Postgresql Golang Martini Inserting key - postgresql

I am working on building a social network type server as an exercise using martini, golang, and postgresql to help develop my skills in all three. The few key things that are eluding me are how to insert the primary key from the user table into the correct row of the user info table (to connect the user info with the specific user). I'm sure there is also a way to consolidate both the queries into a much more concise postgres script...
func CreateUser(ren render.Render, r *http.Request, db *sql.DB) {
_, err := db.Query("INSERT INTO users (first, last, email, password, karma, value) SELECT CAST ($3 AS VARCHAR), $1, $2, $4, $5, $6 WHERE NOT EXISTS (SELECT * FROM users WHERE email = $3)",
r.FormValue("first"),
r.FormValue("last"),
r.FormValue("email"),
r.FormValue("password"),
0,
0)
PanicIf(err)
_, err = db.Query("INSERT INTO userinfo (usr, dob, phonenum, bio, mates, bought, sold) VALUES ($1, $2, $3, $4, $5, $6, $7)",
0,
r.FormValue("dob"),
r.FormValue("phonenum"),
r.FormValue("bio"),
0,
0,
0)
PanicIf(err)
ren.Redirect("/")
}
Here is the script for creating the user table:
DROP TABLE IF EXISTS "public"."users";
CREATE TABLE "public"."users" (
"id" serial NOT NULL,
"first" varchar(40) NOT NULL COLLATE "default",
"last" varchar(40) NOT NULL COLLATE "default",
"email" varchar(40) NOT NULL COLLATE "default",
"password" varchar(40) NOT NULL COLLATE "default",
"karma" int NOT NULL,
"value" int NOT NULL
)
WITH (OIDS=FALSE);
I realize that in production level code I would not want to store the password like this. I have also sort of been stumped by how to pull non-global parameters into the database as its is created by m.get & m.post commands.
m.Post("/newuser", CreateUser)
m.Get("/user", NewUser)
Any advice on consolidating and adding the key from users to the appropriate entry in userinfo would be greatly appreciated... I realize that these are probably incredibly stupid questions but please pardon my naivety.

I did it by returning the inserted user id from the first query and use it in the second one, but I did not use upsert.
So you can try use RETURNING to return the id of newly added user or query it separately by the email (as it looks like the email is unique per user in this case).
Then use it for second query:
_, err = db.Query(`INSERT INTO userinfo (usr, dob, phonenum, bio, mates, bought, sold)
VALUES ($1, $2, $3, $4, $5, $6, $7)`,
id, // there
r.FormValue("dob"),
r.FormValue("phonenum"),
...
I suppose that usr is the user id (foreign key to users.id)
Sample source code:
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
"log"
)
func main() {
db, err := sql.Open("postgres", "user=alex dbname=tmp sslmode=disable")
if err != nil {
log.Fatal(err)
}
rows, err := db.Query(`
INSERT INTO users (first, last, email, password, karma, value)
SELECT CAST ($3 AS VARCHAR), $1, $2, $4, $5, $6
WHERE NOT EXISTS (SELECT * FROM users WHERE email = $3)
RETURNING id`,
"first", "last", "email", "password", 0, 0)
defer rows.Close()
if err != nil {
log.Fatal(err)
}
if rows.Next() {
id := 0
rows.Scan(&id)
_, err = db.Exec(`INSERT INTO userinfo (user_id, info) VALUES ($1, $2)`, id, "info")
if err != nil {
log.Fatal(err)
}
fmt.Println("Inserted:", id)
}
}
I personally prefer QueryRow instead of query as it a bit more obvious that one row is returned, but it will work with Query as well.
Also I recommend to use unique index for email, instead of subquery.

Related

Bulk Insert PostGis Geometry using pgx.copyFrom or pgx.Batch

I've found the way to make a bulk insert with classic postgres types from this post and it works like a charm.
But for whatever reason, i struggle to make it work when trying to insert geometry points:
using pgx.CopyFromRows
rows := [][]interface{}{
{"John", "ST_SetSRID(ST_MakePoint(1.23,2.34), 4326)"},
{"Jane", "ST_SetSRID(ST_MakePoint(1.10,2.12), 4326)"},
}
// GetSession return a *pgxpool.Pool
copyCount, err := postgres.GetSession().CopyFrom(
context.Background(),
pgx.Identifier{"test_db"},
[]string{"first_name", "location"},
pgx.CopyFromRows(rows),
)
this gives me back
"ERROR: Invalid endian flag value encountered. (SQLSTATE XX000)"
using pgx.Batch:
batch := &pgx.Batch{}
batch.Queue("insert into people(first_name, location) values($1, $2)", "Bob", "ST_SetSRID(ST_MakePoint(1.23,1.34), 4326)")
batch.Queue("insert into people(first_name, location) values($1, $2)", "John", "ST_SetSRID(ST_MakePoint(1.23,1.34), 4326)")
batchResult := postgres.GetSession().SendBatch(context.Background(), batch)
_, err := batchResult.Exec()
if err != nil {
return rest_errors.NewInternalServerError("Error processing batch insert", err)
}
batchResult.Close()
I get this error
Error processing batch insert - parse error - invalid geometry (SQLSTATE XX000)
db creation script
CREATE TABLE IF NOT EXISTS public.test_db
(
first_name text COLLATE pg_catalog."default",
location geometry
)
Thank you so much

ERROR: missing FROM-clause entry for table when using GORM

These are the tables in my database
CREATE TABLE vehicles
(
id VARCHAR PRIMARY KEY,
make VARCHAR NOT NULL,
model VARCHAR NOT NULL,
)
CREATE TABLE collisions
(
id VARCHAR PRIMARY KEY,
longitude FLOAT NOT NULL,
latitude FLOAT NOT NULL,
)
CREATE TABLE vehicle_collisions
(
vehicle_id VARCHAR NOT NULL,
collision_id VARCHAR NOT NULL,
PRIMARY KEY (vehicle_id, collision_id)
)
So i need to find list of vehicles with a particular collision_id. I am using gorm .
I tried to implement it in a way
var vehicles []entities.Vehicles
err := r.db.Joins("JOIN vehicles as vh on vh.id=vehicle_collisions.vehicle_id").Where("vehicle_collisions.collision_id=?",
id).Find(&vehicles).Error
if err != nil {
fmt.Println(err)
}
But it is throwing me error
ERROR: missing FROM-clause entry for table "vehicle_collisions" (SQLSTATE 42P01)
Any help would really be appreciated.
Thank you mkopriva as pointed
when you pass &vehicles which is []entities.Vehicles to Find the query generated would be as below:
SELECT
*
FROM
vehicles
JOIN
vehicles vh
ON vh.id = vehicle_collisions.vehicle_id
WHERE vehicle_collisions.collision_id=1
which won't be correct to solve the problem modify the query as:
err := r.db.
Joins("JOIN vehicle_collisions AS vc ON vc.vehicle_id=vehicles.id").
Where("vc.collision_id = ?", id).
Find(&vehicles).Error
As the question lacks some details, I tried to guess them. I hope that the answer provided is relevant to you! Let me present the code that is working on my side:
package main
import (
"gorm.io/driver/postgres"
"gorm.io/gorm"
_ "github.com/lib/pq"
)
type Vehicle struct {
Id string
Make string
Model string
Collisions []Collision `gorm:"many2many:vehicle_collisions"`
}
type Collision struct {
Id string
Longitude float64
Latitude float64
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic(err)
}
db.AutoMigrate(&Vehicle{})
db.AutoMigrate(&Collision{})
// add dummy data to the db
collision := &Collision{"1", 4.4, 4.5}
db.Create(collision)
db.Create(&Collision{"2", 1.1, 1.5})
db.Create(&Vehicle{Id: "1", Make: "ford", Model: "fiesta", Collisions: []Collision{*collision}})
db.Create(&Vehicle{Id: "2", Make: "fiat", Model: "punto", Collisions: []Collision{*collision}})
// get all vehicles for collision 1
var vehicles []Vehicle
db.Debug().Joins("inner join vehicle_collisions vc on vehicles.id = vc.vehicle_id").Find(&vehicles, "vc.collision_id = ?", "1")
}
The code starts with the structs' definitions.
Please note the Gorm annotation on the field Collisions.
After adding some data, the query should be pretty straightforward: we use the Joins method to load data from the table vehicle_collisions and in the Find method we filter out only records with collision_id equal to "1".
Let me know if this helps you or you need something else!

Postgres sqlx upsert causing pq: duplicate key value violates unique constraint

I have a table with a unique index called user_conn_unique and I perform an 'upsert' on that table however intermittently I am getting an error pq: duplicate key value violates unique constraint "user_unique_connection"
I have tried recreating the issue locally by running concurrent request for both insert and update scenarios and also executing the statements directly on the db with many records seeded but cannot recreate it.
From what I see in the logs we receive two requests microseconds apart and one is successful and one fails with the above error
Example Data -
user_id = 592c70b4-48b0-11ec-81d3-0242ac130003
location = EU
group_id = 592c70b4-48b0-11ec-81d3-0242ac131111
In the database there are no duplicates and all the data looks correct.
Below is the table and Go code
PostgreSQL 10.14 - table around 200k rows
create table user_conn (
user_id uuid not null,
location text not null,
group_id uuid not null,
created_at timestamp with time zone not null default current_timestamp,
connected_at timestamp with time zone not null default current_timestamp,
disconnected_at timestamp with time zone,
primary key (user_id, group_id, location)
);
create unique index user_unique_connection on user_conn (location, user_id, group_id, coalesce(disconnected_at, '1970-01-01'));
alter table user_conn add column unlinked_at timestamp with time zone default null;
Go 1.16
"database/sql"
"github.com/lib/pq" // v1.10.3
"github.com/jmoiron/sqlx" // v1.3.4
func (pg *PG) Upsert(userID *uuid.UUID, location string, groupID *uuid.UUID) error {
db := pg.DB() // returns *sqlx.DB
tx, err := db.Beginx()
if err != nil {
return err
}
defer func() {
if err != nil {
tx.Rollback()
} else {
err = tx.Commit()
}
}()
stmt := `
insert into user_conn (
user_id, location, group_id
) values (
$1, $2, $3
)
on conflict (user_id, location, group_id)
do update set disconnected_at=null, unlinked_at=null, connected_at=now()
returning *
`
err = tx.Get(cl, stmt, userID, location, groupID)
if err != nil {
return err
}
_, err = tx.ExecContext(another statement on another table)
if err != nil {
return err
}
return nil
}
What could be causing this issue?

Sqlx Get with prepared statements

I am trying to fetch some data from postgress table using prepared statements
If I try with database.Get() everything is returned.
Table:
create table accounts
(
id bigserial not null
constraint accounts_pkey
primary key,
identificator text not null,
password text not null,
salt text not null,
type smallint not null,
level smallint not null,
created_at timestamp not null,
updated timestamp not null,
expiry_date timestamp,
qr_key text
);
Account struct:
type Account struct {
ID string `db:"id"`
Identificator string `db:"identificator"`
Password string `db:"password"`
Salt string `db:"salt"`
Type int `db:"type"`
Level int `db:"level"`
ExpiryDate time.Time `db:"expiry_date"`
CreatedAt time.Time `db:"created_at"`
UpdateAt time.Time `db:"updated_at"`
QrKey sql.NullString `db:"qr_key"`
}
BTW i tried using ? instead of $1 & $2
stmt, err := database.Preparex(`SELECT * FROM accounts where identificator = $1 and type = $2`)
if err != nil {
panic(err)
}
accounts := []account.Account{}
err = stmt.Get(&accounts, "asd", 123)
if err != nil {
panic(err)
}
The error I get is
"errorMessage": "scannable dest type slice with \u003e1 columns (10) in result",
In the table there are no records I tried to remove all fields except the ID from Account (struct), however it does not work.
Documentation for sqlx described Get and Select as:
Get and Select use rows.Scan on scannable types and rows.StructScan on
non-scannable types. They are roughly analagous to QueryRow and Query,
where Get is useful for fetching a single result and scanning it, and
Select is useful for fetching a slice of results:
For fetching a single record use Get.
stmt, err := database.Preparex(`SELECT * FROM accounts where identificator = $1 and type = $2`)
var account Account
err = stmt.Get(&account, "asd", 123)
If your query returns more than a single record use Select with statement as:
stmt, err := database.Preparex(`SELECT * FROM accounts where identificator = $1 and type = $2`)
var accounts []Account
err = stmt.Select(&accounts, "asd", 123)
In your case if you use stmt.Select instead if stmt.Get. It will work.

Store recursive go structs in Postgresql database

I have two structs (Person and Tenant) that reference each other recursively.
I have no experience with 'SQL' and Im trying to use the https://github.com/jmoiron/sqlx library to store these structs in a way that they keep referencing each other, so that I can retrieve them again as structs.
I don't know which type should the tables be created with or how I am supposed to insert the objects to get it work.
Also If there is any other go library that can handle this case easily i'm open to any suggestions.
Thanks in advance.
type Tenant struct {
Id int `db:"id"`
Name string `db:"name"`
Person []Person `db:"person"`
}
type Person struct {
Id int `db:"id"`
Username string `db:"username"`
Tenants *[]Tenant `db:"tenants"`
}
func main() {
var schema = `
CREATE TABLE IF NOT EXISTS person (
id int,
username text
tenants []text //-> type????
);
CREATE TABLE IF NOT EXISTS tenant (
id int,
name text,
person []text //-> type????
)`
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=%s",
host, port, user, password, dbname, sslmode)
db, err := sqlx.Open("postgres", psqlInfo)
if err != nil {
fmt.Println(err)
return
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err)
}
fmt.Println("Successfully connected!")
db.MustExec(schema)
var tenant1 Tenant
var person1 Person
tenant1 = Tenant{1, "newtenant", []Person{person1}}
person1 = Person{1, "newuser", &[]Tenant{tenant1}}
tx := db.MustBegin()
tx.NamedExec("INSERT INTO tenant (id,name,person) VALUES (:id,:name, :person)", &tenant1)
tx.Commit()
out := []Tenant{}
db.Select(&out, "SELECT * FROM tenant ORDER BY name ASC")
fmt.Println(out)
}
NOTE: This is not a real answer, just a longer comment on the SQL part of the question. Unfortunately I have no experience with sqlx so I cannot help you with that.
What you have there seems to be a many-to-many relationship. A Person can belong to multiple Tenants and a Tenant can have multiple Persons.
In SQL this is usually handled by, what's sometimes called, a linking or junction table.
-- postgresql flavor of SQL
CREATE TABLE IF NOT EXISTS person (
id serial PRIMARY KEY,
username text NOT NULL
);
CREATE TABLE IF NOT EXISTS tenant (
id serial PRIMARY KEY,
name text NOT NULL
);
-- the linking table
CREATE TABLE IF NOT EXISTS person_tenant (
person_id integer NOT NULL REFERENCES person (id),
tenant_id integer NOT NULL REFERENCES tenant (id),
PRIMARY KEY(person_id, tenant_id)
);