How to get row value(s) back after db insert? - postgresql

I am using Golang to insert data into a DB. basically my query looks like below
var cols = "(col1, col2, col3)"
var values = "($1, $2, $3)"
var query = fmt.Sprintf("INSERT INTO %s %s VALUES %s", myTable, cols, values)
res, err := db.Exec(query, thing.val1, thing.val2, thing.val3)
The only things available from res are lastInsertId and # of rows affected. But what I need is the rows affected. The reason being is that I insert data into a psql database which has an AUTOINCREMENT id column - so I want the data back with that.
For example - with Java hibernate I can do what this answer explains. I don't have to re-query the DB for the ID.
EDIT: I tried to use the lastInsertId method and got this error
LastInsertId is not supported by this driver

Assuming you just want the auto-incremented value(s) in a column called id and this is an insert with the pq driver
var cols = "(col1, col2, col3)"
var values = "($1, $2, $3)"
var query = fmt.Sprintf(
"INSERT INTO %s %s VALUES %s RETURNING id",
myTable, cols, values,
)
var id int
if err := db.QueryRow(
query,
thing.val1, thing.val2, thing.val3,
).Scan(&id); err != nil {
panic(err)
}
fmt.Println("ID: ", id)
For multiple inserts:
var cols = "(col1, col2, col3)"
var values = "($1, $2, $3),($4, $5, $6)"
var query = fmt.Sprintf(
"INSERT INTO %s %s VALUES %s RETURNING id",
myTable, cols, values,
)
var ids []int
rows, err := db.Query(
query,
thing.val1, thing.val2, thing.val3,
thing.val4, thing.val5, thing.val6,
)
if err != nil {
panic(err)
}
for rows.Next() {
var id int
if err := rows.Scan(&id); err != nil {
panic(err)
}
ids = append(ids, id)
}
fmt.Println("IDs: ", ids)

res.LastInsertId() is not supported in Postgres Driver. However, It is supported in MySQL Driver.
db.Exec() doesn't return last inserted id but db.QueryRow() does.
For better understanding you can refer this link.
Here, I added one example which might help you.
var id int
err := db.QueryRow("INSERT INTO user (name) VALUES ('John') RETURNING id").Scan(&id)
if err != nil {
...
}

Related

Scan a Sql Query sqlx.Rows into a Nested Structure in golang

I have three following tables:
create table A (
a_id varchar(256) not null unique,
a_name varchar(256)
);
create table B (
b_id varchar(256) not null,
b_a_id varchar(256) not null,
b_name varchar(256),
FOREIGN KEY (b_a_id) REFERENCES a (a_id)
);
create table C (
c_id varchar(256) not null,
c_a_id varchar(256) not null,
c_name varchar(256),
FOREIGN KEY (c_a_id) REFERENCES a (a_id)
);
insert into A(a_id, a_name) values('1234', 'a_name_1');
insert into B(b_id, b_a_id, b_name) values('B1','1234', 'b_name_1');
insert into B(b_id, b_a_id, b_name) values('B2','1234', 'b_name_2');
insert into C(c_id, c_a_id, c_name) values('C1','1234', 'c_name_1');
insert into C(c_id, c_a_id, c_name) values('C2','1234', 'c_name_2');
insert into C(c_id, c_a_id, c_name) values('C3','1234', 'c_name_3');
I have the following Structs in golang:
type A struct {
a_id string `db:"a_id"`
a_name string `db:"a_name"`
b *B `db:"b"`
c *C `db:"c"`
}
type B struct {
b_id string `db:"b_id"`
b_name string `db:"b_name"`
b_a_id string `db:"b_a_id"`
}
type C struct {
c_id string `db:"c_id"`
c_name string `db:"c_name"`
c_a_id string `db:"c_a_id"`
}
I want to scan the rows I get from executing the join query:
SELECT * from A INNER JOIN B ON a_id=b_a_id inner join C on c_a_id=a_id;
Into Struct A using rows.StructScan() in Golang but I am not able to do that. How Do I scan
a join query result into a nested struct and I don't want to individually scan each column as there are a lot of columns as a result of the join query going forward.
After some investigation, I came up with a solution that should work also for you. First, let me present the working code, and then I'll explain the relevant parts:
package main
import (
"fmt"
"github.com/jmoiron/sqlx"
_ "github.com/lib/pq"
)
type A struct {
Id string `db:"a_id"`
Name string `db:"a_name"`
B
C
}
type B struct {
Id string `db:"b_id"`
AId string `db:"b_a_id"`
Name string `db:"b_name"`
}
type C struct {
Id string `db:"c_id"`
AId string `db:"c_a_id"`
Name string `db:"c_name"`
}
func main() {
db, err := sqlx.Open("postgres", "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable")
if err != nil {
panic(err)
}
defer db.Close()
a := []A{}
rows, err := db.Queryx("SELECT * from A INNER JOIN B ON a_id=b_a_id inner join C on c_a_id=a_id;")
if err != nil {
panic(err)
}
defer rows.Close()
for rows.Next() {
var record A
if err := rows.StructScan(&record); err != nil {
panic(err)
}
a = append(a, record)
}
if err := rows.Err(); err != nil {
panic(err)
}
for _, v := range a {
fmt.Println(v)
}
}
Structs definition
Here, I fixed the structs' definition by embedding B and C into the A struct. Furthermore, I also fixed the name as long as they didn't collide with the others (e.g. it's completely safe to have the Id field in all of the three structs).
Fetching data
The other relevant part is fetching the data from Postgres. Here you've to use the Queryx method and provide to it the SQL query you've correctly written.
However, you should use the methods provided by the database/sql package when dealing with multiple rows (as in our case). Next and Err are self-explanatory so I won't spend any time over them.
StructScan is the method that does the trick. It puts the current query into a loop-scoped variable called record of type A (which is our parent struct).
If you give a try to this code it should work also for you, if not let me know!

How to get RETURNING ID

golang-1.18.1, postgresql-12
...
var fk = []int{11, 22}
var img = []string{"a", "b"}
var prim = []string{"a1", "b1"}
err = conn.QueryRow(context.Background(),
"INSERT INTO tb (fk, img, prim) VALUES($1,$2,$3),($4,$5,$6)", fk[0],img[0],prim[0],fk[1],img[1],prim[1]).Scan(&fk[0],&img[0],&prim[0])
...
so it works, insert two records.
How to get returning id
1. if inserting one record
2. or inserting multiple
...
? something like:
var idx []int
err = conn.QueryRow(context.Background(),
"WITH ins AS (INSERT INTO tb (fk, img, prim) VALUES($1,$2,$3),($4,$5,$6) RETURNING id) SELECT array_agg(id) INTO idx FROM ins", fk[0],img[0],prim[0],fk[1],img[1],prim[1]).Scan(&fk[0],&img[0],&prim[0])
tb has fields: id, fk, img, prim
insert into tb (fk, img, prim) values(...) returning id into idx
does not work in my pgx _"github.com/jackc/pgx/v4"
UPD1
**conn.Query**
var fk = []int{11, 22}
var img = []string{"a1", "b1"}
var prim = []string{"a2", "b2"}
rows, err := conn.Query(context.Background(),"INSERT INTO tb (fk,img,prim) VALUES($1,$2,$3),($4,$5,$6) RETURNING id", fk[0],img[0],prim[0],fk[1],img[1],prim[1])
var idx []int
for rows.Next() {
var id int
rows.Scan(&id)
idx = append(idx, id)
}
fmt.Println("idx: ", idx)
**conn.QueryRow**
// works for inserting a single record, but only without a prepared state
var id int
err = conn.QueryRow(context.Background(), "INSERT INTO tb (fk,img,prim) VALUES (11,'aa','bb') RETURNING id").Scan(&id)
fmt.Println("idx: ", id)
// don`t work RETURNING with prepared state, but inserting:
err = conn.QueryRow(context.Background(), "INSERT INTO tb (fk,img,prim) VALUES($1,$2,$3) RETURNING id", fk[0],img[0],prim[0]).Scan(&id,&fk[0],&img[0],&prim[0])
conn.Query works for inserts SINGLE and MULTY records with prepared state and RETURNING.
conn.QueryRow works for inserting a single record, but only without a prepared state.

How to extract postgres timestamp range with Go?

I have a database calendar with tsrange type from postgres. It allows me to have multiple appointments and time range such as :
["2018-11-08 10:00:00","2018-11-08 10:45:00"]
How do I store this value in a Go variable ?
I tried
var tsrange []string
And when I log tsrange[0] it is empty. What is the proper type for it ?
More code :
rows, err := db.Query("SELECT * FROM appointments")
utils.CheckErr(err)
var id int
var userID int
var tsrange []string
rows.Next()
err = rows.Scan(&id, &userID, &tsrange)
fmt.Println(tsrange[0])
When I replace var tsrange []string with var tsrange string the log is ["2018-11-08 10:00:00","2018-11-08 10:45:00"].
You should be able to retrieve the individual bounds of a range at the sql level.
// replace tsrange_col with the name of your tsrange column
rows, err := db.Query("SELECT id, user_id, lower(tsrange_col), upper(tsrange_col) FROM appointments")
utils.CheckErr(err)
var id int
var userID int
var tsrange [2]time.Time
rows.Next()
err = rows.Scan(&id, &userID, &tsrange[0], &tsrange[1])
fmt.Println(tsrange[0]) // from
fmt.Println(tsrange[1]) // to

Inserting array of custom types into postgres

I'm trying to insert a row with a column that is an array of a custom type (ingredient). My tables are:
CREATE TYPE ingredient AS (
name text,
quantity text,
unit text
);
CREATE TABLE IF NOT EXISTS recipes (
recipe_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
name text,
ingredients ingredient[],
// ...
);
Using raw sql, I can insert a row by:
INSERT INTO recipes (name, ingredients) VALUES ('some_name', ARRAY[ROW('aa', 'bb', 'cc'), ROW('xx', 'yy', 'zz')]::ingredient[] );
But I'm struggling to do this in go with the pq lib. I've created a pq.Array interface:
type Ingredient struct {
Name string
Quantity string
Unit string
}
type Ingredients []*Ingredient
func (ings *Ingredients) ConvertValue(v interface{}) (driver.Value, error) {
return "something", nil
}
func (ings *Ingredients) Value() (driver.Value, error) {
val := `ARRAY[]`
for i, ing := range ings {
if i != 0 {
val += ","
}
val += fmt.Printf(`ROW('%v','%v','%v')`, ing.Name, ing.Quantity, ing.Unit)
}
val += `::ingredient[]`
return val, nil
}
// and then trying to insert via:
stmt := `INSERT INTO recipes (
name,
ingredients
)
VALUES ($1, $2)
`
_, err := db.Exec(stmt,
"some_name",
&Ingredients{
&Ingredient{"flour", "3", "cups"},
},
)
But pg keeps throwing the error:
Error insertingpq: malformed array literal: "ARRAY[ROW('flour','3','cups')]::ingredient[]"
Am I returning an incorrect driver.Value?
You can either use this approach outlined here: https://github.com/lib/pq/issues/544
type Ingredient struct {
Name string
Quantity string
Unit string
}
func (i *Ingredient) Value() (driver.Value, error) {
return fmt.Sprintf("('%s','%s','%s')", i.Name, i.Quantity, i.Unit), nil
}
stmt := `INSERT INTO recipes (name, ingredients) VALUES ($1, $2::ingredient[])`
db.Exec(stmt, "some_name", pq.Array([]*Ingredient{{"flour", "3", "cups"}}))
Or if you have records in the table and you query it, you will probably see the ingredient array in its literal form, which you can than mimic during insert.
func (ings *Ingredients) Value() (driver.Value, error) {
val := `{`
for i, ing := range ings {
if i != 0 {
val += ","
}
val += fmt.Sprintf(`"('%s','%s','%s')"`, ing.Name, ing.Quantity, ing.Unit)
}
val += `}`
return val, nil
}
// e.g. `{"('flour','3','cups')"}`
stmt := `INSERT INTO recipes (name, ingredients) VALUES ($1, $2::ingredient[])`
// ...
It seems your database design is very complicated and not taking into account the strengths (and weaknesses) of SQL.
May I suggest you split the ingredients into their own table with a reference to the recipe. Then finding out a full recipe is a JOIN operation.
Creating the DB:
CREATE TABLE ingredients (
recipe_id uuid,
name text,
quantity int,
unit text
);
CREATE TABLE recipes (
recipe_id uuid PRIMARY KEY,
name text
);
Inserting a recipe and querying to read it out:
INSERT INTO recipes VALUES (
'5d1cb631-37bd-46cc-a278-4c8558ed8964', 'cake1'
);
INSERT INTO ingredients (recipe_id, name, quantity, unit) VALUES
('5d1cb631-37bd-46cc-a278-4c8558ed8964', 'flour', 3, 'cups'),
('5d1cb631-37bd-46cc-a278-4c8558ed8964', 'water', 1, 'cups')
;
SELECT r.name, i.name, i.quantity, i.unit
FROM ingredients AS i
INNER JOIN recipes AS r
ON r.recipe_id=i.recipe_id;
SQLFiddle link: http://sqlfiddle.com/#!17/262ad/14

Golang Postgresql Array

If I have a table that returns something like:
id: 1
names: {Jim, Bob, Sam}
names is a varchar array.
How do I scan that back into a []string in Go?
I'm using lib/pg
Right now I have something like
rows, err := models.Db.Query("SELECT pKey, names FROM foo")
for rows.Next() {
var pKey int
var names []string
err = rows.Scan(&pKey, &names)
}
I keep getting:
panic: sql: Scan error on column index 1: unsupported Scan, storing driver.Value type []uint8 into type *[]string
It looks like I need to use StringArray
https://godoc.org/github.com/lib/pq#StringArray
But, I think I'm too new to Go to understand exactly how to use:
func (a *StringArray) Scan(src interface{})
You are right, you can use StringArray but you don't need to call the
func (a *StringArray) Scan(src interface{})
method yourself, this will be called automatically by rows.Scan when you pass it anything that implements the Scanner interface.
So what you need to do is to convert your []string to *StringArray and pass that to rows.Scan, like so:
rows, err := models.Db.Query("SELECT pKey, names FROM foo")
for rows.Next() {
var pKey int
var names []string
err = rows.Scan(&pKey, (*pq.StringArray)(&names))
}
Long Story Short, use like this to convert pgSQL array to GO array, here 5th column is coming as a array :
var _temp3 []string
for rows.Next() {
// ScanRows scan a row into temp_tbl
err := rows.Scan(&_temp, &_temp0, &_temp1, &_temp2, pq.Array(&_temp3))
In detail :
To insert a row that contains an array value, use the pq.Array function like this:
// "ins" is the SQL insert statement
ins := "INSERT INTO posts (title, tags) VALUES ($1, $2)"
// "tags" is the list of tags, as a string slice
tags := []string{"go", "goroutines", "queues"}
// the pq.Array function is the secret sauce
_, err = db.Exec(ins, "Job Queues in Go", pq.Array(tags))
To read a Postgres array value into a Go slice, use:
func getTags(db *sql.DB, title string) (tags []string) {
// the select query, returning 1 column of array type
sel := "SELECT tags FROM posts WHERE title=$1"
// wrap the output parameter in pq.Array for receiving into it
if err := db.QueryRow(sel, title).Scan(pq.Array(&tags)); err != nil {
log.Fatal(err)
}
return
}
Note: that in lib/pq, only slices of certain Go types may be passed to pq.Array().
Another example in which varchar array in pgSQL in generated at runtime in 5th column, like :
--> predefined_allow false admin iam.create {secrets,configMap}
I converted this as,
Q := "SELECT ar.policy_name, ar.allow, ar.role_name, pro.operation_name, ARRAY_AGG(pro.resource_id) as resources FROM iam.authorization_rules ar LEFT JOIN iam.policy_rules_by_operation pro ON pro.id = ar.operation_id GROUP BY ar.policy_name, ar.allow, ar.role_name, pro.operation_name;"
tx := g.db.Raw(Q)
rows, _ := tx.Rows()
defer rows.Close()
var _temp string
var _temp0 bool
var _temp1 string
var _temp2 string
var _temp3 []string
for rows.Next() {
// ScanRows scan a row into temp_tbl
err := rows.Scan(&_temp, &_temp0, &_temp1, &_temp2, pq.Array(&_temp3))
if err != nil {
return nil, err
}
fmt.Println("Query Executed...........\n", _temp, _temp0, _temp1, _temp2, _temp3)
}
Output :
Query Executed...........
predefined_allow false admin iam.create [secrets configMap]