Go and IN clause in Postgres - postgresql

I am trying to execute the following query against the PostgreSQL database in Go using pq driver:
SELECT COUNT(id)
FROM tags
WHERE id IN (1, 2, 3)
where 1, 2, 3 is passed at a slice tags := []string{"1", "2", "3"}.
I have tried many different things like:
s := "(" + strings.Join(tags, ",") + ")"
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN $1`, s,
).Scan(&num); err != nil {
log.Println(err)
}
which results in pq: syntax error at or near "$1". I also tried
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN ($1)`, strings.Join(stringTagIds, ","),
).Scan(&num); err != nil {
log.Println(err)
}
which also fails with pq: invalid input syntax for integer: "1,2,3"
I also tried passing a slice of integers/strings directly and got sql: converting Exec argument #0's type: unsupported type []string, a slice.
So how can I execute this query in Go?

Pre-building the SQL query (preventing SQL injection)
If you're generating an SQL string with a param placeholder for each of the values, it's easier to just generate the final SQL right away.
Note that since values are strings, there's place for SQL injection attack, so we first test if all the string values are indeed numbers, and we only proceed if so:
tags := []string{"1", "2", "3"}
buf := bytes.NewBufferString("SELECT COUNT(id) FROM tags WHERE id IN(")
for i, v := range tags {
if i > 0 {
buf.WriteString(",")
}
if _, err := strconv.Atoi(v); err != nil {
panic("Not number!")
}
buf.WriteString(v)
}
buf.WriteString(")")
Executing it:
num := 0
if err := Db.QueryRow(buf.String()).Scan(&num); err != nil {
log.Println(err)
}
Using ANY
You can also use Postgresql's ANY, whose syntax is as follows:
expression operator ANY (array expression)
Using that, our query may look like this:
SELECT COUNT(id) FROM tags WHERE id = ANY('{1,2,3}'::int[])
In this case you can declare the text form of the array as a parameter:
SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])
Which can simply be built like this:
tags := []string{"1", "2", "3"}
param := "{" + strings.Join(tags, ",") + "}"
Note that no check is required in this case as the array expression will not allow SQL injection (but rather will result in a query execution error).
So the full code:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
param := "{" + strings.Join(tags, ",") + "}"
num := 0
if err := Db.QueryRow(q, param).Scan(&num); err != nil {
log.Println(err)
}

This is not really a Golang issue, you are using a string to compare to integer (id) in your SQL request. That means, SQL receive:
SELECT COUNT(id)
FROM tags
WHERE id IN ("1, 2, 3")
instead of what you want to give it. You just need to convert your tags into integer and passe it to the query.
EDIT:
Since you are trying to pass multiple value to the query, then you should tell it:
params := make([]string, 0, len(tags))
for i := range tags {
params = append(params, fmt.Sprintf("$%d", i+1))
}
query := fmt.Sprintf("SELECT COUNT(id) FROM tags WHERE id IN (%s)", strings.Join(params, ", "))
This will end the query with a "($1, $2, $3...", then convert your tags as int:
values := make([]int, 0, len(tags))
for _, s := range tags {
val, err := strconv.Atoi(s)
if err != nil {
// Do whatever is required with the error
fmt.Println("Err : ", err)
} else {
values = append(values, val)
}
}
And finally, you can use it in the query:
Db.QueryRow(query, values...)
This should do it.

Extending #icza solution, you can use pq.Array instead of building the params yourself.
So using his example, the code can look like this:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
num := 0
if err := Db.QueryRow(q, pq.Array(tags)).Scan(&num); err != nil {
log.Println(err)
}

Related

How can I convert the postgres name type to a go type?

I am getting this error trying to scan in postgres rows.
can't scan into dest[0]: unknown oid 1003 cannot be scanned into string
This is the code I that throws that error.
sourceKeys := make([]string, 0, 1)
targetSchema := ""
targetTable := ""
targetKeys := make([]string, 0, 1)
fks := make([]ForeignKey, 0)
for rows.Next() {
err := rows.Scan(
sourceKeys,
&targetSchema,
&targetTable,
targetKeys,
)
}
…….
}
When I look at postgres documentation (https://pkg.go.dev/github.com/lib/pq/oid) I can see that oid=1003 is for T__name types. How can I read in a name without using the pq library?

Go postgres `SELECT * IN` using array

I have a simple select statement:
Select * FROM X where X.name in ("bob", "joe") and X.phone='123'
That works fine in postgres,
In my Go code I have the following code:
var phone string = "123"
var names []string = []string{"bob", "joe"}
sqlStatement := `Select * FROM X where X.name in ($1) and X.phone=$2`
rows, sqlerr := db.Query(sqlStatement, names, phone)
but for some reason I error out from that sql.
unsupported Scan, storing driver.Value type into type *string
how can i use my names array in side the sqlstatement?
note: if i do a fmt.printf and paste the sql statement into postgres, i do get data back + if i take out the $1, and manually input the strings
Copying and pasting fragments from some working Go PostgreSQL code:
import (
"database/sql"
"github.com/lib/pq"
)
query := `
. . .
WHERE code = ANY($1)
. . .
`
codes := []string{"JFK", "LGA", "EWR"}
rows, err := db.Query(query, pq.Array(codes))
I solved this using http://jmoiron.github.io/sqlx/#inQueries
var phone string = "123"
var names []string = []string{"bob", "joe"}
sqlStatement := `Select * FROM X where X.name in (?) and X.phone=?`
sqlStatement, args, err := sqlx.In(sqlStatement, names, phone)
sqlStatement = db.Rebind(sqlStatement)
rows, sqlerr := db.Queryx(sqlStatement, args...)
this now returns correctly.
another way to solve this was to use fmt.Sprintf() and converting the ? to %s

I'm looking for a way to pass in either a slice of int or a comma delimited string into a database/sql 'in' query [duplicate]

I am trying to execute the following query against the PostgreSQL database in Go using pq driver:
SELECT COUNT(id)
FROM tags
WHERE id IN (1, 2, 3)
where 1, 2, 3 is passed at a slice tags := []string{"1", "2", "3"}.
I have tried many different things like:
s := "(" + strings.Join(tags, ",") + ")"
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN $1`, s,
).Scan(&num); err != nil {
log.Println(err)
}
which results in pq: syntax error at or near "$1". I also tried
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN ($1)`, strings.Join(stringTagIds, ","),
).Scan(&num); err != nil {
log.Println(err)
}
which also fails with pq: invalid input syntax for integer: "1,2,3"
I also tried passing a slice of integers/strings directly and got sql: converting Exec argument #0's type: unsupported type []string, a slice.
So how can I execute this query in Go?
Pre-building the SQL query (preventing SQL injection)
If you're generating an SQL string with a param placeholder for each of the values, it's easier to just generate the final SQL right away.
Note that since values are strings, there's place for SQL injection attack, so we first test if all the string values are indeed numbers, and we only proceed if so:
tags := []string{"1", "2", "3"}
buf := bytes.NewBufferString("SELECT COUNT(id) FROM tags WHERE id IN(")
for i, v := range tags {
if i > 0 {
buf.WriteString(",")
}
if _, err := strconv.Atoi(v); err != nil {
panic("Not number!")
}
buf.WriteString(v)
}
buf.WriteString(")")
Executing it:
num := 0
if err := Db.QueryRow(buf.String()).Scan(&num); err != nil {
log.Println(err)
}
Using ANY
You can also use Postgresql's ANY, whose syntax is as follows:
expression operator ANY (array expression)
Using that, our query may look like this:
SELECT COUNT(id) FROM tags WHERE id = ANY('{1,2,3}'::int[])
In this case you can declare the text form of the array as a parameter:
SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])
Which can simply be built like this:
tags := []string{"1", "2", "3"}
param := "{" + strings.Join(tags, ",") + "}"
Note that no check is required in this case as the array expression will not allow SQL injection (but rather will result in a query execution error).
So the full code:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
param := "{" + strings.Join(tags, ",") + "}"
num := 0
if err := Db.QueryRow(q, param).Scan(&num); err != nil {
log.Println(err)
}
This is not really a Golang issue, you are using a string to compare to integer (id) in your SQL request. That means, SQL receive:
SELECT COUNT(id)
FROM tags
WHERE id IN ("1, 2, 3")
instead of what you want to give it. You just need to convert your tags into integer and passe it to the query.
EDIT:
Since you are trying to pass multiple value to the query, then you should tell it:
params := make([]string, 0, len(tags))
for i := range tags {
params = append(params, fmt.Sprintf("$%d", i+1))
}
query := fmt.Sprintf("SELECT COUNT(id) FROM tags WHERE id IN (%s)", strings.Join(params, ", "))
This will end the query with a "($1, $2, $3...", then convert your tags as int:
values := make([]int, 0, len(tags))
for _, s := range tags {
val, err := strconv.Atoi(s)
if err != nil {
// Do whatever is required with the error
fmt.Println("Err : ", err)
} else {
values = append(values, val)
}
}
And finally, you can use it in the query:
Db.QueryRow(query, values...)
This should do it.
Extending #icza solution, you can use pq.Array instead of building the params yourself.
So using his example, the code can look like this:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
num := 0
if err := Db.QueryRow(q, pq.Array(tags)).Scan(&num); err != nil {
log.Println(err)
}

Custom query with go-pg into []String slice

I'm using https://godoc.org/github.com/go-pg/pg a bunch of other places in the code so I'm hoping I don't have to switch to another client.
I can't get the ORM to write this query (below) correctly, so I just want to pass it thru as a custom string. But I can't figure out how to get the results into my []string slice.
tokens := []string{}
qry := `SELECT p.token
FROM pntokens p
join
(VALUES ('123'), ('456'), ('789')) AS t (userid)
on p.userid = t.userid ;`
I've tried:
err := db.Model(&Pntoken{}, qry).Select(&tokens)
err := db.Query([]string{}, qry, nil).Select(&tokens)
_, err := db.Exec(qry)
res, err := db.Model((*Pntoken)(nil)).Exec(qry)
But cannot get the tool out of my way enough to just get some simple results into my slice.
All tips appreciated!
Use like this for lib https://github.com/go-pg/pg:
tokens := []string
qry := `SELECT p.token FROM pntokens p
join (VALUES ('123'), ('456'), ('789')) AS t (userid)
on p.userid = t.userid ;`
_, err := pc.DB.Query(&tokens,"qry",nil)
if err != nil {
return tokens, err
}

How to insert multiple rows into postgres SQL in a go

Is it possible to insert multiple rows into Postgres database at once? Could someone please suggest if there is a way to insert a slice of slices into database. I have created a slice for each row and created a another slice(multiple rows) by appending all the row slices to it. how do I insert the slice(multiple rows) into db?
When I create a row slice, I'm using row := []interface{}{} . Because I have fields which are strings and int in each row. Looks like I
get an error when I'm inserting data and the error is unsupported type []interface {}, a slice of interface
Implementation:
rowdata := []interface{}{}
row := []interface{}{data.ScenarioUUID, data.Puid, data.Description, data.Status, data.CreatedBy, data.CreatedAt, data.UpdatedBy, data.UpdatedAt, data.ScopeStartsAt, data.ScopeEndsAt, Metric, MetricName, Channel, date, timeRangeValue}
rowdata = append(rowdata, row)
qry2 := `INSERT INTO sample (scenarioUuid,
puId,
description,
status,
createdBy,
createdAt,
updatedBy,
updatedAt,
scopeStartsAt,
scopeEndsAt,
metric,
metric_name,
channel,
time,
value) VALUES ($1, $2, $3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15)`
if _, err := db.Exec(qry2, rowdata); err != nil {
panic(err)
You could do something like this:
samples := // the slice of samples you want to insert
query := `insert into samples (<the list of columns>) values `
values := []interface{}{}
for i, s := range samples {
values = append(values, s.<field1>, s.<field2>, < ... >)
numFields := 15 // the number of fields you are inserting
n := i * numFields
query += `(`
for j := 0; j < numFields; j++ {
query += `$`+strconv.Itoa(n+j+1) + `,`
}
query = query[:len(query)-1] + `),`
}
query = query[:len(query)-1] // remove the trailing comma
db.Exec(query, values...)
https://play.golang.org/p/YqNJKybpwWB