I have a simple select statement:
Select * FROM X where X.name in ("bob", "joe") and X.phone='123'
That works fine in postgres,
In my Go code I have the following code:
var phone string = "123"
var names []string = []string{"bob", "joe"}
sqlStatement := `Select * FROM X where X.name in ($1) and X.phone=$2`
rows, sqlerr := db.Query(sqlStatement, names, phone)
but for some reason I error out from that sql.
unsupported Scan, storing driver.Value type into type *string
how can i use my names array in side the sqlstatement?
note: if i do a fmt.printf and paste the sql statement into postgres, i do get data back + if i take out the $1, and manually input the strings
Copying and pasting fragments from some working Go PostgreSQL code:
import (
"database/sql"
"github.com/lib/pq"
)
query := `
. . .
WHERE code = ANY($1)
. . .
`
codes := []string{"JFK", "LGA", "EWR"}
rows, err := db.Query(query, pq.Array(codes))
I solved this using http://jmoiron.github.io/sqlx/#inQueries
var phone string = "123"
var names []string = []string{"bob", "joe"}
sqlStatement := `Select * FROM X where X.name in (?) and X.phone=?`
sqlStatement, args, err := sqlx.In(sqlStatement, names, phone)
sqlStatement = db.Rebind(sqlStatement)
rows, sqlerr := db.Queryx(sqlStatement, args...)
this now returns correctly.
another way to solve this was to use fmt.Sprintf() and converting the ? to %s
Related
This question already has an answer here:
"Operator does not exist: integer =?" when using Postgres
(1 answer)
Closed 3 years ago.
I am trying to insert incoming form values into my PostgreSQL database table on Heroku. This is the Go function being used:
func Insert(w http.ResponseWriter, r *http.Request) {
db := dbConn()
if r.Method == "POST" {
nameo := r.FormValue("name")
typeo := r.FormValue("asslia")
balanceo := r.FormValue("balance")
insForm, err := db.Prepare("INSERT INTO al(name, asslia, balance) VALUES( ? , ? , ? )")
if err != nil {
panic(err.Error())
}
insForm.Exec(nameo, typeo, balanceo)
log.Println("INSERT: Name: " + nameo + " | Type: " + typeo)
}
defer db.Close()
http.Redirect(w, r, "/", 301)
}
I keep getting the error pq: syntax error at or near "," at the line insForm, err := db.Prepare("INSERT INTO al(name, asslia, balance) VALUES( ? , ? , ? )")
The table was previously created as such:
CREATE TYPE types AS ENUM ('asset', 'liability');
CREATE TABLE IF NOT EXISTS al
(id SERIAL,
asslia TYPES,
balance MONEY,
name VARCHAR(64) NOT NULL UNIQUE,
CHECK (CHAR_LENGTH(TRIM(name)) > 0));
What am I doing wrong?
The right way of doing this is to replace the ? with $N where N is the index of the argument that is passed in as extra arguments to db.Exec()
This new statement does not cause an error:
insForm, err := db.Prepare("INSERT INTO al(name, asslia, balance) VALUES( $1 , $2 , $3 )")
I am trying to execute the following query against the PostgreSQL database in Go using pq driver:
SELECT COUNT(id)
FROM tags
WHERE id IN (1, 2, 3)
where 1, 2, 3 is passed at a slice tags := []string{"1", "2", "3"}.
I have tried many different things like:
s := "(" + strings.Join(tags, ",") + ")"
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN $1`, s,
).Scan(&num); err != nil {
log.Println(err)
}
which results in pq: syntax error at or near "$1". I also tried
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN ($1)`, strings.Join(stringTagIds, ","),
).Scan(&num); err != nil {
log.Println(err)
}
which also fails with pq: invalid input syntax for integer: "1,2,3"
I also tried passing a slice of integers/strings directly and got sql: converting Exec argument #0's type: unsupported type []string, a slice.
So how can I execute this query in Go?
Pre-building the SQL query (preventing SQL injection)
If you're generating an SQL string with a param placeholder for each of the values, it's easier to just generate the final SQL right away.
Note that since values are strings, there's place for SQL injection attack, so we first test if all the string values are indeed numbers, and we only proceed if so:
tags := []string{"1", "2", "3"}
buf := bytes.NewBufferString("SELECT COUNT(id) FROM tags WHERE id IN(")
for i, v := range tags {
if i > 0 {
buf.WriteString(",")
}
if _, err := strconv.Atoi(v); err != nil {
panic("Not number!")
}
buf.WriteString(v)
}
buf.WriteString(")")
Executing it:
num := 0
if err := Db.QueryRow(buf.String()).Scan(&num); err != nil {
log.Println(err)
}
Using ANY
You can also use Postgresql's ANY, whose syntax is as follows:
expression operator ANY (array expression)
Using that, our query may look like this:
SELECT COUNT(id) FROM tags WHERE id = ANY('{1,2,3}'::int[])
In this case you can declare the text form of the array as a parameter:
SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])
Which can simply be built like this:
tags := []string{"1", "2", "3"}
param := "{" + strings.Join(tags, ",") + "}"
Note that no check is required in this case as the array expression will not allow SQL injection (but rather will result in a query execution error).
So the full code:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
param := "{" + strings.Join(tags, ",") + "}"
num := 0
if err := Db.QueryRow(q, param).Scan(&num); err != nil {
log.Println(err)
}
This is not really a Golang issue, you are using a string to compare to integer (id) in your SQL request. That means, SQL receive:
SELECT COUNT(id)
FROM tags
WHERE id IN ("1, 2, 3")
instead of what you want to give it. You just need to convert your tags into integer and passe it to the query.
EDIT:
Since you are trying to pass multiple value to the query, then you should tell it:
params := make([]string, 0, len(tags))
for i := range tags {
params = append(params, fmt.Sprintf("$%d", i+1))
}
query := fmt.Sprintf("SELECT COUNT(id) FROM tags WHERE id IN (%s)", strings.Join(params, ", "))
This will end the query with a "($1, $2, $3...", then convert your tags as int:
values := make([]int, 0, len(tags))
for _, s := range tags {
val, err := strconv.Atoi(s)
if err != nil {
// Do whatever is required with the error
fmt.Println("Err : ", err)
} else {
values = append(values, val)
}
}
And finally, you can use it in the query:
Db.QueryRow(query, values...)
This should do it.
Extending #icza solution, you can use pq.Array instead of building the params yourself.
So using his example, the code can look like this:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
num := 0
if err := Db.QueryRow(q, pq.Array(tags)).Scan(&num); err != nil {
log.Println(err)
}
Is it possible to insert multiple rows into Postgres database at once? Could someone please suggest if there is a way to insert a slice of slices into database. I have created a slice for each row and created a another slice(multiple rows) by appending all the row slices to it. how do I insert the slice(multiple rows) into db?
When I create a row slice, I'm using row := []interface{}{} . Because I have fields which are strings and int in each row. Looks like I
get an error when I'm inserting data and the error is unsupported type []interface {}, a slice of interface
Implementation:
rowdata := []interface{}{}
row := []interface{}{data.ScenarioUUID, data.Puid, data.Description, data.Status, data.CreatedBy, data.CreatedAt, data.UpdatedBy, data.UpdatedAt, data.ScopeStartsAt, data.ScopeEndsAt, Metric, MetricName, Channel, date, timeRangeValue}
rowdata = append(rowdata, row)
qry2 := `INSERT INTO sample (scenarioUuid,
puId,
description,
status,
createdBy,
createdAt,
updatedBy,
updatedAt,
scopeStartsAt,
scopeEndsAt,
metric,
metric_name,
channel,
time,
value) VALUES ($1, $2, $3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15)`
if _, err := db.Exec(qry2, rowdata); err != nil {
panic(err)
You could do something like this:
samples := // the slice of samples you want to insert
query := `insert into samples (<the list of columns>) values `
values := []interface{}{}
for i, s := range samples {
values = append(values, s.<field1>, s.<field2>, < ... >)
numFields := 15 // the number of fields you are inserting
n := i * numFields
query += `(`
for j := 0; j < numFields; j++ {
query += `$`+strconv.Itoa(n+j+1) + `,`
}
query = query[:len(query)-1] + `),`
}
query = query[:len(query)-1] // remove the trailing comma
db.Exec(query, values...)
https://play.golang.org/p/YqNJKybpwWB
I have the following code:
package main
import (
"database/sql"
"fmt"
"github.com/lib/pq"
)
const (
DB_USER = "<username>"
DB_PASSWORD = "<password>"
DB_NAME = "<db>"
)
func main() {
dbinfo := fmt.Sprintf("user=%s password=%s dbname=%s sslmode=disable",
DB_USER, DB_PASSWORD, DB_NAME)
db, err := sql.Open("postgres", dbinfo)
checkErr(err)
defer db.Close()
fmt.Println("# Querying with blacklist in SQL")
rows, err := db.Query(`
SELECT * FROM(
SELECT x from (values ('A'), ('B'), ('C') ) s(x)
) As Res1 WHERE x NOT IN ('A');
`)
checkErr(err)
for rows.Next() {
var str string
err = rows.Scan(&str)
fmt.Println(str) // Prints B,C
}
fmt.Println("Querying with blacklist in Golang")
blacklist := []string{"A"}
q := `
SELECT * FROM(
SELECT x from (values ('A'), ('B'), ('C') ) s(x)
) As Res1 WHERE x NOT IN ($1);
`
rows, err = db.Query(q, pq.Array(blacklist))
checkErr(err)
for rows.Next() {
var str string
err = rows.Scan(&str)
fmt.Println(str) // Prints A, B, C
}
}
func checkErr(err error) {
if err != nil {
panic(err)
}
}
where I pass a pq.Array as a parameter to a Golang Postgres format string $1. However the parameter fails to get passed. When I expect an output of B,C, the program is printing A,B,C.
# Querying with blacklist in SQL
B
C
Querying with blacklist in Golang
A
B
C
Postgres's IN does not take an array like for example ANY or ALL which is where you can use pq.Array.
Instead of using x NOT IN($1) you can use x <> ALL($1).
From ALL's docs.
The left-hand expression is evaluated and compared to each element of
the array using the given operator, which must yield a Boolean result.
The result of ALL is "true" if all comparisons yield true (including
the case where the array has zero elements). The result is "false" if
any false result is found.
I am trying to execute the following query against the PostgreSQL database in Go using pq driver:
SELECT COUNT(id)
FROM tags
WHERE id IN (1, 2, 3)
where 1, 2, 3 is passed at a slice tags := []string{"1", "2", "3"}.
I have tried many different things like:
s := "(" + strings.Join(tags, ",") + ")"
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN $1`, s,
).Scan(&num); err != nil {
log.Println(err)
}
which results in pq: syntax error at or near "$1". I also tried
if err := Db.QueryRow(`
SELECT COUNT(id)
FROM tags
WHERE id IN ($1)`, strings.Join(stringTagIds, ","),
).Scan(&num); err != nil {
log.Println(err)
}
which also fails with pq: invalid input syntax for integer: "1,2,3"
I also tried passing a slice of integers/strings directly and got sql: converting Exec argument #0's type: unsupported type []string, a slice.
So how can I execute this query in Go?
Pre-building the SQL query (preventing SQL injection)
If you're generating an SQL string with a param placeholder for each of the values, it's easier to just generate the final SQL right away.
Note that since values are strings, there's place for SQL injection attack, so we first test if all the string values are indeed numbers, and we only proceed if so:
tags := []string{"1", "2", "3"}
buf := bytes.NewBufferString("SELECT COUNT(id) FROM tags WHERE id IN(")
for i, v := range tags {
if i > 0 {
buf.WriteString(",")
}
if _, err := strconv.Atoi(v); err != nil {
panic("Not number!")
}
buf.WriteString(v)
}
buf.WriteString(")")
Executing it:
num := 0
if err := Db.QueryRow(buf.String()).Scan(&num); err != nil {
log.Println(err)
}
Using ANY
You can also use Postgresql's ANY, whose syntax is as follows:
expression operator ANY (array expression)
Using that, our query may look like this:
SELECT COUNT(id) FROM tags WHERE id = ANY('{1,2,3}'::int[])
In this case you can declare the text form of the array as a parameter:
SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])
Which can simply be built like this:
tags := []string{"1", "2", "3"}
param := "{" + strings.Join(tags, ",") + "}"
Note that no check is required in this case as the array expression will not allow SQL injection (but rather will result in a query execution error).
So the full code:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
param := "{" + strings.Join(tags, ",") + "}"
num := 0
if err := Db.QueryRow(q, param).Scan(&num); err != nil {
log.Println(err)
}
This is not really a Golang issue, you are using a string to compare to integer (id) in your SQL request. That means, SQL receive:
SELECT COUNT(id)
FROM tags
WHERE id IN ("1, 2, 3")
instead of what you want to give it. You just need to convert your tags into integer and passe it to the query.
EDIT:
Since you are trying to pass multiple value to the query, then you should tell it:
params := make([]string, 0, len(tags))
for i := range tags {
params = append(params, fmt.Sprintf("$%d", i+1))
}
query := fmt.Sprintf("SELECT COUNT(id) FROM tags WHERE id IN (%s)", strings.Join(params, ", "))
This will end the query with a "($1, $2, $3...", then convert your tags as int:
values := make([]int, 0, len(tags))
for _, s := range tags {
val, err := strconv.Atoi(s)
if err != nil {
// Do whatever is required with the error
fmt.Println("Err : ", err)
} else {
values = append(values, val)
}
}
And finally, you can use it in the query:
Db.QueryRow(query, values...)
This should do it.
Extending #icza solution, you can use pq.Array instead of building the params yourself.
So using his example, the code can look like this:
tags := []string{"1", "2", "3"}
q := "SELECT COUNT(id) FROM tags WHERE id = ANY($1::int[])"
num := 0
if err := Db.QueryRow(q, pq.Array(tags)).Scan(&num); err != nil {
log.Println(err)
}