I have the following sql statement:
SELECT pk, up FROM mytable WHERE 2 > 1 LIMIT 10
This is just for simplicity, obviously. I am able to parameterize any of the integers:
SELECT pk, up FROM mytable WHERE 2 > $1 LIMIT 10
BUT, when I try to parameterize the operator, eg:
SELECT pk, up FROM mytable WHERE 2 $1 1 LIMIT 10
I get:
pq: syntax error at or near "$1"
Full Code:
package main
import (
"database/sql"
_ "github.com/lib/pq"
"log"
)
func main() {
log.SetFlags(log.Lshortfile)
Db, err := sql.Open("postgres", "user=yoitsmeletmein password=supersecretyo host=what.a.host dbname=mydb sslmode=require")
if err != nil {
log.Fatal("Cannot connect to db: ", err)
}
q := `SELECT pk FROM mytable WHERE 2 $1 1 LIMIT 10`
params := []interface{}{">"}
rows, err := Db.Query(q, params...)
if err != nil {
log.Println(err)
} else {
defer rows.Close()
for rows.Next() {
var pk int64
if err := rows.Scan(&pk); err != nil {
log.Fatal(err)
}
log.Println(pk)
}
}
}
Prepared statements allow to parametrize values, nothing else. It wouldn't make sense to parametrize operators to begin with, a statement cannot be prepared without knowing involved operators. And it would be potentially dangerous, opening vectors for SQL injection.
To switch operators, you'll have to concatenate a new query string in your client or use dynamic SQL with a server-side procedural language, the default being plpgsql.
Related
I'm using SELECT * in a db.query() to return columns from a table. Typically, I would fmt.Scan() the rows into a pre-declared struct{} for further manipulation, but in this case, the table columns change frequently so I'm not able to use a declared struct{} as part of my Scan().
I've been struggling to figure out how I might dynamically build a struct{} based on the column results of the db.query() which I could subsequently call on the use of for Scan(). I've read a little about reflect but I'm struggling to determine if this is right for my use-case or if I might have to think about something else.
Any pointers would be greatly appreciated.
you can get column names from resulting rowset and prepare a slice for the scan.
Example (https://go.dev/play/p/ilYmEIWBG5S) :
package main
import (
"database/sql"
"fmt"
"log"
"github.com/DATA-DOG/go-sqlmock"
)
func main() {
// mock db
db, mock, err := sqlmock.New()
if err != nil {
log.Fatal(err)
}
columns := []string{"id", "status"}
mock.ExpectQuery("SELECT \\* FROM table").
WillReturnRows(sqlmock.NewRows(columns).AddRow(1, "ok"))
// actual code
rows, err := db.Query("SELECT * FROM table")
if err != nil {
log.Fatal(err)
}
cols, err := rows.Columns()
if err != nil {
log.Fatal(err)
}
data := make([]interface{}, len(cols))
strs := make([]sql.NullString, len(cols))
for i := range data {
data[i] = &strs[i]
}
for rows.Next() {
if err := rows.Scan(data...); err != nil {
log.Fatal(err)
}
for i, d := range data {
fmt.Printf("%s = %+v\n", cols[i], d)
}
}
}
This example reads all columns into strings. To detect column type one can use rows.ColumnTypes method.
I am inserting some data in table using SQLx like this
func (*RideRepositoryImpl) insert(entity interface{}, tx persistence.Transaction) (sql.Result, error) {
ride := entity.(*model.Ride)
placeHolders := repository.InsertPlaceholders(len(rideColumns))
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s)", TableName, strings.Join(Columns, ","), placeHolders)
return tx.Exec(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
}
and calling this function like this
func (p *RideRepositoryImpl) Save(ride *model.Ride, tx persistence.Transaction) error {
return repository.Save(ride, p.insert, tx)
Now I want to get UUID of saved record instantly after saving this record . Is there any clean way to do this instantly ?
PostgreSQL has the RETURNING clause for this.
Sometimes it is useful to obtain data from modified rows while they
are being manipulated. The INSERT, UPDATE, and DELETE commands
all have an optional RETURNING clause that supports this. Use of
RETURNING avoids performing an extra database query to collect the
data, and is especially valuable when it would otherwise be difficult
to identify the modified rows reliably.
// add the RETURNING clause to your INSERT query
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s) RETURNING <name_of_uuid_column>", TableName, strings.Join(Columns, ","), placeHolders)
// use QueryRow instead of Exec
row := tx.QueryRow(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
// scan the result of the query
var uuid string
if err := row.Scan(&uuid); err != nil {
panic(err)
}
// ...
For additional INSERT-specific info related to RETURNING you can go to the INSERT documentation and search the page for "returning" with CTRL/CMD+F.
If, in addition, you need your function to still return an sql.Result value to satisfy some requirement, then you can return your own implementation.
var _ sql.Result = sqlresult{} // compiler check
type sqlresult struct { lastid, nrows int64 }
func (r sqlresult) LastInsertId() (int64, error) { return r.lastid, nil }
func (r sqlresult) RowsAffected() (int64, error) { return r.nrows, nil }
func (*RideRepositoryImpl) insert(entity interface{}, tx persistence.Transaction) (sql.Result, error) {
ride := entity.(*model.Ride)
placeHolders := repository.InsertPlaceholders(len(rideColumns))
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s) RETURNING <name_of_uuid_column>", TableName, strings.Join(Columns, ","), placeHolders)
row := tx.QueryRow(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
if err := row.Scan(&ride.<NameOfUUIDField>); err != nil {
return nil, err
}
return sqlresult{0, 1}, nil
}
Consider the following code
func main() {
ctx := context.Background()
tx, err := db.BeginTx(ctx, nil)
row := tx.QueryRow("SELECT timestamp, my_id, my_value FROM my_table ORDER BY timestamp ASC limit 1 ")
var my_id int
var my_time time.Time
var my_value string
err = row.Scan(&my_time, &my_id, &my_value)
if err != nil {
tx.Rollback()
return
}
_, err = tx.ExecContext(ctx, "UPDATE my_table SET status = 'start' WHERE my_id = $1", my_id)
if err != nil {
tx.Rollback()
return
}
err = tx.Commit()
if err != nil {
log.Fatal(err)
}
// do something with my_value that takes a long time
}
Which select the row with the oldest timestamp and set the status to start and do something with the return value, I run this transaction in a multi threaded/server environment, how do I make sure that each thread is getting a unique my_id and that no two thread is processing the same my_id?
I don't think the select statement locks the row that got returned during the first select, so multiple thread can attempt to update the same row.
I could modify the update statement to be
UPDATE my_table SET status = 'start' WHERE my_id = $1 AND status <> `start`
but then I have to reselect another id, is there a way to avoid it?
Use pessimistic locking to put an UPDATE lock on the row when reading:
SELECT ... FOR UPDATE
That will prevent concurrent data modifications.
How can I prevent SQL injection attacks in Go while using "database/sql"?
This solves the single value field problem because you can remove the quotes, but I can't do that filtering a JSON/JSONB field, like in the following because the $1 is considered a string:
`SELECT * FROM foo WHERE bar #> '{"baz": "$1"}'`
The following works but it's prone to SQL Injection:
`SELECT * FROM foo WHERE bar #> '{"baz": "` + "qux" + `"}'`
How do I solve this?
EDITED after #mkopriva's comment:
How would I build this json [{"foo": $1}] with the jsonb_* functions? Tried the below without success:
jsonb_build_array(0, jsonb_build_object('foo', $1::text))::jsonb
There's no sql error. The filter just doesn't work. There's a way that I can check the builded sql? I'm using the database/sql native lib.
Is this what you're looking for?
type MyStruct struct {
Baz string
}
func main() {
db, err := sql.Open("postgres", "postgres://...")
if err != nil {
log.Panic(err)
}
s := MyStruct{
Baz: "qux",
}
val, _ := json.Marshal(s)
if err != nil {
log.Panic(err)
}
if _, err := db.Exec("SELECT * FROM foo WHERE bar #> ?", val); err != nil {
log.Panic(err)
}
}
As a side note, Exec isn't for retrieval (although I kept it for you so the solution would match your example). Check out db.Query (Fantastic tutorial here: http://go-database-sql.org/retrieving.html)
I created a simple sql database with a BYTEA field,
create table testhex (testhex bytea);
insert into testhex (testhex) values ('\x123456');
and then I tried to query it from Go.
package main
import (
"database/sql"
_ "github.com/lib/pq"
)
func main(){
var err error
db, err := sql.Open("postgres", "dbname=testhex sslmode=disable")
if err != nil {
panic(err)
}
var result string
err = db.QueryRow("select testhex from testhex where testhex = $1", `\x123456`).Scan(&result)
if err != nil {
panic(err)
}
}
It doesn't find the row. What am I doing wrong?
When you ran the following query:
insert into testhex (testhex) values ('\x123456');
You inserted the 3 byte sequence [0x12 0x34 0x56] into the table. For the database query you're executing with QueryRow though, you're searching for the 8 character literal string \x123456 so you get no matches in the result.
When you use positional arguments with QueryRow, it is the database adapter's job to convert them to a form the database understands (either by sending them to the database as bound parameters, or by substituting them into the query with appropriate escaping). So by passing an already escaped value you will run into this sort of problem.
Instead, try passing []byte{0x12, 0x34, 0x56} as the positional argument, which should match what is in the database.