How can I prevent SQL injection attacks in Go while using "database/sql"?
This solves the single value field problem because you can remove the quotes, but I can't do that filtering a JSON/JSONB field, like in the following because the $1 is considered a string:
`SELECT * FROM foo WHERE bar #> '{"baz": "$1"}'`
The following works but it's prone to SQL Injection:
`SELECT * FROM foo WHERE bar #> '{"baz": "` + "qux" + `"}'`
How do I solve this?
EDITED after #mkopriva's comment:
How would I build this json [{"foo": $1}] with the jsonb_* functions? Tried the below without success:
jsonb_build_array(0, jsonb_build_object('foo', $1::text))::jsonb
There's no sql error. The filter just doesn't work. There's a way that I can check the builded sql? I'm using the database/sql native lib.
Is this what you're looking for?
type MyStruct struct {
Baz string
}
func main() {
db, err := sql.Open("postgres", "postgres://...")
if err != nil {
log.Panic(err)
}
s := MyStruct{
Baz: "qux",
}
val, _ := json.Marshal(s)
if err != nil {
log.Panic(err)
}
if _, err := db.Exec("SELECT * FROM foo WHERE bar #> ?", val); err != nil {
log.Panic(err)
}
}
As a side note, Exec isn't for retrieval (although I kept it for you so the solution would match your example). Check out db.Query (Fantastic tutorial here: http://go-database-sql.org/retrieving.html)
Related
I have a Golang function to fetch all the records from Postgres database, this function is simply using :
SELECT * from stock_transactions
I want to apply filter to this function to fetch records with some conditions, in-short I want to use :
SELECT * from stock_transactions WHERE symbol = $symb
The problem is to handle the case where if $symb = null the query should act as SELECT * from stock_transactions. I can write an if-else clause for the same but if the number of parameters are more than 2 it could be messy. Is there a better way to handle this?
My function:
func showstocks (w http.ResponseWriter, r *http.Request){
var err error
if r.Method != "GET" {
http.Error(w, http.StatusText(405), http.StatusMethodNotAllowed)
return
}
rows, err := db.Query("SELECT * FROM stock_transaction ORDER BY id DESC")
if err != nil {
http.Error(w, http.StatusText(500), 500)
return
}
defer rows.Close()
sks := make([]stockdata, 0)
for rows.Next() {
sk := stockdata{}
err := rows.Scan(&sk.Sname, &sk.Ttype, &sk.Uprice, &sk.Qty, &sk.Bfee, &sk.Ddate)
if err != nil {
http.Error(w, http.StatusText(500), 500)
return
}
sks = append(sks, sk)
}
if err = rows.Err(); err != nil {
http.Error(w, http.StatusText(500), 500)
return
}
tpl.ExecuteTemplate(w, "dashboard.gohtml", sks)
}
//Suggested by #mkopriva. Tried and tested.
package main
import (
"fmt"
)
func main() {
// ...
// use interface{} because string types cannot be nil
var stock_symbol interface{} // the "zero-value of interface types is nil, so stock_symbol here is nil
// if empty then stock_symbol will be left as nil
if val := r.FormValue("stock_symbol"); len(val) > 0 {
stock_symbol = val // set to the provided value
}
// ...
// now stock_symbol is either the provided string value or nil
db.Query("SELECT ... WHERE (stock_symbol = $1 OR $1 IS NULL)", stock_symbol)
}
You can use the (stock_symbol = $1 OR $1 IS NULL) trick as you have already seen, which is programmatically convenient. However this can often lead to inefficient queries, as the query planner may not be smart enough to optimize them correctly. It might be better to write some code which removes the tautological phrases and also removes the corresponding bind variables, rather than passing each of them to the database. It is slightly messy, but it should be encapsulated into a function, not rewritten each time.
Introduction
database/sql
In the Go standard sql library, the *Stmt type has methods defined like:
func (s *Stmt) Exec(args ...interface{}) (Result, error)
func (s *Stmt) Query(args ...interface{}) (*Rows, error)
The a new (unnamed) statement is prepared by:
func (db *DB) Prepare(query string) (*Stmt, error)
Connection pool is abstracted and not directly accessible
A transaction is prepared on a single connection
If the connection is not available at statment execution time, it will be re-prepared on a new connection.
pgx
The PreparedStatement type doesn't have any methods defined. A new named prepared statement is prepared by:
func (p *ConnPool) Prepare(name, sql string) (*PreparedStatement, error)
Operations are directly on the connection pool
The transaction gets prepared on all connections of the pool
There is no clear way how to execute the prepared statement
In a Github comment, the author explains better the differences of architecture between pgx and database/sql. The documentation on Prepare also states (emphasis mine):
Prepare is idempotent; i.e. it is safe to call Prepare multiple times with the same name and sql arguments. This allows a code path to Prepare and Query/Exec/PrepareEx without concern for if the statement has already been prepared.
Small example
package main
import (
"github.com/jackc/pgx"
)
func main() {
conf := pgx.ConnPoolConfig{
ConnConfig: pgx.ConnConfig{
Host: "/run/postgresql",
User: "postgres",
Database: "test",
},
MaxConnections: 5,
}
db, err := pgx.NewConnPool(conf)
if err != nil {
panic(err)
}
_, err = db.Prepare("my-query", "select $1")
if err != nil {
panic(err)
}
// What to do with the prepared statement?
}
Question(s)
The name argument gives me the impression it can be executed by calling it by name, but how?
The documentation gives the impression that Query/Exec methods somehow leverage the prepared statements. However, those methods don't take a name argument. How does it match them?
Presumably, matching is done by the query content. Then what's the whole point of naming statements?
Possible answers
This is how far I got myself:
There are no methods that refer to the queries by name (assumption)
Matching is done on the query body in conn.ExecEx(). If it is not yet prepared, it will be done:
ps, ok := c.preparedStatements[sql]
if !ok {
var err error
ps, err = c.prepareEx("", sql, nil)
if err != nil {
return "", err
}
}
PosgreSQL itself needs it for something (assumption).
#mkopriva pointed out that the sql text was misleading me. It has a double function here. If the sql variable does not match to a key in the c.preparedStatements[sql] map, the query contained in the sql gets prepared and a new *PreparedStatement struct is appointed to ps. If it did match a key, the ps variable will point to an entry of the map.
So effectively you can do something like:
package main
import (
"fmt"
"github.com/jackc/pgx"
)
func main() {
conf := pgx.ConnPoolConfig{
ConnConfig: pgx.ConnConfig{
Host: "/run/postgresql",
User: "postgres",
Database: "test",
},
MaxConnections: 5,
}
db, err := pgx.NewConnPool(conf)
if err != nil {
panic(err)
}
if _, err := db.Prepare("my-query", "select $1::int"); err != nil {
panic(err)
}
row := db.QueryRow("my-query", 10)
var i int
if err := row.Scan(&i); err != nil {
panic(err)
}
fmt.Println(i)
}
I have a small Go program which uses a a postgresql db. In it there is a query which can return no rows, and the code I'm using to deal with this isn't working correctly.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
err := db.QueryRow("SELECT SUM(delta) FROM karma WHERE nick = $1", nick).Scan(&karma)
var karmaStr string
switch {
case err == sql.ErrNoRows:
karmaStr = fmt.Sprintf("%s has no karma.", nick)
case err != nil:
log.Fatal(err)
default:
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
This logic is taken directly from the Go documentation. When there are no rows corresponding to nick, the following error occurs:
2016/07/24 19:37:07 sql: Scan error on column index 0: converting driver.Value type <nil> ("<nil>") to a int: invalid syntax
I must be doing something stupid - clues appreciated.
I believe your issue is that you're getting a NULL value back from the database, which go translates into nil. However, you're scanning into an integer, which has no concept of nil. One thing you can do is scan into a type that implements the sql.Scanner interface (and can handle NULL values), e.g., sql.NullInt64.
In the example code in the documentation, I'd assume they have a NOT NULL constraint on the username column. I think the reason for this is because they didn't want to lead people to believe that you have to use NULL-able types across the board.
I reworked the code to get the results I wanted.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
rows, err := db.Query("SELECT SUM(delta) FROM karma WHERE nick = $1", nick)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
karmaStr := fmt.Sprintf("%s has no karma.", nick)
if rows.Next() {
rows.Scan(&karma)
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
Tempted to submit a documentation patch of some sort to the database/sql package.
I'm brand new to Go, and I've started working on some postgres queries, and I'm having very little luck.
I have a package that's just going to have some database queries in it. Here's my code.
main.go
package main
import (
"fmt"
)
func main() {
fmt.Println("Querying data")
myqueries.SelectAll("mytable")
}
myqueries.go
package myqueries
import (
"database/sql"
"fmt"
)
func SelectAll (table string) {
db, err := sql.Open("postgres","user=postgres dbname=mydb sslmode=disable")
if err != nil {
fmt.Println(err)
}
defer db.Close()
rows, err := db.Query("SELECT * FROM $1", table)
if err != nil {
fmt.Println(err)
} else {
PrintRows(rows)
}
}
func PrintRows(rows *sql.Rows) {
for rows.Next() {
var firstname string
var lastname string
err := rows.Scan(&firstname, &lastname)
if err != nil {
fmt.Println(err)
}
fmt.Println("first name | last name")
fmt.Println("%v | %v\n", firstname, lastname)
}
}
The error I get is pq: syntax error at or near "$1"
which is from myqueries.go file in the db.Query.
I've tried several variations of this, but nothing has worked yet. Any help is appreciated.
It looks like you are using https://github.com/lib/pq based on the error message and it's docs say that
pq uses the Postgres-native ordinal markers, as shown above
I've never known a database engine that allows the parameterized values in anything other than values. I think you are going to have to resort to string concatenation. I don't have a Go compiler available to me right now, but try something like this. Because you are inserting the table name by concatination, you need it sanitized. pq.QuoteIdentifier should be able to help with that.
func SelectAll (table string) {
db, err := sql.Open("postgres","user=postgres dbname=mydb sslmode=disable")
if err != nil {
fmt.Println(err)
}
defer db.Close()
table = pq.QuoteIdentifier(table)
rows, err := db.Query(fmt.Sprintf("SELECT * FROM %v", table))
if err != nil {
fmt.Println(err)
} else {
PrintRows(rows)
}
}
EDIT: Thanks to hobbs to pointing out pq.QuoteIdentifier
I have the following sql statement:
SELECT pk, up FROM mytable WHERE 2 > 1 LIMIT 10
This is just for simplicity, obviously. I am able to parameterize any of the integers:
SELECT pk, up FROM mytable WHERE 2 > $1 LIMIT 10
BUT, when I try to parameterize the operator, eg:
SELECT pk, up FROM mytable WHERE 2 $1 1 LIMIT 10
I get:
pq: syntax error at or near "$1"
Full Code:
package main
import (
"database/sql"
_ "github.com/lib/pq"
"log"
)
func main() {
log.SetFlags(log.Lshortfile)
Db, err := sql.Open("postgres", "user=yoitsmeletmein password=supersecretyo host=what.a.host dbname=mydb sslmode=require")
if err != nil {
log.Fatal("Cannot connect to db: ", err)
}
q := `SELECT pk FROM mytable WHERE 2 $1 1 LIMIT 10`
params := []interface{}{">"}
rows, err := Db.Query(q, params...)
if err != nil {
log.Println(err)
} else {
defer rows.Close()
for rows.Next() {
var pk int64
if err := rows.Scan(&pk); err != nil {
log.Fatal(err)
}
log.Println(pk)
}
}
}
Prepared statements allow to parametrize values, nothing else. It wouldn't make sense to parametrize operators to begin with, a statement cannot be prepared without knowing involved operators. And it would be potentially dangerous, opening vectors for SQL injection.
To switch operators, you'll have to concatenate a new query string in your client or use dynamic SQL with a server-side procedural language, the default being plpgsql.