I have a small Go program which uses a a postgresql db. In it there is a query which can return no rows, and the code I'm using to deal with this isn't working correctly.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
err := db.QueryRow("SELECT SUM(delta) FROM karma WHERE nick = $1", nick).Scan(&karma)
var karmaStr string
switch {
case err == sql.ErrNoRows:
karmaStr = fmt.Sprintf("%s has no karma.", nick)
case err != nil:
log.Fatal(err)
default:
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
This logic is taken directly from the Go documentation. When there are no rows corresponding to nick, the following error occurs:
2016/07/24 19:37:07 sql: Scan error on column index 0: converting driver.Value type <nil> ("<nil>") to a int: invalid syntax
I must be doing something stupid - clues appreciated.
I believe your issue is that you're getting a NULL value back from the database, which go translates into nil. However, you're scanning into an integer, which has no concept of nil. One thing you can do is scan into a type that implements the sql.Scanner interface (and can handle NULL values), e.g., sql.NullInt64.
In the example code in the documentation, I'd assume they have a NOT NULL constraint on the username column. I think the reason for this is because they didn't want to lead people to believe that you have to use NULL-able types across the board.
I reworked the code to get the results I wanted.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
rows, err := db.Query("SELECT SUM(delta) FROM karma WHERE nick = $1", nick)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
karmaStr := fmt.Sprintf("%s has no karma.", nick)
if rows.Next() {
rows.Scan(&karma)
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
Tempted to submit a documentation patch of some sort to the database/sql package.
Related
Golang, pgx:
I am trying to get all rows from t_example (currently 20 items), however for some reason only one returns (the first one). I tried to debug and rows.Next() returns false after the first iteration.
Could you please help me with advice?
I'm a newbie, but I've tried to find similar cases here in advance :)
My code:
func (ts *exampleStorage) GetAll() *[]Example {
q := `SELECT id, name FROM t_example`
rows := ts.client.Query(context.Background(), q)
example := make([]Example, 0)
for rows.Next() {
var ex Example
rows.Scan(&ex.Id, &ex.Name)
example = append(example, ex)
}
return &example
}
Your code doesn't check for errors :
row.Scan(&ex.Id, &ex.Name) could return an error (and, in pgx implementation, this error is fatal for the rows iteration) :
err := rows.Scan(&ex.Id, &ex.Name)
if err != nil {
fmt.Printf("*** rows.Scan error: %s", err)
return nil, err
}
there is a gotcha with sql.Rows / pgx.Rows error checking : you should check if an error occurred after exiting the for rows.Next() { loop :
for rows.Next() {
...
}
// check rows.Err() after the last rows.Next() :
if err := rows.Err(); err != nil {
// on top of errors triggered by bad conditions on the 'rows.Scan()' call,
// there could also be some bad things like a truncated response because
// of some network error, etc ...
fmt.Printf("*** iteration error: %s", err)
return nil, err
}
return example, nil
a side note : in the vast majority of cases you don't want to return a pointer to a slice (e.g: *[]Example) but a slice (e.g: []Example)
I am inserting some data in table using SQLx like this
func (*RideRepositoryImpl) insert(entity interface{}, tx persistence.Transaction) (sql.Result, error) {
ride := entity.(*model.Ride)
placeHolders := repository.InsertPlaceholders(len(rideColumns))
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s)", TableName, strings.Join(Columns, ","), placeHolders)
return tx.Exec(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
}
and calling this function like this
func (p *RideRepositoryImpl) Save(ride *model.Ride, tx persistence.Transaction) error {
return repository.Save(ride, p.insert, tx)
Now I want to get UUID of saved record instantly after saving this record . Is there any clean way to do this instantly ?
PostgreSQL has the RETURNING clause for this.
Sometimes it is useful to obtain data from modified rows while they
are being manipulated. The INSERT, UPDATE, and DELETE commands
all have an optional RETURNING clause that supports this. Use of
RETURNING avoids performing an extra database query to collect the
data, and is especially valuable when it would otherwise be difficult
to identify the modified rows reliably.
// add the RETURNING clause to your INSERT query
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s) RETURNING <name_of_uuid_column>", TableName, strings.Join(Columns, ","), placeHolders)
// use QueryRow instead of Exec
row := tx.QueryRow(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
// scan the result of the query
var uuid string
if err := row.Scan(&uuid); err != nil {
panic(err)
}
// ...
For additional INSERT-specific info related to RETURNING you can go to the INSERT documentation and search the page for "returning" with CTRL/CMD+F.
If, in addition, you need your function to still return an sql.Result value to satisfy some requirement, then you can return your own implementation.
var _ sql.Result = sqlresult{} // compiler check
type sqlresult struct { lastid, nrows int64 }
func (r sqlresult) LastInsertId() (int64, error) { return r.lastid, nil }
func (r sqlresult) RowsAffected() (int64, error) { return r.nrows, nil }
func (*RideRepositoryImpl) insert(entity interface{}, tx persistence.Transaction) (sql.Result, error) {
ride := entity.(*model.Ride)
placeHolders := repository.InsertPlaceholders(len(rideColumns))
sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s) RETURNING <name_of_uuid_column>", TableName, strings.Join(Columns, ","), placeHolders)
row := tx.QueryRow(sql, ride.ID.String(), ride.DeviceIotID, ride.VehicleID.String(), ride.UserID.String(),ride.AdditionComments)
if err := row.Scan(&ride.<NameOfUUIDField>); err != nil {
return nil, err
}
return sqlresult{0, 1}, nil
}
I am slightly confused by the output i'm receiving from my Postgres when querying it with the use of go.
Since I am very new to this I have a hard time even forming the right question for this problem I have, so I'll just leave a code block here, with the output I'm receiving and what I expected to happen. I hope this makes it more understandable.
The connection to the postgres db seems to work fine
rows, err := db.Query("SELECT title FROM blogs;")
fmt.Println("output", rows)
However, this is the output I am receiving.
output &{0xc4200ea180 0x4c0e20 0xc42009a3c0 0x4b4f90 <nil> {{0 0} 0 0 0 0} false <nil> []}
As I said, I am new to postgres and go, and I have no Idea what I am dealing with here.
I was expecting my entire table to return in a somewhat readable format.
I was expecting my entire table to return in a somewhat readable format.
It does not come back in a "readable" format, why would it?
Query returns a struct that you can use to iterate through the rows that matched the query.
Adapting the example in the docs to your case, and assuming your title field is a VARCHAR, something like this should work for you:
rows, err := db.Query("SELECT title FROM blogs;")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var title string
if err := rows.Scan(&title); err != nil {
log.Fatal(err)
}
fmt.Println(title)
}
if err := rows.Err(); err != nil {
log.Fatal(err)
}
I am using the GOB encoding for my project and i figured out (after a long fight) that empty strings are not encoded/decoded correctly. In my code i use a errormessage (string) to report any problems, this errormessage is most of the time empty. If i encode a empty string, it become nothing, and this gives me a problem with decoding. I don't want to alter the encoding/decoding because these parts are used the most.
How can i tell Go how to encode/decode empty strings?
Example:
Playground working code.
Playground not working code.
The problem isn't the encoding/gob module, but instead the custom MarshalBinary/UnmarshalBinary methods you've declared for Msg, which can't correctly round trip an empty string. There are two ways you could go here:
Get rid of the MarshalBinary/UnmarshalBinary methods and rely on GOB's default encoding for structures. This change alone wont' be enough because the fields of the structure aren't exported. If you're happy to export the fields then this is the simplest option: https://play.golang.org/p/rwzxTtaIh2
Use an encoding that can correctly round trip empty strings. One simple option would be to use GOB itself to encode the struct fields:
func (m Msg) MarshalBinary() ([]byte, error) {
var b bytes.Buffer
enc := gob.NewEncoder(&b)
if err := enc.Encode(m.x); err != nil {
return nil, err
}
if err := enc.Encode(m.y); err != nil {
return nil, err
}
if err := enc.Encode(m.z); err != nil {
return nil, err
}
return b.Bytes(), nil
}
// UnmarshalBinary modifies the receiver so it must take a pointer receiver.
func (m *Msg) UnmarshalBinary(data []byte) error {
dec := gob.NewDecoder(bytes.NewBuffer(data))
if err := dec.Decode(&m.x); err != nil {
return err
}
if err := dec.Decode(&m.y); err != nil {
return err
}
return dec.Decode(&m.z)
}
You can experiment with this example here: https://play.golang.org/p/oNXgt88FtK
The first option is obviously easier, but the second might be useful if your real example is a little more complex. Be careful with custom encoders though: GOB includes a few features that are intended to detect incompatibilities (e.g. if you add a field to a struct and try to decode old data), which are missing from this custom encoding.
Go's zero value for a bool type is false. Postgres supports an undefined BOOL type, represented as NULL. This leads to problems when trying to fetch a BOOL value from Postgres in Go:
rows,err := db.Query("SELECT UNNEST('{TRUE,FALSE,NULL}'::BOOL[])");
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var value bool
if err := rows.Scan(&value); err != nil {
log.Fatal(err)
}
fmt.Printf("Value: %t\n",value);
}
Output:
Value: true
Value: false
2014/11/17 09:34:26 sql: Scan error on column index 0: sql/driver: couldn't convert <nil> (<nil>) into type bool
What's the most idiomatic way around this problem? The two solutions I have imagined are neither very attractive:
Don't use Go's bool type. Instead I would probably use a string, and do my own conversion which accounts for nil
Always make sure BOOLs are either TRUE or FALSE in Postgres by using COALESCE() or some other means.
See http://golang.org/pkg/database/sql/#NullBool in the standard library.
NullBool represents a bool that may be null. NullBool implements the
Scanner interface so it can be used as a scan destination, similar to
NullString.
My preference is using a pointer:
for rows.Next() {
var value *bool
if err := rows.Scan(&value); err != nil {
log.Fatal(err)
}
if value == nil {
// Handle nil
}
fmt.Printf("Value: %t\n", *value);
}
In case of using SQL + graphQL (gqlgen) my preference is using pointers as well.
Why?
Because when I generate a schema it will have pointers on the output and this is much easy to connect with a relevant struct which contains pointers (which handle DB operations).