I use this driver to communicate with psql from Go. Now when I issue an update query, I have no possibility to know whether it actually updated anything (it can update 0 rows if such id is not present).
_, err := Db.Query("UPDATE tags SET name=$1 WHERE id=1", name)
I tried to investigate err variable (in the way the doc suggests for Insert statement):
if err == sql.ErrNoRows {
...
}
But even with non-existent id, err is still null.
I also tried to use QueryRow with returning clause:
id := 0
err := Db.QueryRow("UPDATE tags SET name=$1 WHERE id=1 RETURNING id", name).Scan(&id)
But this one fails to scan &id when id=1 is not present in the database.
So what is the canonical way to check whether my update updated anything?
Try using db.Exec() instead of db.Query() for queries that do not return results. Instead of returning a sql.Rows object (which doesn't have a way to check how many rows were affected), it returns a sql.Result object, which has a method RowsAffected() (int64, error). This returns the number of rows affected (inserted, deleted, updated) by any write operations in the query fed to the Exec() call.
res, err := db.Exec(query, args...)
if err != nil {
return err
}
n, err := res.RowsAffected()
if err != nil {
return err
}
// do something with n
Note that if your query doesn't affect any rows directly, but only does so via a subquery, the rows affected by the subquery will not be counted as rows affected for that method call.
Also, as the method comment notes, this doesn't work for all database types, but I know for a fact it works with pq, as we're using that driver ourselves (and using the RowsAffected() method).
Reference links:
https://golang.org/pkg/database/sql/#DB.Exec
https://golang.org/pkg/database/sql/#Result
I've created an upsert like this in go-pg:
db.Model(myModel).Returning("id").
OnConflict("(fieldA) DO UPDATE set fieldB=EXCLUDED.fieldB").Insert()
and now I'd like to read the returned id. How would I do that? All the examples I've seen ignore the result returned by the insert/update queries.
Judging from the example, the ID will be in myModel.
myModel := &MyModel{
FieldA: `Something something something`
}
_, err := db.Model(myModel).
OnConflict("(fieldA) DO UPDATE").
Set("fieldB = EXCLUDED.fieldB").
Insert()
if err != nil {
panic(err)
}
fmt.Println(myModel.Id)
Looking at the Postgres log, it is doing insert into ... returning "id" to get the ID.
I have a collection, where I save accessTokens and their expiration time.
I want to automatically load all the ones that are expiring within the next X days.
I have already confirmed that the Database and collection does contain an entry, that it should return. I have done this by simply setting an empty query to return everything that is in the collection.
// HighestExptime is the timestamp of the latest day that I want to select
query := bson.M{"expiration": bson.M{"$lte": highestExpTime}}
// dbo is just an instance of a *mongo.Database struct
ctx, _ := context.WithTimeout(context.Background(), 30 * time.Second)
cur, err := dbo.Collection(dbLogin.DBCollection).Find(ctx, query)
The Documents look like this:
{
userID // string
accessToken // string
refreshToken // string
expiration // int
}
I would expect it to return every entry where the expiration is lower than the highestExpTime and is therefore "older" in that sense.
But when I actually execute it, it just returns an empty response and no error.
I actually found the problem myself.
Turns out that my other NodeJS process added the Document only with Strings, which is the reason why $lte returned nothing.
I just had to make sure everything related to this is added as an integer/float.
For the record, I'm learning Go. I'm trying to use and the mgo package and I'd like to insert a new document and return this newly created document to user (I'm trying to write a basic API). I've wrote the following code:
EDIT: Here's the struct for the model:
type Book struct {
ISBN string `json:"isbn"`
Title string `json:"title"`
Authors []string `json:"authors"`
Price string `json:"price"`
}
session := s.Copy()
defer session.Close()
var book Book
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&book)
if err != nil {
ErrorWithJSON(w, "Incorrect body", http.StatusBadRequest)
return
}
c := session.DB("store").C("books")
info, err := c.Upsert(nil, book)
if err != nil {
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed insert book: ", err)
return
}
respBody, err := json.MarshalIndent(info, "", " ")
if err != nil {
log.Fatal(err)
}
ResponseWithJSON(w, respBody, http.StatusOK)
Please note that Book is a struct I have created earlier. The above code does work but what it returns is the upsert result like so:
{
"Updated": 1,
"Removed": 0,
"Matched": 1,
"UpsertedId": null
}
Which is not the recently created object. How can I get the the recently created object to return as a response (please note that ideally I'd like the confirmation that the document was successfully inserted. I have seen other questions where the ID is generated beforehand but for what I've seen it doesn't confirm that the document was created was it?)
Let's clear the concepts first. In MongoDB, each document must have an _id property which acts as its unique document identifier inside a collection. Either you provide the value of this _id or it is assigned automatically by MongoDB.
So it would be ideal (or it's strongly recommended) for your model types to include a field for the _id document identifier. Since we're talking about books here, books already have a unique identifier called ISBN, which you may opt to use as the value of the _id field.
The mapping between MongoDB fields and Go struct fields must be specified using the bson tag (not json). So you should provide bson tag values along with json tags.
So change your model to:
type Book struct {
ISBN string `json:"isbn" bson:"_id"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
If you want to insert a new document (a new book), you should always use Collection.Insert().
And what will be the ID of the newly inserted document? The field you set to the Book.ISBN field as we declared it to be the document ID with the bson:"_id" tag.
You should only use Collection.Upsert() if you are not sure whether the document already exists, but either way you want it to be the document you have at hand. Collection.Upsert() will try to find a document to update, and if one is found, that will be updated. If no document is found, then an insert operation will be performed. The first parameter is the selector to find the document to be updated. Since you passed nil, that means any document may qualify, so one will be selected "randomly". So if you already have books saved, any may get selected and get overwritten. This is certainly not want you want.
Since now the ISBN is the ID, you should specify a selector that filters by ISBN, like this:
info, err := c.Upsert(bson.M{"_id": book.ISBN}, book)
Or since we're filtering by ID, use the more convenient Collection.UpsertId():
info, err := c.UpsertId(book.ISBN, book)
If you want to update an existing document, for that you may use Collection.Update(). This is similar to Collection.Upsert(), but the difference is that if no documents match the selector, an insert will not be performed. Updating a document matched by ID can also be done with the more convenient Collection.UpdateId() (which is analogue to Collection.UpsertId()).
For other documents which do not have a unique identifier naturally (like books having ISBN), you may use generated IDs. The mgo library provides the bson.NewObjectId() function for such purpose, which returns you a value of type bson.ObjectId. When saving new documents with Collection.Insert(), you can acquire a new unique ID with bson.NewObjectId() and assign it to the struct field that is mapped to the MongoDB _id property. If the insert succeeds, you can be sure the document's ID is what you just set before calling Collection.Insert(). This ID generation is designed to work even in a distributed environment, so it will generate unique IDs even if 2 of your nodes attempt to generate an ID at the same time.
So for example if you don't have the ISBN for a book when saving it, then you must have a separate, designated ID field in your Book type, for example:
type Book struct {
ID bson.ObjectId `bson:"_id"`
ISBN string `json:"isbn" bson:"isbn"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
And when saving a new book:
var book Book
// Fill what you have about the book
book.ID = bson.NewObjectId()
c := session.DB("store").C("books")
err = c.Insert(book)
// check error
// If no error, you can refer to this document via book.ID
Banged my head for a while on this.
info, err := c.Upsert(nil, book)
The nil selector for the upsert will match everything so if your collection is empty the selector won't match and all will be fine, the info object will contain the ObjectId in the UpsertedId field, but on every following upsert with nil selector the collection will have records and the nil selector for the upsert will match, therefore it won't return you the UpsertedId and you will be updating the first matched record.
You could use never matching selector for the upsert (which was my solution), but note that this way you will only insert and get the inserted ObjectId and never update a record.
You can checkout the following test:
func TestFoo(t *testing.T) {
foo := struct {
Bar string `bson:"bar"`
}{
Bar: "barrr",
}
session, _ := mgo.DialWithInfo(&mgo.DialInfo{
Addrs: []string{"127.0.0.1:27017"},
})
coll := session.DB("foo").C("bar")
coll.DropCollection()
info, _ := coll.Upsert(nil, foo) // will insert
count, _ := coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35e8ee1f5b80c932b44afb")}, collection records:1
info, _ = coll.Upsert(bson.M{}, foo) // will update
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:1 Removed:0 Matched:1 UpsertedId:<nil>}, collection records:1
info, _ = coll.Upsert(bson.M{"nonsense": -1}, foo) // will insert duplicate record (due to the selector that will never match anything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35ea2a1f5b80c932b44b1d")}, collection records:2
foo.Bar = "baz"
info, _ = coll.Upsert(nil, foo) // will update the first matched (since the selector matches everything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count)
// after the test the collection will have the following records
//> use foo
//> db.bar.find()
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b81"), "bar" : "baz" } // the first matched on the last update with nil selector
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b86"), "bar" : "barrr" } // the duplicated record with the selector that never matches anything
}
EDIT: Note that you should have an index on the never matching field, or your insert query will take too long if you have many records, because the upsert on not indexed filed will scan all the documents in the collection.
According to the documentation (http://godoc.org/launchpad.net/mgo/v2) you can obtain the ID of your "Upserted" document if you use the Upsert method.
There is also an Insert method that does not provide this functionality.
Why is that? What if I want to perform an Insert instead of an Upsert? (or wouldn't ever be any valid reason to want to do that? I'm starting to wonder.)
You use bson.NewObjectId() to generate an ID to be inserted.
This is how you'd insert a new document:
i := bson.NewObjectId()
c.Insert(bson.M{"_id": i, "foo": "bar"})
Since you don't know if you're going to insert or update when you issue an Upsert, it would be superfluous to generate an ID just to drop it right after the query (in case an update happens). That's why it's generated db-side and returned to you when applicable.
This should not happen at all, the mgo should insert and return the Id, since, if we generated the ObjectId from the application itself, If the application is restarted, the Object Id generator will start from the beginning generating the same IDs again and again, thus updating existing records in the database.
That is wrong, MGO should rely on the database in generating those IDs and updating the object or returning the objectId of the inserted object immediately like what other languages binding to MongoDB does like in Python or Java.
You can always try the Upsert function to get the generated ID.
db := service.ConnectDb()
sessionCopy := db.Copy()
defer sessionCopy.Close() // clean up
collection := sessionCopy.DB(service.MongoDB.DTB).C(MessageCol.tbl)
log.Println("before to write: ", msg)
// Update record inserts and creates an ID if wasn't set (Returns created record with new Id)
info, err := collection.Upsert(nil, msg)
if err != nil {
log.Println("Error write message upsert collection: ", err)
return MessageMgo{}, err
}
if info.UpsertedId != nil {
msg.Id = info.UpsertedId.(bson.ObjectId)
}
// gets room from mongo
room, err := GetRoom(msg.Rid)
if err != nil {
return msg, err
}
// increments the msgcount and update it
room.MsgCount = room.MsgCount + 1
err = UpdateRoom(room)
if err != nil {
return msg, err
}
return msg, err
This is a sample code I have and works fine.....