I'm using the mongo-go driver for Go to save some documents on mongoDb. Everyhting works fine, but i'm wondering if there is a way to change how the ID is auto-generated. Right now the document model in the code has the primitive.ObjectID type, is something like this
type Review struct {
ID primitive.ObjectID `json:"id,omitempty" bson:"_id,omitempty"`
Title string `json:"title"`
Text string `json:"text"`
Rate float64 `json:"rate"`
Date time.Time `json:"date"`
Product Product `json:"product"`
}
And the document created is something like this
{
"_id" : ObjectId("5d6f739a20d42db438016cb1"),
"title" : "test-id",
"text" : "rev-text",
"rate" : 5.0,
"date" : ISODate("2019-09-02T12:18:00.000+02:00"),
"status" : "pending"
}
So far so good. However i want the ID to be a UUID, not an ObjectId. I know i can change the struct ID type to UUID or string and set that field when i'm saving the document, however i want to know if there is a way to change the default ObjectID generator of mongoDb so it generates a UUID automatically when saving a new document
You don't have to use an ObjectID for _id:
ID string `json:"id,omitempty" bson:"_id,omitempty"`
Then you can insert any value you want for _id. However, you won't be able to auto-generate ID this way, you have to generate the ID yourself, and insert it with the generated ID.
For the record, I'm learning Go. I'm trying to use and the mgo package and I'd like to insert a new document and return this newly created document to user (I'm trying to write a basic API). I've wrote the following code:
EDIT: Here's the struct for the model:
type Book struct {
ISBN string `json:"isbn"`
Title string `json:"title"`
Authors []string `json:"authors"`
Price string `json:"price"`
}
session := s.Copy()
defer session.Close()
var book Book
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&book)
if err != nil {
ErrorWithJSON(w, "Incorrect body", http.StatusBadRequest)
return
}
c := session.DB("store").C("books")
info, err := c.Upsert(nil, book)
if err != nil {
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed insert book: ", err)
return
}
respBody, err := json.MarshalIndent(info, "", " ")
if err != nil {
log.Fatal(err)
}
ResponseWithJSON(w, respBody, http.StatusOK)
Please note that Book is a struct I have created earlier. The above code does work but what it returns is the upsert result like so:
{
"Updated": 1,
"Removed": 0,
"Matched": 1,
"UpsertedId": null
}
Which is not the recently created object. How can I get the the recently created object to return as a response (please note that ideally I'd like the confirmation that the document was successfully inserted. I have seen other questions where the ID is generated beforehand but for what I've seen it doesn't confirm that the document was created was it?)
Let's clear the concepts first. In MongoDB, each document must have an _id property which acts as its unique document identifier inside a collection. Either you provide the value of this _id or it is assigned automatically by MongoDB.
So it would be ideal (or it's strongly recommended) for your model types to include a field for the _id document identifier. Since we're talking about books here, books already have a unique identifier called ISBN, which you may opt to use as the value of the _id field.
The mapping between MongoDB fields and Go struct fields must be specified using the bson tag (not json). So you should provide bson tag values along with json tags.
So change your model to:
type Book struct {
ISBN string `json:"isbn" bson:"_id"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
If you want to insert a new document (a new book), you should always use Collection.Insert().
And what will be the ID of the newly inserted document? The field you set to the Book.ISBN field as we declared it to be the document ID with the bson:"_id" tag.
You should only use Collection.Upsert() if you are not sure whether the document already exists, but either way you want it to be the document you have at hand. Collection.Upsert() will try to find a document to update, and if one is found, that will be updated. If no document is found, then an insert operation will be performed. The first parameter is the selector to find the document to be updated. Since you passed nil, that means any document may qualify, so one will be selected "randomly". So if you already have books saved, any may get selected and get overwritten. This is certainly not want you want.
Since now the ISBN is the ID, you should specify a selector that filters by ISBN, like this:
info, err := c.Upsert(bson.M{"_id": book.ISBN}, book)
Or since we're filtering by ID, use the more convenient Collection.UpsertId():
info, err := c.UpsertId(book.ISBN, book)
If you want to update an existing document, for that you may use Collection.Update(). This is similar to Collection.Upsert(), but the difference is that if no documents match the selector, an insert will not be performed. Updating a document matched by ID can also be done with the more convenient Collection.UpdateId() (which is analogue to Collection.UpsertId()).
For other documents which do not have a unique identifier naturally (like books having ISBN), you may use generated IDs. The mgo library provides the bson.NewObjectId() function for such purpose, which returns you a value of type bson.ObjectId. When saving new documents with Collection.Insert(), you can acquire a new unique ID with bson.NewObjectId() and assign it to the struct field that is mapped to the MongoDB _id property. If the insert succeeds, you can be sure the document's ID is what you just set before calling Collection.Insert(). This ID generation is designed to work even in a distributed environment, so it will generate unique IDs even if 2 of your nodes attempt to generate an ID at the same time.
So for example if you don't have the ISBN for a book when saving it, then you must have a separate, designated ID field in your Book type, for example:
type Book struct {
ID bson.ObjectId `bson:"_id"`
ISBN string `json:"isbn" bson:"isbn"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
And when saving a new book:
var book Book
// Fill what you have about the book
book.ID = bson.NewObjectId()
c := session.DB("store").C("books")
err = c.Insert(book)
// check error
// If no error, you can refer to this document via book.ID
Banged my head for a while on this.
info, err := c.Upsert(nil, book)
The nil selector for the upsert will match everything so if your collection is empty the selector won't match and all will be fine, the info object will contain the ObjectId in the UpsertedId field, but on every following upsert with nil selector the collection will have records and the nil selector for the upsert will match, therefore it won't return you the UpsertedId and you will be updating the first matched record.
You could use never matching selector for the upsert (which was my solution), but note that this way you will only insert and get the inserted ObjectId and never update a record.
You can checkout the following test:
func TestFoo(t *testing.T) {
foo := struct {
Bar string `bson:"bar"`
}{
Bar: "barrr",
}
session, _ := mgo.DialWithInfo(&mgo.DialInfo{
Addrs: []string{"127.0.0.1:27017"},
})
coll := session.DB("foo").C("bar")
coll.DropCollection()
info, _ := coll.Upsert(nil, foo) // will insert
count, _ := coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35e8ee1f5b80c932b44afb")}, collection records:1
info, _ = coll.Upsert(bson.M{}, foo) // will update
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:1 Removed:0 Matched:1 UpsertedId:<nil>}, collection records:1
info, _ = coll.Upsert(bson.M{"nonsense": -1}, foo) // will insert duplicate record (due to the selector that will never match anything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35ea2a1f5b80c932b44b1d")}, collection records:2
foo.Bar = "baz"
info, _ = coll.Upsert(nil, foo) // will update the first matched (since the selector matches everything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count)
// after the test the collection will have the following records
//> use foo
//> db.bar.find()
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b81"), "bar" : "baz" } // the first matched on the last update with nil selector
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b86"), "bar" : "barrr" } // the duplicated record with the selector that never matches anything
}
EDIT: Note that you should have an index on the never matching field, or your insert query will take too long if you have many records, because the upsert on not indexed filed will scan all the documents in the collection.
I have the following document in my local mongodb:
_id 25dd9d29-efd5-4b4e-8af0-360c49fdba31
name Reykjavik
initialDiseaseColouring blue
In my code I set up a city structure as the following:
type City struct {
ID bson.ObjectId `bson:"_id,omitempty"`
Name string
InitialDiseaseColouring string
}
I'm querying it using
result := City{}
collection.Find(bson.M{"name":"Reykjavik"}).One(&result)
When I try to access the initialDiseaseColouring attribute it's not there
This is result when I print it:
{ObjectIdHex("32356464396432392d656664352d346234652d386166302d333630633439666462613331") Reykjavik }
Does anyone know why?
I was following the example on https://gist.github.com/border/3489566
By default, the bson codec uses the lowercased field name as the key. Use the field tag to specify a different key:
type City struct {
ID bson.ObjectId `bson:"_id,omitempty"`
Name string
InitialDiseaseColouring string `bson:"initialDiseaseColouring"`
}
The addition of the field tag changes the key from "initialdiseasecolouring" to "initialDiseaseColouring".
So recently I've been trying to insert different types of data into structures using mgo, and I've ran into a bit of a snag. When trying to insert an array of object IDs, I can't seem to figure out that format if I were to populate that object ID array within the structure.
Here's the structure as follows
type Group struct {
ID bson.ObjectId `json:"id" bson:"_id"`
GroupName string `json:"groupName"`
UserIDs []*bson.ObjectId `json:"users" bson:"userid"`
Expected []*int `json:"expected"`
Actual []*int `json:"actual"`
}
And the operation I am trying to run is to insert a new "test" table with a single userID into UserIDs.
array := []*bson.ObjectId{&findJ.ID}
c = Session.DB("test").C("Group")
err = c.Insert(&Group{GroupName: "test", UserIDs: array})
ThisPanic(err)
Where findJ has it's own ID from a separate structure. What am I doing wrong here, as it is causing a panic on inserting the array.
I am querying a collection that includes an integer value among it's values, and loading resulting documents into this struct:
type Subscription struct {
Id bson.ObjectId "_id,omitempty"
Listen string
Job string
TimeoutSeconds int
Data string
}
var subscription Subscription
subscriptions := subscriptionsCol.Find(bson.M{"listen": "example_channel"}).Iter()
for subscriptions.Next(&subscription) {
log("Pending job: %s?%s (timeout: %d)\n",
subscription.Job,
subscription.Data,
subscription.TimeoutSeconds)
}
This is what phpMoAdmin shows me:
[_id] => MongoId Object (
[$id] => 502ed8d84eaead30a1351ea7
)
[job] => partus_test_job_a
[TimeoutSeconds] => 30
[listen] => partus.test
[data] => a=1&b=9
It puzzles me that subscription.TimeoutSeconds contains always 0, when I'm positive I have 30 in the document I inserted in the collection.
All other values are retrieved OK.
What's wrong with the int type?
Have you tried setting the "key" value for that field?
Unmarshal
The lowercased field name is used as the key for each exported field,
but this behavior may be changed using the respective field tag.
type Subscription struct {
Id bson.ObjectId "_id,omitempty"
Listen string
Job string
TimeoutSeconds int "TimeoutSeconds"
Data string
}
The other fields are working fine because their lowercase value matches your Mongo fields in the collection, whereas TimeoutSeconds is using the TitleCase. What is happening is the int field is being left at its zero value, since the Unmarshal can't map a field to it.
When UnMarshalling data, there are multiple keys that are supported.
Below are some examples:
type T struct {
A bool
B int "myb"
C string "myc,omitempty"
D string `bson:",omitempty" json:"jsonkey"`
E int64 ",minsize"
F int64 "myf,omitempty,minsize"
}
The general spec for 1 key-value pair during marshal is :
"[<key>][,<flag1>[,<flag2>]]"
`(...) bson:"[<key>][,<flag1>[,<flag2>]]" (...)`
GO provides support for particular keywords like bson (for mongo keys) and json for setting the json key in a resposne.
Check the Marshal GO Reference for more information.
Similarly there are some frameworks which provide further options to define the keys befor parsing. For example, in sql jinzhu github library gives support for setting default values, column ids to map, etc.
Anyone can use this feature and provide customized support.