Obtain ObjectIdHex value from mgo query - mongodb

I'm still new to go and while I see multiple questions on SO similar to this, I'm unable to reproduce the output some OP's had requested (this answer looking the closest).
I'm doing something fairly simple, I'm hitting a users collection in mongo and all I want to do is get the _id value back as a string. I'm going to eventually push these _id's up to NSQ but that's the brunt of my task.
var users []bson.M
err = sess.DB("db_name").C("users").Find(bson.M{}).All(&users)
if err != nil {
os.Exit(1)
}
for _, user := range users {
fmt.Printf("%+v \n", user["_id"])
}
Today this outputs:
ObjectIdHex("537f700b537461b70c5f0000")
ObjectIdHex("537f700b537461b70c600000")
ObjectIdHex("537f700b537461b70c610000")
ObjectIdHex("537f700b537461b70c620000")
I went through the bson#m docs and thought I was correctly using the map in order to extra the value. So I think, my query results in:
{"_id" : ObjectIdHex("Some_ID") }
but if ObjectIdHex("ID") is the value, how do I simply get the string within there.
So ideal output:
"537f700b537461b70c5f0000"
"537f700b537461b70c600000"
"537f700b537461b70c610000"
"537f700b537461b70c620000"

The value associated with key "_id" is of type bson.ObjectId which is simply a string.
bson.M is a type map[string]interface{}, so you need Type assertion to get the id as an ObjectId:
objid, ok := m["_id"].(ObjectId)
if !ok {
panic("Not ObjectId")
}
And the ObjectId has a ObjectId.Hex() method which returns exactly what you want: the object id as a "pure" hex string:
fmt.Println(objid.Hex())
Alternatives
objid can simply be converted to string because its underlying type is string. So you can use a number of further options to convert it to a hex string:
hexid := fmt.Sprintf("%x", string(objid))
If you just want to print it, you can do directly:
fmt.Printf("%x", string(objid))
Note: Converting it to string is important else the fmt package would call its String() method which results in a string like ObjectIdHex("537f700b537461b70c5f0000") and this is what would be converted to hex which is clearly not what you want.
Alternatively you can use the encoding/hex package and the hex.EncodeToString() function:
hexid := hex.EncodeToString([]byte(objid))

Related

Issues running Find().All() using go mongo-driver

I'm new to mongoDB , Currently we are trying to migrate our old mgo driver to go mongo-driver
In our old code we use something like below from global sign mgo driver
//where apps is a struct
apps := []model.App{}
err = mgo.Collection.Find(query).Skip(skipCount).Limit(MaxResults).Sort("-starttime").All(&apps)
so with the new mongo-driver I tried something like below using Find but that did not work .
// Set FindOneOptions
findOpt := options.Find()
findOpt.SetSkip(int64(skipCount))
limitVal := appsbody.MaxResults
findOpt.SetLimit(int64(limitVal))
findOpt.SetSort("-starttime")
err = mgo.Collection.Find(query, findOpt).All(context.TODO(), &apps)
In the Above snippet params query is of type map[string]interface{}.
when I tried to log the query Key = type Value = dbuser both are of type string
The query value is originally passed by using query := url.Values{}, this case the query type will be map[string][]string
I think later that is passed as map[string]interface{} not sure if that is causing this problem & not able to blend with the right format for query params , So I even tried to converting that using below code , still that did not help me resolve the problem .
//do a type conversion for the original query
q := make(map[string]interface{}, len(query))
for key, value := range query {
q[key] = value
}
when I try to run the code it fails to do Find operation & I get the below err & throws nil pointer
cannot transform type string to a BSON Document: WriteString can only write while positioned on a Element or Value but is positioned on a TopLevel
panic: runtime error: invalid memory address or nil pointer dereference
panic: runtime error: invalid memory address or nil pointer dereference
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x564171020634]
goroutine 1 [running]:
go.mongodb.org/mongo-driver/mongo.(*Cursor).closeImplicitSession(0x5641708ab4e0?)
go.mongodb.org/mongo-driver#v1.10.3/mongo/cursor.go:309 +0x14
panic({0x5641716c9440, 0x56417200c2d0})
runtime/panic.go:884 +0x212
go.mongodb.org/mongo-driver/mongo.(*Cursor).Close(0xa1?, {0x5641718f9c30?, 0xc00003a078?})
go.mongodb.org/mongo-driver#v1.10.3/mongo/cursor.go:222 +0x5f
panic({0x5641716c9440, 0x56417200c2d0})
runtime/panic.go:884 +0x212
go.mongodb.org/mongo-driver/mongo.(*Cursor).All(0x0, {0x5641718f9c30, 0xc00003a078}, {0x5641715fa1c0?, 0xc0001d6480?})
go.mongodb.org/mongo-driver#v1.10.3/mongo/cursor.go:251 +0x1ff
cr/dev/usvcs/apps/appDBAccess.DbService.GetApps({{0x5641718fda98, 0xc000484960}, {0x5641718f7228, 0xc000012750}, {0x5641718fce68, 0xc000014030}, {0x564171901cb8, 0x5641720bc5d8}}, 0xc0001dbbf0, {0x0, ...})
Not sure what mistake am i doing here , Can anyone help me with this ?
The problem is with the sort value. It must be a document, not a simple string. It may be a map, a bson.M (it's also a map) or a bson.D value (or any other values that marshals "nicely" into BSON, e.g. a struct).
If you only use a single field to sort by, simplest is a bson.M. Also note that method calls on options can be chained (they return the receiver):
findOpt := options.Find().
SetSkip(int64(skipCount)).
SetLimit(int64(appsbody.MaxResults)).
SetSort(bson.M{"starttime": -1})
If you'd have multiple sort keys, order does matter, in which case use a bson.D document (maps are unordered, bson.D is an ordered list of key-value pairs):
findOpt := options.Find().
SetSkip(int64(skipCount)).
SetLimit(int64(appsbody.MaxResults)).
SetSort(bson.D{{Key:"starttime", Value: -1}, {Key:"other", Value: 1}})

How to use Find().Select().One() in go mongo-driver library

This code is working fine in go mgo library
result interface{}
err = getCollection.Find(bson.M{}).Select(bson.M{"_id": 1}).One(&result)
but I want to perform this using go mongo-driver library
I have tried below code but it is not working as the above one
err = getCollection.FindOne(ctx, bson.M{}, options.FindOne().SetProjection(bson.M{"_id": 1})).Decode(&result)
My test collection data is
example{
"_id":ObjectId(),
"Name":"qwert"
}
Anyone suggest me how can we achieve this in mongo-driver?
i can't comment on your question because i am new contributor here, i am using mongo-driver in my project now, i have tried to fetch only projection only some fields to be show up,
can you specific the argument on second for filtering ?
var (
opt options.FindOneOptions
modelStruct model.Person
)
filter := bson.M{"email": "hello#test.com"}
opt.SetProjection(bson.M{"name": 1})
err := collection.findOne(context.Background(), filter, opt).Decode(&modelStruct)
if that doesn't work, then you should limit the struct , make sure in your model.Person has data like this
type Person struct {
Name string `json:"name" bson:"name"`
Gender string `json:"gender" bson:"gender"`
}
or you can just make own model for limit the fields:
var personLimitOnlyGetName struct {
Name string `json:"name" bson:"name"`
}
// please look carefully in your collection field for camelCase
opt.SetProjection(bson.M{"name": 1})

Mongo Go Driver is getting interface conversion error when SetSort used

I would like to change order my documents in Mongo DB using Go. I have valid json string code and i can marshal it successfully in to map[string]int. The sample of this type just like:
[{year 1}, {lastupdated -1}]. The value presents order year field ascending and lastupdated field descending. This struct is aspect of MongoDB understands. Also I'm pass this data into bson.D type. Here is my code:
if queries["order"] != nil {
var unmarshalledOrder map[string]int
json.Unmarshal(queries["order"].([]byte), &unmarshalledOrder)
docRes := make(bson.D, 0)
for field, sort := range unmarshalledOrder {
docRes = append(docRes, bson.DocElem{field, sort})
}
log.Println(docRes)
}
When I print docRes, everthing goes well. But i pass the data to options.Sort function, the function throws interface conversion: interface {} is runtime.errorString, not string panic. Is it a bug on mongo go driver or am i wrong?
Can you post the code you've written which uses the driver? Based on the use of bson.DocElem, I think you're using mgo, but mgo's Query.Sort method takes strings, not documents (https://pkg.go.dev/github.com/globalsign/mgo?tab=doc#Query.Sort).

How to insert a document with mgo and get the value returned

For the record, I'm learning Go. I'm trying to use and the mgo package and I'd like to insert a new document and return this newly created document to user (I'm trying to write a basic API). I've wrote the following code:
EDIT: Here's the struct for the model:
type Book struct {
ISBN string `json:"isbn"`
Title string `json:"title"`
Authors []string `json:"authors"`
Price string `json:"price"`
}
session := s.Copy()
defer session.Close()
var book Book
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&book)
if err != nil {
ErrorWithJSON(w, "Incorrect body", http.StatusBadRequest)
return
}
c := session.DB("store").C("books")
info, err := c.Upsert(nil, book)
if err != nil {
ErrorWithJSON(w, "Database error", http.StatusInternalServerError)
log.Println("Failed insert book: ", err)
return
}
respBody, err := json.MarshalIndent(info, "", " ")
if err != nil {
log.Fatal(err)
}
ResponseWithJSON(w, respBody, http.StatusOK)
Please note that Book is a struct I have created earlier. The above code does work but what it returns is the upsert result like so:
{
"Updated": 1,
"Removed": 0,
"Matched": 1,
"UpsertedId": null
}
Which is not the recently created object. How can I get the the recently created object to return as a response (please note that ideally I'd like the confirmation that the document was successfully inserted. I have seen other questions where the ID is generated beforehand but for what I've seen it doesn't confirm that the document was created was it?)
Let's clear the concepts first. In MongoDB, each document must have an _id property which acts as its unique document identifier inside a collection. Either you provide the value of this _id or it is assigned automatically by MongoDB.
So it would be ideal (or it's strongly recommended) for your model types to include a field for the _id document identifier. Since we're talking about books here, books already have a unique identifier called ISBN, which you may opt to use as the value of the _id field.
The mapping between MongoDB fields and Go struct fields must be specified using the bson tag (not json). So you should provide bson tag values along with json tags.
So change your model to:
type Book struct {
ISBN string `json:"isbn" bson:"_id"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
If you want to insert a new document (a new book), you should always use Collection.Insert().
And what will be the ID of the newly inserted document? The field you set to the Book.ISBN field as we declared it to be the document ID with the bson:"_id" tag.
You should only use Collection.Upsert() if you are not sure whether the document already exists, but either way you want it to be the document you have at hand. Collection.Upsert() will try to find a document to update, and if one is found, that will be updated. If no document is found, then an insert operation will be performed. The first parameter is the selector to find the document to be updated. Since you passed nil, that means any document may qualify, so one will be selected "randomly". So if you already have books saved, any may get selected and get overwritten. This is certainly not want you want.
Since now the ISBN is the ID, you should specify a selector that filters by ISBN, like this:
info, err := c.Upsert(bson.M{"_id": book.ISBN}, book)
Or since we're filtering by ID, use the more convenient Collection.UpsertId():
info, err := c.UpsertId(book.ISBN, book)
If you want to update an existing document, for that you may use Collection.Update(). This is similar to Collection.Upsert(), but the difference is that if no documents match the selector, an insert will not be performed. Updating a document matched by ID can also be done with the more convenient Collection.UpdateId() (which is analogue to Collection.UpsertId()).
For other documents which do not have a unique identifier naturally (like books having ISBN), you may use generated IDs. The mgo library provides the bson.NewObjectId() function for such purpose, which returns you a value of type bson.ObjectId. When saving new documents with Collection.Insert(), you can acquire a new unique ID with bson.NewObjectId() and assign it to the struct field that is mapped to the MongoDB _id property. If the insert succeeds, you can be sure the document's ID is what you just set before calling Collection.Insert(). This ID generation is designed to work even in a distributed environment, so it will generate unique IDs even if 2 of your nodes attempt to generate an ID at the same time.
So for example if you don't have the ISBN for a book when saving it, then you must have a separate, designated ID field in your Book type, for example:
type Book struct {
ID bson.ObjectId `bson:"_id"`
ISBN string `json:"isbn" bson:"isbn"`
Title string `json:"title" bson:"title"`
Authors []string `json:"authors" bson:"authors"`
Price string `json:"price" bson:"price"`
}
And when saving a new book:
var book Book
// Fill what you have about the book
book.ID = bson.NewObjectId()
c := session.DB("store").C("books")
err = c.Insert(book)
// check error
// If no error, you can refer to this document via book.ID
Banged my head for a while on this.
info, err := c.Upsert(nil, book)
The nil selector for the upsert will match everything so if your collection is empty the selector won't match and all will be fine, the info object will contain the ObjectId in the UpsertedId field, but on every following upsert with nil selector the collection will have records and the nil selector for the upsert will match, therefore it won't return you the UpsertedId and you will be updating the first matched record.
You could use never matching selector for the upsert (which was my solution), but note that this way you will only insert and get the inserted ObjectId and never update a record.
You can checkout the following test:
func TestFoo(t *testing.T) {
foo := struct {
Bar string `bson:"bar"`
}{
Bar: "barrr",
}
session, _ := mgo.DialWithInfo(&mgo.DialInfo{
Addrs: []string{"127.0.0.1:27017"},
})
coll := session.DB("foo").C("bar")
coll.DropCollection()
info, _ := coll.Upsert(nil, foo) // will insert
count, _ := coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35e8ee1f5b80c932b44afb")}, collection records:1
info, _ = coll.Upsert(bson.M{}, foo) // will update
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:1 Removed:0 Matched:1 UpsertedId:<nil>}, collection records:1
info, _ = coll.Upsert(bson.M{"nonsense": -1}, foo) // will insert duplicate record (due to the selector that will never match anything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count) // &{Updated:0 Removed:0 Matched:0 UpsertedId:ObjectIdHex("5c35ea2a1f5b80c932b44b1d")}, collection records:2
foo.Bar = "baz"
info, _ = coll.Upsert(nil, foo) // will update the first matched (since the selector matches everything)
count, _ = coll.Count()
fmt.Printf("%+v, collecton records:%v\n", info, count)
// after the test the collection will have the following records
//> use foo
//> db.bar.find()
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b81"), "bar" : "baz" } // the first matched on the last update with nil selector
//{ "_id" : ObjectId("5c35ebf41f5b80c932b44b86"), "bar" : "barrr" } // the duplicated record with the selector that never matches anything
}
EDIT: Note that you should have an index on the never matching field, or your insert query will take too long if you have many records, because the upsert on not indexed filed will scan all the documents in the collection.

to find the last index of the array in mongodb

In the back end i m using go lang and for database i use mongoDB. I m trying to find the last document inserted in the embedded array so i can retrieve the document in the last array index without knowing its index.Right now i m getting all the documents of the employee and then find the last index.It is like overloading my RAM as i need to retrieve 1000 of record of employee and store it in ram before finding the last index of the array
My struct is as follows
type (
Employee struct {
Name string
Password string
EmpId string
EmailAddress string
Position string
Gender string
Nationality string
Department string
MaritalStatus string
Approvedby string
JoinDate time.Time
ConfirmationDate time.Time
EndDate time.Time
Leave []*LeaveInfo
}
LeaveInfo struct {
Total float64
Id int
Days float64
From time.Time
To time.Time
Status string
Certificate []*CertificateInfo
}
CertificateInfo struct {
FileName string
FileType string
FileSize int
}
Here is how i do in my application
My code is follows
employee := Employee{}
err = c.Find(bson.M{
"empid": "1234"
}).One(&result)
if err != nil {
log.Fatal(err)
}
name:=result.Name
lastindex:= result.LeaveInfo[len(result.LeaveInfo)-1].Id
As you can see i retrive the whole employee data and then find the Id of the last document.Is there any better way to do this.Appreciate any help.Please ...Thanks..
Newly Added Codes
This code is based on your example
var result bson.M
query := bson.M{"empid": "1234"} // gets the employee you are interested in
match := bson.M{"$match": query} // Set up the match part of the pipeline
unwind := bson.M{"$unwind": "$leave"} // sets up the leave field to be unwound
pipeline := []bson.M{match, unwind,{
"$project":bson.M{
"ID":bson.M{
"$slice": []interface{}{"$leave.id", -1},
}
}
iter := postCollection.Pipe(pipeline).Iter()
for iter.Next(&result) {
fmt.Printf("%+v\n", result)
}
iter.Close()
This code gives me lot of same document like 542 documents .But all these document are same...
If you are running a version on Mongo which has $slice in it, It was introduced in 2.4, you can use it in a Find with the addition of a Select Function, which works like project.
err = c.Find(bson.M{"empid": "1234"}).Select(bson.M{"name": 1, "leave": bson.M{"$slice": -1}}).One(&result) // result is an Employee struct
Otherwise, aggregation is your friend. You need to use a pipeline and unwind the Employee Leave field.
There are a few simple steps:
1) Define a result record based on your Employee record where the Leave field is defined as a single LeaveInfo rather than a slice of LeaveInfos, eg
EmployeeResult struct {
Name string `bson:"name"`
Leave LeaveInfo `bson:"leave"`
}
Do not forget to make the bson tag the same name as the LeaveInfo tag in the Employee struct.
2) Then create a pipeline with a couple of stages:
query := bson.M{"empid": "1234"} // gets the employee you are interested in
match := bson.M{"$match": query} // Set up the match part of the pipeline
unwind := bson.M{"$unwind": "$leave"} // sets up the leave field to be unwound
pipeline := []bson.M{match, unwind} // the pipeline you are passing to pipe. I like to split the parts of the pipe call up for clarity and ease of later modification
3) Call Pipe with the pipeline as a parameter then Iter over the results, this should give you one LeaveInfo at a time
var (
result EmployeeResult
)
iter := postCollection.Pipe(pipeline).Iter()
for iter.Next(&result) {
fmt.Printf("%+v\n", result)
}
iter.Close()
4) At the end of the loop, result will have the last item in the list, or be blank if nothing was read.