I try to get max values from MongoDB collection from my Go code.
What type should I use to decode result?
When I use bson.D{} as val2 type the result looks like [{_id <nil>} {max 66} {cnt 14}].
Here's the code:
filter := []bson.M{{
"$group": bson.M{
"_id": nil,
"max": bson.M{"$max": "$hellid"},
}},
}
cursor, err := collection.Aggregate(ctx, filter)
for cursor.Next(ctx) {
val2 := ???
err := cursor.Decode(&val2)
fmt.Printf("cursor: %v, value: %v\n", cursor.Current, val2)
}
}
Using bson.D already works as you presented. The problem may be you can't "easily" get out the max and cnt values.
Model your result document with a struct like this:
type result struct {
Max int `bson:"max"`
Count int `bson:"cnt"
}
Although cnt is not produced by the example code you provided.
And then:
var res result
err := cursor.Decode(&res)
Related
showInfoCursor, err := collection.Aggregate(context.TODO(), mongo.Pipeline{unwindStage, groupStage})
if err != nil {
panic(err)
}
var showsWithInfo []bson.M
if err = showInfoCursor.All(context.TODO(), &showsWithInfo); err != nil {
panic(err)
}
I'm iterating the showsWithInfo array. And each bson.M contains a primitive M type value for a particular key. I've tried to convert it to a struct, but it was no successful.
map[operatorId:1 channel: XYZ]
This is what I got once I print the value of that primitive M.
I need to get those two values from that. (operatorId, channel)
bson.M is a type alias to primitive.M:
type M = primitive.M
And primitive.M is a "simple" map:
type M map[string]interface{}
So you may index the value as you would any maps:
m := primitive.M{
"operatorId": 1,
"channel": "XYZ",
}
fmt.Println(m)
fmt.Println("Operator ID:", m["operatorId"])
fmt.Println("Channel:", m["channel"])
This outputs (try it on the Go Playground):
map[channel:XYZ operatorId:1]
Operator ID: 1
Channel: XYZ
I'm working with golang and the MongoDB driver, I want to patch one of my objects according to the data I get from the outside:
I have a struct:
type Pivot struct {
Email string `json:"email"`
Base string `json:"base"`
}
And the patch (with MongoDB Update)
setMap := bson.D{
{"$set", setElements},
}
res, err := collection.UpdateMany(
ctx,
filter,
setMap,
)
And I want to make the setObject a little bit dynamic:
if len(pivot.Base) > 0 {
setElements.append("base", pivot.Base) //this doesn't work...
}
if len(pivot.Email) > 0 {
setElements.append("email", pivot.Email)
}
I' ve seen that the setObject can be built like
{"$set", bson.D{
{"processed", pivot.Processed},
}
But how can I make it dynamic?
Append a DocElem (mgo) or an E (go.mongodb.org) to the slice depending on the client you are using.
var setElements bson.D
if len(pivot.Base) > 0 {
setElements = append(setElements, bson.E{"base", pivot.Base})
}
if len(pivot.Email) > 0 {
setElements = append(setElements, bson.E{"email", pivot.Email})
}
setMap := bson.D{
{"$set", setElements},
}
Replace bson.E with bson.DocElem for mgo.
I am new to mongodb-go-driver and i am stuck.
I have a date inside a struct like:
type Email struct {
Date string `json:"date"`
}
the Dates on my mongoDB and mapped in my struct have the values like "02/10/2018 11:55:20".
I want to find on my DB the elements that Date are after an other date, i'm trying this but the response is always null.
initDate, _ := time.Parse("02012006", initialDate)
cursor, err := emails.Find(context.Background(), bson.NewDocument(bson.EC.SubDocumentFromElements("date", bson.EC.DateTime("$gt", initDate.Unix()))))
what am i doing wrong?
the Dates on my mongoDB and mapped in my struct have the values like "02/10/2018 11:55:20".
There are a number of ways you could do. The first, as mentioned on the comment, is to convert the string date into an actual date format. See also MongoDB Date. Storing date values in the proper date format is the recommended way for performance.
If you have a document:
{ "a": 1, "b": ISODate("2018-10-02T11:55:20Z") }
Using mongo-go-driver (current v1.2.x) you can do as below to find and compare using date:
initDate, err := time.Parse("02/01/2006 15:04:05", "01/10/2018 11:55:20")
filter := bson.D{
{"b", bson.D{
{"$gt", initDate},
}},
}
cursor, err := collection.Find(context.Background(), filter)
Please note the layout value in the example above for time.Parse(). It needs to match the string layout/format.
An alternative way, without converting the value is to use MongoDB Aggregation Pipeline. You can use $dateFromString operator to convert the string date into date then use $match stage to filter by date.
For example, given documents:
{ "a": 1, "b": "02/10/2018 11:55:20" }
{ "a": 2, "b": "04/10/2018 10:37:19" }
You can try:
// Add a new field called 'newdate' to store the converted date value
addFieldsStage := bson.D{
{"$addFields", bson.D{
{"newdate", bson.D{
{"$dateFromString", bson.D{
{"dateString", "$b"},
{"format", "%d/%m/%Y %H:%M:%S"},
}},
}},
}},
}
initDate, err := time.Parse("02/01/2006 15:04:05", "02/10/2018 11:55:20")
// Filter the newly added field with the date
matchStage := bson.D{
{"$match", bson.D{
{"newdate", bson.D{
{"$gt", initDate},
}},
}},
}
pipeline := mongo.Pipeline{addFieldsStage, matchStage}
cursor, err := collection.Aggregate(context.Background(), pipeline)
The unstable bsonx package in mongodb-go-driver has a DateTime Type.
You can add the field in your struct like this:
type Email struct {
Date bsonx.Val
}
To declare the struct use bsonx.DateTime(millis int64):
Email{
Date: bsonx.DateTime(time.Now().UnixNano()/1e6)
}
*time.Now().UnixNano()/1e6 basically gets the unix millis.
And you can convert it to time.Time with email.Date.Time()
I read in the following article that it is more efficient to use the natural ordering of _id to perform pagination because skip always starts from the beginning of the collection.
Fast and Efficient Pagination in MongoDB
// Page 1
db.students.find().limit(10)
// Page 2
last_id = ... # logic to get last_id
db.students.find({'_id': {'$gt': last_id}}).limit(10)
But I have no idea how to perform the above using the mongodb/mongo-go-driver.
you can create a new func, dont forget to pass http.writer to read parameter.
func Pagination(r *http.Request, FindOptions *options.FindOptions) (int64, int64) {
if r.URL.Query().Get("page") != "" && r.URL.Query().Get("limit") != "" {
page, _ := strconv.ParseInt(r.URL.Query().Get("page"), 10, 32)
limit, _ := strconv.ParseInt(r.URL.Query().Get("limit"), 10, 32)
if page == 1 {
FindOptions.SetSkip(0)
FindOptions.SetLimit(limit)
return page, limit
}
FindOptions.SetSkip((page - 1) * limit)
FindOptions.SetLimit(limit)
return page, limit
}
FindOptions.SetSkip(0)
FindOptions.SetLimit(0)
return 0, 0
}
just call
Pagination(r, options)
example
options := options.Find()
page, limit := parameter.Pagination(r, options)
// page, limit as response for header payload
The cursor.skip() method requires the server to scan from the beginning of the input results set before beginning to return results. As the offset increases, cursor.skip() will become slower. While range queries can use indexes to avoid scanning unwanted documents, typically yielding better performance as the offset grows compared to using cursor.skip() for pagination. See more information on MongoDB: Pagination Example
Using the current version of mongo-go-driver (v0.0.15).An example to perform pagination showing latest entry first:
func Paginate(collection *mongo.Collection, startValue objectid.ObjectID, nPerPage int64) ([]bson.Document, *bson.Value, error) {
// Query range filter using the default indexed _id field.
filter := bson.VC.DocumentFromElements(
bson.EC.SubDocumentFromElements(
"_id",
bson.EC.ObjectID("$gt", startValue),
),
)
var opts []findopt.Find
opts = append(opts, findopt.Sort(bson.NewDocument(bson.EC.Int32("_id", -1))))
opts = append(opts, findopt.Limit(nPerPage))
cursor, _ := collection.Find(context.Background(), filter, opts...)
var lastValue *bson.Value
var results []bson.Document
for cursor.Next(context.Background()) {
elem := bson.NewDocument()
err := cursor.Decode(elem)
if err != nil {
return results, lastValue, err
}
results = append(results, *elem)
lastValue = elem.Lookup("_id")
}
return results, lastValue, nil
}
An example to call the pagination function above:
database := client.Database("databaseName")
collection := database.Collection("collectionName")
startObjectID, _ := objectid.FromHex("5bbafea2b5e14ee3a298fa4a")
// Paginate only the latest 20 documents
elements, lastID, err := Paginate(collection, startObjectID, 20)
for _, e := range elements {
fmt.Println(&e)
}
// Last seen ObjectID can be used to call next Paginate()
fmt.Println("Last seen ObjectID: ", lastID.ObjectID())
Note that you can also substitute the _id field with another indexed field.
Set page=0 and limit=0 if no pagination is required.
func GetUsers (page, limit int) {
filter := bson.D{{}} // selects all documents
options := new(options.FindOptions)
if limit != 0 {
if page == 0 {
page = 1
}
options.SetSkip(int64((page - 1) * limit))
options.SetLimit(int64(limit))
}
cursor, err := mongoCollection.Find(context.TODO(), filter, options)
...
}
I have a document which has this following structure
{ "_id" : "736722976", "value" : { "total_visit" : 4, "FIFA World Cup 2014" : 1, "Germany" : 1, "Algeria" : 1, "Thomas Muller" : 1, "Mesut Ozil" : 1, "Monsoon" : 1, "India Meteorological Department (IMD)" : 1, "Web Exclusive" : 2, "Specials" : 1, "Tapas Pal" : 1, "Twitter Trends" : 1, "Sunanda Pushkar" : 1, "Shashi Tharoor" : 1, "AIIMS" : 1, "special" : 1 } }
THE MOST IMPORTANT thing is that the sub document structure under the key "value" is variable so I can not create a structure for that. I tried to follow the suggestion here - Unstructured MongoDB collections with mgo
And I came with this code ---
package main
import ("fmt"
"labix.org/v2/mgo" //importing mgo
"labix.org/v2/mgo/bson"
_ "reflect"
)
type AnalysisStruct struct{
Id string `bson:"_id,omitempty"`
Value bson.M `bson:",inline"`
}
func main() {
var m AnalysisStruct
//connecting to localhost mongodb
session, err := mgo.Dial("localhost")
if err != nil {
panic(err)
}
defer session.Close()
c := session.DB("my_analysis_db").C("analysis_mid_2")
iter := c.Find(nil).Iter()
for{
if iter.Next(&m){
fmt.Println(m.Value["value"]["total_visit"])
}else{
break
}
}
}
When I try to build this using go build -v -o analyzer it shows me this error---
./analyzer.go:32: invalid operation: m.Value["value"]["total_visit"] (index of type interface {})
I am terribly stuck with this. Can not get anything going. Please can somebody help?
Thanks
I cam up with this code after doing some research. Not the most optimized one for sure. But for my case it works. Took help from
http://blog.denevell.org/golang-interface-type-assertions-switch.html
https://groups.google.com/forum/#!topic/mgo-users/JYE-CP15az4
package main
import ("fmt"
"labix.org/v2/mgo" //importing mgo
"labix.org/v2/mgo/bson"
_ "reflect"
)
type AnalysisStruct struct{
Id string `bson:"_id,omitempty"`
Value bson.M `bson:",inline"`
}
func main() {
var m AnalysisStruct
//connecting to localhost mongodb
session, err := mgo.Dial("localhost")
if err != nil {
panic(err)
}
defer session.Close()
c := session.DB("consumergenepool_db").C("analysis_mid_2")
iter := c.Find(nil).Iter()
for{
if iter.Next(&m){
s := m.Value["value"].(bson.M)
data, _ := bson.Marshal(s)
var m bson.M
_ = bson.Unmarshal(data, &m)
fmt.Println(m)
for k, v := range m{
fmt.Print(k)
fmt.Print(" :: ")
fmt.Println(v)
}
}else{
break
}
}
}
Let me know your thoughts on this.
Thanks
When testing something new always use a lot of fmt.Printf's to get a feel for it, that being said.
Value bson.M `bson:",inline"`
Should be
Value bson.M `bson:"value,omitempty"`
And
fmt.Println(m.Value["value"]["total_visit"])
Should be:
fmt.Printf("%#v\n", m)
fmt.Println(m.Value["total_visit"])
Your m.Value is "value", so you can use m.Value["total_visit"] directly.
playground
//edit
You can only use inline to catch any fields that that aren't a part of the original struct, but since your struct only has 2 fields (Id and Value), you don't need it.
Now if you were to keep ,inline you would use it like this:
if v, ok := m.Value["value"].(bson.M); ok {
fmt.Println(v["total_visit"])
}
Because m.Value["value"] is an interface{}, you have to type assert it back to it's original value, bson.M before you could use it.