I am using Go and MongoDB for my Project.I used
db.collection.aggregate([{$match:{}},{$lookup:{}},{$addFields:{}}])
it is working fine in MongoDB, but when I use pipeline in Go, it gives below error
The 'cursor' option is required, except for aggregate with the explain argument
Go code is
matchStage:=bson.M{"$match":bson.M{}}
pipeline := collection.Pipe([]bson.M{matchStage})
err = pipeline.All(&resp)
This is not how you are supposed to implement Mongo-Aggregation query in Go.
It should be of the format
cursor, err := collection.Aggregate(
ctx,
mongo.Pipeline{<PIPELINE-STAGES>},
options.Aggregate().SetAllowDiskUse(true),
)
Hence your code should be:
ctx, _ = context.WithTimeout(context.Background(), 2*time.Second)
matchStage := bson.D{
{"$match", bson.D{}},
}
lookupStage := bson.D{
{"from", ""},
{"let": bson.D{{}}},
{"pipeline": bson.A{}},
{"as": ""},
}
addFieldsStage := bson.D{
{"$addFields", bson.D{}},
}
cursor, err := collection.Aggregate(
ctx,
mongo.Pipeline{matchStage, lookupStage, addFieldsStage},
options.Aggregate().SetAllowDiskUse(true), // Mongo-Aggregate options if any
)
if err != nil {
panic(err)
}
for cursor.Next(ctx) {
var cursorResult bson.M
err := cursor.Decode(&cursorResult) // I world recommend to decode it using a struct instead
if err != nil {
panic(err)
}
fmt.Printf("Decoded Cursor: %v", cursorResult)
}
err = cursor.Close(ctx)
if err != nil {
panic(err)
}
Note: I haven't tested the code in my local. So let me know in case of errors.
Related
I'm trying to make a function that watches the database for a certain document with a certain id to update but it does not work. It just stays alive while updating the document while the function should return. I've tried multiple things and the rest of the code works fine. When i remove the id part and listen for all document updates in that collection the function does as it should
func iterateChangeStream(routineCtx context.Context,stream *mongo.ChangeStream, chn chan string) {
defer stream.Close(routineCtx)
for stream.Next(routineCtx) {
var data bson.M
if err := stream.Decode(&data); err != nil {
fmt.Println(err)
}
chn <- "updated"
err := stream.Close(routineCtx)
if err != nil {
return
}
return
}
return
}
func (s Storage) ListenForScannerUpdateById(id primitive.ObjectID) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
chn := make(chan string)
coll := s.db.Collection("scanners")
scan, err := s.GetScannerById(id)
fmt.Println(scan)
matchPipeline := bson.D{
{
"$match", bson.D{
{"operationType", "update"},
{"fullDocument._id", bson.D{
{"$eq", id},
}},
},
},
}
scannerStream, err := coll.Watch(ctx, mongo.Pipeline{matchPipeline})
if err != nil {
err := scannerStream.Close(ctx)
if err != nil {
panic( err)
}
fmt.Printf("err: %v", err)
}
routineCtx, _ := context.WithCancel(context.Background())
go iterateChangeStream(routineCtx, scannerStream, chn)
msg, _ := <- chn
defer close(chn)
fmt.Println(msg)
return
}
Ok, so after reading the documentation for a seccond time i found this:
For update operations, this field only appears if you configured the change stream with fullDocument set to updateLookup. This field then represents the most current majority-committed version of the document modified by the update operation. This document may differ from the changes described in updateDescription if other majority-committed operations modified the document between the original update operation and the full document lookup.
so after setting the fullDocument option to updateLookup like this it works perfect:
scannerStream, err := coll.Watch(ctx, mongo.Pipeline{matchPipeline}, options.ChangeStream().SetFullDocument(options.UpdateLookup))
I have a problem finding an objectid through the query param called id.
I can see the id that arrives at the function until the moment of doing the query.
But when I try to use ObjectIDFromHex it returns 00000000000000000000000 and doesn't get the document from mongodb.
I'll leave a screenshot so you can see the full problem.
screenshot with IDE
The code is this.
func RetrieveUser(ID string) (models.User, error) {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
db := MongoCN.Database("mydatabase")
col := db.Collection("users")
var user models.User
objID, _ := primitive.ObjectIDFromHex(ID)
fmt.Println(ID)
fmt.Println(objID)
condition := bson.M{
"_id": objID,
}
err := col.FindOne(ctx, condition).Decode(&user)
user.Password = ""
if err != nil {
fmt.Println("User not found" + err.Error())
return user, err
}
return user, nil
}
This is my database collection:
With this go code, I try to get all the users who are either involved in the story or created the story with the given id.
func main() {
for stf.DB == nil {
}
collection := stf.DB.Collection("user")
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
id, _ := primitive.ObjectIDFromHex("5cb4dd7e29d3dca573a73d4c")
fter := bson.M{"_id": id}
involvedFilter := bson.M{"stories_involved": fter}
createdFilter := bson.M{"stories_created": fter}
filter := bson.M{"$or": []bson.M{involvedFilter, createdFilter}}
cur, err := collection.Find(ctx, filter)
if err != nil {
log.Fatal(err.Error())
}
defer cur.Close(ctx)
for cur.Next(ctx) {
var result bson.M
err := cur.Decode(&result)
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(result)
}
if err := cur.Err(); err != nil {
log.Fatal(err.Error())
}
}
The code doesn't output any errors, but it also doesn't output any objects...
Thanks for your help in advance!
Your query translates into:
{"$or":[
{"stories_involved":{
"_id": ObjectId("5cb4dd7e29d3dca573a73d4c")}},
{"stories_created":{
"_id":ObjectId("5cb4dd7e29d3dca573a73d4c")}}
]}
Which means that it's searching for either a document with a nested document
i.e:
{stories_involved: {_id: <value>}} OR {stories_created: {_id: <value>}}.
However, the documents in the collection contains nested document array i.e:
{stories_involved: [{_id:<value>}]} OR {stories_created: [{_id:<value>}]}
This is the reason your query is not returning any value (and no error because the query syntax is correct).
There are two ways of Querying a document nested in an array using dot notation. If you know the index of the array for the document, you can just specify the position:
id, _ := primitive.ObjectIDFromHex("5cb4dd7e29d3dca573a73d4c")
involvedFilter := bson.M{"stories_involved.0._id": id}
createdFilter := bson.M{"stories_created.0._id": id}
filter := bson.M{"$or": []bson.M{involvedFilter, createdFilter}}
cur, err := collection.Find(ctx, filter)
If you do not know the index position of the document nested in the array, concatenate the name of the array field, with a dot (.) and the name of the field in the nested document:
id, _ := primitive.ObjectIDFromHex("5cb4dd7e29d3dca573a73d4c")
involvedFilter := bson.M{"stories_involved._id": id}
createdFilter := bson.M{"stories_created._id": id}
filter := bson.M{"$or": []bson.M{involvedFilter, createdFilter}}
cur, err := collection.Find(ctx, filter)
See also MongoDB: Query Documents
I checked out the answer here but this uses the old and unmaintained mgo. How can I find all documents in a collection using the mongo-go-driver?
I tried passing a nil filter, but this does not return any documents and instead returns nil. I also checked the documentation but did not see any mention of returning all documents. Here is what I've tried with aforementioned result.
client, err := mongo.Connect(context.TODO(), "mongodb://localhost:27017")
coll := client.Database("test").Collection("albums")
if err != nil { fmt.Println(err) }
// we can assume we're connected...right?
fmt.Println("connected to mongodb")
var results []*Album
findOptions := options.Find()
cursor, err := coll.Find(context.TODO(), nil, findOptions)
if err != nil {
fmt.Println(err) // prints 'document is nil'
}
Also, I'm about confused about why I need to specify findOptions when I've called the Find() function on the collection (or do I not need to specify?).
Here is what I came up with using the official MongoDB driver for golang. I am using godotenv (https://github.com/joho/godotenv) to pass the database parameters.
//Find multiple documents
func FindRecords() {
err := godotenv.Load()
if err != nil {
fmt.Println(err)
}
//Get database settings from env file
//dbUser := os.Getenv("db_username")
//dbPass := os.Getenv("db_pass")
dbName := os.Getenv("db_name")
docCollection := "retailMembers"
dbHost := os.Getenv("db_host")
dbPort := os.Getenv("db_port")
dbEngine := os.Getenv("db_type")
//set client options
clientOptions := options.Client().ApplyURI("mongodb://" + dbHost + ":" + dbPort)
//connect to MongoDB
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
log.Fatal(err)
}
//check the connection
err = client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to " + dbEngine)
db := client.Database(dbName).Collection(docCollection)
//find records
//pass these options to the Find method
findOptions := options.Find()
//Set the limit of the number of record to find
findOptions.SetLimit(5)
//Define an array in which you can store the decoded documents
var results []Member
//Passing the bson.D{{}} as the filter matches documents in the collection
cur, err := db.Find(context.TODO(), bson.D{{}}, findOptions)
if err !=nil {
log.Fatal(err)
}
//Finding multiple documents returns a cursor
//Iterate through the cursor allows us to decode documents one at a time
for cur.Next(context.TODO()) {
//Create a value into which the single document can be decoded
var elem Member
err := cur.Decode(&elem)
if err != nil {
log.Fatal(err)
}
results =append(results, elem)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
//Close the cursor once finished
cur.Close(context.TODO())
fmt.Printf("Found multiple documents: %+v\n", results)
}
Try passing an empty bson.D instead of nil:
cursor, err := coll.Find(context.TODO(), bson.D{})
Also, FindOptions is optional.
Disclaimer: I've never used the official driver, but there are a few examples at https://godoc.org/go.mongodb.org/mongo-driver/mongo
Seems like their tutorial is outdated :/
I need to build a query using comparison operators, equivalent of db.inventory.find( { qty: { $gt: 20 } using the official driver. Any idea how to do that?
Connecting to a server is something like:
client, err := mongo.NewClient("mongodb://foo:bar#localhost:27017")
if err != nil { log.Fatal(err) }
err = client.Connect(context.TODO())
if err != nil { log.Fatal(err) }
Then obtain the inventory mongo.Collection like:
coll := client.Database("baz").Collection("inventory")
Then you can execute your query using Collection.Find() like:
ctx := context.Background()
cursor, err := coll.Find(ctx,
bson.NewDocument(
bson.EC.SubDocumentFromElements("qty",
bson.EC.Int32("$gt", 20),
),
),
)
defer cursor.Close(ctx) // Make sure you close the cursor!
Reading the results using the mongo.Cursor:
doc := bson.NewDocument()
for cursor.Next(ctx) {
doc.Reset()
if err := cursor.Decode(doc); err != nil {
// Handle error
log.Printf("cursor.Decode failed: %v", err)
return
}
// Do something with doc:
log.Printf("Result: %v", doc)
}
if err := cursor.Err(); err != nil {
log.Printf("cursor.Err: %v", err)
}
Note: I used a single bson.Document value to read all documents, and used its Document.Reset() in the beginning of each iteration to clear it and "prepare" it to read a new document into it. If you want to store the documents (e.g. in a slice), then you can obviously not do this. In that case just create a new doc in each iteration like doc := bson.NewDocument().