Struggling to convert a MongoDB singleResult object into Go struct - mongodb

I tried to follow documentation here and here but had no luck.
I want to get a singleResult from FindOne on the Collection named moviesCollection and then use Decode or Unmarshal to put those values into a struct. The values in the struct JSONData are exactly the same as in each Document
I am using the official mongodb driver github.com/mongodb/mongo-go-driver
Here is an example of what I have tried:
mongoContext, cancelContext := context.WithTimeout(context.Background(), 10*time.Second)
defer cancelContext()
mongoClient, _ := mongo.Connect(mongoContext, options.Client().ApplyURI("mongodb://localhost:27017"))
moviesCollection := mongoClient.Database("Entertainment").Collection("Movies")
moviesCollection.InsertOne(mongoContext, bson.M{"_id": "Deadpool", "Path": "path/to/file"})
singleResult := moviesCollection.FindOne(mongoContext, bson.M{"_id": "Deadpool"})
if singleResult.Err() != nil {
log.Println("Find error: ", singleResult.Err())
}
JSONData := struct {
Path string `json:"Path"`
}{}
decodeError := singleResult.Decode(&JSONData)
if decodeError != nil {
log.Println("Decode error: ", decodeError)
}
fmt.Println("Path: ", JSONData.Path)
However no errors are produced and JSON.Path produces and empty string.
I have also tried using bson.D{{"_id", "Deadpool"}} instead of bson.M{"_id": "Deadpool"}
I can confirm that JSON.Path is not empty string as I have checked the database natively using MongoDB Compass. The entry contains the following:
{"_id":"Deadpool","Path":"path/to/file"}

Internally, MongoDB uses bson. Change your struct as below should work.
From
JSONData := struct {
Path string `json:"Path"`
}{}
to
JSONData := struct {
Path string `bson:"Path"`
}{}

Hey so as simagix mentioned you should be able to change your tag from JSON to bson:
`bson:"Path"`
Another option, incase you need to obtain a more generic result is to pass it a D object like so:
JSONData := &bson.D{}
decodeError := singleResult.Decode(JSONData)
You can then obtain all the information through a map using the JSON.Data.Map function.

If you are using mongo-go-driver >= v.0.1.0 then, taking a look to the go-doc it looks pretty straightforward:
filter := bson.D{{"hello", "world"}}
err := collection.FindOne(context.Background(), filter).Decode(&result)
if err != nil { return err }
// do something with result...
So, what you need is:
package main
import (
"context"
"github.com/mongodb/mongo-go-driver/bson"
"github.com/mongodb/mongo-go-driver/mongo
)
func main() {
ctx := context.Background()
client, err := mongo.NewClient("mongodb://localhost:27017")
if err != nil {
...
}
if err := client.Connect(ctx); err != nil {
...
}
defer client.Disconnect(ctx)
collection := client.Database("myDb").Collection("movies")
filter := bson.D{{"_id", "sometitle"}}
var result JSONData
err := collection.FindOne(ctx, filter).Decode(&result)
if err != nil {
...
}
// Do Something cool here
}
I see you are using Title to filter the doc. Pay attention.

Related

How to insert array of struct into MongoDB

I am trying to insert data that is stored in an array of struct into a MongoDB using the go.mongodb.org/mongo-driver library. My struct is
type Statement struct {
ProductID string `bson:"product_id" json:"product_id"`
ModelNum string `bson:"model_num" json:"model_num"`
Title string `bson:"title" json:"title"`
}
and my insertion code is
func insert(stmts []Statement) {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
quickstartDatabase := client.Database("quickstart")
testCollection := quickstartDatabase.Collection("test")
testCollection.InsertMany(ctx, stmts) // This is giving error
}
The compiler gives the error cannot use stmts (variable of type []Statement) as []interface{} value in argument to testCollection.InsertMany at the InsertMany command.
I have tried marshalling the struct before inserting using bson.Marshal but even that doesn't work. How do I insert this data into the DB?
insertMany accept []interface{}
i would make like this
newValue := make([]interface{}, len(statements))
for i := range statements {
newValue[i] = statements[i]
}
col.InsertMany(ctx, newValue)

Appending Data to an Interface Pointing to a Defined Struct using 'reflect'

I am trying to create a function where it gets all the documents from a Mongo Collection and queries them to declared structs. To achieve this I set the parameters for the function of type interface so it can work with two structs. Here is my code:
In package entities:
type Project struct {
Title string
Position string
....
}
type Projects struct {
Projects []Project
}
In the current package:
var docs entities.Projects
var doc entities.Project
//doc represents a document from Mongo Collection
//docs represents an array of documents, each element is a document
//collection has type *mongo.Collection and points to the desired collection on MongoDB.
createQuery(&doc, &docs, collection)
func createQuery(doc interface{}, docs interface{}, c *mongo.Collection) {
documents := reflect.ValueOf(docs).Elem()
document := reflect.ValueOf(doc)
cur, err := c.Find(context.Background(), bson.D{{}})
if err != nil {
log.Fatal(err)
}
for cur.Next(context.Background()) {
err = cur.Decode(document.Interface())
if err != nil {
log.Fatal(err)
}
//Error is thrown here
documents.Set(reflect.Append(documents, document))
fmt.Println(doc)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
if err != nil {
fmt.Printf("oh shit this is the error %s \n", err)
}
cur.Close(context.Background())
fmt.Printf("documents: %+v\n", documents.Interface())
fmt.Printf("document: %+v\n", document.CanSet())
}
---ERROR OUTPUT---
panic: reflect: call of reflect.Append on struct Value
I was able to set data to doc using the document variable, although when doing document.CanSet() is false (so it may not even work). Where the program breaks is when I try to append the document to the documents interface.
The code in the question passes the struct docs to a function that expects a slice. Pass the address of the slice field in docs instead of docs itself.
The createQuery function can determine the slice element type from the slice itself. There's no need to pass in an example value.
var docs entities.Projects
createQuery(&docs.Projects, collection)
for _, doc := range docs.Projects {
fmt.Println(doc.Title)
}
The call to cur.Decode requires a pointer to an uninitialized value. Use reflect.New to create that value.
func createQuery(docs interface{}, c *mongo.Collection) {
docsv := reflect.ValueOf(docs).Elem()
doct := docsv.Type().Elem()
cur, err := c.Find(context.Background(), bson.D{{}})
if err != nil {
log.Fatal(err)
}
for cur.Next(context.Background()) {
docpv := reflect.New(doct)
err = cur.Decode(docpv.Interface())
if err != nil {
log.Fatal(err)
}
docsv.Set(reflect.Append(docsv, docpv.Elem()))
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
cur.Close(context.Background())
}
As an aside, the entities.Projects struct type is not needed if that struct type will only ever have one field. Use []Project instead:
var docs []entities.Project
createQuery(&docs, collection)
for _, doc := range docs {
fmt.Println(doc.Title)
}

What is the bson syntax for $set in UpdateOne for the official mongo-go-driver

I am trying to get some familiarity with the official mongo-go-driver and the right syntax for UpdateOne.
My simplest full example follows:
(NOTE: in order to use this code you will need to substitute in your own user and server names as well as export the login password to the environment as MONGO_PW):
package main
import (
"context"
"fmt"
"os"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/primitive"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type DB struct {
User string
Server string
Database string
Collection string
Client *mongo.Client
Ctx context.Context
}
var db = DB{
User: <username>,
Server: <server_IP>,
Database: "test",
Collection: "movies",
Ctx: context.TODO(),
}
type Movie struct {
ID primitive.ObjectID `bson:"_id" json:"id"`
Name string `bson:"name" json:"name"`
Description string `bson:"description" json:"description"`
}
func main() {
if err := db.Connect(); err != nil {
fmt.Println("error: unable to connect")
os.Exit(1)
}
fmt.Println("connected")
// The code assumes the original entry for dunkirk is the following
// {"Name":"dunkirk", "Description":"a world war 2 movie"}
updatedMovie := Movie{
Name: "dunkirk",
Description: "movie about the british evacuation in WWII",
}
res, err := db.UpdateByName(updatedMovie)
if err != nil {
fmt.Println("error updating movie:", err)
os.Exit(1)
}
if res.MatchedCount < 1 {
fmt.Println("error: update did not match any documents")
os.Exit(1)
}
}
// UpdateByName changes the description for a movie identified by its name
func (db *DB) UpdateByName(movie Movie) (*mongo.UpdateResult, error) {
filter := bson.D{{"name", movie.Name}}
res, err := db.Client.Database(db.Database).Collection(db.Collection).UpdateOne(
db.Ctx,
filter,
movie,
)
if err != nil {
return nil, err
}
return res, nil
}
// Connect assumes that the database password is stored in the
// environment variable MONGO_PW
func (db *DB) Connect() error {
pw, ok := os.LookupEnv("MONGO_PW")
if !ok {
fmt.Println("error: unable to find MONGO_PW in the environment")
os.Exit(1)
}
mongoURI := fmt.Sprintf("mongodb+srv://%s:%s#%s", db.User, pw, db.Server)
// Set client options and verify connection
clientOptions := options.Client().ApplyURI(mongoURI)
client, err := mongo.Connect(db.Ctx, clientOptions)
if err != nil {
return err
}
err = client.Ping(db.Ctx, nil)
if err != nil {
return err
}
db.Client = client
return nil
}
The function signature for UpdateOne from the package docs is:
func (coll *Collection) UpdateOne(ctx context.Context, filter interface{},
update interface{}, opts ...*options.UpdateOptions) (*UpdateResult, error)
So I am clearly making some sort of mistake in creating the update interface{} argument to the function because I am presented with this error
error updating movie: update document must contain key beginning with '$'
The most popular answer here shows that I need to use a document sort of like this
{ $set: {"Name" : "The Matrix", "Decription" "Neo and Trinity kick butt" } }
but taken verbatim this will not compile in the mongo-go-driver.
I think I need some form of a bson document to comply with the Go syntax. What is the best and/or most efficient syntax to create this bson document for the update?
After playing around with this for a little while longer I was able to solve the problem after A LOT OF TRIAL AND ERROR using the mongodb bson package by changing the UpdateByName function in my code above as follows:
// UpdateByName changes the description for a movie identified by its name
func (db *DB) UpdateByName(movie Movie) (*mongo.UpdateResult, error) {
filter := bson.D{{"name", movie.Name}}
update := bson.D{{"$set",
bson.D{
{"description", movie.Description},
},
}}
res, err := db.Client.Database(db.Database).Collection(db.Collection).UpdateOne(
db.Ctx,
filter,
update,
)
if err != nil {
return nil, err
}
return res, nil
}
Note the use of bson.D{{$"set", .... It is unfortunate the way MongoDB has implemented the bson package this syntax still does not pass the go-vet. If anyone has a comment to fix the lint conflict below it would be appreciated.
go.mongodb.org/mongo-driver/bson/primitive.E composite literal uses unkeyed fields
In many cases you can replace construction
filter := bson.D{{"name", movie.Name}}
with
filter := bson.M{"name": movie.Name}
if arguments order dowsn't matter

Find all documents in a collection with mongo go driver

I checked out the answer here but this uses the old and unmaintained mgo. How can I find all documents in a collection using the mongo-go-driver?
I tried passing a nil filter, but this does not return any documents and instead returns nil. I also checked the documentation but did not see any mention of returning all documents. Here is what I've tried with aforementioned result.
client, err := mongo.Connect(context.TODO(), "mongodb://localhost:27017")
coll := client.Database("test").Collection("albums")
if err != nil { fmt.Println(err) }
// we can assume we're connected...right?
fmt.Println("connected to mongodb")
var results []*Album
findOptions := options.Find()
cursor, err := coll.Find(context.TODO(), nil, findOptions)
if err != nil {
fmt.Println(err) // prints 'document is nil'
}
Also, I'm about confused about why I need to specify findOptions when I've called the Find() function on the collection (or do I not need to specify?).
Here is what I came up with using the official MongoDB driver for golang. I am using godotenv (https://github.com/joho/godotenv) to pass the database parameters.
//Find multiple documents
func FindRecords() {
err := godotenv.Load()
if err != nil {
fmt.Println(err)
}
//Get database settings from env file
//dbUser := os.Getenv("db_username")
//dbPass := os.Getenv("db_pass")
dbName := os.Getenv("db_name")
docCollection := "retailMembers"
dbHost := os.Getenv("db_host")
dbPort := os.Getenv("db_port")
dbEngine := os.Getenv("db_type")
//set client options
clientOptions := options.Client().ApplyURI("mongodb://" + dbHost + ":" + dbPort)
//connect to MongoDB
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
log.Fatal(err)
}
//check the connection
err = client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to " + dbEngine)
db := client.Database(dbName).Collection(docCollection)
//find records
//pass these options to the Find method
findOptions := options.Find()
//Set the limit of the number of record to find
findOptions.SetLimit(5)
//Define an array in which you can store the decoded documents
var results []Member
//Passing the bson.D{{}} as the filter matches documents in the collection
cur, err := db.Find(context.TODO(), bson.D{{}}, findOptions)
if err !=nil {
log.Fatal(err)
}
//Finding multiple documents returns a cursor
//Iterate through the cursor allows us to decode documents one at a time
for cur.Next(context.TODO()) {
//Create a value into which the single document can be decoded
var elem Member
err := cur.Decode(&elem)
if err != nil {
log.Fatal(err)
}
results =append(results, elem)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
//Close the cursor once finished
cur.Close(context.TODO())
fmt.Printf("Found multiple documents: %+v\n", results)
}
Try passing an empty bson.D instead of nil:
cursor, err := coll.Find(context.TODO(), bson.D{})
Also, FindOptions is optional.
Disclaimer: I've never used the official driver, but there are a few examples at https://godoc.org/go.mongodb.org/mongo-driver/mongo
Seems like their tutorial is outdated :/

Querying mongodb from golang using the _id stored in an array

So here is my question. I have an array which are stored the _ids of mongodbs objects. Whats the right way to retrieve them all in one query using the mgo and bson package?
So if the array is like that: ids:=["543d171c5b2c12420dd016","543d171c5b2dd016"]
How we make the query ? I tried that but I know its wrong.
query := bson.M{"_id": bson.M{"$in": ids}}
c.Find(query).All()
Thanks in advance
If the documents are stored with string ids, then the code looks correct.
The ids look like hex encoded object ids. If the object identifiers are object ids, then you need to the convert the hex strings to object ids:
oids := make([]bson.ObjectId, len(ids))
for i := range ids {
oids[i] = bson.ObjectIdHex(ids[i])
}
query := bson.M{"_id": bson.M{"$in": oids}}
MongoDB syntax for go.mongodb.org/mongo-driver has been updated, this should work using the official driver.
oids := make([]primitive.ObjectID, len(ids))
for i := range ids {
objID, err := primitive.ObjectIDFromHex(ids[i])
if err == nil {
oids = append(oids, objID)
}
}
This is to convert back to a struct that can be used through out the app
type MongoUser struct {
ID *primitive.ObjectID `json:"id" bson:"_id"`
FirstName string `json:"first_name" bson:"firstName"`
LastName string `json:"last_name" bson:"lastName"`
Email string `json:"email" bson:"email"`
}
This is a helper method that takes your slice of ids and turns it into the object id type.
func formatObjectIdMultiple(hex []string) ([]primitive.ObjectID, error) {
var list []primitive.ObjectID
oids := make([]primitive.ObjectID, len(hex))
for _, i := range hex {
objectId, err := primitive.ObjectIDFromHex(i)
if err != nil {
return nil, err
}
oids = append(oids, objectId)
}
return list, nil
}
Here is my method for the db. Its important you use bson.M for some reason bson.D does not work with this. Also dont forget to close your cursor the defer function will close it at the end of your GetMultipleUser function.
func (mongo *Mongo) GetMultipleUser(ids []string) ([]*MongoUser, error) {
objectIDs, err := formatObjectIdMultiple(ids)
if err != nil {
return nil, err
}
query := bson.M{"_id": bson.M{"$in": objectIDs}}
coll := mongo.Con.Database("dbName").Collection("users")
cursor, err := coll.Find(context.Background(), query)
if err != nil {
return nil, err
}
defer func() {
cursor.Close(context.Background())
}()
var output []*MongoUser
for cursor.Next(context.Background()) {
var temp *MongoUser
cursor.Decode(&temp)
output = append(output, temp)
}
return output, nil
}