Currently I learn to create restful api with golang and mongodb. Actually I am beginner in both. I use mongodb-go-driver and I learn to use filter when we want to use find() function. But I have some that I don't understand. What is the different between filter := bson.M{"_id": "abcd"} and filter := bson.M{{"_id": "abcd"}}? Thank you
Refer to the source code, https://github.com/mongodb/mongo-go-driver/blob/master/bson/primitive/primitive.go
bson.D, internally is primitive.D, which is []primitive.E, which is a struct. bson.M, internally is primitive.M, which is map[string]interface{}. You put in key/value in bson.M but use document (struct) in bson.D.
It is better to explain it using 2 parameters, e.g. search for a = 1 and b = 2. You syntax will be: bson.M{"a": 1, "b": 2} or bson.D{{"a": 1}, {"b": 2}}
//Filter Part
if filter != nil {
if len(filter.Status) > 0 {
query = append(query, bson.M{"status": bson.M{"$in": filter.Status}})
}
}
d.Shared.BsonToJSONPrint(query)
//Getting Total count
totalCount, err := col.Find(func() bson.M {
if query != nil {
if len(query) > 0 {
return bson.M{"$and": query}
}
}
return bson.M{}
}
var user []models.User
var findQuery []bson.M
coll := db.C(constants.USERTABLE)
if name != nil {
if len(name) > 0 {
query = append(query, bson.M{"name": bson.M{"$in": name}})
}
}
if status != nil {
if len(status) > 0 {
query = append(query, bson.M{"status": bson.M{"$in": status}})
}
}
findQuery = append(findQuery, bson.M{"$match": func() bson.M {
if query != nil {
if len(query) > 0 {
return bson.M{"$and": query}
}
}
return bson.M{}
}()})
shared.BsonToJSONPrint(query)
err = coll.Pipe(findQuery).All(&user)
if err != nil {
return nil, err
}
return user, nil
}
The client decodes the result to a bson.D document by default, this is represented as an array, which is not the document shape we want.
To overwrite this behaviour you have to decode to a bson.M, which is a simple map and has the same shape of the document in database
res := collection.FindOne(ctx, p.Where)
if res.Err() == mongo.ErrNoDocuments {
return nil, nil
}
if res.Err() != nil {
return nil, res.Err()
}
var document bson.M = make(bson.M) // important
err = res.Decode(document)
Related
This is my code. I always get maximum value of 999, even when there are more blocks (e.g 3000 blocks).
This is what the document(s) look like.
func GetLatestBlockFromMongoDB() int{
if contains(ReturnBlocksExists(), "blocks") == true {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
var blockheights []models.Block
defer cancel()
options := options.Find().SetProjection(bson.M{"block.header.blockNumber": 1})
options.SetSort(bson.D{{"block.header.blockNumber", -1}})
options.SetLimit(1)
results, err := blocksCollections.Find(context.TODO(), bson.D{}, options)
if err != nil {
fmt.Println(err)
}
//reading from the db in an optimal way
defer results.Close(ctx)
for results.Next(ctx) {
var singleBlock models.Block
if err = results.Decode(&singleBlock); err != nil {
fmt.Println(err)
}
blockheights = append(blockheights, singleBlock)
}
fmt.Println("%v", blockheights[0].Block.Header.BlockNumber)
if len(strings.TrimSpace(blockheights[0].Block.Header.BlockNumber)) > 0{
i, err := strconv.Atoi(blockheights[0].Block.Header.BlockNumber)
if err != nil {
glog.Fatalf("%v", err)
}
return i
} else {
return 0
}
} else {
return 0
}
}
How can i get the maximum value of blockNumber? I think the problem might be because blockNumber is a string but not an integer. I'm not sure. I did set sort and also limit 1 so it should normally work.
Yes, you are right, from the image, I can see that the blockNumber field is a string, so you are not comparing integers, you are comparing string, where "999" is greater than "3000":
For example:
package main
import (
"fmt"
)
func findMax(a []string) string {
m := a[0]
for _, value := range a {
if value > m {
m = value
}
}
return m
}
func main() {
blocks := []string{"999", "2000", "3000"}
ma := findMax(blocks)
fmt.Println(ma)
}
$ go run .
999
Check out this question too.
Currently I am able to return all my products from the collection.
I however want to be able to return products that come after a specific product ID (which would be the last one on the client side so they could load more)
Current way (return all)
query := bson.M{}
var product ReturnedProdcut
var products []ReturnedProduct
cur, err := mg.Db.Collection("products").Find(c.Request().Context(), query)
if err != nil {
fmt.Println(err)
}
for cur.Next(c.Request().Context()) {
err := cur.Decode(&product)
if err != nil {
fmt.Println(err)
}
products = append(products, product)
}
// return products list in JSON format
return c.JSON(http.StatusOK, products)
New Way Attempt(return based on page)
afterID := c.QueryParam("afterID")
if afterID == "" {
// get from start of collection based on latest date
}
// get 10 products after this ID, if no ID then get from start
query := bson.M{}
var product ReturnedProduct
var products []Returnedproduct
//.find(afterId).limit(10) - something like this?
cur, err := mg.Db.Collection("products").Find(c.Request().Context(), query)
if err != nil {
fmt.Println(err)
}
for cur.Next(c.Request().Context()) {
err := cur.Decode(&product)
if err != nil {
fmt.Println(err)
}
products = append(products, product)
}
// return products list in JSON format
return c.JSON(http.StatusOK, products)
The official MongoDB Go driver also has a *FindOptions optional parameter you could also explore.
pageOptions := options.Find()
pageOptions.SetSkip(int64(page)) //0-i
pageOptions.SetLimit(int64(limit)) // number of records to return
cur, err := userCollection.Find(c.Request().Context(), bson.D{{}}, pageOptions)
if err != nil {
// handle error
}
defer cur.Close(ctx)
var products []Returnedproduct
for cur.Next(c.Request().Context()) {
var product Returnedproduct
if err := cur.Decode(&product); err != nil {
// handle error
}
products = append(products, &product)
}
if err := cur.Err(); err != nil {
// handle error
}
You may construct a query where _id is greater than afterID, in which case you should also specify sorting by _id. For sorting and for setting a limit, you may use options.FindOptions.
You also should use Cursor.All() to decode all results and not one-by-one.
This is how it could look like:
query := bson.M{"_id": bson.M{"$gt": afterID}}
opts := options.Find().
SetSort(bson.M{"_id": 1}).
SetLimit(10)
ctx := c.Request().Context()
curs, err := mg.Db.Collection("products").Find(ctx, query, opts)
if err != nil {
// Handle error
}
var products []Returnedproduct
if err = curs.All(ctx, &products); err != nil {
// Handle error
}
return c.JSON(http.StatusOK, products)
I want to update one of the documents CLIENTDATA USING _id and clientID as the filter keys how can I update it and is their any method through aggregation. How can I update swapnil name to something else using id and clientID as my filters
//UpdateClient is used to update clientData
func UpdateClient(Data structure.ClientDataUpdate) bool {
connection := GetConnection()
if connection == nil {
return false
}
collection := connection.Database("IAGENT").Collection("CLIENTDATA")
filter := bson.M{"$and": []interface{}{bson.M{"_id": Data.ID}, bson.M{"clientData.clientID": Data.ClientID}}}
update := bson.M{"$set": bson.M{"clientData.name": Data.Name, "clientData.policy": Data.Policy, "clientData.expiryDate": Data.ExpiryDate,"clientData.metaData":Data.Metadata,"clientData.mobile":Data.Phone}}
_, err := collection.UpdateOne(context.TODO(), filter, update)
if err != nil {
fmt.Println("updating the Data", err)
return false
}
return true
}
Here is the image of my MongoDB database with the above collection.
You need to use positional operator to update element in an array, so instead of using clientData.name you should use clientData.$.name
//UpdateClient is used to update clientData
func UpdateClient(Data structure.ClientDataUpdate) bool {
connection := GetConnection()
if connection == nil {
return false
}
collection := connection.Database("IAGENT").Collection("CLIENTDATA")
filter := bson.M{"$and": []interface{}{bson.M{"_id": Data.ID}, bson.M{"clientData.clientID": Data.ClientID}}}
update := bson.M{"$set": bson.M{"clientData.$.name": Data.Name, "clientData.$.policy": Data.Policy, "clientData.$.expiryDate": Data.ExpiryDate,"clientData.$.metaData":Data.Metadata,"clientData.$.mobile":Data.Phone}}
_, err := collection.UpdateOne(context.TODO(), filter, update)
if err != nil {
fmt.Println("updating the Data", err)
return false
}
return true
}
While querying the below data, returned cursor is empty. while there is 100s of documents which satisfy the condition.
{
"_id": "5dd68c51a39809125944ffba",
"status": "success",
"balance": "0.000",
"request_params": {
"username": "test_user",
"service_code": "MR"
}
using below code
MongoDB driver "go.mongodb.org/mongo-driver/mongo"
func saveLog(data Log) bool {
mongo, err := openMongo()
if err != nil {
log.Println(err)
fmt.Println("Connection failed")
return false
} else {
LogCollection := mongo.Database(LogDb).Collection(CollectionLog)
insertedApi, err := LogCollection.InsertOne(context.TODO(), data)
if err != nil {
log.Println(err)
fmt.Println("Insert failed")
return false
} else {
log.Println(insertedApi.InsertedID)
return true
}
}
}
func parseLog() {
db, err := openMongo()
if err != nil {
fmt.Println(err)
fmt.Println("Connection failed")
return
} else {
logCollection := db.Database(LogDb).Collection(CollectionLog)
var results [] *Log
find := bson.D{{"status","success"},{"request_params",bson.D{{"username","test_user"}}}}
fmt.Println(find)
cur, err := logCollection.Find(context.TODO(), find)
if err != nil {
log.Fatal(err)
}else {
for cur.Next(context.TODO()) {
var elem Log
err := cur.Decode(&elem)
if err != nil {
fmt.Println("Parse error : ",err)
}
fmt.Println("Log : ",elem)
results = append(results, &elem)
}
}
}
}
Log write
saveLog(Log{"success","0.000",RequestParams{"test_user","MR"}})
Log read
parseLog()
Log struct
type Log struct {
Status string `bson:"status"`
Balance string `bson:"balance"`
RequestParams RequestParams `bson:"request_params"`
}
type RequestParams struct {
Username string `bson:"username"`
ServiceCode string `bson:"service_code"`
}
MongoDB data
status only is returning whole 8k documents
bson.D{{"status","success"}}
Isn't collection.Find() function the right one for it.
Shell command is returning documents correctly
db.log.find({"status":"success","request_params.username":"test_user"}).limit(10).pretty()
The issue here is because of the query filter. There is a difference between the following queries:
// Query A: {"status": "success", "request_params": {"username":"test_user"}}
find := bson.D{{"status","success"},{"request_params",bson.D{{"username","test_user"}}}}
// Query B: {"status": "success", "request_params.username":"test_user"}
find := bson.D{{"status","success"},{"request_params.username","test_user"}}
Query A means that you would like to match an exact document of request_params where the value object exactly equal to {"username":"test_user"}. None of the documents in your collection matches this criteria. The documents also contains {"service_code":"MR"}. While query B uses dot notation, which means that you would like to match request_params field where it contains a value of {"username":"test_user"}.
See also Query on Nested Field for more information.
I'm trying to do a query in MongoDB, just like the group by in SQL. So I use the Aggregate() API, to find all the unique _id(MarketType+Symbol).
And the Aggregate() runs successful, but when I decode the return values, I got nothing. Aggregate() return type is the same as Find(), and I decode in this way when Find() is useful. Why it can't work when I Aggregate().
type MarketSymbol struct {
MarketType string `json:"Market,omitempty" bson:"MarketType"`
Symbol string `json:"Symbol,omitempty" bson:"Symbol"`
}
func GetSymbol() {
pipeline := bson.M {
"$group" :
bson.M{"_id":bson.M{"MarketType":"$MarketType","Symbol":"$Symbol"}},
}
cur,err := mongoSt.Collection.Aggregate(context.TODO(), []bson.M{pipeline})
if err != nil {
fmt.Println("Aggregate Error:", err)
} else {
fmt.Println("Aggregate Success!") //this line works
}
var results []MarketSymbol
count := 1
for cur.Next(context.TODO()) {
var elem MarketSymbol
err := cur.Decode(&elem)
if err != nil {
fmt.Printf("[count]=%d Decode Error\n", count)
count++
continue
}
fmt.Printf("[count]=%d, [MarketType]:%s , [Symbol]:%s\n", count, elem.MarketType, elem.Symbol) //this line works,but value is nil
count++
results = append(results, elem)
}
}
I got it. I also tried to use _id to save the Decode value before, but failed. So I thought it doesn't work. But it's right, what actually caused the error is I gave a wrong type of MarketType, it should be int32 but I gave it a string. So...
Anyway, the problem solved, here is the code.
type ID struct {
Id MarketSymbol `json:"ID,omitempty" bson:"_id"`
}
type MarketSymbol struct {
MarketType int32 `json:"Market,omitempty" bson:"MarketType"`
Symbol string `json:"Symbol,omitempty" bson:"Symbol"`
}
func GetSymbol{
pipeline := bson.M {
"$group" : bson.M{"_id":bson.M{"MarketType":"$MarketType","Symbol":"$Symbol"}},
}
cur,err := mongoSt.Collection.Aggregate(context.TODO(), []bson.M{pipeline})
if err != nil {
fmt.Println("Aggregate Error:", err)
} else {
fmt.Println("Aggregate Success!")
}
var results []ID
count := 1
for cur.Next(context.TODO()) {
var elem ID
err := cur.Decode(&elem)
if err != nil {
fmt.Printf("[count]=%d Decode Error\n", count)
count++
continue
}
fmt.Printf("[count]=%d, [MarketType]:%d , [Symbol]:%s\n", count, elem.Id.MarketType, elem.Id.Symbol)
count++
results = append(results, elem)
}
}