I’m writing a code for receiving a data from mongodb in golang.
My code is as below:
type DataContent struct {
Create time.Time `bson:"create"`
Desc string `bson:"desc"`
}
type Data struct {
Id bson.ObjectId `bson:"_id,omitempty"`
Desc string `bson:"desc"`
Content DataContent `bson:"content"`
}
func get() error {
result := []Data{}
coll := session.DB(“”).C(“aaa”)
project := bson.M{"$project": bson.M{"_id": 1, "desc": 1, "content": 1 }}
err := coll.Pipe([]bson.M{project}).All(&result)
if err != nil {
return err
}
data, err := json.Marshal(result)
fmt.Printf("#####\n%s\n#####\n", string(data))
return nil
}
Result of execution is as below:
#####
[{"Id":"58133f92cf4abf18c834750d", "Desc”:”reg1\n”,"Content":{"Create":"0001-01-01T00:00:00Z","Desc":""}},
{"Id":"58134bbbcf4abf18c8347513", "Desc”:”reg2\n”,”Content":{"Create":"0001-01-01T00:00:00Z","Desc”:””}}]
#####
Values for subfield of “Content” was not come.
I’ve run same process via terminal also. Result is as below:
> db.aaa.aggregate([{$project: { _id:1, desc:1, content:1}}])
[{"Id":"58133f92cf4abf18c834750d", "Desc”:”reg1\n”,"Content":{"Create":ISODate("2016-10-28T13:13:13.520Z"),"Desc”:”aaa”}},
{"Id":"58134bbbcf4abf18c8347513", "Desc”:”reg2\n”,”Content":{"Create":ISODate("2016-10-28T13:09:32.810Z"),"Desc”:””}}]
Does anyone let me know how to get subfield-value via pipe function?
In addition, “Content” has following structure.
Content : [
{
Create : ISODate(“…”),
Desc : “…”
},
{
Create : ISODate(“…”),
Desc : “…”
}
]
Your Content structure is not a single value but an array of values. This means it cannot be loaded into the field Data.Content which is a single value of type DataContent.
Change your Data.Content field to a slice type ([]DataContent) and it will work:
type Data struct {
Id bson.ObjectId `bson:"_id,omitempty"`
Desc string `bson:"desc"`
Content []DataContent `bson:"content"`
}
Related
Mongo driver used: https://pkg.go.dev/go.mongodb.org/mongo-driver
I have some data saved in the mongodb like below:
{
"title" : "elem_1_3_title",
"list" : "elem_1_3_list"
}
When I receive this data using mongodb driver then it sorts the map in the alphabetical order:
cursor, err := collection.Aggregate(context.TODO(), pipeline)
if err != nil {
// handle err
}
pages := make([]map[string]interface{})
err = cursor.All(context.TODO(), &pages)
if err != nil {
// handle err
}
output:
{
"list" : "elem_1_3_list",
"title" : "elem_1_3_title"
}
updated:
type PageResp struct {
Id int `json:"_id,omitempty" bson:"_id,omitempty"`
Status int `json:"status" bson:"status"`
AddedSections []string `json:"added_sections" bson:"added_sections"`
Sections *orderedmap.OrderedMap `json:"sections,omitempty" bson:"sections,omitempty"`
}
The data from database is received in this struct & sections field is the map which needs to be ordered.
NOTE: I can not define structs for this as I have a very long list of fields & some new fields can be added in the future.
Is there any possible way to receive the same order which is saved under mongodb ?
You might need an ordered map in order to preserve order.
See for instance elliotchance/orderedmap described in "An Ordered Map in Go".
Or wk8/go-ordered-map with Go 1.18 and parameter types.
In both instances, that would replace your make([]map[string]interface{})
om := orderedmap.New[string, string]()
Getting this error while binding mongodb document to GO Struct using docstore collection iterator.
Database: Azure CosmosDB mongoDb API
Go Driver: Docstore
Code details:
Cosmos DB Constructors Code
client, err := c.newClient(ctx, true)
if err != nil {
log.Error("error connecting to mongodb cluster", zap.Error(err))
return nil, err
}
database := client.Database(c.dbName)
collection := database.Collection(c.collName)
return mongodocstore.OpenCollection(collection, "", nil)
Go struct for mapping with Mongo Db document
type utterance struct {
ID primitive.ObjectID `docstore:"_id,omitempty"`
User string `docstore:"user,omitempty"`
Locale string `docstore:"Locale,omitempty"`
Text string `docstore:"Text,omitempty"`
Source string `docstore:"Source,omitempty"`
Timestamp time.Time `docstore:"Timestamp,omitempty"`
DocstoreRevision interface{}
}
MondoDb Document
{
"_id" : ObjectId("60d5e18539864e948a8851a6"),
"User" : "auth0|6049b5ef5d79540071db6a0a",
"Locale" : "en_US",
"Text" : "Hi",
"Source" : "UTTERANCE_SOURCE_USER",
"Timestamp" : {
"$date" : 1624629637002
},
"DocstoreRevision" : "bf3b35d8-54ed-4a23-a08f-7d41b5c34085"
}
Method to call the docstore collection and iterate
i := s.collection.Query().Get(ctx)
defer i.Stop()
var results []*services.Utterance
for {
fmt.Println("for every document: ")
//var native utterance
doc := &utterance{}
err := i.Next(ctx, doc) //error at this line
if err == io.EOF {
break
} else if err != nil {
fmt.Println(“getting to this err block“)
return nil, err
}
u := doc.ToProto()
results = append(results, u)
}
Try setting the ID to a pointer type:
type utterance struct {
ID *primitive.ObjectID `docstore:"_id,omitempty"`
User string `docstore:"user,omitempty"`
Locale string `docstore:"Locale,omitempty"`
Text string `docstore:"Text,omitempty"`
Source string `docstore:"Source,omitempty"`
Timestamp time.Time `docstore:"Timestamp,omitempty"`
DocstoreRevision interface{}
}
Probably when you create the struct to be decoded the primitive.ObjectID is being initialized and the driver don't know to decode it.
And be careful. The docstore is a generic Document driver. It is not specific for MongoDB. So it probably don't work well with MongoDB specific types.
Reading the driver for mongodb https://pkg.go.dev/gocloud.dev/docstore/mongodocstore
There is a way to set what is stored in the _id field. I understand docstore don't know how to decode ObjectID from MongoDB.
I try to read and write and delete data from a Go application with the official mongodb driver for go (go.mongodb.org/mongo-driver).
Here is my struct I want to use:
Contact struct {
ID xid.ID `json:"contact_id" bson:"contact_id"`
SurName string `json:"surname" bson:"surname"`
PreName string `json:"prename" bson:"prename"`
}
// xid is https://github.com/rs/xid
I omit code to add to the collection as this is working find.
I can get a list of contacts with a specific contact_id using the following code (abbreviated):
filter := bson.D{}
cursor, err := contactCollection.Find(nil, filter)
for cur.Next(context.TODO()) {
...
}
This works and returns the documents. I thought about doing the same for delete or a matched get:
// delete - abbreviated
filter := bson.M{"contact_id": id}
result, _ := contactCollection.DeleteMany(nil, filter)
// result.DeletedCount is always 0, err is nil
if err != nil {
sendError(c, err) // helper function
return
}
c.JSON(200, gin.H{
"ok": true,
"message": fmt.Sprintf("deleted %d patients", result.DeletedCount),
}) // will be called, it is part of a webservice done with gin
// get complete
func Get(c *gin.Context) {
defer c.Done()
id := c.Param("id")
filter := bson.M{"contact_id": id}
cur, err := contactCollection.Find(nil, filter)
if err != nil {
sendError(c, err) // helper function
return
} // no error
contacts := make([]types.Contact, 0)
for cur.Next(context.TODO()) { // nothing returned
// create a value into which the single document can be decoded
var elem types.Contact
err := cur.Decode(&elem)
if err != nil {
sendError(c, err) // helper function
return
}
contacts = append(contacts, elem)
}
c.JSON(200, contacts)
}
Why does the same filter does not work on delete?
Edit: Insert code looks like this:
_, _ = contactCollection.InsertOne(context.TODO(), Contact{
ID: "abcdefg",
SurName: "Demo",
PreName: "on stackoverflow",
})
Contact.ID is of type xid.ID, which is a byte array:
type ID [rawLen]byte
So the insert code you provided where you use a string literal to specify the value for the ID field would be a compile-time error:
_, _ = contactCollection.InsertOne(context.TODO(), Contact{
ID: "abcdefg",
SurName: "Demo",
PreName: "on stackoverflow",
})
Later in your comments you clarified that the above insert code was just an example, and not how you actually do it. In your real code you unmarshal the contact (or its ID field) from a request.
xid.ID has its own unmarshaling logic, which might interpret the input data differently, and might result in an ID representing a different string value than your input. ID.UnmarshalJSON() defines how the string ID will be converted to xid.ID:
func (id *ID) UnmarshalJSON(b []byte) error {
s := string(b)
if s == "null" {
*id = nilID
return nil
}
return id.UnmarshalText(b[1 : len(b)-1])
}
As you can see, the first byte is cut off, and ID.UnmarshalText() does even more "magic" on it (check the source if you're interested).
All-in-all, to avoid such "transformations" happen in the background without your knowledge, use a simple string type for your ID, and do necessary conversions yourself wherever you need to store / transmit your ID.
For the ID Field, you should use the primitive.ObjectID provided by the bson package.
"go.mongodb.org/mongo-driver/bson/primitive"
ID primitive.ObjectID `json:"_id" bson:"_id"`
As I know, we can use
> db['twitter-3'].find({}, {"text": 1})
to select all texts in collection.
How can we use mgo to find specific field in golang?
I tried
var result []string
err = conn.Find(bson.M{}, bson.M{"text", 1}).All(&result)
But it is not correct.
Use the query Select method to specify the fields to return:
var result []struct{ Text string `bson:"text"` }
err := c.Find(nil).Select(bson.M{"text": 1}).All(&result)
if err != nil {
// handle error
}
for _, v := range result {
fmt.Println(v.Text)
}
In this example, I declared an anonymous type with the one selected field. It's OK to use a type with all document fields.
to select multiple fields:
var result []struct{
Text string `bson:"text"`
Otherfield string `bson:"otherfield"`
}
err := c.Find(nil).Select(bson.M{"text": 1, "otherfield": 1}).All(&result)
if err != nil {
// handle error
}
for _, v := range result {
fmt.Println(v.Text)
}
var result interface{}
err = c.Find(nil).Select(bson.M{"text": 1}).All(&result)
The mongodb documentation says:
The fields and values of both the and parameters if the parameter contains only update operator expressions. The update creates a base document from the equality clauses in the parameter, and then applies the update expressions from the parameter.
And the mgo documentation says:
Upsert finds a single document matching the provided selector document and modifies it according to the update document. If no document matching the selector is found, the update document is applied to the selector document and the result is inserted in the collection.
But if i do an upsert like this:
session.UpsertId(data.Code, data)
I end up with an entry which have an ObjectID generated automatically by mongodb, instead of data.Code.
this means that UpsertId expect data to be formated with update operators and you can't use a an arbitrary struct? Or what i'm missing here?
Pd. Mongo 2.4.9 mgo v2 golang go version devel +f613443bb13a
EDIT:
This is a sample of what i mean, using the sample code from Neil Lunn:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
// "gopkg.in/mgo.v2/bson"
)
type Person struct {
Code string
Name string
}
func main() {
session, err := mgo.Dial("admin:admin#localhost");
if err != nil {
fmt.Println("Error: ", err)
return
// panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
Code: "1234",
Name: "Bill",
}
_, err = c.UpsertId( p.Code, &p )
result := Person{}
err = c.FindId(p.Code).One(&result)
if err != nil {
fmt.Println("FindId Error: ", err)
return
// panic(err)
}
fmt.Println("Person", result)
}
I found the documentation of the MongoDB was right. The correct way to do this is to wrap the struct to insert into an update operator.
The sample code provided by Neil Lunn, would look like:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Person struct {
Code string
Name string
}
func main() {
session, err := mgo.Dial("admin:admin#localhost");
if err != nil {
fmt.Println("Error: ", err)
return
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
Code: "1234",
Name: "Bill",
}
upsertdata := bson.M{ "$set": p}
info , err2 := c.UpsertId( p.Code, upsertdata )
fmt.Println("UpsertId -> ", info, err2)
result := Person{}
err = c.FindId(p.Code).One(&result)
if err != nil {
fmt.Println("FindId Error: ", err)
return
}
fmt.Println("Person", result)
}
Thank you very much for your interest and help Neil.
You seem to be talking about assigning a struct with a custom _id field here. This really comes down to how you define your struct. Here is a quick example:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Person struct {
ID string `bson:"_id"`
Name string
}
func main() {
session, err := mgo.Dial("127.0.0.1");
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
ID: "1",
Name: "Bill",
}
_, err = c.UpsertId( p.ID, &p )
result := Person{}
err = c.Find(bson.M{"_id": p.ID}).One(&result)
if err != nil {
panic(err)
}
fmt.Println("Person", result)
}
So in the custom definition here I am mapping the ID field to bson _id and defining it's type as string. As shown in the example this is exactly what happens when serialized via UpsertId and then retrieved.
Now you have elaborated I'll point to the difference on the struct definition.
What I have produces this:
{ "_id": 1, "name": "Bill" }
What you have ( without the same mapping on the struct ) does this:
{ "_id": ObjectId("53cfa557e248860d16e1f7e0"), "code": 1, "name": "Bill" }
As you see, the _id given in the upsert will never match because none of your fields in the struct are mapped to _id. You need the same as I have:
type Person struct {
Code string `bson:"_id"`
Name string
}
That maps a field to the mandatory _id field, otherwise one is automatically produced for you.