Golang/mgo : How can I store Date (not ISODate) in mongodb? - mongodb

If I store current time like this:
type Test struct {
Id string `bson:"id" json:"id,omitempty"`
TestTime time.Time `bson:"testTime" json:"testTime,omitempty"`
}
...
t := Test {
Id : "TEST0001",
TestTime : time.Now(),
}
...
c.Insert(t)
Then I use mongochef to search it :
{
"_id" : ObjectId("576bc7a48114a14b47920d60"),
"id" : "TEST0001",
"testTime" : ISODate("2016-06-23T11:27:30.447+0000")
}
So, mgo store ISODate by default, how can I store Date not ISODate ?

mgo automagically converts time.Time into a Mongo internal date data type (source, actually it's just a timestamp with no timezone info and it's always corrected to UTC). Any other functionaly has to be manually implemented by you.
You can force mgo to correctly (de)serialize your types by implementing the Getter and Setter interfaces from package mgo/bson thought it's pretty low-level so be careful.

You should define a custom struct that saves the timezone like this.
The you can define a custom Unmarshal that changes the location of the date on loading.
func (t *TimeWithTimezone) Unmarshal(in []byte, out interface{}) (err error) {
type decode TimeWithTimezone
var d decode
if err := bson.NewDecoder(in).Decode(&d); err != nil {
return err
}
loc, err := FixedZone(d.Timezone, d.Timezone)
if err != nil {
return fmt.Errorf("Invalid Timezone: %s", d.Timezone)
}
t.Time = d.Time.In(loc)
t.Timezone = d.Timezone
return nil
}
Something like this should do the trick, it's not tested, just to give you an idea!

Related

MongoDB returns object data as array of key-value pair in Go

I have written following query which returns me records where updated_at date is greater than synced_at date from all records using mongodb library in Golang.
pipeline := []bson.M{}
filter := []string{"$updated_at", "$synced_at"}
pipeline = append(pipeline, bson.M{"$match": bson.M{"$expr": bson.M{"$gte": filter}}})
opts := options.Aggregate().SetMaxTime(2 * time.Second)
cursor, err := collection.Aggregate(ctx, pipeline, opts)
for cursor.Next(context.Background()) {
records := model.DData{}
err = cursor.Decode(&records)
}
The structure of Data is:
type DData struct {
Name string `json:"name" bson:"name"`
Data interface{} `json:"data" bson:"data"`
UpdatedAt time.Time `json:"updated_at" bson:"updated_at"`
SyncedAt time.Time `json:"synced_at" bson:"synced_at"`
}
Data in collection is of the form:
{
"name":"Vallabh",
"data":{
"field1":"value1",
"field2":"value2",
"field3":"value3",
},
"updated_at":2021-08-17T09:43:27Z,
"synced_at":2021-08-07T09:43:27Z
}
But with above query I am getting data in the form:
{
"name":"Vallabh",
"data":[
{
"key":"field1",
"value":"value1"
},
{
"key":"field2",
"value":"value2"
},
{
"key":"field3",
"value":"value3"
}
}],
"updated_at":"2021-08-17T09:43:27Z",
"synced_at":"2021-08-07T09:43:27Z"
}
What am I doing wrong? Its happening only when field type is an interface in struct.
Try with this
type DData struct {
Name string `json:"name" bson:"name"`
Data map[string]string `json:"data" bson:"data"`
UpdatedAt time.Time `json:"updated_at" bson:"updated_at"`
SyncedAt time.Time `json:"synced_at" bson:"synced_at"`
}
Always use the type if you know it beforehand, with interface{} the burden is on the library to find out what the type is. If you expect the varied values in the map you can use map[string]interface{}
Found a very weird solution to this problem where I decoded it to bson.M{}, then marshalled it and unmarshalled it to my struct. I am sure there are better ways but this one worked for me.
for cursor.Next(context.Background()) {
tempResult := bson.M{}
err = decode(cursor, &tempResult)
if err != nil {
logger.LogErrorf(err, "error while decoding")
continue
}
obj, err := marshal(tempResult)
if err != nil {
logger.LogErrorf(err, "error while marshalling")
continue
}
var data model.DData
err = json.Unmarshal(obj, &data)
if err != nil {
tenant.LogError(err, "error while marshalling")
continue
}
}
#VallabhLakade I had similar concern like this and tried the below way that helped .
So basically what's the problem is that the mongo-driver defaults to unmarshalling as bson.D for structs of type interface{} where as mgo mgo-driver defaults to bson.M .
So we will have to add the below code while trying to establish connection with mongo-db , SetRegistry() options as clientOpts To map the old mgo behavior, so that mongo-driver defaults to bson.M while unmarshalling structs of type interface{} , and this should not display the values back as key-value pair
tM := reflect.TypeOf(bson.M{})
reg := bson.NewRegistryBuilder().RegisterTypeMapEntry(bsontype.EmbeddedDocument, tM).Build()
clientOpts := options.Client().ApplyURI(SOMEURI).SetAuth(authVal).SetRegistry(reg)
client, err := mongo.Connect(ctx, clientOpts)
Reference -> MongoDB document returns array of key value pair in go mongo-driver
#Vallabh you shall use mgocompat too something like below this should mimic the old mgo behavior
registry := mgocompat.NewRegistryBuilder().Build()
connectOpts := options.Client().ApplyURI(SOMEURI).SetAuth(info).SetRegistry(registry)

cannot set type primitive.ObjectID to ObjectID

Getting this error while binding mongodb document to GO Struct using docstore collection iterator.
Database: Azure CosmosDB mongoDb API
Go Driver: Docstore
Code details:
Cosmos DB Constructors Code
client, err := c.newClient(ctx, true)
if err != nil {
log.Error("error connecting to mongodb cluster", zap.Error(err))
return nil, err
}
database := client.Database(c.dbName)
collection := database.Collection(c.collName)
return mongodocstore.OpenCollection(collection, "", nil)
Go struct for mapping with Mongo Db document
type utterance struct {
ID primitive.ObjectID `docstore:"_id,omitempty"`
User string `docstore:"user,omitempty"`
Locale string `docstore:"Locale,omitempty"`
Text string `docstore:"Text,omitempty"`
Source string `docstore:"Source,omitempty"`
Timestamp time.Time `docstore:"Timestamp,omitempty"`
DocstoreRevision interface{}
}
MondoDb Document
{
"_id" : ObjectId("60d5e18539864e948a8851a6"),
"User" : "auth0|6049b5ef5d79540071db6a0a",
"Locale" : "en_US",
"Text" : "Hi",
"Source" : "UTTERANCE_SOURCE_USER",
"Timestamp" : {
"$date" : 1624629637002
},
"DocstoreRevision" : "bf3b35d8-54ed-4a23-a08f-7d41b5c34085"
}
Method to call the docstore collection and iterate
i := s.collection.Query().Get(ctx)
defer i.Stop()
var results []*services.Utterance
for {
fmt.Println("for every document: ")
//var native utterance
doc := &utterance{}
err := i.Next(ctx, doc) //error at this line
if err == io.EOF {
break
} else if err != nil {
fmt.Println(“getting to this err block“)
return nil, err
}
u := doc.ToProto()
results = append(results, u)
}
Try setting the ID to a pointer type:
type utterance struct {
ID *primitive.ObjectID `docstore:"_id,omitempty"`
User string `docstore:"user,omitempty"`
Locale string `docstore:"Locale,omitempty"`
Text string `docstore:"Text,omitempty"`
Source string `docstore:"Source,omitempty"`
Timestamp time.Time `docstore:"Timestamp,omitempty"`
DocstoreRevision interface{}
}
Probably when you create the struct to be decoded the primitive.ObjectID is being initialized and the driver don't know to decode it.
And be careful. The docstore is a generic Document driver. It is not specific for MongoDB. So it probably don't work well with MongoDB specific types.
Reading the driver for mongodb https://pkg.go.dev/gocloud.dev/docstore/mongodocstore
There is a way to set what is stored in the _id field. I understand docstore don't know how to decode ObjectID from MongoDB.

How to get ObjectId (_id) of my document in MGO mongo db driver for golang

I'm working with MGO (cause I didn't found anything better that it).
I'have played with it and have got some result but I don't understand how to get the _id (internal Mongo ObjectId) of document received?
For ex:
type FunnyNumber struct {
Value int
_id string
}
session, err := mgo.Dial("127.0.0.1:27017")
if err != nil {
panic(err)
}
defer session.Close()
// Optional. Switch the session to a monotonic behavior.
session.SetMode(mgo.Monotonic, true)
c := session.DB("m101").C("funnynumbers")
funnynumber := FunnyNumber{}
err = c.Find(bson.M{}).One(&funnynumber)
if err != nil {
log.Fatal(err)
}
fmt.Println("Id one:", funnynumber._id) // Nothing here? WHy? How to get it.
fmt.Println("Value one:", funnynumber.Value) // 62. It's OK!
Could someone help me, please? Ot where might I read some info about it? I haven't found anything in the MGO doc
Schema of my document is:
{ "_id" : ObjectId("50778ce69331a280cf4bcf90"), "value" : 62 }
Thanks!
Change _id variable to uppercase(ID) to make it exportable.
Use bson.ObjectID as its type.
Add tag for struct FunnyNumber Id variable.
field
Above three things should be done to get object Id value.
import "labix.org/v2/mgo/bson"
type FunnyNumber struct {
Value int `json:"value"`
Id bson.ObjectId `bson:"_id,omitempty"`` // only uppercase variables can be exported
}
Take a look at package BSON for more understanding on using bson tags when working with mongodb

query by date in mongodb

I'm able to insert an entry into MongoDB using the golang driver gopkg.in/mgo.vs and gopkg.in/mgo.vs/bson but I'm not able to pull it out. In the mongo shell, if I do
db.Items.find({ date : 1428762411980 })
it shows me the entry that I just inserted with the Go code. However, if I try to do the following to fetch it in Go, it's telling me that the record isn't found
func fetch(w http.ResponseWriter, r *http.Request){
var result SomeStruct
date := r.FormValue("date")
err := Items.Find(bson.M{"date":date}).One(&result)
...code omitted...
}
func Items() *mgo.Collection {
return DB().C("Items")
}
func DB() *mgo.Database {
return DBSession().DB("mydb")
}
One thing I noticed was that, in the shell, the date is stored as a NumberLong
"date" : NumberLong("1428762411980")
I'm wondering if I have to do something with the date value that I receive from the form in the fetch function before using it to query the database?
Update
Before saving the data to the db, it comes in as a json string like this
"date":1428762411980
I then decode it into a struct
type blah struct{
Id bson.ObjectId `json:"id" bson:"_id"`
Date int64 `json:"date" bson: "date"`
And it gets saved like this (as shown in the shell)
"date" : NumberLong("1428762411980")
r.FormValue returns a string, but you need an int64. Use strconv.ParseInt. Then your query should work.
date, err := strconv.ParseInt(r.FormValue("date"), 10, 64)
// handle err
err = Items.Find(bson.M{"date":date}).One(&result)

How to unmarshal a named type alias from a document with mgo?

I have a struct with updated_at field which I want to be JSON encoded as a unix timestamp.
I tried the following which doesn’t seem to work,
the updated_at field is never unmarshalled from the MongoDB document:
type Timestamp time.now
func (t Timestamp) MarshalJSON() ([]byte, error) {
ts := time.Time(t).Unix()
fmt.Println(ts)
stamp := fmt.Sprint(ts)
return []byte(stamp), nil
}
type User struct {
UpdatedAt *Timestamp `bson:"updated_at,omitempty" json:"updated_at,omitempty"`
}
I found a temp solution, to write the struct’s MarshalJSON function, doing something like this (changing the UpdatedAt type to *time.Time):
func (u *User) MarshalJSON() ([]byte, error) {
out := make(map[string]interface{})
if u.UpdatedAt != nil && !u.UpdatedAt.IsZero() {
out["updated_at"] = u.UpdatedAt.Unix()
}
return json.Marshal(out)
}
is there a better or more elegant solution for doing this?
found the solution elsewhere and wrote a post about it - https://medium.com/coding-and-deploying-in-the-cloud/time-stamps-in-golang-abcaf581b72f
for handling mgo's marshaling/unmarshaling one has to implement the GetBSON() and SetBSON() functions.
Your code isn't working because you need to implement MarshalJSON on *Timestamp not Timestamp.
func (t *Timestamp) MarshalJSON() ([]byte, error) { .... }