Merge two MongoDB databases from old driver to official driver - mongodb

I am using go and I need to merge these two databases to keep the clients happy. I have changed the go code to use the official driver but it refuses to accept time.Time.
So in short, I need to transfer this model database
MgoResults struct {
ID bson.ObjectId `bson:"_id"`
ZipID string `bson:"zid"`
Message string `bson:"message"`
Result string `bson:"res"`
Error error `bson:"errors"`
ExpirationDate time.Time `bson:"expirationDate"`
}
into this model database
MgoResults struct {
ID primitive.ObjectID `bson:"_id"`
ZipID string `bson:"zid"`
Message string `bson:"message"`
Result string `bson:"res"`
Error error `bson:"errors"`
ExpirationDate primitive.DateTime `bson:"expirationDate"`
}
I don't have any strategies yet, but I welcome any examples with go code.
I have thought about iterating through all items and changing types then saving it to new model. Both databases work perfectly.
My main problem is during decoding and rebuilding the struct. I am probably doing something wrong.
var v bson.M
err = result.Decode(&v)
if err != nil {
log.Println("error at decode result : ", err)
}
//start filling the struct
res.ZipID = v["zid"].(string)
if v["errors"] == nil{
res.Error = nil
}else{
res.Error = v["errors"].(error)
}
res.Message= v["message"].(string)
res.ID = v["_id"].(primitive.ObjectID)
res.Result = v["res"].(string)
if v["expirationDate"] == nil{
//res.ExpirationDate = time.Now()
}else{
res.ExpirationDate = v["expirationDate"].(time.Time )
//res.ExpirationDate = v["expirationDate"].(primitive.DateTime)
}

Thanks to #icza and this slightly related answer here: https://stackoverflow.com/a/64419437/2912740 .
I was able to modify my code a bit and it worked. So it looks like mongo returns its own time that needs some conversion to regular time.
mytime := v["expirationDate"].(primitive.DateTime)
res.ExpirationDate = mytime.Time()

Related

question about inserting a variable into a mongodb

I need help. I am a self learner and I am trying to understand how to insert a variable that I read from a csv. I want to use that variable and insert it into the mongo db repeatedly again until it finishes reading the csv file. The code is like this. I would really appreciate the help. I am going crazy looking over all the info and on stack overflow. if i can get this right so i can build some confidence in coding with go. thank you
reader, err := csvreader.Read()
if err == io.EOF {
break
} else if err != nil {
panic(err)
}
fmt.Println(reader)
type Event struct {
event_id string `json: "event_id"`
date_added string `bson: time.time`
}
var event = Event{
event_id: {}string reader,
date_added: time.Now().Format("2006-01-02 15:04"),
}
result, err := cli.InsertOne(ctx, event)
fmt.Println(result)```

How to insert slice

I have a postgres db that I would like to generate tables for and write to using Gorp, however I get an error message when I try to insert due to the slices contained within my structs "sql: converting argument $4 type: unsupported type []core.EmbeddedStruct, a slice of struct.
My structs look as follows:
type Struct1 struct {
ID string
Name string
Location string
EmbeddedStruct []EmbeddedStruct
}
type EmbeddedStruct struct {
ID string
Name string
struct1Id string
EmbeddedStruct2 []EmbeddedStruct2
}
type EmbeddedStruct2 struct {
ID string
Name string
embeddedStructId string
}
func (repo *PgStruct1Repo) Write(t *core.Struct1) error {
trans, err := createTransaction(repo.dbMap)
defer closeTransaction(trans)
if err != nil {
return err
}
// Check to see if struct1 item already exists
exists, err := repo.exists(t.ID, trans)
if err != nil {
return err
}
if !exists {
log.Debugf("saving new struct1 with ID %s", t.ID)
err = trans.Insert(t)
if err != nil {
return err
}
return nil
}
return nil
}
Does anyone have any experience with/or know if Gorp supports inserting slices? From what I've read it seems to only support slices for SELECT statements
Gorp supports inserting a variadic number of slices, so if you have a slice records, you can do:
err = db.Insert(records...)
However, from your question it seems you want to save a single record that has a slice struct field.
https://github.com/go-gorp/gorp
gorp doesn't know anything about the relationships between your structs (at least not yet).
So, you have to handle the relationship yourself. The way I personally would solve this issue is to have Gorp ignore the slice on the parent:
type Struct1 struct {
ID string
Name string
Location string
EmbeddedStruct []EmbeddedStruct `db:"-"`
}
And then use the PostInsert hook to save the EmbeddedStruct (side note, this is a poor name as it is not actually an embedded struct)
func (s *Struct1) PostInsert(sql gorp.SqlExecutor) error {
for i := range s.EmbeddedStruct {
s.EmbeddedStruct[i].struct1Id = s.ID
}
return sql.Insert(s.EmbeddedStruct...)
}
And then repeat the process on EmbeddedStruct2.
Take care to setup the relationships properly on the DB side to ensure referential integrity (e.g. ON DELETE CASCADE / RESTRICT), and it would probably be a good idea to wrap the whole thing in a transaction.

How to extract fields from mongodb record when iterating with cursor in Golang

I am fairly new to golang programming and the mongodb interface.
I've got a dbase of records created by another application. I am trying to walk the dbase and examine specific fields of each record. I can decode the full records as bson, but I cannot get the specific values.
This struct defines the 3 fields I would like to extract:
type myDbaseRec struct {
aid string `bson:"pon-util-aid"`
ingressPct string `bson:"ingress-bucket-percent"`
egressPct string `bson:"egress-bucket-percent"`
}
Here is my code to iterate through the cursor after the collection.Find(ctx, queryFilter) and decode the results both as bson and as my struct:
var myResult myDbaseRec
var bsonMResult bson.M
var count int
for cursor.Next(ctx) {
err := cursor.Decode(&myResult)
if err != nil {
fmt.Println("cursor.Next() error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(myResult))
fmt.Printf("result: %+v\n", myResult)
}
err = cursor.Decode(&bsonMResult)
if err != nil {
fmt.Println("bson decode error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(bsonMResult))
fmt.Println("\nresult:", bsonMResult)
}
}
Here is an example of one iteration of the loop. The bson decode appears to work, but my struct is empty:
result type: internal.myDbaseRec
result: {aid: ingressPct: egressPct:}
result type: primitive.M
result: map[pon-util-aid:ROLT-1-MONTREAL/1/1/xp2 _id:ObjectID("5d70b4d1b3605301ef72228b")
admitted-assured-upstream-bw:0 admitted-excess-upstream-bw:0 admitted-fixed-upstream-bw:0
assured-upstream-bytes:0 available-excess-upstream-bw:0 available-fixed-upstream-bw:622080
app_counters_key_field:ROLT-1-MONTREAL/1/1/xp2 app_export_time:1567665626 downstream-octets:52639862633214
egress-bucket-bps:8940390198 egress-bucket-percent:91 egress-bucket-seconds:559
excess-upstream-bytes:0 fixed-upstream-bytes:0 ingress-bucket-bps:8253153852
ingress-bucket-percent:84 ingress-bucket-seconds:559 sample-time:0 upstream-octets:48549268162714]
I would have expected to get
result: {aid:"ROLT-1-MONTREAL/1/1/xp2" ingressPct:84 egressPct:91}
Any suggestion on how to properly find these 3 fields from each record?
=== UPDATE: The first comment below answered my question.
First, in Go only fields starting with a (Unicode) upper case letter are exported. See also Exported identifiers. The default decoder will try to decode only to the exported fields. So you should change the struct into:
type myDbaseRec struct {
Aid string `bson:"pon-util-aid"`
IngressPct int32 `bson:"ingress-bucket-percent"`
EgressPct int32 `bson:"egress-bucket-percent"`
}
Also notice that the struct above for IngressPct and EgressPct have type int32. This is because the value in the document is represented in numbers (int/double), and not string. You may change it to other number type accordingly i.e. int16, int64, etc.

How to get ObjectId (_id) of my document in MGO mongo db driver for golang

I'm working with MGO (cause I didn't found anything better that it).
I'have played with it and have got some result but I don't understand how to get the _id (internal Mongo ObjectId) of document received?
For ex:
type FunnyNumber struct {
Value int
_id string
}
session, err := mgo.Dial("127.0.0.1:27017")
if err != nil {
panic(err)
}
defer session.Close()
// Optional. Switch the session to a monotonic behavior.
session.SetMode(mgo.Monotonic, true)
c := session.DB("m101").C("funnynumbers")
funnynumber := FunnyNumber{}
err = c.Find(bson.M{}).One(&funnynumber)
if err != nil {
log.Fatal(err)
}
fmt.Println("Id one:", funnynumber._id) // Nothing here? WHy? How to get it.
fmt.Println("Value one:", funnynumber.Value) // 62. It's OK!
Could someone help me, please? Ot where might I read some info about it? I haven't found anything in the MGO doc
Schema of my document is:
{ "_id" : ObjectId("50778ce69331a280cf4bcf90"), "value" : 62 }
Thanks!
Change _id variable to uppercase(ID) to make it exportable.
Use bson.ObjectID as its type.
Add tag for struct FunnyNumber Id variable.
field
Above three things should be done to get object Id value.
import "labix.org/v2/mgo/bson"
type FunnyNumber struct {
Value int `json:"value"`
Id bson.ObjectId `bson:"_id,omitempty"`` // only uppercase variables can be exported
}
Take a look at package BSON for more understanding on using bson tags when working with mongodb

Golang mgo result into simple slice

I'm fairly new to both Go and MongoDB. Trying to select a single field from the DB and save it in an int slice without any avail.
userIDs := []int64{}
coll.Find(bson.M{"isdeleted": false}).Select(bson.M{"userid": 1}).All(&userIDs)
The above prints out an empty slice. However, if I create a struct with a single ID field that is int64 with marshalling then it works fine.
All I am trying to do is work with a simple slice containing IDs that I need instead of a struct with a single field. All help is appreciated.
Because mgo queries return documents, a few lines of code is required to accomplish the goal:
var result []struct{ UserID int64 `bson:"userid"` }
err := coll.Find(bson.M{"isdeleted": false}).Select(bson.M{"userid": 1}).All(&result)
if err != nil {
// handle error
}
userIDs := make([]int64, len(result))
for i := range result {
userIDs[i] = result.UserID
}