I need help. I am a self learner and I am trying to understand how to insert a variable that I read from a csv. I want to use that variable and insert it into the mongo db repeatedly again until it finishes reading the csv file. The code is like this. I would really appreciate the help. I am going crazy looking over all the info and on stack overflow. if i can get this right so i can build some confidence in coding with go. thank you
reader, err := csvreader.Read()
if err == io.EOF {
break
} else if err != nil {
panic(err)
}
fmt.Println(reader)
type Event struct {
event_id string `json: "event_id"`
date_added string `bson: time.time`
}
var event = Event{
event_id: {}string reader,
date_added: time.Now().Format("2006-01-02 15:04"),
}
result, err := cli.InsertOne(ctx, event)
fmt.Println(result)```
Related
I have a postgres db that I would like to generate tables for and write to using Gorp, however I get an error message when I try to insert due to the slices contained within my structs "sql: converting argument $4 type: unsupported type []core.EmbeddedStruct, a slice of struct.
My structs look as follows:
type Struct1 struct {
ID string
Name string
Location string
EmbeddedStruct []EmbeddedStruct
}
type EmbeddedStruct struct {
ID string
Name string
struct1Id string
EmbeddedStruct2 []EmbeddedStruct2
}
type EmbeddedStruct2 struct {
ID string
Name string
embeddedStructId string
}
func (repo *PgStruct1Repo) Write(t *core.Struct1) error {
trans, err := createTransaction(repo.dbMap)
defer closeTransaction(trans)
if err != nil {
return err
}
// Check to see if struct1 item already exists
exists, err := repo.exists(t.ID, trans)
if err != nil {
return err
}
if !exists {
log.Debugf("saving new struct1 with ID %s", t.ID)
err = trans.Insert(t)
if err != nil {
return err
}
return nil
}
return nil
}
Does anyone have any experience with/or know if Gorp supports inserting slices? From what I've read it seems to only support slices for SELECT statements
Gorp supports inserting a variadic number of slices, so if you have a slice records, you can do:
err = db.Insert(records...)
However, from your question it seems you want to save a single record that has a slice struct field.
https://github.com/go-gorp/gorp
gorp doesn't know anything about the relationships between your structs (at least not yet).
So, you have to handle the relationship yourself. The way I personally would solve this issue is to have Gorp ignore the slice on the parent:
type Struct1 struct {
ID string
Name string
Location string
EmbeddedStruct []EmbeddedStruct `db:"-"`
}
And then use the PostInsert hook to save the EmbeddedStruct (side note, this is a poor name as it is not actually an embedded struct)
func (s *Struct1) PostInsert(sql gorp.SqlExecutor) error {
for i := range s.EmbeddedStruct {
s.EmbeddedStruct[i].struct1Id = s.ID
}
return sql.Insert(s.EmbeddedStruct...)
}
And then repeat the process on EmbeddedStruct2.
Take care to setup the relationships properly on the DB side to ensure referential integrity (e.g. ON DELETE CASCADE / RESTRICT), and it would probably be a good idea to wrap the whole thing in a transaction.
I am fairly new to golang programming and the mongodb interface.
I've got a dbase of records created by another application. I am trying to walk the dbase and examine specific fields of each record. I can decode the full records as bson, but I cannot get the specific values.
This struct defines the 3 fields I would like to extract:
type myDbaseRec struct {
aid string `bson:"pon-util-aid"`
ingressPct string `bson:"ingress-bucket-percent"`
egressPct string `bson:"egress-bucket-percent"`
}
Here is my code to iterate through the cursor after the collection.Find(ctx, queryFilter) and decode the results both as bson and as my struct:
var myResult myDbaseRec
var bsonMResult bson.M
var count int
for cursor.Next(ctx) {
err := cursor.Decode(&myResult)
if err != nil {
fmt.Println("cursor.Next() error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(myResult))
fmt.Printf("result: %+v\n", myResult)
}
err = cursor.Decode(&bsonMResult)
if err != nil {
fmt.Println("bson decode error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(bsonMResult))
fmt.Println("\nresult:", bsonMResult)
}
}
Here is an example of one iteration of the loop. The bson decode appears to work, but my struct is empty:
result type: internal.myDbaseRec
result: {aid: ingressPct: egressPct:}
result type: primitive.M
result: map[pon-util-aid:ROLT-1-MONTREAL/1/1/xp2 _id:ObjectID("5d70b4d1b3605301ef72228b")
admitted-assured-upstream-bw:0 admitted-excess-upstream-bw:0 admitted-fixed-upstream-bw:0
assured-upstream-bytes:0 available-excess-upstream-bw:0 available-fixed-upstream-bw:622080
app_counters_key_field:ROLT-1-MONTREAL/1/1/xp2 app_export_time:1567665626 downstream-octets:52639862633214
egress-bucket-bps:8940390198 egress-bucket-percent:91 egress-bucket-seconds:559
excess-upstream-bytes:0 fixed-upstream-bytes:0 ingress-bucket-bps:8253153852
ingress-bucket-percent:84 ingress-bucket-seconds:559 sample-time:0 upstream-octets:48549268162714]
I would have expected to get
result: {aid:"ROLT-1-MONTREAL/1/1/xp2" ingressPct:84 egressPct:91}
Any suggestion on how to properly find these 3 fields from each record?
=== UPDATE: The first comment below answered my question.
First, in Go only fields starting with a (Unicode) upper case letter are exported. See also Exported identifiers. The default decoder will try to decode only to the exported fields. So you should change the struct into:
type myDbaseRec struct {
Aid string `bson:"pon-util-aid"`
IngressPct int32 `bson:"ingress-bucket-percent"`
EgressPct int32 `bson:"egress-bucket-percent"`
}
Also notice that the struct above for IngressPct and EgressPct have type int32. This is because the value in the document is represented in numbers (int/double), and not string. You may change it to other number type accordingly i.e. int16, int64, etc.
This is function that tries to update the code matching by its value
the res collection as having the code of Marque it will be compared with doc.Marque if it is the case it will be replaced by the value of the marque.
This code is working perfectly in mongoDB CLI, but as i am working with GO.
I tried to convert it into mgo as you may see below but it is not working , i did not find the foreach function in mgo , is there something to be replaced with in this case ? thanks for the help
db.res.find().forEach(function(doc){
var v = db.brands.findOne({code: doc.Marque});
if(v){
db.res.update({"Marque": doc.Marque},
{$set: {"Marque":v.value}}, {multi: true});
}
});
Here is what i tried :
result:=Results{}
pipe:=res.find(bson.M{}).Iter()
for pipe.Next(&result) {
brands:=brands.findOne({code: doc.Marque});
if(v){
pipe.update({"Marque": doc.Marque},
{$set: {"Marque": v.value}}, {multi: true});
}
}
Visit the mgo Godoc may help you understand how it works.
Second, exported types/functions in Golang are begin with a capital letter. So res.find, brands.findOne, ... should be res.Find, brands.FineOne respectively, if such functions exist.
// let's say you have a type like this
type myResult struct {
ID bson.ObjectId `bson:"_id"`
Marque string `bson:"Marque"`
// other fields...
}
// and another type like this
type myCode struct {
Code string `bson:"code"`
// other fields...
}
res := db.C("res")
brands := db.C("brands")
result := myResult{}
// iterate all documents
iter := res.Find(nil).Iter()
for iter.Next(&result) {
var v myCode
err := brands.Find(bson.M{"code": result.Marque}).One(&v)
if err != nil {
// maybe not found or other reason,
// it is recommend to have additional check
continue
}
query := bson.M{"_id": result.ID}
update := bson.M{"Marque": v.value}
if err = res.Update(query, update); err != nil {
// handle error
}
}
if err := iter.Close(); err != nil {
fmt.Println(err)
}
Basically I want update in one value from the mongodb document by given fully struct interface as a change parameter in collection.Upsert(selector,change). how we do this without lose other values into empty. Other(type,category.rerportby,createon,info) values should be keep on existing values only update plant and location values into PLANT07 and BAR)
NOTE: I want use completely Service Notification Struct Object for
do this.
DatabaseName:WO
CollectionName:SERVICE_NOTIFICATIONS
package models
//models.ServiceNotification
type ServiceNotification struct {
NotificationNo string `json:"notification_no" bson:"notification_no"`
Type string `json:"type" bson:"type"`
Category string `json:"category" bson:"category"`
Plant string `json:"plant" bson:"plant"`
Location string `json:"location" bson:"location"`
ReportedBy string `json:"reportedby" bson:"reportedby"`
Info map[string]interface{}`json:"info" bson:"info"`
SAPInfo SAPNotificationInfo `json:"sapinfo" bson:"sapinfo"`
CreateOn string `json:"createon" bson:"createon"`
UpdateOn string `json:"updateon" bson:"updateon"`
}
package main
func main(){
input := models.ServiceNotification{
NotificationNo:000120,
Plant:"Plant07",
Location:"BAR",
}
Change_ServiceNotification(input)
}
I want update plant and location by given complete struct interface to the mongo Upsert function. because I want to decide dynamically what should
update . But when I update plant and location other values going
to be LOST. in mongo record.
func Change_ServiceNotification(notification models.ServiceNotification) error {
session, err := commons.GetMongoSession()
if err != nil {
return errors.New("Cannot create mongodb session" + err.Error())
}
defer session.Close()
var col = session.DB(WO).C(SERVICE_NOTIFICATIONS)
selector := bson.M{"notification_no": notification.NotificationNo}
_, err = col.Upsert(selector, notification)
if err != nil {
errMsg := "Cannot update service notification " + err.Error()
return errors.New(errMsg)
}
return nil
}
Appreciate your help
Thanks in advance
You cannot do it this way, but you can use the $set operator of MongoDB (Skipping error checking):
input := Bson.M{
"$set": bson.M{
"plant": "Plant07",
// Further fields here...
}
}
selector := bson.M{"notification_no": notification.NotificationNo}
col.Upsert(selector, input)
This will update only the provided fields.
I have a small Go program which uses a a postgresql db. In it there is a query which can return no rows, and the code I'm using to deal with this isn't working correctly.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
err := db.QueryRow("SELECT SUM(delta) FROM karma WHERE nick = $1", nick).Scan(&karma)
var karmaStr string
switch {
case err == sql.ErrNoRows:
karmaStr = fmt.Sprintf("%s has no karma.", nick)
case err != nil:
log.Fatal(err)
default:
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
This logic is taken directly from the Go documentation. When there are no rows corresponding to nick, the following error occurs:
2016/07/24 19:37:07 sql: Scan error on column index 0: converting driver.Value type <nil> ("<nil>") to a int: invalid syntax
I must be doing something stupid - clues appreciated.
I believe your issue is that you're getting a NULL value back from the database, which go translates into nil. However, you're scanning into an integer, which has no concept of nil. One thing you can do is scan into a type that implements the sql.Scanner interface (and can handle NULL values), e.g., sql.NullInt64.
In the example code in the documentation, I'd assume they have a NOT NULL constraint on the username column. I think the reason for this is because they didn't want to lead people to believe that you have to use NULL-able types across the board.
I reworked the code to get the results I wanted.
// Get the karma value for nick from the database.
func getKarma(nick string, db *sql.DB) string {
var karma int
rows, err := db.Query("SELECT SUM(delta) FROM karma WHERE nick = $1", nick)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
karmaStr := fmt.Sprintf("%s has no karma.", nick)
if rows.Next() {
rows.Scan(&karma)
karmaStr = fmt.Sprintf("Karma for %s is %d.", nick, karma)
}
return karmaStr
}
Tempted to submit a documentation patch of some sort to the database/sql package.