I am facing a issue with update document using golang mongo driver.
Scenario: I want to update a field that is nested in a struct. For ex: StructOuter -> structInner -> field1, field2, field3. Now if I want to update the field3 and I have the corresponding value as another struct, how can i go ahead by just updating this field alone. I tried with code below but it updates the whole structInner leaving only field3:
conv, _ := bson.Marshal(prod)
bson.Unmarshal(conv, &updateFields)
update := bson.M{
"$set": updateFields,
}
model.SetUpdate(update).
Sample JSON:
{
"field_one": "value",
"data": {
"field_two": [
"data1",
"data2"
],
"field_three": "check",
"field_four": "abc",
"field_five": "work",
}
}
I want to avoid hard coded field query for updating.
Just want to know if this is supported, if yes can you help me with it and also point to some deep dive links on this.
If you have control over the code, you could try creating methods on the struct. These methods can help you construct the fields path to perform partial update. For example, if you have the following structs:
type Outer struct {
Data Inner `bson:"data"`
}
type Inner struct {
FieldThree string `bson:"field_three"`
FieldFour string `bson:"field_four"`
}
You can try adding methods as below to construct update statements. These are returned in the dot-notation format.
func (o *Outer) SetFieldThree(value string) bson.E {
return bson.E{"data.field_three", value}
}
func (o *Outer) SetFieldFour(value string) bson.E {
return bson.E{"data.field_four", value}
}
To update, you can construct the statements like below:
x := Outer{}
var updateFields bson.D
updateFields = append(updateFields, x.SetFieldThree("updated"))
updateFields = append(updateFields, x.SetFieldFour("updated"))
statement := bson.D{{"$set", updateFields}}
result, err := collection.UpdateOne(ctx, bson.M{}, statement)
Related
I have the following structure in my database:
{
"_id": {
"$oid": "5fc4fc68fcd604bac9f61f71"
},
"init": {
"fullname": "Besikta Anibula",
"parts": [
"Besikta",
"Anibula"
],
"alt": "Besikta Ani."
},
"industry": "E-Commerce"
}
I´m trying to just access the init object and write the results to a structured variable in Go:
var InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string`bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
collectionRESULTS.FindOne(ctx, options.FindOne().SetProjection(bson.M{"init": 1})).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
But the result is empty:
{"fullname":"","parts":null,"alt":""}
It is working when not using a projection like:
var InputData struct {
Ident primitive.ObjectID `bson:"_id" json:"id"`
}
collectionRESULTS.FindOne(ctx, bson.M{}).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
Result as expected:
{"id":"5fc4fc68fcd604bac9f61f71"}
You set the projection to only retrieve the init property of the result documents, and then you try to unmarshal into a struct value that does not have any matching field for the init value, so nothing will be set in that struct value.
You must use a value that has an init field, like this wrapper Result struct:
type Result struct {
Init InputData `bson:"init"`
}
type InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string `bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
Use it like this:
var result Result
err := collectionRESULTS.FindOne(ctx, bson.M{}, options.FindOne().
SetProjection(bson.M{"init": 1})).Decode(&result)
if err != nil {
// handle error
}
I'd like to get the names of all the keys in a MongoDB collection.
For example, from this:
"Id": ObjectId("5f5a010d431c4519dcda0e3d")
"title": "App"
"query": ""
"db": ""
"widgettype": ""
"tablename": "active_instance"
fields:Object
user:"name",
key:"passcode"
"status": "active"
"inlibrary": ""
"createdts": 1599733804
Using "gopkg.in/mgo.v2" and "gopkg.in/mgo.v2/bson" packages.
err := mongodbSession.DB(dbName).C(collectionName).Find(bson.M{}).One(&result)
var keyset []string
for index, _ := range result {
fmt.Printf("%+v\n", index)
keyset = append(keyset, index)
}
fmt.Println(keyset)
getting output as this
[_id title query db widgettype status fields inlibrary createdts ]
child key is not being featched that is user and key.
Embedded documents will appear as another bson.M values inside your result, so you have to use a recursion to also traverse those.
Here's how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
keys = append(keys, getKeys(m2)...)
}
}
return
}
Example using it:
m := bson.M{"Id": bson.ObjectId("5f5a010d431c4519dcda0e3d"),
"title": "App",
"query": "",
"db": "",
"widgettype": "",
"tablename": "active_instance",
"fields": bson.M{
"user": "name",
"key": "passcode",
},
"status": "active",
"inlibrary": "",
"createdts": 1599733804,
}
keys := getKeys(m)
fmt.Println(keys)
Which will output (try it on the Go Playground):
[db widgettype createdts inlibrary _id title query tablename
fields user key status]
If you look at the results, user and key are included, but it's not possible to tell if they were fields of the document or that of an embedded document.
You may choose to prefix fields of embedded documents with the field name of the embedded document field itself, e.g. to get fields.user and fields.key.
This is how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
for _, k2 := range getKeys(m2) {
keys = append(keys, k+"."+k2)
}
}
}
return
}
Which would output (try it on the Go Playground):
[createdts title query db status inlibrary _id widgettype tablename
fields fields.user fields.key]
Also note that the above solutions do not handle arrays. If you have arrays, you should also iterate over them, and if they contain another array or object, you should do the same (recursively). It's an exercise for you to extend it to handle arrays too.
I have a function for comparing two structs and making a bson document as input to mongodb updateOne()
Example struct format
type event struct {
...
Name string
StartTime int32
...
}
Diff function, please ignore that I have not checked for no difference yet.
func diffEvent(e event, u event) (bson.M, error) {
newValues := bson.M{}
if e.Name != u.Name {
newValues["name"] = u.Name
}
if e.StartTime != u.StartTime {
newValues["starttime"] = u.StartTime
}
...
return bson.M{"$set": newValues}, nil
}
Then I generated a test function like so:
func Test_diffEvent(t *testing.T) {
type args struct {
e event
u event
}
tests := []struct {
name string
args args
want bson.M
wantErr bool
}{
{
name: "update startime",
args: args{
e: event{StartTime: 1},
u: event{StartTime: 2},
},
want: bson.M{"$set": bson.M{"starttime": 2}},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := diffEvent(tt.args.e, tt.args.u)
if (err != nil) != tt.wantErr {
t.Errorf("diffEvent() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("diffEvent() = %v, want %v", got, tt.want)
}
})
}
}
This fails with a
--- FAIL: Test_diffEvent/update_startime (0.00s)
models_test.go:582: diffEvent() = map[$set:map[starttime:2]], want map[$set:map[starttime:2]]
For me this seem to be the same. I have played around with this and bool fields, string fields, enum fields, and fields as struct or fields as arrays of structs seems to work fine with deepequal, but it gives an error for int32 fields.
As a go beginner; what am I missing here? I would assume that if bool/string works then int32 would too.
This:
bson.M{"starttime": 2}
Sets the "starttime" key to the value of the literal 2. 2 is an untyped integer constant, and since no type is provided, its default type will be used which is int.
And 2 values stored in interface values are only equal if the dynamic value stored in them have identical type and value. So a value 2 with int type cannot be equal to a value 2 of type int32.
Use explicit type to tell you want to specify a value of int32 type:
bson.M{"starttime": int32(2)}
I am trying to insert an array into a MongoDB instance using Go. I have the [] string slice in Go and want to convert it into a BSON array to pass it to the DB using the github.com/mongodb/mongo-go-driver driver.
var result bson.Array
for _, data := range myData {
value := bson.VC.String(data)
result.Append(value)
}
This loops over each element of my input data and tries to append it to the BSON array. However the line with the Append() fails with panic: document is nil. How should I do this conversion?
Edit: The code in the question and this answer is no longer relevant because the bson.Array type was deleted from the package. At the time of this edit, the bson.A and basic slice operations should be used to construct arrays.
Use the factory function NewArray to create the array:
result := bson.NewArray()
for _, data := range myData {
value := bson.VC.String(data)
result.Append(value)
}
As mentioned by #Cerise bson.Array has since been deleted. I do this with multiple utility functions as follows:
func BSONStringA(sa []string) (result bson.A) {
result = bson.A{}
for_, e := range sa {
result = append(result, e)
}
return
}
func BSONIntA(ia []string) (result bson.A) {
// ...
}
Converting a slice of string (ids) to BSON array
var objIds bson.A
for _, val := range ids {
objIds = append(objIds, val)
}
log.Println(objIds)
I'm working with OpenStreeMap data dump into a MongoDB instance, the following collections exists nodes, ways and relations.
I'm querying all nodes within a radius from a given geospatial point, and to know how these nodes relate I'm working with the ways collection trying to retrieve all ways that contain any node from my previous geospatial query.
Then, I'm trying to include the geospatial coordinates in the way document (it already have a loc.coordinates field which is empty for some reason) using the node IDs it contains in the field loc.nodes. Along with the help provided in this answer I have come to the following code:
package main
import (
"fmt"
mgo "gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
// GeoJSON Holds data of geospatial points
type GeoJSON struct {
Type string `json:"-"`
Coordinates []float64 `json:"coordinates"`
}
type waynodes struct {
Type string
Coordinates []float64
Nodes []int
}
// OSMNode Represet a single point in space.
// https://wiki.openstreetmap.org/wiki/Node
//
// A node is one of the core elements in the OpenStreetMap data model. It
// consists of a single point in space defined by its latitude, longitude and
// node id. A third, optional dimension (altitude) can also be included:
// key:ele (abrev. for "elevation"). A node can also be defined as part of a
// particular layer=* or level=*, where distinct features pass over or under
// one another; say, at a bridge. Nodes can be used to define standalone point
// features, but are more often used to define the shape or "path" of a way.
type OSMNode struct {
ID int `bson:"_id"`
Location GeoJSON `bson:"loc"`
Tags map[string]interface{} `bson:"tags,omitempty"`
}
// OSMWay Represent an ordered list of nodes
// https://wiki.openstreetmap.org/wiki/Way
//
// A way is an ordered list of nodes which normally also has at least one tag
// or is included within a Relation. A way can have between 2 and 2,000 nodes,
// although it's possible that faulty ways with zero or a single node exist. A
// way can be open or closed. A closed way is one whose last node on the way is
// also the first on that way. A closed way may be interpreted either as a
// closed polyline, or an area, or both.
//
// The nodes defining the geometry of the way are enumerated in the correct
// order, and indicated only by reference using their unique identifier. These
// nodes must have been already defined separately with their coordinates.
type OSMWay struct {
ID int `bson:"_id"`
Location waynodes `bson:"loc"`
Tags map[string]interface{}
}
// km2miles convert a distance in kilometers to miles and then return the
// radius of such distance.
func km2miles(dist float64) float64 {
r := dist * 0.621371
// https://en.wikipedia.org/wiki/Earth_radius#Fixed_radius
return r / 3963.2
}
// nodes2list return a string list of node IDs from a list of OSMNode objects
func nodes2list(l []OSMNode) []int {
var list []int
for _, v := range l {
list = append(list, v.ID)
}
return list
}
// GetGeoWithinPos Return all points in a given point of Earth within the
// radius of `dist`.
func (db *DB) GetGeoWithinPos(long, lat, dist float64) ([]OSMWay, error) {
// Look at `nodes` document in our `osm` database
c := db.m.DB("osm").C("nodes")
// Query all nodes within a range from a spatial point: It should be
// equivalent to:
// db.nodes.find(
// {loc:
// {$geoWithin:
// {$centerSphere: [[-83.4995983, 10.1033002], 0.186411 / 3963.2]
// }
// }
// }, {"_id": 1});
var nodesresult []OSMNode
err := c.Find(bson.M{
"loc": bson.M{
"$geoWithin": bson.M{
"$centerSphere": []interface{}{
[]interface{}{long, lat}, km2miles(dist),
},
},
},
}).Select(bson.M{"_id": 1}).All(&nodesresult)
if err != nil {
return nil, err
} else if nodesresult == nil {
return nil, fmt.Errorf("Nodes not found on %f lat, %f long in a radius of %f km", lat, long, dist)
} else if nodesresult[0].ID == 0 {
return nil, fmt.Errorf("Nodes incorrectly unmarshall: %#v", nodesresult[0:3])
}
// Prepare a pipeline
pipe := []bson.M{
{
// Match only ways that contains the ID of the nodes
// from the query on `qsn`
"$match": bson.M{
"loc.nodes": bson.M{
"$in": nodes2list(nodesresult), // Return []int
},
},
},
{
// Now look for the nodes at `nodes` collection present
// at `loc.nodes` field...
"$lookup": bson.M{
"from": "nodes",
"localField": "loc.nodes",
"foreignField": "_id",
"as": "loc.coordinates",
},
},
{
// ...and set the field `loc.coordinates` with the
// coordinates of all nodes.
"$addField": bson.M{
"loc.coordinates": bson.M{
"$reduce": bson.M{
"input": "$loc.coordinates.loc.coordinates",
"initialValue": []float64{},
"in": bson.M{"$concatArrays": []string{"$$this", "$$value"}},
},
},
},
},
}
// Query ways collection
w := db.m.DB("osm").C("ways")
var ways []OSMWay
// Execute the pipeline 🤞
err = w.Pipe(pipe).All(&ways)
if ways == nil {
return nil, fmt.Errorf("Ways not found within %0.2f km/radius (%f mil/radius)", dist, km2miles(dist))
}
return ways, err
}
But the pipeline at the end returns nothing.
$ go test
--- FAIL: TestFetchData (1.80s)
db_test.go:16: from -83.4995983long, 10.1033002lat: Ways not found within 1.00 km/radius (0.000157 mil/radius)
I would like to know what I'm doing wrong here and why mgo cannot do what I'm looking to do.
For sake of completeness here is the test definition:
func TestFetchData(t *testing.T) {
db, err := NewDBConn("", "", "localhost", 27017)
if err != nil {
t.Fatalf("Could not establish connection with MongoDB: %s", err)
}
// Get data from some location in my hometown
_, err := db.GetGeoWithinPos(-83.4995983, 10.1033002, 1.0)
if err != nil {
t.Fatalf("from -83.4995983long, 10.1033002lat: %s", err)
}
}
Example documents
This is an example document from the ways collection:
{
"_id":492464922,
"tags":{
"maxspeed":"20",
"surface":"asphalt",
"highway":"residential",
"oneway":"yes",
"name":"Avenida 1"
},
"loc":{
"type":"Polygon",
"coordinates":[
],
"nodes":[
445848963,
4844871065,
432568566
]
}
}
This would be an example document from the nodes collection:
{
"_id":445848963,
"loc":{
"type":"Point",
"coordinates":[
-83.5047254,
10.0984515
]
}
}
And this would be the example output that I'm looking to return with query I'm trying to pass to the pipeline:
{
"_id":492464922,
"tags":{
"maxspeed":"20",
"surface":"asphalt",
"highway":"residential",
"oneway":"yes",
"name":"Avenida 1"
},
"loc":{
"type":"Polygon",
"coordinates":[
-83.5047254,
10.0984515,
-83.5052237,
10.0987132,
-83.5056339,
10.0989286
],
"nodes":[
445848963,
4844871065,
432568566
]
}
}
This is because there is a typo in your aggregation pipeline. The operator is called $addFields not $addField (missing an s).
The method invocation of w.Pipe() should throw an error something along the lines of Unrecognized pipeline stage name: '$addField'. However, your code is not checking the err variable that is returned by Pipe(). Since you're only checking variable ways which would be nil due to the error, your method returns (nil, "Ways not found within %0.2f km/radius (%f mil/radius)"); thus masking the pipeline error.
I would suggest to check the content check err first:
err = w.Pipe(pipe).All(&ways)
if err != nil {
//handle error
}