Get names of all keys in the collection using Go - mongodb

I'd like to get the names of all the keys in a MongoDB collection.
For example, from this:
"Id": ObjectId("5f5a010d431c4519dcda0e3d")
"title": "App"
"query": ""
"db": ""
"widgettype": ""
"tablename": "active_instance"
fields:Object
user:"name",
key:"passcode"
"status": "active"
"inlibrary": ""
"createdts": 1599733804
Using "gopkg.in/mgo.v2" and "gopkg.in/mgo.v2/bson" packages.
err := mongodbSession.DB(dbName).C(collectionName).Find(bson.M{}).One(&result)
var keyset []string
for index, _ := range result {
fmt.Printf("%+v\n", index)
keyset = append(keyset, index)
}
fmt.Println(keyset)
getting output as this
[_id title query db widgettype status fields inlibrary createdts ]
child key is not being featched that is user and key.

Embedded documents will appear as another bson.M values inside your result, so you have to use a recursion to also traverse those.
Here's how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
keys = append(keys, getKeys(m2)...)
}
}
return
}
Example using it:
m := bson.M{"Id": bson.ObjectId("5f5a010d431c4519dcda0e3d"),
"title": "App",
"query": "",
"db": "",
"widgettype": "",
"tablename": "active_instance",
"fields": bson.M{
"user": "name",
"key": "passcode",
},
"status": "active",
"inlibrary": "",
"createdts": 1599733804,
}
keys := getKeys(m)
fmt.Println(keys)
Which will output (try it on the Go Playground):
[db widgettype createdts inlibrary _id title query tablename
fields user key status]
If you look at the results, user and key are included, but it's not possible to tell if they were fields of the document or that of an embedded document.
You may choose to prefix fields of embedded documents with the field name of the embedded document field itself, e.g. to get fields.user and fields.key.
This is how you can do that:
func getKeys(m bson.M) (keys []string) {
for k, v := range m {
keys = append(keys, k)
if m2, ok := v.(bson.M); ok {
for _, k2 := range getKeys(m2) {
keys = append(keys, k+"."+k2)
}
}
}
return
}
Which would output (try it on the Go Playground):
[createdts title query db status inlibrary _id widgettype tablename
fields fields.user fields.key]
Also note that the above solutions do not handle arrays. If you have arrays, you should also iterate over them, and if they contain another array or object, you should do the same (recursively). It's an exercise for you to extend it to handle arrays too.

Related

Using MongoDB Projection

I have the following structure in my database:
{
"_id": {
"$oid": "5fc4fc68fcd604bac9f61f71"
},
"init": {
"fullname": "Besikta Anibula",
"parts": [
"Besikta",
"Anibula"
],
"alt": "Besikta Ani."
},
"industry": "E-Commerce"
}
I´m trying to just access the init object and write the results to a structured variable in Go:
var InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string`bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
collectionRESULTS.FindOne(ctx, options.FindOne().SetProjection(bson.M{"init": 1})).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
But the result is empty:
{"fullname":"","parts":null,"alt":""}
It is working when not using a projection like:
var InputData struct {
Ident primitive.ObjectID `bson:"_id" json:"id"`
}
collectionRESULTS.FindOne(ctx, bson.M{}).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
Result as expected:
{"id":"5fc4fc68fcd604bac9f61f71"}
You set the projection to only retrieve the init property of the result documents, and then you try to unmarshal into a struct value that does not have any matching field for the init value, so nothing will be set in that struct value.
You must use a value that has an init field, like this wrapper Result struct:
type Result struct {
Init InputData `bson:"init"`
}
type InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string `bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
Use it like this:
var result Result
err := collectionRESULTS.FindOne(ctx, bson.M{}, options.FindOne().
SetProjection(bson.M{"init": 1})).Decode(&result)
if err != nil {
// handle error
}

Nested field update using golang struct in mongoDB

I am facing a issue with update document using golang mongo driver.
Scenario: I want to update a field that is nested in a struct. For ex: StructOuter -> structInner -> field1, field2, field3. Now if I want to update the field3 and I have the corresponding value as another struct, how can i go ahead by just updating this field alone. I tried with code below but it updates the whole structInner leaving only field3:
conv, _ := bson.Marshal(prod)
bson.Unmarshal(conv, &updateFields)
update := bson.M{
"$set": updateFields,
}
model.SetUpdate(update).
Sample JSON:
{
"field_one": "value",
"data": {
"field_two": [
"data1",
"data2"
],
"field_three": "check",
"field_four": "abc",
"field_five": "work",
}
}
I want to avoid hard coded field query for updating.
Just want to know if this is supported, if yes can you help me with it and also point to some deep dive links on this.
If you have control over the code, you could try creating methods on the struct. These methods can help you construct the fields path to perform partial update. For example, if you have the following structs:
type Outer struct {
Data Inner `bson:"data"`
}
type Inner struct {
FieldThree string `bson:"field_three"`
FieldFour string `bson:"field_four"`
}
You can try adding methods as below to construct update statements. These are returned in the dot-notation format.
func (o *Outer) SetFieldThree(value string) bson.E {
return bson.E{"data.field_three", value}
}
func (o *Outer) SetFieldFour(value string) bson.E {
return bson.E{"data.field_four", value}
}
To update, you can construct the statements like below:
x := Outer{}
var updateFields bson.D
updateFields = append(updateFields, x.SetFieldThree("updated"))
updateFields = append(updateFields, x.SetFieldFour("updated"))
statement := bson.D{{"$set", updateFields}}
result, err := collection.UpdateOne(ctx, bson.M{}, statement)

Converting MongoDB $max result to golang data

I try to get max values from MongoDB collection from my Go code.
What type should I use to decode result?
When I use bson.D{} as val2 type the result looks like [{_id <nil>} {max 66} {cnt 14}].
Here's the code:
filter := []bson.M{{
"$group": bson.M{
"_id": nil,
"max": bson.M{"$max": "$hellid"},
}},
}
cursor, err := collection.Aggregate(ctx, filter)
for cursor.Next(ctx) {
val2 := ???
err := cursor.Decode(&val2)
fmt.Printf("cursor: %v, value: %v\n", cursor.Current, val2)
}
}
Using bson.D already works as you presented. The problem may be you can't "easily" get out the max and cnt values.
Model your result document with a struct like this:
type result struct {
Max int `bson:"max"`
Count int `bson:"cnt"
}
Although cnt is not produced by the example code you provided.
And then:
var res result
err := cursor.Decode(&res)

mgo with aggregation, filtering with another query and field alteration

I'm working with OpenStreeMap data dump into a MongoDB instance, the following collections exists nodes, ways and relations.
I'm querying all nodes within a radius from a given geospatial point, and to know how these nodes relate I'm working with the ways collection trying to retrieve all ways that contain any node from my previous geospatial query.
Then, I'm trying to include the geospatial coordinates in the way document (it already have a loc.coordinates field which is empty for some reason) using the node IDs it contains in the field loc.nodes. Along with the help provided in this answer I have come to the following code:
package main
import (
"fmt"
mgo "gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
// GeoJSON Holds data of geospatial points
type GeoJSON struct {
Type string `json:"-"`
Coordinates []float64 `json:"coordinates"`
}
type waynodes struct {
Type string
Coordinates []float64
Nodes []int
}
// OSMNode Represet a single point in space.
// https://wiki.openstreetmap.org/wiki/Node
//
// A node is one of the core elements in the OpenStreetMap data model. It
// consists of a single point in space defined by its latitude, longitude and
// node id. A third, optional dimension (altitude) can also be included:
// key:ele (abrev. for "elevation"). A node can also be defined as part of a
// particular layer=* or level=*, where distinct features pass over or under
// one another; say, at a bridge. Nodes can be used to define standalone point
// features, but are more often used to define the shape or "path" of a way.
type OSMNode struct {
ID int `bson:"_id"`
Location GeoJSON `bson:"loc"`
Tags map[string]interface{} `bson:"tags,omitempty"`
}
// OSMWay Represent an ordered list of nodes
// https://wiki.openstreetmap.org/wiki/Way
//
// A way is an ordered list of nodes which normally also has at least one tag
// or is included within a Relation. A way can have between 2 and 2,000 nodes,
// although it's possible that faulty ways with zero or a single node exist. A
// way can be open or closed. A closed way is one whose last node on the way is
// also the first on that way. A closed way may be interpreted either as a
// closed polyline, or an area, or both.
//
// The nodes defining the geometry of the way are enumerated in the correct
// order, and indicated only by reference using their unique identifier. These
// nodes must have been already defined separately with their coordinates.
type OSMWay struct {
ID int `bson:"_id"`
Location waynodes `bson:"loc"`
Tags map[string]interface{}
}
// km2miles convert a distance in kilometers to miles and then return the
// radius of such distance.
func km2miles(dist float64) float64 {
r := dist * 0.621371
// https://en.wikipedia.org/wiki/Earth_radius#Fixed_radius
return r / 3963.2
}
// nodes2list return a string list of node IDs from a list of OSMNode objects
func nodes2list(l []OSMNode) []int {
var list []int
for _, v := range l {
list = append(list, v.ID)
}
return list
}
// GetGeoWithinPos Return all points in a given point of Earth within the
// radius of `dist`.
func (db *DB) GetGeoWithinPos(long, lat, dist float64) ([]OSMWay, error) {
// Look at `nodes` document in our `osm` database
c := db.m.DB("osm").C("nodes")
// Query all nodes within a range from a spatial point: It should be
// equivalent to:
// db.nodes.find(
// {loc:
// {$geoWithin:
// {$centerSphere: [[-83.4995983, 10.1033002], 0.186411 / 3963.2]
// }
// }
// }, {"_id": 1});
var nodesresult []OSMNode
err := c.Find(bson.M{
"loc": bson.M{
"$geoWithin": bson.M{
"$centerSphere": []interface{}{
[]interface{}{long, lat}, km2miles(dist),
},
},
},
}).Select(bson.M{"_id": 1}).All(&nodesresult)
if err != nil {
return nil, err
} else if nodesresult == nil {
return nil, fmt.Errorf("Nodes not found on %f lat, %f long in a radius of %f km", lat, long, dist)
} else if nodesresult[0].ID == 0 {
return nil, fmt.Errorf("Nodes incorrectly unmarshall: %#v", nodesresult[0:3])
}
// Prepare a pipeline
pipe := []bson.M{
{
// Match only ways that contains the ID of the nodes
// from the query on `qsn`
"$match": bson.M{
"loc.nodes": bson.M{
"$in": nodes2list(nodesresult), // Return []int
},
},
},
{
// Now look for the nodes at `nodes` collection present
// at `loc.nodes` field...
"$lookup": bson.M{
"from": "nodes",
"localField": "loc.nodes",
"foreignField": "_id",
"as": "loc.coordinates",
},
},
{
// ...and set the field `loc.coordinates` with the
// coordinates of all nodes.
"$addField": bson.M{
"loc.coordinates": bson.M{
"$reduce": bson.M{
"input": "$loc.coordinates.loc.coordinates",
"initialValue": []float64{},
"in": bson.M{"$concatArrays": []string{"$$this", "$$value"}},
},
},
},
},
}
// Query ways collection
w := db.m.DB("osm").C("ways")
var ways []OSMWay
// Execute the pipeline 🤞
err = w.Pipe(pipe).All(&ways)
if ways == nil {
return nil, fmt.Errorf("Ways not found within %0.2f km/radius (%f mil/radius)", dist, km2miles(dist))
}
return ways, err
}
But the pipeline at the end returns nothing.
$ go test
--- FAIL: TestFetchData (1.80s)
db_test.go:16: from -83.4995983long, 10.1033002lat: Ways not found within 1.00 km/radius (0.000157 mil/radius)
I would like to know what I'm doing wrong here and why mgo cannot do what I'm looking to do.
For sake of completeness here is the test definition:
func TestFetchData(t *testing.T) {
db, err := NewDBConn("", "", "localhost", 27017)
if err != nil {
t.Fatalf("Could not establish connection with MongoDB: %s", err)
}
// Get data from some location in my hometown
_, err := db.GetGeoWithinPos(-83.4995983, 10.1033002, 1.0)
if err != nil {
t.Fatalf("from -83.4995983long, 10.1033002lat: %s", err)
}
}
Example documents
This is an example document from the ways collection:
{
"_id":492464922,
"tags":{
"maxspeed":"20",
"surface":"asphalt",
"highway":"residential",
"oneway":"yes",
"name":"Avenida 1"
},
"loc":{
"type":"Polygon",
"coordinates":[
],
"nodes":[
445848963,
4844871065,
432568566
]
}
}
This would be an example document from the nodes collection:
{
"_id":445848963,
"loc":{
"type":"Point",
"coordinates":[
-83.5047254,
10.0984515
]
}
}
And this would be the example output that I'm looking to return with query I'm trying to pass to the pipeline:
{
"_id":492464922,
"tags":{
"maxspeed":"20",
"surface":"asphalt",
"highway":"residential",
"oneway":"yes",
"name":"Avenida 1"
},
"loc":{
"type":"Polygon",
"coordinates":[
-83.5047254,
10.0984515,
-83.5052237,
10.0987132,
-83.5056339,
10.0989286
],
"nodes":[
445848963,
4844871065,
432568566
]
}
}
This is because there is a typo in your aggregation pipeline. The operator is called $addFields not $addField (missing an s).
The method invocation of w.Pipe() should throw an error something along the lines of Unrecognized pipeline stage name: '$addField'. However, your code is not checking the err variable that is returned by Pipe(). Since you're only checking variable ways which would be nil due to the error, your method returns (nil, "Ways not found within %0.2f km/radius (%f mil/radius)"); thus masking the pipeline error.
I would suggest to check the content check err first:
err = w.Pipe(pipe).All(&ways)
if err != nil {
//handle error
}

mgo find convert a single value array to string

This is part of my collection schema in mongodb:
{ "_id" : ObjectId("55e1eef5255da6d384754642"), "name" : [ "Web, Mobile & Software Dev", "Movil y desarrollo de software" ] } { "_id" : ObjectId("55e1f2d0255da6d38475464b"), "name" : [ "IT & Networking", "TI y Redes" ] } ...
Right now i can get the info like this:
err := r.Coll.Find(bson.M{}).Select(bson.M{"name": bson.M{"$slice": []int{1, 1}}}).All(&result.Data)
but i want "name" to return a string instead of a single value array, so i dont have to index it inside my frontend if no need.
very limited comments i need 2000 poits for editing my post and add more things it seems, this is not answer but maybe, so i have to loop it?, isnt better way?
err := r.Coll.Find(bson.M{}).Select(bson.M{"name": bson.M{"$slice": []int{1, 1}}}).All(&result.Data)
if err != nil {
return result, err
}
type skillnew struct {
Id bson.ObjectId `json:"id,omitempty" bson:"_id,omitempty"`
Name string `bson:"name,omitempty" json:"name,omitempty"`
}
skillsallnew := make([]skillnew, len(result.Data))
for i := range result.Data {
skillsallnew[i] = skillnew{result.Data[i].Id, result.Data[i].Name[0]}
}