I'm creating an API in Go using MongoDB and mgo as storage engine.
I wrote sort of an abstraction for GET requests letting user filter the results by fields in query string parameters, but it only works for string fields.
I'm searching for a way to get a field's type with only field name, in order to cast parameter to correct type before searching in collection.
Here is the code:
func (db *DataBase) GetByFields(fields *map[string]interface{}, collection string) ([]DataModel, error) {
var res []interface{}
Debug("Getting " + collection + " by fields: ")
for i, v := range *fields {
Debug("=> " + i + " = " + v.(string))
// Here would be the type checking
}
if limit, ok := (*fields)["limit"]; ok {
limint, err := strconv.Atoi(limit.(string))
if err != nil {...} // Err Handling
delete(*fields, "limit")
err = db.DB.C(collection).Find(fields).Limit(limint).All(&res)
if err != nil {...} // Err Handling
} else {
err := db.DB.C(collection).Find(fields).All(&res)
if err != nil {...} // Err Handling
}
resModel := ComputeModelSlice(res, collection)
return resModel, nil
}
With mongodb I can check type with:
db.getCollection('CollectionName').findOne().field_name instanceof typeName
But I can't find a way to perform that with mgo.
Any idea?
I'm not sure about a way to get the type of the field before doing the query, but one approach is to simply query into an bson.M and then do type detection on the retrieved values:
var res bson.M
// ...
err = db.DB.C(collection).Find(fields).Limit(limint).All(&res)
// ...
for key, val := range res {
switch val.(type) {
case string:
// handle
case int:
// handle
// ...
default:
// handle
}
}
Related
I'm trying to search a document in golang using mongodb driver. But the result is always blank.
This is my code:
var bsonString bson.M
var jsonString string
fmt.Printf("[%s] > request for url\n", req.RemoteAddr)
w.Header().Set("Access-Control-Allow-Origin", "*")
dataSource := client.Database(dbName)
collection := dataSource.Collection(collectionName)
err := collection.FindOne(context.Background(), bson.D{{"question.title", "Question"}}).Decode(&bsonString)
if err != nil {
if err == mongo.ErrNoDocuments {
// This error means your query did not match any documents.
log.Println("No matched documents!")
return
}
panic(err)
}
finalBytes, _ := bson.Marshal(bsonString)
bson.Unmarshal(finalBytes, &jsonString)
fmt.Println(jsonString)
My data is:
{"_id":{"$oid":"631c5c78e606582e2ad78e2d"},"question":{"title":"Question","create_by":"AZ","time":{"$numberLong":"1661394765044"},"detail":"<h4>info</h4>"},"answers":[{"create_by":"baa","time":{"$numberLong":"1661394765044"},"detail":"<h4>abc</h4>"}]}
You are unmarshalling a document into a string. If you did error handling you would see the error:
// no error haldling
json.Unmarshal(finalBytes, &jsonString)
// with error handling
err = json.Unmarshal(finalBytes, &jsonString)
if err != nil {
fmt.Println(err)
// prints: json: cannot unmarshal object into Go value of type string
}
You can create a struct to match your data or unmarshal into interface (for unstructured data).
// declare variables
var bsonString bson.M
var jsonString string
dataSource := client.Database(dbName)
collection := dataSource.Collection(collectionName)
//
dataObject := map[string]interface{}{}
fmt.Printf("[%s] > request for url\n", req.RemoteAddr)
// set headers
w.Header().Set("Access-Control-Allow-Origin", "*")
err := collection.FindOne(context.Background(), bson.D{{"question.title", "Question"}}).Decode(&bsonString)
if err != nil {
if err == mongo.ErrNoDocuments {
// This error means your query did not match any documents.
log.Println("No matched documents!")
return
}
panic(err)
}
// marshal & unmarshall
finalBytes, _ := bson.Marshal(bsonString)
bson.Unmarshal(finalBytes, &dataObject)
fmt.Println(dataObject)
I'm trying to convert the structure I'm decoding with my query to map[string]interface.
Here is my code:
var m map[string]interface
var result []Result
type Result struct {
Id ResultId `bson:"_id"`
Filename string `bson:"filename"`
}
type ResultId struct {
Host string `bson:"host"`
}
group := bson.D{{"$group", bson.D{{"_id", bson.D{{"host","$host"}}}, {"filename", bson.D{{"$last","$filename"}}}}}}
collection := client.Database("mongodb").Collection("Meta")
cursor, err := collection.Aggregate(ctx, mongo.Pipeline{group})
if err != nil {
return c.JSON(http.StatusInternalServerError, err)
}
defer cursor.Close(ctx)
if err = cursor.All(ctx, &results); err != nil {
fmt.Printf("cursor.All() error:", err)
return c.JSON(http.StatusInternalServerError, err)
}
for _, value := range results {
m = append(m,&bson.M{value.Id.Host:value.Filename})
}
But it does not return a map[string]interface and for information I use the go.mongodb.org package.
append only works on slices, (refer to effective go's section for more on append)
The way to add elements to a map, is simply:
m["key"] = value
Also keep in mind maps need to be initialised which I don't see in your code. Either with make or by giving an initial value (can be an empty value)
I have the following Mongo query I'm trying to translate to Go using github.com/globalsign/mgo:
db.getCollection('cluster').find({"clusterName": {"$in": ["clusterA", "clusterB"]}})
"clusterName" is a string field. Basically the naive alternative would be to perform multiple calls to mongo for each value in the list.
The query I wrote:
func ReadClusters(clusterNames []string) (*[]kusto.Cluster, error) {
var clusters *[]kusto.Cluster
err := readObjects(clusterCollection, bson.M{"clusterName": bson.M{"$in": clusterNames}}, &clusters, "" /* sortField */)
if err != nil {
return nil, err
}
return clusters, nil
}
And my helper functions:
func readObjects(collection string, query bson.M, dest interface{}, sortField string) error {
err := getDocuments(collection, query, dest, sortField)
if err != nil {
if err == mgo.ErrNotFound {
return ErrNotFound
}
return err
}
return nil
}
func getDocuments(collectionName string, query bson.M, dest interface{}, sortField string) error {
session := client.Copy()
defer session.Close()
collection := getCollection(session, collectionName)
find := collection.Find(query)
if sortField != "" {
find = find.Sort(sortField)
}
return find.All(dest)
}
I'm getting the error:
2020/07/09 11:58:46 http: panic serving [::1]:54085: result argument
must be a slice address
I'm currently using Go1.11, and the mgo version I see under go.mod is github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
clusters is already of pointer type to slice, so taking its address will be a pointer to pointer to slice.
Declare it to be a non-pointer to slice:
var clusters []kusto.Cluster
Now I'm doing:
sess := mongodb.DB("mybase").C("mycollection")
var users []struct {
Username string `bson:"username"`
}
err = sess.Find(nil).Select(bson.M{"username": 1, "_id": 0}).All(&users)
if err != nil {
fmt.Println(err)
}
var myUsers []string
for _, user := range users{
myUsers = append(myUsers, user.Username)
}
Is there a more effective way to get slice with usernames from Find (or another search function) directly, without struct and range loop?
The result of a MongoDB find() is always a list of documents. So if you want a list of values, you have to convert it manually just as you did.
Using a custom type (derived from string)
Also note that if you would create your own type (derived from string), you could override its unmarshaling logic, and "extract" just the username from the document.
This is how it could look like:
type Username string
func (u *Username) SetBSON(raw bson.Raw) (err error) {
doc := bson.M{}
if err = raw.Unmarshal(&doc); err != nil {
return
}
*u = Username(doc["username"].(string))
return
}
And then querying the usernames into a slice:
c := mongodb.DB("mybase").C("mycollection") // Obtain collection
var uns []Username
err = c.Find(nil).Select(bson.M{"username": 1, "_id": 0}).All(&uns)
if err != nil {
fmt.Println(err)
}
fmt.Println(uns)
Note that []Username is not the same as []string, so this may or may not be sufficient to you. Should you need a user name as a value of string instead of Username when processing the result, you can simply convert a Username to string.
Using Query.Iter()
Another way to avoid the slice copying would be to call Query.Iter(), iterate over the results and extract and store the username manually, similarly how the above custom unmarshaling logic does.
This is how it could look like:
var uns []string
it := c.Find(nil).Select(bson.M{"username": 1, "_id": 0}).Iter()
defer it.Close()
for doc := (bson.M{}); it.Next(&doc); {
uns = append(uns, doc["username"].(string))
}
if err := it.Err(); err != nil {
fmt.Println(err)
}
fmt.Println(uns)
I don't see what could be more effective than a simple range loop with appends. Without all the Mongo stuff your code basically is this and that's exactly how I would do this.
package main
import (
"fmt"
)
type User struct {
Username string
}
func main() {
var users []User
users = append(users, User{"John"}, User{"Jane"}, User{"Jim"}, User{"Jean"})
fmt.Println(users)
// Interesting part starts here.
var myUsers []string
for _, user := range users {
myUsers = append(myUsers, user.Username)
}
// Interesting part ends here.
fmt.Println(myUsers)
}
https://play.golang.com/p/qCwENmemn-R
As I know, we can use
> db['twitter-3'].find({}, {"text": 1})
to select all texts in collection.
How can we use mgo to find specific field in golang?
I tried
var result []string
err = conn.Find(bson.M{}, bson.M{"text", 1}).All(&result)
But it is not correct.
Use the query Select method to specify the fields to return:
var result []struct{ Text string `bson:"text"` }
err := c.Find(nil).Select(bson.M{"text": 1}).All(&result)
if err != nil {
// handle error
}
for _, v := range result {
fmt.Println(v.Text)
}
In this example, I declared an anonymous type with the one selected field. It's OK to use a type with all document fields.
to select multiple fields:
var result []struct{
Text string `bson:"text"`
Otherfield string `bson:"otherfield"`
}
err := c.Find(nil).Select(bson.M{"text": 1, "otherfield": 1}).All(&result)
if err != nil {
// handle error
}
for _, v := range result {
fmt.Println(v.Text)
}
var result interface{}
err = c.Find(nil).Select(bson.M{"text": 1}).All(&result)