how to fetch the unknown mongo doc via mgo - mongodb

Here is the piece of code that is trying to fetch all the docs from the mongodb.
func fetchAll(db *mgo.Database) map[string]interface {
var msg map[string]interface{}
err := db.C("msg").Find(nil).All(&msg)
if err != nil {
panic(err)
}
return msg
}
I got the error: syntax error: unexpected var
What is wrong here? And is there a better way to fetch the arbitrary mongo docs via mgo?
thanks

First, fix the syntax error:
func fetchAll(db *mgo.Database) map[string]interface{} {
var msg map[string]interface{}
err := db.C("msg").Find(nil).All(&msg)
if err != nil {
panic(err)
}
return msg
}
Note the {} in the function return type declaration.
But there's more. All() retrieves all documents from the result set to a slice. Change the return type to a slice of maps:
func fetchAll(db *mgo.Database) []map[string]interface{} {
var msgs []map[string]interface{}
err := db.C("msg").Find(nil).All(&msgs)
if err != nil {
panic(err)
}
return msgs
}
While we are at it, let's return the error instead of panicking.
func fetchAll(db *mgo.Database) ([]map[string]interface{}, error) {
var msgs []map[string]interface{}
err := db.C("msg").Find(nil).All(&msgs)
return msgs, err
}

Related

Custom BSON marshal and unmarshal using mongo-driver

I have a struct field like this below. I also store raw protobuf of the same struct in db. Now every time fetch or save data to mongo. I have to update ReallyBigRaw, from the proto when I want to save to DB and when fetch I have to unmarshal ReallyBigRaw to ReallyBigObj to give out responses. Is there a way I can implement some interface or provide some callback functions so that the mongo driver does this automatically before saving or fetching data from DB.
Also, I am using the offical golang mongo driver not mgo, I have read some answers where can be done in mgo golang library.
import (
"github.com/golang/protobuf/jsonpb"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/primitive"
proto "github.com/dinesh/api/go"
)
type ReallyBig struct {
ID string `bson:"_id,omitempty"`
DraftID string `bson:"draft_id,omitempty"`
// Marshaled ReallyBigObj proto to map[string]interface{} stored in DB
ReallyBigRaw map[string]interface{} `bson:"raw,omitempty"`
ReallyBigObj *proto.ReallyBig `bson:"-"`
CreatedAt primitive.DateTime `bson:"created_at,omitempty"`
UpdatedAt primitive.DateTime `bson:"updated_at,omitempty"`
}
func (r *ReallyBig) GetProto() (*proto.ReallyBig, error) {
if r.ReallyBigObj != nil {
return r.ReallyBigObj, nil
}
Obj, err := getProto(r.ReallyBigRaw)
if err != nil {
return nil, err
}
r.ReallyBigObj = Obj
return r.ReallyBigObj, nil
}
func getRaw(r *proto.ReallyBig) (map[string]interface{}, error) {
m := jsonpb.Marshaler{}
b := bytes.NewBuffer([]byte{})
// marshals proto to json format
err := m.Marshal(b, r)
if err != nil {
return nil, err
}
var raw map[string]interface{}
// unmarshal the raw data to an interface
err = json.Unmarshal(b.Bytes(), &raw)
if err != nil {
return nil, err
}
return raw, nil
}
func getProto(raw map[string]interface{}) (*proto.ReallyBig, error) {
b, err := json.Marshal(raw)
if err != nil {
return nil, err
}
u := jsonpb.Unmarshaler{}
var reallyBigProto proto.ReallyBig
err = u.Unmarshal(bytes.NewReader(b), &recipeProto)
if err != nil {
return nil, err
}
return &reallyBigProto, nil
}
I implemented the Marshaler and Unmarshaler interface. Since mongo driver calls MarshalBSON and UnmarshalBSON if the type implements Marshaler and Unmarshaler we also end up in infinite loop. To avoid that we create a Alias of the type. Alias in Golang inherit only the fields not the methods so we end up calling normal bson.Marshal and bson.Unmarshal
func (r *ReallyBig) MarshalBSON() ([]byte, error) {
type ReallyBigAlias ReallyBig
reallyBigRaw, err := getRaw(r.ReallyBigObj)
if err != nil {
return nil, err
}
r.ReallyBigRaw = reallyBigRaw
return bson.Marshal((*ReallyBigAlias)(r))
}
func (r *ReallyBig) UnmarshalBSON(data []byte) error {
type ReallyBigAlias ReallyBig
err := bson.Unmarshal(data, (*ReallyBigAlias)(r))
if err != nil {
return err
}
reallyBigProto, err := getProto(r.ReallyBigRaw)
if err != nil {
return err
}
r.ReallyBigObj = reallyBigProto
return nil
}

how to create a mongo db package

I'd like to build an infrastructure, which is a package for the project. So that other developers can import this package to perform CRUD operations on the DB.
But I've got an error during the test:
type Students struct {
Name string
Age int
}
type InsertOneResult struct {
InsertedID interface{}
}
func dbGetOne(coll, document interface{}) (*InsertOneResult, error) {
...
}
func dbUpdateOne(coll, document interface{}) (*InsertOneResult, error) {
...
}
func dbDeleteOne(coll, document interface{}) (*InsertOneResult, error) {
...
}
func dbInsertOne(coll, document interface{}) (*InsertOneResult, error) {
res, err := coll.InsertOne(context.TODO(), document)
if err != nil {
log.Fatal(err)
}
return &InsertOneResult{InsertedID: res[0]}, err
}
func main() {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://<user>:<password>#<host>:<port>/<dbname>"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
coll := client.Database("db").Collection("students")
data := Students{"Amy", 10}
res, err := dbInsertOne(coll, data)
if err != nil {
log.Fatal(err)
}
fmt.Printf("inserted document with ID %v\n", res.InsertedID)
}
Here's the error:
./main.go:24:18: coll.InsertOne undefined (type interface {} is interface with no methods)
Is there any way to solve this? Thanks in advance.
Hey it looks like the error could be coming from a type conversion issue. The solution would be to clearly define the type for coll as *mongo.Collection in the dbInsertOne() function. This allows the compiler at compile time to figure out the structure of the input instead of having to rely on an abstract interface.
func dbInsertOne(coll *mongo.Collection, document interface{}) (*InsertOneResult, error) {
res, err := coll.InsertOne(context.TODO(), document)
if err != nil {
log.Fatal(err)
}
return &InsertOneResult{InsertedID: res.InsertedID}, err
}
I would further suggest that the 2nd argument document should also be a typed known term if possible. e.g.
func dbInsertOne(coll *mongo.Collection, document Students)
Static typing will help quite a bit and clear up any confusion.

How to handle duplicate unique index error?

I'm using MongoDB. Code to add data to the collection:
type User struct {
Firstname string `json:"firstname" bson:"firstname"`
Lastname *string `json:"lastname,omitempty" bson:"lastname"`
Username string `json:"username" bson:"username"`
RegistrationDate primitive.DateTime `json:"registrationDate" bson:"registrationData"`
LastLogin primitive.DateTime `json:"lastLogin" bson:"lastLogin"`
}
var client *mongo.Client
func AddUser(response http.ResponseWriter, request *http.Request) {
collection := client.Database("hattip").Collection("user")
var user User
_ = json.NewDecoder(request.Body).Decode(&user)
insertResult, err := collection.InsertOne(context.TODO(), user)
if err != nil {
// here i need to get the kind of error.
fmt.Println("Error on inserting new user", err)
response.WriteHeader(http.StatusPreconditionFailed)
} else {
fmt.Println(insertResult.InsertedID)
response.WriteHeader(http.StatusCreated)
}
}
func main() {
client = GetClient()
err := client.Ping(context.Background(), readpref.Primary())
if err != nil {
log.Fatal("Couldn't connect to the database", err)
} else {
log.Println("Connected!")
}
router := mux.NewRouter()
router.HandleFunc("/person", AddUser).Methods("POST")
err = http.ListenAndServe("127.0.0.1:8080", router)
if err == nil {
fmt.Println("Server is listening...")
} else {
fmt.Println(err.Error())
}
}
func GetClient() *mongo.Client {
clientOptions := options.Client().ApplyURI("mongodb://127.0.0.1:27017")
client, err := mongo.NewClient(clientOptions)
if err != nil {
log.Fatal(err)
}
err = client.Connect(context.Background())
if err != nil {
log.Fatal(err)
}
return client
}
If I add a record with a username that already exists in the database, I get -
Error on inserting new user multiple write errors: [{write errors:
[{E11000 duplicate key error collection: hattip.user index:
username_unique dup key: { username: "dd" }}]}, {}]
in the line fmt.Println("Error on inserting new user", err) The record with the string dd in the username field is already there, and the username field is a unique index.
I want to be sure that the error is exact E11000 error (a repeating collection of key errors).
So far i compare err to whole error string that appears on duplication of a unique field, but it's completely wrong. If there is a way to get error code from err object, or there are other ways to solve this problem?
Also, i found mgo package, but to use it properly i have to learn it, rewrite current code and so on, but honestly, it looks good:
if mgo.IsDup(err) {
err = errors.New("Duplicate name exists")
}
According to the driver docs, InsertOne may return a WriteException, so you can check if the error is a WriteException, and if it is so, then check the WriteErrors in it. Each WriteError contains an error code.
if we, ok:=err.(WriteException); ok {
for _,e:=range we.WriteErrors {
// check e.Code
}
}
You can write an IsDup based on this.

How to get DecodeBytes() output without canonical extended JSON additions

I use DecodeBytes() function to get the data from mongoDB (as the struct of the data can varies) with mongo-driver for Go.
My problem is when one of the values is int/double (and not string).
In that case it add adds some stuff of canonical extended JSON, for example 3 to "$numberDouble": "3.0".
How can I remove those additions of the canonical extended JSON?
func (m *Mongoclient) Find(collection string, filter interface{}) string {
findResult := m.Db.Collection(collection).FindOne(m.Ctx, filter)
if findResult.Err() != nil {
fmt.Println(findResult.Err().Error())
return ""
}
db, err := findResult.DecodeBytes()
if err != nil {
fmt.Println(err.Error())
return ""
}
return db.String()
}
The solution was to use Decode function to bson.M and than json.Marshal it:
var document bson.M
err := findResult.Decode(&document)
if err != nil {
fmt.Println(err.Error())
return ""
}
resBytes, err := json.Marshal(document)
if err != nil {
fmt.Println(err)
return ""
}
return string(resBytes)

Golang Mongodb %!(EXTRA

I'm trying to marshal a struct into JSON and then insert it into my Mongo database, but keep on getting this error: %!(EXTRA main.Test={575590180 Me}). What am I doing wrong? I took this code exactly from another project I worked on which could insert documents without any problems.
package main
import (
"utils"
"hash/fnv"
"log"
"gopkg.in/mgo.v2"
"encoding/json"
)
type Test struct {
Id uint32
Name string
}
func ConnectDB() *mgo.Session {
session, err := mgo.Dial("localhost:27017")
if err != nil {
panic(err)
}
return session
}
func SaveMgoDoc(dbName string, collectionName string, file Test) bool {
session, err := mgo.Dial("localhost:27017")
if err != nil {
panic(err)
}
defer session.Close()
fileJson, err := json.Marshal(file)
if err != nil {
log.Printf("failed to marshal struct to json...\n", file)
return false
}
collection := session.DB(dbName).C(collectionName)
err = collection.Insert(&fileJson)
if err != nil {
log.Printf("failed to insert doc into database...\n", file)
return false
}
return true
}
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
func main() {
utils.SaveMgoDoc("mydb", "mydoc", Test{hash("Me"), "Me"})
}
Insert expects a pointer to a struct, not a json string. So, in this case, just use:
err = collection.Insert(&file)