I am getting the following error while deleting from key from array of JSON objects using Go.
Error:
repository/orderRepository.go:394:11: first argument to delete must be map; have interface {}
repository/orderRepository.go:395:11: first argument to delete must be map; have interface {}
repository/orderRepository.go:396:11: first argument to delete must be map; have interface {}
repository/orderRepository.go:397:11: first argument to delete must be map; have interface {}
repository/orderRepository.go:398:11: first argument to delete must be map; have interface {}
I am explaining my code below.
func SyncOrders() map[string]interface{} {
logger.Log.Println("OrderRepository SyncOrders Begin")
resourceManager := resources.ResourceManager{}
session, error := driver.Connect()
db := session.DB(config.Configuration.Database)
var resp map[string]interface{}
if error != nil {
resp := utils.Message(resourceManager.GetProperty(constants.ERROR), resourceManager.GetProperty(constants.DB_SERVER_NOT_REACHABLE_CODE), resourceManager.GetProperty(constants.DB_SERVER_NOT_REACHABLE_DESC))
return resp
} else {
var result []interface{}
//filter := bson.M{"Customer.CustomerID": id, "PaymentDetails.PaymentStatus": "Payment Received"}
//fmt.Println(filter)
err := db.C(ORDERCOLLECTION).Find(nil).All(&result)
if err == nil {
resp = utils.Message(resourceManager.GetProperty(constants.SUCCESS), resourceManager.GetProperty(constants.PRODUCT_GETBYID_CODE), resourceManager.GetProperty(constants.PRODUCT_GETBYID_DESC))
for i := 1; i < len(result); i++ {
delete(result[i],"_id");
delete(result[i],"CreatedAt");
delete(result[i],"CreatedBy");
delete(result[i],"UpdatedAt");
delete(result[i],"UpdatedBy");
}
resp["data"] = result
} else {
//fmt.Println(err)
resp = utils.Message(resourceManager.GetProperty(constants.ERROR), resourceManager.GetProperty(constants.PRODUCT_GETBYID_NOTFOUND_CODE), resourceManager.GetProperty(constants.PRODUCT_GETBYID_NOTFOUND_DESC))
}
defer session.Close()
return resp
}
}
Here I am fetching some record from MongoDB and delete some key value from each record but when I am running the server I am getting these errors. As I am beginner to Go. Can anybody help me to resolve these errors?
The error message says it all: the first argument to the builtin delete() must be a value of static type map.
Your result variable is of type []interface{}, so indexing it like result[i] will result in a value of type interface{}.
If it holds a map, you may use type assertion to obtain the map value from it. Since you use the mgo driver, it is of type bson.M (which is a map[string]interface{}), so you may do it like this:
delete(result[i].(bson.M), "_id")
But it would be better if you would declare result to be a slice of maps in the first place:
var result []bson.M
So then no type assertion will be needed, and the following will be valid code:
delete(result[i], "_id")
Also note that if you want to remove these properties from the results, it would be best if you would tell MongoDB you don't need these fields and so the server wouldn't even send these (saving network traffic) and then you wouldn't have to remove them (saving time and memory).
Use projection to tell you don't need these fields. In mgo you can set a projection using the Query.Select() method.
For example:
err := db.C(ORDERCOLLECTION).Find(nil).Select(bson.M{
"_id": 0,
"CreatedAt": 0,
"CreatedBy": 0,
"UpdatedAt": 0,
"UpdatedBy": 0,
}).All(&result)
The above query will result in documents where the listed fields will not be present, so you don't have to manually remove them using delete().
You are trying to delete key from variable of type interface{}.
#icza gives a good solution.
You can use .Select() to select which fields should be retrieved for the results.
Then you don't need to delete those fields from every object.
For example, the following query would only retrieve the name and age field:
err := db.C(ORDERCOLLECTION).Find(nil).Select(bson.M{"name": 1, "age": 1}).All(&result)
Related
I am writing a web app in go and using the GORM for my ORM. I need to be able to retrieve all the metrics of a certain user and return it via JSON to be displayed on the front end. The query seems to run successfully but I only see a memory address when printing the results and receive an error when trying to cast the results the standard way.
Here is my current code
func DisplayData(w http.ResponseWriter, r *http.Request) {
//Get the data from the database
var metric models.Metric
results := db.Where("user_id = ?", "1").Find(&metric)
//Write a json response
w.WriteHeader(http.StatusCreated)
w.Header().Set("Content-Type", "application/json")
resp := make(map[string]string)
resp["message"] = results
jsonResp, err := json.Marshal(resp)
if err != nil {
log.Fatalf("Error happened in JSON marshal. Err: %s", err)
}
w.Write(jsonResp)
return
}
This results in the error
controllers/statsCont.go:116:18: cannot use results (type *gorm.DB) as type string in assignment
note: module requires Go 1.17
When I try to cast by surrounding result in string() it gives the following error.
controllers/statsCont.go:116:26: cannot convert results (type *gorm.DB) to type string
note: module requires Go 1.17
As stated by #BaytaDarell the query result is added in the variable passed inside Find method
The return value of Find is different in the context it is called in case when it is called with db type the return type is (tx *DB) and when called with associations type the return type is Error
To solve the issue remove below lines
resp := make(map[string]string)
resp["message"] = results
And update it has
resp := map[string]interface{}{"message": metric}
I would like to know if there's any approach that would allow me to ignore null types while unmarshalling a MongoDB document into a Go struct.
Right now I have some auto-generate Go structs, something like this:
type User struct {
Name string `bson:"name"`
Email string `bson:"email"`
}
Changing the types declared in this struct is not an option, and here's the problem; in a MongoDB database, which I do not have total control, some of the documents have been inserted with null values were originally I was not expecting nulls. Something like this:
{
"name": "John Doe",
"email": null
}
As the string types declared inside my struct are not pointers, they can't receive a nil value, so whenever I try to unmarshall this document in my struct, it returns an error.
Preventing the insertion of this kind of document into the database would be the ideal solution, but for my use case, ignoring the null values would also be acceptable. So after unmarshalling the document my User instance would look like this
User {
Name: "John Doe",
Email: "",
}
I'm trying to find, either some annotation flag, or an option that could be passed to the method Find/FindOne, or maybe even a query parameter to prevent returning any field containing null values from the database. Without any success until now.
Are there any built-in solutions in the mongo-go-driver for this problem?
The problem is that the current bson codecs do not support encoding / decoding string into / from null.
One way to handle this is to create a custom decoder for string type in which we handle null values: we just use the empty string (and more importantly don't report error).
Custom decoders are described by the type bsoncodec.ValueDecoder. They can be registered at a bsoncodec.Registry, using a bsoncodec.RegistryBuilder for example.
Registries can be set / applied at multiple levels, even to a whole mongo.Client, or to a mongo.Database or just to a mongo.Collection, when acquiring them, as part of their options, e.g. options.ClientOptions.SetRegistry().
First let's see how we can do this for string, and next we'll see how to improve / generalize the solution to any type.
1. Handling null strings
First things first, let's create a custom string decoder that can turn a null into a(n empty) string:
import (
"go.mongodb.org/mongo-driver/bson/bsoncodec"
"go.mongodb.org/mongo-driver/bson/bsonrw"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type nullawareStrDecoder struct{}
func (nullawareStrDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if !val.CanSet() || val.Kind() != reflect.String {
return errors.New("bad type or not settable")
}
var str string
var err error
switch vr.Type() {
case bsontype.String:
if str, err = vr.ReadString(); err != nil {
return err
}
case bsontype.Null: // THIS IS THE MISSING PIECE TO HANDLE NULL!
if err = vr.ReadNull(); err != nil {
return err
}
default:
return fmt.Errorf("cannot decode %v into a string type", vr.Type())
}
val.SetString(str)
return nil
}
OK, and now let's see how to utilize this custom string decoder to a mongo.Client:
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(
bson.NewRegistryBuilder().
RegisterDecoder(reflect.TypeOf(""), nullawareStrDecoder{}).
Build(),
)
client, err := mongo.Connect(ctx, clientOpts)
From now on, using this client, whenever you decode results into string values, this registered nullawareStrDecoder decoder will be called to handle the conversion, which accepts bson null values and sets the Go empty string "".
But we can do better... Read on...
2. Handling null values of any type: "type-neutral" null-aware decoder
One way would be to create a separate, custom decoder and register it for each type we wish to handle. That seems to be a lot of work.
What we may (and should) do instead is create a single, "type-neutral" custom decoder which handles just nulls, and if the BSON value is not null, should call the default decoder to handle the non-null value.
This is surprisingly simple:
type nullawareDecoder struct {
defDecoder bsoncodec.ValueDecoder
zeroValue reflect.Value
}
func (d *nullawareDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if vr.Type() != bsontype.Null {
return d.defDecoder.DecodeValue(dctx, vr, val)
}
if !val.CanSet() {
return errors.New("value not settable")
}
if err := vr.ReadNull(); err != nil {
return err
}
// Set the zero value of val's type:
val.Set(d.zeroValue)
return nil
}
We just have to figure out what to use for nullawareDecoder.defDecoder. For this we may use the default registry: bson.DefaultRegistry, we may lookup the default decoder for individual types. Cool.
So what we do now is register a value of our nullawareDecoder for all types we want to handle nulls for. It's not that hard. We just list the types (or values of those types) we want this for, and we can take care of all with a simple loop:
customValues := []interface{}{
"", // string
int(0), // int
int32(0), // int32
}
rb := bson.NewRegistryBuilder()
for _, v := range customValues {
t := reflect.TypeOf(v)
defDecoder, err := bson.DefaultRegistry.LookupDecoder(t)
if err != nil {
panic(err)
}
rb.RegisterDecoder(t, &nullawareDecoder{defDecoder, reflect.Zero(t)})
}
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(rb.Build())
client, err := mongo.Connect(ctx, clientOpts)
In the example above I registered null-aware decoders for string, int and int32, but you may extend this list to your liking, just add values of the desired types to the customValues slice above.
You can go through the operator $exists and Query for Null or Missing Fields for a detail explanation.
In the mongo-go-driver, you can try below query:
The email => nil query matches documents that either contains the email field whose value is nil or that do not contain the email field.
cursor, err := coll.Find(
context.Background(),
bson.D{
{"email", nil},
})
You have to just add the $ne operator in the above query to get the records that do not have the field email or do not have the value nil in email. For more details about the operator $ne
If you know ahead of time which fields could potentially be null in your mongoDB records, you can use pointers in your structs instead:
type User struct {
Name string `bson:"name"` // Will still fail to decode if null in Mongo
Email *string `bson:"email"` // Will be nil in go if null in Mongo
}
Just remember that now you'll need to code more defensively around anything that uses this value after decoding from mongo, eg:
var reliableVal string
if User.Email != nil {
reliableVal = *user.Email
} else {
reliableVal = ""
}
I would like to know if there's any approach that would allow me to ignore null types while unmarshalling a MongoDB document into a Go struct.
Right now I have some auto-generate Go structs, something like this:
type User struct {
Name string `bson:"name"`
Email string `bson:"email"`
}
Changing the types declared in this struct is not an option, and here's the problem; in a MongoDB database, which I do not have total control, some of the documents have been inserted with null values were originally I was not expecting nulls. Something like this:
{
"name": "John Doe",
"email": null
}
As the string types declared inside my struct are not pointers, they can't receive a nil value, so whenever I try to unmarshall this document in my struct, it returns an error.
Preventing the insertion of this kind of document into the database would be the ideal solution, but for my use case, ignoring the null values would also be acceptable. So after unmarshalling the document my User instance would look like this
User {
Name: "John Doe",
Email: "",
}
I'm trying to find, either some annotation flag, or an option that could be passed to the method Find/FindOne, or maybe even a query parameter to prevent returning any field containing null values from the database. Without any success until now.
Are there any built-in solutions in the mongo-go-driver for this problem?
The problem is that the current bson codecs do not support encoding / decoding string into / from null.
One way to handle this is to create a custom decoder for string type in which we handle null values: we just use the empty string (and more importantly don't report error).
Custom decoders are described by the type bsoncodec.ValueDecoder. They can be registered at a bsoncodec.Registry, using a bsoncodec.RegistryBuilder for example.
Registries can be set / applied at multiple levels, even to a whole mongo.Client, or to a mongo.Database or just to a mongo.Collection, when acquiring them, as part of their options, e.g. options.ClientOptions.SetRegistry().
First let's see how we can do this for string, and next we'll see how to improve / generalize the solution to any type.
1. Handling null strings
First things first, let's create a custom string decoder that can turn a null into a(n empty) string:
import (
"go.mongodb.org/mongo-driver/bson/bsoncodec"
"go.mongodb.org/mongo-driver/bson/bsonrw"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type nullawareStrDecoder struct{}
func (nullawareStrDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if !val.CanSet() || val.Kind() != reflect.String {
return errors.New("bad type or not settable")
}
var str string
var err error
switch vr.Type() {
case bsontype.String:
if str, err = vr.ReadString(); err != nil {
return err
}
case bsontype.Null: // THIS IS THE MISSING PIECE TO HANDLE NULL!
if err = vr.ReadNull(); err != nil {
return err
}
default:
return fmt.Errorf("cannot decode %v into a string type", vr.Type())
}
val.SetString(str)
return nil
}
OK, and now let's see how to utilize this custom string decoder to a mongo.Client:
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(
bson.NewRegistryBuilder().
RegisterDecoder(reflect.TypeOf(""), nullawareStrDecoder{}).
Build(),
)
client, err := mongo.Connect(ctx, clientOpts)
From now on, using this client, whenever you decode results into string values, this registered nullawareStrDecoder decoder will be called to handle the conversion, which accepts bson null values and sets the Go empty string "".
But we can do better... Read on...
2. Handling null values of any type: "type-neutral" null-aware decoder
One way would be to create a separate, custom decoder and register it for each type we wish to handle. That seems to be a lot of work.
What we may (and should) do instead is create a single, "type-neutral" custom decoder which handles just nulls, and if the BSON value is not null, should call the default decoder to handle the non-null value.
This is surprisingly simple:
type nullawareDecoder struct {
defDecoder bsoncodec.ValueDecoder
zeroValue reflect.Value
}
func (d *nullawareDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if vr.Type() != bsontype.Null {
return d.defDecoder.DecodeValue(dctx, vr, val)
}
if !val.CanSet() {
return errors.New("value not settable")
}
if err := vr.ReadNull(); err != nil {
return err
}
// Set the zero value of val's type:
val.Set(d.zeroValue)
return nil
}
We just have to figure out what to use for nullawareDecoder.defDecoder. For this we may use the default registry: bson.DefaultRegistry, we may lookup the default decoder for individual types. Cool.
So what we do now is register a value of our nullawareDecoder for all types we want to handle nulls for. It's not that hard. We just list the types (or values of those types) we want this for, and we can take care of all with a simple loop:
customValues := []interface{}{
"", // string
int(0), // int
int32(0), // int32
}
rb := bson.NewRegistryBuilder()
for _, v := range customValues {
t := reflect.TypeOf(v)
defDecoder, err := bson.DefaultRegistry.LookupDecoder(t)
if err != nil {
panic(err)
}
rb.RegisterDecoder(t, &nullawareDecoder{defDecoder, reflect.Zero(t)})
}
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(rb.Build())
client, err := mongo.Connect(ctx, clientOpts)
In the example above I registered null-aware decoders for string, int and int32, but you may extend this list to your liking, just add values of the desired types to the customValues slice above.
You can go through the operator $exists and Query for Null or Missing Fields for a detail explanation.
In the mongo-go-driver, you can try below query:
The email => nil query matches documents that either contains the email field whose value is nil or that do not contain the email field.
cursor, err := coll.Find(
context.Background(),
bson.D{
{"email", nil},
})
You have to just add the $ne operator in the above query to get the records that do not have the field email or do not have the value nil in email. For more details about the operator $ne
If you know ahead of time which fields could potentially be null in your mongoDB records, you can use pointers in your structs instead:
type User struct {
Name string `bson:"name"` // Will still fail to decode if null in Mongo
Email *string `bson:"email"` // Will be nil in go if null in Mongo
}
Just remember that now you'll need to code more defensively around anything that uses this value after decoding from mongo, eg:
var reliableVal string
if User.Email != nil {
reliableVal = *user.Email
} else {
reliableVal = ""
}
Basically I want update in one value from the mongodb document by given fully struct interface as a change parameter in collection.Upsert(selector,change). how we do this without lose other values into empty. Other(type,category.rerportby,createon,info) values should be keep on existing values only update plant and location values into PLANT07 and BAR)
NOTE: I want use completely Service Notification Struct Object for
do this.
DatabaseName:WO
CollectionName:SERVICE_NOTIFICATIONS
package models
//models.ServiceNotification
type ServiceNotification struct {
NotificationNo string `json:"notification_no" bson:"notification_no"`
Type string `json:"type" bson:"type"`
Category string `json:"category" bson:"category"`
Plant string `json:"plant" bson:"plant"`
Location string `json:"location" bson:"location"`
ReportedBy string `json:"reportedby" bson:"reportedby"`
Info map[string]interface{}`json:"info" bson:"info"`
SAPInfo SAPNotificationInfo `json:"sapinfo" bson:"sapinfo"`
CreateOn string `json:"createon" bson:"createon"`
UpdateOn string `json:"updateon" bson:"updateon"`
}
package main
func main(){
input := models.ServiceNotification{
NotificationNo:000120,
Plant:"Plant07",
Location:"BAR",
}
Change_ServiceNotification(input)
}
I want update plant and location by given complete struct interface to the mongo Upsert function. because I want to decide dynamically what should
update . But when I update plant and location other values going
to be LOST. in mongo record.
func Change_ServiceNotification(notification models.ServiceNotification) error {
session, err := commons.GetMongoSession()
if err != nil {
return errors.New("Cannot create mongodb session" + err.Error())
}
defer session.Close()
var col = session.DB(WO).C(SERVICE_NOTIFICATIONS)
selector := bson.M{"notification_no": notification.NotificationNo}
_, err = col.Upsert(selector, notification)
if err != nil {
errMsg := "Cannot update service notification " + err.Error()
return errors.New(errMsg)
}
return nil
}
Appreciate your help
Thanks in advance
You cannot do it this way, but you can use the $set operator of MongoDB (Skipping error checking):
input := Bson.M{
"$set": bson.M{
"plant": "Plant07",
// Further fields here...
}
}
selector := bson.M{"notification_no": notification.NotificationNo}
col.Upsert(selector, input)
This will update only the provided fields.
I'm fairly new to both Go and MongoDB. Trying to select a single field from the DB and save it in an int slice without any avail.
userIDs := []int64{}
coll.Find(bson.M{"isdeleted": false}).Select(bson.M{"userid": 1}).All(&userIDs)
The above prints out an empty slice. However, if I create a struct with a single ID field that is int64 with marshalling then it works fine.
All I am trying to do is work with a simple slice containing IDs that I need instead of a struct with a single field. All help is appreciated.
Because mgo queries return documents, a few lines of code is required to accomplish the goal:
var result []struct{ UserID int64 `bson:"userid"` }
err := coll.Find(bson.M{"isdeleted": false}).Select(bson.M{"userid": 1}).All(&result)
if err != nil {
// handle error
}
userIDs := make([]int64, len(result))
for i := range result {
userIDs[i] = result.UserID
}