I have a function for comparing two structs and making a bson document as input to mongodb updateOne()
Example struct format
type event struct {
...
Name string
StartTime int32
...
}
Diff function, please ignore that I have not checked for no difference yet.
func diffEvent(e event, u event) (bson.M, error) {
newValues := bson.M{}
if e.Name != u.Name {
newValues["name"] = u.Name
}
if e.StartTime != u.StartTime {
newValues["starttime"] = u.StartTime
}
...
return bson.M{"$set": newValues}, nil
}
Then I generated a test function like so:
func Test_diffEvent(t *testing.T) {
type args struct {
e event
u event
}
tests := []struct {
name string
args args
want bson.M
wantErr bool
}{
{
name: "update startime",
args: args{
e: event{StartTime: 1},
u: event{StartTime: 2},
},
want: bson.M{"$set": bson.M{"starttime": 2}},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := diffEvent(tt.args.e, tt.args.u)
if (err != nil) != tt.wantErr {
t.Errorf("diffEvent() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("diffEvent() = %v, want %v", got, tt.want)
}
})
}
}
This fails with a
--- FAIL: Test_diffEvent/update_startime (0.00s)
models_test.go:582: diffEvent() = map[$set:map[starttime:2]], want map[$set:map[starttime:2]]
For me this seem to be the same. I have played around with this and bool fields, string fields, enum fields, and fields as struct or fields as arrays of structs seems to work fine with deepequal, but it gives an error for int32 fields.
As a go beginner; what am I missing here? I would assume that if bool/string works then int32 would too.
This:
bson.M{"starttime": 2}
Sets the "starttime" key to the value of the literal 2. 2 is an untyped integer constant, and since no type is provided, its default type will be used which is int.
And 2 values stored in interface values are only equal if the dynamic value stored in them have identical type and value. So a value 2 with int type cannot be equal to a value 2 of type int32.
Use explicit type to tell you want to specify a value of int32 type:
bson.M{"starttime": int32(2)}
Related
As the title shows, I have a struct defined an uint64 field, but error returned when I set its value as math.MaxUint64.
This is my code:
type MyDoc struct {
Number uint64 `bson:"_id"`
Timestamp int64 `bson:"time"`
}
// I just want to know whether uint64 overflows or not.
func main() {
mydoc := &MyDoc{
Number: math.MaxUint64,
Timestamp: time.Now().UnixNano(),
}
v, err := bson.Marshal(mydoc)
if err != nil {
panic(err)
}
fmt.Println(v)
}
After executed, error is following:
panic: 18446744073709551615 overflows int64 [recovered]
panic: 18446744073709551615 overflows int64
Obviously, uint64 types of data are processed as int64 which is not I expect.
So, how to store an uint64 data but not overflows in MongoDB?? I can not use string type instead, because I need to compare the size of number so that sorts documents.
I am using MongoDB official Go Driver.
Thanks in advance!
You could use a custom encoder and decoder.
With the code you provided, you could do something like that:
// define a custom type so you can register custom encoder/decoder for it
type MyNumber uint64
type MyDoc struct {
Number MyNumber `bson:"_id"`
Timestamp int64 `bson:"time"`
}
// create a custom registry builder
rb := bsoncodec.NewRegistryBuilder()
// register default codecs and encoders/decoders
var primitiveCodecs bson.PrimitiveCodecs
bsoncodec.DefaultValueEncoders{}.RegisterDefaultEncoders(rb)
bsoncodec.DefaultValueDecoders{}.RegisterDefaultDecoders(rb)
primitiveCodecs.RegisterPrimitiveCodecs(rb)
// register custom encoder/decoder
myNumberType := reflect.TypeOf(MyNumber(0))
rb.RegisterTypeEncoder(
myNumberType,
bsoncodec.ValueEncoderFunc(func(_ bsoncodec.EncodeContext, vw bsonrw.ValueWriter, val reflect.Value) error {
if !val.IsValid() || val.Type() != myNumberType {
return bsoncodec.ValueEncoderError{
Name: "MyNumberEncodeValue",
Types: []reflect.Type{myNumberType},
Received: val,
}
}
// IMPORTANT STEP: cast uint64 to int64 so it can be stored in mongo
vw.WriteInt64(int64(val.Uint()))
return nil
}),
)
rb.RegisterTypeDecoder(
myNumberType,
bsoncodec.ValueDecoderFunc(func(_ bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
// IMPORTANT STEP: read sore value in mongo as int64
read, err := vr.ReadInt64()
if err != nil {
return err
}
// IMPORTANT STEP: cast back to uint64
val.SetUint(uint64(read))
return nil
}),
)
// build the registry
reg := rb.Build()
To use this custom registry, you can pass it as option when creating your collection
collection := client.Database("testing").Collection("myCollection", &options.CollectionOptions{
Registry: reg,
})
I am using PostgreSQL as a database for my service, with a go backend.
After querying the database, I can access the rows with rows.Next() and rows.Scan(XX) :
func All(receptacle ...interface{}) (error) {
[...]
for rows.Next() {
err = rows.Scan(receptacle...)
if err != nil {
tx.Rollback()
return nil, err
}
}
rows.Close()
[...]
}
func main() {
var myInt1 int
var myInt2 int
var myString string
All(&myInt1, &myInt2, &myString)
//all 3 variables are correctly set
}
Here, receptacle is variadic.
If my query returns two int and one string, instead of doing rows.Scan(&myInt1, &myInt2, &myString), I am using the function's arguments rows.Scan(args...).
This is perfectly working when one, and only one, a row is requested from the BD.
What I would like to do is to use the same process to store multiple rows :
func main() {
var myInts1 []int
var myInts2 []int
var myStrings []string
All(&myInts1, &myInts2, &myStrings)
//myInts1 == []int{firstRowInt1, secondRowInt1, ...}
//myInts2 == []int{firstRowInt2, secondRowInt2, ...}
//myStrings == []string{firstRowString, secondRowString, ...}
}
Is there a way to tell a function to use the index X of a variadic argument, something like this: rows.Scan(receptacle[0]...) ?
My guess (and tries) is no, but wanted to confirm this hypothesis
I have a struct referencing a *big.Int. When storing this struct naively into MongoDB (using the official driver) the field turns to be nil when fetching the struct back. What is the proper/best way to store a big.Int into MongoDB?
type MyStruct struct {
Number *big.Int
}
nb := MyStruct{Number: big.NewInt(42)}
r, _ := db.Collection("test").InsertOne(context.TODO(), nb)
result := &MyStruct{}
db.Collection("test").FindOne(context.TODO(), bson.D{{"_id", r.InsertedID}}).Decode(result)
fmt.Println(result) // <== Number will be 0 here
My best idea so far would be to create a wrapper around big.Int that implements MarshalBSON and UnmarshalBSON (which I am not even sure how to do properly to be honest). But that'd be quite inconvenient.
Here's a possible implementation I came up with that stores the big.Int as plain text into MongoDb. It is also possible to easily store as byte array by using methods Bytes and SetBytes of big.Int instead of MarshalText/UnmarshalText.
package common
import (
"fmt"
"math/big"
"go.mongodb.org/mongo-driver/bson"
)
type BigInt struct {
i *big.Int
}
func NewBigInt(bigint *big.Int) *BigInt {
return &BigInt{i: bigint}
}
func (bi *BigInt) Int() *big.Int {
return bi.i
}
func (bi *BigInt) MarshalBSON() ([]byte, error) {
txt, err := bi.i.MarshalText()
if err != nil {
return nil, err
}
a, err := bson.Marshal(map[string]string{"i": string(txt)})
return a, err
}
func (bi *BigInt) UnmarshalBSON(data []byte) error {
var d bson.D
err := bson.Unmarshal(data, &d)
if err != nil {
return err
}
if v, ok := d.Map()["i"]; ok {
bi.i = big.NewInt(0)
return bi.i.UnmarshalText([]byte(v.(string)))
}
return fmt.Errorf("key 'i' missing")
}
the field turns to be nil when fetching the struct back
The reason why it returns 0, is because there is no bson mapping available for big.Int. If you check the document inserted into MongoDB collection you should see something similar to below:
{
"_id": ObjectId("..."),
"number": {}
}
Where there is no value stored in number field.
What is the proper/best way to store a big.Int into MongoDB?
Let's first understand what BigInt is. Big integer data type is intended for use when integer values might exceed the range that is supported by the int data type. The range is -2^63 (-9,223,372,036,854,775,808) to 2^63-1 (9,223,372,036,854,775,807) with storage size of 8 bytes. Generally this is used in SQL.
In Go, you can use a more precise integer types. Available built-in types int8, int16, int32, and int64 (and their unsigned counterparts) are best suited for data. The counterparts for big integer in Go is int64. With range -9,223,372,036,854,775,808 through 9,223,372,036,854,775,807, and storage size of 8 bytes.
Using mongo-go-driver,you can just use int64 which will be converted to bson RawValue.Int64. For example:
type MyStruct struct {
Number int64
}
collection := db.Collection("tests")
nb := MyStruct{Number: int64(42)}
r, _ := collection.InsertOne(context.TODO(), nb)
var result MyStruct
collection.FindOne(context.TODO(), bson.D{{"_id", r.InsertedID}}).Decode(&result)
This question already has answers here:
Type converting slices of interfaces
(9 answers)
Closed 3 years ago.
func GetFromDB(tableName string, m *bson.M) interface{} {
var (
__session *mgo.Session = getSession()
)
//if the query arg is nil. give it the null query
if m == nil {
m = &bson.M{}
}
__result := []interface{}{}
__cs_Group := __session.DB(T_dbName).C(tableName)
__cs_Group.Find(m).All(&__result)
return __result
}
call
GetFromDB(T_cs_GroupName, &bson.M{"Name": "Alex"}).([]CS_Group)
runtime will give me panic:
panic: interface conversion: interface is []interface {}, not []mydbs.CS_Group
how convert the return value to my struct?
You can't automatically convert between a slice of two different types – that includes []interface{} to []CS_Group. In every case, you need to convert each element individually:
s := GetFromDB(T_cs_GroupName, &bson.M{"Name": "Alex"}).([]interface{})
g := make([]CS_Group, 0, len(s))
for _, i := range s {
g = append(g, i.(CS_Group))
}
You need to convert the entire hierarchy of objects:
rawResult := GetFromDB(T_cs_GroupName, &bson.M{"Name": "Alex"}).([]interface{})
var result []CS_Group
for _, m := range rawResult {
result = append(result,
CS_Group{
SomeField: m["somefield"].(typeOfSomeField),
AnotherField: m["anotherfield"].(typeOfAnotherField),
})
}
This code is for the simple case where the type returned from mgo matches the type of your struct fields. You may need to sprinkle in some type conversions and type switches on the bson.M value types.
An alternate approach is to take advantage of mgo's decoder by passing the output slice as an argument:
func GetFromDB(tableName string, m *bson.M, result interface{}) error {
var (
__session *mgo.Session = getSession()
)
//if the query arg is nil. give it the null query
if m == nil {
m = &bson.M{}
}
__result := []interface{}{}
__cs_Group := __session.DB(T_dbName).C(tableName)
return __cs_Group.Find(m).All(result)
}
With this change, you can fetch directly to your type:
var result []CS_Group
err := GetFromDB(T_cs_GroupName, bson.M{"Name": "Alex"}, &result)
See also: FAQ: Can I convert a []T to an []interface{}?
I wrote a function that would return a sorted slice of strings from a map[string]Foo. I'm curious what is the best way to create a generic routine that can return a sorted slice of strings from any type that is a map with strings as keys.
Is there a way to do it using an interface specification? For example, is there any way to do something like:
type MapWithStringKey interface {
<some code here>
}
To implement the interface above, a type would need strings as keys. I could then write a generic function that returns a sorted list of keys for fulfilling types.
This is my current best solution using the reflect module:
func SortedKeys(mapWithStringKey interface{}) []string {
keys := []string{}
typ := reflect.TypeOf(mapWithStringKey)
if typ.Kind() == reflect.Map && typ.Key().Kind() == reflect.String {
switch typ.Elem().Kind() {
case reflect.Int:
for key, _ := range mapWithStringKey.(map[string]int) {
keys = append(keys, key)
}
case reflect.String:
for key, _ := range mapWithStringKey.(map[string]string) {
keys = append(keys, key)
}
// ... add more cases as needed
default:
log.Fatalf("Error: SortedKeys() does not handle %s\n", typ)
}
sort.Strings(keys)
} else {
log.Fatalln("Error: parameter to SortedKeys() not map[string]...")
}
return keys
}
Click for Go Playground version
I'm forced to code type assertions for each supported type even though at compile time, we should know the exact type of the mapWithStringKey parameter.
You cannot make partial types. But you can define an interface which serves your purpose:
type SortableKeysValue interface {
// a function that returns the strings to be sorted
Keys() []string
}
func SortedKeys(s SortableKeysValue) []string {
keys := s.Keys()
sort.Strings(keys)
return keys
}
type MyMap map[string]string
func (s MyMap) Keys() []string {
keys := make([]string, 0, len(s))
for k, _ := range s {
keys = append(keys, k)
}
return keys
}
Try it here: http://play.golang.org/p/vKfri-h4Cp
Hope that helps (go-1.1):
package main
import (
"fmt"
"reflect"
)
var m = map[string]int{"a": 3, "b": 4}
func MapKeys(m interface{}) (keys []string) {
v := reflect.ValueOf(m)
for _, k := range v.MapKeys() {
keys = append(keys, k.Interface().(string))
}
return
}
func main() {
fmt.Printf("%#v\n", MapKeys(m))
}