In mongoose(node.js) I can define a model schema with a default Date.now like so:
...
type: Date,
default: Date.now
...
How do I achieve the same without having to insert the time.Time every time I create a document with mgo?
type User struct {
CreatedAt time.Time `json:"created_at" bson:"created_at"` // Make this field filled automatically with time.Now() every time a document of this `struct` is inserted
}
In Go you can't define default values for fields, they will always be the zero-value of their type when a new struct value is created (unless you use a composite literal where you can give a different value explicitly).
So one option would be to create a constructor-like function NewUser() which would set this field, and use always this function to create new users:
func NewUser() *User {
return &User{
CreatedAt: time.Now(),
}
}
Of course this can't be forced, and also this will hold the timestamp of the User struct value creation and not when it is saved.
Another, better approach is to use a custom marshaling logic.
You can write custom marshaling logic by implementing bson.Getter. GetBSON() is responsible to provide a value that will actually be saved. We want the same User instance to be saved, just its CreatedAt field set prior:
type User struct {
CreatedAt time.Time `json:"created_at" bson:"created_at"`
}
func (u *User) GetBSON() (interface{}, error) {
u.CreatedAt = time.Now()
type my *User
return my(u), nil
}
Note that a new my type is created and returned. The reason for this is to avoid stack overflow. Simply returning a value of type *User is bad, because it implements bson.Getter, so GetBSON() would get called endlessly. The new my type does not have this method, so endless "recursion" does not happen (the type keyword creates a new type, and it does not "inherit" methods of the underlying type).
Note that this solution will also overwrite (re-set) the CreatedAt field) even if you just want to re-save a User. So we should add a check whether the CreatedAt field is filled, and only set it if it's the zero value:
func (u *User) GetBSON() (interface{}, error) {
if u.CreatedAt.IsZero() {
u.CreatedAt = time.Now()
}
type my *User
return my(u), nil
}
Also see related / similar question: Accesing MongoDB from Go
Related
I have a slice of structs that I want to pass into a stored procedure to be used as an array of user defined types t but I can't figure out a way of doing this is in Go.
For example the structs in go:
type a struct {
ID int `db:"id"`
Name string `db:"name"`
Created time.Time `db:"created"`
IsNew bool `db:"is_new"`
}
And the create statement for the user defined type
CREATE TYPE custom_type AS
(
id int,
name varchar,
created timestamp,
is_new boolean
)
and then the stored procedure
create or replace procedure custom_procedure(
input custom_type[]
)
So far I have tried doing
func Save(records []a) error {
_, err := p.client.Exec("CALL custom_procedure($1)", pq.Array(records))
return err
}
but I just get an error "sql: converting argument $1 type: unsupported type a, a struct"
You'll have to implement the driver.Valuer interface on the a type and have the Value method return a postgres composite type literal of the instance of a.
You can read this documentation on how to properly construct composite row type values. Just keep in mind that, since you're using pq.Array, which will quote the output of the Value method, you yourself SHOULD NOT put quotes around the output and also you SHOULD NOT use the ROW keyword.
For example:
type a struct {
ID int `db:"id"`
Name string `db:"name"`
Created time.Time `db:"created"`
IsNew bool `db:"is_new"`
}
func (v a) Value() (driver.Value, error) {
s := fmt.Sprintf("(%d,%q,%s,%t)",
v.ID,
v.Name,
v.Created.Format("2006-01-02 15:04:05"),
v.IsNew,
)
return []byte(s), nil
}
TL;DR: Does the MongoDB driver provide a function to marshal and unmarshal a single field of a document?
This is a pretty straightforward question, but here's some context:
I have a worker responsible for synchronizing data between 2 separated databases. When it receives an event message, signalizing some document must sync, it selects the document in the primary database, and replicates it in another (it's a whole different database, not a replica set).
The thing is: I don't know the full structure of that document, so to preserve the data, I must unmarshal this document in a map map[string]interface{}, or a bson.M that works in the same fashion. But this seems like a lot of overhead, to unmarshal all this data I'm not even using, only to marshal it back to the other database.
So I thought about creating a structure that would just store the binary value of that document, without performing any marshal or unmarshal in order to reduce the overhead, like this:
type Document = map[string]Field
type Field struct {
Type bsontype.Type
Value []byte
}
func (f Field) MarshalBSONValue() (bsontype.Type, []byte, error) {
return f.Type, f.Value, nil
}
func (f *Field) UnmarshalBSONValue(btype bsontype.Type, value []byte) error {
f.Type = btype
f.Value = value
return nil
}
With this structure I can indeed reduce how much of the data will be parsed, but now, I need to manually unmarshal the one value in this document I'll need to use.
So I'm wondering if the MongoDB driver would have some function such as:
// Hypothetical function to get the value of a BSON
var status string
if err := decodeBSON(doc['status'].Type, doc['status'].Value, &status); err != nil {
return err
}
And
// Hypothetical function to set the value of a BSON
createdAt, err := encodeBSON(bsontype.Date, time.Now())
if err != nil {
return err
}
doc["createdAt"] = Field{Type: bsontype.Date, Value: createdAt}
How can I achieve this?
The Field type in your code is equivalent to the driver's bson.RawValue type. By switching to RawValue, you can decode individual fields using the RawValue.Unmarshal method and encode fields using bson.MarshalValue, which returns the two components (type and data) that you need to construct a new RawValue.
An example of how you can use these methods to change a field based on its original value: The Field type in your code is equivalent to the driver's bson.RawValue type. By switching to RawValue, you can decode individual fields using the RawValue.Unmarshal method and encode fields using bson.MarshalValue, which returns the two components (type and data) that you need to construct a new RawValue.
An example of how you can change a field depending on its original value without unmarshalling all of the original document's fields: https://gist.github.com/divjotarora/06c5188138456070cee26024f223b3ee
(sorry this question turned out longer than I had thought...)
I'm using Go and MongoDB with the mgo driver. I'm trying to persist and retrieve different structs (implementing a common interface) in the same MongoDB collection. I'm coming from the Java world (where this is very easily done with Spring with literally no config) and I'm having a hard time doing something similar with Go.
I have read every last related article or post or StackExchange question I could find, but still I haven't found a complete solution. This includes:
Unstructured MongoDB collections with mgo
How do you create a new instance of a struct from its type at run time in Go?
Golang reflect: Get Type representation from name?
Here's a simplified setup I use for testing. Suppose two structs S1 and S2, implementing a common interface I. S2 has an implicit field of type S1, meaning that structure-wise S2 embeds a S1 value, and that type-wise S2 implements I.
type I interface {
f1()
}
type S1 struct {
X int
}
type S2 struct {
S1
Y int
}
func (*S1) f1() {
fmt.Println("f1")
}
Now I can save an instance of S1 or S2 easily using mgo.Collection.Insert(), but to properly populate a value using mgo.Collection.Find().One() for example, I need to pass a pointer to an existing value of S1 or S2, which means I already know the type of the object I want to read!!
I want to be able to retrieve a document from the MongoDB collection without knowing if it's a S1, or a S2, or in fact any object that implements I.
Here's where I am so far: instead of directly saving the object I want to persist, I save a Wrapper struct that holds the MongoDB id, an identifier of the type, and the actual value. The type identifier is a concatenation of packageName + "." + typeName, and it is used to lookup the type in a type registry, since there is no native mechanism to map from a type name to a Type object in Go. This means I need to register the types that I want to be able to persist and retrieve, but I could live with that. Here's how it goes:
typeregistry.Register(reflect.TypeOf((*S1)(nil)).Elem())
typeregistry.Register(reflect.TypeOf((*S2)(nil)).Elem())
Here's the code for the type registry:
var types map[string]reflect.Type
func init() {
types = make(map[string]reflect.Type)
}
func Register(t reflect.Type) {
key := GetKey(t)
types[key] = t
}
func GetKey(t reflect.Type) string {
key := t.PkgPath() + "." + t.Name()
return key
}
func GetType(key string) reflect.Type {
t := types[key]
return t
}
The code for saving an object is quite straightforward:
func save(coll *mgo.Collection, s I) (bson.ObjectId, error) {
t := reflect.TypeOf(s)
wrapper := Wrapper{
Id: bson.NewObjectId(),
TypeKey: typeregistry.GetKey(t),
Val: s,
}
return wrapper.Id, coll.Insert(wrapper)
}
The code for retrieving an object is a bit more tricky:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// FIXME populate value using wrapper.Val (type bson.M)
// HOW ???
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
This partially works as the returned object is typed correctly, but what i can't figure out is how to populate the value pt with the data retrieved from MongoDB which is stored in wrapper.Val as a bson.M.
I have tried the following but it doesn't work:
m := wrapper.Val.(bson.M)
bsonBytes, _ := bson.Marshal(m)
bson.Unmarshal(bsonBytes, pt)
So basically the remaining problem is: how to populate an unknown structure from a bson.M value? I'm sure there has to be an easy solution...
Thanks in advance for any help.
Here's a Github gist with all the code: https://gist.github.com/ogerardin/5aa272f69563475ba9d7b3194b12ae57
First, you should always check returned errors, always. bson.Marshal() and bson.Unmarshal() return errors which you don't check. Doing so reveals why it doesn't work:
unmarshal can't deal with struct values. Use a pointer
pt is of type reflect.Value (which happens to be a struct), not something you should pass to bson.Unmarshal(). You should pass e.g. a pointer to a struct value you want to unmarshal into (which will be wrapped in an interface{} value). So call Value.Interface() on the value returned by reflect.New():
pt := reflect.New(t).Interface()
You can pass this to bson.Unmarshal():
bsonBytes, err := bson.Marshal(m)
if err != nil {
panic(err)
}
if err = bson.Unmarshal(bsonBytes, pt); err != nil {
panic(err)
}
(In your real code you want to do something else than panic, this is just to show you should always check errors!)
Also note that it is possible to directly convert maps to structs (directly meaning without marshaling and unmarshaling). You may implement it by hand or use a ready 3rd party lib. For details, see Converting map to struct
Also note that there are more clever ways to solve what you want to do. You could store the type in the ID itself, so if you have the ID, you can construct a value of the type to unmarshal into the query result, so you could skip this whole process. It would be a lot more simple and a lot more efficient.
For example you could use the following ID structure:
<type>-<id>
For example:
my.package.S1-123
When fetching / loading this document, you could use reflection to create a value of my.package.S1, and unmarshal into that directly (pass that to Query.One()).
As per #icza 's comments, here's a modified version of getById() that actually works:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// populate value using wrapper.Val
err = mapstructure.Decode(wrapper.V, pt.Interface())
if err != nil {
return nil, err
}
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
Conversion from bson.M to the struct is handled by https://github.com/mitchellh/mapstructure instead of marshalling-unmarshaling.
I'm trying to set some value for a record in my CloudKit database. I haven't problem to set any other kind of value on other fields like string or int but for Reference filedname(storyid) i get this error :
"invalid attempt to set value type STRING for field 'storyid' for type
'Storypage', defined to be: REFERENCE"
func saveToCloud(note: String)
{
let newpage = CKRecord (recordType: "Storypage")
newpage.setValue(note, forKey: "pagecontent")
newpage.setValue("UTENTE1", forKey:"username" )
newpage.setValue("84EC8E60-1467-6411-5CDC-7E85DDB51C89", forKey: "storyid")
database.save(newpage) { (record, error) in
guard record != nil else {return}
}
A reference field stores a CKReference object, not a String:
Linking to Another Record
To link records together and create a strong
relationship between them, create a new CKReference object, initialize
it with the owner record, and assign that reference object to a field
of the owned record. When you design the relationships among your own
records, make the owner the more important of two related records.
So you'll want to get the recordID of the record you're relating newpage to, then use that to create a reference object, which you can then use to set the storyid field.
I am working on a project that uses combination of Go and MongoDB. I am stuck at a place where I have a struct like:
type Booking struct {
// booking fields
Id int `json:"_id,omitempty" bson:"_id,omitempty"`
Uid int `json:"uid,omitempty" bson:"uid,omitempty"`
IndustryId int `json:"industry_id,omitempty" bson:"industry_id,omitempty"`
LocationId int `json:"location_id,omitempty" bson:"location_id,omitempty"`
BaseLocationId int `json:"base_location_id,omitempty" bson:"base_location_id,omitempty"`
}
In this struct, the field Id is of int type. But as we know MongoDB's default id is of bsonObject type. Some times, the system generates the default MongoDB id in Id field.
To overcome this I have modified the struct like this:
type Booking struct {
// booking fields
Id int `json:"_id,omitempty" bson:"_id,omitempty"`
BsonId bson.ObjectId `json:"bson_id" bson:"_id,omitempty"`
Uid int `json:"uid,omitempty" bson:"uid,omitempty"`
IndustryId int `json:"industry_id,omitempty" bson:"industry_id,omitempty"`
LocationId int `json:"location_id,omitempty" bson:"location_id,omitempty"`
BaseLocationId int `json:"base_location_id,omitempty" bson:"base_location_id,omitempty"`
}
In above struct, I have mapped the same _id field in two different struct fields Id (type int) and BsonId (type bson.ObjectId). I want if id of integer type comes, it maps under Id otherwise under BsonId.
But this thing is giving following error:
Duplicated key '_id' in struct models.Booking
How can I implement this type of thing with Go Structs ??
Update:
Here is the code I have written for custom marshaling/unmarshaling:
func (booking *Booking) SetBSON(raw bson.Raw) (err error) {
type bsonBooking Booking
if err = raw.Unmarshal((*bsonBooking)(booking)); err != nil {
return
}
booking.BsonId, err = booking.Id
return
}
func (booking *Booking) GetBSON() (interface{}, error) {
booking.Id = Booking.BsonId
type bsonBooking *Booking
return bsonBooking(booking), nil
}
But this is giving type mismatch error of Id and BsonId fields. What should I do now ?
In your original struct you used this tag for the Id field:
bson:"_id,omitempty"
This means if the value of the Id field is 0 (zero value for the int type), then that field will not be sent to MongoDB. But the _id property is mandatory in MongoDB, so in this case the MongoDB server will generate an ObjectId for it.
To overcome this, the easiest is to ensure the Id will is always non-zero; or if the 0 is a valid id, remove the omitempty option from the tag.
If your collection must allow ids of mixed type (int and ObjectId), then the easiest would be to define the Id field with type of interface{} so it can accomodate both (actually all) types of key values:
Id interface{} `json:"_id,omitempty" bson:"_id,omitempty"`
Yes, working with this may be a little more cumbersome (e.g. if you explicitly need the id as int, you need to use type assertion), but this will solve your issue.
If you do need 2 id fields, one with int type and another with ObjectId type, then your only option will be to implement custom BSON marshaling and unmarshaling. This basically means to implement the bson.Getter and / or bson.Setter interfaces (one method for each) on your struct type, in which you may do anything you like to populate your struct or assemble the data to be actually saved / inserted. For details and example, see Accessing MongoDB from Go.
Here's an example how using custom marshaling it may look like:
Leave out the Id and BsonId fields from marshaling (using the bson:"-" tag), and add a 3rd, "temporary" id field:
type Booking struct {
Id int `bson:"-"`
BsonId bson.ObjectId `bson:"-"`
TempId interface{} `bson:"_id"`
// rest of your fields...
}
So whatever id you have in your MongoDB, it will end up in TempId, and only this id field will be sent and saved in MongoDB's _id property.
Use the GetBSON() method to set TempId from the other id fields (whichever is set) before your struct value gets saved / inserted, and use SetBSON() method to "copy" TempId's value to one of the other id fields based on its dynamic type after the document is retrieved from MongoDB:
func (b *Booking) GetBSON() (interface{}, error) {
if b.Id != 0 {
b.TempId = b.Id
} else {
b.TempId = b.BsonId
}
return b, nil
}
func (b *Booking) SetBSON(raw bson.Raw) (err error) {
if err = raw.Unmarshal(b); err != nil {
return
}
if intId, ok := b.TempId.(int); ok {
b.Id = intId
} else bsonId, ok := b.TempId.(bson.ObjectId); ok {
b.BsonId = bsonId
} else {
err = errors.New("invalid or missing id")
}
return
}
Note: if you dislike the TempId field in your Booking struct, you may create a copy of Booking (e.g. tempBooking), and only add TempId into that, and use tempBooking for marshaling / unmarshaling. You may use embedding (tempBooking may embed Booking) so you can even avoid repetition.