My data structure is
{
_id: ObjectID,
...
fields: [
{ name: "Aryan" },
{ books : [ 1,2,3 ] },
]
}
In our application a user can define his own fields data structure but with a key value structure. So, we had no way of knowing the structure of the data.
So in our document struct we had
type Document struct {
Fields map[string]interface{}
}
as the second parameter returned by mongo was primitive.A ( []interface{} under the hood ), the individual item could have been an array, map, anything.
But we couldn't range over that for it being.
How can I get the individual values like the book ids [1,2,3] or the name value "Aryan" ?
After a couple of attempts to solving this, I was at a state where my current element it self is an interface{} ( [1,2,3] in this case ) and I couldn't get the individual 1,2,3s.
But finally managed to solve it
for k, val := range doc.Fields {
v := reflect.ValueOf(val)
switch reflect.TypeOf(t).Kind() {
case reflect.Slice:
// getting the individual ids
for i := 0; i < v.Len(); i++ {
fmt.Println(v.Index(i))
}
case reflect.String:
default:
// handle
}
}
Note: v.Index(i) returns a data type of Value .
to change the data type
v.Index(i).Interface().(string) // string
v.Index(i).Interface().(float64) // float
Ideally you should avoid working with interface{} type since it's very error prone and the compiler can't help you. The idiomatic way is to define a struct for your model with BSON tags like in this example`
type MyType struct {
ID primitive.ObjectID `bson:"_id,omitempty"`
Fields []Field `bson:"fields,omitempty"`
}
type Field struct {
Name string `bson:"name,omitempty"`
Books []int `bson:"books,omitempty"`
}
Field here is defined as the reunion of all possible fields which again is not ideal but at least the compiler can help you and developers know what to expect from the database document.
Related
I need to insert into a Postgresql database using gorm. But the struct in question has fields containing arrays of custom types:
type VideoFile struct {
UID string
FileName string
Format string
Path string
}
type myStruct struct {
ID string
UserId uint64
UserType int
FaceVideoFiles []VideoFile
MeetingVideoFiles []VideoFile
}
func (r *videoRepo) CreateRecord(input *myStruct) error {
tx := r.base.GetDB().Begin()
err := tx.Create(input).Error
if err != nil {
tx.Rollback()
return err
}
tx.Commit()
return nil
}
ID field is the primary key in database. I am not sure which tags to add to the struct fields so I leave it empty here (e.g. gorm:"primaryKey" json:"id", etc). If the FaceVideoFiles is not an array of struct, things seem to work, but that is not the case with an array of struct. What should I do so that I can insert those arrays of structs (not individual structs) into my database?
You have to use one to many. More information related to this. Please follow documentation. https://gorm.io/docs/has_many.html#Has-Many
type Dog struct {
ID int
Name string
Toys []Toy `gorm:"polymorphic:Owner;"`
}
type Toy struct {
ID int
Name string
OwnerID int
OwnerType string
}
db.Create(&Dog{Name: "dog1", Toys: []Toy{{Name: "toy1"}, {Name: "toy2"}}})
// INSERT INTO `dogs` (`name`) VALUES ("dog1")
// INSERT INTO `toys` (`name`,`owner_id`,`owner_type`) VALUES ("toy1","1","dogs"), ("toy2","1","dogs")
I am working on a project that uses combination of Go and MongoDB. I am stuck at a place where I have a struct like:
type Booking struct {
// booking fields
Id int `json:"_id,omitempty" bson:"_id,omitempty"`
Uid int `json:"uid,omitempty" bson:"uid,omitempty"`
IndustryId int `json:"industry_id,omitempty" bson:"industry_id,omitempty"`
LocationId int `json:"location_id,omitempty" bson:"location_id,omitempty"`
BaseLocationId int `json:"base_location_id,omitempty" bson:"base_location_id,omitempty"`
}
In this struct, the field Id is of int type. But as we know MongoDB's default id is of bsonObject type. Some times, the system generates the default MongoDB id in Id field.
To overcome this I have modified the struct like this:
type Booking struct {
// booking fields
Id int `json:"_id,omitempty" bson:"_id,omitempty"`
BsonId bson.ObjectId `json:"bson_id" bson:"_id,omitempty"`
Uid int `json:"uid,omitempty" bson:"uid,omitempty"`
IndustryId int `json:"industry_id,omitempty" bson:"industry_id,omitempty"`
LocationId int `json:"location_id,omitempty" bson:"location_id,omitempty"`
BaseLocationId int `json:"base_location_id,omitempty" bson:"base_location_id,omitempty"`
}
In above struct, I have mapped the same _id field in two different struct fields Id (type int) and BsonId (type bson.ObjectId). I want if id of integer type comes, it maps under Id otherwise under BsonId.
But this thing is giving following error:
Duplicated key '_id' in struct models.Booking
How can I implement this type of thing with Go Structs ??
Update:
Here is the code I have written for custom marshaling/unmarshaling:
func (booking *Booking) SetBSON(raw bson.Raw) (err error) {
type bsonBooking Booking
if err = raw.Unmarshal((*bsonBooking)(booking)); err != nil {
return
}
booking.BsonId, err = booking.Id
return
}
func (booking *Booking) GetBSON() (interface{}, error) {
booking.Id = Booking.BsonId
type bsonBooking *Booking
return bsonBooking(booking), nil
}
But this is giving type mismatch error of Id and BsonId fields. What should I do now ?
In your original struct you used this tag for the Id field:
bson:"_id,omitempty"
This means if the value of the Id field is 0 (zero value for the int type), then that field will not be sent to MongoDB. But the _id property is mandatory in MongoDB, so in this case the MongoDB server will generate an ObjectId for it.
To overcome this, the easiest is to ensure the Id will is always non-zero; or if the 0 is a valid id, remove the omitempty option from the tag.
If your collection must allow ids of mixed type (int and ObjectId), then the easiest would be to define the Id field with type of interface{} so it can accomodate both (actually all) types of key values:
Id interface{} `json:"_id,omitempty" bson:"_id,omitempty"`
Yes, working with this may be a little more cumbersome (e.g. if you explicitly need the id as int, you need to use type assertion), but this will solve your issue.
If you do need 2 id fields, one with int type and another with ObjectId type, then your only option will be to implement custom BSON marshaling and unmarshaling. This basically means to implement the bson.Getter and / or bson.Setter interfaces (one method for each) on your struct type, in which you may do anything you like to populate your struct or assemble the data to be actually saved / inserted. For details and example, see Accessing MongoDB from Go.
Here's an example how using custom marshaling it may look like:
Leave out the Id and BsonId fields from marshaling (using the bson:"-" tag), and add a 3rd, "temporary" id field:
type Booking struct {
Id int `bson:"-"`
BsonId bson.ObjectId `bson:"-"`
TempId interface{} `bson:"_id"`
// rest of your fields...
}
So whatever id you have in your MongoDB, it will end up in TempId, and only this id field will be sent and saved in MongoDB's _id property.
Use the GetBSON() method to set TempId from the other id fields (whichever is set) before your struct value gets saved / inserted, and use SetBSON() method to "copy" TempId's value to one of the other id fields based on its dynamic type after the document is retrieved from MongoDB:
func (b *Booking) GetBSON() (interface{}, error) {
if b.Id != 0 {
b.TempId = b.Id
} else {
b.TempId = b.BsonId
}
return b, nil
}
func (b *Booking) SetBSON(raw bson.Raw) (err error) {
if err = raw.Unmarshal(b); err != nil {
return
}
if intId, ok := b.TempId.(int); ok {
b.Id = intId
} else bsonId, ok := b.TempId.(bson.ObjectId); ok {
b.BsonId = bsonId
} else {
err = errors.New("invalid or missing id")
}
return
}
Note: if you dislike the TempId field in your Booking struct, you may create a copy of Booking (e.g. tempBooking), and only add TempId into that, and use tempBooking for marshaling / unmarshaling. You may use embedding (tempBooking may embed Booking) so you can even avoid repetition.
I'm using the MongoKitten library to fetch documents from mongoDB.
I have the following document in my mongoDB:
{
...
foo: ["A", "B"]
...
}
I can query the db, but I can't loop through the foo array of the returned documents. For instance, let's say that I save the results of my query into mongoDocs.
for Doc in mongoDocs {
print(Doc["foo"] as Any) // prints ["A", "B"]
var tempFoos = [String]()
for foo in Doc["foo"] { // gives error: Type 'Value' does not conform to protocol "Sequence"
print("Foo: " + foo)
tempFoos.append(foo)
}
}
I understand the error. Basically, my foo array doesn't conform to the Sequence protocol that allows me to loop over it. But how do I fix that?
Edit - Here's the code I'm using to fetch the mongoDocs. I've printed the results and used other properties from them. I just can't seem to iterate through this array.
mongoDocs = try self.geographiesCollection!.find(matching: q, projecting: projection, limitedTo: 100)
Here's the relevant function in the MongoKitten source code. The function returns Cursor<Document>
Here you can check out how a framework dev explained handling this situation. MongoKitten closed issue 27
here are some quotes from his explanation incase the link becomes invalid.
"MongoKitten BSON library always returns an enum (Value) when subscripting a document."
"A BSON array is really just a document with keys from 0 to x, so the enum case for array has a document as it's associated value. Because Value can also be, say, a double or a date, it doesn't conform to the Sequence protocol.
The easiest way to iterate over the array is by using the document convenience accessor on Value. This returns the underlying document if Value is either an array or document, or an empty document if it's something else. You can then iterate like this:"
for (key, val) in doc["vals"].document {
print("Value is \(val)")
}
Convert it into an array:
for Doc in mongoDocs {
guard let arr = Doc["foo"] as? [String] else { continue }
for foo in arr {
// do your things
}
}
I think you're using BSON 3. Value is an enum with multiple cases, which you can see here. Here's my stab at it:
for doc in mongoDocs {
guard case Value.array(let fooArray)? = doc["foo"] {
fatalError("doc[\"foo\"] is not an array")
}
print(fooArray)
}
I tried to retrieve a document from my collection with unique id.
I have a collection with fields: name, age, city, and rank. I want to get 'city' results from mongodb using golang.
My struct code
type exp struct {
name string `bson:"name"`
age int `bson:"age"`
city string `bson:"city"`
rank int `bson:"rank"`
}
With the following code to retrieve results from mongodb:
var result []exp //my struct type
err = coll.Find(bson.M{"City":bson.M{}}).Distinct("City",&result)
fmt.Println(result)
With this code I get an empty array as the result. How would I get all the cities?
Try this code
var result []string
err = c.Find(nil).Distinct("city", &result)
if err != nil {
log.Fatal(err)
}
fmt.Println(result)
Due to restrictions in reflection, mgo (as well as encoding/json and other similar packages) is unable to use unexported fields to marshal or unmarshal data. What you need to do is to export your fields by capitalize the first letter:
type exp struct {
Name string `bson:"name"`
Age int `bson:"age"`
City string `bson:"city"`
Rank int `bson:"rank"`
}
A side note: you do not need to specify the bson tags if the desired name is the same as the lowercase field name. The documentation for bson states:
The lowercased field name is used as the key for each exported field,
but this behavior may be changed using the respective field tag.
Edit:
I just realized you did get an empty slice and not a slice with empty struct fields. My answer is then not an actual answer to the question, but it is still an issue that you need to consider.
Using mgo I'm unable to store any meaningful data. Only the _id gets stored
type Person struct {
name string
age int
}
func main() {
session, err := mgo.Dial("localhost")
if err != nil {
log.Fatal(err)
}
defer session.Close()
p := Person{"Joe", 50}
ppl := session.DB("rest").C("people")
ppl.Insert(p)
}
The result in Mongo is just the _id field - no sign of "Joe".
Using go 1.1.2 on Arch linux, MongoDB 2.4.6.
type Person struct {
name string
age int
}
The mgo package can't access unexported (lowercase) fields of your struct (i.e. no other package than the one the struct is defined in can). You need to export them (first letter must be upper case), like this:
type Person struct {
Name string
Age int
}
If you wish to have the field names in lower case in the DB you must provide a struct tag for them, like this:
type Person struct {
Name string `bson:"name"`
Age int `bson:"age"`
}
See the documentation on names:
Names are as important in Go as in any other language. They even have
semantic effect: the visibility of a name outside a package is
determined by whether its first character is upper case. [...]
EDIT:
Gustavo Niemeyer (author of the mgo and bson packages) noted in the comments that unlike the json package, the bson marshaller will lowercase all struct field names when commiting to the database, effectively making the last step in this answer superfluous.