I'm posting the following json string:
{'foods':[{'vName':'bean','color':'green','size':'small'},
{'vName':'carrot','color':'orange', 'size':'medium'}]}
I'm posting to Go using Restangular, and the receiving func is:
func CreateFoods(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
var food Food //this needs to be an array or something?
dec := json.NewDecoder(r.Body)
dec.Decode(&food)
}
My Food struct:
type Food struct{
VName string `json:"vName"`
Color string `json:"color"`
Size string `json:"size"`
}
I've used this routine for cases where where I post a single entity, but now I want to post multiple entities, and I cannot figure out how to map this json example to multiple entities.
Also, I'm trying to "see" the JSON POST, to see the JSON string, then if I have to, I can work with the string to make the entities. I cannot figure out how to get a hold of the JSON string from http.Request.
Add this:
// You might use lowercase foods since it is maybe not something you want to export
type Foods struct {
Foods []Food
}
When decoding use this:
var foods Foods
dec.Decode(&foods)
To see the body of a response as a string:
bytes, err := ioutil.ReadAll(r.Body)
fmt.Println(string(bytes))
Small detail: After the last two lines you now read the body contents. You should then decode the json not using json.NewDecoder and Decode but json.Unmarshal. Complete example for CreateFoods() to prevent confusion:
bytes, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println("error reading body")
return
}
fmt.Println(string(bytes))
var foods Foods
json.Unmarshal(bytes, &foods)
Hope it works, didn't test, let me know!
The proper way to handle decoding json if you have an io.Reader is to use json.NewDecoder, using ioutil.ReadAll then json.Unmarshal is slow and creates a lot of unneeded buffers.
To handle an array like your json example you need to create a struct like this:
type Foods struct {
Foods []Food `json:"foods"`
}
However if the json is just an array [{'vName':'bean','color':'green','size':'small'} {'vName':'carrot','color':'orange', 'size':'medium'}], you can just use:
type Foods []Food
type Food struct {
VName string `json:"vName"`
Color string `json:"color"`
Size string `json:"size"`
}
func CreateFoods(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
var food Foods //this needs to be an array or something?
dec := json.NewDecoder(r.Body)
dec.Decode(&foods)
}
Related
I have a mongoDb instance connected and my request (from my React app) gets a 200 back. I can see the collection in Mondodb as "cats" with a record for each cat but the cat records looks like:
_id:5f66641bad25efa40b810d83 - fine
name:"" - should not be blank
favorite:false -should be true
I had had some cors problems at first so I changed what was "Content-Type" to "content-type" (for Access-Control-Allow-Headers) and that "fixed" the cors issue. I have no idea if that is what is breaking the code though.
Since this is my first time setting up a database on my own I am just not sure where I have broken the code. Any help would be appreciated.
Here is my CreateCat method:
func CreateCat(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Context-Type", "application/json; charset=utf-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "POST")
w.Header().Set("Access-Control-Allow-Headers", "content-type")
var cat models.Cat
_ = json.NewDecoder(r.Body).Decode(&cat)
fmt.Println(cat)
insertOneFavoriteCat(cat)
json.NewEncoder(w).Encode(cat)
}
Here is my model:
type Cat struct {
ID primitive.ObjectID `json:"_id,omitempty" bson:"_id,omitempty"`
Name string `json:"name,string"`
Favorite bool `json:"favorite,string"`
}
This is what println prints: { false}
and if you are curious if it's my frontend, this is the request getting sent:
{"cat":{"name":"tabby","favorite":true}}
Problem is that JSON MongoDB returns doesn't match how your struct would be decoded. That false comes from Favorite field which cannot be found in JSON object with cat key returned by MongoDB. To decode this type of response you need to define a wrapping struct (as CatModel in my example below) because your Cat is not top level object but just a member key of upper level JSON object:
type CatModel struct {
Cat struct {
ID primitive.ObjectID `json:"_id,omitempty" bson:"_id,omitempty"`
Name string `json:"name"`
Favorite bool `json:"favorite"`
} `json:"cat"`
}
Also you have typo (Context-Type instead of Content-Type) in header name. Correct Header().Set() is:
w.Header().Set("Content-Type", "application/json; charset=utf-8")
TL;DR: Does the MongoDB driver provide a function to marshal and unmarshal a single field of a document?
This is a pretty straightforward question, but here's some context:
I have a worker responsible for synchronizing data between 2 separated databases. When it receives an event message, signalizing some document must sync, it selects the document in the primary database, and replicates it in another (it's a whole different database, not a replica set).
The thing is: I don't know the full structure of that document, so to preserve the data, I must unmarshal this document in a map map[string]interface{}, or a bson.M that works in the same fashion. But this seems like a lot of overhead, to unmarshal all this data I'm not even using, only to marshal it back to the other database.
So I thought about creating a structure that would just store the binary value of that document, without performing any marshal or unmarshal in order to reduce the overhead, like this:
type Document = map[string]Field
type Field struct {
Type bsontype.Type
Value []byte
}
func (f Field) MarshalBSONValue() (bsontype.Type, []byte, error) {
return f.Type, f.Value, nil
}
func (f *Field) UnmarshalBSONValue(btype bsontype.Type, value []byte) error {
f.Type = btype
f.Value = value
return nil
}
With this structure I can indeed reduce how much of the data will be parsed, but now, I need to manually unmarshal the one value in this document I'll need to use.
So I'm wondering if the MongoDB driver would have some function such as:
// Hypothetical function to get the value of a BSON
var status string
if err := decodeBSON(doc['status'].Type, doc['status'].Value, &status); err != nil {
return err
}
And
// Hypothetical function to set the value of a BSON
createdAt, err := encodeBSON(bsontype.Date, time.Now())
if err != nil {
return err
}
doc["createdAt"] = Field{Type: bsontype.Date, Value: createdAt}
How can I achieve this?
The Field type in your code is equivalent to the driver's bson.RawValue type. By switching to RawValue, you can decode individual fields using the RawValue.Unmarshal method and encode fields using bson.MarshalValue, which returns the two components (type and data) that you need to construct a new RawValue.
An example of how you can use these methods to change a field based on its original value: The Field type in your code is equivalent to the driver's bson.RawValue type. By switching to RawValue, you can decode individual fields using the RawValue.Unmarshal method and encode fields using bson.MarshalValue, which returns the two components (type and data) that you need to construct a new RawValue.
An example of how you can change a field depending on its original value without unmarshalling all of the original document's fields: https://gist.github.com/divjotarora/06c5188138456070cee26024f223b3ee
(sorry this question turned out longer than I had thought...)
I'm using Go and MongoDB with the mgo driver. I'm trying to persist and retrieve different structs (implementing a common interface) in the same MongoDB collection. I'm coming from the Java world (where this is very easily done with Spring with literally no config) and I'm having a hard time doing something similar with Go.
I have read every last related article or post or StackExchange question I could find, but still I haven't found a complete solution. This includes:
Unstructured MongoDB collections with mgo
How do you create a new instance of a struct from its type at run time in Go?
Golang reflect: Get Type representation from name?
Here's a simplified setup I use for testing. Suppose two structs S1 and S2, implementing a common interface I. S2 has an implicit field of type S1, meaning that structure-wise S2 embeds a S1 value, and that type-wise S2 implements I.
type I interface {
f1()
}
type S1 struct {
X int
}
type S2 struct {
S1
Y int
}
func (*S1) f1() {
fmt.Println("f1")
}
Now I can save an instance of S1 or S2 easily using mgo.Collection.Insert(), but to properly populate a value using mgo.Collection.Find().One() for example, I need to pass a pointer to an existing value of S1 or S2, which means I already know the type of the object I want to read!!
I want to be able to retrieve a document from the MongoDB collection without knowing if it's a S1, or a S2, or in fact any object that implements I.
Here's where I am so far: instead of directly saving the object I want to persist, I save a Wrapper struct that holds the MongoDB id, an identifier of the type, and the actual value. The type identifier is a concatenation of packageName + "." + typeName, and it is used to lookup the type in a type registry, since there is no native mechanism to map from a type name to a Type object in Go. This means I need to register the types that I want to be able to persist and retrieve, but I could live with that. Here's how it goes:
typeregistry.Register(reflect.TypeOf((*S1)(nil)).Elem())
typeregistry.Register(reflect.TypeOf((*S2)(nil)).Elem())
Here's the code for the type registry:
var types map[string]reflect.Type
func init() {
types = make(map[string]reflect.Type)
}
func Register(t reflect.Type) {
key := GetKey(t)
types[key] = t
}
func GetKey(t reflect.Type) string {
key := t.PkgPath() + "." + t.Name()
return key
}
func GetType(key string) reflect.Type {
t := types[key]
return t
}
The code for saving an object is quite straightforward:
func save(coll *mgo.Collection, s I) (bson.ObjectId, error) {
t := reflect.TypeOf(s)
wrapper := Wrapper{
Id: bson.NewObjectId(),
TypeKey: typeregistry.GetKey(t),
Val: s,
}
return wrapper.Id, coll.Insert(wrapper)
}
The code for retrieving an object is a bit more tricky:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// FIXME populate value using wrapper.Val (type bson.M)
// HOW ???
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
This partially works as the returned object is typed correctly, but what i can't figure out is how to populate the value pt with the data retrieved from MongoDB which is stored in wrapper.Val as a bson.M.
I have tried the following but it doesn't work:
m := wrapper.Val.(bson.M)
bsonBytes, _ := bson.Marshal(m)
bson.Unmarshal(bsonBytes, pt)
So basically the remaining problem is: how to populate an unknown structure from a bson.M value? I'm sure there has to be an easy solution...
Thanks in advance for any help.
Here's a Github gist with all the code: https://gist.github.com/ogerardin/5aa272f69563475ba9d7b3194b12ae57
First, you should always check returned errors, always. bson.Marshal() and bson.Unmarshal() return errors which you don't check. Doing so reveals why it doesn't work:
unmarshal can't deal with struct values. Use a pointer
pt is of type reflect.Value (which happens to be a struct), not something you should pass to bson.Unmarshal(). You should pass e.g. a pointer to a struct value you want to unmarshal into (which will be wrapped in an interface{} value). So call Value.Interface() on the value returned by reflect.New():
pt := reflect.New(t).Interface()
You can pass this to bson.Unmarshal():
bsonBytes, err := bson.Marshal(m)
if err != nil {
panic(err)
}
if err = bson.Unmarshal(bsonBytes, pt); err != nil {
panic(err)
}
(In your real code you want to do something else than panic, this is just to show you should always check errors!)
Also note that it is possible to directly convert maps to structs (directly meaning without marshaling and unmarshaling). You may implement it by hand or use a ready 3rd party lib. For details, see Converting map to struct
Also note that there are more clever ways to solve what you want to do. You could store the type in the ID itself, so if you have the ID, you can construct a value of the type to unmarshal into the query result, so you could skip this whole process. It would be a lot more simple and a lot more efficient.
For example you could use the following ID structure:
<type>-<id>
For example:
my.package.S1-123
When fetching / loading this document, you could use reflection to create a value of my.package.S1, and unmarshal into that directly (pass that to Query.One()).
As per #icza 's comments, here's a modified version of getById() that actually works:
func getById(coll *mgo.Collection, id interface{}) (*I, error) {
// read wrapper
wrapper := Wrapper{}
err := coll.Find(bson.M{"_id": id}).One(&wrapper)
if err != nil {
return nil, err
}
// obtain Type from registry
t := typeregistry.GetType(wrapper.TypeKey)
// get a pointer to a new value of this type
pt := reflect.New(t)
// populate value using wrapper.Val
err = mapstructure.Decode(wrapper.V, pt.Interface())
if err != nil {
return nil, err
}
// return the value as *I
i := pt.Elem().Interface().(I)
return &i, nil
}
Conversion from bson.M to the struct is handled by https://github.com/mitchellh/mapstructure instead of marshalling-unmarshaling.
I tried to retrieve a document from my collection with unique id.
I have a collection with fields: name, age, city, and rank. I want to get 'city' results from mongodb using golang.
My struct code
type exp struct {
name string `bson:"name"`
age int `bson:"age"`
city string `bson:"city"`
rank int `bson:"rank"`
}
With the following code to retrieve results from mongodb:
var result []exp //my struct type
err = coll.Find(bson.M{"City":bson.M{}}).Distinct("City",&result)
fmt.Println(result)
With this code I get an empty array as the result. How would I get all the cities?
Try this code
var result []string
err = c.Find(nil).Distinct("city", &result)
if err != nil {
log.Fatal(err)
}
fmt.Println(result)
Due to restrictions in reflection, mgo (as well as encoding/json and other similar packages) is unable to use unexported fields to marshal or unmarshal data. What you need to do is to export your fields by capitalize the first letter:
type exp struct {
Name string `bson:"name"`
Age int `bson:"age"`
City string `bson:"city"`
Rank int `bson:"rank"`
}
A side note: you do not need to specify the bson tags if the desired name is the same as the lowercase field name. The documentation for bson states:
The lowercased field name is used as the key for each exported field,
but this behavior may be changed using the respective field tag.
Edit:
I just realized you did get an empty slice and not a slice with empty struct fields. My answer is then not an actual answer to the question, but it is still an issue that you need to consider.
I have a function as below which decodes some json data and returns it as an interface
package search
func SearchItemsByUser(r *http.Request) interface{} {
type results struct {
Hits hits
NbHits int
NbPages int
HitsPerPage int
ProcessingTimeMS int
Query string
Params string
}
var Result results
er := json.Unmarshal(body, &Result)
if er != nil {
fmt.Println("error:", er)
}
return Result
}
I'm trying to access the data fields ( e.g. Params) but for some reasons it says that the interface has no such field. Any idea why ?
func test(w http.ResponseWriter, r *http.Request) {
result := search.SearchItemsByUser(r)
fmt.Fprintf(w, "%s", result.Params)
An interface variable can be used to store any value that conforms to the interface, and call methods that art part of that interface. Note that you won't be able to access fields on the underlying value through an interface variable.
In this case, your SearchItemsByUser method returns an interface{} value (i.e. the empty interface), which can hold any value but doesn't provide any direct access to that value. You can extract the dynamic value held by the interface variable through a type assertion, like so:
dynamic_value := interface_variable.(typename)
Except that in this case, the type of the dynamic value is private to your SearchItemsByUser method. I would suggest making two changes to your code:
Define your results type at the top level, rather than within the method body.
Make SearchItemsByUser directly return a value of the results type instead of interface{}.