My GoLang struct:
type myPojo struct {
ID bson.ObjectId `json:"id" bson:"_id,omitempty"`
Start time.Time `json:"start"`
}
POST API JSON input request:
{
"Start":ISODate("2013-10-01T00:00:00.000Z")
}
My code to convert input JSON request to Golang Struct:
func myPostApi(w http.ResponseWriter, r *http.Request, db mongoDB) {
w.Header().Set("Content-Type", "application/json")
decoder := json.NewDecoder(r.Body)
var inputObj myPojo
err := decoder.Decode(&inputObj)
if err != nil {
//This gets executed
log.Println("Error occurred converting POST input json to myPojo data.")
log.Println(err)
}
}
Above code is not able to convert, and it goes inside error if block and prints below, please help.
2018/02/25 22:12:44 Error occurred converting POST input json to myPojo data.
2018/02/25 22:12:44 invalid character 'I' looking for beginning of value
The
ISODate("2013...")
value is not valid JSON. That looks like a symbol or function call, neither of which are allowed in JSON. And there is no date type in JSON:
The "right" JSON date format
Related
I have a custom type for the purposes of formatting a time object
type MyTime time.Time
To use this with JSON I have implemented:
func (t *MyTime) UnmarshalJSON(b []byte) error
func (t MyTime) MarshalJSON() ([]byte, error)
These functions marshal the type to/from a formatted string and work fine.
I now need to read/write this field to/from mongoDb. I have implemented
func (t *MyTime) UnmarshalBSON(b []byte) error
Which lets me read my type from mongo with no trouble (mongo doc has same format as above). this implementation simply strips the size and trailing zero bytes from the BSON string and parses as for JSON.
So far so good
My issue is how I write this value in the same format. I started by implementing:
func (t MyTime) MarshalBSON() ([]byte, error)
with a reversal of the unmarshal above (inserting size and trailing zero) and verified that the byte slice was identical to that I read earlier but this generated an error.
Unlike the JSON version, the MarshalBSON implementation will not allow me to write a string. Debugging into the mgo library shows that it is expecting me to marshal a structure, not a string field.
After googling the problem, I tried implementing
func (t *MyTime) GetBSON() (output interface{}, err error)
But this is not called by the InsertIntoMongo code.
There are multiple ways of marshalling the object but this format is defined externally and I cannot change it.
How can I achieve this last piece of the puzzle?
Namely, make BSON marshal my custom type as a string value (the reverse of the unmarshal already implemented)
edit:
My implementation of MarshalBSON mirrored the JSON as a reverse of Unmarshal.
In JSON unmarshal is passed a string (as []byte), Marshal returns the same string.
In BSON, unmarshal is passed size (4 bytes = len(string) + 1) + string + \x00. Marshal returns the same stream (verified as identical)
the call
mongoClient.Database.Collection.InsertOne(ctx, data, nil)
Where data is a structure containing a number of instances of MyTime
error = go.mongodb.org/mongo-driver/mongo.CommandError
error.Code = 22
error.Name = InvalidBSON
error.Message = invalid bson type in element with field name '021-07-16T09:12:31.793395300' in object with _id: "..."
(The field name is the string value I tried to write with the first character (byte) missing)
It should be a string, not a field
With the Marshal Function missing, the parent element (MyTime) was written to mongo as an (empty) object where it should be a string.
Slighty old question, but in case its of help...
MarshalBSON() and UnmarshalBSON() are really for things that marshal/unmarshal to BSON document. Whereas, your type MyTime time.Time is probably better marshalled/unmarshalled as a BSON value - for which there are specific methods that need to be implemented, e.g.
func (mt *MyTime) UnmarshalBSONValue(btype bsontype.Type, data []byte) error {
if btype != bsontype.String {
return errors.New("cannot unmarshal non-string bson value to MyTime")
}
vr := bsonrw.NewBSONValueReader(btype, data)
dec, err := bson.NewDecoder(vr)
if err != nil {
return err
}
var str string
err = dec.Decode(&str)
if err != nil {
return err
}
dt, err := time.Parse(time.RFC3339, str)
if err != nil {
return err
}
*mt = MyTime(dt)
return nil
}
func (mt MyTime) MarshalBSONValue() (bsontype.Type, []byte, error) {
str := time.Time(mt).Format(time.RFC3339)
return bson.MarshalValue(str)
}
I am fairly new to golang programming and the mongodb interface.
I've got a dbase of records created by another application. I am trying to walk the dbase and examine specific fields of each record. I can decode the full records as bson, but I cannot get the specific values.
This struct defines the 3 fields I would like to extract:
type myDbaseRec struct {
aid string `bson:"pon-util-aid"`
ingressPct string `bson:"ingress-bucket-percent"`
egressPct string `bson:"egress-bucket-percent"`
}
Here is my code to iterate through the cursor after the collection.Find(ctx, queryFilter) and decode the results both as bson and as my struct:
var myResult myDbaseRec
var bsonMResult bson.M
var count int
for cursor.Next(ctx) {
err := cursor.Decode(&myResult)
if err != nil {
fmt.Println("cursor.Next() error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(myResult))
fmt.Printf("result: %+v\n", myResult)
}
err = cursor.Decode(&bsonMResult)
if err != nil {
fmt.Println("bson decode error:", err)
panic(err)
// If there are no cursor.Decode errors
} else {
fmt.Println("\nresult type:", reflect.TypeOf(bsonMResult))
fmt.Println("\nresult:", bsonMResult)
}
}
Here is an example of one iteration of the loop. The bson decode appears to work, but my struct is empty:
result type: internal.myDbaseRec
result: {aid: ingressPct: egressPct:}
result type: primitive.M
result: map[pon-util-aid:ROLT-1-MONTREAL/1/1/xp2 _id:ObjectID("5d70b4d1b3605301ef72228b")
admitted-assured-upstream-bw:0 admitted-excess-upstream-bw:0 admitted-fixed-upstream-bw:0
assured-upstream-bytes:0 available-excess-upstream-bw:0 available-fixed-upstream-bw:622080
app_counters_key_field:ROLT-1-MONTREAL/1/1/xp2 app_export_time:1567665626 downstream-octets:52639862633214
egress-bucket-bps:8940390198 egress-bucket-percent:91 egress-bucket-seconds:559
excess-upstream-bytes:0 fixed-upstream-bytes:0 ingress-bucket-bps:8253153852
ingress-bucket-percent:84 ingress-bucket-seconds:559 sample-time:0 upstream-octets:48549268162714]
I would have expected to get
result: {aid:"ROLT-1-MONTREAL/1/1/xp2" ingressPct:84 egressPct:91}
Any suggestion on how to properly find these 3 fields from each record?
=== UPDATE: The first comment below answered my question.
First, in Go only fields starting with a (Unicode) upper case letter are exported. See also Exported identifiers. The default decoder will try to decode only to the exported fields. So you should change the struct into:
type myDbaseRec struct {
Aid string `bson:"pon-util-aid"`
IngressPct int32 `bson:"ingress-bucket-percent"`
EgressPct int32 `bson:"egress-bucket-percent"`
}
Also notice that the struct above for IngressPct and EgressPct have type int32. This is because the value in the document is represented in numbers (int/double), and not string. You may change it to other number type accordingly i.e. int16, int64, etc.
I am looking at this example.
I would never coome up with solution like this,I would go for bson.raw.
type Movie struct {
ID bson.ObjectId `json:"id" bson:"_id,omitempty"`
Name string `json:"name" bson:"name"`
Year string `json:"year" bson:"year"`
Directors []string `json:"directors" bson:"directors"`
Writers []string `json:"writers" bson:"writers"`
BoxOffice BoxOffice `json:"boxOffice" bson:"boxOffice"`
}
GetMovie function reads data from MongoDB and returns JSON
func (db *DB) GetMovie(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
w.WriteHeader(http.StatusOK)
var movie Movie
err := db.collection.Find(bson.M{"_id": bson.ObjectIdHex(vars["id"])}).One(&movie)
if err != nil {
w.Write([]byte(err.Error()))
} else {
w.Header().Set("Content-Type", "application/json")
response, _ := json.Marshal(movie)
w.Write(response)
}
}
I do not understand how generic map bson.M was created. Why did the author used bson.ObjectIdHex(vars["id"]?
bson.M is a map under the hood:
type M map[string]interface{}
And this:
bson.M{"_id": bson.ObjectIdHex(vars["id"])}
Is a composite literal creating a value of type bson.M. It has a single pair where key is "_id" and the associated value is a bson.ObjectId returned by the function bson.ObjectIdHex().
The document ID to look up and return is most likely coming as a hexadecimal string in vars["id"], and bson.ObjectIdHex() converts (parses) this into an ObjectId.
Tips: to query a document by ID, easier is to use Collection.FindId, e.g.:
err := db.collection.FindId(bson.ObjectIdHex(vars["id"])).One(&movie)
Also to avoid a runtime panic in case an invalid ID is stored in vars["id"], you could use bson.IsObjectIdHex() to check it first. For details, see Prevent runtime panic in bson.ObjectIdHex.
Also, marshaling the result into a byte slice and then writing it to the response is inefficient, the response could be streamed to the output using json.Encoder. For details, see Ouput json to http.ResponseWriter with template.
I have created an API server using golang and ravel. In one of the POST methods, I need to read the body and decode it to a model before saving it. But it is failing to do so. This is the guide I used https://medium.com/#kyawmyintthein/revel-mgo-restful-generator-for-revel-web-framework-mongodb-86209de3977e
The expected behaviour is to create the user object in the mongoDB. But I am getting an error response. Something is going wrong while Decoding it to user struct.
Controller method:
func (c UserController) Create() revel.Result {
fmt.Print("Body: ")
fmt.Println(c.Request.Body)
var (
user models.User
err error
)
err = json.NewDecoder(c.Request.Body).Decode(&user)
if err != nil {
errResp := buildErrResponse(err, "403")
c.Response.Status = 403
return c.RenderJSON(errResp)
}
user, err = models.AddUser(user)
if err != nil {
errResp := buildErrResponse(err, "500")
c.Response.Status = 500
return c.RenderJSON(errResp)
}
c.Response.Status = 201
return c.RenderJSON(user)
}
User Model:
package models
import (
"gopkg.in/mgo.v2/bson"
"time"
"userAPI/app/models/mongodb"
)
type User struct {
ID bson.ObjectId `json:"id" bson:"_id"`
Name string `json:"name" bson:"name"`
Email string `json:"email" bson:"email"`
Phone string `json:"phone" bson:"phone"`
Username string `json:"username" bson:"username"`
CreatedAt time.Time `json:"created_at" bson:"created_at"`
UpdatedAt time.Time `json:"updated_at" bson:"updated_at"`
}
JSON body used in POST request
{
"name":"kanishka",
"email":"kanishka#gmail.com",
"phone":"91238901238",
"username":"k"
}
This is the response I am getting
{
"error_code": "403",
"error_message": "EOF"
}
The output of println(c.Request.Body) on top of the controller method shows
&{0xc4202ffa80 false true {0 0} true false false
0x12a1430}
This is my first attempt at golang. Kindly help me to proceed further.
After searching, I found this issue #1011. Quote to the response:
Revel automatically calls ParseForm, ParseMultipartForm or whatever function is needed for a request, so req.Body is parsed and all valuable information is stored to controller.Params.(Fixed, Route, Query, Form, Files).
It means that, before calling your controller's method, the request body already being read by revel, so when you try to read it again the result will be EOF. From the docs, instead of using json.Decoder and request body, try the following:
c.Params.BindJSON(&user)
I'm able to insert an entry into MongoDB using the golang driver gopkg.in/mgo.vs and gopkg.in/mgo.vs/bson but I'm not able to pull it out. In the mongo shell, if I do
db.Items.find({ date : 1428762411980 })
it shows me the entry that I just inserted with the Go code. However, if I try to do the following to fetch it in Go, it's telling me that the record isn't found
func fetch(w http.ResponseWriter, r *http.Request){
var result SomeStruct
date := r.FormValue("date")
err := Items.Find(bson.M{"date":date}).One(&result)
...code omitted...
}
func Items() *mgo.Collection {
return DB().C("Items")
}
func DB() *mgo.Database {
return DBSession().DB("mydb")
}
One thing I noticed was that, in the shell, the date is stored as a NumberLong
"date" : NumberLong("1428762411980")
I'm wondering if I have to do something with the date value that I receive from the form in the fetch function before using it to query the database?
Update
Before saving the data to the db, it comes in as a json string like this
"date":1428762411980
I then decode it into a struct
type blah struct{
Id bson.ObjectId `json:"id" bson:"_id"`
Date int64 `json:"date" bson: "date"`
And it gets saved like this (as shown in the shell)
"date" : NumberLong("1428762411980")
r.FormValue returns a string, but you need an int64. Use strconv.ParseInt. Then your query should work.
date, err := strconv.ParseInt(r.FormValue("date"), 10, 64)
// handle err
err = Items.Find(bson.M{"date":date}).One(&result)