I was wondering if there is any way to have a stuct field that doesn't get commited to mgo even if it isn't empty.
The only way I have found to do this is to make the field lowercase, which makes it a pain to access. Is there another way?
This is an example, and my goal here is to not commit the SSN into the database but still have it uppercase.
package main
import (
"fmt"
"crypto/sha1"
"encoding/base64"
"labix.org/v2/mgo"
)
type Person struct{
Name string
SSN string
HashedSSN string
}
func main() {
bob := Person{"Bob", "fake_ssn", ""}
hasher := sha1.New()
hasher.Write( []byte(bob.SSN))
sha := base64.URLEncoding.EncodeToString(hasher.Sum(nil))
bob.HashedSSN = sha
mgoSession, err := mgo.Dial("localhost:27017")
if err != nil {
fmt.Println("mongo_config#initMongoSessions : Could not dial to mgoSession", err)
} else {
mgoSession.DB("test").C("person").Insert(bob)
}
}
Thanks,
You can do that by using the field tag as follows:
type T struct {
Field string `bson:"-"`
}
Related
I am creating simple REST API using MongoDB and golang as a driver.
I was able to create POST request which can be found here:
terminal output.
However, when creating GET request, i always need to get it by bson _id. Would someone be able to let me know how to retrieve from json id not bson _id from golang script. If this is not possible, I would appreciate if someone let me know how to convert id to _id.
models/user.go
package models
import (
"gopkg.in/mgo.v2/bson"
)
type User struct {
Id bson.ObjectId `json:"id" bson: "_id"`
Name string `json:"name" bson: "name"`
Gender string `json:"gender" bson: "gender"`
Age int `json:"age" bson: "age"`
}
controllers/user.go file
type UserController struct {
session *mgo.Session}
func httpResponse(w http.ResponseWriter, jsonOut []byte, code int) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
fmt.Fprintf(w, "%s", jsonOut)
}
func NewUserCOntroller(s *mgo.Session) *UserController {
// return the address of UserController
return &UserController{s}
}
func (uc UserController) GetUser(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
id := p.ByName("id")
if !bson.IsObjectIdHex(id) {
w.WriteHeader(http.StatusNotFound)
}
// oid is something you use in mongo
oid := bson.ObjectIdHex(id)
u := models.User{}
if err := uc.session.DB("mongolang").C("users").FindId(oid).One(&u); err != nil {
w.WriteHeader(404)
return
}
uj, err := json.Marshal(u)
if err != nil {
fmt.Println(err)
}
httpResponse(w, uj, http.StatusOK)
}
func (uc UserController) CreateUser(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// its empty for now
u := models.User{}
json.NewDecoder(r.Body).Decode(&u)
u.Id = bson.NewObjectId()
uc.session.DB("mongolang").C("users").Insert(u)
jsonOut, _ := json.Marshal(u)
httpResponse(w, jsonOut, http.StatusOK)
fmt.Println("Response:", string(jsonOut), " 201 OK")
}
main.go
package main
import (
"net/http"
"github.com/julienschmidt/httprouter"
"gopkg.in/mgo.v2"
"mongo-golang/controllers"
)
func main() {
// create new instance
r := httprouter.New()
// new session
uc := controllers.NewUserCOntroller(getSession())
r.GET("/user/:id", uc.GetUser)
r.POST("/user", uc.CreateUser)
r.DELETE("/user/:id", uc.DeleteUser)
http.ListenAndServe("localhost:9000", r)
}
func getSession() *mgo.Session {
// get session and connect with mongo
s, err := mgo.Dial("mongodb://localhost")
if err != nil {
panic(err)
}
return s
}
Would someone be able to let me know how to retrieve from json id not bson _id from golang script.
You can do that only if you save bson Id as a Hex string in the additional field in an object, but don't do that. That is unnecessary and you won't get anything.
If this is not possible, I would appreciate if someone let me know how to convert id to _id.
There are two functions that converting bson ObjectID to string and string to bson ObjectID
for id to _id, you already use that function in your code:
oid := bson.ObjectIdHex(id)
bson.ObjectIdHex() convert hex representation of objectID to bson.objectID. Hex representation is what you would see in JSON output.
After you invoke that function you get bson.ObjetID. There is a method .Hex() that can get you Hex (text/json) representation of that object.
oid := bson.ObjectIdHex(id)
json_represenation_of_bson_object_id = oid.Hex()
Also, you use the old mongo driver, a new driver written and maintained by MongoDB is what you should use:
https://github.com/mongodb/mongo-go-driver
I'm a beginner in both go and mongodb.
I try to decode a DocumentResult into a struct using bson tags, and it does not work for a custom type wrapping a string.
Can it be done without changing the field's type to a string?
import (
"context"
"github.com/mongodb/mongo-go-driver/mongo"
)
type MyDoc struct {
SomeInt int `bson:"some_int"`
SomeString string `bson:"some_string,omitempty"`
CustomType MyType `bson:"custom_type,omitempty"`
}
type MyType string
const myType MyType = "ABCD"
func main() {
//Connect to db
client, _ := mongo.Connect(context.Background(), "mongodb://localhost:27017", nil)
db := client.Database("example_db")
collection := db.Collection("col")
//Insert document
docToInsert := MyDoc{42, "The Answer", myType}
collection.InsertOne(nil, docToInsert)
//Retrieve document
filterDoc := MyDoc{SomeInt: 42}
resultDoc := &MyDoc{}
result := collection.FindOne(nil, filterDoc)
result.Decode(resultDoc)
println(resultDoc.SomeInt, resultDoc.SomeString, resultDoc.CustomType)
PRINTED RESULT: "42 The Answer" //"ABCD" is missing
Thanks in advance
I try to decode a DocumentResult into a struct using bson tags, and it does not work for a custom type wrapping a string
With your current MyType, the document that would be stored in MongoDB would be as below:
{
"_id": ObjectId("..."),
"some_int": NumberLong("42"),
"some_string": "The Answer",
"custom_type": "ABCD"
}
Even though the underlying type is a string, this could be tricky to decode with the current version of mongo-go-driver (v0.0.12) due to the type wrapping.
However, if you would like to have a custom type as such, you could change the struct into an embedded field instead. For example:
type MyDoc struct {
SomeInt int `bson:"some_int"`
SomeString string `bson:"some_string,omitempty"`
CustomType MyType `bson:"custom_type,omitempty"`
}
type MyType struct {
Value string `bson:"value,omitempty"`
}
var myType = MyType{Value: "ABCD"}
docToInsert := MyDoc{42, "The Answer", "ABCD"}
insertResult, err := collection.InsertOne(nil, docToInsert)
resultDoc := collection.FindOne(context.Background(), nil)
if err != nil {
log.Fatal(err)
}
elem := &MyDoc{}
err = resultDoc.Decode(elem)
if err != nil {
log.Fatal(err)
}
fmt.Println(elem.SomeInt, elem.SomeString, elem.CustomType.Value)
// 42 The Answer ABCD
The document would be stored in MongoDB as below:
{
"_id": ObjectId("..."),
"some_int": NumberLong("42"),
"some_string": "The Answer",
"custom_type": {
"value": "ABCD"
}
}
Otherwise just use string type directly because the resulting document in the database would be the same as the type wrapping version:
type MyDoc struct {
SomeInt int `bson:"some_int"`
SomeString string `bson:"some_string,omitempty"`
CustomType string `bson:"custom_type,omitempty"`
}
You may also find MongoDB Data Modeling a useful reference.
Foreword: Custom types having string as their underlying types are now handled automatically by the driver. This answer predates the driver 1.x versions where this was necessary.
Unfortunately you're out of luck. The current state of the official mongo go driver does not support unmarshaling string values from BSON to a Go value whose type is a custom type having string as its underlying type. This may change in the future, but for now this is not supported.
The way decoding into a struct field is handled is implemented in bson/decode.go, currently line #387:
case 0x2:
str := v.StringValue()
switch containerType {
case tString, tEmpty:
val = reflect.ValueOf(str)
case tJSONNumber:
_, err := strconv.ParseFloat(str, 64)
if err != nil {
return val, err
}
val = reflect.ValueOf(str).Convert(tJSONNumber)
case tURL:
u, err := url.Parse(str)
if err != nil {
return val, err
}
val = reflect.ValueOf(u).Elem()
default:
return val, nil
}
0x02 is the BSON string type. It is only attempted to decode into the struct field if the struct field's type is any of the following: string, interface{}, json.Number or url.URL (or a pointer to these).
Unfortunately implementing bson.Unmarshaler on your custom type does not help either, as it is not checked in case of struct fields, only if the struct itself implements it. But implementing on the struct itself, you would have to duplicate the struct with the field being one of the above listed supported types (or use a map or a bson.Document type).
This is a serious limitation on the library's part which can very easily be solved, so let's hope for the best that they add support for this in the near future.
With the 1.x versions of MongoDB driver for Go (latest version at the time of writing is 1.3.1) it is fully possible to encode and decode aliased types.
Your example now works as expected, given one adjusts the mongo.Connect to match new 1.x API.
package main
import (
"context"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type MyDoc struct {
SomeInt int `bson:"some_int"`
SomeString string `bson:"some_string,omitempty"`
CustomType MyType `bson:"custom_type,omitempty"`
}
type MyType string
const myType MyType = "ABCD"
func main() {
// Connect to db
clientOpts := options.Client().
ApplyURI("mongodb://localhost/example_db")
client, _ := mongo.Connect(context.Background(), clientOpts)
db := client.Database("example_db")
collection := db.Collection("col")
// Insert document
docToInsert := MyDoc{42, "The Answer", myType}
collection.InsertOne(nil, docToInsert)
// Retrieve document
filterDoc := MyDoc{SomeInt: 42}
resultDoc := &MyDoc{}
result := collection.FindOne(nil, filterDoc)
result.Decode(resultDoc)
println(resultDoc.SomeInt, resultDoc.SomeString, resultDoc.CustomType)
}
This returns: 42 The Answer ABCD as expected
So we have a Person with a Name. A First and Last name.
Let's insert the Person with the First and Last name and query for the Person by Name.First again. How?
package main
import (
"fmt"
"log"
"github.com/jinzhu/gorm"
)
var (
pgHost string
pgUser string
pgDatabase string
pgPass string
)
type Person struct {
gorm.Model
Name *Name
NameID uint
}
type Name struct {
gorm.Model
PersonID uint
First string
Last string
}
func main() {
// let's assume they're set
db, e := gorm.Open("postgres", fmt.Sprintf("host=%s user=%s dbname=%s password=%s sslmode=disable", pgHost, pgUser, pgDatabase, pgPass))
if e != nil {
log.Fatal(e)
}
defer db.Close()
db.AutoMigrate(&Person{}, &Name{})
p := &Person{
Name: &Name{First: "First", Last: "Last"},
}
tx := db.Begin()
if c := tx.Create(p); c.Error != nil {
log.Fatal("tx.create", c.Error)
}
tx.Commit()
// let's find it
pq := &Person{Name: &Name{First: "First"}}
pr := new(Person)
if c := db.Where(pq).First(pr); c.Error != nil {
log.Fatal("db.Where", c.Error)
}
}
Error Message
converting Exec argument $1 type: unsupported type Name, a struct
Feel free to suggest alternatives, change structs, etc.
No, adding NameFirst and NameLast fields isn't acceptable. The 2 structs should remain intact.
Remove the pointer and it will work
Reference:
http://jinzhu.me/gorm/associations.html
The mongodb documentation says:
The fields and values of both the and parameters if the parameter contains only update operator expressions. The update creates a base document from the equality clauses in the parameter, and then applies the update expressions from the parameter.
And the mgo documentation says:
Upsert finds a single document matching the provided selector document and modifies it according to the update document. If no document matching the selector is found, the update document is applied to the selector document and the result is inserted in the collection.
But if i do an upsert like this:
session.UpsertId(data.Code, data)
I end up with an entry which have an ObjectID generated automatically by mongodb, instead of data.Code.
this means that UpsertId expect data to be formated with update operators and you can't use a an arbitrary struct? Or what i'm missing here?
Pd. Mongo 2.4.9 mgo v2 golang go version devel +f613443bb13a
EDIT:
This is a sample of what i mean, using the sample code from Neil Lunn:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
// "gopkg.in/mgo.v2/bson"
)
type Person struct {
Code string
Name string
}
func main() {
session, err := mgo.Dial("admin:admin#localhost");
if err != nil {
fmt.Println("Error: ", err)
return
// panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
Code: "1234",
Name: "Bill",
}
_, err = c.UpsertId( p.Code, &p )
result := Person{}
err = c.FindId(p.Code).One(&result)
if err != nil {
fmt.Println("FindId Error: ", err)
return
// panic(err)
}
fmt.Println("Person", result)
}
I found the documentation of the MongoDB was right. The correct way to do this is to wrap the struct to insert into an update operator.
The sample code provided by Neil Lunn, would look like:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Person struct {
Code string
Name string
}
func main() {
session, err := mgo.Dial("admin:admin#localhost");
if err != nil {
fmt.Println("Error: ", err)
return
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
Code: "1234",
Name: "Bill",
}
upsertdata := bson.M{ "$set": p}
info , err2 := c.UpsertId( p.Code, upsertdata )
fmt.Println("UpsertId -> ", info, err2)
result := Person{}
err = c.FindId(p.Code).One(&result)
if err != nil {
fmt.Println("FindId Error: ", err)
return
}
fmt.Println("Person", result)
}
Thank you very much for your interest and help Neil.
You seem to be talking about assigning a struct with a custom _id field here. This really comes down to how you define your struct. Here is a quick example:
package main
import (
"fmt"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Person struct {
ID string `bson:"_id"`
Name string
}
func main() {
session, err := mgo.Dial("127.0.0.1");
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
var p = Person{
ID: "1",
Name: "Bill",
}
_, err = c.UpsertId( p.ID, &p )
result := Person{}
err = c.Find(bson.M{"_id": p.ID}).One(&result)
if err != nil {
panic(err)
}
fmt.Println("Person", result)
}
So in the custom definition here I am mapping the ID field to bson _id and defining it's type as string. As shown in the example this is exactly what happens when serialized via UpsertId and then retrieved.
Now you have elaborated I'll point to the difference on the struct definition.
What I have produces this:
{ "_id": 1, "name": "Bill" }
What you have ( without the same mapping on the struct ) does this:
{ "_id": ObjectId("53cfa557e248860d16e1f7e0"), "code": 1, "name": "Bill" }
As you see, the _id given in the upsert will never match because none of your fields in the struct are mapped to _id. You need the same as I have:
type Person struct {
Code string `bson:"_id"`
Name string
}
That maps a field to the mandatory _id field, otherwise one is automatically produced for you.
I try to populate the exported fields of a struct with content fetched from a MongoDb-database using the labix.org/v2/mgo package.
mgo uses the labix.org/v2/mgo/bson package to unmarshal the data. But the unmarshaller sets all unexported fields to their zero value.
Is there any way to prevent this behavior?
Working example:
package main
import (
"fmt"
"labix.org/v2/mgo/bson"
)
type Sub struct{ Int int }
type Player struct {
Name string
unexpInt int
unexpPoint *Sub
}
func main() {
dta,err := bson.Marshal(bson.M{"name": "ANisus"})
if err != nil {
panic(err)
}
p := &Player{unexpInt: 12, unexpPoint: &Sub{42}}
fmt.Printf("Before: %+v\n", p)
err = bson.Unmarshal(dta, p)
if err != nil {
panic(err)
}
fmt.Printf("After: %+v\n", p)
}
Output:
Before: &{Name: unexpInt:12 unexpPoint:0xf84005f500}
After: &{Name:ANisus unexpInt:0 unexpPoint:<nil>}
This is not possible. As you can see in the source code, struct values are explicitly being set to their zero value before filling in any fields.
There is no option to disable this behaviour. It is presumably in place to make sure the result of Unmarshal() only depends on the BSON data and not any prior state.