In my DB i have a table that require a coordinate value.
such value is stored as a
Point DataType
however when i try to obtain such value, GO responds with an error:
unsupported Scan, storing driver.Value type []uint8 into type *models.Point
this is my Model
type Business struct {
ID int `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Owner string `json:"owner,omitempty"`
Coordinates Point `json:"coordinates,omitempty"`
Reason string `json:"reason,omitempty"`
BusinessBranches []BusinessBranches `json:"reason,omitempty"`
}
and this is the Point type model
type Point struct {
X float64 `json:"lat"`
Y float64 `json:"lon"`
}
in my controller i call the selection of Business.
func (b *BusinessRepoImpl) Select() ([]models.Business, error) {
business_list := make([]models.Business, 0)
rows, err := b.Db.Query("SELECT id, name, owner, coordinates, reason FROM business")
if err != nil {
return business_list, err
}
for rows.Next() {
business := models.Business{}
err := rows.Scan(&business.ID, &business.Name, &business.Owner, &business.Coordinates, &business.Reason)
if err != nil {
fmt.Println(err)
break
}
business_list = append(business_list, business)
}
err = rows.Err()
if err != nil {
return business_list, err
}
return business_list, nil
}
anyone know how to parse such DataType, so i can read it and store it?
Thanks. :)
As i can see you need to set to set a structure with the current lat, long and then unmarshal from the database.
Related
I'm trying to convert the structure I'm decoding with my query to map[string]interface.
Here is my code:
var m map[string]interface
var result []Result
type Result struct {
Id ResultId `bson:"_id"`
Filename string `bson:"filename"`
}
type ResultId struct {
Host string `bson:"host"`
}
group := bson.D{{"$group", bson.D{{"_id", bson.D{{"host","$host"}}}, {"filename", bson.D{{"$last","$filename"}}}}}}
collection := client.Database("mongodb").Collection("Meta")
cursor, err := collection.Aggregate(ctx, mongo.Pipeline{group})
if err != nil {
return c.JSON(http.StatusInternalServerError, err)
}
defer cursor.Close(ctx)
if err = cursor.All(ctx, &results); err != nil {
fmt.Printf("cursor.All() error:", err)
return c.JSON(http.StatusInternalServerError, err)
}
for _, value := range results {
m = append(m,&bson.M{value.Id.Host:value.Filename})
}
But it does not return a map[string]interface and for information I use the go.mongodb.org package.
append only works on slices, (refer to effective go's section for more on append)
The way to add elements to a map, is simply:
m["key"] = value
Also keep in mind maps need to be initialised which I don't see in your code. Either with make or by giving an initial value (can be an empty value)
I am writing an abstraction over the official mongo driver which consists of a struct containing a pointer to the collection needed and CRUD methods on it. In order to be able to work with multiple types(all of which have bson adnotations) I use an interface called Storable, but I don't see a way in which I could decode the field without knowing the type exactly. Code snippet:
func (c *Collection) GetAll() ([]models.Storable, error) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cursor, err := c.coll.Find(ctx, bson.M{})
if err != nil {
return nil, err
}
var result []models.Storable
for cursor.Next(ctx) {
var doc models.Storable
err = cursor.Decode(&doc)
if err != nil {
return nil, err
}
result = append(result, doc)
}
return result, nil
}
type example:
type User struct {
ID primitive.ObjectID `json:"id" bson:"_id,omitempty"`
FirstName string `json:"firstName" bson:"firstName"`
LastName string `json:"lastName" bson:"lastName"`
Email string `json:"email" bson:"email"`
Password string `json:"password" bson:"password"`
}
You need to pass in the correct type to decode to somehow. To be able to pass in different types you could use the empty interface{}. However if you pass in the entire slice (e.g. []User) into interface{} you cannot append to it any more without complex usage of reflection.
Using a function to create a new row
As #mkopriva mentioned in the comments, we could pass a function creating a new row to it (or pass it to the collection on initialisation):
func (c *Collection) GetAll(newRow func() models.Storable) ([]models.Storable, error) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cursor, err := c.coll.Find(ctx, bson.M{})
if err != nil {
return nil, err
}
var result []models.Storable
for cursor.Next(ctx) {
nRow := newRow()
err = cursor.Decode(nRow)
if err != nil {
return nil, err
}
result = append(result, nRow.(models.Storable))
}
return result, nil
}
You would then call with a function returning the correct type (must be a pointer):
rows, err := c.GetAll(func() models.Storable {
return new(User)
})
Here my reference implementation using json decoding to test this: Playground
Using reflection to create a new row
You could pass in the variable type for a single row:
func (c *Collection) GetAll(row models.Storable) ([]models.Storable, error) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cursor, err := c.coll.Find(ctx, bson.M{})
if err != nil {
return nil, err
}
var result []models.Storable
for cursor.Next(ctx) {
nRow := reflect.New(reflect.TypeOf(p)).Interface()
err = cursor.Decode(nRow)
if err != nil {
return nil, err
}
result = append(result, nRow.(models.Storable))
}
return result, nil
}
You would then call this with the correct type:
var user User
rows, err := c.GetAll(user)
Note that I don't use a pointer type here! The pointer is taken via reflection in the function.
Here my reference implementation using json decoding to test this: Playground
The downside of the "one function for all" is that you now have a slice of models.Storable with different data inside and you have to use a type switch or type assertion to work with the data.
For this reason I don't use generic functions for database calls but create a new function for every call I need: e.g. GetAllUsers, GetUserByID, GetAllContacts, etc.
Note: this is one of the things I will rethink when generics get added to Go.
I am storing users and treating them as the center of the universe in my application, i am now trying to introduce the concept of an Org whereby users can be a member of many Orgs and then certain settings etc will belong to the Org. The function I am trying to create is to search for all Orgs where the users ID can be found and either the Owner or one of the Members and return a list of Orgs to then render the details client-side.
The issue I am having relates to the handling and conversion of results from the Mongo Find and then how to handle that and convert to a format I can return safely at the end.
Currently im unable to return the data with the error
cannot use &org (value of type *[]*model.Org) as *model.Org value in
return statement
Org Model
package model
// Org is the structure of a org
type Org struct {
ID string `json:"id" bson:"_id"`
Name string `json:"name" bson:"name"`
Owner string `json:"owner" bson:"owner"`
Members []string `json:"members" bson:"members"`
Token string `json:"token" bson:"-"`
VerifyToken string `json:"verifyToken" bson:"verifyToken"`
}
Function
// GetOrgByUserID returns a user by his id
func (db *DB) GetOrgByUserID(id string) (*model.Org, error) {
findOptions := options.Find()
var org []*model.Org
cur, err := db.collections.orgs.Find(context.TODO(), bson.D{{"owner", id}}, findOptions)
if err != nil {
return nil, err
}
// Iterate through the cursor
for cur.Next(context.TODO()) {
var elem model.Org
err := cur.Decode(&elem)
if err != nil {
return nil, err
}
org = append(org, &elem)
}
if err := cur.Err(); err != nil {
return nil, err
}
// Close the cursor once finished
cur.Close(context.TODO())
return &org, nil
}
Fix by declaring the return value as a slice. Also, simplify the code by using the cursor All method:
func (db *DB) GetOrgByUserID(id string) ([]*model.Org, error) {
findOptions := options.Find()
cur, err := db.collections.orgs.Find(context.TODO(), bson.D{{"owner", id}}, findOptions)
if err != nil {
return nil, err
}
defer cur.Close(ctx.TODO())
var org []*model.Org
err = cur.All(ctx.TODO(), &org)
return org, nil
}
I am trying to create a function where it gets all the documents from a Mongo Collection and queries them to declared structs. To achieve this I set the parameters for the function of type interface so it can work with two structs. Here is my code:
In package entities:
type Project struct {
Title string
Position string
....
}
type Projects struct {
Projects []Project
}
In the current package:
var docs entities.Projects
var doc entities.Project
//doc represents a document from Mongo Collection
//docs represents an array of documents, each element is a document
//collection has type *mongo.Collection and points to the desired collection on MongoDB.
createQuery(&doc, &docs, collection)
func createQuery(doc interface{}, docs interface{}, c *mongo.Collection) {
documents := reflect.ValueOf(docs).Elem()
document := reflect.ValueOf(doc)
cur, err := c.Find(context.Background(), bson.D{{}})
if err != nil {
log.Fatal(err)
}
for cur.Next(context.Background()) {
err = cur.Decode(document.Interface())
if err != nil {
log.Fatal(err)
}
//Error is thrown here
documents.Set(reflect.Append(documents, document))
fmt.Println(doc)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
if err != nil {
fmt.Printf("oh shit this is the error %s \n", err)
}
cur.Close(context.Background())
fmt.Printf("documents: %+v\n", documents.Interface())
fmt.Printf("document: %+v\n", document.CanSet())
}
---ERROR OUTPUT---
panic: reflect: call of reflect.Append on struct Value
I was able to set data to doc using the document variable, although when doing document.CanSet() is false (so it may not even work). Where the program breaks is when I try to append the document to the documents interface.
The code in the question passes the struct docs to a function that expects a slice. Pass the address of the slice field in docs instead of docs itself.
The createQuery function can determine the slice element type from the slice itself. There's no need to pass in an example value.
var docs entities.Projects
createQuery(&docs.Projects, collection)
for _, doc := range docs.Projects {
fmt.Println(doc.Title)
}
The call to cur.Decode requires a pointer to an uninitialized value. Use reflect.New to create that value.
func createQuery(docs interface{}, c *mongo.Collection) {
docsv := reflect.ValueOf(docs).Elem()
doct := docsv.Type().Elem()
cur, err := c.Find(context.Background(), bson.D{{}})
if err != nil {
log.Fatal(err)
}
for cur.Next(context.Background()) {
docpv := reflect.New(doct)
err = cur.Decode(docpv.Interface())
if err != nil {
log.Fatal(err)
}
docsv.Set(reflect.Append(docsv, docpv.Elem()))
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
cur.Close(context.Background())
}
As an aside, the entities.Projects struct type is not needed if that struct type will only ever have one field. Use []Project instead:
var docs []entities.Project
createQuery(&docs, collection)
for _, doc := range docs {
fmt.Println(doc.Title)
}
I have a table in Postgres that is a Jsonb
Create Table Business(
id serial not null primary key,
id_category integer not null,
name varchar(50) not null,
owner varchar(200) not null,
coordinates jsonb not null,
reason varchar(300) not null,
foreign key(id_category) references Category(id)
);
as you can see i store the coordinates as a jsonb
ex:
Insert into Business(id_category, name, owner, coordinates, reason)
values
(1,'MyName','Owner', [{"latitude": 12.1268142, "longitude": -86.2754}]','Description')
the way that I extract the data and assign it is like this.
type Business struct {
ID int `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Owner string `json:"owner,omitempty"`
Category string `json:"category,omitempty"`
Departments []string `json:"departments,omitempty"`
Location []Coordinates `json:"location,omitempty"`
Reason string `json:"reason,omitempty"`
}
type Coordinates struct {
Latitude float64 `json:"latitude,omitempty"`
Longitude float64 `json:"longitude,omitempty"`
}
func (a Coordinates) Value() (driver.Value, error) {
return json.Marshal(a)
}
func (a *Coordinates) Scan(value []interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
return json.Unmarshal(b, &a)
}
However, I keep receiving this message.
sql: Scan error on column index 3, name "coordinates": unsupported
Scan, storing driver.Value type []uint8 into type *models.Coordinates
And the controller that I use to extract the information is this.
func (b *BusinessRepoImpl) Select() ([]models.Business, error) {
business_list := make([]models.Business, 0)
rows, err := b.Db.Query("SELECT business.id, business.name, business.owner, business.coordinates, business.reason_froggy, category.category FROM business INNER JOIN category on category.id = business.id_category group by business.id, business.name, business.owner, business.coordinates, business.reason_froggy, category.category")
if err != nil {
return business_list, err
}
for rows.Next() {
business := models.Business{}
err := rows.Scan(&business.ID, &business.Name, &business.Owner, &business.Location, &business.Reason, &business.Category)
if err != nil {
break
}
business_list = append(business_list, business)
}
err = rows.Err()
if err != nil {
return business_list, err
}
return business_list, nil
}
Can anyone please tell me how to solve this issue? Retrieve the json array of objects and assign it to the coordinates field inside Business.
1.
As you can see from the documentation the Scanner interface, to be satisfied, requires the method
Scan(src interface{}) error
But your *Coordinates type implements a different method
Scan(value []interface{}) error
The types interface{} and []interface{} are two very different things.
2.
The Scanner interface must be implemented on the type of the field which you want to pass as an argument to rows.Scan. That is, you've implemented your Scan method on *Coordinates but the type of the Location field is []Coordinates.
Again, same thing, the types *Coordinates and []Coordinates are two very different things.
So the solution is to implement the interface properly and on the proper type.
Note that since Go doesn't allow adding methods to unnamed types, and []Coordinates is an unnamed type, you need to declare a new type that you'll then use in place of []Coordinates.
type CoordinatesSlice []Coordinates
func (s *CoordinatesSlice) Scan(src interface{}) error {
switch v := src.(type) {
case []byte:
return json.Unmarshal(v, s)
case string:
return json.Unmarshal([]byte(v), s)
}
return errors.New("type assertion failed")
}
// ...
type Business struct {
// ...
Location CoordinatesSlice `json:"location,omitempty"`
// ...
}
NOTE
If the business location will always have only one pair of coordinates store into the db as a jsonb object and change the Location type from CoordinatesSlice to Coordinates and accordingly move the Scanner implementation from *CoordinatesSlice to *Coordinates.
I know that this solution is really unoptimized, but it was the only way that it works.
basically i have to obtain the json and then do an unmarshal into the Location attribute.
var location string = ""
if err := json.Unmarshal([]byte(location), &business.Location); err != nil { panic(err) }