I have a struct Huygens:
type Huygens struct {
Worlds map[string]World `json:"worlds" sql:"type:JSONB"`
}
type World struct {
Diameter int
Name string
}
When I try to insert into a Postgres DB using GORM I get:
sql: converting argument $1 type: unsupported type map[string]huygens.world, a map
To insert I simply use db.Create(&World{})
Does anyone know how to insert that column as a JSONB column and avoid that error?
Thanks
You need to specify that Huygens is a gorm.Model and dump your World struct to a []byte type.
import (
"encoding/json"
"fmt"
"gorm.io/datatypes"
)
type Huygens struct {
gorm.Model
Worlds datatypes.JSON
}
type World struct {
Diameter int
Name string
}
world := World{Diameter: 3, Name: Earth}
b, err := json.Marshal(world)
if err != nil {
fmt.Println("error:", err)
}
db.Create(&Huygens{Worlds: datatypes.JSON(b)})
You can see more here.
Another way, which makes the model more usable, is to create a type that implements Scan/Value, and in those Unmarshal from/Marshal to JSON:
type Huygens struct {
Worlds WorldsMap `json:"worlds"`
}
type World struct {
Diameter int
Name string
}
type WorldsMap map[string]World
func (WorldsMap) GormDataType() string {
return "JSONB"
}
func (w *WorldsMap) Scan(value interface{}) error {
var bytes []byte
switch v := value.(type) {
case []byte:
bytes = v
case string:
bytes = []byte(v)
default:
return errors.New(fmt.Sprint("Failed to unmarshal JSONB value:", value))
}
err := json.Unmarshal(bytes, w)
return err
}
func (w WorldsMap) Value() (driver.Value, error) {
bytes, err := json.Marshal(s)
return string(bytes), err
}
Then you can use the Worlds field in your code like a map, as opposed to an awkward byte slice.
Related
I have a json object sent from a mobile application, looks like this:
{
"product_id": "0123456789",
"name": "PRODUCT_NAME",
"manufacturer": "PRODUCT_MANUFACTURER",
"image_url": "IMAGE_URL",
"additional_info": "",
"store_id": "STORE_ID",
"store_name": "STORE_NAME",
"owner_id": "OWNER_ID",
"quantities": {
"1000": 10.0,
"1500": 12.0,
}
}
The key value in quantities is a for example, can be grams, and the value is a representes a price.
So for example, 1000 grams of rice will cost 10$, and 1500 grams of rice will cost 12.0$ (as a sale or something)
I have a Model Object in my Go code, that has the quantities filed as map[int]float32
I'm trying to find a way to insert this map into a PostgreSQL database I have, but can't figure out how.
This is my Go Model:
type Product struct {
ID string
UUID string
Name string
Manufacturer string
ImageURL string
AdditionalInfo string
StoreID string
StoreName string
OwnerID string
Quantities map[int]float32
}
I read about JSONB but won't it return a json when I retrieve the data? I need it to return a map[int]float32 and not json.
You need to implement driver.Valuer and sql.Scanner interface for save as JSONB
The driver.Valuer interface, such that it marshals the object into a JSON byte slice that can be understood by the database.
The sql.Scanner interface, such that it unmarshals a JSON byte slice from the database into the struct fields.
And for json Marshal you need to convert map[int]float32 into map[string]float32
type cusjsonb map[int]float32
// Returns the JSON-encoded representation
func (a cusjsonb) Value() (driver.Value, error) {
// Convert to map[string]float32 from map[int]float32
x := make(map[string]float32)
for k, v := range a {
x[strconv.FormatInt(int64(k), 10)] = v
}
// Marshal into json
return json.Marshal(x)
}
// Decodes a JSON-encoded value
func (a *cusjsonb) Scan(value interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
// Unmarshal from json to map[string]float32
x := make(map[string]float32)
if err := json.Unmarshal(b, &x); err != nil {
return err
}
// Convert to map[int]float32 from map[string]float32
*a = make(cusjsonb, len(x))
for k, v := range x {
if ki, err := strconv.ParseInt(k, 10, 32); err != nil {
return err
} else {
(*a)[int(ki)] = v
}
}
return nil
}
I'm using MarshalBSONValue to marshal an inner struct field to a custom string representation.
I can't figure out how to implement the inverse operation, UnmarshalBSONValue, in order to parse the custom string representation into an inner struct.
import (
"fmt"
"testing"
"github.com/stretchr/testify/assert"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type inner struct {
value string
}
func (i inner) MarshalBSONValue() (bsontype.Type, []byte, error) {
return bsontype.String, []byte(i.value), nil
}
// How to implement this?
//
// func (i *inner) UnmarshalBSONValue(t bsontype.Type, data []byte) error {
// ...
// }
type Outer struct {
Inner inner `bson:"inner"`
}
func TestMarshalBSON(t *testing.T) {
var outer1 = Outer{Inner: inner{value: "value"}}
doc, err := bson.Marshal(outer1)
assert.NoError(t, err)
fmt.Printf("%#v\n", string(doc)) // "\x11\x00\x00\x00\x02inner\x00value\x00"
var outer2 Outer
err = bson.Unmarshal(doc, &outer2) // error
assert.NoError(t, err)
assert.Equal(t, outer1, outer2)
}
I would greatly appreciate it if anyone could provide a working implementation of UnmarshalBSONValue for the example test above.
You can use bsoncore.ReadString to parse the given value.
func (i inner) MarshalBSONValue() (bsontype.Type, []byte, error) {
return bson.MarshalValue(i.value)
}
func (i *inner) UnmarshalBSONValue(t bsontype.Type, value []byte) error {
if t != bsontype.String {
return fmt.Errorf("invalid bson value type '%s'", t.String())
}
s, _, ok := bsoncore.ReadString(value)
if !ok {
return fmt.Errorf("invalid bson string value")
}
i.value = s
return nil
}
I have 2 database table;
A has 3 columns and they are X, Y, Z
B has 2 columns and they are X, W
My Go structs are like this;
type Base struct {
X int
Y int
}
type A struct {
Base
Z int
}
type B struct {
Base
W int
}
And I initialize my structs like this;
a := A{Base: Base{X: 1, Y:2}, Z: 3}
b := B{Base: Base{X: 1}, W: 4}
When I want to insert these to database using gorm.io ORM, "a" is inserted without any problem but "b" can't be inserted because postgresql gives me an error something like
pq: column "y" of relation "B" does not exist
How can I insert "b" to database without creating another base model that doesn't have "Y" field?
When you assigning struct to another struct, and you create instance of struct, all struct fields has been filled with they default data-type value.
for example: int default value is 0.
So you have 2 solution for this question.
create two different struct(without Base struct), just A and B. like this: (maybe you know this solution.).
type A struct {
X int
Y int
Z int
}
type B struct {
X int
W int
}
use struct tags to prevent from inserting with gorm.
Note: I didn`t test this.
type Base struct {
X int
Y int `json:"y,omitempty"`
}
type A struct {
Base
Z int
}
type B struct {
Base
W int
}
I think you are not declaring the models correctly as documentation 'Base' should have tag embedded inside both A and B then it is ok. .. . for your better understanding I am posting your code with modification. Please be noted that I have tested in my machine it works as charm. .
here is the model declaration.
type Base struct {
X int `json:"x"`
Y int `json:"y"`
}
type A struct {
Base `gorm:"embedded"`
Z int `json:"z"`
}
type B struct {
Base `gorm:"embedded"`
W int `json:"w"`
}
here is the main function for your understanding. . .
func main() {
fmt.Println("vim-go")
db, err := gorm.Open("postgres", "host=localhost port=5432 user=postgres
dbname=testd password=postgres sslmode=disable")
if err != nil {
log.Panic("error occured", err)
}
db.AutoMigrate(&A{}, &B{})
a := &A{Base: Base{X: 1, Y: 2}, Z: 3}
b := &B{Base: Base{X: 1}, W: 3}
if err := db.Create(a).Error; err != nil {
log.Println(err)
}
if err := db.Create(b).Error; err != nil {
log.Println(err)
}
fmt.Println("everything is ok")
defer db.Close()
}
Please be sure to check the documentation of model declaration and the tags. . .
gorm model declaration
Note: Y field will be there in B table but with zero value. . .
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type SearchResult struct {
OrderNumber string `json:"orderNumber"`
Qunatity interface{} `json:"qty"`
Price string `json:"price"`
OrderType interface{} `json:"type"`
ItemQty string `json:"itemQty"`
}
type Or []SearchResult
func fieldSet(fields ...string) map[string]bool {
set := make(map[string]bool, len(fields))
for _, s := range fields {
set[s] = true
}
return set
}
func (s *SearchResult) SelectFields(fields ...string) map[string]interface{} {
fs := fieldSet(fields...)
rt, rv := reflect.TypeOf(*s), reflect.ValueOf(*s)
out := make(map[string]interface{}, rt.NumField())
for i := 0; i < rt.NumField(); i++ {
field := rt.Field(i)
jsonKey := field.Tag.Get("json")
if fs[jsonKey] {
out[jsonKey] = rv.Field(i).Interface()
}
}
return out
}
func main() {
result := &SearchResult{
Date: "to be honest you should probably use a time.Time field here,
just sayin",
Industry: "rocketships",
IdCity: "interface{} is kinda inspecific, but this is the idcity field",
City: "New York Fuckin' City",
}
b, err := json.MarshalIndent(result.SelectFields("orderNumber", "qty"), "", " ")
if err != nil {
panic(err.Error())
}
var or Or
fmt.Print(string(b))
or=append(or,*result)
or=append(or,*result)
for i := 0; i < len(or); i++ {
c, err := json.MarshalIndent(or[i].SelectFields("idCity", "city", "company"),
"", " ")
if err != nil {
panic(err.Error())
}
fmt.Print(string(c))
}
}
The one way of doing this using omitempty fields in struct and other way is to traverse through struct fields which is expensive.
In case of performance doesn`t matter for you then you can take reference of above code snippet.
I'm struggling to marshal/unmarshal bson array with polymorphic struct with mongo-go-driver。I plan to save a struct discriminator into the marshaled data, and write a custom UnmarshalBSONValue function to decode it according to the struct discriminator. But I don't know how to do it correctly.
package polymorphism
import (
"fmt"
"testing"
"code.byted.org/gopkg/pkg/testing/assert"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type INode interface {
GetName() string
}
type TypedNode struct {
NodeClass string
}
type Node struct {
TypedNode `bson:"inline"`
Name string
Children INodeList
}
func (n *Node) GetName() string {
return n.Name
}
type INodeList []INode
func (l *INodeList) UnmarshalBSONValue(t bsontype.Type, data []byte) error {
fmt.Println("INodeList.UnmarshalBSONValue")
var arr []bson.Raw // 1. First, try to decode data as []bson.Raw
err := bson.Unmarshal(data, &arr) // error: cannot decode document into []bson.Raw
if err != nil {
fmt.Println(err)
return err
}
for _, item := range arr { // 2. Then, try to decode each bson.Raw as concrete Node according to `nodeclass`
class := item.Lookup("nodeclass").StringValue()
fmt.Printf("class: %v\n", class)
if class == "SubNode1" {
bin, err := bson.Marshal(item)
if err != nil {
return err
}
var sub1 SubNode1
err = bson.Unmarshal(bin, &sub1)
if err != nil {
return err
}
*l = append(*l, &sub1)
} else if class == "SubNode2" {
//...
}
}
return nil
}
type SubNode1 struct {
*Node `bson:"inline"`
FirstName string
LastName string
}
type SubNode2 struct {
*Node `bson:"inline"`
Extra string
}
With code above, I'm trying to decode INodeList data as []bson.Raw, then decode each bson.Raw as a concrete Node according to nodeclass. But it report error:
cannot decode document into []bson.Raw
at line
err := bson.Unmarshal(data, &arr).
So, how to do it correctly?
You need to pass a bson.Raw’s pointer to bson.Unmarshal(data, &arr) and then slice its value to an array of raw values like:
func (l *INodeList) UnmarshalBSONValue(t bsontype.Type, data []byte) error {
fmt.Println("INodeList.UnmarshalBSONValue")
var raw bson.Raw // 1. First, try to decode data as bson.Raw
err := bson.Unmarshal(data, &raw)
if err != nil {
fmt.Println(err)
return err
}
// Slice the raw document to an array of valid raw values
rawNodes, err := raw.Values()
if err != nil {
return err
}
// 2. Then, try to decode each bson.Raw as concrete Node according to `nodeclass`
for _, rawNode := range rawNodes {
// Convert the raw node to a raw document in order to access its "nodeclass" field
d, ok := rawNode.DocumentOK()
if !ok {
return fmt.Errorf("raw node can't be converted to doc")
}
class := d.Lookup("nodeclass").StringValue()
// Decode the node's raw doc to the corresponding struct
var node INode
switch class {
case "SubNode1":
node = &SubNode1{}
case "SubNode2":
node = &SubNode2{}
//...
default:
// ...
}
bson.Unmarshal(d, node)
*l = append(*l, node)
}
return nil
}
Note that Note.Children must be a INodeList's pointer, and the inline fields must be a struct or a map (not a pointer):
type Node struct {
TypedNode `bson:",inline"`
Name string
Children *INodeList
}
type SubNode1 struct {
Node `bson:",inline"`
FirstName string
LastName string
}
type SubNode2 struct {
Node `bson:",inline"`
Extra string
}
I want to create function that accept 'dynamic array struct' and use it to map a data from database *mgodb
type Cats struct {
Meow string
}
func getCatsPagination() {
mapStructResult("Animality","Cat_Col", Cats)
}
type Dogs struct {
Bark string
}
func getDogsPagination() {
mapStructResult("Animality","Dog_Col", Dogs)
}
func mapStructResult(db string, collection string, model interface{}) {
result := []model{} //gets an error here
err := con.Find(param).Limit(int(limit)).Skip(int(offset)).All(&result) // map any database result to 'any' struct provided
if err != nil {
log.Fatal(err)
}
}
and gets an error as "model is not a type", why is it?
any answer will be highly appreciated !
Pass the ready slice to the mapStructResult function.
type Cats struct {
Meow string
}
func getCatsPagination() {
cats := []Cats{}
mapStructResult("Animality", "Cat_Col", &cats)
}
type Dogs struct {
Bark string
}
func getDogsPagination() {
dogs := []Dogs{}
mapStructResult("Animality", "Dog_Col", &dogs)
}
func mapStructResult(db string, collection string, model interface{}) {
err := con.Find(param).Limit(int(limit)).Skip(int(offset)).All(result) // map any database result to 'any' struct provided
if err != nil {
log.Fatal(err)
}
}
First of all there is no term in go called dynamic struct. Struct fields are declared before using them and cannot be changed. We can use bson type to handle the data. Bson type is like map[string]interface{} used to save the data dynamically.
func mapStructResult(db string, collection string, model interface{}) {
var result []bson.M // bson.M works like map[string]interface{}
err := con.Find(param).Limit(int(limit)).Skip(int(offset)).All(&result) // map any database result to 'any' struct provided
if err != nil {
log.Fatal(err)
}
}
For more information on bson type. Look this godoc for BSON