go scan Postgres array_agg - postgresql

I have a one-to-many relationship in postgres (Event has many EventUser), and would like to scan and store into a struct Event.
// EventUser struct
type EventUser struct {
ID int64
CheckedIn bool
PaidAmount float32
}
// Event struct
type Event struct {
ID int64 `json:"id"`
Name string `json:"name"`
StartTime string `json:"startTime"`
EventUsers string
}
Here is the query:
SELECT events.id, events.name, events."startTime", e."eventUsers" as "eventUsers"
FROM "Events" as events
LEFT JOIN (
SELECT events.id as id, array_to_json(array_agg(eu.*)) as "eventUsers"
FROM "EventUsers" as eu
JOIN "Events" AS "events" ON events.id = eu."eventId"
WHERE eu.status = 'RESERVED'
GROUP BY events.id
) AS e USING (id)
WHERE events.status = 'COMPLETED'
The query returns this:
{
id: 2,
name: "2 Events are 48 days from now",
startTime: 1590471343345,
eventUsers: [
{
id: 2,
checkedIn: false,
paidAmount: 8
},
{
id: 3,
checkedIn: false,
paidAmount: 8,
},
],
};
This is what I am trying to do, which directly scan each item under eventUsers into struct, and store in the Event's struct.
got := []Event{}
for rows.Next() {
var r Event
err = rows.Scan(&r.ID, &r.Name, &r.StartTime, &r.EventUsers)
if err != nil {
panic(err)
}
}
I think I could achieve this by storing the array as a string and then unmarshal it.
What I want is something similar to sql.NullString.

You could define a slice type that implements the sql.Scanner interface. Note that when you pass an instance of a type that implements Scanner to a (*sql.Rows).Scan or (*sql.Row).Scan call, the impelmenter's Scan method will be invoked automatically.
type EventUserList []*EventUser
func (list *EventUserList) Scan(src interface{}) error {
if data, ok := src.([]byte); ok && len(data) > 0 {
if err := json.Unmarshal(data, list); err != nil {
return err
}
}
return nil
}
Then, assuming that the e."eventUsers" in the select query is a json array, you could use it like this:
// EventUser struct
type EventUser struct {
ID int64
CheckedIn bool
PaidAmount float32
}
// Event struct
type Event struct {
ID int64 `json:"id"`
Name string `json:"name"`
StartTime string `json:"startTime"`
EventUsers EventUserList `json:"eventUsers"`
}
// ...
var events []*Event
for rows.Next() {
e := new(Event)
if err := rows.Scan(&e.ID, &e.Name, &e.StartTime, &e.EventUsers); err != nil {
return err
}
events = append(events, e)
}

Related

How to marshal/unmarshal bson array with polymorphic struct with mongo-go-driver

I'm struggling to marshal/unmarshal bson array with polymorphic struct with mongo-go-driver。I plan to save a struct discriminator into the marshaled data, and write a custom UnmarshalBSONValue function to decode it according to the struct discriminator. But I don't know how to do it correctly.
package polymorphism
import (
"fmt"
"testing"
"code.byted.org/gopkg/pkg/testing/assert"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type INode interface {
GetName() string
}
type TypedNode struct {
NodeClass string
}
type Node struct {
TypedNode `bson:"inline"`
Name string
Children INodeList
}
func (n *Node) GetName() string {
return n.Name
}
type INodeList []INode
func (l *INodeList) UnmarshalBSONValue(t bsontype.Type, data []byte) error {
fmt.Println("INodeList.UnmarshalBSONValue")
var arr []bson.Raw // 1. First, try to decode data as []bson.Raw
err := bson.Unmarshal(data, &arr) // error: cannot decode document into []bson.Raw
if err != nil {
fmt.Println(err)
return err
}
for _, item := range arr { // 2. Then, try to decode each bson.Raw as concrete Node according to `nodeclass`
class := item.Lookup("nodeclass").StringValue()
fmt.Printf("class: %v\n", class)
if class == "SubNode1" {
bin, err := bson.Marshal(item)
if err != nil {
return err
}
var sub1 SubNode1
err = bson.Unmarshal(bin, &sub1)
if err != nil {
return err
}
*l = append(*l, &sub1)
} else if class == "SubNode2" {
//...
}
}
return nil
}
type SubNode1 struct {
*Node `bson:"inline"`
FirstName string
LastName string
}
type SubNode2 struct {
*Node `bson:"inline"`
Extra string
}
With code above, I'm trying to decode INodeList data as []bson.Raw, then decode each bson.Raw as a concrete Node according to nodeclass. But it report error:
cannot decode document into []bson.Raw
at line
err := bson.Unmarshal(data, &arr).
So, how to do it correctly?
You need to pass a bson.Raw’s pointer to bson.Unmarshal(data, &arr) and then slice its value to an array of raw values like:
func (l *INodeList) UnmarshalBSONValue(t bsontype.Type, data []byte) error {
fmt.Println("INodeList.UnmarshalBSONValue")
var raw bson.Raw // 1. First, try to decode data as bson.Raw
err := bson.Unmarshal(data, &raw)
if err != nil {
fmt.Println(err)
return err
}
// Slice the raw document to an array of valid raw values
rawNodes, err := raw.Values()
if err != nil {
return err
}
// 2. Then, try to decode each bson.Raw as concrete Node according to `nodeclass`
for _, rawNode := range rawNodes {
// Convert the raw node to a raw document in order to access its "nodeclass" field
d, ok := rawNode.DocumentOK()
if !ok {
return fmt.Errorf("raw node can't be converted to doc")
}
class := d.Lookup("nodeclass").StringValue()
// Decode the node's raw doc to the corresponding struct
var node INode
switch class {
case "SubNode1":
node = &SubNode1{}
case "SubNode2":
node = &SubNode2{}
//...
default:
// ...
}
bson.Unmarshal(d, node)
*l = append(*l, node)
}
return nil
}
Note that Note.Children must be a INodeList's pointer, and the inline fields must be a struct or a map (not a pointer):
type Node struct {
TypedNode `bson:",inline"`
Name string
Children *INodeList
}
type SubNode1 struct {
Node `bson:",inline"`
FirstName string
LastName string
}
type SubNode2 struct {
Node `bson:",inline"`
Extra string
}

Projections in Mongodb not working properly

I have this data in my mongoDB database.
{
"_id":"5d9ce9fd270eae22adb95d70",
...
"isdriver":true,
"driver":{
"walletmoney":0,
"license":"6eef8271-62d7-4a1c-972a-2c40a773b35a",
"vehicle":{
"image":"b6c3619b-86e6-49d0-8734-e2c48815dfc1",
"insurance":"5f8229c4-4700-4059-8b72-9344a2bc6092",
"manufacturer":"Tesla",
"model":"Model 3",
"vin":"12345678912345678",
"year":2018
},
"verified":false
...
}
}
Here is my driver struct
type Driver struct {
...
Verified bool `json:"verified,omitempty"`
License string `json:"licenseimage,omitempty"`
...
Vehicle Vehicle `json:"vehicle,omitempty"`
}
Here is my Student Struct
type Student struct {
ID primitive.ObjectID `bson:"_id,omitempty"`
...
IsDriver bool `json:"isdriver,omitempty"`
Driver Driver `json:"driver,omitempty"`
}
Vehicle struct
type Vehicle struct {
Image string `json:"vehicleimage,omitempty"`
Insurance string `json:"insuranceimage,omitempty"`
VIN string `json:"vin,omitempty"`
Manufacturer string `json:"manufacturer,omitemptyr"` <-----(Edit) Find out this is also wrong
Model string `json:"model,omitempty"`
Year uint16 `json:"year,omitempty"`
}
And I'm using this function to get all the drivers from the database
func GetAllDrivers() []model.Driver {
// Options
projections := bson.D{
{"driver", 1},
/* {"driver.verified", 1},
{"driver.license", 1}, */
}
// Filter for search
filter := bson.M{"isdriver": true}
// Return student collection (*mongo.Collection)
studentCollection := GetStudentCollection()
cur, err := studentCollection.Find(context.TODO(), filter, options.Find().SetProjection(projections))
// Error while finding documents
if err != nil {
fmt.Print(err)
return []model.Driver{}
}
var drivers []model.Driver
var driver model.Driver
// Get the next result from the cursor
for cur.Next(context.TODO()) {
err := cur.Decode(&driver)
if err != nil {
fmt.Print(err)
}
drivers = append(drivers, driver)
}
if err := cur.Err(); err != nil {
fmt.Print(err)
}
cur.Close(context.TODO())
return drivers
}
But the response I'm getting in the postman is ridiculous
[
{
"vehicle": {
"manufacturer": ""
}
},
{
"vehicle": {
"manufacturer": ""
}
}
]
One thing is okay that is in response I'm getting two objects which are fine because as my filter suggestest isdriver: true I have total three documents in the database in which two of those have isdriver: true.
Can anybody help me with this? Why I'm getting this response?
You are doing a find in the students collection, but you decode into a driver.
This needs to be changed.
var drivers []Driver
var student Student
// Get the next result from the cursor
for cur.Next(context.TODO()) {
err := cur.Decode(&student)
if err != nil {
fmt.Println(err)
}
drivers = append(drivers, student.Driver)
}
Furthermore , you are lacking an Inline struct tag for the Driver field of Student:
type Student struct {
ID primitive.ObjectID `bson:"_id,omitempty"`
IsDriver bool `json:"isdriver,omitempty"`
// Note that Inline is uppercase.
Driver Driver `json:"driver,omitempty" bson:"driver,Inline"`
}
Same, of course, goes for all referenced structs. Working sample code: https://gist.github.com/mwmahlberg/c46ec3ad3ccee028f0666ff7d5d8d98b

How to access array values present in mongodb?

I want to access array values (access SpecCode) present in mongodb database from Go.
type MTopic struct {
SpecCodes []struct {
SpecCode string `json:speccode`
}
TopicCode string `json:topiccode`
TopicDesc string `json:topicdesc`
TopicBigDesc string `json:topicbigdesc`
TopicSource string `json:topicsource`
TopicSources []struct {
Topic string `json:topic`
}
CreatedBy string `json:createdby`
CreatedOn string `json:createdon`
UpdatedBy string `json:updatedby`
UpdatedOn string `json:updatedon`
}
using the following code:
func (m *TopicMaster) GetTopic(userdetails string) (list []MTopic, err error) {
collection := dbConnect7.Use("masterdata", "topic_master")
err = collection.Find(bson.M{"speccodes": userdetails}).All(&list)
return list, err
}
I have to get all values which have speccodes of userdetails in the collection topic_master. It's gin framework. This code is from models.
Just try like this
type MTopic struct {
SpecCodes []struct {
SpecCode string `json:"speccode"`
} `json:"speccodes"`
TopicCode string `json:"topiccode"`
TopicDesc string `json:"topicdesc"`
TopicBigDesc string `json:"topicbigdesc"`
TopicSource string `json:"topicsource"`
TopicSources []struct {
Topic string `json:"topic"`
}
CreatedBy string `json:"createdby"`
CreatedOn string `json:"createdon"`
UpdatedBy string `json:"updatedby"`
UpdatedOn string `json:"updatedon"`
}
and your function should be like this
func (m *TopicMaster) GetTopic(userdetails string) (list []MTopic, err error) {
collection := dbConnect7.Use("masterdata", "topic_master")
findQ := bson.M{"speccodes.speccode": userdetails}
list := make([]MTopic, 0)
if err = collection.Find(findQ).All(&list); err != nil {
err=errors.Wrapf(err, "failed to fetch topic info for user detail %s", userdetails)
return nil, err
}
return list, err
}

MongoDB bson.M query

I am trying to query using bison all JSON data in MongoDB with two fields but am getting null as result.
{
"allowedList": [
{
"List": [
{
"allow": {
"ss": 1,
},
"Information": [
{
"Id": "Id1"
}
]
}
]
}
]
}
I was able to filter all using the MongoDB at command line using
db.slicedb.find({"allowedList.List.allow.ss":1,"allowedList.List.Information.nsiId":"Id-Id21"})
but using
query := bson.M{"allowedList.List.allow": bson.M{"ss": sst}, "allowedList.List.Information": bson.M{"Id": Id}}
sst and Id are integer and string input to the query function
err := db.C(COLLECTION).Find(query).All(&specificSlices)
but is not working, am getting null even though there are json data that match the two field. Can someone help point out what was wrong with my query?
Server and database config
type SliceDataAccess struct {
Server string
Database string
}
var db *mgo.Database
const (
COLLECTION = "slicedb"
)
Establish a connection to database
func (m *SliceDataAccess) Connect() {
session, err := mgo.DialWithTimeout(m.Server, 20*time.Second)
if err != nil {
log.Fatal(err)
}
db = session.DB(m.Database)
}
Structs fields
type InstanceInfo struct {
ID string `json:"nfId" bson:"_id"`
AllowedList []AllowedNssai `json:"allowedList" bson:"allowedList"`
}
type AllowedNssai struct {
List []AllowedSnssai `json:"List,omitempty" bson:"List"`
...
}
type AllowedSnssai struct {
Allow *Snssai `json:"allow,omitempty" bson:"allow"`
Information []NsiInformation `json:"Information,omitempty" bson:"Information"`
}
type NsiInformation struct {
Id string `json:"Id" bson:"Id"`
}
type Snssai struct {
Ss int32 `json:"sst" bson:"ss"`
}
Query function defined
func (m *SliceDataAccess) FindAll(sst int32, nsiId string ([]InstanceInfo, error) {
var specificSlices []InstanceInfo
query := bson.M{"allowedList.List.allow": bson.M{"ss": sst}, "allowedList.List.Information": bson.M{"Id": nsiId}}
err := db.C(COLLECTION).Find(query).All(&specificSlices)
if err != nil {
return specificSlices, err
}
return specificSlices, nil
}
HTTP handler function for request and response
func AvailabilityGet(w http.ResponseWriter, r *http.Request)
var slice InstanceInfo
err := json.NewDecoder(r.Body).Decode(&slice)
if err != nil {
respondWithError(w, http.StatusBadRequest, "Object body not well decoded")
return
}
sst := slice.AllowedList[0].List[0].Allow.Sst
nsiId := slice.AllowedList[0].List[0].Information[0].Id
specificSlices, err := da.FindAll(sst, nsiId)
json.NewEncoder(w).Encode(specificSlices)
}
Attached is my the full go code i have done.
this worked
query := bson.M{"allowedNssaiList.allowedSnssaiList.allowedSnssai.sst": sst, "allowedNssaiList.allowedSnssaiList.nsiInformationList.nsiId": nsiId}

Create new struct and append it to array in a one to many relationship

I'm trying to append a new item to an array in a one-to-many relationship. The problem is that one of the IDs is always undefined and the model I want to append to does not get updated.
I have the following models:
type Station struct {
gorm.Model
Name string
Measurements []Measurement
PlantID uint64
Plant Plant
}
type Measurement struct {
ID uint64 `gorm:"primary_key"`
StationID uint64
TempSensor float32
LightSensor float32
HumiditySensor float32
CreatedAt time.Time
}
type Plant struct {
gorm.Model
Name string
}
This is the route to which I'm sending the post request:
/stations/:id/measurements
This is the current route handler that I have:
func CreateMeasurement(c *gin.Context) {
id := c.Params.ByName("id")
var station Station
if err := db.Where("id = ?", id).First(&station).Error; err != nil {
c.AbortWithStatus(404)
fmt.Println(err)
} else {
var measurement Measurement
c.BindJSON(&measurement)
// Convert params string to uint
convertedID, err := strconv.ParseUint(id, 10, 64)
if err != nil {
fmt.Println(err)
}
measurement.StationID = convertedID
db.Model(&station).Association("Measurements").Append(&measurement)
db.Save(&station)
c.JSON(200, station)
}
}
Question: How can I create a new Measurement item and append it to the []Measurement array in a specific Station which is specified by the route parameters?
Solved the problem. It turns out there was some problem with the database table. Although I got auto migrate enable, there was a problem with some ids that were null.
Here is the working route:
func CreateMeasurement(c *gin.Context) {
id := c.Params.ByName("id")
var station Station
if err := db.Where("id = ?", id).First(&station).Error; err != nil {
c.AbortWithStatus(404)
fmt.Println(err)
} else {
var measurement Measurement
c.BindJSON(&measurement)
db.Model(&station).Association("Measurements").Append(&measurement)
c.JSON(200, station)
}
}