postgres, go-gorm - can't preload data - postgresql

I am trying to preload some data when querying for a row on my postgresql with gorm
I have the following types defined in go with a belongs to association:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
Config json.RawMessage `gorm:"column:config" sql:"type:json" json:"config"`
Components []uint8 `gorm:"column:components;type:integer[]" json:"components"`
}
and the tables in the postgres db with a row as follows
my_schema.my_object
id | name | owner_id | ...
----|-----------|-----------|-----
0 | testobj | 0 | ...
my_schema.owner
id | name | config | components
----|-----------|-----------|-----------
0 | testowner | <jsonb> | {0,1,2,3}
I am running the following query with gorm:
object := &models.MyObject{}
result := ls.Table("my_schema.my_object").
Preload("Owner").
Where("id = ?", "0").
First(object)
but as a result, in object i get the following struct:
{"id": 0, "name": "testobj", "owner_id":0, "owner": {"id": 0, "name": "", "config": nil, "components": nil}}
I get no errors or warnings of any kind
the foreignkey relation was specified in the database creation sql script

I was able to achieve what I wanted by implementing the TableName() method for each of my models as such:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
func (MyObject) TableName() {
return "my_schema.my_object"
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
...
}
func (Owner) TableName() {
return "my_schema.owner"
}
Then, I can then use it in a query like the following:
myObject := &models.MyObject{}
result := ls.Model(myObject).
Where("id = ?", id). //assume any valid value for id
First(myObject).
Related(&myObject.Owner)
return myObject, processResult(result)
in the end, when making a request for MyObject to my server I get all the data of the related Owner object in the database:
$ curl "http://localhost:8080/api/objetcs/related/1" | jq // passing 1 as object id value in url params
{
"id": "1", // taken from UrlParam
"name": "testobj",
"owner": {
"id": "1", // using foreign key owner_id
"name": "testowner",
"config": configJson // json object taken from row in my_schema.owner table in the DB
}
}
}

Related

Create nested struct based on url query parameters

My goal is to get some filtered records from database. Filtration is based on a struct which depends on another struct:
type Group struct {
ID primitive.ObjectID
Name string
}
type Role struct {
ID primitive.ObjectID
Name string
Description string
Groups []*group.Group
}
I create an object of Role struct from URL query parameters:
var roleWP Role
if r.URL.Query().Has("name") {
name := r.URL.Query().Get("name")
roleWP.Name = name
}
if r.URL.Query().Has("description") {
description := r.URL.Query().Get("description")
roleWP.Description = description
}
if r.URL.Query().Has("groups") {
//How would look groups parameter?
}
Filling name and description fields of Role struct is pretty simple. The whole url would be: myhost/roles?name=rolename&description=roledescription
But how would look url if I want to pass data for Group struct? Is it possible to pass data as a json object in query parameter? Also, I want to mention that groups field in Role is an array. My ideal dummy url would look like: myhost/roles?name=rolename&description=roledescription&groups={name:groupname1}&groups={name:groupname2}
Loop through the groups, split on :, create group and append to slice:
roleWP := Role{
Name: r.FormValue("name"),
Description: r.FormValue("description"),
}
for _, g := range r.Form["groups"] {
g = strings.TrimPrefix(g, "{")
g = strings.TrimSuffix(g, "}")
i := strings.Index(g, ":")
if i < 0 {
// handle error
}
roleWP.Groups = append(roleWP.Groups, &Group{g[:i], g[i+1:]})
}
Here's how to use JSON instead of OP's ideal format:
roleWP := Role{
Name: r.FormValue("name"),
Description: r.FormValue("description"),
}
for _, s := range r.Form["groups"] {
var g Group
err := json.Unmarshal([]byte(s), &v)
if err != nil {
// handle error
}
roleWP.Groups = append(roleWP.Groups, &g)
}

Vapor 4: how to include siblings including extra properties from the pivot table?

I am struggling a lot with how to return a model that contains a many-to-many relationship via a pivot table that contains extra fields. Basically, I want to return the full pivot table with the extra fields, but I can't figure how to do this.
Let's consider the following 3 models: Course, User, and the pivot between them, a Student. The Student model contains the extra field progress.
final class Course: Model, Content {
static let schema = "courses"
#ID(key: .id)
var id: UUID?
#Field(key: "name")
var name: String
init() { }
}
final class Student: Model {
static let schema = "students"
#ID(key: .id)
var id: UUID?
#Parent(key: "course_id")
var course: Course
#Parent(key: "user_id")
var user: User
#Field(key: "progress")
var progress: Int
init() { }
}
final class User: Model, Content {
static let schema = "users"
#ID(key: .id)
var id: UUID?
#Field(key: "name")
var name: String
#Field(key: "private")
var somePrivateField: String
init() { }
}
I have a route like this, which returns an array of courses:
func list(req: Request) throws -> EventLoopFuture<[Course]> {
return Course
.query(on: req.db)
.all()
.get()
}
The resulting JSON looks something like this:
[
{
"id": 1,
"name": "Course 1"
}
]
How can I also include the array of students, so that the end result is something like this?
[
{
"id": 1,
"name": "Course 1",
"students": [
{
"user": {
"id": 1,
"name": "User 1"
},
"progress": 0
},
{
"user": {
"id": 2,
"name": "User 2"
},
"progress": 100
},
]
]
I can add the users to the Course model like this:
#Siblings(through: Student.self, from: \.$course, to: \.$user)
public var users: [User]
And then change my route like this:
func list(req: Request) throws -> EventLoopFuture<[Course]> {
return Course
.query(on: req.db)
.with(\.$user)
.all()
.get()
}
But that only adds the user info to the result, NOT the extra properties on the pivot table (namely, progress). It kinda seems to me that even though pivot tables can have extra properties and the docs even specifically point that out, there are no good ways of actually dealing with this scenario since #Siblings don't point to the pivot at all.
Bonus question: I'd want the User model be mapped to a PublicUser model, so that private/internal fields are not part of the JSON result. See this question for what I mean. I want to do that same thing, but with the Student pivot's User model. Complicated, I know 😬
I encountered the same issue about accessing additional fields in the pivot table, but there is a fairly tidy way of accomplishing this. In addition to your siblings relationship, define a #Children relation from Course to Student. Then, in your query, do a nested with.
Put this in your Course model:
#Children(for:\.$course) var students: [Student]
Query:
let query = Course.query(on: req.db).with(\.$students){ $0.with(\.$user) }.all()
The first with gets the additional fields of the pivot table and then the nested with get the User model.
You can query and use the extra properties on the pivot table directly using the pivots property on the Sibling relation which eagerly loads the pivot table objects through the sibling relation for easy access.
Eg:
Course.query(on: db)
.with(\.$user)
.with(\.$user.$pivots).all().map { course in
// you can now access the loaded students using:
let students = course.$user.pivots
...
}

Using MongoDB Projection

I have the following structure in my database:
{
"_id": {
"$oid": "5fc4fc68fcd604bac9f61f71"
},
"init": {
"fullname": "Besikta Anibula",
"parts": [
"Besikta",
"Anibula"
],
"alt": "Besikta Ani."
},
"industry": "E-Commerce"
}
I´m trying to just access the init object and write the results to a structured variable in Go:
var InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string`bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
collectionRESULTS.FindOne(ctx, options.FindOne().SetProjection(bson.M{"init": 1})).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
But the result is empty:
{"fullname":"","parts":null,"alt":""}
It is working when not using a projection like:
var InputData struct {
Ident primitive.ObjectID `bson:"_id" json:"id"`
}
collectionRESULTS.FindOne(ctx, bson.M{}).Decode(&InputData)
js, _ := json.Marshal(InputData)
fmt.Fprint(writer, string(js))
Result as expected:
{"id":"5fc4fc68fcd604bac9f61f71"}
You set the projection to only retrieve the init property of the result documents, and then you try to unmarshal into a struct value that does not have any matching field for the init value, so nothing will be set in that struct value.
You must use a value that has an init field, like this wrapper Result struct:
type Result struct {
Init InputData `bson:"init"`
}
type InputData struct {
Fullname string `bson:"fullname" json:"fullname"`
Parts []string `bson:"parts" json:"parts"`
Alt string `bson:"alt" json:"alt"`
}
Use it like this:
var result Result
err := collectionRESULTS.FindOne(ctx, bson.M{}, options.FindOne().
SetProjection(bson.M{"init": 1})).Decode(&result)
if err != nil {
// handle error
}

Convert JSON.RawMessage to JSON

I am using gqlgen, sqlx and pgx. And I'm trying to use a custom scalar for sqlx's types.JSONText.
I have this attributes jsonb field in items table.
-- migrations/001_up.sql
CREATE TABLE IF NOT EXISTS items (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
quantity INT NOT NULL,
attributes JSONB
);
I have these model structs:
// graph/model/item.go
type Item struct {
ID string `json:"id,omitempty" db:"id,omitempty"`
Quantity int `json:"quantity" db:"quantity"`
Attributes *Attributes `json:"attributes,omitempty" db:"attributes,omitempty"`
}
type Attributes types.JSONText
I have this graphql schema:
// graph/schema.graphql
type Item {
id: ID!
quantity: Int!
attributes: Attributes
}
scalar Attributes
I can successfully inserted into database, but got error at retrieving.
| id | quantity | attributes |
|---------------|----------|------------------------------------|
| 031e1489-... | 100 | {"size": "medium", "color": "red"} |
This is the log I got from db query:
>> items.Db: &{
031e1489-02c9-46d3-924d-6a2edf1ca3ba // id
100 // quantity
0xc000430600 // attributes
}
I tried to marshal attributes scalar:
// graph/model/item.go
...
func (a *Attributes) MarshalGQL(w io.Writer) {
b, _ := json.Marshal(a)
w.Write(b)
}
// Unmarshal here
...
Add custom scalar type in gqlgen.yml:
...
Attributes:
model:
- github.com/my-api/graph/model.Attributes
But I got the string instead of json:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": "eyJjb2xvciI6ICJyZWQifQ==",
}
}
}
The desired output is:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"size": "medium",
"color": "red",
}
}
}
}
What I am doing wrong?
Here are my attempts:
If I removed pointer from Attributes in Item struct, gqlgen throws error:
go generate ./...
go: finding module for package github.com/my-api/graph/generated
generating core failed: type.gotpl: template: type.gotpl:49:28: executing "type.gotpl" at <$type.Elem.GO>: nil pointer evaluating *config.TypeReference.GOexit status 1
graph/resolver.go:3: running "go": exit status 1
make: *** [Makefile:2: gengql] Error 1
This returns the desired result, but I don't know how to use it with real data:
func (a *Attributes) MarshalGQL(w io.Writer) {
raw := json.RawMessage(`{"foo":"bar"}`)
j, _ := json.Marshal(&raw)
s := string(j)
w.Write([]byte(s))
}
Query result:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"foo": "bar"
}
}
}
}
Attributes is defined using types.JSONText which is defined using json.RawMessage which is defined using []byte. This means that the underlying type of all 4 types, including []byte, is []byte, which in turn means that all of the 4 types can be converted to []byte.
Hence it should be enough to do this:
func (a *Attributes) MarshalGQL(w io.Writer) {
w.Write([]byte(*a))
}

How to group Postgres data correctly

In PostgreSQL, I have such table:
| QUESTION_TEXT | CATEGORY | AGREE_PERCENT | DISAGREE_PERCENT |
|----------------------------------------|----------|---------------|------------------|
| Do you support the President's policy? | Policy | 50 | 50 |
| Do you support Democrats? | Policy | 32 | 68 |
| Do you support the Lannisters? | Cinema | 45 | 55 |
| Do you support Spielberg's work? | Cinema | 60 | 40 |
In my Go application with the help of gorm library I make SQL request to PostgreSQL database like that:
type Entry struct {
QuestionText string `json:"question_text"`
Category string `json:"category"`
AgreePercent float64 `json:"agree_percent"`
DisagreePercent float64 `json:"disagree_percent"`
}
rows, _ := database.DBGORM.Raw("SELECT * FROM SPECIFICATION").Rows()
for rows.Next() {
entry := &Entry{}
if err = rows.Scan(&entry.QuestionText, & entry.Category, &entry.AgreePercent, &entry.DisagreePercent); err != nil {
utils.Logger().Println(err)
}
}
How to get a similar result? As you can see each object inside the array is grouped by value in the category column:
[
{
category: "Policy",
questions: ["Do you support the President's policy?", "Do you support Democrats?"],
series: [
{
name: "Agree, %",
data: [50, 32]
},
{
name: "Disagree, %",
data: [50, 68]
},
]
},
{
category: "Cinema",
questions: ["Do you support the Lannisters?", "Do you support Spielberg's work?"],
series: [
{
name: "Agree, %",
data: [45, 60]
},
{
name: "Disagree, %",
data: [55, 40]
},
]
},
]
Well, I can't say that it's elegant way, but finally I solve my task:
// Create struct called "Series".
type Series struct {
Name string `json:"name"`
Data []float64 `json:"data"`
}
// Create struct called "Specification".
type Specification struct {
Category string `json:"category"`
Questions []string `json:"questions"`
Series []Series `json:"series"`
}
// Initialize an array of struct.
var specifications []Specification
// Initialize several variables.
var catogoryName string
var arrayQuestionText []string
var arrayAgreePercent []float64
var arrayDisagreePercent []float64
for rows.Next() {
// Check the change in the name of the category.
if entry.Category == catogoryName {
// Add new elements to arrays.
arrayQuestionText = append(arrayQuestionText, entry.QuestionText)
arrayDisagreePercent = append(arrayDisagreePercent, entry.DisagreePercent)
arrayAgreePercent = append(arrayAgreePercent, entry.AgreePercent)
} else {
if len(catogoryName) > 0 {
// Fill the struct with data.
specification := Specification{
Category: catogoryName,
Questions: arrayQuestionText,
Series: []Series{
{
Name: "Agree, %",
Data: arrayAgreePercent,
},
{
Name: "Disagree, %",
Data: arrayDisagreePercent,
},
},
}
// Add new struct to array.
specifications = append(specifications, specification)
}
}
// Update data in arrays.
catogoryName = entry.Category
arrayQuestionText = nil
arrayQuestionText = append(arrayQuestionText, entry.QuestionText)
arrayDisagreePercent = nil
arrayDisagreePercent = append(arrayDisagreePercent, entry.DisagreePercent)
arrayAgreePercent = nil
arrayAgreePercent = append(arrayAgreePercent, entry.AgreePercent)
}