How to group Postgres data correctly - postgresql

In PostgreSQL, I have such table:
| QUESTION_TEXT | CATEGORY | AGREE_PERCENT | DISAGREE_PERCENT |
|----------------------------------------|----------|---------------|------------------|
| Do you support the President's policy? | Policy | 50 | 50 |
| Do you support Democrats? | Policy | 32 | 68 |
| Do you support the Lannisters? | Cinema | 45 | 55 |
| Do you support Spielberg's work? | Cinema | 60 | 40 |
In my Go application with the help of gorm library I make SQL request to PostgreSQL database like that:
type Entry struct {
QuestionText string `json:"question_text"`
Category string `json:"category"`
AgreePercent float64 `json:"agree_percent"`
DisagreePercent float64 `json:"disagree_percent"`
}
rows, _ := database.DBGORM.Raw("SELECT * FROM SPECIFICATION").Rows()
for rows.Next() {
entry := &Entry{}
if err = rows.Scan(&entry.QuestionText, & entry.Category, &entry.AgreePercent, &entry.DisagreePercent); err != nil {
utils.Logger().Println(err)
}
}
How to get a similar result? As you can see each object inside the array is grouped by value in the category column:
[
{
category: "Policy",
questions: ["Do you support the President's policy?", "Do you support Democrats?"],
series: [
{
name: "Agree, %",
data: [50, 32]
},
{
name: "Disagree, %",
data: [50, 68]
},
]
},
{
category: "Cinema",
questions: ["Do you support the Lannisters?", "Do you support Spielberg's work?"],
series: [
{
name: "Agree, %",
data: [45, 60]
},
{
name: "Disagree, %",
data: [55, 40]
},
]
},
]

Well, I can't say that it's elegant way, but finally I solve my task:
// Create struct called "Series".
type Series struct {
Name string `json:"name"`
Data []float64 `json:"data"`
}
// Create struct called "Specification".
type Specification struct {
Category string `json:"category"`
Questions []string `json:"questions"`
Series []Series `json:"series"`
}
// Initialize an array of struct.
var specifications []Specification
// Initialize several variables.
var catogoryName string
var arrayQuestionText []string
var arrayAgreePercent []float64
var arrayDisagreePercent []float64
for rows.Next() {
// Check the change in the name of the category.
if entry.Category == catogoryName {
// Add new elements to arrays.
arrayQuestionText = append(arrayQuestionText, entry.QuestionText)
arrayDisagreePercent = append(arrayDisagreePercent, entry.DisagreePercent)
arrayAgreePercent = append(arrayAgreePercent, entry.AgreePercent)
} else {
if len(catogoryName) > 0 {
// Fill the struct with data.
specification := Specification{
Category: catogoryName,
Questions: arrayQuestionText,
Series: []Series{
{
Name: "Agree, %",
Data: arrayAgreePercent,
},
{
Name: "Disagree, %",
Data: arrayDisagreePercent,
},
},
}
// Add new struct to array.
specifications = append(specifications, specification)
}
}
// Update data in arrays.
catogoryName = entry.Category
arrayQuestionText = nil
arrayQuestionText = append(arrayQuestionText, entry.QuestionText)
arrayDisagreePercent = nil
arrayDisagreePercent = append(arrayDisagreePercent, entry.DisagreePercent)
arrayAgreePercent = nil
arrayAgreePercent = append(arrayAgreePercent, entry.AgreePercent)
}

Related

Convert JSON.RawMessage to JSON

I am using gqlgen, sqlx and pgx. And I'm trying to use a custom scalar for sqlx's types.JSONText.
I have this attributes jsonb field in items table.
-- migrations/001_up.sql
CREATE TABLE IF NOT EXISTS items (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
quantity INT NOT NULL,
attributes JSONB
);
I have these model structs:
// graph/model/item.go
type Item struct {
ID string `json:"id,omitempty" db:"id,omitempty"`
Quantity int `json:"quantity" db:"quantity"`
Attributes *Attributes `json:"attributes,omitempty" db:"attributes,omitempty"`
}
type Attributes types.JSONText
I have this graphql schema:
// graph/schema.graphql
type Item {
id: ID!
quantity: Int!
attributes: Attributes
}
scalar Attributes
I can successfully inserted into database, but got error at retrieving.
| id | quantity | attributes |
|---------------|----------|------------------------------------|
| 031e1489-... | 100 | {"size": "medium", "color": "red"} |
This is the log I got from db query:
>> items.Db: &{
031e1489-02c9-46d3-924d-6a2edf1ca3ba // id
100 // quantity
0xc000430600 // attributes
}
I tried to marshal attributes scalar:
// graph/model/item.go
...
func (a *Attributes) MarshalGQL(w io.Writer) {
b, _ := json.Marshal(a)
w.Write(b)
}
// Unmarshal here
...
Add custom scalar type in gqlgen.yml:
...
Attributes:
model:
- github.com/my-api/graph/model.Attributes
But I got the string instead of json:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": "eyJjb2xvciI6ICJyZWQifQ==",
}
}
}
The desired output is:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"size": "medium",
"color": "red",
}
}
}
}
What I am doing wrong?
Here are my attempts:
If I removed pointer from Attributes in Item struct, gqlgen throws error:
go generate ./...
go: finding module for package github.com/my-api/graph/generated
generating core failed: type.gotpl: template: type.gotpl:49:28: executing "type.gotpl" at <$type.Elem.GO>: nil pointer evaluating *config.TypeReference.GOexit status 1
graph/resolver.go:3: running "go": exit status 1
make: *** [Makefile:2: gengql] Error 1
This returns the desired result, but I don't know how to use it with real data:
func (a *Attributes) MarshalGQL(w io.Writer) {
raw := json.RawMessage(`{"foo":"bar"}`)
j, _ := json.Marshal(&raw)
s := string(j)
w.Write([]byte(s))
}
Query result:
{
"data": {
"item": {
"id": "031e1489-02c9-46d3-924d-6a2edf1ca3ba",
"quantity": 100,
"attributes": {
"foo": "bar"
}
}
}
}
Attributes is defined using types.JSONText which is defined using json.RawMessage which is defined using []byte. This means that the underlying type of all 4 types, including []byte, is []byte, which in turn means that all of the 4 types can be converted to []byte.
Hence it should be enough to do this:
func (a *Attributes) MarshalGQL(w io.Writer) {
w.Write([]byte(*a))
}

postgres, go-gorm - can't preload data

I am trying to preload some data when querying for a row on my postgresql with gorm
I have the following types defined in go with a belongs to association:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
Name string `gorm:"column:name" json:"name"`
Config json.RawMessage `gorm:"column:config" sql:"type:json" json:"config"`
Components []uint8 `gorm:"column:components;type:integer[]" json:"components"`
}
and the tables in the postgres db with a row as follows
my_schema.my_object
id | name | owner_id | ...
----|-----------|-----------|-----
0 | testobj | 0 | ...
my_schema.owner
id | name | config | components
----|-----------|-----------|-----------
0 | testowner | <jsonb> | {0,1,2,3}
I am running the following query with gorm:
object := &models.MyObject{}
result := ls.Table("my_schema.my_object").
Preload("Owner").
Where("id = ?", "0").
First(object)
but as a result, in object i get the following struct:
{"id": 0, "name": "testobj", "owner_id":0, "owner": {"id": 0, "name": "", "config": nil, "components": nil}}
I get no errors or warnings of any kind
the foreignkey relation was specified in the database creation sql script
I was able to achieve what I wanted by implementing the TableName() method for each of my models as such:
type MyObject struct {
ID uint `grom:"column:id" json:"id"`
OwnerID uint `gorm:"column:owner_id" json:"owner_id"`
Owner Owner `json:"owner"`
...
}
func (MyObject) TableName() {
return "my_schema.my_object"
}
type Owner struct {
ID uint `gorm:"column:id" json:"id"`
...
}
func (Owner) TableName() {
return "my_schema.owner"
}
Then, I can then use it in a query like the following:
myObject := &models.MyObject{}
result := ls.Model(myObject).
Where("id = ?", id). //assume any valid value for id
First(myObject).
Related(&myObject.Owner)
return myObject, processResult(result)
in the end, when making a request for MyObject to my server I get all the data of the related Owner object in the database:
$ curl "http://localhost:8080/api/objetcs/related/1" | jq // passing 1 as object id value in url params
{
"id": "1", // taken from UrlParam
"name": "testobj",
"owner": {
"id": "1", // using foreign key owner_id
"name": "testowner",
"config": configJson // json object taken from row in my_schema.owner table in the DB
}
}
}

How to query the additional column (other than siblings) added to pivot table?

Here is my RecipeIngredientPivot table schema to which I added an addition quantity column.
-----------------------------------------------------
id | ingredientID | recipeID | quantity
-----------------------------------------------------
| 1 | A001 | R001 | 100 |
| 2 | A002 | R001 | 50 |
| 3 | C004 | R001 | 23 |
| 4 | A001 | R002 | 75 |
I was able to create a quantity column to the Pivot table which has the ingredient_id and recipe_id as siblings by only using Pivot instead of ModifiablePivot.
extension RecipeIngredientPivot: Pivot {
init(_ recipe: Recipe, _ ingredient: Ingredient, _ quantity: Double) throws {
self.recipeID = try recipe.requireID()
self.ingredientID = try ingredient.requireID()
self.quantity = quantity
}
}
However, I am not sure how to query and get quantities of a recipe.
extension Recipe {
var ingredients: Siblings<Recipe, Ingredient, RecipeIngredientPivot> {
return siblings()
}
var quantities: [Double] {
// Not sure how to get this.
}
}
Any help is much appreciated.
I figured it out. I had to do query(on.req) & pivot(on: req) on the siblings() and wait for both to complete successfully before mapping on the results.
let ingredientFuture = try recipe.ingredients.query(on: req).all()
let pivotFuture = try recipe.ingredients.pivots(on: req).all()
return ingredientFuture.and(pivotFuture).map { ingredients, pivots in
for (ingredient,pivot) in zip(ingredients, pivots) {
print("\(ingredient) - \(pivot.quantity)")
}
}

Tabular Data to Nested JSON

I am trying to convert tabular query output to nested JSON using MuleSoft. My query output is like below:
----------------------------------
Customer | Order | Items
----------------------------------
C | Order1 | Itm1
C | Order1 | Itm2
C | Order2 | Itm1
C | Order2 | Itm4
C | Order3 | Itm3
C | Order3 | Itm4
and using Mule4 dataweave I am trying to convert it in flowing JSON output:
Customer: C
Orders: {
Order1:{
Items: {
Item: Item1
Item: Item2
}
}
Order2:{
Items: {
Item: Item1
Item: Item4
}
}
Order3:
}
So far I have tried code like below with no luck:
%dw 2.0
output application/json
---
payload map ((st, stindex) -> {
Customer: st.Customer,
Orders: payload filter(($.Customer == st.Customer) and ($.Order == st.Order)) map ((f, fIndex) ->{
Order: f.Order
Items : payload filter (($.ItemName == f.ItemName) and ($.Order == f.Order)) map ((i, iIndex) -> {
item: i.ItemName
})
})
})
it seems I am missing something important. Appreciate your help!!
I needed a multilevel groupBy(), mapObjects and a reduce() for the item.
%dw 2.0
output application/json
---
(payload map {
Customer: $.Customer,
Order: $.Order,
Items: $.Items
} groupBy (item) -> item.Customer)
mapObject ((value, key, index) -> {
Customer : key,
Orders: (value groupBy (orders)->orders.Order)
mapObject ((value1, key1, index1) -> {
"$(key1)": value1 reduce ((item, accumulator={}) -> accumulator ++ {Item: item.Items} )
}
)
}
)
Output:
{
"Customer": "C",
"Orders": {
"Order2": {
"Item": "Itm1",
"Item": "Itm4"
},
"Order1": {
"Item": "Itm1",
"Item": "Itm2"
},
"Order3": {
"Item": "Itm3",
"Item": "Itm4"
}
}
}
The indenting of the script is bad but it is too late for me to fix it.

What's the difference and/or preferred way of updating a deep property?

This is a babel/ES7 question (for redux reducer use)
I want to update "dan" only in some properties. What's the preferred way with immutability mind?
It seems that only TRY 1 and TRY 3 merge/update correctly.
Is there any difference between the two? For me TRY 3 wins because it's the shortest (if no difference between TRY 1)
Thanks
const people = { byID: {
gaston : { name:'Gaston', age: 22 },
dan : { name: 'gaston', age: 44 }
}
}
const currentID = "dan"
///
// TRY 1
const thisID = {}
thisID[currentID] = {...people.byID[currentID],
age: 20,
sex: 'male',
}
const newPeople = {...people,
byID: {...people.byID,
...thisID
}
}
console.log( newPeople ) // OK
//
// TRY 2
const newPeople2 = {}
newPeople2.byID = {}
newPeople2.byID[currentID] = {}
newPeople2.byID[currentID]["age"] = 20
newPeople2.byID[currentID]["sex"] = "male"
const newPeople3 = {...people, ...newPeople2}
console.log( newPeople3 ) // NOPE (not merge)
//
// TRY 3
const newPeople4 = {...people}
newPeople4.byID = newPeople4.byID || {}
newPeople4.byID[currentID] = newPeople4.byID[currentID] || {}
newPeople4.byID[currentID]["age"] = 20
newPeople4.byID[currentID]["sex"] = "male"
console.log(newPeople4) // OK
Here are the outputs
TRY 1
{"byID":{"gaston":{"name":"Gaston","age":22},"dan":{"name":"gaston","age":20,"sex":"male"}}}
TRY 2
{"byID":{"dan":{"age":20,"sex":"male"}}}
TRY 3
{"byID":{"gaston":{"name":"Gaston","age":22},"dan":{"name":"gaston","age":20,"sex":"male"}}}
Using the spread operator, you can do:
const updatedPeople = {
byID: {
...people.byID,
dan: {
...people.byID.dan,
age: 20,
sex: "male"
}
}
};
If you need the id to be dynamic, using computed property keys (an other ES6 feature, part of destructuring):
const id = 'dan';
const updatedPeople = {
byID: {
...people.byID,
[id]: {
...people.byID[id],
age: 20,
sex: "male"
}
}
};
Those solutions are immutability full proof, though, if you're not familiar with ES6 syntax, they can be hard to read and if you have more complex data, you might consider using immutablejs (or some kind of immutable library)