Iteration for insert into values - postgresql

I will try and present the current setup as an abstract view, the focus being on the logical approach to batch insert.
CREATE TABLE factv (id, ..other columns..);
CREATE TABLE facto (id, ..other columns..);
CREATE TABLE dims (id serial, dimensions jsonb);
The 2 fact tables share the same dimensions, but they've got different columns.
There's an event stream which send messages to a table and there's function which is executed for each row, the logic is similar to:
CREATE OR REPLACE FUNCTION insert_event() RETURNS TRIGGER AS $$
IF event_json ->> 'type' = 'someEvent' THEN
WITH factv_insert AS (
INSERT INTO factv VALUES (id,..other columns..)
fn_createDimId,
NEW.event_json->>..,
...
RETURNING id
)
INSERT INTO facto VALUES (id,..other columns..)
(select id from factv_insert),
NEW.event_json->>..
ELSE DoSomethingElse...
END IF;
RETURN NEW;
END;
$$ LANGUAGE PLPGSQL;
The function called in here fn_createDimId just looks up the dims table and if these dimension are not found, they're inserted. If those are already in there, just give me that id for those dimensions as the id for this fact insert.
I do have some new events coming through and I need to grab some information which breaks the rules around insert into..values with ERROR: more than one row returned by a subquery used as an expression
The event structure is similar, but not limited to
{
"type": "someEvent",
"instruction": {
"contains": {
"id": "containerid",
"map": {
"50561:null:null": {
"productid": "50561",
"quantity": 3
},
"50562:null:null": {
"productid": "50562",
"quantity": 8
},
"50559:null:null": {
"productid": "50559",
"quantity": 5
}
}
},
"target": {
"50561": "Random",
"50562": "Random",
"50559": "Mix",
}
}
}
What is causing the problems here is the information around target and the respective quantities for those ids. From the event shown above, I need to aggregate and insert into the fact table:
-------|-----
target | qty
-------|-----
Random | 11
Mixed | 5
-------------
If I was to query to grab the information, I would run the following:
WITH meta_data as (
SELECT
json_object_keys(event_json -> 'instruction' ->'target') as prodid
,event_json -> 'instruction' ->'target'->>json_object_keys(event_json -> 'instruction' ->'target') as target
,event_json -> 'instruction' ->'contains'->'map'->json_object_keys(event_json -> 'instruction' ->'contains'->'map')->>'quantity' as qty
FROM event_table )
select
target,
sum(qty::int)
from meta_data
group by target;
I am looking to find a solution which makes it possible to do the same logical operations, but overcomes the error on the multiple returned rows, ideally an iteration for each event which returns more than 1 row.

Related

Return an object of objects in postgresql 15

I am trying to create an object of objects in a psql query, but I'm struggling to get the formatting right. I tried to do it with a nested jsonb_object_agg function but got this error:
ERROR: aggregate function calls cannot be nested
The return value needs to look like this:
"characteristics": {
"Size": {
"id": 14,
"value": "4.0000"
},
"Width": {
"id": 15,
"value": "3.5000"
},
"Comfort": {
"id": 16,
"value": "4.0000"
},
There names of the characteristics are in one table and the values in another. Here is the structure of the two tables:
CREATE TABLE IF NOT EXISTS characteristics
(
id integer PRIMARY KEY UNIQUE,
product_id integer,
name text
);
CREATE TABLE IF NOT EXISTS ratingschar
(
id serial PRIMARY KEY,
characteristics_id integer,
review_id integer,
value integer
);
Here is my query right now, it has gone through many iterations:
(select jsonb_object_agg(name, jsonb_object_agg(some_value,therating)) as chars
from (select name, AVG(value) as some_value, ratingschar.id as therating from
ratingschar
inner join characteristics
on ratingschar.id = characteristics.id
where product_id = 465464
GROUP BY name, ratingschar.id
) as b
)
I figured it out, I needed to use jsonb_build_object instead of jsonb_object_agg. I'm not entirely sure why I can't nest agg functions like that though. Here was my final query:
(select jsonb_object_agg(name, json_build_object('id',the_id, 'rating',the_rating)) as chars
from (select name, AVG(value) as the_rating, ratingschar.id as the_id from
ratingschar
inner join characteristics
on ratingschar.id = characteristics.id
where product_id = 465464
GROUP BY name, ratingschar.id
) as b
)

PostgreSQL update a jsonb column multiple times

Consider the following:
create table query(id integer, query_definition jsonb);
create table query_item(path text[], id integer);
insert into query (id, query_definition)
values
(100, '{"columns":[{"type":"integer","field":"id"},{"type":"str","field":"firstname"},{"type":"str","field":"lastname"}]}'::jsonb),
(101, '{"columns":[{"type":"integer","field":"id"},{"type":"str","field":"firstname"}]}'::jsonb);
insert into query_item(path, id) values
('{columns,0,type}'::text[], 100),
('{columns,1,type}'::text[], 100),
('{columns,2,type}'::text[], 100),
('{columns,0,type}'::text[], 101),
('{columns,1,type}'::text[], 101);
I have a query table which has a jsonb column named query_definition.
The jsonb value looks like the following:
{
"columns": [
{
"type": "integer",
"field": "id"
},
{
"type": "str",
"field": "firstname"
},
{
"type": "str",
"field": "lastname"
}
]
}
In order to replace all "type": "..." with "type": "string", I've built the query_item table which contains the following data:
path |id |
----------------+---+
{columns,0,type}|100|
{columns,1,type}|100|
{columns,2,type}|100|
{columns,0,type}|101|
{columns,1,type}|101|
path matches each path from the json root to the "type" entry, id is the corresponding query's id.
I made up the following sql statement to do what I want:
update query q
set query_definition = jsonb_set(q.query_definition, query_item.path, ('"string"')::jsonb, false)
from query_item
where q.id = query_item.id
But it partially works, as it takes the 1st matching id and skips the others (the 1st and 4th line of query_item table).
I know I could build a for statement, but it requires a plpgsql context and I'd rather avoid its use.
Is there a way to do it with a single update statement?
I've read in this topic it's possible to make it with strings, but I didn't find out how to adapt this mechanism with jsonb treatment.

why is column display order changed in json_agg funtion than that of a temp table - PostgreSQL

I am creating a temp table in a PostgreSQL function\proc.
CREATE TEMP TABLE tbl_temp_class(id SERIAL PRIMARY key, batch_id INT, class_id INT, class_name VARCHAR);
later I am dynamically adding columns to this table, using dynamic sql.
the l_column_counter is incremented with in a for loop untill n
l_sql_query := CONCAT('ALTER TABLE tbl_temp_class ADD column ', 'col', '_', l_column_counter, ' varchar default('''');');
EXECUTE l_sql_query;
At then end I want the tbl_temp_class result as a json array. Hence I'm doing below.
select json_agg(ut)
from (
select *
from tbl_temp_class
order by id) ut;
I expect the result for the above query to be
[
{
"id":1,
"batch_id":1,
"class_id":1,
"class_name":"Maths",
"col_1":"",
"col_2":"",
"col_3":"",
"col_4":"",
"col_5":""
},
{
"id":2,
"batch_id":1,
"class_id":2,
"class_name":"History",
"col_1":"",
"col_2":"",
"col_3":"",
"col_4":"",
"col_5":""
}
]
however, the result I am getting is as below. The column display order is scrambled.
Any idea how to fix this? Is this because the json is generated out of a temp table?
I need the column display order in the final json array to be same as the column display order in the temp table.
[
{
"id":1,
"col_1":"",
"col_2":"",
"col_3":"",
"col_4":"",
"col_5":"",
"class_id":1,
"batch_id":1,
"class_name":"Maths",
},
{
"id":2,
"col_1":"",
"col_2":"",
"col_3":"",
"col_4":"",
"col_5":"",
"class_id":2,
"batch_id":1,
"class_name":"History",
}
]
Did you try an ORDER BY in the aggregation?
SELECT json_agg(ut ORDER BY id) -- this where you want to use the ORDER BY
FROM (
SELECT *
FROM tbl_temp_class
ORDER BY id) ut;

Json_query check which rows have a special value in their json list

I have a table that each row contains a json column. Inside the json column I have an object containing an array of tags. What I want to do is to see which rows in my table have the tag that I am searching for.
Here is an example of my data:
Row 1:
Id :xxx
Jsom Column:
{
"tags":[
{"name":"blue dragon", weight:0.80},
{"name":"Game", weight:0.90}
]
}
Row 2:
Id : yyy
Jsom Column:
{
"tags":[
{"name":"Green dragon", weight:0.70},
{"name":"fantasy", weight:0.80}
]
}
So I want to write a code that if I search for Green, it returns only row 2 and if I search for dragon it returns both row 1 and 2. How can I do that?
I know I can write this to access my array, but more than that I am clueless :\
I am looking for something like this
Select * from myTable
where JSON_query([JsonColumn], '$.tags[*].name') like '%dragon%'
update
My final query looking like this
select DISTINCT t.id, dv.[key], dv.value
from #t t
cross apply openjson(doc,'$.tags') as d
where json_Value( d.value,'$.name') like '%dragon%'
Something like this:
declare #t table(id int, doc nvarchar(max))
insert into #t(id,doc) values
(1,'
{
"tags":[
{"name":"blue dragon", "weight":"0.80"},
{"name":"Game", "weight":"0.90"}
]
}'),(2,'
{
"tags":[
{"name":"Green dragon", "weight":"0.70"},
{"name":"fantasy", "weight":"0.80"}
]
}')
select t.id, dv.[key], dv.value
from #t t
cross apply openjson(doc,'$.tags') as d
cross apply openjson(d.value) dv
where dv.value like '%dragon%'

PostgreSQL - jsonb_each

I have just started to play around with jsonb on postgres and finding examples hard to find online as it is a relatively new concept.I am trying to use jsonb_each_text to printout a table of keys and values but get a csv's in a single column.
I have the below json saved as as jsonb and using it to test my queries.
{
"lookup_id": "730fca0c-2984-4d5c-8fab-2a9aa2144534",
"service_type": "XXX",
"metadata": "sampledata2",
"matrix": [
{
"payment_selection": "type",
"offer_currencies": [
{
"currency_code": "EUR",
"value": 1220.42
}
]
}
]
}
I can gain access to offer_currencies array with
SELECT element -> 'offer_currencies' -> 0
FROM test t, jsonb_array_elements(t.json -> 'matrix') AS element
WHERE element ->> 'payment_selection' = 'type'
which gives a result of "{"value": 1220.42, "currency_code": "EUR"}", so if i run the below query I get (I have to change " for ')
select * from jsonb_each_text('{"value": 1220.42, "currency_code": "EUR"}')
Key | Value
---------------|----------
"value" | "1220.42"
"currency_code"| "EUR"
So using the above theory I created this query
SELECT jsonb_each_text(data)
FROM (SELECT element -> 'offer_currencies' -> 0 AS data
FROM test t, jsonb_array_elements(t.json -> 'matrix') AS element
WHERE element ->> 'payment_selection' = 'type') AS dummy;
But this prints csv's in one column
record
---------------------
"(value,1220.42)"
"(currency_code,EUR)"
The primary problem here, is that you select the whole row as a column (PostgreSQL allows that). You can fix that with SELECT (jsonb_each_text(data)).* ....
But: don't SELECT set-returning functions, that can often lead to errors (or unexpected results). Instead, use f.ex. LATERAL joins/sub-queries:
select first_currency.*
from test t
, jsonb_array_elements(t.json -> 'matrix') element
, jsonb_each_text(element -> 'offer_currencies' -> 0) first_currency
where element ->> 'payment_selection' = 'type'
Note: function calls in the FROM clause are implicit LATERAL joins (here: CROSS JOINs).
WITH testa AS(
select jsonb_array_elements
(t.json -> 'matrix') -> 'offer_currencies' -> 0 as jsonbcolumn from test t)
SELECT d.key, d.value FROM testa
join jsonb_each_text(testa.jsonbcolumn) d ON true
ORDER BY 1, 2;
tetsa get the temporal jsonb data. Then using lateral join to transform the jsonb data to table format.