How to cross join with json and get all data - postgresql

I'm tryng to use a join function in my SQL request.
I have two table :
tbl_jsontesting
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| id | data | description |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 1 | {"complexProperties":[{"properties":{"key":"Registred","Value":"123456789"}},{"properties":{"key":"Urgency","Value":"Total"}},{"properties":{"key":"ImpactScope","Value":"All"}}]} | Some Text |
| 2 | {"complexProperties":[{"properties":{"key":"Registred","Value":"123456788"}},{"properties":{"key":"Urgency","Value":"Total"}},{"properties":{"key":"ImpactScope","Value":"All"}}]} | Some Text2 |
| 3 | {"complexProperties":[{"properties":{"key":"Urgency","Value":"Total"}},{"properties":{"key":"ImpactScope","Value":"All"}}]} | Some Text3 |
| 4 | {} | Some Text4 |
| 5 | {"complexProperties":[]} | Some Text5 |
| 6 | | Some Text6 |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tbl_registred
----------------------
| id | name |
----------------------
| 123456789 | Source |
| 123456788 | Cars |
----------------------
My current query :
select jt.id, rg.id as id_registred, rg.name, jt.description
from tbl_jsontesting jt
cross join jsonb_array_elements(jt.data::jsonb -> 'complexProperties') as p(props)
join tbl_registred rg
on rg.id::text = (p.props -> 'properties' ->> 'Value')
and p.props -> 'properties' ->> 'key' = 'Registred'
;
Result :
--------------------------------------------
| id | id_registred | name | description |
--------------------------------------------
| 2 | 123456788 | Cars | Some Text2 |
| 1 | 123456789 | Source | Some Text |
--------------------------------------------
Expected result :
--------------------------------------------
| id | id_registred | name | description |
--------------------------------------------
| 6 | | | Some Text6 |
| 5 | | | Some Text5 |
| 4 | | | Some Text4 |
| 3 | | | Some Text3 |
| 2 | 123456788 | Cars | Some Text2 |
| 1 | 123456789 | Source | Some Text |
--------------------------------------------
Fiddle : https://www.db-fiddle.com/f/5Jvq4SXUpBvJsY7H3G13xm/5

in your fiddle colunm tbl_jsontesting.description is named name :)
other than that
you can do a full join in place of the second join to keep all the line ;
then add the line whitout id_registred to your first result :
WITH all_line as (
select jt.id, rg.id as id_registred, rg.name, jt.description
from tbl_jsontesting jt
cross join jsonb_array_elements(jt.data::jsonb -> 'complexProperties') as p(props)
LEFT join tbl_registred rg
on rg.id::text = (p.props -> 'properties' ->> 'Value')
and p.props -> 'properties' ->> 'key' = 'Registred'
),line_registred as (
SELECT *
FROM all_line
WHERE id_registred IS NOT NULL
)
SELECT * FROM line_registred
UNION ALL
SELECT distinct * --distinct because of line generate by sub properties in json
FROM all_line
WHERE id NOT IN (SELECT id FROM line_registred)
probably not the shortest way, but you have what you excpected :
| id | id_registred | name | description |
| --- | ------------ | ------ | ----------- |
| 1 | 123456789 | Source | Some Text |
| 2 | 123456788 | Cars | Some Text2 |
| 3 | | | Some Text3 |
next to your edit :
line 6 in is not valid ('' is not json)
line 4 and 5 doesn't have 'properties'
so
select id, jsonb_array_elements(data::jsonb -> 'complexProperties') FROM tbl_jsontesting
does'nt return anything for that 2 lines
you could do the same tricks that we done for line not 'Registred'
with matched line then all line
give you some thing like :
**Query #1**
WITH line_with_properties as (
select id,description, jsonb_array_elements(data::jsonb -> 'complexProperties') as props
FROM tbl_jsontesting
),all_json_line as (
SELECT id,description, props
FROM line_with_properties
UNION ALL
SELECT id,description, '{}' as props
FROM tbl_jsontesting
WHERE id NOT IN (SELECT id from line_with_properties)
), all_line as(
select ajl.id, rg.id as id_registred, rg.name, ajl.description
from all_json_line ajl
LEFT join tbl_registred rg
on rg.id::text = (ajl.props -> 'properties' ->> 'Value')
and ajl.props -> 'properties' ->> 'key' = 'Registred'
),line_registred as (
SELECT *
FROM all_line
WHERE id_registred IS NOT NULL
)
SELECT * FROM line_registred
UNION ALL
SELECT distinct *
FROM all_line
WHERE id NOT IN (SELECT id FROM line_registred)
order by id;
and again what you expected tested in your fiddle (without line 6):
| id | id_registred | name | description |
| --- | ------------ | ------ | ----------- |
| 1 | 123456789 | Source | Some Text |
| 2 | 123456788 | Cars | Some Text2 |
| 3 | | | Some Text3 |
| 4 | | | Some Text4 |
| 5 | | | Some Text5 |

Related

Create a PostgreSQL function that becomes a formula field of a table retrieving related data from other table

The example above can be done on a SQL Server. It is a function that performs the calculation on another table while getting the current table field Id to list data from other table, return a single value.
Question: how to do the exact thing with PostgreSQL
SELECT TOP(5) * FROM Artists;
+------------+------------------+--------------+-------------+
| ArtistId | ArtistName | ActiveFrom | CountryId |
|------------+------------------+--------------+-------------|
| 1 | Iron Maiden | 1975-12-25 | 3 |
| 2 | AC/DC | 1973-01-11 | 2 |
| 3 | Allan Holdsworth | 1969-01-01 | 3 |
| 4 | Buddy Rich | 1919-01-01 | 6 |
| 5 | Devin Townsend | 1993-01-01 | 8 |
+------------+------------------+--------------+-------------+
SELECT TOP(5) * FROM Albums;
+-----------+------------------------+---------------+------------+-----------+
| AlbumId | AlbumName | ReleaseDate | ArtistId | GenreId |
|-----------+------------------------+---------------+------------+-----------|
| 1 | Powerslave | 1984-09-03 | 1 | 1 |
| 2 | Powerage | 1978-05-05 | 2 | 1 |
| 3 | Singing Down the Lane | 1956-01-01 | 6 | 3 |
| 4 | Ziltoid the Omniscient | 2007-05-21 | 5 | 1 |
| 5 | Casualties of Cool | 2014-05-14 | 5 | 1 |
+-----------+------------------------+---------------+------------+-----------+
The function
CREATE FUNCTION [dbo].[ufn_AlbumCount] (#ArtistId int)
RETURNS smallint
AS
BEGIN
DECLARE #AlbumCount int;
SELECT #AlbumCount = COUNT(AlbumId)
FROM Albums
WHERE ArtistId = #ArtistId;
RETURN #AlbumCount;
END;
GO
Now, (at SQL Server), after update the first table fields with ALTER TABLE Artists ADD AlbumCount AS dbo.ufn_AlbumCount(ArtistId); whe can list and get the following result.
+------------+------------------+--------------+-------------+--------------+
| ArtistId | ArtistName | ActiveFrom | CountryId | AlbumCount |
|------------+------------------+--------------+-------------+--------------|
| 1 | Iron Maiden | 1975-12-25 | 3 | 5 |
| 2 | AC/DC | 1973-01-11 | 2 | 3 |
| 3 | Allan Holdsworth | 1969-01-01 | 3 | 2 |
| 4 | Buddy Rich | 1919-01-01 | 6 | 1 |
| 5 | Devin Townsend | 1993-01-01 | 8 | 3 |
| 6 | Jim Reeves | 1948-01-01 | 6 | 1 |
| 7 | Tom Jones | 1963-01-01 | 4 | 3 |
| 8 | Maroon 5 | 1994-01-01 | 6 | 0 |
| 9 | The Script | 2001-01-01 | 5 | 1 |
| 10 | Lit | 1988-06-26 | 6 | 0 |
+------------+------------------+--------------+-------------+--------------+
but how to achieve this on postgresql?
Postgres doesn't support "virtual" computed column (i.e. computed columns that are generated at runtime), so there is no exact equivalent. The most efficient solution is a view that counts this:
create view artists_with_counts
as
select a.*,
coalesce(t.album_count, 0) as album_count
from artists a
left join (
select artist_id, count(*) as album_count
from albums
group by artist_id
) t on a.artist_id = t.artist_id;
Another option is to create a function that can be used as a "virtual column" in a select - but as this is done row-by-row, this will be substantially slower than the view.
create function album_count(p_artist artists)
returns bigint
as
$$
select count(*)
from albums a
where a.artist_id = p_artist.artist_id;
$$
language sql
stable;
Then you can include this as a column:
select a.*, a.album_count
from artists a;
Using the function like that, requires to prefix the function reference with the table alias (alternatively, you can use album_count(a))
Online example

How do I join tables while putting the results in a json array?

Table name: people
+----+------+-------------+-------+
| id | name | city | state |
+----+------+-------------+-------+
| 1 | Joe | Los Angeles | CA |
+----+------+-------------+-------+
| 2 | Jill | Miami | FL |
+----+------+-------------+-------+
| 3 | Asa | Portland | OR |
+----+------+-------------+-------+
Table name: pets
+----+----------+------+
| id | pet_name | type |
+----+----------+------+
| 1 | Spike | dog |
+----+----------+------+
| 1 | Fluffy | cat |
+----+----------+------+
| 2 | Oscar | dog |
+----+----------+------+
How would I join the two tables above to include a column containing JSON of results matched in the 'pets' table (PostgreSQL)?
+----+------+------------------------------------------------------------+
| id | name | pets |
+----+------+------------------------------------------------------------+
| 1 | Joe | [{name:'Spike', type:'dog'}, {name: 'Fluffy', type:'cat'}] |
+----+------+------------------------------------------------------------+
| 2 | Jill | [{name:'Oscar', type:'dog'}] |
+----+------+------------------------------------------------------------+
| 3 | Asa | [] |
+----+------+------------------------------------------------------------+
Use json_agg() to aggregate over json-objects:
SELECT people.id
, name
, json_agg(
CASE WHEN pet_name IS NOT NULL THEN
json_build_object(
'name', pet_name
, 'type', type
)
END
)
FROM people
LEFT JOIN pets ON people.id = pets.id
GROUP BY
people.id
, name
ORDER BY
people.id;

TSQL - PIVOT but CONCATENATE Fields

in this thread I was assisted with my initial question. The answer supplied has been accepted because it was the actual answer to that question.
As an extention to that answer, please consider the same table:
+------------------------------------------------------------------------------+
| GUID | DeviceGUID | DetailGUID | sValue | iValue | gValue | DateStored |
| ENTRY1 | DEVICE1 | Detail1 | SN112 | | | 01/01/2020 |
| ENTRY2 | DEVICE1 | Detail4 | | 1241 | | 01/01/2020 |
| ENTRY3 | DEVICE1 | Detail7 | | | GUID12 | 01/01/2020 |
| ENTRY8 | DEVICE1 | Detail7 | | | GUID13 | 01/01/2020 |
| ENTRY9 | DEVICE1 | Detail7 | | | GUID14 | 01/01/2020 |
| ENTRY4 | DEVICE2 | Detail1 | SN111 | | | 01/01/2020 |
| ENTRY5 | DEVICE2 | Detail2 | RND123 | | | 01/01/2020 |
| ENRTY6 | DEVICE2 | Detail4 | | 2351 | | 03/01/2020 |
| ENTRY7 | DEVICE3 | Detail1 | SN100 | | | 02/01/2020 |
| [...] | [...] | [...] | | | | |
| | | | | | | |
+------------------------------------------------------------------------------+
The issue arises when there are multiple records with the same DetailGUID; PIVOT has been set to select 'MAX', I would not know exactly how that selects the actual record in this case, but that is not important.
Insteas of selecting one record and having it displayed, I need the records to be concatenated in a Comma Separated List, in the Pivot.
the current SQL query is as follows:
DECLARE #columns NVARCHAR(MAX), #sql NVARCHAR(MAX), #OrderGUID uniqueidentifier;
SET #OrderGUID = '1B470FFB-7410-4950-A3BC-B9D778C459D3';
SET #columns = N'';
SELECT #columns+=N', p.'+QUOTENAME([Name])
FROM
(
SELECT GUID AS [Name]
FROM [dbo].Details AS p
) AS x;
SET #sql =
N'
SELECT *
FROM
(
SELECT DeviceObjectGUID
,DetailGUID
,CONCAT(sValue, iValue, gValue) as [value]
,DateStored
FROM DeviceDetails
WHERE (DeviceObjectGUID IN (SELECT DeviceObjectGUID FROM DevicesPerOrder WHERE OrderGUID = ''' + CAST(#OrderGUID as nVarchar(MAX) )+ '''))
) DS
PIVOT
(MAX([value]) FOR DetailGUID IN ('+STUFF(REPLACE(#columns, ', p.[', ',['), 1, 1, '')+')) PVT';
EXEC sp_executesql #sql
this will dynamically select all the detailGUIDS and transform them to headers; but I am unsure where I would start to input the CONCAT or TO XML statements

Select common values when using group by [Postgres]

I have three main tables meetings, persons, hobbies with two relational tables.
Table meetings
+---------------+
| id | subject |
+----+----------+
| 1 | Kickoff |
| 2 | Relaunch |
| 3 | Party |
+----+----------+
Table persons
+------------+
| id | name |
+----+-------+
| 1 | John |
| 2 | Anna |
| 3 | Linda |
+----+-------+
Table hobbies
+---------------+
| id | name |
+----+----------+
| 1 | Soccer |
| 2 | Tennis |
| 3 | Swimming |
+----+----------+
Relation Table meeting_person
+-----------------+-----------+
| id | meeting_id | person_id |
+----+------------+-----------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 2 | 1 |
| 5 | 2 | 2 |
| 6 | 3 | 1 |
+----+------------+-----------+
Relation Table person_hobby
+----------------+----------+
| id | person_id | hobby_id |
+----+-----------+----------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 2 | 1 |
| 5 | 2 | 2 |
| 6 | 3 | 1 |
+----+-----------+----------+
Now I want to to find the common hobbies of all person attending each meeting.
So the desired result would be:
+------------+-----------------+------------------------+
| meeting_id | persons | common_hobbies |
| | (Aggregated) | (Aggregated) |
+------------+-----------------+------------------------+
| 1 | John,Anna,Linda | Soccer |
| 2 | John,Anna | Soccer,Tennis |
| 3 | John | Soccer,Tennis,Swimming |
+------------+-----------------+------------------------+
My current work in progress is:
select
m.id as "meeting_id",
(
select string_agg(distinct p.name, ',')
from meeting_person mp
inner join persons p on mp.person_id = p.id
where m.id = mp.meeting_id
) as "persons",
string_agg(distinct h2.name , ',') as "common_hobbies"
from meetings m
inner join meeting_person mp2 on m.id = mp2.meeting_id
inner join persons p2 on mp2.person_id = p2.id
inner join person_hobby ph2 on p2.id = ph2.person_id
inner join hobbies h2 on ph2.hobby_id = h2.id
group by m.id
But this query lists not the common_hobbies but all hobbies which are at least once mentioned.
+------------+-----------------+------------------------+
| meeting_id | persons | common_hobbies |
+------------+-----------------+------------------------+
| 1 | John,Anna,Linda | Soccer,Tennis,Swimming |
| 2 | John,Anna | Soccer,Tennis,Swimming |
| 3 | John | Soccer,Tennis,Swimming |
+------------+-----------------+------------------------+
Does anyone have any hints for me, on how I could solve this problem?
Cheers
This problem can be solved by implement custom aggregation function (found it here):
create or replace function array_intersect(anyarray, anyarray)
returns anyarray language sql
as $$
select
case
when $1 is null then $2
when $2 is null then $1
else
array(
select unnest($1)
intersect
select unnest($2))
end;
$$;
create aggregate array_intersect_agg (anyarray)
(
sfunc = array_intersect,
stype = anyarray
);
So, the solution can be next:
select
meeting_id,
array_agg(ph.name) persons,
array_intersect_agg(hobby) common_hobbies
from meeting_person mp
join (
select p.id, p.name, array_agg(h.name) hobby
from person_hobby ph
join persons p on ph.person_id = p.id
join hobbies h on h.id = ph.hobby_id
group by p.id, p.name
) ph on ph.id = mp.person_id
group by meeting_id;
Look the example fiddle
Result:
meeting_id | persons | common_hobbies
-----------+-----------------------+--------------------------
1 | {John,Anna,Linda} | {Soccer}
3 | {John} | {Soccer,Tennis,Swimming}
2 | {John,Anna} | {Soccer,Tennis}

how to return number of records as a part of a select statement?

I'd like to know if there is a way to include row numbers (basically telling me how many records I'm getting back from a database query).
I have the following SQL query
SELECT w.widget_id, w.class_id, wg.name classname, wg.label AS classgroup, c.label, c.seq,
g.name AS group, p.name, p.type, CASE WHEN v.value IS NOT NULL THEN v.value WHEN g2p.value IS NOT NULL THEN g2p.value ELSE p.value END AS value
FROM widgets_to_categories w
INNER JOIN widget_classes c ON w.class_id = c.class_id
JOIN classes_to_param_groups t2g ON c.class_id = t2g.class_id
JOIN widget_groups g ON t2g.group_id = g.group_id
JOIN param_groups_to_params g2p ON t2g.group_id = g2p.group_id
JOIN provisioning_params p ON g2p.param_id = p.param_id
INNER JOIN widget_cat_groups wg ON c.class_group_id = wg.class_group_id
LEFT JOIN widget_values v ON(w.widget_id=v.device_id AND p.param_id=v.param_id AND g.name=v.group_name )
WHERE w.widget_id=8 ORDER BY c.class_id ASC
And it returns data like:
widget_id | class_id | classname | classgroup | label | seq | group | name | type | value
8 | 1 | toy | group A | test label | 1 | toy | reg | text | af
8 | 1 | toy | group A | test label | 1 | reg2 | fall | text | 25327
8 | 1 | toy | group A | test label | 1 | reg2 | pd | text | dvaa
8 | 1 | toy | group A | test label | 1 | reg2 | ext | text | 28235
8 | 1 | toy | group A | test label | 1 | reg1 | ext | text | 28230
8 | 1 | toy | group A | test label | 1 | toy | meec | text | 094F22DE501
8 | 1 | toy | group A | test label | 1 | toy | mmap | text | 0|
8 | 1 | toy | group A | test label | 1 | reg1 | fna | text | 26014
8 | 1 | toy | group A | test label | 1 | reg1 | fall | text | t-123
8 | 1 | toy | group A | test label | 1 | toy | uen | boolean | false
8 | 1 | toy | group A | test label | 1 | toy | adminpd |
I'd like to know if there's a way to have the database auto generate and return another column that is just an identifier for the row, like so:
id |widget_id | class_id | classname | classgroup | label | seq | group | name | type | value
1 | 8 | 1 | toy | group A | test label | 1 | toy | reg | text | af
2 | 8 | 1 | toy | group A | test label | 1 | reg2 | fall | text | 25327
3 | 8 | 1 | toy | group A | test label | 1 | reg2 | pd | text | dvaa
4 | 8 | 1 | toy | group A | test label | 1 | reg2 | ext | text | 28235
5 | 8 | 1 | toy | group A | test label | 1 | reg1 | ext | text | 28230
6 | 8 | 1 | toy | group A | test label | 1 | toy | meec | text | 094F22DE501
7 | 8 | 1 | toy | group A | test label | 1 | toy | mmap | text | 0|
8 | 8 | 1 | toy | group A | test label | 1 | reg1 | fna | text | 26014
9 | 8 | 1 | toy | group A | test label | 1 | reg1 | fall | text | t-123
10 | 8 | 1 | toy | group A | test label | 1 | toy | uen | boolean | false
11 | 8 | 1 | toy | group A | test label | 1 | toy | adminpd | boolean | false
I think I can do this by selecting into a temporary table.. I haven't figured out the syntax on how to do it yet... But I'm also wondering if there's another simpler way.
Once I get the data back from the database, having this ID field makes it eaiser to manipulate.
Thanks.
You can use the row_number window function to keep track of each row number.
Like so:
create table foo
(
id serial,
val text
);
INSERT INTO foo (val)
VALUES ('One'), ('Two'), ('Three');
SELECT f.*, row_number() OVER(ORDER BY val)
FROM foo AS f
ORDER BY val;
Here's an SQL Fiddle which shows this:
http://sqlfiddle.com/#!15/0c434/2
Additional options:
You could count the result with a query of the form:
SELECT count(*)
FROM
(
SELECT *
FROM foo
);
Or you may be able to get the row count back as part of the Postgres library you're using. For example, psycopg2 (Python) and DBI (Perl) allow for this (with some caveats). The library you're using may offer something similar.