I'm using Ultralite 16.0 and trying to aggregate multiple rows into one.
I have a film table and a genre table, that have a many-to-many relationship through the film_genre table.
I want a result like this:
| mFilm | idGenre |
| --------- | ------- |
| Film_One | 1, 2 |
| Film_Two | 1, 3, 4 |
I can do this easily in SQL Server Management Studio (SSMS) with
SELECT f.nmFilm,
(
REPLACE(
STUFF(
(SELECT g.idGenre FROM genre g
JOIN film_genre fg on g.idGenre = fg.idGenre
WHERE fg.idFilm = f.idFilm
FOR XML PATH('')
) , 1 , 1 , ''
)
, '&', '&')
) AS genres FROM film f;
However, any time I try to use FOR XML in SQLAnywhere such as in the query below
SELECT f.nmFilm,
(SELECT g.idGenre FROM genre g
JOIN film_genre fg on g.idGenre = fg.idGenre
WHERE fg.idFilm = f.idFilm
FOR XML AUTO
) AS genres FROM film f;
I get syntax error
Could not execute statement.
[UltraLite Database] Syntax error near 'XML' [SQL Offset: 57]
SQLCODE=-131, ODBC 3 State="42000"
Line 1, column 1
I can't find any reference to FOR XML in the documentation, so I'm unsure if this is available in version 16.0.
How could I achieve this?
In case someone else needs this, all I had to do was use the LIST() function.
SELECT f.nmFilm,
(SELECT LIST(g.idGenre) FROM genre g
JOIN film_genre fg on g.idGenre = fg.idGenre
WHERE fg.idFilm = f.idFilm
) AS genres
FROM film f;
Related
I want to be able to reference the error table for msg and description based on the err_id on the results table for err_map jsonb column, I'd also want to be able relate which error occurred against which column whether the independent columns c1,c2 or val_map jsonb column c3, c4)
the only reason the val_map stores data(with .) as "val_map.c3": 3 so we can identify that these columns were from val_map when mapping errors to columns.
I have a result table
here the err_map column values 1,3 reference to below error table
id | c1 | c2 | val_map | err_map
----------------------------------------------------------------
1 | chk1 | chk2 | {"c3":3, "c4":4} | {"c1": 1, "val_map.c3": 3}
Error Table
id | msg | description
----------------------------------------------------------------
1 | msg1 | an error1 occurred
----------------------------------------------------------------
3 | msg3 | an error3 occurred
I looked at jsonb_each and jsonb_object_keys but can't really figure out how to use it to join these tables. Any help/hints will be appreciated.
Pardon if something is unclear, please ask to provide more detail.
[Edit 1]: removed foreign key reference as it was misleading
[Edit 2]: I've got it working but it's quite inefficient
select
e.error_key,
e.error_message,
T2.key as key
from result.error e
inner join (
select
substring(T1.key, 11) as key,
T1.value
from (
select em.key, em.value
from result rd, jsonb_each(rd.error_map) as em
) as T1
where T1.key like '%value_map%'
union all
select T1.key , T1.value
from (
select em.key, em.value
from result rd, jsonb_each(rd.error_map) as em
) as T1
where T1.key not like '%value_map%'
) as T2 on T2.value::bigint = e.id;
You can simplify that UNION ALL to just
select
e.error_key,
e.error_message,
T2.key as key
from result.error e
inner join (
select
case when T1.key like 'val_map.%'
then substring(T1.key, 9)
else T1.key
end as key,
T1.value
from result rd, jsonb_each(rd.error_map) as T1
) as T2 on T2.value::bigint = e.id;
I have one array with column values as
{james=UC/james,adam=C/james,chris=UC/james,john=U/james}
The above column values are not json. They are in string in the following form:
{ username=privilegestring/grantor }
How to convert above column into multiple rows
Edit #3:
Updated the query to specifically target pg_catalog.pg_namespace for acl permissions grants, via CTE pg_catalog. Currently this CTE is filtered in the where clause to select a single namespace name ('avengers'); if you want to select from multiple namespace names, you should be able to add them into the WHERE clause of this CTE directly, or in the case of wanting all namespace names, remove the clause altogether.
It's worth noting as well, that you will need to expand the case statements in access_privilege_types to handle all permissions cases: 'r', 'w', 'a', 'd', and 'x', for the operations: SELECT, UPDATE, INSERT, DELETE, REFERENCE, respectively.
Edit #2:
The final posted version of the query below should get you the data you want in the format that you want it in. I don't know how many possible values there are for the permissions types; if you have more than the two specified currently, you will need to expand the case statements in the CTE* access_privilege_types*. Obviously you'll also need to replace your table name within the query, etc.. Let me know if you run into any trouble and I'll help as necessary.
Edit #1:
Was able to validate that this query works in Redshift. Updated the query to break out separate rows by grantee and owner. The current version doesn't break out individual permissions by row yet -- Will take a look later tonight to see if I can get that working as well.
Original:
I don't have access to my Redshift cluster to test this at the moment, but I will when I get home. The general idea behind the following method, is to create a numbered index table to cross join against that will expand the data in the permissions field into a row-based representation.
I had inquired about the size limit, because this will currently only handle 10,000 possible delimited values, however you can adjust the CTEs to scale up to larger amounts if needed for your specific application:
Revision 3 Query:
WITH
pg_namespace AS (
SELECT
nspname
, nspowner
, rtrim(ltrim(array_to_string(nspacl, ','), '{'), '}') as nspacl
FROM pg_catalog.pg_namespace
WHERE nspname = 'public'
),
-- Generating a table with the numbers 1 - 10 in a single column.
ten_numbers AS (
SELECT
1 AS num
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 0
),
-- Expands the values in ten_numbers to create a single column with the values 1 - 10,000.
depivot_index AS (
SELECT
(1000 * t1.num) + (100 * t2.num) + (10 * t3.num) + t4.num AS gen_num
FROM ten_numbers AS t1
JOIN ten_numbers AS t2 ON 1 = 1
JOIN ten_numbers AS t3 ON 1 = 1
JOIN ten_numbers AS t4 ON 1 = 1
),
-- Filters down generated_numbers to house only the numbers up to the maximum times that the delimiter appears.
splitter AS (
SELECT
*
FROM depivot_index
WHERE gen_num BETWEEN 1 AND (
SELECT max(REGEXP_COUNT(nspacl, '\\,') + 1)
FROM pg_namespace
)
),
-- Cross joins permissions_groups and splitter to populate all requests, delimited on ','.
expanded_input AS (
SELECT
pg.nspname
, pg.nspacl
, trim(split_part(pg.nspacl, ',', s.gen_num)) AS raw_permissions_string
FROM pg_namespace AS pg
JOIN splitter AS s ON 1 = 1
WHERE split_part(nspacl, ',', s.gen_num) <> ''
),
-- Breaks out the owner and grantee fields into their own columns respectively.
users_with_raw_permissions_data AS (
SELECT
e.raw_permissions_string
, e.nspname
, trim(split_part(e.raw_permissions_string, '=', 1)) AS grantee
, trim(split_part(trim(split_part(e.raw_permissions_string, '=', 2)), '/', 2)) AS owner
, trim(split_part(trim(split_part(e.raw_permissions_string, '=', 2)), '/', 1)) AS raw_permissions_data
FROM
expanded_input e
),
-- Mines privilege types from raw string data.
access_privilege_types AS (
SELECT
u.nspname
, u.owner
, u.grantee
,CASE
WHEN position('C*' IN u.raw_permissions_data) > 0 THEN 'C*'
WHEN position('U*' IN u.raw_permissions_data) > 0 THEN 'U*'
WHEN position('C' IN u.raw_permissions_data) > 0 THEN 'C'
WHEN position('U' IN u.raw_permissions_data) > 0 THEN 'U'
ELSE u.raw_permissions_data
END AS first_access_privilege
, CASE
WHEN position('U*' IN u.raw_permissions_data) > 0 THEN 'U*'
WHEN position('C*' IN u.raw_permissions_data) > 0 THEN 'C*'
WHEN position('U' IN u.raw_permissions_data) > 0 THEN 'U'
WHEN position('C' IN u.raw_permissions_data) > 0 THEN 'C'
ELSE u.raw_permissions_data
END AS second_access_privilege
, first_access_privilege || ',' || second_access_privilege AS merged_access_privileges
FROM users_with_raw_permissions_data u
),
-- Cross joins access_privilge_types and splitter to populate all privilege_types, delimited on ','.
expanded_access_privilege_types AS (
SELECT
a.nspname
, a.owner
, a.grantee
, trim(split_part(a.merged_access_privileges, ',', s.gen_num)) AS access_privileges
FROM access_privilege_types AS a
JOIN splitter AS s ON 1 = 1
WHERE split_part(a.merged_access_privileges, ',', s.gen_num) <> ''
GROUP BY 1, 2, 3, 4
)
SELECT
ea.nspname
, ea.owner
, ea.grantee
, LEFT(ea.access_privileges, 1) AS access_privilege
, CASE
WHEN POSITION('*' IN ea.access_privileges) > 0 THEN 'YES'
ELSE 'NO'
END AS is_grantable
FROM expanded_access_privilege_types ea
ORDER BY 1, 2, 3, 4, 5
Edit #4:
Adding some clarification on how the ten_numbers, depivot_index, and splitter tables work to break apart the pg_catalog.pg_namespace.nspacl field. The general overview, is that ten_numbers and depivot_index are created purely to return tables with numbered rows to use as an index when joining in thesplit_partvalues ofnspacl`.
ten_numbers generates a table with a single column, containing the numbers 0-9:
-------
| num |
-------
| 0 |
-------
| 1 |
-------
| etc |
-------
| 9 |
-------
This table is then expanded to house the range 0-9999 during the CTE depivot_index:
-----------
| gen_num |
-----------
| 0 |
-----------
| 1 |
-----------
| 2 |
-----------
| etc |
-----------
| 9998 |
-----------
| 9999 |
-----------
splitter then narrows down the table to house only the numbers up to the maximum count of the specified delimiter within the nspacl field:
-------
| num |
-------
| 0 |
-------
| 1 |
-------
| etc |
-------
| 6 |
-------
The table returned by splitter is then used as the target of a CROSS JOIN via the join on 1 = 1 in CTE expanded_input. This ensures that each member returned by split_part will have its own row:
---------------------------------------------------------------------------
| nspname | nspacl | raw_permissions_string |
---------------------------------------------------------------------------
| avengers | "{james=UC/james,adam=C/james}" | "james=UC/james" |
---------------------------------------------------------------------------
| avengers | "{james=UC/james,adam=C/james}" | "adam=C/james" |
---------------------------------------------------------------------------
| avengers | etc. | etc. |
---------------------------------------------------------------------------
I'm new to SQL but I'm trying to join two tables. However, it's not working as I expected. This is in Postgresql.
Here are the tables I'm trying to join.
My Tables
SELECT * FROM houses;
id | name | address | picture
----+----------------+-------------+------------
1 | House 1 | 440 S 3rd W | long-link2.jpg
2 | House 2 | 538 S 5th E | long-link.jpg
SELECT house_id, trunc(avg(score), 1) FROM house_reviews GROUP BY house_id;
house_id | trunc
----------+-------
1 | 3.0
2 | 3.0
My JOIN statements
Attempt 1 (works)
SELECT * FROM houses
LEFT JOIN (SELECT house_id, trunc(avg(score), 1) FROM house_reviews GROUP BY house_id) AS r
ON houses.id = r.house_id;
Attempt 2 (does not work)
SELECT id, name, address FROM houses
LEFT JOIN (SELECT house_id, trunc(avg(score), 1) FROM house_reviews GROUP BY house_id) AS r
ON houses.id = r.house_id;
The only difference between the two is that I don't select the picture in the attempt 2. But attempt 2 doesn't seem to join at all. Instead it displays
id | name | address
----+----------------+-------------
1 | Tuscany | 440 S 2nd W
2 | Mountain Lofts | 538 S 2nd W
meaning that it failed to join and is instead just displaying the houses table.
My Question
I'm confused why the join failed in the second table because I removed only one arbitrary column (pictures).
Is there a way that I can join the two tables together but also exclude the pictures column from the "houses" table?
Thank you!
You're only seeing data from houses because that's all you've selected. Try this:
SELECT
h.id, h.name, h.address,
r.avg_score
FROM houses h
LEFT JOIN (
SELECT house_id, trunc(avg(score), 1) avg_score
FROM house_reviews
GROUP BY house_id
) AS r
ON houses.id = r.house_id;
create table dupt(cat varchar(10), num int)
insert dupt(cat,num) values ('A',1),('A',2),('A',3),
('B',1),('B',2),
('C',1),('C',2), ('C',3),
('D',1),('D',2), ('D',4),
('E',1),('E',2),
('F',1),('F',2)
I need to create a report which finds out duplicate data. From the sample data above, report needs to show that data for cat A is duplicated by cat C (notice the num value and no. of records) and cat B is duplicated by cat E and F. What is the best way to show that?
Example output
-------------
|cat | dupby|
-------------
| A | C |
| B | E, F |
-------------
Updated: switched to traditional set matching using common table expression and the stuff() with select ... for xml path ('') method of string concatenation only on the final results:
;with cte as (
select *
, cnt = count(*) over (partition by cat)
from t
)
, duplicates as (
select
x.cat
, dup_cat = x2.cat
from cte as x
inner join cte as x2
on x.cat < x2.cat
and x.num = x2.num
and x.cnt = x2.cnt
group by x.cat, x2.cat, x.cnt
having count(*) = x.cnt
)
select
d.cat
, dupby = stuff((
select ', '+i.dup_cat
from duplicates i
where i.cat = d.cat
for xml path (''), type).value('.','varchar(8000)')
,1,2,'')
from duplicates d
where not exists (
select 1
from duplicates i
where d.cat = i.dup_cat
)
group by d.cat
rextester demo: http://rextester.com/KHAG98718
returns:
+-----+-------+
| cat | dupby |
+-----+-------+
| A | C |
| B | E, F |
+-----+-------+
Say I have this table
id | data | value
-----------------
1 | a | A
2 | a | A
3 | a | A
4 | a | B
5 | b | C
6 | c | A
7 | c | C
8 | c | C
I want to remove those rows with duplicated value for each data while keeping the one with the min id, e.g. the result will be
id | data | value
-----------------
1 | a | A
4 | a | B
5 | b | C
6 | c | A
7 | c | C
I know a way to do it is to do a union like:
SELECT 1 [id], 'a' [data], 'A' [value] INTO #test UNION SELECT 2, 'a', 'A'
UNION SELECT 3, 'a', 'A' UNION SELECT 4, 'a', 'B'
UNION SELECT 5, 'b', 'C' UNION SELECT 6, 'c', 'A'
UNION SELECT 7, 'c', 'C' UNION SELECT 8, 'c', 'C'
SELECT * FROM #test WHERE id NOT IN (
SELECT MIN(id) FROM #test
GROUP BY [data], [value]
HAVING COUNT(1) > 1
UNION
SELECT MIN(id) FROM #test
GROUP BY [data], [value]
HAVING COUNT(1) <= 1
)
but this solution has to repeat the same group by twice (consider the real case is a massive group by with > 20 columns)
I would prefer a simpler answer with less code as oppose to complex ones. Is there any more concise way to code this?
Thank you
You can use one of the methods below:
Using WITH CTE:
WITH CTE AS
(SELECT *,RN=ROW_NUMBER() OVER(PARTITION BY data,value ORDER BY id)
FROM TableName)
DELETE FROM CTE WHERE RN>1
Explanation:
This query will select the contents of the table along with a row number RN. And then delete the records with RN >1 (which would be the duplicates).
This Fiddle shows the records which are going to be deleted using this method.
Using NOT IN:
DELETE FROM TableName
WHERE id NOT IN
(SELECT MIN(id) as id
FROM TableName
GROUP BY data,value)
Explanation:
With the given example, inner query will return ids (1,6,4,5,7). The outer query will delete records from table whose id NOT IN (1,6,4,5,7).
This fiddle shows the records which are going to be deleted using this method.
Suggestion: Use the first method since it is faster than the latter. Also, it manages to keep only one record if id field is also duplicated for the same data and value.
I want to add MYSQL solution for this query
Suggestion 1 : MySQL prior to version 8.0 doesn't support the WITH clause
Suggestion 2 : throw this error (you can't specify table TableName for update in FROM clause
So the solution will be
DELETE FROM TableName WHERE id NOT IN
(SELECT MIN(id) as id
FROM (select * from TableName) as t1
GROUP BY data,value) as t2;