TSQL - PIVOT but CONCATENATE Fields - tsql

in this thread I was assisted with my initial question. The answer supplied has been accepted because it was the actual answer to that question.
As an extention to that answer, please consider the same table:
+------------------------------------------------------------------------------+
| GUID | DeviceGUID | DetailGUID | sValue | iValue | gValue | DateStored |
| ENTRY1 | DEVICE1 | Detail1 | SN112 | | | 01/01/2020 |
| ENTRY2 | DEVICE1 | Detail4 | | 1241 | | 01/01/2020 |
| ENTRY3 | DEVICE1 | Detail7 | | | GUID12 | 01/01/2020 |
| ENTRY8 | DEVICE1 | Detail7 | | | GUID13 | 01/01/2020 |
| ENTRY9 | DEVICE1 | Detail7 | | | GUID14 | 01/01/2020 |
| ENTRY4 | DEVICE2 | Detail1 | SN111 | | | 01/01/2020 |
| ENTRY5 | DEVICE2 | Detail2 | RND123 | | | 01/01/2020 |
| ENRTY6 | DEVICE2 | Detail4 | | 2351 | | 03/01/2020 |
| ENTRY7 | DEVICE3 | Detail1 | SN100 | | | 02/01/2020 |
| [...] | [...] | [...] | | | | |
| | | | | | | |
+------------------------------------------------------------------------------+
The issue arises when there are multiple records with the same DetailGUID; PIVOT has been set to select 'MAX', I would not know exactly how that selects the actual record in this case, but that is not important.
Insteas of selecting one record and having it displayed, I need the records to be concatenated in a Comma Separated List, in the Pivot.
the current SQL query is as follows:
DECLARE #columns NVARCHAR(MAX), #sql NVARCHAR(MAX), #OrderGUID uniqueidentifier;
SET #OrderGUID = '1B470FFB-7410-4950-A3BC-B9D778C459D3';
SET #columns = N'';
SELECT #columns+=N', p.'+QUOTENAME([Name])
FROM
(
SELECT GUID AS [Name]
FROM [dbo].Details AS p
) AS x;
SET #sql =
N'
SELECT *
FROM
(
SELECT DeviceObjectGUID
,DetailGUID
,CONCAT(sValue, iValue, gValue) as [value]
,DateStored
FROM DeviceDetails
WHERE (DeviceObjectGUID IN (SELECT DeviceObjectGUID FROM DevicesPerOrder WHERE OrderGUID = ''' + CAST(#OrderGUID as nVarchar(MAX) )+ '''))
) DS
PIVOT
(MAX([value]) FOR DetailGUID IN ('+STUFF(REPLACE(#columns, ', p.[', ',['), 1, 1, '')+')) PVT';
EXEC sp_executesql #sql
this will dynamically select all the detailGUIDS and transform them to headers; but I am unsure where I would start to input the CONCAT or TO XML statements

Related

Do multiple UPDATE using CTE

I'm trying to use a CTE to do two update statements in Postgres and I'm not sure if there is a better approach to what I'm trying to do.
-- This script will update user records where the duplicate record value will overwrite the old value
WITH
update_users_Cte
AS
(
SELECT
new.id new_user_id,
old.id old_user_id,
new.employee_id new_employee_id,
old.employee_id old_employee_id,
new.display_name new_display_name,
old.display_name old_display_name,
new.first_name new_first_name,
old.first_name old_first_name,
new.last_name new_last_name ,
old.last_name old_last_name,
new.phone_number new_phone_number,
old.phone_number old_phone_number,
new.job_type new_job_type,
old.job_type old_job_type,
new.department_id new_department_id,
old.department_id old_department_id
FROM users old
JOIN users new ON (CONCAT(0, new.employee_id) = old.employee_id)
WHERE LENGTH(new.employee_id) = 5 AND
new.display_name != old.display_name OR
new.first_name != old.first_name OR
new.last_name != old.last_name OR
new.phone_number != old.phone_number OR
new.job_type != old.job_type OR
new.department_id != old.department_id
)
UPDATE
users
SET
phone_number = NULL
FROM
update_users_Cte
WHERE
employee_id = update_users_Cte.new_employee_id
UPDATE
users
SET
display_name = update_users_Cte.new_display_name,
first_name = update_users_Cte.new_first_name,
last_name = update_users_Cte.new_last_name,
phone_number = update_users_Cte.new_phone_number,
job_type = update_users_Cte.new_job_type,
department_id = update_users_Cte.new_department_id
FROM
update_users_Cte
WHERE
employee_id = update_users_Cte.old_employee_id
This is the error:
ERROR: syntax error at or near "UPDATE"
LINE 41: UPDATE
I would like to be able to do both UPDATEs and use the CTE as I need to check it in both cases. I'm not sure if I have to wrap the whole thing in a transaction.
Any help would be appreciated.
new_user_id | old_user_id | new_employee_id | old_employee_id | new_display_name | old_display_name | new_first_name | old_first_name | new_last_name | old_last_name | new_phone_number | old_phone_number | new_updated_at | old_updated_at | new_job_type | old_job_type | new_department_id | old_department_id
-------------+-------------+-----------------+-----------------+------------------+------------------+----------------+----------------+---------------+---------------+------------------+------------------+---------------------+---------------------+--------------+---------------+-------------------+-------------------
474 | 19710 | 35275 | 035275 | | | David | David | Coyle | Coyle | +447584208902 | | 2017-06-22 17:09:43 | 2021-01-27 15:14:43 | | | 418 | 418
19701 | 432 | 21239 | 021239 | | | Piotr | Piotr | Mierniczek | Mierniczek | | +447404050330 | 2021-02-08 14:36:59 | 2017-06-22 17:09:42 | | | 249 | 73
19702 | 479 | 35568 | 035568 | | | Manjita | Manjita | Kunwar | Kunwar | | +447847370860 | 2021-01-15 15:51:44 | 2021-01-15 15:45:20 | | | 317 | 317
19707 | 19680 | 11111 | 011111 | | Sarika | Sarika | Sarika | Sharma | Sharma | | +447700000000 | 2021-01-20 12:46:09 | 2021-01-20 12:45:12 | | C.S. Employee | |

How to convert row into column in PostgreSQL of below table

I was trying to convert the trace table to resulted table in postgress. I have hug data in the table.
I have table with name : Trace
entity_id | ts | key | bool_v | dbl_v | str_v | long_v |
---------------------------------------------------------------------------------------------------------------
1ea815c48c5ac30bca403a1010b09f1 | 1593934026155 | temperature | | | | 45 |
1ea815c48c5ac30bca403a1010b09f1 | 1593934026155 | operation | | | Normal | |
1ea815c48c5ac30bca403a1010b09f1 | 1593934026155 | period | | | | 6968 |
1ea815c48c5ac30bca403a1010b09f1 | 1593933202984 | temperature | | | | 44 |
1ea815c48c5ac30bca403a1010b09f1 | 1593933202984 | operation | | | Reverse | |
1ea815c48c5ac30bca403a1010b09f1 | 1593933202984 | period | | | | 3535 |
Trace Table
convert the above table into following table in PostgreSQL
Output Table: Result
entity_id | ts | temperature | operation | period |
----------------------------------------------------------------------------------------|
1ea815c48c5ac30bca403a1010b09f1 | 1593934026155 | 45 | Normal | 6968 |
1ea815c48c5ac30bca403a1010b09f1 | 1593933202984 | 44 | Reverse | 3535 |
Result Table
Have you tried this yet?
select entity_id, ts,
max(long_v) filter (where key = 'temperature') as temperature,
max(str_v) filter (where key = 'operation') as operation,
max(long_v) filter (where key = 'period') as period
from trace
group by entity_id, ts;

T-SQL : Pivot table without aggregate

I am trying to understand how to pivot data within T-SQL but can't seem to get it working. I have the following table structure
+-------------------+-----------------------+
| Name | Value |
+-------------------+-----------------------+
| TaskId | 12417 |
| TaskUid | XX00044497 |
| TaskDefId | 23 |
| TaskStatusId | 4 |
| Notes | |
| TaskActivityIndex | 0 |
| ModifiedBy | Orange |
| Modified | /Date(1554540200000)/ |
| CreatedBy | Apple |
| Created | /Date(2121212100000)/ |
| TaskPriorityId | 40 |
| OId | 2 |
+-------------------+-----------------------+
I want to pivot the name column to be columns expected output
+--------+------------------------+-----------+--------------+-------+-------------------+------------+-----------------------+-----------+-----------------------+----------------+-----+
| TASKID | TASKUID | TASKDEFID | TASKSTATUSID | NOTES | TASKACTIVITYINDEX | MODIFIEDBY | MODIFIED | CREATEDBY | CREATED | TASKPRIORITYID | OID |
+--------+------------------------+-----------+--------------+-------+-------------------+------------+-----------------------+-----------+-----------------------+----------------+-----+
| | | | | | | | | | | | |
| 12417 | XX00044497 | 23 | 4 | | 0 | Orange | /Date(1554540200000)/ | Apple | /Date(2121212100000)/ | 40 | 2 |
+--------+------------------------+-----------+--------------+-------+-------------------+------------+-----------------------+-----------+-----------------------+----------------+-----+
Is there an easy way of doing it? The columns are fixed (not dynamic).
Any help appreciated
Try this:
select * from yourtable
pivot
(
min(value)
for Name in ([TaskID],[TaskUID],[TaskDefID]......)
) as pivotable
You can also use case statements.
You must use the aggregate function in the pivot table.
If you want to learn more, here is the reference:
https://learn.microsoft.com/en-us/sql/t-sql/queries/from-using-pivot-and-unpivot?view=sql-server-2017
Output (I only tried three columns):
DB<>Fiddle

PostgreSQL crosstab doesn't work as desired

In this example, I expect the resulting pivot table to have values for 4 columns, but instead there's only values for 2.
It should've returned something like this:
| time | trace1 | trace2 | trace3 | trace4 |
| -----------------------------------------|
| t | v | v | v | v |
| t | v | v | v | null |
| t | null | v | v | v |
| t | v | v | null | v |
| t | v | null | v | v |
|------------------------------------------|
but I got this instead:
| time | trace1 | trace2 | trace3 | trace4 |
| -----------------------------------------|
| t | v | v | null | null |
| t | v | v | null | null |
| t | v | v | null | null |
| t | v | null | null | null |
| t | v | null | null | null |
|------------------------------------------|
Even worse, if I remove
order by unixdatetime
, everything will be smashed into only 1 column as below:
| time | trace1 | trace2 | trace3 | trace4 |
| -----------------------------------------|
| t | v | null | null | null |
| t | v | null | null | null |
| t | v | null | null | null |
| t | v | null | null | null |
| t | v | null | null | null |
|------------------------------------------|
Here's the code:
select *
from crosstab(
$$
select
unixdatetime,
gaugesummaryid,
value::double precision
from
(values
(1546300800,187923,1.5),
(1546387200,187923,1.5),
(1546473600,187923,1.5),
(1546560000,187923,1.75),
(1546646400,187923,1.75),
(1546732800,187923,1.75),
(1546819200,187923,1.75),
(1546905600,187923,1.5),
(1546992000,187923,1.5),
(1547078400,187923,1.5),
(1547164800,187923,1.5),
(1547337600,187924,200),
(1547424000,187924,200),
(1547510400,187924,200),
(1547596800,187924,200),
(1547683200,187924,200),
(1547769600,187924,200),
(1547856000,187924,200),
(1547942400,187924,200),
(1548028800,187924,200),
(1548115200,187924,200),
(1548201600,187924,200),
(1548288000,187924,200),
(1546300800,187926,120),
(1546387200,187926,120),
(1546473600,187926,120),
(1546560000,187926,110),
(1546646400,187926,110),
(1546732800,187926,110),
(1546819200,187926,110),
(1546905600,187926,115),
(1546992000,187926,115),
(1547078400,187926,115),
(1547942400,187927,100),
(1548028800,187927,100),
(1548115200,187927,100),
(1548201600,187927,100),
(1548288000,187927,100)
) as t (unixdatetime, gaugesummaryid, value)
order by unixdatetime
$$
) as final_result (
unixdatetime int,
trace1 double precision,
trace2 double precision,
trace3 double precision,
trace4 double precision
);
Here's the link in case you'd like to play around:
https://dbfiddle.uk/?rdbms=postgres_11&fiddle=2c4f6098fb89b78898ba1bf6afa7f439
How to get the desired result?
I would recommend you to use filter (where ...) clause instead of a pivot table.
select
unixdatetime,
min(value) filter (where gaugesummaryid = 187923) as trace_1,
min(value) filter (where gaugesummaryid = 187924) as trace_2,
min(value) filter (where gaugesummaryid = 187926) as trace_3,
min(value) filter (where gaugesummaryid = 187927) as trace_4
from table
group by 1;
Note, that you have to use an aggregate function to be able to use the clause. In your case, it does not matter if you use min, max, avg or sum.
Use the 2-argument form of the crosstab function:
SELECT *
FROM crosstab(
$$
SELECT
unixdatetime,
gaugesummaryid,
value::double precision
FROM test
ORDER BY unixdatetime
$$
, 'SELECT DISTINCT gaugesummaryid FROM test ORDER BY 1 LIMIT 4'
) as final_result (
unixdatetime int,
trace1 double precision,
trace2 double precision,
trace3 double precision,
trace4 double precision
)
yields
| unixdatetime | trace1 | trace2 | trace3 | trace4 |
|--------------+--------+--------+--------+--------|
| 1546300800 | 1.5 | | 120 | |
| 1546387200 | 1.5 | | 120 | |
| 1546473600 | 1.5 | | 120 | |
| 1546560000 | 1.75 | | 110 | |
| 1546646400 | 1.75 | | 110 | |
| 1546732800 | 1.75 | | 110 | |
| 1546819200 | 1.75 | | 110 | |
| 1546905600 | 1.5 | | 115 | |
| 1546992000 | 1.5 | | 115 | |
| 1547078400 | 1.5 | | 115 | |
| 1547164800 | 1.5 | | | |
| 1547337600 | | 200 | | |
| 1547424000 | | 200 | | |
| 1547510400 | | 200 | | |
| 1547596800 | | 200 | | |
| 1547683200 | | 200 | | |
| 1547769600 | | 200 | | |
| 1547856000 | | 200 | | |
| 1547942400 | | 200 | | 100 |
| 1548028800 | | 200 | | 100 |
| 1548115200 | | 200 | | 100 |
| 1548201600 | | 200 | | 100 |
| 1548288000 | | 200 | | 100 |
Using this setup:
DROP TABLE IF EXISTS test;
CREATE TABLE test (
unixdatetime bigint,
gaugesummaryid int,
value double precision
);
INSERT INTO test VALUES
(1546300800,187923,1.5),
(1546387200,187923,1.5),
(1546473600,187923,1.5),
(1546560000,187923,1.75),
(1546646400,187923,1.75),
(1546732800,187923,1.75),
(1546819200,187923,1.75),
(1546905600,187923,1.5),
(1546992000,187923,1.5),
(1547078400,187923,1.5),
(1547164800,187923,1.5),
(1547337600,187924,200),
(1547424000,187924,200),
(1547510400,187924,200),
(1547596800,187924,200),
(1547683200,187924,200),
(1547769600,187924,200),
(1547856000,187924,200),
(1547942400,187924,200),
(1548028800,187924,200),
(1548115200,187924,200),
(1548201600,187924,200),
(1548288000,187924,200),
(1546300800,187926,120),
(1546387200,187926,120),
(1546473600,187926,120),
(1546560000,187926,110),
(1546646400,187926,110),
(1546732800,187926,110),
(1546819200,187926,110),
(1546905600,187926,115),
(1546992000,187926,115),
(1547078400,187926,115),
(1547942400,187927,100),
(1548028800,187927,100),
(1548115200,187927,100),
(1548201600,187927,100),
(1548288000,187927,100);
While some of the target values may be missing , you need the 2-argument form of crosstab() (like unutbu provided).
But it makes no sense to use a query producing unstable results as 2nd parameter. Use a VALUES expression (or similar) to provide a stable set of target columns in sync with the resulting column definition list. Like:
SELECT *
FROM crosstab(
$$
SELECT *
FROM (
VALUES
(bigint '1546300800', 187923, float8 '1.5')
, (1546387200,187923,1.5)
, (1546473600,187923,1.5)
-- , ...
, (1548288000,187927,100)
) t (unixdatetime, gaugesummaryid, value)
ORDER BY 1,2
$$
, 'VALUES (187923), (187924), (187926), (187927)' -- !!
) final_result (unixdatetime int
, trace1 float8
, trace2 float8
, trace3 float8
, trace4 float8);
db<>fiddle here
Detailed explanation:
PostgreSQL Crosstab Query
It would be nice to get results for a dynamic number of target columns from a single query. Alas, SQL does not work like that. There are various workarounds. See:
Execute a dynamic crosstab query

Optimization of Sybase 15.5 union query

im having trouble trying to optimize the following query on Sybase 15.5. Does anyone know how could i improve it. Each one of the tables used there have about 30 million rows each. I tried my best to optimize it but still taking lot of time(1.5 hours).
create table #tmp1( f_id smallint, a_date smalldatetime )
create table #tmp2( f_id smallint, a_date smalldatetime )
insert #tmp1
select f_id, a_date = max( a_date )
FROM audit_table
WHERE i_date = #pIDate
group by f_id
insert #tmp2
select f_id , a_date = max( a_date )
FROM n_audit_table
WHERE i_date = #pIDate
group by f_id
create table #tmp(
t_account varchar(32) not null,
t_id varchar(32) not null,
product varchar(64) null
)
insert into #tmp
select t_account,t_id, product
FROM audit_table nt, #tmp1 a
WHERE i_date = #pIDate
and nt.a_date = a.a_date
and nt.f_id = a.f_id
union
select t_account,t_id, product
FROM n_audit_table t, #tmp2 a
WHERE t.item_date = #pIDate
and t.a_date = a.a_date
and t.f_id = a.f_id
Both the tables having indexes on i_date, a_date, f_id. Please find below showplan where it is long time.
QUERY PLAN FOR STATEMENT 2 (at line 24).
Optimized using Serial Mode
STEP 1
The type of query is INSERT.
10 operator(s) under root
|ROOT:EMIT Operator (VA = 10)
|
| |INSERT Operator (VA = 9)
| | The update mode is direct.
| |
| | |HASH UNION Operator (VA = 8) has 2 children.
| | | Using Worktable1 for internal storage.
| | | Key Count: 3
| | |
| | | |NESTED LOOP JOIN Operator (VA = 3) (Join Type: Inner Join)
| | | |
| | | | |SCAN Operator (VA = 0)
| | | | | FROM TABLE
| | | | | #tmp1
| | | | | a
| | | | | Table Scan.
| | | | | Forward Scan.
| | | | | Positioning at start of table.
| | | | | Using I/O Size 2 Kbytes for data pages.
| | | | | With LRU Buffer Replacement Strategy for data pages.
| | | |
| | | | |RESTRICT Operator (VA = 2)(5)(0)(0)(0)(0)
| | | | |
| | | | | |SCAN Operator (VA = 1)
| | | | | | FROM TABLE
| | | | | | audit_table
| | | | | | nt
| | | | | | Index : IX_audit_table
| | | | | | Forward Scan.
| | | | | | Positioning by key.
| | | | | | Keys are:
| | | | | | i_date ASC
| | | | | | a_date ASC
| | | | | | Using I/O Size 2 Kbytes for index leaf pages.
| | | | | | With LRU Buffer Replacement Strategy for index leaf pages.
| | | | | | Using I/O Size 2 Kbytes for data pages.
| | | | | | With LRU Buffer Replacement Strategy for data pages.
| | |
| | | |NESTED LOOP JOIN Operator (VA = 7) (Join Type: Inner Join)
| | | |
| | | | |SCAN Operator (VA = 4)
| | | | | FROM TABLE
| | | | | #tmp2
| | | | | a
| | | | | Table Scan.
| | | | | Forward Scan.
| | | | | Positioning at start of table.
| | | | | Using I/O Size 2 Kbytes for data pages.
| | | | | With LRU Buffer Replacement Strategy for data pages.
| | | |
| | | | |RESTRICT Operator (VA = 6)(5)(0)(0)(0)(0)
| | | | |
| | | | | |SCAN Operator (VA = 5)
| | | | | | FROM TABLE
| | | | | | n_audit_table
| | | | | | t
| | | | | | Index : IX_n_audit_table
| | | | | | Forward Scan.
| | | | | | Positioning by key.
| | | | | | Keys are:
| | | | | | i_date ASC
| | | | | | a_date ASC
| | | | | | Using I/O Size 2 Kbytes for index leaf pages.
| | | | | | With LRU Buffer Replacement Strategy for index leaf pages.
| | | | | | Using I/O Size 2 Kbytes for data pages.
| | | | | | With LRU Buffer Replacement Strategy for data pages.
| |
| | TO TABLE
| | #tmp
| | Using I/O Size 2 Kbytes for data pages.
Total estimated I/O cost for statement 2 (at line 24): 29322945.
I doubt its a union issue. Queries are more probable troublemaker.
I suppose you should start from adding indexes on your temp tables:
create table #tmp1( f_id smallint, a_date smalldatetime )
Create clustered index IX1Temp on #tmp1(f_id )
Create clustered index IX2Temp on #tmp1(a_date )
...
Also, I see not much sense in #tmp1, #tmp2 the way you use them. You could call CTE instead. Also. I would recommend you to try PARTITION BY instead GROUP BY statement.
According to the query execution plan, the problem is the table scans on the temporary tables.
Please get the execution plan for the following query:
insert into #tmp
select t_account,t_id, product
FROM
audit_table nt,
(
select f_id, a_date = max(a_date)
FROM audit_table
WHERE i_date = #pIDate
group by f_id
) a
WHERE
i_date = #pIDate
and nt.a_date = a.a_date
and nt.f_id = a.f_id
union
select t_account,t_id, product
FROM
n_audit_table t,
(
select f_id , a_date = max( a_date )
FROM n_audit_table
WHERE i_date = #pIDate
group by f_id
) a
WHERE
t.item_date = #pIDate
and t.a_date = a.a_date
and t.f_id = a.f_id
How many rows end up in each of the temporary tables?
Looks like the temporary tables could be replaced by using HAVING, I would need to test it, it is always complicated when your group by is on a single column and you require more columns in the output.
Try running this statement with SET STATISTICS PLANCOST ON and SET STATISTICS IO ON as that would give a good idea of the number of pages that are scanned and if Sybase is going wrong somewhere while optimising the query.