How Dynamicaly columns in UNPIVOT operator - tsql

I currently have the following query:
WITH History AS (
SELECT
kz.*,
kz.__$operation AS operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT
FROM cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all') AS kz
INNER JOIN [cdc].[lsn_time_mapping] map
ON kz.[__$start_lsn] = map.start_lsn
where kz.GUID_BalanceHC_Zalezh = 'DDA9AB3A-A0AF-4623-9362-0000C8C83D63'
),
UnpivotedValues AS(
SELECT guid, GUID_another, field, val, operation, beginT, endT
FROM History
UNPIVOT ( [val] FOR field IN
(
area,
oilwidthmin,
oilwidthmax,
efectivwidthmin,
efectivwidthmax,
etc...
))t
),
UnpivotedWithLastValue AS (
SELECT
*,
--Use LAG() to get the last value for the same field
LAG(val, 1) OVER (PARTITION BY guid, GUID_another, field ORDER BY BeginT) LastVal
FROM UnpivotedValues
)
SELECT * FROM UnpivotedWithLastValue WHERE val <> LastVal OR LastVal IS NULL ORDER BY guid
This query returns the changed values for a single table that has CDC (Change Data Capture) enabled.
I want to create a stored procedure that receives the columns to be unpivoted, and the cdc function (e.g. cdc.fn_cdc_get_all_...) as parameters and returns the result set.
The result for this tables must be joined in one report.
In my case parameter 1 is cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all'). This is the CDC function.
How should I send the list of fields that i want in the result? How's the string?
Also, is there a way to do without dynamic SQL? Dynamic SQL it is not better solution for performance.

As you know SQL Server is declarative by design and does not support macro substitution.
UNPIVOT would clearly be more performant, but here is a simplified example of a UNPIVOT which does not require Dynamic SQL, but only a little XML.
Example
Let's assume your table/results looks like this:
You may notice that I only we only specify key fields to EXCLUDE in the final WHERE
Declare #YourData table (ID int,Active bit,First_Name varchar(50),Last_Name varchar(50),EMail varchar(50),Salary decimal(10,2))
Insert into #YourData values
(1,1,'John','Smith','john.smith#email.com',85600),
(2,0,'Jane','Doe' ,'jane.doe#email.com',83200)
;with cte as (
-- Replace with your Complex Query
Select * from #YourData
)
Select A.ID
,A.Active
,C.*
From cte A
Cross Apply (Select XMLData=cast((Select A.* for XML RAW) as xml)) B
Cross Apply (
Select Item = attr.value('local-name(.)','varchar(100)')
,Value = attr.value('.','varchar(max)')
From XMLData.nodes('/row') C1(n)
Cross Apply C1.n.nodes('./#*') C2(attr)
Where attr.value('local-name(.)','varchar(100)') not in ('ID','Active')
) C
Returns

Related

How to translate SQL to DAX, Need to add FILTER

I want to create calculated table that will summarize In_Force Premium from existing table fact_Premium.
How can I filter the result by saying:
TODAY() has to be between `fact_Premium[EffectiveDate]` and (SELECT TOP 1 fact_Premium[ExpirationDate] ORDE BY QuoteID DESC)
In SQL I'd do that like this:
`WHERE CONVERT(date, getdate()) between CONVERT(date, tblQuotes.EffectiveDate)
and (
select top 1 q2.ExpirationDate
from Table2 Q2
where q2.ControlNo = Table1.controlno
order by quoteid` desc
)
Here is my DAX statement so far:
In_Force Premium =
FILTER(
ADDCOLUMNS(
SUMMARIZE(
//Grouping necessary columns
fact_Premium,
fact_Premium[QuoteID],
fact_Premium[Division],
fact_Premium[Office],
dim_Company[CompanyGUID],
fact_Premium[LineGUID],
fact_Premium[ProducerGUID],
fact_Premium[StateID],
fact_Premium[ExpirationDate]
),
"Premium", CALCULATE(
SUM(fact_Premium[Premium])
),
"ControlNo", CALCULATE(
DISTINCTCOUNT(fact_Premium[ControlNo])
)
), // Here I need to make sure TODAY() falls between fact_Premium[EffectiveDate] and (SELECT TOP 1 fact_Premium[ExpirationDate] ORDE BY QuoteID DESC)
)
Also, what would be more efficient way, to create calculated table from fact_Premium or create same table using sql statement (--> Get Data--> SQL Server) ?
There are 2 potential ways in T-SQL to get the next effective date. One is to use LEAD() and another is to use an APPLY operator. As there are few facts to work with here are samples:
select *
from (
select *
, lead(EffectiveDate) over(partition by CompanyGUID order by quoteid desc) as NextEffectiveDate
from Table1
join Table2 on ...
) d
or
select table1.*, oa.NextEffectiveDate
from Table1
outer apply (
select top(1) q2.ExpirationDate AS NextEffectiveDate
from Table2 Q2
where q2.ControlNo = Table1.controlno
order by quoteid desc
) oa
nb. an outer apply is a little similar to a left join in that it will allow rows with a NULL to be returned by the query, if that is not needed than use cross apply instead.
In both these approaches you may refer to NextEffectiveDate in a final where clause, but I would prefer to avoid using the convert function if that is feasible (this depends on the data).

Tracking changes for field using CDC or T-SQL with CDC

Folowing CDC results set for update operation. Only area field has been updated.
On previous sreenshot only part of gields that contains in table. Fields are much more. Time to time some from them updates some not. In following query i try to show statistic of changes by fields in usable view.
with History AS (
SELECT
cz.GUID as Id,
cz.category,
isnull(cz.area, 0) as area,
isnull(cz.oilwidthmin,0) as oilwidthmin,
isnull(cz.oilwidthmax,0) as oilwidthmax,
isnull(cz.efectivwidthmin,0) as efectivwidthmin,
isnull(cz.efectivwidthmax,0) as efectivwidthmax,
isnull(cz.koafporistmin,0) as koafporistmin,
isnull(cz.koafporistmax,0) as koafporistmax,
CASE cz.__$operation
WHEN 1 THEN 'DELETE'
WHEN 2 THEN 'INSERT'
WHEN 3 THEN 'Before UPDATE'
WHEN 4 THEN 'After UPDATE'
END operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT
FROM cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all') AS cz
INNER JOIN [cdc].[lsn_time_mapping] map
ON cz.[__$start_lsn] = map.start_lsn
)
SELECT field, val, operation, beginT, endT FROM History
unpivot ( [val] for field in
(
--category,
area,
oilwidthmin,
oilwidthmax,
efectivwidthmin,
efectivwidthmax,
koafporistmin,
koafporistmax))t where id = '2D166098-7CBD-4622-9EB0-000070506FE6'
Result of the query is following:
But previous result contains extra data.
Expected result must be following:
I know that the CDC tracks changes by rows. Or maybe i'm wrong? If no, how can i doing some comparer for val field in SQL. I have not deep knowledge in t-sql, and everything that comes to my mind is somehow used by cursors. Any ideas?
Maybe somehow use CT (change tracking)? Maybe somehow using group by?
Almost correct answer. Folowing query return expected result:
WITH History AS (
SELECT
*,
CASE cz.__$operation
WHEN 1 THEN 'DELETE'
WHEN 2 THEN 'INSERT'
WHEN 3 THEN 'Before UPDATE'
WHEN 4 THEN 'After UPDATE'
END operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT
FROM cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all') AS cz
INNER JOIN [cdc].[lsn_time_mapping] map
ON cz.[__$start_lsn] = map.start_lsn
where cz.GUID = '2D166098-7CBD-4622-9EB0-000070506FE6'
),
UnpivotedValues AS(
SELECT guid, field, val, operation, beginT, endT
FROM History
UNPIVOT ( [val] FOR field IN
(
area,
oilwidthmin,
oilwidthmax,
efectivwidthmin,
efectivwidthmax,
koafporistmin,
koafporistmax
))t
),
UnpivotedWithLastValue AS (
SELECT
*,
--Use LAG() to get the last value for the same field
LAG(val, 1) OVER (PARTITION BY field ORDER BY BeginT) LastVal
FROM UnpivotedValues
)
--Filter out record where the value equals the last value for the same field
SELECT * FROM UnpivotedWithLastValue WHERE val <> LastVal OR LastVal IS NULL ORDER BY guid
Result of this query looks like this:
But when WHERE cz.GUID = is absent or if query using several GUIDs in WHERE predicate i get folowing result:
This result for two GUIDs. Value for LastVal in first row must be 16691. Like val from row 4.
You cannot set up CDC to only track values for the columns that have changed. However, you can quite easily filter out the unchanged values in your query.
Consider the following query, which is a simplified copy of your original query:
WITH History AS (
SELECT
*,
CASE cz.__$operation
WHEN 1 THEN 'DELETE'
WHEN 2 THEN 'INSERT'
WHEN 3 THEN 'Before UPDATE'
WHEN 4 THEN 'After UPDATE'
END operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT
FROM cdc.fn_cdc_get_all_changes_Dbo_YourTable(sys.fn_cdc_get_min_lsn('Dbo_YourTable'), sys.fn_cdc_get_max_lsn(), 'all') AS cz
INNER JOIN [cdc].[lsn_time_mapping] map
ON cz.[__$start_lsn] = map.start_lsn
),
UnpivotedValues AS(
SELECT id, field, val, operation, beginT, endT, t.tran_id
FROM History
UNPIVOT ( [val] FOR field IN
(Column1, Column2, Column3))t
),
UnpivotedWithLastValue AS (
SELECT
*,
--Use LAG() to get the last value for the same field
LAG(val, 1) OVER (PARTITION BY id, field ORDER BY BeginT) LastVal
FROM UnpivotedValues
)
--Filter out record where the value equals the last value for the same field
SELECT * FROM UnpivotedWithLastValue WHERE val <> LastVal OR LastVal IS NULL
ORDER BY Id, beginT
In this query I have used the LAG() function to get the last value for each field. Based on this value you can filter out the unchanged records in your final query, and shown above.
In your case, you can use the ROW_NUMBER function to sequentially number the changes - after this, you can join each sequential change with the previous one (based on field and id) and output only the rows with the diffent value.
Something like this:
WITH
History AS
(
SELECT
cz.GUID as Id,
cz.category,
isnull(cz.area, 0) as area,
isnull(cz.oilwidthmin,0) as oilwidthmin,
isnull(cz.oilwidthmax,0) as oilwidthmax,
isnull(cz.efectivwidthmin,0) as efectivwidthmin,
isnull(cz.efectivwidthmax,0) as efectivwidthmax,
isnull(cz.koafporistmin,0) as koafporistmin,
isnull(cz.koafporistmax,0) as koafporistmax,
CASE
cz.__$operation
WHEN 1 THEN 'DELETE'
WHEN 2 THEN 'INSERT'
WHEN 3 THEN 'Before UPDATE'
WHEN 4 THEN 'After UPDATE'
END operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT,
ROW_NUMBER() OVER (PARTITION BY cz.GUID ORDER BY map.tran_end_time ASC) as rn
FROM
cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all') AS cz
INNER JOIN
[cdc].[lsn_time_mapping] map ON cz.[__$start_lsn] = map.start_lsn
),
History2 AS
(
SELECT id, field, val, operation, beginT, endT, rn FROM History
unpivot ( [val] for field in
(
--category,
area,
oilwidthmin,
oilwidthmax,
efectivwidthmin,
efectivwidthmax,
koafporistmin,
koafporistmax))t
where id = '2D166098-7CBD-4622-9EB0-000070506FE6'
)
-- return the values that were inserted first
SELECT
a.*
FROM
History2 a
WHERE
a.rn=1
UNION ALL
-- ... and then return only the values that are different from the previous ones
SELECT
a.*
FROM
History2 a
INNER JOIN
History2 b ON a.id = b.id AND a.field=b.field AND a.rn = b.rn-1 AND a.value<>b.value
WHERE
a.rn>1
BTW; You could also configure CDC to track changes only in certain columns, not in the entire table. Look at the #captured_column_list of the sys.sp_cdc_enable_table stored procedure.

Display formatted JSON in SSRS report

I have a table where one field is JSON string.
"CX.UW.001": "03/08/2017", "CX.UW.001.AUDIT": "admin",
I want to produce an SSRS report where it appears in readable format like:
CX.UW.001: 03/08/2017
CX.UW.001.AUDIT: admin
Is it possible?
If you are looking for multiple records, just about any parse/split function will do, or you can use a simple CROSS APPLY in concert with a little XML
Declare #YourTable table (ID int, JSON varchar(max))
Insert Into #YourTable values
(1,'"CX.UW.001": "03/08/2017", "CX.UW.001.AUDIT": "admin"')
Select A.ID
,DisplayAs = replace(B.RetVal,'"','')
From #YourTable A
Cross Apply (
Select RetSeq = Row_Number() over (Order By (Select null))
,RetVal = LTrim(RTrim(B.i.value('(./text())[1]', 'varchar(max)')))
From (Select x = Cast('<x>' + replace((Select replace(A.JSON,',','§§Split§§') as [*] For XML Path('')),'§§Split§§','</x><x>')+'</x>' as xml).query('.')) as X
Cross Apply x.nodes('x') AS B(i)
) B
Returns
ID DisplayAs
1 CX.UW.001: 03/08/2017
1 CX.UW.001.AUDIT: admin
Or If you want the string to wrap
Select A.ID
,DisplayAs = replace(replace(JSON,',',char(13)),'"','')
From #YourTable A
Returns
1 CX.UW.001: 03/08/2017
CX.UW.001.AUDIT: admin
Right click that field, choose expression, locate Text from Common Functions category, use Replace function, should be the syntax like :
Replace (Fields!Yours.Value.Value,"""","")
Or in TSQL:
Select Replace(JSON_COLUMN,'"','')
From table

TSQL Update Query behaving unexpectedly

I have a nested select query that is returning the proper amount of rows. The query builds a recordset and compares it to a table and returns the records in the query that are not in the table.
I converted the select query to an update query. I am trying to populate the table with the rows returned from the query. When I run the update query it is returning with zero rows to update. I dont understand why because the select query is returning record and I am using the same code in the update query.
Thanks
Select Query: (This is returning several records)
Select *
From
(SELECT DISTINCT
ProductClass,SalProductClass.[Description],B.Branch,B.BranchDesc,B.Salesperson,B.Name,
CAST(0 AS FLOAT) AS Rate,'N' AS Split
FROM (SELECT SalBranch.Branch,SalBranch.[Description] AS BranchDesc,A.Salesperson,A.Name
FROM (SELECT DISTINCT
Salesperson,Name
FROM SalSalesperson
) A
CROSS JOIN SalBranch
) B
CROSS JOIN SalProductClass
) C
Left Outer Join RateComm On
RateComm.ProductClass = C.ProductClass and
RateComm.Branch = C.Branch And RateComm.Salesperson = C.Salesperson
Where RateComm.ProductClass is Null
Update Query: (This is returning zero records)
UPDATE RateComm
SET RateComm.ProductClass=C.ProductClass,RateComm.ProdClassDesc=C.ProdClassDesc,
RateComm.Branch=C.Branch,RateComm.BranchDesc=C.BranchDesc,RateComm.Salesperson=C.Salesperson,
RateComm.Name=C.Name,RateComm.Rate=C.Rate,RateComm.Split=C.Split
FROM (SELECT DISTINCT
ProductClass,SalProductClass.[Description] AS ProdClassDesc,B.Branch,B.BranchDesc,B.Salesperson,B.Name,
CAST(0 AS FLOAT) AS Rate,'N' AS Split
FROM (SELECT SalBranch.Branch,SalBranch.[Description] AS BranchDesc,A.Salesperson,A.Name
FROM (SELECT DISTINCT
Salesperson,Name
FROM SalSalesperson
) A
CROSS JOIN SalBranch
) B
CROSS JOIN SalProductClass
) C
LEFT OUTER JOIN RateComm ON C.ProductClass=RateComm.ProductClass AND
C.Salesperson=RateComm.Salesperson AND C.Branch=RateComm.Branch
WHERE RateComm.ProductClass IS NULL
It's difficult to update what doesn't exist. Have you tried an INSERT query instead?

Filter union result

I'm making select with a union.
SELECT * FROM table_1
UNION
SELECT * FROM table_2
Is it possible to filter query results by column values?
Yes, you can enclose your entire union inside another select:
select * from (
select * from table_1 union select * from table_2) as t
where t.column = 'y'
You have to introduce the alias for the table ("as t"). Also, if the data from the tables is disjoint, you might want to consider switching to UNION ALL - UNION by itself works to eliminate duplicates in the result set. This is frequently not necessary.
A simple to read solution is to use a CTE (common table expression). This takes the form:
WITH foobar AS (
SELECT foo, bar FROM table_1
UNION
SELECT foo, bar FROM table_2
)
Then you can refer to the CTE in subsequent queries by name, as if it were a normal table:
SELECT foo,bar FROM foobar WHERE foo = 'value'
CTEs are quite powerful, I recommend further reading here
One tip that you will not find in that MS article is; if you require more than one CTE put a comma between the expression statements. eg:
WITH foo AS (
SELECT thing FROM place WHERE field = 'Value'
),
bar AS (
SELECT otherthing FROM otherplace WHERE otherfield = 'Other Value'
)
If you want to filter the query based on some criteria then you could do this -
Select * from table_1 where table_1.col1 = <some value>
UNION
Select * from table_2 where table_2.col1 = <some value>
But, I would say if you want to filter result to find the common values then you can use joins instead
Select * from table_1 inner join table_2 on table_1.col1 = table_2.col1