hive window function - row change in value - date

I have data which has an id /flag and date fields
I need to populate flag_date field in below way
login_date id flag flag_date
5/1/2018 100 N NULL
5/2/2018 100 N NULL
5/3/2018 100 Y 5/3/2018
5/4/2018 100 Y 5/3/2018
5/5/2018 100 Y 5/3/2018
5/6/2018 100 N NULL
5/7/2018 100 N NULL
5/8/2018 100 Y 5/8/2018
5/9/2018 100 Y 5/8/2018
5/10/2018 100 Y 5/8/2018
When Flag value changes to N to Y, flag_date value changes accordingly.
please help.

select login_date
,id
,flag
,case when flag = 'Y' then min(login_date) over(partition by id,grp) end as flag_date
from (select login_date,id,flag
,row_number() over(partition by id order by login_date) -
row_number() over(partition by id,flag order by login_date) as grp
from tbl
) t
First classify rows into groups, i.e. consecutive 'Y's and 'N's starting a new value when the series is broken. This can be done with a difference of row numbers approach. (Run the inner query to see how group numbers are assigned)
Once the groups are assigned, it is trivial to compute flag_date with conditional aggregation.
One more approach to solve this involves generating a new group whenever a 'N' value is encountered. The outer query remains the same, only the inner one changes.
select login_date
,id
,flag
,case when flag = 'Y' then min(login_date) over(partition by id,grp) end as flag_date
from (select login_date,id,flag
,sum(case when flag = 'N' then 1 else 0 end) over(partition by id order by login_date) as grp
from tbl
) t

Related

In PostgreSQL, how can I optimize a query with which I obtain the differences between the current column and the immediately previous one?

I have this audit table
User
date
text
text 2
u1
2023-01-01
hi
yes
u1
2022-12-20
hi
no
u1
2022-12-01
hello
maybe
And I need as a result, something like this:
User
date
text
text 2
u1
2023-01-01
null
x
u1
2022-12-20
x
x
u1
2022-12-01
null
null
So I can know which column changed from the last time.
Something like this is working, but I think may be a way to optimize it? or at least generate a "more easy to look" query? (i need the information for almost 20 columns, not only 3)
SELECT
ta.audit_date,
ta.audit_user,
CASE
WHEN ta.audit_operation = 'I' THEN 'Insert'
WHEN ta.audit_operation = 'U' THEN 'Update'
END AS action,
CASE WHEN ta.column1 <> (SELECT column1
FROM audit_table ta1
WHERE ta1.id = 9207 AND ta1.audit_date < ta.audit_date
ORDER BY ta1.audit_date DESC
LIMIT 1)
THEN 'X' ELSE null END column1,
CASE WHEN ta.column2 <> (SELECT column2
FROM audit_table ta1
WHERE ta1.id = 9207 AND ta1.audit_date < ta.audit_date
ORDER BY ta1.audit_date DESC
LIMIT 1)
THEN 'X' ELSE null END column2,
CASE WHEN ta.column3 <> (SELECT column3
FROM audit_table ta1
WHERE ta1.id = 9207 AND ta1.audit_date < ta.audit_date
ORDER BY ta1.audit_date DESC
LIMIT 1)
THEN 'X' ELSE null END column3
FROM
audit_table ta
WHERE
ta.id = 9207
ORDER BY
audit_date DESC
Thank you!
I think you can just use the LAG() analytic function here. If I understand correctly:
SELECT *, CASE WHEN text != LAG(text) OVER (ORDER BY date) THEN 'x' END AS text_label,
CASE WHEN text2 != LAG(text) OVER (ORDER BY date) THEN 'x' END AS text2_label
FROM yourTable
ORDER BY date;

Sql Server - Running Totals Based on Conditions

I have been banging my head trying to come up with the correct logic (SQL Server 2012) needed to achieve something I would imagine would be fairly routine but I have been unable to find any examples of this anywhere. Basically, I have 3 columns in a table: product, flag, value. It is possible for a product to be listed multiple times within the table but only once with a unique flag (i.e. product1 can have flag1 or flag2 with different/identical but there will never be 2 records with product1 and flag1 and different/identical values).
The flag represents a pre-defined value (1,2,3,4) and the intention behind this field is to be able to assign a unique mathematical equation based on the value of the flag. The end result would yield a single product, the unique flag, and a new cumulative total based on the mathematical equation output. For instance, let's say product1 was listed 4 times with flag values of flag1, flag2, flag3, flag4 (see below):
Product-----Flag-----Value
Product1----Flag1----1.00
Product1----Flag2----3.00
Product1----Flag3----5.00
Product1----Flag4----7.00
Product-----Flag-----Value
Product1----Flag1----1.00 (flag1 value)
Product1----Flag2----4.00 (flag1+flag2 value)
Product1----Flag3----6.00 (flag1+flag3 value)
Product1----Flag4----10.00 (flag2+flag4 value)
Flag1 is defined as add flag1 only. Flag2 is defined as add flag1 and flag2. Flag 3 is defined as add flag1 and flag 3. Flag 4 is defined as add flag2 and flag4. the new output would be product1 listed four times with flag values of flag1, flag2, flag3, flag4 but new values as flag1, flag1_flag2, flag1+flag3, flag2+flag4.
I have tried to apply the logic via a case statement but I can't figure out how to traverse all the products for each condition and I have tried to go with a running totals solution but I am not sure how to incorporate the flag condition into it so it only performs a running total for when those conditions are true. Any assistance and/or article to help get me going down the right path would be greatly appreciated.
While I'm not sure I fully understand your question I think this might be what you want. For this to work it assumes flag1 is always present when flags 1 through 3 are and that flag2 is present when flag4 is.
;with cte as (
select
product,
max(case when flag = 'Flag1' then Value end) as f1Value,
max(case when flag = 'Flag2' then Value end) as f2Value,
max(case when flag = 'Flag3' then Value end) as f3Value,
max(case when flag = 'Flag4' then Value end) as f4Value
from flags group by Product
)
select
flags.Product,
flags.Flag,
flags.Value as "Org. value",
case flag
when 'Flag1' then f1Value
when 'Flag2' then f1Value + f2Value
when 'Flag3' then f1Value + f3Value
when 'Flag4' then f2Value + f4Value
else flags.Value -- take the present value when flag is not Flag1-4
end as "New value"
from flags
inner join cte on flags.Product = cte.Product
Take a look at this Sample SQL Fiddle to see it in action.
You can join a table to itself, and pick the conditions appropriately:
SELECT p1.product,p1.Flag,p1.Value + COALESCE(p2.Value,0)
FROM
Products p1
left join
Products p2
on
p1.Product = p2.Product and
p2.Flag = CASE p1.Flag
--1 doesn't need a previous value
WHEN 2 THEN 1
WHEN 3 THEN 1
WHEN 4 THEN 2
END
I assumed and tried on Range values.
CREATE TABLE #tmp (Product VARCHAR(10), flag VARCHAR(10),value numeric(13,2))
GO
INSERT INTO #tmp
SELECT 'Product1' , 'Flag1',1
UNION
SELECT 'Product1' , 'Flag2',3
UNION
SELECT 'Product1' , 'Flag3',5
UNION
SELECT 'Product1' , 'Flag4',7
GO
;WITH cte
AS
(
SELECT row_number () OVER(
ORDER BY flag) 'row',*
FROM #tmp
)
SELECT *,value 'RT'
FROM cte
WHERE row = 1
UNION
SELECT * ,(
SELECT cte.value
FROM cte
WHERE row = 1
) + value 'RT'
FROM cte
WHERE row BETWEEN 2
AND 3
UNION
SELECT * ,(
SELECT cte.value
FROM cte
WHERE row =2
) + value 'RT'
FROM cte
WHERE row >3
GO
DROP TABLE #tmp

GROUP BY getting the second highest date

I'm currently doing this group by to retrieve the max date :
SELECT A, MAX(B) FROM X GROUP BY A
This is perfectly working. However, when I try to retrieve the second highest value, I'm totally lost.
If anyone has an idea...
Try this:
SELECT X.A,
MAX(X.B)
FROM YourTable X
JOIN
(
SELECT
X1.A,
MAX(X1.B)
FROM YourTable X1
GROUP BY X1.A
) X1 ON X1.A = X.A
AND X.B < X1.B
GROUP BY X.A
Basically this says get the max of all the ones that are less than the max.
You can use the ranking function ROW_NUMBER in a cte:
WITH CTE AS
(
SELECT A,
MaxB = MAX(B)OVER(PARTITION BY A),
RN = ROW_NUMBER() OVER (PARTITION BY A ORDER By B DESC)
FROM dbo.X
)
SELECT A, MaxB
FROM CTE
WHERE RN <= 2
This will return the two highest values for each group (if that is what you want).
You're columns are rather ambiguous, but if A is max_date then, B is some other value you wish to sort by, then one way to do it could be:
SELECT A FROM X ORDER BY B DESC LIMIT 2
Which will give you 2 rows with the second highest displayed first.

How to match records for two different groups?

I have one main table called Event_log which contains all of the records that I need for this query. Within this table there is one column that I'm calling "Grp". To simplify things, assume that there are only two possible values for this Grp: A and B. So now we have one table, Event_log, with one column "Grp" and one more column called "Actual Date". Lastly I want to add one more Flag column to this table, which works as follows.
First, I order all of the records in descending order by date as demonstrated below. Then, I want to flag each Group "A" row with a 1 or a 0. For all "A" rows, if the previous record (earlier in date) = "B" row then I want to flag 1. Otherwise flag a 0. So this initial table looks like this before setting this flag:
Actual Date Grp Flag
1-29-13 A
12-27-12 B
12-26-12 B
12-23-12 A
12-22-12 A
But after these calculations are done, it should look like this:
Actual Date Grp Flag
1-29-13 A 1
12-27-12 B NULL
12-26-12 B NULL
12-23-12 A 0
12-22-12 A 0
How can I do this? This is simpler to describe than it is to query!
You can use something like:
select el.ActualDate
, el.Grp
, Flag = case
when el.grp = 'B' then null
when prev.grp = 'B' then 1
else 0
end
from Event_log el
outer apply
(
select top 1 prev.grp
from Event_log prev
where el.ActualDate > prev.ActualDate
order by prev.ActualDate desc
) prev
order by el.ActualDate desc
SQL Fiddle with demo.
Try this
;with cte as
(
SELECT CAST('01-29-13' As DateTime) ActualDate,'A' Grp
UNION ALL SELECT '12-27-12','B'
UNION ALL SELECT '12-26-12','B'
UNION ALL SELECT '12-23-12','A'
UNION ALL SELECT '12-22-12','A'
)
, CTE2 as
(
SELECT *, ROW_NUMBER() OVER (order by actualdate desc) rn
FROM cte
)
SELECT a.*,
case
when A.Grp = 'A' THEN
CASE WHEN b.Grp = 'B' THEN 1 ELSE 0 END
ELSE NULL
END Flag
from cte2 a
LEFT OUTER JOIN CTE2 b on a.rn + 1 = b.rn

TSQL - COUNT number of rows in a different state than current row

It's kind of hard to explain, but from this example it should be clear.
Table TABLE:
Name State Time
--------------------
A 1 1/4/2012
B 0 1/3/2012
C 0 1/2/2012
D 1 1/1/2012
Would like to
select * from TABLE where state=1 order by Time desc
plus an additional column 'Skipped' containing the number of rows after one where state=1 in state 0, in other words the output should look like this:
Name State Time Skipped
A 1 1/4/2012 2 -- 2 rows after A where State != 1
D 1 1/1/2012 0 -- 0 rows after D where State != 1
0 should also be reported in case of 2 consecutive rows are in state = 1, i.e. there is nothing between these rows in a state other than 1.
It seems like CTE are must here, but can't figure out how to count rows where state != 1.
Any help will be appreciated.
(MS Sql Server 2008)
I've used a CTE to establish RowNo, so that you're not dependent on consecutive dates:
WITH CTE_Rows as
(
select name,state,time,
rowno = ROW_NUMBER() over (order by [time])
from MyTable
)
select name,state,time,
gap = isnull(r.rowno - x.rowno - 1,0)
from
CTE_Rows r
outer apply (
select top 1 rowno
from CTE_Rows sub
where sub.rowno < r.rowno and sub.state = 1
order by sub.rowno desc) x
where r.state = 1
If you just want to do it by date, then its simpler - just need an outer apply:
select name,state,r.time,
gap = convert(int,isnull(r.time - x.time - 1,0))
from
MyTable r
outer apply (
select top 1 time
from MyTable sub
where sub.time < r.time and sub.state = 1
order by sub.time desc) x
where r.state = 1
FYI the test data is used was created as follows:
create table MyTable
(Name char(1), [state] tinyint, [Time] datetime)
insert MyTable
values
('E',1,'2012-01-05'),
('A',1,'2012-01-04'),
('B',0,'2012-01-03'),
('C',0,'2012-01-02'),
('D',1,'2012-01-01')
Okay, here you go (it gets a little messy):
SELECT U.CurrentTime,
(SELECT COUNT(*)
FROM StateTable AS T3
WHERE T3.State=0
AND T3.Time BETWEEN U.LastTime AND U.CurrentTime) AS Skipped
FROM (SELECT T1.Time AS CurrentTime,
(SELECT TOP 1 T2.Time
FROM StateTable AS T2
WHERE T2.Time < T1.Time AND T2.State=1
ORDER BY T2.Time DESC) AS LastTime
FROM StateTable AS T1 WHERE T1.State = 1) AS U