Quartiles calculation in Postgresql Query - postgresql

I am having a hard time trying to get this done. I have the following table:
cod_prod seller price date
A Andres 10 anydate
A Paul 5 anydate
A Mike 2.5 anydate
A Josh 1.75 anydate
A Karen 7.5 anydate
.... ..... ... .......
I am trying to calculate quartiles of the price for each product and classify each seller's price into 4 quartiles.
The output I am expecting is:
Cod_Prod Seller Price Quartile 1stQ 2ndQ 3rdQ 4thQ
A Andres 10 4 2.5 5 7.5 10
A Karen 7.5 3 2.5 5 7.5 10
A Paul 5 2 2.5 5 7.5 10
A Mike 2.5 1 2.5 5 7.5 10
A Josh 1.75 1 2.5 5 7.5 10
.. ..... .... .... .... .. ... ...
This table has thousands of distinct cod_prod and thousands of sellers.
I am trying this query:
with cte as (
select seller, cod_prod, sum(price) as sum_price
from tablename
group by 2,1
)
select seller,
cod_prod,
sum_price,
ntile(4) over (partition by seller order by sum_price asc) quartile
from cte
But this not doing what I expect and still mising the 1stQ to 4thQ indicators bins
I tried many different things but this is the closest I got from what I want.
Can someone help me to solve it?

I am not sure if this query is exactly what you want, but I think can help you.
I calculated quartiles grouping by cod_prod.
WITH cte AS (SELECT seller, cod_prod, sum(price) as sum_price
FROM t
GROUP BY seller, cod_prod),
quartiles AS (SELECT
cod_prod,
percentile_cont(0.25) within group (order by sum_price asc) as "1stQ",
percentile_cont(0.50) within group (order by sum_price asc) as "2ndQ",
percentile_cont(0.75) within group (order by sum_price asc) as "3rdQ",
percentile_cont(1) within group (order by sum_price asc) as "4thQ"
FROM cte
GROUP BY cod_prod)
SELECT cte.*,
ntile(4) over (PARTITION BY cte.cod_prod ORDER BY sum_price ASC) quartile,
quartiles.*
FROM cte
INNER JOIN quartiles ON cte.cod_prod = quartiles.cod_prod;

Related

A query to get per month data for all months and calculate percentage per month per type

From the DB (Postgresql) I want to get the percentage per month (of all months) of stock items with a certain condition. So the total of the whole month is 100% and per condition it would be a percentage of that. I'm trying all kinds of 'partition by' queries, but i quite can't get it right.
In the example there would be an extra column and on each row there would be the percentage of that month. So the value for the new column for the first row it would be 25/506*100.
Right now I have and works is:
select to_char(created_at, 'YYYY-MM') as maand, count(si.id) as aantal,
case
when condition_id=1 then 'Nieuw'
when condition_id=2 then 'Als nieuw'
when condition_id=3 then 'Goed'
when condition_id=4 then 'Redelijk'
when condition_id=5 then 'Matig'
else 'Onbepaald'
end
from stock_items si
group by maand, condition_id
order by maand desc, condition_id asc
maand
aantal
case
new column
2022-01
25
Nieuw
25/506*100
2022-01
234
Als nieuw
234/506*100
2022-01
127
Goed
127/506*100
2022-01
16
Redelijk
16/506*100
2022-01
104
Matig
104/506*100
2021-12
456
Nieuw
other month
I hope it's all clear. Thanks!
I got what I wanted. To realise i want it a little different, but this is the answer to my question.
select
to_char(created_at, 'YYYY-MM') as maand,
count(id) as aantal,
round((count(id) / (sum(count(id)) over (partition by to_char(created_at, 'YYYY-MM'))) * 100), 2) as percentage,
case
when condition_id=1 then 'Nieuw'
when condition_id=2 then 'Als nieuw'
when condition_id=3 then 'Goed'
when condition_id=4 then 'Redelijk'
when condition_id=5 then 'Matig'
else 'Onbepaald'
end
from stock_items
group by maand, condition_id
order by maand desc, condition_id asc
just warp it with CTE.
with a as (
select to_char(created_at, 'YYYY-MM') as maand, count(si.id) as aantal,
case
when condition_id=1 then 'Nieuw'
when condition_id=2 then 'Als nieuw'
when condition_id=3 then 'Goed'
when condition_id=4 then 'Redelijk'
when condition_id=5 then 'Matig'
else 'Onbepaald'
end as case
from stock_items si
group by maand, condition_id
order by maand desc, condition_id asc)
select a.*, aantal * 100 / sum(aantal) over (PARTITION BY maand) as anntal_rate from a;
/* some characters so the edit is accepted */

Display MAX and 2nd MAX SALARY from the EMPLOYEE table

SELECT max(salary),
(SELECT MAX(SALARY) FROM EMPLOYEE
WHERE SALARY NOT IN(SELECT MAX(SALARY) FROM EMPLOYEE)) as 2ND_MAX_SALARY;
This is giving me the error: FROM keyword not found where expected
You want the top 2 of your table ordered by one of the columns (the FETCH NEXT clause is available from Oracle 12c R1)
SELECT Salary FROM Employee ORDER BY Salary DESC LIMIT 2
FETCH NEXT 2 ROWS ONLY;
Use
SELECT Salary FROM Employee ORDER BY Salary DESC LIMIT 2
FETCH NEXT 2 ROWS WITH TIES;
if you want to return all employees that have the 1st or 2nd highest salary: There might only be one highest salary amount in the company, but more than one employee who gets that amount. Those rows are the ties.
If you're on Oracle database version lower than 12c, rank analytic function might help.
For sample rows:
SQL> select * from employee order by salary desc;
ENAME SALARY
---------- ----------
KING 5000 --> highest salary
FORD 3000 --> Ford and Scott "share" the 2nd
SCOTT 3000 --> highest salary
JONES 2975
BLAKE 2850
CLARK 2450
ALLEN 1600
TURNER 1500
MILLER 1300
WARD 1250
MARTIN 1250
ADAMS 1100
JAMES 950
SMITH 800
14 rows selected.
In a subquery (or a CTE, as I did), calculate rank for each salary and then, in the main query, select rows that rank as to top salaries:
SQL> with temp as
2 (select ename,
3 salary,
4 rank() over (order by salary desc) rnk
5 from employee
6 )
7 select ename, salary
8 from temp
9 where rnk <= 2
10 order by rnk desc;
ENAME SALARY
---------- ----------
SCOTT 3000
FORD 3000
KING 5000
SQL>
SELECT MAX(salary) AS max_salary,
(SELECT MAX(salary)
FROM employee
WHERE salary NOT IN (SELECT MAX(salary)
FROM employee
)
) AS 2nD_max_salary
FROM employee;

Cohort Analysis with RedShift by Month

I am trying to build a cohort analysis for monthly retention but experiencing challenge getting the Month Number column right. The month number is supposed to return month(s) user transacted i.e 0 for registration month, 1 for the first month after registration month, 2 for the second month until the last month but currently, it returns negative month numbers in some cells.
It should be like this table:
cohort_month total_users month_number percentage
---------- ----------- -- ------------ ---------
January 100 0 40
January 341 1 90
January 115 2 90
February 103 0 73
February 100 1 40
March 90 0 90
Here is the SQL:
with cohort_items as (
select
extract(month from insert_date) as cohort_month,
msisdn as user_id
from mfscore.t_um_user_detail where extract(year from insert_date)=2020
order by 1, 2
),
user_activities as (
select
A.sender_msisdn,
extract(month from A.insert_date)-C.cohort_month as month_number
from mfscore.t_wm_transaction_logs A
left join cohort_items C ON A.sender_msisdn = C.user_id
where extract(year from A.insert_date)=2020
group by 1, 2
),
cohort_size as (
select cohort_month, count(1) as num_users
from cohort_items
group by 1
order by 1
),
B as (
select
C.cohort_month,
A.month_number,
count(1) as num_users
from user_activities A
left join cohort_items C ON A.sender_msisdn = C.user_id
group by 1, 2
)
select
B.cohort_month,
S.num_users as total_users,
B.month_number,
B.num_users * 100 / S.num_users as percentage
from B
left join cohort_size S ON B.cohort_month = S.cohort_month
where B.cohort_month IS NOT NULL
order by 1, 3
I think the RANK window function is the right solution. So the idea is to assigne a rank to months of user activities for each user, order by year and month.
Something like:
WITH activity_per_user AS (
SELECT
user_id,
event_date,
RANK() OVER (PARTITION BY user_id ORDER BY DATE_PART('year', event_date) , DATE_PART('month', event_date) ASC) AS month_number
FROM user_activities_table
)
RANK number starts from 1, so you may want to substract 1.
Then, you can group by user_id and month_number to get the number of interactions for each user per month from the subscription (adapt to your use case accordingly).
SELECT
user_id,
month_number,
COUNT(1) AS n_interactions
FROM activity_per_user
GROUP BY 1, 2
Here is the documentation:
https://docs.aws.amazon.com/redshift/latest/dg/r_WF_RANK.html

Calculate past 3 month average for every past 3rd month

I am using SQL Server 2014. I have a table like this
create table revenue (id varchar(2), trasdate date, revenue int);
insert into revenue(id, trasdate, revenue)
values ('aa', '2018/09/01', 1234.5),
('aa' , '2018/08/04', 450),
('aa', '2018/07/03',500),
('aa', '2018/06/04',600),
('ab', '2018/09/01', 1234.5),
('ab' , '2018/08/04', 450),
('ab', '2018/07/03',500),
('ab', '2018/06/04',600),
('ab', '2018/05/03', 200),
('ab', '2018/04/02', 150),
('ab', '2018/03/01', 350),
('ab', '2018/02/05', 700),
('aa', '2018/01/07', 400)
;
I am preparing a SQL query to create a SSRS report. I want to calculate a past 3 month average for current and every past 3rd month with result like below. As we are in month of September right now. The result should show something like this:
**id Period Revenue_3Mon**
aa March-May 233
aa June-Aug 516
ab March-May 233
ab June-Aug 516
Though I can figure out about the Period column. I was mainly focussing on getting the Revenue_3Mon. So I initially tried with the below query after some googling. But this query throws an error as incorrect syntax near 'rows' and if I remove rows from the query then it throws an error as Incorrect syntax near the keyword 'between'. And incorrect syntax near i.
select i.id,i.mon,
avg([i.mon_revenue]) over (partition by i.id, i.mon order by [i.id],
[i.mon] rows between 3 preceding and 1 preceding row) as revenue_3mon --
-- using 3 preceding and 1 preceding row you exclude the current row
from (select a.id, month(a.trasdate) as mon,
sum(a.revenue) as mon_revenue
from revenue a
group by a.id, month(a.trasdate)) i
group by i.id, i.mon
order by i.id,i.mon;
After few efforts, I gave up on this query and came up with new solution which was a bit close to my expectation (after lots of trial and errors).
Declare #count as int;
declare #max as int;
set #count = 4
declare #temp as table (id varchar(2), monthoftrasdate int, revenue int,
[3monavg] int);
SET #MAX = (SELECT distinct MAX(a.ROWNUM) FROM (SELECT id, month(trasdate)
as mon, SUM(revenue) TotalRevenue,
-- sum(revenue) as mon_revenue,
ROW_NUMBER() OVER(PARTITION BY ID ORDER BY MONTH(TRASDATE)) AS ROWNUM
FROM revenue
GROUP BY ID, MONTH(TRASDATE)
) A GROUP BY A.ID);
while (#count <= #max )
begin
WITH CTE AS (
SELECT id, month(trasdate) as mon, SUM(revenue) TotalRevenue,
-- sum(revenue) as mon_revenue,
ROW_NUMBER() OVER(PARTITION BY ID ORDER BY MONTH(TRASDATE)) AS
ROWNUM
FROM revenue
GROUP BY ID, MONTH(TRASDATE)
)
insert into #temp
SELECT A.ID,A.MON, a.TotalRevenue
,( SELECT avg(b.TotalRevenue) as avgrev
FROM CTE B
WHERE B.ROWNUM BETWEEN A.ROWNUM-3 AND A.ROWNUM-1
AND A.ID = B.ID --AND A.mon = B.mon
--and b.ROWNUM < a.ROWNUM
and (a.mon > 3 and a.ROWNUM > 3)
GROUP BY B.id
) AS REVENUE_3MON
FROM CTE A
set #count = #count + 1
end
select distinct a.* from #temp a
The reason I had to use 'distinct' is because the query was showing duplicate records for every id and every month. So far the result shows like below
id MonthofTrasdate Revenue 3MonAvg
aa 1 400 NULL
aa 2 700 NULL
aa 3 350 NULL
aa 4 150 483
aa 5 200 400
aa 6 600 233
aa 7 500 316
aa 8 450 433
aa 9 1234 516
ab 1 400 NULL
ab 2 700 NULL
ab 3 350 NULL
ab 4 150 483
ab 5 200 400
ab 6 600 233
ab 7 500 316
ab 8 450 433
ab 9 1234 516
This pulls out past 3 month average for every month. But i will just manipulate the rest on SSRS the way i want it.
As currently my table has no data for previous year. This works for me showing the appropriate result for next couple of months for now. But my concern is when I have to show my boss for next year Jan, Feb and March then it should be able to pull also for these months as well like Oct-Dec (Previous year), Nov-Jan and Dec - Feb. I am struggling to figure out the proper way to put this in my query.
Can you please help me out with this query? And also let me know what is wrong with my former query.
Problems with your first attempt:
You enclosed some of the aliases and column names in square brackets like [i.mon_revenue]. There is no need for square brackets, but if you want to use them, you have to break them up at the dot: [i].[mon_revenue].
In your window function expression, there is one row too many (in the end).
Window functions are applied at the very end (after the rest of the respective query), so you also have to include i.mon_revenue in your GROUP BY clause of the outer query.
Knowing that the inner query will produce one row per id and mon, there will never be preceding rows in an id-mon partition. Therefore, you must not partition by both, but only by id.
To simplify the query after resolving the issues: ordering by a partition column generally makes no sense, and since - as already mentioned - the inner query returns unique id-mon combinations, you don't have to group by these in the outer query. Looking at that query, we see that the outer query just directly selects and uses the values from the inner query, which makes a separation in two queries unneccessary. So, in fact, you wanted to perform the following query, which will produce the rolling 3-month average (I added the monthly TotalRevenue as well):
SELECT id, MONTH(trasdate) AS mon, SUM(revenue) AS TotalRevenue,
AVG(SUM(revenue)) OVER (PARTITION BY id ORDER BY MONTH(trasdate) ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING) AS revenue_3mon
FROM revenue
GROUP BY id, MONTH(trasdate)
ORDER BY id, MONTH(trasdate);
Suggestions on your second attempt:
When calculating the #MAX value, you rely on the fact that each id has revenues for the same number of months. Are you sure?
The code inside the WHILE loop does not depend on #count, so it will add the same data into the #temp table multiple times, which is probably the reason why you thought you needed a DISTINCT. Therfore: No need for the variables, no need for a loop and a #temp, no need for DISTINCT.
The conditions A.mon > 3 and A.rownum > 3 are redundant with your current data. In general, I guess, you don't want to explicitly excluse the months from January to March, so A.mon > 3 should be removed. A.rownum > 3 could be removed, too, unless you really don't want to see a 3-month average when there are only 2 preceding months or less.
As the subquery for the average is restricted to only one id, there's no need for a GROUP BY.
Since the ROW_NUMBER function doesn't care about gaps in the months, I suggest to use a different numbering function, for example DATEDIFF(month, MAX(trasdate), GETDATE()) AS mnum. Of course, the comparison in the WHERE clause of the subquery then has to be changed to B.mnum BETWEEN A.mnum+1 AND A.mnum+3.
So, your second attempt can be reduced to this, which will produce the same result as the above, at least with your sample data, where no gaps in the months exist:
WITH CTE AS (
SELECT id, MONTH(trasdate) AS mon, SUM(revenue) AS TotalRevenue,
DATEDIFF(month, MAX(trasdate), GETDATE()) AS mnum
FROM revenue
GROUP BY id, MONTH(trasdate)
)
SELECT id, mon, TotalRevenue
, (SELECT AVG(B.TotalRevenue)
FROM CTE B
WHERE B.mnum BETWEEN A.mnum+1 AND A.mnum+3
AND A.id = B.id
) AS revenue_3mon
FROM CTE A
ORDER BY id, mnum DESC;
Now, guess what, an expression like my mnum using DATEDIFF increases by one every month as we move to the past, regardless of a change of years, so this might be useful for grouping as well, whether you want to (or can?) use Window functions or not:
With OVER()
SELECT id, MONTH(MIN(trasdate)) AS mon, YEAR(MIN(trasdate)) AS yr, SUM(revenue) AS TotalRevenue,
AVG(SUM(revenue)) OVER (PARTITION BY id ORDER BY MIN(trasdate) ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING) AS revenue_3mon
FROM revenue
GROUP BY id, DATEDIFF(month, trasdate, GETDATE())
ORDER BY id, DATEDIFF(month, trasdate, GETDATE()) DESC;
Without OVER()
WITH CTE AS (
SELECT id, MIN(trasdate) AS min_dt, SUM(revenue) AS TotalRevenue,
DATEDIFF(month, trasdate, GETDATE()) AS mnum
FROM revenue
GROUP BY id, DATEDIFF(month, trasdate, GETDATE())
)
SELECT id, MONTH(min_dt) AS mon, YEAR(min_dt) AS yr, TotalRevenue
, (SELECT AVG(B.TotalRevenue)
FROM CTE B
WHERE B.mnum BETWEEN A.mnum+1 AND A.mnum+3
AND A.id = B.id
) AS revenue_3mon
FROM CTE A
ORDER BY id, mnum DESC;
Both queries allow for retrieving the minimum and maximum date for each period (including month and year).
If you instead wanted what you originally posted under The result should show something like this (just grouping by previous 3-months intervals), you just would have to group your original revenue table by id and (DATEDIFF(month, trasdate, GETDATE())-1)/3 (filtering WHERE DATEDIFF(month, trasdate, GETDATE()) > 0). If so, this kind of grouping and aggregation could, of course, be done also by the Report Server.
I think this should do what you want:
select r.*,
avg(r.mon_revenue) over (partition by r.id
order by r.mon_min
rows between 3 preceding and 1 preceding row
) as revenue_3mon
-- using 3 preceding and 1 preceding row you exclude the current row
from (select r.id, month(r.trasdate) as mon,
min(r.trasdate) as mon_min,
sum(r.revenue) as mon_revenue
from revenue r
group by r.id, year(r.trasdate), month(r.trasdate)
) 4
order by r.id, r.mon, r.mon_min;
Notes:
I fixed the code so it recognizes years as well as dates.
The expression [i.mon_revenue] is not a valid column reference (in your case). You have no column with the name "i.mon_revenue" (with the . in the name).
I changed the column alias to r to match the table.
I added a date column for each month to make it easier to express the ordering.
The outer group by is not necessary.
There are several syntax errors in your code. This should give you what you need. The inner query is the important bit but hopefully this will be enough to get you on your way.
I switch our the temp table for variable and changed the revenue column to not be INT as you have decimal values in there but other than that your original sample table is unchanged
DECLARE #revenue table (id varchar(2), trasdate date, revenue float)
insert into #revenue(id, trasdate, revenue)
values ('aa', '2018/09/01', 1234.5),
('aa' , '2018/08/04', 450),
('aa', '2018/07/03',500),
('aa', '2018/06/04',600),
('ab', '2018/09/01', 1234.5),
('ab' , '2018/08/04', 450),
('ab', '2018/07/03',500),
('ab', '2018/06/04',600),
('ab', '2018/05/03', 200),
('ab', '2018/04/02', 150),
('ab', '2018/03/01', 350),
('ab', '2018/02/05', 700),
('aa', '2018/01/07', 400)
SELECT
*
FROM
(
SELECT
*
, MONTH(trasdate) as MonthNumber
, AVG(revenue) OVER (PARTITION BY id
ORDER BY
id
, MONTH(trasdate) ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING) as ThreeMonthAvg
FROM #revenue
) a
WHERE MONTH(GETDATE()) - MonthNumber IN (0, 3, 6, 9)
This gives the following results
aa 2018-06-04 600 6 400
aa 2018-09-01 1234.5 9 516.666666666667
ab 2018-03-01 350 3 700
ab 2018-06-04 600 6 233.333333333333
ab 2018-09-01 1234.5 9 516.666666666667

t-sql - compute the difference between dates for each row

Can you show how can this be done in t-sql?
sample records
accountnumber trandate
-------------------------
1000 02-11-2010
1000 02-12-2010
1000 02-13-2010
2000 02-10-2010
2000 02-15-2010
How to compute the # of days between each transactions for each accountnumber?
like this
accountnumber trandate # of days
----------------------------------------
1000 02-11-2010 0
1000 02-12-2010 1
1000 02-13-2010 1
2000 02-10-2010 0
2000 02-15-2010 5
Thanks a lot!
SELECT accountnumber,
trandate,
Datediff(DAY, a.trandate, (SELECT TOP 1 trandate
FROM mytable b
WHERE b.trandate > a.trandate
ORDER BY trandate))
FROM mytable a
ORDER BY trandate
you can use between and
select * from table1 where trandate between 'date1' and 'date2'
Hope this helps.
Select A.AccountNo, A.TranDate, B.TranDate as PreviousTranDate, A.TranDate - B.Trandate as NoOfDays
from
(Select AccountNo, TranDate, Row_Number() as RNO over (Partition by AccountNo order by TranDate)) as A,
(Select AccountNo, TranDate, Row_Number() as RNO over (Partition by AccountNo order by TranDate)) as B
Where A.AccountNo = B.AccountNo and A.RNO -1 = B.RNO
You can also use a CTE expression to increase preformance.