in Poweryquery/pivot Merging two tables based on multiple column with unique values in both relative to other - merge

I have two tables: one for Sale and one for Target as shown below:
Target table:
Date
mat
Tar
01/01/2020
A
10
01/01/2020
B
12
01/01/2020
C
5
01/02/2020
A
10
01/02/2020
B
12
01/02/2020
C
5
01/03/2020
A
10
01/03/2020
B
12
01/03/2020
C
5
Sale table:
Date
mat
S
01/01/2020
A
5
01/01/2020
B
6
01/01/2020
C
8
01/01/2020
D
1
01/02/2020
A
1
01/02/2020
B
2
01/02/2020
D
12
01/03/2020
B
1
01/03/2020
C
4
01/03/2020
A
5
01/03/2020
F
2
As you can see, there are certain material date combinations in the Target table that is not in the Sale table and vice versa.
I want to combine them in such a way that any missing material date combo not in Target will be added as a row with the new material and the sales will be added as a new column. Below is the ideal output:
Date
mat
Tar
S
01/01/2020
A
10
5
01/01/2020
B
12
6
01/01/2020
C
5
8
01/02/2020
A
10
1
01/02/2020
B
12
2
01/02/2020
C
5
0
01/03/2020
A
10
5
01/03/2020
B
12
1
01/03/2020
C
5
4
01/01/2020
D
0
1
01/02/2020
D
0
12
01/03/2020
F
0
2
However, I am not getting this in PowerQuery when I choose a merge based on columns Date and mat when I choose a full outer-join to keep rows from both tables. My output is creating two date and material columns instead of one consolidated as shown above.

One way to do this would be to append both tables together and group by the Date and mat columns to get all of the combinations of those two that you need and then join Target and Sale to that new table.
let
Source = Table.Combine({Target, Sale}),
#"Grouped Rows" = Table.Group(Source, {"Date", "mat"}, {}),
#"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"Date", Order.Ascending}, {"mat", Order.Ascending}}),
#"Merged Queries" = Table.NestedJoin(#"Sorted Rows", {"Date", "mat"}, Target, {"Date", "mat"}, "Target", JoinKind.LeftOuter),
#"Expanded Target" = Table.ExpandTableColumn(#"Merged Queries", "Target", {"Tar"}, {"Tar"}),
#"Merged Queries1" = Table.NestedJoin(#"Expanded Target", {"Date", "mat"}, Sale, {"Date", "mat"}, "Sale", JoinKind.LeftOuter),
#"Expanded Sale" = Table.ExpandTableColumn(#"Merged Queries1", "Sale", {"S"}, {"S"})
in
#"Expanded Sale"
Replace null with 0 if desired.
Edit: I'm being overly complicated. You can take care Tar and S in the Group By by taking the max over those columns.
let
Source = Table.Combine({Target, Sale}),
#"Grouped Rows" = Table.Group(Source, {"Date", "mat"}, {{"Tar", each List.Max([Tar])}, {"S", each List.Max([S])}})
in
#"Grouped Rows"

Related

Getting data from alternate dates of same ID column

I've a table data as below, now I need to fetch the record with in same code, where (Value2-Value1)*2 of one row >= (Value2-Value1) of consequtive date row. (all dates are uniform with in all codes)
---------------------------------------
code Date Value1 Value2
---------------------------------------
1 1-1-2018 13 14
1 2-1-2018 14 16
1 4-1-2018 15 18
2 1-1-2019 1 3
2 2-1-2018 2 3
2 4-1-2018 3 7
ex: output needs to be
1 1-1-2018 13 14
as I am begginer to SQL coding, tried my best, but cannot get through with compare only on consequtive dates.
Use a self join.
You can specify all the conditions you've listed in the ON clause:
SELECT T0.code, T0.Date, T0.Value1, T0.Value2
FROM Table As T0
JOIN Table As T1
ON T0.code = T1.code
AND T0.Date = DateAdd(Day, 1, T1.Date)
AND (T0.Value2 - T0.Value1) * 2 >= T1.Value2 - T1.Value1

select all columns with suffix _test in q kdb

I have a partitioned table, similar to below table:
q)t:([]date:3#2019.01.01; a:1 2 3; a_test:2 3 4; b_test:3 4 5; c: 6 7 8);
date a a_test b_test c
----------------------------
2019.01.01 1 2 3 6
2019.01.01 2 3 4 7
2019.01.01 3 4 5 8
Now, I want to fetch date column and all columns have names with suffix "_test" from table t.
Expected output:
date a_test b_test
------------------------
2019.01.01 2 3
2019.01.01 3 4
2019.01.01 4 5
In my original table, there are more than 100 columns with name having _test so below is not a practical solution in this case.
q)select date, a_test, b_test from t where date=2019.01.01
I tried various options like below, but of no use:
q)delete all except date, *_test from select from t where date=2019.01.01
If the columns you are selecting are variable then you should use a functional qSQL statement to perform the query. The following can be used in your case
q)query:{[tab;dt;c]?[tab;enlist (=;`date;dt);0b;(`date,c)!`date,c]}
q)query[t;2019.01.01;cols[t] where cols[t] like "*_*"]
date a_test b_test
------------------------
2019.01.01 2 3
2019.01.01 3 4
2019.01.01 4 5
In order to craft a particular functional statement, you can parse your query, putting dummy columns in place if you aren't sure what they should be
q)parse "select date,c1,c2 from tab where date=dt"
?
`tab
,,(=;`date;`dt)
0b
`date`c1`c2!`date`c1`c2
A functional select is probably the best way to go here if you require adding further filters.
?[`t;();0b;{x!x}`date,exec c from meta t where c like "*_test"]
The functional form of any select quesry can be obtained by using the -5! operator on any SQL style statement.
In the example below I have created a table with 20 fields, each one beginning with either a or b.
I then use the functional form to define which fields I want.
q)tab:{[x] enlist x!count[x]#0}`$"_" sv ' raze string `a`b,/:\:til 10
q){[t;s]?[t;();0b;{[x] x!x} cols[t] where cols[t] like s]}[tab;"b*"]
b_0 b_1 b_2 b_3 b_4 b_5 b_6 b_7 b_8 b_9
---------------------------------------
0 0 0 0 0 0 0 0 0 0
q){[t;s]?[t;();0b;{[x] x!x} cols[t] where cols[t] like s]}[tab;"a*"]
a_0 a_1 a_2 a_3 a_4 a_5 a_6 a_7 a_8 a_9
---------------------------------------
0 0 0 0 0 0 0 0 0 0
q)-5!" select a,b from c"
?
`c
()
0b
`a`b!`a`b
Alternatively, if I don't require any filtering I can use the # operator as in below:
{[x;s] (cols[x] where cols[x] like s)#x}[ tab;"a*"]

Create Pivot Table using PostgreSQL

I have a table like this:
type code desc store Sales/Day Stock
-----------------------------------------------
1 AA1 abc 101 3 6
1 AA2 abd 101 4 0
1 AA3 abf 101 4 3
2 BA1 bba 101 5 1
2 BA2 bbc 101 2 1
1 AA1 abc 102 1 4
1 AA2 abd 102 2 0
2 BA1 bba 102 4 2
2 BA2 bbc 102 5 5
etc.
How I can show the result table like this:
type code desc Store 101 Store 102
Sales/Day | Stock Sales/Day | Stock
--------------------------------------------------------------
1 AA1 abc 3 6 1 4
1 AA2 abd 4 0 2 0
1 AA3 abf 4 3 0 0
2 BA1 bba 5 1 4 2
2 BA2 bbc 2 1 5 5
etc.
Note:
Colspan is only display.
demo:db<>fiddle
First way: FILTER
SELECT
type,
code,
"desc",
COALESCE(SUM(sales_day) FILTER (WHERE store = 101)) as sales_day_101,
COALESCE(SUM(stock) FILTER (WHERE store = 101), 0) as stock_101,
COALESCE(SUM(sales_day) FILTER (WHERE store = 102), 0) as sales_day_102,
COALESCE(SUM(stock) FILTER (WHERE store = 102), 0) as stock_102
FROM mytable
GROUP BY type, code, "desc"
ORDER BY type, code
Aggregating your values. I took SUM but in your case with distinct rows many other aggregate functions would do it. FILTER allows you to aggregate only one store.
The COALESCE is to avoid NULL values if no values are present for one aggregation (like AA3 in store 102).
Second way, CASE WHEN
SELECT
type,
code,
"desc",
SUM(CASE WHEN store = 101 THEN sales_day ELSE 0 END) as sales_day_101,
SUM(CASE WHEN store = 101 THEN stock ELSE 0 END) as stock_101,
SUM(CASE WHEN store = 102 THEN sales_day ELSE 0 END) as sales_day_102,
SUM(CASE WHEN store = 102 THEN stock ELSE 0 END) as stock_102
FROM mytable
GROUP BY type, code, "desc"
ORDER BY type, code
The idea is the same, but the newer FILTER function is replace by the more common CASE clause.
Notice that "desc" is a reserved word in Postgres. So I strictly recommend to rename your column.

PostgreSQL window function & difference between dates

Suppose I have data formatted in the following way (FYI, total row count is over 30K):
customer_id order_date order_rank
A 2017-02-19 1
A 2017-02-24 2
A 2017-03-31 3
A 2017-07-03 4
A 2017-08-10 5
B 2016-04-24 1
B 2016-04-30 2
C 2016-07-18 1
C 2016-09-01 2
C 2016-09-13 3
I need a 4th column, let's call it days_since_last_order which, in the case where order_rank = 1 then 0 else calculate the number of days since the previous order (with rank n-1).
So, the above would return:
customer_id order_date order_rank days_since_last_order
A 2017-02-19 1 0
A 2017-02-24 2 5
A 2017-03-31 3 35
A 2017-07-03 4 94
A 2017-08-10 5 38
B 2016-04-24 1 0
B 2016-04-30 2 6
C 2016-07-18 1 79
C 2016-09-01 2 45
C 2016-09-13 3 12
Is there an easier way to calculate the above with a window function (or similar) rather than join the entire dataset against itself (eg. on A.order_rank = B.order_rank - 1) and doing the calc?
Thanks!
use the lag window function
SELECT
customer_id
, order_date
, order_rank
, COALESCE(
DATE(order_date)
- DATE(LAG(order_date) OVER (PARTITION BY customer_id ORDER BY order_date))
, 0)
FROM <table_name>

Get Data from 2 Tables using Join

I have 2 tables :
1. transfer
2. data
in table data 2 records :
id name
1. 2 PQR
2. 3 XYZ
in table transfer 5 records :
id to from amount type
1. 1 2 3 100.00 C
2. 2 3 2 200.00 C
3. 3 2 3 150.00 D
4. 4 3 2 150.00 C
5. 5 2 3 300.00 D
now I want to form query that will take 2 in where condition and give me result
from transfer table that when 2 is in to column then from data should be shown
and when 2 is in from column then to data should be print.
And in result I want other columns that are amount and type.
I want data using join (Any), I am totally confused that how to perform this task.
Expected Result :
from/to amount type
3 100.00 C
3 200.00 C
3 150.00 D
3 300.00 D
Any Guidance on this..
Try Like this
select
case when "from"=2 then "to" when "to"=2 then "from" end "from/to"
,amount,type from transfer
Out put is
form/to amount type
3 100 C
3 200 C
3 150 D
3 150 C
3 100 D
OR
select case when "from"=2 then d.name when "to"=2 then data.name end "from/to",
amount,type from transfer inner join data on ("to"=data.id)
inner join data as d on("from"=d.id)
Out put is
form/to amount type
XYZ 100 C
XYZ 200 C
XYZ 150 D
XYZ 150 C
XYZ 100 D
ADDITION:
prove of working query: http://ideone.com/64kIov