For example, let's say that I have an invoice for 1000 CUA (Currency A) and the exchange rate is 1 CUA = 20.20 CUB (Currency B). So I make 10 payments of 2019.90 CUB
#
Payment (CUB)
Payment (CUA)
Balance
0
1000.00
1
2019.90
100.00
900.00
2
2019.90
100.00
800.00
3
2019.90
100.00
700.00
4
2019.90
100.00
600.00
5
2019.90
100.00
500.00
6
2019.90
100.00
400.00
7
2019.90
100.00
300.00
8
2019.90
100.00
200.00
9
2019.90
100.00
100.00
10
2019.90
100.00
0.00
Σ
20199.00
1000.00
1000.00 CUA is 20200.00 CUB but total payments were only 20199.00 CUB
Related
I have a datasource that changed recently (same table however) and I am trying to clean up my table and having an issue with a pesky character " that I am trying to replace with null
When the table is pushed to kdb, it is a symbol column that can have a single double quote (ascii 34.) I have been running ssr to replace it with null and using fills to populate which it had worked at one point before the datasource change. I thought it might be a leading/trailing spaces so I checked with trim which seems to be fine so no rogue spaces are included.
For some reason, I am unable to perform ssr function on it. I'm verifying that the char is correct. I thought I had it working however my update below doesn't.
Any thoughts? I'm assuming its my regex?
P.S. I was hoping to avoid having to use the 'int $ seg to cast it as ascii but that is my next idea.
// update query does not fail but doesn't update the single, double quote in seg
t: update seg: fills `$ssr[;"\"\"";""] each string seg from t;
//verify the data types are symbols
meta t;
c | t f a
-------------| -----
id | s
seg | s
Here is my attempt at casting the seg as string to see how to escape the quote as well as the desired goal.
id seg Displaystring Ticker ---->> Desired output
------------------------------------------------------
AAA 1 GOOG "GOOG" GOOG
AAA 2 " ,"\"" GOOG
AAA 3 " ,"\"" GOOG
AAA 4 " ,"\"" GOOG
AAA 5 " ,"\"" GOOG
AAA 6 " ,"\"" GOOG
BBB 1 AMZN "AMZN" AMZN
BBB 2 " ,"\"" AMZN
BBB 3 " ,"\"" AMZN
CCC 1 AAPL "AAPL" AAPL
CCC 2 " ,"\"" AAPL
CCC 3 " ,"\"" AAPL
DDD 1 TSLA ,"\"" TSLA
DDD 2 " ,"\"" TSLA
This should do the job
q)tab:([]id:`$raze{x,/:string 1+til y}'[("AAA ";"BBB ";"CCC ";"DDD ");6 3 3 2];seg:#[`$'14#"\"";0 6 9 12;:;`GOOG`AMZN`AAPL`TSLA])
q)tab
id seg
----------
AAA 1 GOOG
AAA 2 "
AAA 3 "
AAA 4 "
AAA 5 "
AAA 6 "
BBB 1 AMZN
BBB 2 "
BBB 3 "
CCC 1 AAPL
CCC 2 "
CCC 3 "
DDD 1 TSLA
DDD 2 "
q)update fills?[seg=`$"\"";`;seg]from tab
id seg
----------
AAA 1 GOOG
AAA 2 GOOG
AAA 3 GOOG
AAA 4 GOOG
AAA 5 GOOG
AAA 6 GOOG
BBB 1 AMZN
BBB 2 AMZN
BBB 3 AMZN
CCC 1 AAPL
CCC 2 AAPL
CCC 3 AAPL
DDD 1 TSLA
DDD 2 TSLA
I have sensors data as below wherein under Data Column, there are 6rows containing value 45 in between preceding and following rows containing value 50. The requirement is to clean this data and impute with 50 (prev value) in the new_data column. Moreover, the no of noise records (shown as 45 in table) might either vary in number or with level of rows.
Case 1 (sample data) :-
Sl.no
Timestamp
Data
New_data
1
1/1/2021 0:00:00
50
50
2
1/1/2021 0:15:00
50
50
3
1/1/2021 0:30:00
50
50
4
1/1/2021 0:45:00
50
50
5
1/1/2021 1:00:00
50
50
6
1/1/2021 1:15:00
50
50
7
1/1/2021 1:30:00
50
50
8
1/1/2021 1:45:00
50
50
9
1/1/2021 2:00:00
50
50
10
1/1/2021 2:15:00
50
50
11
1/1/2021 2:30:00
45
50
12
1/1/2021 2:45:00
45
50
13
1/1/2021 3:00:00
45
50
14
1/1/2021 3:15:00
45
50
15
1/1/2021 3:30:00
45
50
16
1/1/2021 3:45:00
45
50
17
1/1/2021 4:00:00
50
50
18
1/1/2021 4:15:00
50
50
19
1/1/2021 4:30:00
50
50
20
1/1/2021 4:45:00
50
50
21
1/1/2021 5:00:00
50
50
22
1/1/2021 5:15:00
50
50
23
1/1/2021 5:30:00
50
50
I am thinking of a need to group these data ordered by timestamp asc (like below) and then could have a condition in place where it will have to check group by group in large sample data and if group 1 is same as group 3 , replace group 2 with group 1 values.
Sl.no
Timestamp
Data
New_data
group
1
1/1/2021 0:00:00
50
50
1
2
1/1/2021 0:15:00
50
50
1
3
1/1/2021 0:30:00
50
50
1
4
1/1/2021 0:45:00
50
50
1
5
1/1/2021 1:00:00
50
50
1
6
1/1/2021 1:15:00
50
50
1
7
1/1/2021 1:30:00
50
50
1
8
1/1/2021 1:45:00
50
50
1
9
1/1/2021 2:00:00
50
50
1
10
1/1/2021 2:15:00
50
50
1
11
1/1/2021 2:30:00
45
50
2
12
1/1/2021 2:45:00
45
50
2
13
1/1/2021 3:00:00
45
50
2
14
1/1/2021 3:15:00
45
50
2
15
1/1/2021 3:30:00
45
50
2
16
1/1/2021 3:45:00
45
50
2
17
1/1/2021 4:00:00
50
50
3
18
1/1/2021 4:15:00
50
50
3
19
1/1/2021 4:30:00
50
50
3
20
1/1/2021 4:45:00
50
50
3
21
1/1/2021 5:00:00
50
50
3
22
1/1/2021 5:15:00
50
50
3
23
1/1/2021 5:30:00
50
50
3
Moreover, there is also a need to add an exception like, if the next group is having similar pattern, not to change but to retain the data as it is.
Ex below : If group 1 and group 3 are same , impute group 2 with group 1 value.
But if group 2 and group 4 are same, do not change group 3 , retain same data in New_data.
Case 2:-
Sl.no
Timestamp
Data
New_data
group
1
1/1/2021 0:00:00
50
50
1
2
1/1/2021 0:15:00
50
50
1
3
1/1/2021 0:30:00
50
50
1
4
1/1/2021 0:45:00
50
50
1
5
1/1/2021 1:00:00
50
50
1
6
1/1/2021 1:15:00
50
50
1
7
1/1/2021 1:30:00
50
50
1
8
1/1/2021 1:45:00
50
50
1
9
1/1/2021 2:00:00
50
50
1
10
1/1/2021 2:15:00
50
50
1
11
1/1/2021 2:30:00
45
50
2
12
1/1/2021 2:45:00
45
50
2
13
1/1/2021 3:00:00
45
50
2
14
1/1/2021 3:15:00
45
50
2
15
1/1/2021 3:30:00
45
50
2
16
1/1/2021 3:45:00
45
50
2
17
1/1/2021 4:00:00
50
50
3
18
1/1/2021 4:15:00
50
50
3
19
1/1/2021 4:30:00
50
50
3
20
1/1/2021 4:45:00
50
50
3
21
1/1/2021 5:00:00
50
50
3
22
1/1/2021 5:15:00
50
50
3
23
1/1/2021 5:30:00
50
50
3
24
1/1/2021 5:45:00
45
45
4
25
1/1/2021 6:00:00
45
45
4
26
1/1/2021 6:15:00
45
45
4
27
1/1/2021 6:30:00
45
45
4
28
1/1/2021 6:45:00
45
45
4
29
1/1/2021 7:00:00
45
45
4
30
1/1/2021 7:15:00
45
45
4
31
1/1/2021 7:30:00
45
45
4
Reaching out for help in coding in postgresql to address above scenario. Please feel free to suggest any alternative approaches to solve above problem.
The query below should answer the need.
The first query identifies the rows which correspond to a change of
data.
The second query groups the rows between two successive changes of data and set up the corresponding range of timestamp
The third query is a recursive query which calculates the new_data in an
iterative way according to the timestamp order.
The last query display the expected result.
WITH RECURSIVE list As
(
SELECT no
, timestamp
, lag(data) OVER w AS previous
, data
, lead(data) OVER w AS next
, data IS DISTINCT FROM lag(data) OVER w AS first
, data IS DISTINCT FROM lead(data) OVER w AS last
FROM sensors
WINDOW w AS (ORDER BY timestamp ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)
), range_list AS
(
SELECT tsrange(timestamp, lead(timestamp) OVER w, '[]') AS range
, previous
, data
, lead(next) OVER w AS next
, first
FROM list
WHERE first OR last
WINDOW w AS (ORDER BY timestamp ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)
), rec_list (range, previous, data, next, new_data, arr) AS
(
SELECT range
, previous
, data
, next
, data
, array[range]
FROM range_list
WHERE previous IS NULL
UNION ALL
SELECT c.range
, p.data
, c.data
, c.next
, CASE
WHEN p.new_data IS NOT DISTINCT FROM c.next
THEN p.data
ELSE c.data
END
, p.arr || c.range
FROM rec_list AS p
INNER JOIN range_list AS c
ON lower(c.range) = upper(p.range) + interval '15 minutes'
WHERE NOT array[c.range] <# p.arr
AND first
)
SELECT s.*, r.new_data
FROM sensors AS s
INNER JOIN rec_list AS r
ON r.range #> s.timestamp
ORDER BY timestamp
see the test result in dbfiddle
I am new to kdb and researching it for a use case to generate time series data using a table of various function inputs. Each row of the table consists of function inputs keyed by an id and segment and will call one function per row. I have figured out how to identify which function albeit using brute force nested conditions.
My question is 2 part
How does one employ kicking off the execution of these functions?
Once the time series data is generated for each id and segment, how best can the the output be compiled into a singular table (sample output noted below - I have thought about one table for each id and then compile in two steps which would work as well but we'll have thousands of ids)
Below is a sample table and some conditions to add meta data including which function to apply
//Create sample table and add columns to identify unknown and desired function
t:([id:`AAA`AAA`AAA`BBB`CCC;seg:1 2 3 1 1];aa: 1500 0n 400 40 900;bb:0n 200 30 40 0n;cc: .40 .25 0n 0n .35)
t: update Uknown:?[0N = aa;`aa;?[0N = bb;`bb;?[0N = cc;`cc;`UNK]]] from t
t: update Call_Function:?[0N = aa;`Solveaa;?[0N = bb;`Solvebb;?[0N = cc;`Solvecc;`NoFunction]]] from t
A sample function below uses the inputs from table t to generate time series data (limited to 5 periods for example here) and test using X
//dummy function to generate output for first 5 time periods
Solvebb:{[aa;cc]
(aa%cc)*(1-exp(neg cc*1+til 5))
}
//test the function as an example for dummy output in result table below
x: flip enlist Solvebb[1500;.40] //sample output for AAA seg1 from t for example
The result would ideally be a sample table similar to below
t2: `id`seg xkey ("SIIIS";enlist",") 0:`:./Data/sampleOutput.csv
id seg| seg_idx tot_idx result
-------| ------------------------
AAA 1 | 1 1 1,236.30
AAA 1 | 2 2 2,065.02
AAA 1 | 3 3 2,620.52
AAA 1 | 4 4 2,992.89
AAA 1 | 5 5 3,242.49
AAA 2 | 1 6
AAA 2 | 2 7
AAA 2 | 3 8
AAA 2 | 4 9
AAA 2 | 5 10
AAA 3 | 1 11
AAA 3 | 2 12
AAA 3 | 3 13
AAA 3 | 4 14
AAA 3 | 5 15
BBB 1 | 1 1
BBB 1 | 2 2
BBB 1 | 3 3
BBB 1 | 4 4
BBB 1 | 5 5
..
It's difficult without more details, but something like the following may help.
First, it may be easier to define Solvebb so that it can take 3 inputs and simple ignores the middle one:
q)Solvebb:{[aa;bb;cc](aa%cc)*(1-exp(neg cc*1+til 5))}
And adding dummy functions for the other two in your table (NB. it's important for the use of ungroup later that the output of these functions are lists):
q)Solveaa:{[aa;bb;cc] (bb+cc;bb*cc)}
q)Solvecc:{[aa;bb;cc] (aa+bb;aa*bb)}
You can apply each call each function on all three vectors of input with:
q)update result:first[Call_Function]'[aa;bb;cc] by Call_Function from t
id seg| aa bb cc Uknown Call_Function result
-------| -------------------------------------------------------------------------------
AAA 1 | 1500 0.4 bb Solvebb 1236.3 2065.016 2620.522 2992.888 3242.493
AAA 2 | 200 0.25 aa Solveaa 200.25 50
AAA 3 | 400 30 cc Solvecc 430 12000f
BBB 1 | 40 40 cc Solvecc 80 1600f
CCC 1 | 900 0.35 bb Solvebb 759.3735 1294.495 1671.589 1937.322 2124.581
and you can unravel this table by applying the ungroup function
q)ungroup update result:first[Call_Function]'[aa;bb;cc] by Call_Function from t
id seg aa bb cc Uknown Call_Function result
---------------------------------------------------
AAA 1 1500 0.4 bb Solvebb 1236.3
AAA 1 1500 0.4 bb Solvebb 2065.016
AAA 1 1500 0.4 bb Solvebb 2620.522
AAA 1 1500 0.4 bb Solvebb 2992.888
AAA 1 1500 0.4 bb Solvebb 3242.493
AAA 2 200 0.25 aa Solveaa 200.25
AAA 2 200 0.25 aa Solveaa 50
AAA 3 400 30 cc Solvecc 430
AAA 3 400 30 cc Solvecc 12000
BBB 1 40 40 cc Solvecc 80
BBB 1 40 40 cc Solvecc 1600
CCC 1 900 0.35 bb Solvebb 759.3735
CCC 1 900 0.35 bb Solvebb 1294.495
CCC 1 900 0.35 bb Solvebb 1671.589
CCC 1 900 0.35 bb Solvebb 1937.322
CCC 1 900 0.35 bb Solvebb 2124.581
We need to create a postgres SQL view to generate a chart. Chart creating tool allow only a single SQL view as input. The chart has the filter option by studentname, cousecode and feecode. Other than the chart display, we need to show the sum of the total course fee and fee amount paid by all the students from the same view.
table1: student
id name address
1 John USA
2 Robert UK
3 Tinger NZ
table2: student_course
id std_id coursecode fee
1 1 CHEM 3000
2 1 PHY 4000
3. 1 BIO 2000
4. 2 CHEM 3000
5. 2 GEO 1500
6. 3 ENG 2000
table3: student_fees
id std_name coursecode feecode amount
1 1 CHEM BKFEE 100
2 1 CHEM SPFEE 140
3 1 CHEM MATFEE 250
4 1 PHY BKFEE 150
5 1 PHY SPFEE 200
6 1 BIO LBFEE 300
7 1 BIO MATFEE 350
9 1 BIO TECFEE 200
10 2 CHEM BKFEE 100
11 2 CHEM SPFEE 140
12 2 GEO BKFEE 150
13 3 ENG BKFEE 75
14 3 ENG SPFEE 140
15 3 ENG LBFEE 180
Am able to create a view like this. But this view is not enough for my operation. Because from this view I couldn't calculate the sum of the total course fee(course fee is repeating). In this case, the grouping will not work. Because of the need to filter the data by studentname,coursecode and feecode.
View:
id std_id coursecode course_fee feecode fee_amount
1 John CHEM 3000 BKFEE 100
2 John CHEM 3000 SPFEE 140
3 John CHEM 3000 MATFEE 250
4 John PHY 4000 BKFEE 150
5 John PHY 4000 SPFEE 200
6 John BIO 4000 LBFEE 300
7 John BIO 4000 MATFEE 350
8 John BIO 4000 TECFEE 200
9 Robert CHEM 3000 BKFEE 100
10 Robert CHEM 3000 SPFEE 140
11 Robert GEO 1500 BKFEE 150
12 Tinger ENG 2000 BKFEE 75
13 Tinger ENG 2000 SPFEE 140
14 Tinger ENG 2000 LBFEE 180
So in any way can we create a view like this ?
View:
id std_id coursecode course_fee feecode fee_amount
1 John CHEM 3000 BKFEE 100
2 John CHEM 0 SPFEE 140
3 John CHEM 0 MATFEE 250
4 John PHY 4000 BKFEE 150
5 John PHY 0 SPFEE 200
6 John BIO 4000 LBFEE 300
7 John BIO 0 MATFEE 350
8 John BIO 0 TECFEE 200
9 Robert CHEM 3000 BKFEE 100
10 Robert CHEM 0 SPFEE 140
11 Robert GEO 1500 BKFEE 150
12 Tinger ENG 2000 BKFEE 75
13 Tinger ENG 0 SPFEE 140
14 Tinger ENG 0 LBFEE 180
Any help appreciated...
I guess you are looking for rollup functionality in your view query i am shearing you 2 links fist one is for the basics how rollup works and the 2nd one is specific to Postgresql
first link , second link Hope this will help you
I have work out one demo for you please check rollup query
Not similar to the answer you are expecting but you can explore GROUPING SET
select name, sf.coursecode, amount, sum(fee)
from student s, student_course sc, student_fees sf
where s.id = sc.std_id
and sf.std_name = s.id
and sf.coursecode = sc.coursecode
group by
GROUPING SETS (
(name, sf.coursecode, amount, fee),
(name, sf.coursecode, fee),
()
)
order by name, sf.coursecode asc
Hi Can anyone help me please to get unique group number?
I need to give unique rows for each group even when same group repeats after some groups.
I have following data:
id version product startdate enddate
123 0 2443 2010/09/01 2011/01/02
123 1 131 2011/01/03 2011/03/09
123 2 131 2011/08/10 2012/09/10
123 3 3009 2012/09/11 2014/03/31
123 4 668 2014/04/01 2014/04/30
123 5 668 2014/05/01 2016/01/01
123 6 668 2016/01/02 2017/09/08
123 7 131 2017/09/09 2017/10/10
123 8 131 2018/10/11 2019/01/01
123 9 550 2019/01/02 2099/01/01
select *,
dense_rank()over(partition by id order by id,product)
from table
Expected results:
id version product startdate enddate count
123 0 2443 2010/09/01 2011/01/02 1
123 1 131 2011/01/03 2011/03/09 2
123 2 131 2011/08/10 2012/09/10 2
123 3 3009 2012/09/11 2014/03/31 3
123 4 668 2014/04/01 2014/04/30 4
123 5 668 2014/05/01 2016/01/01 4
123 6 668 2016/01/02 2017/09/08 4
123 7 131 2017/09/09 2017/10/10 5
123 8 131 2018/10/11 2019/01/01 5
123 9 550 2019/01/02 2099/01/01 6
Try the following
SELECT
id,version,product,startdate,enddate,
1+SUM(v)OVER(PARTITION BY id ORDER BY version) n
FROM
(
SELECT
*,
IIF(LAG(product)OVER(PARTITION BY id ORDER BY version)<>product,1,0) v
FROM TestTable
) q