I have a table:
q)t:([] sym:`AAPL`MSFT`AMZN`AAPL`MSFT`AMZN; px:1 2 3 11 22 33; sh:100 200 300 1000 2000 3000)
I want to get the last px and sh by sym which can be obtained using last function two times:
q)select last px, last sh by sym from t
sym | px sh
----| -------
AAPL| 11 1000
AMZN| 33 3000
MSFT| 22 2000
How can we use last keyword only once to get above output?(Because in practical cases sometimes we need to use last on more than 30 columns)
My Failed attempts:
q)select last (px;sh) by sym from t
q)select {last x}#'(px;sh) by sym from t
For cases like this, using a by phrase together with no selection of columns is the same as applying last to all columns.
select by sym from t
Should do the trick
q)(select last px, last sh by sym from t)~select by sym from t
1b
A common approach to the problem is to use fby which allows you to apply a function such as first or last (or a lambda) across all columns:
t:([]c1:10?`A`B`C;c2:10?10;c3:10?"xyz";c4:.z.D-til 10)
q)select from t where i=(last;i)fby c1
c1 c2 c3 c4
-------------------
A 9 z 2019.10.01
C 7 y 2019.09.29
B 0 x 2019.09.28
q)select from t where i=({first x};i)fby c1
c1 c2 c3 c4
-------------------
B 6 x 2019.10.07
C 6 x 2019.10.06
A 4 y 2019.10.02
To answer your question in a comment regarding applying a different function per column, you would have to use a functional select such as:
q)?[t;();{x!x}1#`c1;{x!((first;last)x in `c2`c4),'x}cols[t]except`c1]
c1| c2 c3 c4
--| ----------------
A | 9 y 2019.10.01
B | 0 x 2019.09.28
C | 7 x 2019.09.29
This uses last for columns c2 and c4, then uses first for the other columns
Related
I have a table of the form:
sym date o h l c v d sp ao ah al ac av
-------------------------------------------------------------------------------------------------------
A 1999.11.18 45.5 50 40 44 4.47399e+07 0 1 30.10473 33.08212 26.46569 29.11226 4.47399e+07
A 1999.11.19 42.94 43 39.81 40.38 1.08971e+07 0 1 28.41092 28.45062 26.33998 26.71712 1.08971e+07
A 1999.11.22 41.31 44 40.06 44 4705200 0 1 27.33244 29.11226 26.50539 29.11226 4705200
I'm trying to bring down the previous close down to today:
select sym, date, c, prev c from daily
but this doesn't respect the sym ticker groups. How do I apply this transformation at the ticker level?
Edit:
Also, is there a way that I can enforce a sort on date with this schema?
This is a bit messy, but is this roughly what you are looking for?
q)t: ([] sym: `a`b`a`b`a; date: 2021.01.01 2021.01.01 2021.01.02 2021.01.02 2021.01.03; c: 10 11 8 9 10)
q)t
sym date c
-----------------
a 2021.01.01 10
b 2021.01.01 11
a 2021.01.02 8
b 2021.01.02 9
a 2021.01.03 10
q)ungroup select date, c, prevClose: prev c by sym from `date xasc t
sym date c prevClose
---------------------------
a 2021.01.01 10
a 2021.01.02 8 10
a 2021.01.03 10 8
b 2021.01.01 11
b 2021.01.02 9 11
If not, could you give some example output?
I have been stuck on this for a while now, but cannot come up with a solution, any help would be appriciated
I have 2 table like
q)x
a b c d
--------
1 x 10 1
2 y 20 1
3 z 30 1
q)y
a b| c d
---| ----
1 x| 1 10
3 h| 2 20
Would like to sum the common columns and append the new ones. Expected result should be
a b c d
--------
1 x 11 11
2 y 20 1
3 z 30 1
3 h 2 20
pj looks to only update the (1,x) but doesn't insert the new (3,h). I am assuming there has to be a way to do some sort of union+plus join in kdb
You can take advantage of the plus (+) operator here by simply keying x and adding the table y to get the desired table:
q)(2!x)+y
a b| c d
---| -----
1 x| 11 11
2 y| 20 1
3 z| 30 1
3 h| 2 20
The same "plus if there's a matching key, insert if not" behaviour works for dictionaries too:
q)(`a`b!1 2)+`a`c!10 30
a| 11
b| 2
c| 30
got it :)
q) (x pj y), 0!select from y where not ([]a;b) in key 2!x
a b c d
--------
1 x 11 11
2 y 20 1
3 z 30 1
3 h 2 20
Always open for a better implementation :D I am sure there is one.
I have one table:
a b
x 5
y 20
z 10
i want to have a sum of column b but without aggregation, thus i would get this as result:
a b c
x 5 35
y 20 35
z 10 35
This should be possible right? How this the Select comming from the table above?
You can use window functions for that:
select a, b, sum(b) over () as c
from the_table
order by a
Assuming I have a dict
d:flip(100 200 400 800 1600; 1 3 4 6 10)
how can I create a lookup function that returns the value of the smallest key that is larger than x? Given a table
tbl:flip `sym`val!(`a`b`c`d; 50 280 1200 1800)
I would like to do something like
{[x] : update new:fun[x[`val]] from x} each tbl
to end up at a table like this
tbl:flip `sym`val`new!(`a`b`c`d; 50 280 1200 1800; 1 4 10 0N)
sym val new
a 50 1
b 280 4
c 1200 10
d 1800
stepped dictionaries may help
http://code.kx.com/q/cookbook/temporal-data/#stepped-attribute
q)d:`s#-0W 100 200 400 800 1600!1 3 4 6 10 0N
q)d 50 280 1200 1800
1 4 10 0N
I think you will want to use binr to return the next element greater than or equal to x. Note that you should use a sorted list for this to work correctly. For the examples above, converting d to a dictionary with d:(!). flip d I came up with:
q)k:asc key d
q)d k k binr tbl`val
1 4 10 0N
q)update new:d k k binr val from tbl
sym val new
------------
a 50 1
b 280 4
c 1200 10
d 1800
Where you get the dictionary keys to use with: k k binr tbl`val.
Edit: if the value in the table needs to be mapped to a value greater than x but not equal to, you could try:
q)show tbl:update val:100 from tbl where i=0
sym val
--------
a 100
b 280
c 1200
d 1800
q)update new:d k (k-1) binr val from tbl
sym val new
------------
a 100 3
b 280 4
c 1200 10
d 1800
I'm currently learning kdb+/q.
I have a table of data. I want to take 2 columns of data (just numbers), compare them and create a new Boolean column that will display whether the value in column 1 is greater than or equal to the value in column 2.
I am comfortable using the update command to create a new column, but I don't know how to ensure that it is Boolean, how to compare the values and a method to display the "greater-than-or-equal-to-ness" - is it possible to do a simple Y/N output for that?
Thanks.
/ dummy data
q) show t:([] a:1 2 3; b: 0 2 4)
a b
---
1 0
2 2
3 4
/ add column name 'ge' with value from b>=a
q) update ge:b>=a from t
a b ge
------
1 0 0
2 2 1
3 4 1
Use a vector conditional:
http://code.kx.com/q/ref/lists/#vector-conditional
q)t:([]c1:1 10 7 5 9;c2:8 5 3 4 9)
q)r:update goe:?[c1>=c2;1b;0b] from t
c1 c2 goe
-------------
1 8 0
10 5 1
7 3 1
5 4 1
9 9 1
Use meta to confirm the goe column is of boolean type:
q)meta r
c | t f a
-------| -----
c1 | j
c2 | j
goe | b
The operation <= works well with vectors, but in some cases when a function needs atoms as input for performing an operation, you might want to use ' (each-both operator).
e.g. To compare the length of symbol string with another column value
q)f:{x<=count string y}
q)f[3;`ab]
0b
q)t:([] l:1 2 3; s: `a`bc`de)
q)update r:f'[l;s] from t
l s r
------
1 a 1
2 bc 1
3 de 0