Say I have a list of tables. (sym1, sym2, sym3 etc)
How would I add a new column to each table called Sym containing the table name?
Thank you
You could use something like:
q){#[value x;`Sym;:;x]}each tables[]
+`a`b`c`Sym!(0 1 2 3 4;0 1 2 3 4;0 1 2 3 4;`sym1`sym1`sym1`sym1`sym1)
+`a`b`c`Sym!(0 1 2 3 4;0 1 2 3 4;0 1 2 3 4;`sym2`sym2`sym2`sym2`sym2)
+`a`b`c`Sym!(0 1 2 3 4;0 1 2 3 4;0 1 2 3 4;`sym3`sym3`sym3`sym3`sym3)
If you remove value from the first argument of #, this will update the tables in place.
Otherwise, since this returns a list, you can use indexing to return the table you want from the list:
q)({#[value x;`Sym;:;x]}each tables[])0
a b c Sym
----------
0 0 0 sym1
1 1 1 sym1
2 2 2 sym1
3 3 3 sym1
4 4 4 sym1
Hope this helps,
James
Another way to achieve this :
q){update Sym:x from x}each `sym1`sym2`sym3
q)raze (sym1;sym2;sym3)
p s Sym
----------------
2.08725 75 sym1
2.065687 6 sym1
2.058972 63 sym2
2.095509 62 sym2
2.036151 90 sym3
2.090895 63 sym3
If you are getting these tables (sym1,sym2,sym3) as the output of another function call like :
f each `s1`s2`s3
then I'll suggest updating the function to add the column Sym just before return these individual tables.
f:{
/some logic
update Sym:x from t
}
This will save an operation of adding a new column separately
Related
I have a table of returns that I want to calculate correlation for every n rows, but I am not sure how to do that. To be more illustrative, my table t is
sym1
sym2
sym3
sym4
3
4
5
1
0
-1
6
4
-3
10
8
9
-4
19
-1
6
How can I calculate the correlation between sym1 and sym2/sym3/sym4 for row 1-3, then row 2-4 etc?
Currently I can only specify column then calculate, but I am trying to find a way that I can fix sym1 and iterate through sym2-4. Here's my current code
(cor).'flip flip each {(til x)xprev\:y}[3;]each (sym1;sym2) from t
Here's a solution which may work for you.
q)update s1s2:(cor).'flip{-3#'reverse{-1_x}\[x;y]}[-1+count i]each(sym1;sym2)from t
sym1 sym2 sym3 sym4 s1s2
------------------------------
3 4 5 1
0 -1 6 4 1
-3 10 8 9 -0.5447048
-4 19 -1 6 -0.9751578
This is only one correlation found between columns, namely sym1 & sym2. The first two results can be discarded since they do not have the 3 values required for the correlation.
I have tried using the following function to derive a table consisting of 3 columns with one column data holding a list of an arbitrary schema.
fn:{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
which works well on input with multiple rows i.e.:
q)x:flip `a`b`c`d`e!(5#enlist 5?10)
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
-----------------
8 8 5 2 8 6 8
5 8 5 2 8 6 5
2 8 5 2 8 6 2
8 8 5 2 8 6 8
6 8 5 2 8 6 6
However fails when using input with a single row i.e.
q)x:`a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
------------
8 7 7
8 8 7
8 4 7
8 4 7
8 6 7
which is obviously incorrect.
One might fix this by using enlist i.e.
q)x:enlist `a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time| 8
data| 7 8 4 4 6
id | 7
Which is correct, however if one were to apply this in the function i.e.
fn:{
flip enlist `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
...
time| 2 5 8 7 9
data| 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9
id | 2 5 8 7 9
Which has the wrong format of data values.
My question here is how might one avert this conversion issue and derive the same field values whether the argument is a multi row or single row table.
Or otherwise what is the canonical implementation of this in kdb+/q
Thanks
Edit:
To clarify: my problem isn't necessarily with the data input as one could just apply enlist if it is only one row. My question pertains to how one might use enlist in the fn function to make single row input conform to the logic seen when using multi row tables. i.e. how to replace fn enlist input with fn data (how to make the function input agnostic) Thanks
Are you meaning to flip the data perpendicular to the rest of the table? Your 5 row example works because there are 5 rows and 5 columns. The single row doesn't work due to 1 row to 5 columns.
Correct me if I'm wrong but I think this is what you want:
fn:{([]time:x`b;data:flip x`a`b`c`d`e;id:x`a)};
--------------------------------------------------
t1:flip `a`b`c`d`e!(5#enlist til 5);
a b c d e
---------
0 0 0 0 0
1 1 1 1 1
2 2 2 2 2
3 3 3 3 3
4 4 4 4 4
fn[t1]
time data id
-----------------
0 0 0 0 0 0 0
1 1 1 1 1 1 1
2 2 2 2 2 2 2
3 3 3 3 3 3 3
4 4 4 4 4 4 4
--------------------------------------------------
t2:enlist `a`b`c`d`e!til 5;
a b c d e
---------
0 1 2 3 4
fn[t2]
time data id
-----------------
1 0 1 2 3 4 0
Note without the flip you get this:
([]time:t1`b;data:t1`a`b`c`d`e;id:t1`a)
time data id
-----------------
0 0 1 2 3 4 0
1 0 1 2 3 4 1
2 0 1 2 3 4 2
3 0 1 2 3 4 3
4 0 1 2 3 4 4
In this case the time is no longer in line with the data but it works because of 5 row and cols.
Edit - I can't think of a better way to convert a dictionary to a table when needed other than using count first in a conditional. Note if the first key is a nested list this wouldn't work
{ $[1 = count first x;enlist x;x] } `a`b`c`d`e!til 5
Note, your provided function doesn't work with this:
{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
}{$[1 = count first x;enlist x;x]} `a`b`c`d`e!til 5
I have a list of items and need to merge them into a single column
using the list
list:(1 2;3 4 5 7;0 1 3)
index value
0 1 2
1 3 4 5 7
2 0 1 3
my goal is
select from list2
value
1
2
3
4
5
7
0
1
3
'raze' function flattens out 1 level of the list.
q) raze (1 2;3 4 5 7;0 1 3)
q) 1 2 3 4 5 7 0 1 3
If you have list with multi level indexing then use 'over' adverb with raze:
q) (raze/)(1 2 3;(11 12;33 44);5 6)
To convert that to table column:
q) t:([]c:raze list)
ungroup would also work provided your table doesn't have multiple columns with different nesting (or strings)
q)ungroup ([]list)
list
----
1
2
3
4
5
7
0
1
3
If you just wanted your list to appear like that I would do the following.
1 cut raze list
I see that you have used a select statement, however if you want your column defined as this in your table do the following
a:raze list
tab:([] b:a)
Your output from this should look like this
q)tab
b
-
1
2
3
4
5
7
0
1
3
Overall, a more concise way to achieve what you want to do would be
select from ([]raze list)
To avoid any errors you should not call the column header 'value' as this is a protected keyword in kdb+ and when you try to reassign it as a column header kdb will through an assign error
`assign
Hope this helps
All
I need to get the combinations and permutations of a list.
A function have been implemented for permutations.
perm:{[N;l]$[N=1;l;raze .z.s[N-1;l]{x,/:y except x}\:l]}
However, I have no idea about combinations, just like this:
l: 1 2 3
comb[2;l]
1 2
1 3
2 3
l: 1 2 3 4
comb[3;l]
1 2 3
1 2 4
1 3 4
2 3 4
Thanks!
From your solution, you can do:
q)comb:{[N;l]$[N=1;l;raze .z.s[N-1;l]{x,/:y where y>max x}\:l]}
q)comb[2;1 2 3]
1 2
1 3
2 3
Another approach using over:
q)perm:{{raze x{x,/:y except x}\:y}[;y]/[x-1;y]}
q)comb:{{raze x{x,/:y where y>max x}\:y}[;y]/[x-1;y]}
One option is to use your permutation function like this:
q) comb:{[N;l] distinct asc each perm[N;l] }
q)l: 1 2 3 4
q) comb[3;l]
output:
1 2 3
1 2 4
1 3 4
2 3 4
Note: This will change the order of elements because of asc. So if your list should have (1 3 2) in answer, it will give (1 2 3).
To maintain order, use any other function/logic in place of asc to filter duplicate elements in sets (ex: (1 2 3) and (1 3 2) are duplicates).
I'd like to calculate a new column which is a function of several columns using select.
My actual application will involve a grouping in the select so the columns entries which I will pass to the function will contain lists. But this simple example illustrates my question
t:([] a:1 2 3; b:10 20 30; c:5 6 7)
/ Pass one argument, using projection (set first two arguments to 1)
select s:{[x;y;z] x+y+z}[1;1;] each a from t
/ Pass two arguments using each-both (set first arg to 1)
select s:a {[x;y;z] x+y+z}[1;;]'b from t
Now, how can I pass three or more arguments?
Each' will work in general but it's best to use vector operations where possible. Here I use the . operator to apply our function, \t to time both methods. I store their results to r1/r2 to show they are the same:
q)t:([]a:til n;b:til n;c:til n:1200300)
q)\t r1:update d:{x+y+z}'[a;b;c] from t
289
q)\t r2:update d:{x+y+z} . (a;b;c) from t
20
q)r1~r2
1b
q)r2
a b c d
-----------
0 0 0 0
1 1 1 3
2 2 2 6
3 3 3 9
4 4 4 12
5 5 5 15
..
Cheers,
Ryan
The following form works in general
q)t:([]a:til 10;b:til 10;c:til 10)
q)select d:{x+y+z}'[a;b;c] from t
d
--
0
3
6
9
..