I'd like to calculate a new column which is a function of several columns using select.
My actual application will involve a grouping in the select so the columns entries which I will pass to the function will contain lists. But this simple example illustrates my question
t:([] a:1 2 3; b:10 20 30; c:5 6 7)
/ Pass one argument, using projection (set first two arguments to 1)
select s:{[x;y;z] x+y+z}[1;1;] each a from t
/ Pass two arguments using each-both (set first arg to 1)
select s:a {[x;y;z] x+y+z}[1;;]'b from t
Now, how can I pass three or more arguments?
Each' will work in general but it's best to use vector operations where possible. Here I use the . operator to apply our function, \t to time both methods. I store their results to r1/r2 to show they are the same:
q)t:([]a:til n;b:til n;c:til n:1200300)
q)\t r1:update d:{x+y+z}'[a;b;c] from t
289
q)\t r2:update d:{x+y+z} . (a;b;c) from t
20
q)r1~r2
1b
q)r2
a b c d
-----------
0 0 0 0
1 1 1 3
2 2 2 6
3 3 3 9
4 4 4 12
5 5 5 15
..
Cheers,
Ryan
The following form works in general
q)t:([]a:til 10;b:til 10;c:til 10)
q)select d:{x+y+z}'[a;b;c] from t
d
--
0
3
6
9
..
Related
I have a method of achieving this which also explains my question.
a:1 2 3 4;
b:5 6 7;
cond:1101001b;
comb:(count cond) # 0N;
comb[where cond]:a;
comb[where not cond]:b
But q has so many utilities for manipulating lists, I am wondering if there is a more direct way of doing this.
rank is what you need.
q)comb
1 2 5 3 6 7 4
q)(b,a)rank cond
1 2 5 3 6 7 4
You could write the expression in a single line
comb:#[;where not cond;:;b] #[;where cond;:;a] (count cond)#0N
Alternatively, assuming the 1s and 0s of cond matches the lengths of a and b:
(a,b) iasc where[cond],where not cond
In q/kdb, we can apply a function to a number of arguments, as below:
f each (1;2;3)
We can also apply a defined argument to a list of functions:
flist: (f1:{x+y+z},f2:{x+y-z},f3:{x-y+z});
flist .\: 1 2 3
What is the most efficient way to combine both of these- to apply every function in a list to each value in a list as parameters. For example, to apply 3 unary functions, f1, f2 and f3, to a list containing values 1, 2 and 3 (producing 9 calls).
Any help with this is much appreciated!
You can use the eachboth (') operator:
q)f1:1+;f2:2+;f3:3+
q)(f1;f2;f3) #' 10 20 30
11 22 33
or in the case of multi-argument functions,
q)g1:+;g2:-;g3:*
q)(g1;g2;g3) .' (2 1;3 2;2 2)
3 1 4
and if you want to apply each function to each value, you need to form a cross product first:
q)(#/)each(f1;f2;f3) cross 10 20 30
11 21 31 12 22 32 13 23 33
You can use the unary apply-at # (since you are dealing with unary functions), in combination with each-left & each-right. For example:
q)({x+1};{neg x};{x*x}) #\:/: (1 2 3)
2 -1 1
3 -2 4
4 -3 9
I have a list of items and need to merge them into a single column
using the list
list:(1 2;3 4 5 7;0 1 3)
index value
0 1 2
1 3 4 5 7
2 0 1 3
my goal is
select from list2
value
1
2
3
4
5
7
0
1
3
'raze' function flattens out 1 level of the list.
q) raze (1 2;3 4 5 7;0 1 3)
q) 1 2 3 4 5 7 0 1 3
If you have list with multi level indexing then use 'over' adverb with raze:
q) (raze/)(1 2 3;(11 12;33 44);5 6)
To convert that to table column:
q) t:([]c:raze list)
ungroup would also work provided your table doesn't have multiple columns with different nesting (or strings)
q)ungroup ([]list)
list
----
1
2
3
4
5
7
0
1
3
If you just wanted your list to appear like that I would do the following.
1 cut raze list
I see that you have used a select statement, however if you want your column defined as this in your table do the following
a:raze list
tab:([] b:a)
Your output from this should look like this
q)tab
b
-
1
2
3
4
5
7
0
1
3
Overall, a more concise way to achieve what you want to do would be
select from ([]raze list)
To avoid any errors you should not call the column header 'value' as this is a protected keyword in kdb+ and when you try to reassign it as a column header kdb will through an assign error
`assign
Hope this helps
If I am to split a list into 2 rows, I can use:
q)2 0N#til 10
However, the following syntax does not work:
q)n:2
q)n 0N#til 10
how I can achieve such reshaping?
Need brackets and semi colon
q)2 0N#til 10
0 1 2 3 4
5 6 7 8 9
q)n:2
q)(n;0N)#til 10
0 1 2 3 4
5 6 7 8 9
Here is the general syntax to split a list in matrix form:
(list1)#(list2)
As you can see, left part and right part of '#' is list. So here is one example:
q)list1: (4;3) / or simply (4 3)
q)list2: til 12
q)list1#list2
We can make an integer list in 2 way:
Using semicolon as list1:(2;3;4)
Using spaces as list1:(2 3 4)
But when you have variable, option 2 doesn't work;
q)list1: (n 3) / where n:2
q) `type error
So for your question, solution is to use semicolon to create list:
q) list1:(n;0N)
q) list1#til 10
I have a table rCom which has various columns. I would like to sum across each row..
for example:
Date TypeA TypeB TypeC TypeD
date1 40.5 23.1 45.1 65.2
date2 23.3 32.2 56.1 30.1
How can I write a q query to add a fourth column 'Total' that sums across each row?
why not just:
update Total: TypeA+TypeB+TypeC+TypeD from rCom
?
Sum will work just fine:
q)flip`a`b`c!3 3#til 9
a b c
-----
0 3 6
1 4 7
2 5 8
q)update d:sum(a;b;c) from flip`a`b`c!3 3#til 9
a b c d
--------
0 3 6 9
1 4 7 12
2 5 8 15
Sum has map reduce which will be better for a huge table.
One quick point regarding summing across rows. You should be careful about nulls in 1 column resulting in a null result for the sum. Borrowing #WooiKent Lee's example.
We put a null into the first position of the a column. Notice how our sum now becomes null
q)wn:.[flip`a`b`c!3 3#til 9;(0;`a);first 0#] //with null
q)update d:sum (a;b;c) from wn
a b c d
--------
3 6
1 4 7 12
2 5 8 15
This is a direct effect of the way nulls in q are treated. If you sum across a simple list, the nulls are ignored
q)sum 1 2 3 0N
6
However, a sum across a general list will not display this behavior
q)sum (),/:1 2 3 0N
,0N
So, for your table situation, you might want to fill in with a zero beforehand
q)update d:sum 0^(a;b;c) from wn
a b c d
--------
3 6 9
1 4 7 12
2 5 8 15
Or alternatively, make it s.t. you are actually summing across simple lists rather than general lists.
q)update d:sum each flip (a;b;c) from wn
a b c d
--------
3 6 9
1 4 7 12
2 5 8 15
For a more complete reference on null treatment please see the reference website
This is what worked:
select Answer:{[x;y;z;a] x+y+z+a }'[TypeA;TypeB;TypeC;TypeD] from
([] dt:2014.01.01 2014.01.02 2014.01.03; TypeA:4 5 6; TypeB:1 2 3; TypeC:8 9 10; TypeD:3 4 5)