In q/kdb, we can apply a function to a number of arguments, as below:
f each (1;2;3)
We can also apply a defined argument to a list of functions:
flist: (f1:{x+y+z},f2:{x+y-z},f3:{x-y+z});
flist .\: 1 2 3
What is the most efficient way to combine both of these- to apply every function in a list to each value in a list as parameters. For example, to apply 3 unary functions, f1, f2 and f3, to a list containing values 1, 2 and 3 (producing 9 calls).
Any help with this is much appreciated!
You can use the eachboth (') operator:
q)f1:1+;f2:2+;f3:3+
q)(f1;f2;f3) #' 10 20 30
11 22 33
or in the case of multi-argument functions,
q)g1:+;g2:-;g3:*
q)(g1;g2;g3) .' (2 1;3 2;2 2)
3 1 4
and if you want to apply each function to each value, you need to form a cross product first:
q)(#/)each(f1;f2;f3) cross 10 20 30
11 21 31 12 22 32 13 23 33
You can use the unary apply-at # (since you are dealing with unary functions), in combination with each-left & each-right. For example:
q)({x+1};{neg x};{x*x}) #\:/: (1 2 3)
2 -1 1
3 -2 4
4 -3 9
Related
I want to maintain a list L. So I want to write a function func[] that always take the last table of list L, work on it, return a value that appends to the end of L.
For example:
Let L be [a,b,c]
func() will take c, compute d = func[c]
Now L becomes [a,b,c,d]
func() will take d, compute e = func[d]
Now L becomes [a,b,c,d,e]
Et cetera.
How can I implement this in kdb/q?
Do or while should allow you to achieve this e.g.
q)l:0 2;f:{x,2*last x}
q)
q)5 f/ l // do op 5 times
0 2 4 8 16 32 64
q)
q)(64>last#) f/ l // perform op while last item in list is less than 64
0 2 4 8 16 32 64
I think the over function is what you need.
https://code.kx.com/q/ref/over/
q)l:1 2 3
q)f:{if[100<count x;:x];x,1+last x}
q)over[f;l]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29..
I added a condition to break the loop in f. When using over if the same result is returned twice it will stop.
You could also use .z.s. This will repeat the current function with new input.
https://code.kx.com/q/ref/dotz/#zs-self
Adding on to #CWD's answer, you could do this with explicit recursion using .z.s like so:
func: {[x;f;threshold] $[threshold <= count x;x;.z.s [x,enlist f last x;f;threshold]]}
func[1 2 3;{x*10};5} /returns 1 2 3 30 300
This will recursively apply your function to the last element until the total count of the list hits your specified threshold. Of course, set the threshold too high, and you will end up with a stack error. As such, it is preferable to use the standard iterators in production-grade code.
I have two lists:
l1:`a`b`c;
l2: til 20;
I am trying to create a dictionary 'd' that contains the elements of 'l1' as key and the elements of 'l2' evenly distributed over it. So like this:
d:(`a`b`c)!(0j, 3j, 6j, 9j, 12j, 15j, 18j;1j, 4j, 7j, 10j, 13j, 16j, 19j;2j, 5j, 8j, 11j, 14j, 17j)
The order of the elements is not relevant, I just need them balanced. I was able to achieve that in an iterative way (happy to add the code, if that's considered helpful), but there must be a more elegant way (potentially with adverbs?).
It can be done using the group :
q)group (count[l2]#l1)
(`a`b`c)!(0j, 3j, 6j, 9j, 12j, 15j, 18j;1j, 4j, 7j, 10j, 13j, 16j, 19j;2j, 5j, 8j, 11j, 14j, 17j)
If your l2 is something else instead of til 20 , then you have to lookup the items back after grouping :
q)l2: 20#.Q.a
q)l2
"abcdefghijklmnopqrst"
q)l2 group (count[l2]#l1) // lookup the items back from l2 after grouping
(`a`b`c)!("adgjmps";"behknqt";"cfilor")
You can use the reshape functionality of the take operator #. It takes two arguments: a LHS of at least 2 dimensions and the list to reshape.
For example (3;4)#til 12 will reshape the list 0 1 ... 12 into a 3 by 4 matrix
In our case, the number of the number of elements in l1 will will not necessary divide exactly into the number of elements in l2 (we don't want a rectangular matrix). Instead we can supply a null as the second dimension which will take care of distributing the remainders.
q) l1!(count[l1];0N)#l2
a| 0 1 2 3 4 5
b| 6 7 8 9 10 11 12
c| 13 14 15 16 17 18 19
This method performs very well for larger input lists.
As a side note, when using .Q.fc to split a vector argument over n slaves for multi-threading, kdb uses the # operator to reshape the vector into n vectors, one for each slave.
q)d:`a`b`c!{a where x = (a:til 20) mod y}'[til 3;3]
q)d
a| 0 3 6 9 12 15 18
b| 1 4 7 10 13 16 19
c| 2 5 8 11 14 17
If I am to split a list into 2 rows, I can use:
q)2 0N#til 10
However, the following syntax does not work:
q)n:2
q)n 0N#til 10
how I can achieve such reshaping?
Need brackets and semi colon
q)2 0N#til 10
0 1 2 3 4
5 6 7 8 9
q)n:2
q)(n;0N)#til 10
0 1 2 3 4
5 6 7 8 9
Here is the general syntax to split a list in matrix form:
(list1)#(list2)
As you can see, left part and right part of '#' is list. So here is one example:
q)list1: (4;3) / or simply (4 3)
q)list2: til 12
q)list1#list2
We can make an integer list in 2 way:
Using semicolon as list1:(2;3;4)
Using spaces as list1:(2 3 4)
But when you have variable, option 2 doesn't work;
q)list1: (n 3) / where n:2
q) `type error
So for your question, solution is to use semicolon to create list:
q) list1:(n;0N)
q) list1#til 10
I'd like to calculate a new column which is a function of several columns using select.
My actual application will involve a grouping in the select so the columns entries which I will pass to the function will contain lists. But this simple example illustrates my question
t:([] a:1 2 3; b:10 20 30; c:5 6 7)
/ Pass one argument, using projection (set first two arguments to 1)
select s:{[x;y;z] x+y+z}[1;1;] each a from t
/ Pass two arguments using each-both (set first arg to 1)
select s:a {[x;y;z] x+y+z}[1;;]'b from t
Now, how can I pass three or more arguments?
Each' will work in general but it's best to use vector operations where possible. Here I use the . operator to apply our function, \t to time both methods. I store their results to r1/r2 to show they are the same:
q)t:([]a:til n;b:til n;c:til n:1200300)
q)\t r1:update d:{x+y+z}'[a;b;c] from t
289
q)\t r2:update d:{x+y+z} . (a;b;c) from t
20
q)r1~r2
1b
q)r2
a b c d
-----------
0 0 0 0
1 1 1 3
2 2 2 6
3 3 3 9
4 4 4 12
5 5 5 15
..
Cheers,
Ryan
The following form works in general
q)t:([]a:til 10;b:til 10;c:til 10)
q)select d:{x+y+z}'[a;b;c] from t
d
--
0
3
6
9
..
I came across some MATLAB syntax with a colon that I don't fully understand.
First Question:
The expression: 0:pi/4:pi
results in the answer: 0 0.7854 1.5708 2.3562 3.1416
Why is this the case? I thought that colon operator is used as a quick way to refer to indices so that we don't have to write out the full list. (e.g. 1:3 -> 1 2 3)
Second Question:
Similar to above, say if I have a matrix X = [1 2 3 4 5 6 7 8 9]. How can I interpret the expression X(:,1:3)? Specifically, what does the colon operator without the left and right numbers mean?
Actually a:b generates a vector. You could use it as index only because the (...) accepts a list also, e.g.
octave-3.0.3:10> a = [1,4,7]
a =
1 4 7
octave-3.0.3:11> b = [1,4,9,16,25,36,49]
b =
1 4 9 16 25 36 49
octave-3.0.3:12> b(a) # gets [b(1), b(4), b(7)]
ans =
1 16 49
Now, the a:b:c syntax is equivalent to [a, a+b, a+2*b, ...] until c, e.g.
octave-3.0.3:15> 4:7:50
ans =
4 11 18 25 32 39 46
which explains what you get in 0:pi/4:pi.
A lone : selects the whole axes (row/column), e.g.
octave-3.0.3:16> a = [1,2,3;4,5,6;7,8,9]
a =
1 2 3
4 5 6
7 8 9
octave-3.0.3:17> a(:,1) # means a(1:3, 1)
ans =
1
4
7
octave-3.0.3:18> a(1,:) # means a(1, 1:3)
ans =
1 2 3
See the official MATLAB doc on colon (:) for detail.
My two pennies to KennyTM's answer.
Actually scalar and vector variables in MATLAB have 2 dimensions. Scalar has 1 row and 1 column, and vector has either 1 row or column. Just try size(X).
Colon (:) operator for indexing simply means all. Syntax X(:,1:3) means get all rows and columns from 1 to 3. Since your variable X has only 1 row, you will get first 3 values in this row.