Please help me with colon : operator, I'm stuck on how it works. It works as an assignment, assignment through x+:1, global assignment/view ::, I/O 0:, 1:, to return value from the middle of the function :r, and to get an unary form of operator #:.
But what happend if one apply an adverb to it? I tried this way:
$ q
KDB+ 3.6 2019.04.02 Copyright (C) 1993-2019 Kx Systems
q)(+')[100;2 3 4]
102 103 104
q)(:')[x;2 3 4]
'x
[0] (:')[x;2 3 4]
^
q)(:')[100;2 3 4]
2 3 4
I expect evaluations in order: x:2, then x:3, then x:4. To get x:4 as a result. But I've got an error. And also :' works with a number 100 for some unknown reason.
What :' is actually doing?
q)parse "(:')[100;2 3 4]"
(';:)
100
2 3 4
Parsing didn't shed much light to me, so I'm asking for your help.
When modified by an iterator (also known as an adverb in q speak), : behaves just like any other binary operator. In your example
q)(:')[100;2 3 4]
2 3 4
an atom 100 is extended to a conformant list 100 100 100 and then : is applied to elements of the two lists pairwise. The final result is returned. It might look confusing (: tries to modify a constant value, really?) but if you compare this to any other binary operator and notice that they never modify their operands but return a result of expression everything should click into place.
For example, compare
q)+'[100; 2 3 4]
102 103 104
and
q)(:')[100;2 3 4]
2 3 4
In both cases an a temporary vector 100 100 100 is created implicitly and an operator is applied to it and 2 3 4. So the former is semantically equivalent to
(t[0]+2;t[1]+2;t[2]+4)
and the latter to
(t[0]:2;t[1]:2;t[2]:4)
where t is that temporary vector.
This explains why (:')[x;2 3 4] gives an error -- if x doesn't exist kdb can't extend it to a list.
Related
I'm totally new to kdb+/q, and I found this problem below quite confusing to me. Just to simplify, we say we have this one line function f returns an one-row table with preset values, and I want to run this function over a combination of inputs x and y, like dates (list) and metas (table, with columns like orderid, px, size etc).
Now, I listed two ways to do so below. Since the function f doesn't really use any of the input, I would suppose the order of x and y doesn't matter since the difference is just which one is passed to f before another and only when two inputs passed would f starts to operate.
But why I got error in the second way, i.e. table follows the list?
Any idea and explanation is much appreciated.
f: {[x;y]
([] m: enlist `M; n: enlist `N)
};
x: 1 2 3;
y: ([] a: 4 5 6; b: 7 8 9);
raze raze f ' [y] ' [x]; // this one works
raze raze f ' [x] ' [y]; // this one gives ERROR: length Explanation: Arguments do not conform
What you're doing is effectively equivalent to:
f:{y;1};
q)(f'[([]a:1 2 3;b:4 5 3)])#/:1 2 3
1 1 1
1 1 1
1 1 1
(using extra brackets to make it clear the order of operation).
In this situation each one reduces to
q)f'[([]a:1 2 3;b:4 5 3);1]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);2]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);3]
1 1 1
The "length" is ok here because the "y" values are atomic and kdb automatically expands those atomic values to match the length of the table. In order words, kdb treats these as:
q)f'[([]a:1 2 3;b:4 5 3);1 1 1]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);2 2 2]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);3 3 3]
1 1 1
However, when you change the order it becomes:
(f'[1 2 3])#/:([]a:1 2 3;b:4 5 3)
which is equivalent to:
f'[1 2 3;`a`b!1 4]
f'[1 2 3;`a`b!2 5]
f'[1 2 3;`a`b!3 3]
but now you do have a length problem because the dictionaries in the "y" variable are not atomic, they have length 2. Which doesn't match the length of the list (3).
You don’t say so but it looks like you are studying how to iterate a binary function f over list arguments, which has brought you to projecting f' onto x, which gives you a unary f'[x] that you then iterate over y. If that’s how we got here, what you want might be as simple as x f'y, which iterates f over corresponding items in x and y.
However, you mention combinations of inputs. If you want effectively a Cartesian product based on f, then combine the iterators Each Right and Each Left to get x f:/:\:y.
That returns a matrix. You have razed your result. Depending on your argument types, you might be able to use cross to generate all the argument pair combinations, and Apply Each .' to apply f to each pair:
f .' x cross y
Recently, I wanted to calculate the next multiple of 5 of several values.
I was very confused by the output of this code, which should have done the trick:
7:11 - mod(7:11, 5) + 5
ans =
7 8 9 10 11 12 13 14
While the actual working solution was this:
(7:11) - mod(7:11, 5) + 5
ans =
10 10 10 15 15
So this seems to be related to operator precedence! But what exactly does the first command do, and why does it output a (1,8) vector?
Addendum: I have found that the first command can also be written as:
7:(11 - mod(7:11, 5) + 5)
Which already hints towards the explanation of the observed result, but I am still curious about the whole explanation.
Here's the list of MATLAB operator precedence
As you can see, parentheses, (), are solved first, meaning that mod(7:11,5) will be done first. Then point 6), the addition and subtraction are taken care of from left to right, i.e. 11-mod(7:11,5) and then 11-mod(7:11,5)+5. Then point 7), the colon, :, gets evaluated, thus 7:11-mod(7:11,5)+5.
As you noted correctly 7:11 - mod(7:11, 5) + 5 is the same as 7:(11 - mod(7:11, 5) + 5), as seen above using operator precedence.
Now to the second part: why do you obtain 8 values, rather than 5? The problem here is "making an array with an array". Basically:
1:3
ans =
1 2 3
1:(3:5)
ans =
1 2 3
This shows what's going on. If you initialise an array with the colon, but have the end point as an array, MATLAB uses only the first value. As odd as it may sound, it's documented behaviour.
mod(7:11,5) generates an array, [2 3 4 0 1]. This array is then subtracted from 11 and 5 is added [14 13 12 16 15]. Now, as we see in the documentation, only the first element is then considered. 7:[14 13 12 16 15] gets parsed as 7:14 and will result in 8 values, as you've shown.
Doing (7:11) - mod(7:11, 5) + 5 first creates two arrays: 7:11 and mod(7:11,5). It then subtracts the two arrays elementwise and adds 5 to each of the elements. Interesting to note here would be that 7:12 - mod(7:11, 5) + 5 would work, whereas (7:12) - mod(7:11, 5) + 5 would result in an error due to incompatible array sizes.
Basic questions about negative numbers and mod in Kdb
Below gives -1 as expected
q) neg 7 mod 2
but
q) a:neg 7
q) a mod 2
gives 1
And below
q) -7 mod 2
gives 1
Anyone please explain this?
KDB execute statements from right to left. So statement neg 7 mod 2 is same as neg(7 mod 2).
First KDB executes 7 mod 2 and then apply neg function on the result like below.
q) 7 mod 2 // 1
q) neg 1 // -1
which is same as
q) neg 7 mod 2 // -1
Last 2 cases -7 mod 2 and neg[7] mod 2 are equivalent. And the result for that is 1.
The mod function, as shown in the kx reference page(https://code.kx.com/v2/ref/mod/), only returns positive values. Therefore, 1 is the expect answer for -7 mod 2, and a mod 2 in your example.
The reason that neg 7 mod 2 returns -1 is that q evaluates arithmetic from right to left.
As 7 mod 2 return 1, the neg function returns -1 after taking in the value from 7 mod 2.
Hope this helps!
As Rahul has covered, this is expected behaviour that occurs as a result of the right to left execution of KDB, in conjunction with the fact that mod will always return a positive result in kdb. If you want to better understand how the execution of a given command is being implemented you can always parse it out, which will show the underly k parse tree.
q)mod
k){x-y*x div y}
q)neg
-:
q)parse "neg 7 mod 2"
-:
(k){x-y*x div y};7;2)
Here we can see that neg (-:) is being applied to the result of the mod (k){x-y*x div y}) of 7 and 2.
Right to left trips up many that are learning kdb. It will be useful to keep this aspect in mind as a possible cause for any problem that you encounter with kdb as you learn the basics, I can guarantee that it will trip you up at least a few more times.
I'd really recommend that you read/work through Q For Mortals 3, which has been made free by Kx
For the following list:
q)a:("ua#1100#1";"sba#2220#2";"r#4444#a")
I want following output :
("1100#1";"2220#2";"4444#a")
? gives first index of #
q)(a?\:"#")
2 3 1`
but using cut does not give the desired result :
q)(a?\:"#")cut'a
(("ua";"#1";"10";"0#";"1");("sba";"#22";"20#";"2");("r";"#";"4";"4";"4";"4";"#";"a"))`
You can also parse the data rather than drop chars from each string.
It'll be somewhat more efficient if your dataset is large.
q)("J#*"0:/:a)[;1]
"1100#1"
"2220#2"
"4444#a"
Notice I've set the 'key' to 'J' which will result in nulls in your example case, but you only care about the values anyway.
If you can join (sv) the strings together, it'll be even better too
q)last "J#;"0:";" sv a
"1100#1"
"2220#2"
"4444#a"
HTH,
Sean
When the left argument of cut is atom , cut behaves differently than _.
q)2 cut 2 3 4 5 6
(2 3;4 5;,6)
q)2 _ 2 3 4 5 6
4 5 6
Use _ to cut the string
q)(1+a?\:"#")_'a
("1100#1";"2220#2";"4444#a")
or
q)"#"sv/:1_/:"#" vs/:a
("1100#1";"2220#2";"4444#a")
To form a matrix consisting of identical rows, one could use
x:1 2 3
2 3#x,x
which produces (1 2 3i;1 2 3i) as expected. However, attempting to generalise this thus:
2 (count x)#x,x
produces a type error although the types are equal:
(type 3) ~ type count x
returns 1b. Why doesn't this work?
The following should work.
q)(2;count x)#x,x
1 2 3
1 2 3
If you look at the parse tree of both your statements you can see that the second is evaluated differently. In the second only the result of count is passed as an argument to #.
q)parse"2 3#x,x"
#
2 3
(,;`x;`x)
q)parse"2 (count x)#x,x"
2
(#;(#:;`x);(,;`x;`x))
If you're looking to build matrices with identical rows you might be better off using
rownum#enlist x
q)x:100000?100
q)\ts do[100;v1:5 100000#x,x]
157 5767696j
q)\ts do[100;v2:5#enlist x]
0 992j
q)v1~v2
1b
I for one find this more natural (and its faster!)