How to properly apply a monadic functions and projections in k?
KDB+ 3.6 2018.05.17 Copyright (C) 1993-2018 Kx Systems
q) \
(5*;10*)#\:2
10 20
({x};{x*x})#\:2
2 4
(#;#)#\:2
(#[2];#[2])
Why 2 first examples work properly and the last one doesn't? I thought it would be:
(#;#)#\:2
1 1
but it gives me a strange result.
# (take) is a diadic function, unlike count which is monadic. This is why you were getting a projection when applying only a single argument to it.
q)count
#:
q)type (count)
101h
q)type (#)
102h
You can use the . (dot-apply) operator on diadic functions with two operands to return a result that is not a projection.
(#;#) .\: (3;til 10)
0 1 2
0 1 2
Got it!
q)\
(#;#)#\:2
(#[2];#[2])
(#:;#:)#\:2
1 1
For the purpose of completeness, this relates to unary forms which is documented here: https://code.kx.com/q/basics/exposed-infrastructure/#unary-forms
Related
The Q Tips book (Nick Psaris) shows the
following function (Chapter 10):
q)merge:`time xdesc upsert
As it is stated, it corresponds to function composition. I see the pattern: the
expression supplies a function that takes both arguments for upsert and then
uses its result to feed time xdesc. However the syntax feels weird, since
I would expect upsert to be the second argument of the xdesc invocation.
Aiming at simplifying the expression, I could see that the very same scenario
applies here:
q)f:1+*
q)f[2;3]
7
If we show its result, we can clearly see that f behaves as expected:
q)f
+[1]*
However, If we slightly modify the function, the meaning of the expression is
completely different:
q)g:+[1;]*
q)g[2;3]
'rank
[0] g[2;3]
^
In fact, +[1;] is passed as first argument to the * operator instead,
leading us to a rank error:
q)g
*[+[1;]]
I could also notice that the pattern breaks when the first function is
"monadic":
q)h:neg *
q)h[2;3]
'rank
[0] h[2;3]
^
Also here:
q)i:neg neg
'type
[0] i:neg neg
^
At this point, my intuition is that this pattern only applies when we are
interested on composing dyadic standard (vs user-defined) operators that exploit infix
notation. Am I getting it right? Is this syntactic sugar actually more general? Is there any
documentation where the pattern is fully described? Thanks!
There are some documented ways to achieve what you wish:
https://code.kx.com/q/ref/apply/#composition
You can create a unary train using #
q)r:neg neg#
q)r 1
1
https://code.kx.com/q/ref/compose/
You can use ' to compose a unary value with another of rank >=1
q)f:('[1+;*])
q)f[2;3]
7
Likely the behaviour you are seeing is not officially there to be exploited by users in q so should not be relied upon. This link may be of interest:
https://github.com/quintanar401/DCoQ
I'm trying to understand this:
100+\ 1 2 3
101 103 106
Which works fine.
Question 1:
When I wrap this in brackets, I get an error I wasn't expecting:
(100+\) 1 2 3
'Cannot write to handle 100. OS reports: Bad file descriptor
What am I doing wrong here? It doesn't look like I'm writing a file to me.
Question 2:
Given the +[1;2] = 3, I believe this:
+[100;]\ 1 2 3
'
[0] +[100;]\ 1 2 3
(or perhaps +[;100]\ 1 2 3) should also work with projection, but it doesn't. What am I doing wrong here?
Question 1:
Use parse to determine order of execution
q)show pt:parse "(100+\\)1 2 3"; // need to escape \
((\;+);100)
1 2 3
q)eval each pt // should be clearer now
100
1 2 3
q)
q)value eval each pt // attempting to apply 100 to list which cannot be done
'Cannot write to handle 100. OS reports: Bad file descriptor
[0] value eval each pt
^
Question 2:
The projection is unary & is applied to the entire right argument. With unary application, evaluations will (attempt to) continue until convergence - https://code.kx.com/q/ref/accumulators/#unary-values
q)(neg\)1 2 3
1 2 3
-1 -2 -3
q)+[100]\[1 2 3]
'wsfull
m 0 68157440
Question 1
When an iterator (here \) is applied postfix (as is usual) to a function (here +) it derives a function (here +\) that is both variadic (as per #mturkington) and has infix syntax. You can apply it as a unary or as a binary. Your example 100+\1 2 3 applies it as a binary.
The parser needs a clue if you want to apply +\ as a unary. You can apply any function using bracket notation. Or you can parenthesise it: (+\) has noun syntax, as does the list (+;-;*;%). You can apply or index a noun with prefix syntax.
q)100+\1 2 3 / binary application, infix syntax
101 103 106
q)+\[100;1 2 3] / binary application, bracket syntax
101 103 106
q)+\[1 2 3] / unary application, bracket syntax
1 3 6
q)(+\)1 2 3 / unary application, prefix syntax
1 3 6
Question 2
You don’t say what result you expect from using the projection. I’ll assume you’re exploring a different way of getting the same result as in Q1.
The key issue here is that the projection of binary Add on 100 is a unary +[;100] (or +[100] or just 100+), and the accumulators \ and / applied to a unary are the Converge, Do and While iterators.
None of these gives you the Q1 result. For unary f, the derived function f\ just keeps applying f successively.
q)5 +[100]\ 1 2 3 / do 100+ five times
1 2 3
101 102 103
201 202 203
301 302 303
401 402 403
501 502 503
In this case, +\ is the underlying code for the sums keyword. This is one of several keywords that are known as variadic because their rank is not fixed. When you try (100+\) 1 2 3, kdb is actually applying the equivalent of sums to your input list, then trying to write that to handle 100, which of course doesn't exist. So that's why you get the error you get.
As for the syntax in Question 2, the following should work (adapted from this page on the variadic syntax)
q)+\[100;1 2 3]
101 103 106
I'm trying to write a function using kdb+ which will look at the list, and find the values that simply meet two conditions.
Let's call the list DR (for data range). And I want a function that will combine these two conditions
"DR where (DR mod 7) in 2"
and
"DR where (DR.dd) in 1"
I'm able to apply them one at a time but I really need to combine them into one function. I was hoping I could do this
"DR was (DR.dd mod 7) in 2 and DR where (DR.dd) in 1"
but this obviously didn't work. Any advice?
You can utilize the and function to help with this, which is the same as &:
q)dr:.z.d+til 100
q)and
&
q)2=dr mod 7
10000001000000100000010000001000000100000010000001000000100000010000001000000..
q)1=dr.dd
00000000000000000000000001000000000000000000000000000000100000000000000000000..
q)(1=dr.dd)&2=dr mod 7
00000000000000000000000000000000000000000000000000000000100000000000000000000..
q)dr where(1=dr.dd)&2=dr mod 7
2021.02.01 2021.03.01
Its necessary wrap the first part in brackets due to how kdb reads code from right to left. This format changes slightly when doing this in a where clause, the brackets arent needed due to how each where clause is parsed, that is each clause between the commas are parsed seperately. However it is essentially doing the same thing as the code above.
q)t:([]date:dr)
q)select from t where 1=date.dd,2=date mod 7
date
----------
2021.02.01
2021.03.01
You could also do this using min to achieve similar results, like so:
DR where min(1=DR.dd;2=DR mod 7)
What is (!). called in kdb?
and are below use cases valid to use (!). to convert a list to a dictionary or are there better ways and other uses of (!). ?
Example:
q)(!). (`A`B;(`C`D`E;`F`G`H));
q).[(!);flip (`A`B;`C`D;`E`F)]
I cannot find any documentation on the use cases on (!). in kdb tutorials. Please share any information on (!). and its uses?
It's a version of apply & yep your use case is valid. The reason the operator is wrapped in parentheses is because it itself is a dyadic infix operator as is dot apply (.)
If you attempt to apply it as is, your expression is like so, which Q doesn't like
// infixOp infixOp operand
q)+ . 4 5
'
[0] + . 4 5
^
Wrapping the operator within parentheses effectively transforms it so the expression now becomes
// operand infixOp operand
q)(+). 4 5
9
If you define a function which can't be used infix, then there's no need to wrap it
q)f:+
q)4 f 5
'type
[0] 4 f 5
^
q)f . 4 5
9
If using apply with bracket notation as in your example, there's no need to wrap the function
q).[+;4 5]
9
https://code.kx.com/q/ref/apply/#apply-index
https://code.kx.com/q/basics/syntax/#parentheses-around-a-function-with-infix-syntax
Jason
In terms of use-cases, I find it very useful when defining dictionaries/tables as configs particularly when dictionaries are too wide (horizontal) for the screen or when it's more useful to see fields/mappings vertically as pairs. From a code/script point of view that is.
For example:
mapping:(!) . flip(
(`one; 1);
(`two; 2);
(`three; 3));
is much easier to read when scanning through a q script than
mapping2:`one`two`three!1 2 3
when the latter gets very wide.
It makes no difference to the actual dictionary of course because as Jason pointed out it's the same thing.
How can I get the array element range of first to second last?
For example,
$array = 1,2,3,4,5
$array[0] - will give me the first (1)
$array[-2] - will give me the second last (4)
$array[0..2] - will give me first to third (1,2,3)
$array[0..-2] - I'm expecting to get first to second last (1,2,3,4) but I get 1,5,4 ???
I know I can do long hand and go for($x=0;$x -lt $array.count;$x++), but I was looking for the square bracket shortcut.
You just need to calculate the end index, like so:
$array[0..($array.length - 2)]
Do remember to check that you actually have more than two entries in your array first, otherwise you'll find yourself getting duplicates in the result.
An example of such a duplicate would be:
#(1)[0..-1]
Which, from an array of a single 1 gives the following output
1
1
There might be a situation where you are processing a list, but you don't know the length. Select-object has a -skiplast parameter.
(1,2,3,4,5 | select -skiplast 2)
1
2
3
As mentioned earlier the best solution here:
$array[0..($array.length - 2)]
The problem you met with $array[0..-2] can be explained with the nature of "0..-2" expression and the range operator ".." in PowerShell. If you try to evaluate just this part "0..-2" in PowerShell you will see that result will be an array of numbers from 0 to -2.
>> 0..-2
0
-1
-2
And when you're trying to do $array[0..-2] in PowerShell it's the same as if you would do $array[0,-1,-2]. That's why you get results as 1, 5, 4 instead of 1, 2, 3, 4.
It could be kind of counterintuitive at first especially if you have some Python or Ruby background, but you need to take it into account when using PowerShell.
Robert Westerlund answer is excellent.
This answer I just saw on the Everything you wanted to know about arrays page and wanted to try it out.
I like it because it seems to describe exactly what the goal is, end at one short of the upper bound.
$array[0..($array.GetUpperBound(0) - 1)]
1
2
3
4
I used this variation of your original attempt to uninstall all but the latest version from Get-InstalledModule. It's really short, but not perfect because if there are more than 9 items it still returns just 8, but you can put a larger negative number, though.
$array[-9..-2]
1
2
3
4