To form a matrix consisting of identical rows, one could use
x:1 2 3
2 3#x,x
which produces (1 2 3i;1 2 3i) as expected. However, attempting to generalise this thus:
2 (count x)#x,x
produces a type error although the types are equal:
(type 3) ~ type count x
returns 1b. Why doesn't this work?
The following should work.
q)(2;count x)#x,x
1 2 3
1 2 3
If you look at the parse tree of both your statements you can see that the second is evaluated differently. In the second only the result of count is passed as an argument to #.
q)parse"2 3#x,x"
#
2 3
(,;`x;`x)
q)parse"2 (count x)#x,x"
2
(#;(#:;`x);(,;`x;`x))
If you're looking to build matrices with identical rows you might be better off using
rownum#enlist x
q)x:100000?100
q)\ts do[100;v1:5 100000#x,x]
157 5767696j
q)\ts do[100;v2:5#enlist x]
0 992j
q)v1~v2
1b
I for one find this more natural (and its faster!)
Related
I have a table
t:([]a:`a`b`c;b:1 2 3;c:`x`y`z)
I would like to iterate and process each row.
The thing is that the processing logic for each row may result in arbitrary lines of data, after the full iteration the result maybe as such e.g.
results:([]a:`a1`b1`b2`b3`c1`c2;x:1 2 2 2 3 3)
I have the following idea so far but doesn't seem to work:
uj { // some processing function } each t
But how does one return arbitrary number of data append the results into a new table?
Assuming you are using something from the table entries to indicate your arbitrary value, you can use a dictionary to indicate a number (or a function) which can be used to apply these values.
In this example, I use the c column of the original table to indicate the number of rows to return (and the number from 1 to count to).
As each entry of the table is a dictionary, I can index using the column names to get the values and build a new table.
I also use raze to join each of the results together, as they will each have the same schema.
raze {[x]
d:`x`y`z!1 3 2;
([]a:((),`$string[x[`a]],/:string 1+til d[x[`c]]);x:((),d[x[`c]])#x[`b])
} each t
Not sure if this is what you want, but you can try something like this:
ungroup select a:`${y,/:x}[string b]'[string a],b from t
Or you can use accumulators if you need the result of the previous row calculations like this:
{y[`b]+:last[x]`b;x,y}/[t;t]
If your processing function is outputting tables that conform, just raze should suffice:
raze {y#enlist x}'[t;1 3 2]
a b c
-----
a 1 x
b 2 y
b 2 y
b 2 y
c 3 z
c 3 z
Otherwise use (uj/)
(uj/) {y#enlist x}'[t;1 3 2]
a b c
-----
a 1 x
b 2 y
b 2 y
b 2 y
c 3 z
c 3 z
Your best answer will depend very much on how you want to use the results computed from each row of t. It might suit you to normalise t; it might not. The key point here:
A table cell can be any q data structure.
The minimum you can do in this regard is to store the result of your processing function in a new column.
Below, an arbitrary binary function f returns its result as a dictionary.
q)f:{n:1+rand 3;(`$string[x],/:"123" til n)!n#y}
q)f [`a;2]
a1| 2
a2| 2
q)update d:a f'b from t
a b c d
---------------------
a 1 x `a1`a2`a3!1 1 1
b 2 y (,`b1)!,2
c 3 z `c1`c2!3 3
But its result could be any q data structure.
You were considering a unary processing function:
q)pf:{#[x;`d;:;] f . x`a`b}
q)pf each t
a b c d
---------------------
a 1 x `a1`a2`a3!1 1 1
b 2 y `b1`b2!2 2
c 3 z `c1`c2`c3!3 3 3
You might find other suggestions at KX Community.
If I understand correctly your question you need something like this :
(uj/){}each t
Check this bit :
(uj/)enlist[t],{x:update x:i from?[rand[20]#enlist x;();0b;{x!x}rand[4]#cols[x]];{(x;![x;();0b;(enlist`a)!enlist($;enlist`;((';{raze string(x;y)});`a;`i))])[y~`a]}/[x;cols x]}each t
This part :
x:update x:i from
// functional form of a function that takes random rows/columns
?[rand[20]#enlist x;();0b;{x!x}rand[4]#cols[x]];
// some for of if-else and an update to generate column a (not bullet proof)
{(x;![x;();0b;(enlist`a)!enlist($;enlist`;((';{raze string(x;y)});`a;`i))])[y~`a]}/[x;cols x]
Basically the above gives something like :
q){x:update x:i from?[rand[20]#enlist x;();0b;{x!x}rand[4]#cols[x]];{(x;![x;();0b;(enlist`a)!enlist($;enlist`;((';{raze string(x;y)});`a;`i))])[y~`a]}/[x;cols x]}each t
+`a`b`c`x!(`a0`a1`a2`a3`a4`a5`a6`a7;1 1 1 1 1 1 1 1;`x`x`x`x`x`x`x`x;0 1 2 3 ..
+`a`x!(`a0`a1`a2`a3`a4`a5;0 1 2 3 4 5)
+`a`b`c`x!(`a0`a1`a2;1 1 1;`x`x`x;0 1 2)
+`a`b`c`x!(`a0`a1`a2`a3`a4`a5`a6`a7`a8`a9`a10`a11;1 1 1 1 1 1 1 1 1 1 1 1;`x`..
or taking the first one :
q)first{x:update x:i from?[rand[20]#enlist x;();0b;{x!x}rand[4]#cols[x]];{(x;![x;();0b;(enlist`a)!enlist($;enlist`;((';{raze string(x;y)});`a;`i))])[y~`a]}/[x;cols x]}each t
a b x
--------
a0 1 0
a1 1 1
a2 1 2
a3 1 3
a4 1 4
a5 1 5
a6 1 6
a7 1 7
a8 1 8
a9 1 9
a10 1 10
You can do
(uj/)enist[t],{ // some function }each t
to get what you want. Drop the enlist[t] if you don't want the table you start with in your result
Hope this helps.
I'm totally new to kdb+/q, and I found this problem below quite confusing to me. Just to simplify, we say we have this one line function f returns an one-row table with preset values, and I want to run this function over a combination of inputs x and y, like dates (list) and metas (table, with columns like orderid, px, size etc).
Now, I listed two ways to do so below. Since the function f doesn't really use any of the input, I would suppose the order of x and y doesn't matter since the difference is just which one is passed to f before another and only when two inputs passed would f starts to operate.
But why I got error in the second way, i.e. table follows the list?
Any idea and explanation is much appreciated.
f: {[x;y]
([] m: enlist `M; n: enlist `N)
};
x: 1 2 3;
y: ([] a: 4 5 6; b: 7 8 9);
raze raze f ' [y] ' [x]; // this one works
raze raze f ' [x] ' [y]; // this one gives ERROR: length Explanation: Arguments do not conform
What you're doing is effectively equivalent to:
f:{y;1};
q)(f'[([]a:1 2 3;b:4 5 3)])#/:1 2 3
1 1 1
1 1 1
1 1 1
(using extra brackets to make it clear the order of operation).
In this situation each one reduces to
q)f'[([]a:1 2 3;b:4 5 3);1]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);2]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);3]
1 1 1
The "length" is ok here because the "y" values are atomic and kdb automatically expands those atomic values to match the length of the table. In order words, kdb treats these as:
q)f'[([]a:1 2 3;b:4 5 3);1 1 1]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);2 2 2]
1 1 1
q)f'[([]a:1 2 3;b:4 5 3);3 3 3]
1 1 1
However, when you change the order it becomes:
(f'[1 2 3])#/:([]a:1 2 3;b:4 5 3)
which is equivalent to:
f'[1 2 3;`a`b!1 4]
f'[1 2 3;`a`b!2 5]
f'[1 2 3;`a`b!3 3]
but now you do have a length problem because the dictionaries in the "y" variable are not atomic, they have length 2. Which doesn't match the length of the list (3).
You don’t say so but it looks like you are studying how to iterate a binary function f over list arguments, which has brought you to projecting f' onto x, which gives you a unary f'[x] that you then iterate over y. If that’s how we got here, what you want might be as simple as x f'y, which iterates f over corresponding items in x and y.
However, you mention combinations of inputs. If you want effectively a Cartesian product based on f, then combine the iterators Each Right and Each Left to get x f:/:\:y.
That returns a matrix. You have razed your result. Depending on your argument types, you might be able to use cross to generate all the argument pair combinations, and Apply Each .' to apply f to each pair:
f .' x cross y
Is there a way to find the valence (rank) of a function in q?
ie I'm looking for a function v such that
v[{42}] gives 0
v[{x*x}] gives 1
v[{x+y}] gives 2
etc
What you want to do is
q)f:{x+y+z}
q)/the second item returned by value f is the args of f, so we simply count them.
q)value f
0x63624161410003
`x`y`z <--- args of f
`symbol$()
,`
7 5 6 3 4 2 2
"..f"
""
-1
"{x+y+z}"
q)count value[f]1
3
q)/ wrap it in a function
q)v:{count value[x]1}
(although should be noted that a niladic func like {42} will still show 1 param, x)
see below:
https://code.kx.com/q/ref/value/#lambda
Please help me with colon : operator, I'm stuck on how it works. It works as an assignment, assignment through x+:1, global assignment/view ::, I/O 0:, 1:, to return value from the middle of the function :r, and to get an unary form of operator #:.
But what happend if one apply an adverb to it? I tried this way:
$ q
KDB+ 3.6 2019.04.02 Copyright (C) 1993-2019 Kx Systems
q)(+')[100;2 3 4]
102 103 104
q)(:')[x;2 3 4]
'x
[0] (:')[x;2 3 4]
^
q)(:')[100;2 3 4]
2 3 4
I expect evaluations in order: x:2, then x:3, then x:4. To get x:4 as a result. But I've got an error. And also :' works with a number 100 for some unknown reason.
What :' is actually doing?
q)parse "(:')[100;2 3 4]"
(';:)
100
2 3 4
Parsing didn't shed much light to me, so I'm asking for your help.
When modified by an iterator (also known as an adverb in q speak), : behaves just like any other binary operator. In your example
q)(:')[100;2 3 4]
2 3 4
an atom 100 is extended to a conformant list 100 100 100 and then : is applied to elements of the two lists pairwise. The final result is returned. It might look confusing (: tries to modify a constant value, really?) but if you compare this to any other binary operator and notice that they never modify their operands but return a result of expression everything should click into place.
For example, compare
q)+'[100; 2 3 4]
102 103 104
and
q)(:')[100;2 3 4]
2 3 4
In both cases an a temporary vector 100 100 100 is created implicitly and an operator is applied to it and 2 3 4. So the former is semantically equivalent to
(t[0]+2;t[1]+2;t[2]+4)
and the latter to
(t[0]:2;t[1]:2;t[2]:4)
where t is that temporary vector.
This explains why (:')[x;2 3 4] gives an error -- if x doesn't exist kdb can't extend it to a list.
Say I have a list
b:1 1 2 3 4
and I want to find the location of the element in list b using another list
a:1 2
When I type in b in\ a, I got
11000b
00000b
where it should be
11000b
00100b
What is going on and how to get the desired answer?
Thanks in advance!
You need to use each-right /:
q)b in/:a
11000b
00100b
With b in\a the first output is getting passed back in as b. Effectively:
q)1 1 2 3 4 in 1
11000b
q)11000b in 2
00000b
You can also use each-both ':
q)in[b]'[a]
11000b
00100b