kdb passing column names into functions - kdb

I have a table
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10)
ref a b c d e
0 5 3 3 9 1
1 1 9 0 0 0
2 5 9 4 1 7
3 0 0 5 1 3
4 2 6 8 9 3
5 3 2 0 6 6
6 7 6 4 9 8
7 4 8 9 7 2
8 7 0 8 8 3
9 7 9 0 4 8
how can I set all values in columns a,b,c ,.. to 0Ni if their value equals the value in column ref without having to do a single line update for every columns?
So something taht would look a bit like (which returns ERROR:type)
{update x:?[x=t;0Ni;x] from t} each `a`b`c`....

The previous answer involves working with strings, this can often get very messy. To avoid this it is possible to build up the query by passing column names as symbols instead. The dictionary for the functional select can be built up using the following function:
q){y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]
a| ({?[y=x;0Ni;x]};`ref) `a
b| ({?[y=x;0Ni;x]};`ref) `b
c| ({?[y=x;0Ni;x]};`ref) `c
The initial column is x and it allows to any number of columns to be passed as y for comparison.
This can then be added into the functional select:
q)![t;();0b;{y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]]
ref a b c d e
-------------
0 4 5 8 4 8
1 2 6 9 1
2 8 4 7 2 9
3 0 1 2 7 5
4 5 3 0 4
5 8 3 1 6
6 5 7 4 9 6
7 2 8 2 2 1
8 2 7 1 8
9 6 1 8 8 5

You may reuse the next code snippet
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10);
columns: `a`b`c`d`e;
![t;();0b;columns!{parse "?[",x,"=ref;0Ni;",x,"]" }each string columns]
Where updated columns are put in columns list.
And functional update, which maps every column X to value ?[x=ref;0Ni;x], is used

Related

How do I do a scan over a table in KDB?

t: ([] a: til 10; b: til 10)
a b
---
0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
I'm trying to get it to sum a, and upsert it, leaving b in place. I don't want to use q-sql for it.
I think it should be something like:
({x[`a]: x`a + y`a}\) t
I keep getting 'type errors though, on the indexing operation.
What am I doing wrong?
EDIT:
An even simpler example,
({x[`a]: 3}\) t
Same error. Expected result:
q)
a b
---
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
What am I'm trying to achieve in pseudocode:
assign case:
for each row in t
row[`a]: 3
For the summation case:
for each row in t
row[`a]: row_prior[`a] + row[`a]
I finally figured out the answer; you have to return after the assignment:
({y[`a]: y[`a]+x[`a]; y}\) t
a b
----
0 0
1 1
3 2
6 3
10 4
15 5
21 6
28 7
36 8
45 9

KDB+/Q:Input agnostic function for single and multi row tables

I have tried using the following function to derive a table consisting of 3 columns with one column data holding a list of an arbitrary schema.
fn:{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
which works well on input with multiple rows i.e.:
q)x:flip `a`b`c`d`e!(5#enlist 5?10)
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
-----------------
8 8 5 2 8 6 8
5 8 5 2 8 6 5
2 8 5 2 8 6 2
8 8 5 2 8 6 8
6 8 5 2 8 6 6
However fails when using input with a single row i.e.
q)x:`a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
------------
8 7 7
8 8 7
8 4 7
8 4 7
8 6 7
which is obviously incorrect.
One might fix this by using enlist i.e.
q)x:enlist `a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time| 8
data| 7 8 4 4 6
id | 7
Which is correct, however if one were to apply this in the function i.e.
fn:{
flip enlist `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
...
time| 2 5 8 7 9
data| 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9
id | 2 5 8 7 9
Which has the wrong format of data values.
My question here is how might one avert this conversion issue and derive the same field values whether the argument is a multi row or single row table.
Or otherwise what is the canonical implementation of this in kdb+/q
Thanks
Edit:
To clarify: my problem isn't necessarily with the data input as one could just apply enlist if it is only one row. My question pertains to how one might use enlist in the fn function to make single row input conform to the logic seen when using multi row tables. i.e. how to replace fn enlist input with fn data (how to make the function input agnostic) Thanks
Are you meaning to flip the data perpendicular to the rest of the table? Your 5 row example works because there are 5 rows and 5 columns. The single row doesn't work due to 1 row to 5 columns.
Correct me if I'm wrong but I think this is what you want:
fn:{([]time:x`b;data:flip x`a`b`c`d`e;id:x`a)};
--------------------------------------------------
t1:flip `a`b`c`d`e!(5#enlist til 5);
a b c d e
---------
0 0 0 0 0
1 1 1 1 1
2 2 2 2 2
3 3 3 3 3
4 4 4 4 4
fn[t1]
time data id
-----------------
0 0 0 0 0 0 0
1 1 1 1 1 1 1
2 2 2 2 2 2 2
3 3 3 3 3 3 3
4 4 4 4 4 4 4
--------------------------------------------------
t2:enlist `a`b`c`d`e!til 5;
a b c d e
---------
0 1 2 3 4
fn[t2]
time data id
-----------------
1 0 1 2 3 4 0
Note without the flip you get this:
([]time:t1`b;data:t1`a`b`c`d`e;id:t1`a)
time data id
-----------------
0 0 1 2 3 4 0
1 0 1 2 3 4 1
2 0 1 2 3 4 2
3 0 1 2 3 4 3
4 0 1 2 3 4 4
In this case the time is no longer in line with the data but it works because of 5 row and cols.
Edit - I can't think of a better way to convert a dictionary to a table when needed other than using count first in a conditional. Note if the first key is a nested list this wouldn't work
{ $[1 = count first x;enlist x;x] } `a`b`c`d`e!til 5
Note, your provided function doesn't work with this:
{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
}{$[1 = count first x;enlist x;x]} `a`b`c`d`e!til 5

Take a column of a matrix and make it a row in kdb

Consider the matrix:
1 2 3
4 5 6
7 8 9
I'd like to take the middle column, assign it to a variable, and replace the middle row with it, giving me
1 2 3
2 5 8
7 8 9
I'm extracting the middle column using
a:m[;enlist1]
which returns
2
5
8
How do I replace the middle row with a? Is a flip necessary?
Thanks.
If you want to update the matrix in place you can use
q)show m:(3;3)#1+til 10
1 2 3
4 5 6
7 8 9
q)a:m[;1]
q)m[1]:a
q)show m
1 2 3
2 5 8
7 8 9
q)
cutting out "a" all you need is:
m[1]:m[;1]
You can use dot amend -
q)show m:(3;3)#1+til 10
1 2 3
4 5 6
7 8 9
q)show a:m[;1]
2 5 8
q).[m;(1;::);:;a]
1 2 3
2 5 8
7 8 9
Can see documentation here:
http://code.kx.com/wiki/Reference/DotSymbol
http://code.kx.com/wiki/JB:QforMortals2/functions#Functional_Forms_of_Amend
Making it slightly more generic where you can define the operation, row, and column
q)m:3 cut 1+til 9
1 2 3
4 5 6
7 8 9
Assigning the middle column to middle row :
q){[ m;o;i1;i2] .[m;enlist i1;o; flip[m] i2 ] }[m;:;1;1]
1 2 3
2 5 8
7 8 9
Adding the middle column to middle row by passing o as +
q){[ m;o;i1;i2] .[m;enlist i1;o; flip[m] i2 ] }[m;+;1;1]
1 2 3
6 10 14
7 8 9

MATLAB - generating vector with sequence of values

Given two parameters:
n %number of repetitions per value
k %max value to repeat
I would like to create a vector of size n*k, which is a concatenation of k vectors of size n, such that the i'th vector contains the value i at each coordinate.
Example:
n = 5;
k = 9;
Desired result:
[1,1,1,1,1,2,2,2,2,2,3,3,3,3,3,4,4,4,4,4,5,5,5,5,5,6,6,6,6,6,7,7,7,7,7,8,8,8,8,8,9,9,9,9,9];
Is there an elegant way to achieve this?
Thanks!
quite a few ways to do it:
method 1:
A=1:k
repelem(A',n,1)'
method 2:
A=1:k
kron(A', ones(n,1))'
method 3:
A=1:k
B=repmat(A, n, 1)
B(:)'
method 4:
A=1:k
B=ones(n,1)*A
B(:)'
Here is an alternative method
A = reshape(mtimes((1:k).',ones(1,n)).',1,n*k)
A =
Columns 1 through 22
1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5
Columns 23 through 44
5 5 5 6 6 6 6 6 7 7 7 7 7 8 8 8 8 8 9 9 9 9
Column 45
9
It multiplies each element by ones n times
>> mtimes((1:k).',ones(1,5)).'
ans =
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
and then reshapes the whole matrix to one vector

Adding a row to a matrix in Q?

I currently have a 3 by 3 matrix "m":
1 2 3
4 5 6
7 8 9
I would like to add a row to matrix 'm' to have a resultant matrix of:
1 2 3
4 5 6
7 8 9
10 11 12
A matrix in q is just a list of lists where inner lists represent rows.
m: ((1 2 3);(4 5 6);(7 8 9))
In order to add one more row all you have to do is add one more inner list to it:
m: m,enlist 10 11 12
enlist is important here, without it you'll end up with this:
q)((1 2 3);(4 5 6);(7 8 9)),10 11 12
1 2 3
4 5 6
7 8 9
10
11
12
I agree; using 0N!x to view the structure is very useful.
To achieve what you want then you can simply do;
q)show m:3 cut 1+til 9 /create matrix
1 2 3
4 5 6
7 8 9
q)show m,:10 11 12 /join new 'row'
1 2 3
4 5 6
7 8 9
10 11 12
q)