calling a function and its arguments from a table in q - kdb

I am trying to call a function and its arguments from a table. So I call the function by:
(first exec function from table where id=jobId)
In this example I get: +
Then I call the arguments as:
[first exec args from table where id=jobId]
and I get as an example [2 2]
Running the two lines after each other as:
`(first exec func from table where id=jobId)[first exec args from .table where id=jobId]`
Will give me: +[2 2]
But I need: +[2;2].
I am reading this statement from the documentations but I could not quite implement.
"
Arguments to the functional form of exec (?[;;;]) must be passed in parsed form and, depending upon the desired results, in a particular data structure. "
I tried to convert the passing of the arguments part into a functional call like this:
?[table;jobId=id;args;()]

To cover the general case of functions with single inputs, multiple inputs etc you'd need something like:
q)t:([]func:({x+1};floor;{x+y};first;{x+100});args:(12;1.5;2 2;"abc";`foo))
q)t
func args
-------------
{x+1} 12
_: 1.5
{x+y} 2 2
*: "abc"
{x+100} `foo
q)update res:{.[x;(),y;#[x;y;]`$]}'[func;args] from t
func args res
-------------------
{x+1} 12 13
_: 1.5 1
{x+y} 2 2 4
*: "abc" "a"
{x+100} `foo `type

To call your function and arguments from a table, you can use a . apply, like so:
q)table:([] function:+;args:enlist 2 2;id:1)
q)table
function args id
----------------
+ 2 2 1
q)exec .[first function;first args] from table where id=1
4
Hope this helps.

If you are trying to execute function on arguments then you could directly do that inside select/exec statement.
q) exec function .' args from table where id=1

I'm supposing that your table looks something like this
q)show table:([] function:(+;*;{x+2*y}); args:(2 2;9 7;1 3); id:0 1 2)
function args id
----------------
+ 2 2 0
* 9 7 1
{x+2*y} 1 3 2
When you have your function and arguments you can apply using .(apply):
q)jobId:1
q)(first exec function from table where id=jobId) . first exec args from table where id=jobId
63
You can simplify the query above to exec only once if you use the '(each-both) adverb:
q)first exec function .' args from table where id=1
63
And if you drop the where clause above you can get the result off applying each function to each of its arguments:
q)update result:function .' args from table
function args id result
-----------------------
+ 2 2 0 4
* 9 7 1 63
{x+2*y} 1 3 2 7

Related

Positional arguments with embedpy on KDB

\l p.q
np:.p.import`numpy
Logistic:.p.import[`sklearn.linear_model;`:LogisticRegression]
train_X_np: np[`:array](1 2 3)
train_Y_np: np[`:array](0 1 1)
Logistic[`:fit][train_X_np;train_Y_np]
When I run this I get:
call: fit() missing 1 required positional argument: 'y'
What am I doing wrong?
(Also normally, there should be a .reshape(-1,1) on the X array, I wonder if this is the cause?)
You need to initialise the LogisticRegression object
You need to reshape the x input
q)Logistic:.p.import[`sklearn.linear_model;`:LogisticRegression][]
q)train_X_np: np[`:array][1 2 3][`:reshape;-1 1]
q)Logistic[`:fit][train_X_np;train_Y_np]
/ To predict
q)Logistic[`:predict][np[`:array][0 1 2][`:reshape;-1 1]]`
0 1 1
You could also use flip enlist 1 2 3 kdb list before passing it to numpy instead of reshaping

kdb apply function in select by row

I have a table
t: flip `S`V ! ((`$"|A|B|"; `$"|B|C|D|"; `$"|B|"); 1 2 3)
and some dicts
t1: 4 10 15 20 ! 1 2 3 5;
t2: 4 10 15 20 ! 0.5 2 4 5;
Now I need to add a column with values on the the substrings in S and the function below (which is a bit pseudocode because I am stuck here).
f:{[s;v];
if[`A in "|" vs string s; t:t1;];
else if[`B in "|" vs string s; t:t2;];
k: asc key t;
:t k k binr v;
}
problems are that s and v are passed in as full column vectors when I do something like
update l:f[S,V] from t;
How can I make this an operation that works by row?
How can I make this a vectorized function?
Thanks
You will want to use the each-both adverb to apply a function over two columns by row.
In your case:
update l:f'[S;V] from t;
To help with your pseudocode function, you might want to use $, the if-else operator, e.g.
f:{[s;v]
t:$["A"in ls:"|"vs string s;t1;"B"in ls;t2;()!()];
k:asc key t;
:t k k binr v;
};
You've not mentioned a final else clause in your pseudocode but $ expects one hence the empty dictionary at the end.
Also note that in your table the columns S and V have been cast to a symbol. vs expects a string to split so I've had to use the stringoperation - this could be removed if you are able to redefine your original table.
Hope this helps!

KDB+/Q: About unused parameters in inner functions

I have this function f
f:{{z+x*y}[x]/[y]}
I am able to call f without a 3rd parameter and I get that, but how is the inner {z+x*y} able to complete without a third parameter?
kdb will assume, if given a single list to a function which takes two parameters, that you want the first one to be x and the remainder to be y (within the context of over and scan, not in general). For example:
q){x+y}/[1;2 3 4]
10
can also be achieved by:
q){x+y}/[1 2 3 4]
10
This is likely what's happening in your example.
EDIT:
In particular, you would use this function like
q){{z+x*y}[x]/[y]}[2;3 4 5 6]
56
which is equivalent to (due to the projection of x):
q){y+2*x}/[3 4 5 6]
56
which is equivalent to (due to my original point above):
q){y+2*x}/[3;4 5 6]
56
Which explains why the "third" parameter wasn't needed
You need to understand 2 things: 'over' behavior with dyadic functions and projection.
1. Understand how over/scan works on dyadic function:
http://code.kx.com/q/ref/adverbs/#over
If you have a list like (x1,x2,x3) and funtion 'f' then
f/(x1,x2,x3) ~ f[ f[x1;x2];x3]
So in every iteration it takes one element from list which will be 'y' and result from last iteration will be 'x'. Except in first iteration where first element will be 'x' and second 'y'.
Ex:
q) f:{x*y} / call with -> f/ (4 5 6)
first iteration : x=4, y=5, result=20
second iteration: x=20, y=6, result=120
2. Projection:
Lets take an example funtion f3 which takes 3 parameters:
q) f3:{[a;b;c] a+b+c}
now we can project it to f2 by fixing (passing) one parameter
q) f2:f3[4] / which means=> f2={[b;c] 4+b+c}
so f2 is dyadic now- it accepts only 2 parameters.
So now coming to your example and applying above 2 concepts, inner function will eventually become dyadic because of projection and then finally 'over' function works on this new dyadic function.
We can rewrite the function as :
f:{
f3:{z+x*y};
f2:f3[x];
f2/y
}

Give each variable a name based on an already existing logical-ID vector (MATLAB)

I have length(C) number of variables. Each index represents a uniqe type of variable (in my optimization model), e.g. wheter it is electricity generation, transmission line capacity etc..
However, I have a logical vector with the same length as C (all variables) indicating if it is e.g. generation:
% length(genoidx)=length(C), i.e. the number of variables
genoidx = [1 1 1 1 1 1 0 0 ... 1 1 1 1 1 1 0 0]
In this case, there are 6 generators in 2 time steps, amounting to 12 variables.
I want to name each variable to get a better overview of the output from the optimization model, f.ex. like this:
% This is only a try on pseudo coding
varname = cell(length(C),1)
varname(genoidx) = 'geno' (1 2 3 4 5 6 ... 1 2 3 4 5 6)
varname(lineidx) = 'line' (...
Any suggestions on how to name the variables in C with string and number, based on logical ID-vector?
Thanks!
Using dynamic names is maybe OK for the seeing the results of a calculation in the workspace, but I wouldn't use them if any code is ever going to read them.
You can use the assignin('base') function to do this.
I'm not quite sure what your pseudo code is attempting to do, but you could do something like:
>> varname={'aaa','bbb','ccc','ddd'}
varname =
'aaa' 'bbb' 'ccc' 'ddd'
>> genoidx=logical([1,0,1,1])
genoidx =
1 0 1 1
>> assignin('base', sprintf('%s_',varname{genoidx}), 22)
which would create the variable aaa_ccc_ddd_ in the workspace and assign the number 22 to it.
Alternatively you could use an expression like:
sum(genoidx.*(length(genoidx):-1:1))
to calculate a decimal value and index a cell array of bespoke names:
>> varname={'aaa','bbb','ccc','ddd','eee','fff','ggg','hhh'}
varname =
'aaa' 'bbb' 'ccc' 'ddd' 'eee' 'fff' 'ggg' 'hhh'
>> assignin('base', varname{sum(genoidx.*(length(genoidx):-1:1))}, 33)
which would create the variable ggg and assign 33 to it.

How to remove element from list in KDB?

For instance, I have:
x: (`a`b!(1;2); (); `a`b!(3;4))
and I want to remove the (). I have tried using ? (match)
x[x ? not ()]
but that just gives ().
What is the correct expression?
Background
I often use peach to execute a series of queries, and often some will return missing data in the form of (). Hence I want to remove the () and get back to a nice table.
Well, if you are returning tables already (rather than dictionaries), you could always just raze and that would be enough to get rid of the empty list. Technically, it is appended, but if you append an empty list on a list, you get the original list. Recall that a table is simply a list of dictionaries, so this remains the case here as well.
//I'm enlisting your dictionaries,
//so that they become tables and then raze
q)raze #[x; 0 2; enlist]
a b
---
1 2
3 4
Snippets such as raze f each x or raze f peach x are very common, and I suspect this is the idiom that would be best for your use case.
You can use except as well:
q)x except 1#()
a b
---
1 2
3 4
() is an empty list. So it has count of zero. Therefore we can use:
q)x where not 0 = count each x
a b
---
1 2
3 4
A slight more generic way to get rid of unwanted elements :
q)x where not x~\:()
a b
---
1 2
3 4
q)x where not x~\:`a`b!(1;2)
()
`a`b!3 4