How to remove element from list in KDB? - kdb

For instance, I have:
x: (`a`b!(1;2); (); `a`b!(3;4))
and I want to remove the (). I have tried using ? (match)
x[x ? not ()]
but that just gives ().
What is the correct expression?
Background
I often use peach to execute a series of queries, and often some will return missing data in the form of (). Hence I want to remove the () and get back to a nice table.

Well, if you are returning tables already (rather than dictionaries), you could always just raze and that would be enough to get rid of the empty list. Technically, it is appended, but if you append an empty list on a list, you get the original list. Recall that a table is simply a list of dictionaries, so this remains the case here as well.
//I'm enlisting your dictionaries,
//so that they become tables and then raze
q)raze #[x; 0 2; enlist]
a b
---
1 2
3 4
Snippets such as raze f each x or raze f peach x are very common, and I suspect this is the idiom that would be best for your use case.

You can use except as well:
q)x except 1#()
a b
---
1 2
3 4

() is an empty list. So it has count of zero. Therefore we can use:
q)x where not 0 = count each x
a b
---
1 2
3 4

A slight more generic way to get rid of unwanted elements :
q)x where not x~\:()
a b
---
1 2
3 4
q)x where not x~\:`a`b!(1;2)
()
`a`b!3 4

Related

kdb apply function in select by row

I have a table
t: flip `S`V ! ((`$"|A|B|"; `$"|B|C|D|"; `$"|B|"); 1 2 3)
and some dicts
t1: 4 10 15 20 ! 1 2 3 5;
t2: 4 10 15 20 ! 0.5 2 4 5;
Now I need to add a column with values on the the substrings in S and the function below (which is a bit pseudocode because I am stuck here).
f:{[s;v];
if[`A in "|" vs string s; t:t1;];
else if[`B in "|" vs string s; t:t2;];
k: asc key t;
:t k k binr v;
}
problems are that s and v are passed in as full column vectors when I do something like
update l:f[S,V] from t;
How can I make this an operation that works by row?
How can I make this a vectorized function?
Thanks
You will want to use the each-both adverb to apply a function over two columns by row.
In your case:
update l:f'[S;V] from t;
To help with your pseudocode function, you might want to use $, the if-else operator, e.g.
f:{[s;v]
t:$["A"in ls:"|"vs string s;t1;"B"in ls;t2;()!()];
k:asc key t;
:t k k binr v;
};
You've not mentioned a final else clause in your pseudocode but $ expects one hence the empty dictionary at the end.
Also note that in your table the columns S and V have been cast to a symbol. vs expects a string to split so I've had to use the stringoperation - this could be removed if you are able to redefine your original table.
Hope this helps!

q - internal state in while loop

In q, I am trying to call a function f on an incrementing argument id while some condition is not met.
The function f creates a table of random length (between 1 and 5) with a column identifier which is dependent on the input id:
f:{[id] len:(1?1 2 3 4 5)0; ([] identifier:id+til len; c2:len?`a`b`c)}
Starting with id=0, f should be called while (count f[id])>1, i.e. so long until a table of length 1 is produced. The id should be incremented each step.
With the "repeat" adverb I can do the while condition and the starting value:
{(count x)>1} f/0
but how can I keep incrementing the id?
Not entirely sure if this will fix your issue but I was able to get it to work by incrementing id inside the function and returning it with each iteration:
q)g:{[id] len:(1?1 2 3 4 5)0; id[0]:([] identifier:id[1]+til len; c2:len?`a`b`c);#[id;1;1+]}
In this case id is a 2 element list where the first element is the table you are returning (initially ()) and the second item is the id. By amending the exit condition I was able to make it stop whenever the output table had a count of 1:
q)g/[{not 1=count x 0};(();0)]
+`identifier`c2!(,9;,`b)
10
If you just need the table you can run first on the output of the above expression:
q)first g/[{not 1=count x 0};(();0)]
identifier c2
-------------
3 a
The issue with the function f is that when using over and scan the output if each iteration becomes the input for the next. In your case your function is working on a numeric value put on the second pass it would get passed a table.

KDB+/Q: About unused parameters in inner functions

I have this function f
f:{{z+x*y}[x]/[y]}
I am able to call f without a 3rd parameter and I get that, but how is the inner {z+x*y} able to complete without a third parameter?
kdb will assume, if given a single list to a function which takes two parameters, that you want the first one to be x and the remainder to be y (within the context of over and scan, not in general). For example:
q){x+y}/[1;2 3 4]
10
can also be achieved by:
q){x+y}/[1 2 3 4]
10
This is likely what's happening in your example.
EDIT:
In particular, you would use this function like
q){{z+x*y}[x]/[y]}[2;3 4 5 6]
56
which is equivalent to (due to the projection of x):
q){y+2*x}/[3 4 5 6]
56
which is equivalent to (due to my original point above):
q){y+2*x}/[3;4 5 6]
56
Which explains why the "third" parameter wasn't needed
You need to understand 2 things: 'over' behavior with dyadic functions and projection.
1. Understand how over/scan works on dyadic function:
http://code.kx.com/q/ref/adverbs/#over
If you have a list like (x1,x2,x3) and funtion 'f' then
f/(x1,x2,x3) ~ f[ f[x1;x2];x3]
So in every iteration it takes one element from list which will be 'y' and result from last iteration will be 'x'. Except in first iteration where first element will be 'x' and second 'y'.
Ex:
q) f:{x*y} / call with -> f/ (4 5 6)
first iteration : x=4, y=5, result=20
second iteration: x=20, y=6, result=120
2. Projection:
Lets take an example funtion f3 which takes 3 parameters:
q) f3:{[a;b;c] a+b+c}
now we can project it to f2 by fixing (passing) one parameter
q) f2:f3[4] / which means=> f2={[b;c] 4+b+c}
so f2 is dyadic now- it accepts only 2 parameters.
So now coming to your example and applying above 2 concepts, inner function will eventually become dyadic because of projection and then finally 'over' function works on this new dyadic function.
We can rewrite the function as :
f:{
f3:{z+x*y};
f2:f3[x];
f2/y
}

Quicksort in Q/KDB+

I found this quicksort implementation on a website:
q:{$[2>distinct x;x;raze q each x where each not scan x < rand x]};
I don't understand this part:
raze q each x where each not scan x < rand x
Can someone explain it to me step by step?
Lets do it step by step . I assume you have basic understanding of Quick Sort algo. Also, there is one correction in code you mentioned which I have corrected in step 5.
Example list:
q)x: 1 0 5 4 3
Take a random element from list which will act as pivot.
q) rand x
Suppose it gives us '4' from list.
Split list 'x' in 2 lists. One contains elements lesser that '4' and other greater(or equal) to '4'.
2.a) First compare all elements with pivot (4 in our case)
q) (x<rand x) / 11001b : output is boolean list
2.b) Using above boolean list we can get all elements from 'x' lesser than '4'. Here is the way:
q) x where 11001b / ( 1 0 3) : output
So we require other expression to get all elements greater(or equal) than pivot '4'. There are many ways to do it
but lets see the one used in code:
q)not scan (x<rand x) / (11001b;00110b) : output
So it gives the list which has 2 lists. First is result of (x < rand x) which is used to get elements lesser than pivot '4' and other is negation of this list which is done by 'not' and it is used to get all elements greater(or equal) that pivot '4'.
2.c) So now we can generate 2 lists using sample code from (2.b)
q) x where each (not scan (x<rand x)) / ((1 0 3);(5 4)): output list which has 2 lists
Now apply same function to each list to sort each of them
i.e. recursive call on each list of list ((1 0 3);(5 4))
q) q each x where each (not scan (x<rand x))
After all calculations , apply 'raze' to flatten all lists that are returned from each recursive call to output one single list.
End condition for recursive call is: when input list has only 1 distinct element just return it.
q) 2>count distinct x
Note: There is one correction. 'count' was missing in original code.

Is this the simplified version of this boolean expression? Or is this reviewer wrong

Cause I've tried doing the truth table unfortunately one has 3 literals and the other has 4 so i got confused.
F = (A+B+C)(A+B+D')+B'C;
and this is the simplified version
F = A + B + C
http://www.belley.org/etc141/Boolean%20Sinplification%20Exercises/Boolean%20Simplification%20Exercise%20Questions.pdf
cause I think there's something wrong with this reviewer.. or is it accurate?
btw is simplification different from minimizing from Sum of Minterms to Sum of Products?
Yes, it is the same.
Draw the truth table for both expressions, assuming that there are four input variables in both. The value of D will not play into the second truth table: values in cells with D=1 will match values in cells with D=0. In other words, you can think of the second expression as
F = A +B + C + (0)(D)
You will see that both tables match: the (A+B+C)(A+B+D') subexpression has zeros in ABCD= {0000, 0001, 0011}; (A+B+C) has zeros only at {0000, 0001}. Adding B'C patches zero at 0011 in the first subexpressions, so the results are equivalent.