(q/kdb+) Generating an automated list - kdb

Example 1)
I have the code below
5#10+1*2
that generates
index value
0 12
1 12
2 12
3 12
4 12
How can I replace the number "1" by the index?
then generating
5#10+index*2
index value
0 10
1 12
2 14
3 16
4 18
update Example 2)
Now, if I have, let's say
mult:5;
t:select from ([]numC:1 3 6 4 1;[]s:50 16 53 6 33);
update lst:(numC#'s) from t
the last update will generate
numC s lst
1 50 50
3 16 16 16 16
6 53 53 53 53 53 53 53
4 6 6 6 6 6
1 33 33
How can I generate the "lst" column as per below?
numC s lst
1 50 50+0*mult
3 16 16+0*mult 16+1*mult 16+2*mult
6 53 53+0*mult 53+1*mult 53+2*mult 53+3*mult 53+4*mult 53+5*mult
4 6 6+0*mult 6+1*mult 6+2*mult 6+3*mult
1 33 33+0*mult
I tried something like
update lst:(numC#'s + (til numC)*mult) from t
but I am getting an error
ERROR: 'type
Thanks vm

Is this what you're looking for:
q)x:5
q)x#10+(til x)*2
10 12 14 16 18
http://code.kx.com/q/ref/arith-integer/#til

You can remove take # and use til to simplify to:
q)10+2*til 5
10 12 14 16 18
Using til will create a list of a list of 5 elements (0->4), so you will not need take 5 elements from the resulting list. Take will only be required if your list of indices is greater than 5.
Update:
For your second example the following should work:
q)update lst:{y+x*til z}'[mult;s;numC] from t
q)update lst:s+mult*til each numC from t
numC s lst
-------------------------
1 50 ,50
3 16 16 21 26
6 53 53 58 63 68 73 78
4 6 6 11 16 21
1 33 ,33

There are many ways with which we can get achieve this:
1) 10+2*til 5
2) (2*til 5) + 10
/ take operator: The dyadic take function creates lists. The left argument specifies the count and shape and the right argument provides the data.
It is useful for selecting from the front or end of a list.
https://code.kx.com/wiki/Reference/NumberSign
q)5#0 1 2 3 4 5 6 7 8 / take the first 5 items
0 1 2 3 4
q)-5#0 1 2 3 4 5 6 7 8 / take the last 5 elements
4 5 6 7 8
use take operator # only when it is required.
say we have 10 elements, of which we need five on output, then we can use:
5#10+2*til 10
/ The til function takes a non-negative integer argument X and returns the first X integers

Related

KDB/Q: multiple PEACH?

I have a function that takes 2 parameters: date and sym. I would like to do this for multiple dates and multiple sym. I have a list for each parameter. I can currently loop through 1 list using
raze function[2020.07.07;] peach symlist
How can I do something similar but looping through the list of dates too?
You may try following:
Create list of pairs of input parameters.
Write anonymous function which calls your function and use peachon list op paired parameters
For example
symlist: `A`B`C; // symlist defined for testing
function: {(x;y)}; // function defined for testing
raze {function . x} peach (2020.07.07 2020.07.08 2020.07.09) cross symlist
I think this could work:
raze function'[2020.07.07 2020.07.08 2020.07.09;] peach symlist
If not some more things to consider. Could you change your function to accept a sym list instead of individual syms by including an each/peach inside it? Then you could each the dates.
Also, you could create a new list of each date matched with the symlist and create a new function which takes this list and does whatever the initial function did by separating the elements of the list.
q)dates
2020.08.31 2020.09.01 2020.09.02
q)sym
`llme`obpb`dhca`mhod`mgpg`jokg`kgnd`nhke`oofi`fnca`jffe`hjca`mdmc
q)func
{[date;syms]string[date],/:string peach syms}
q)func2
{[list]func[list 0;list 1]}
q)\t res1:func[;sym]each dates
220
q)\t res2:func[;sym]peach dates
102
q)
q)func2
{[list]func[list 0;list 1]}
q)dateSymList:dates,\:enlist sym
q)\t res3:func2 peach dateSymList
80
q)res3~res2
1b
q)res3~res1
1b
Let us know if any of those solutions work, thanks.
Some possible ways to do this
Can project dyadic f as monadic & parallelise over list of argument pairs
q)a:"ABC";b:til 3;f:{(x;y)}
q)\s 4
q)(f .)peach l:raze a,\:/:b
"A" 0
"B" 0
"C" 0
"A" 1
"B" 1
"C" 1
"A" 2
"B" 2
"C" 2
Or could define function to take a dictionary argument & parallelise over a table
q)f:{x`c1`c2}
q)f peach flip`c1`c2!flip l
"A" 0
"B" 0
"C" 0
"A" 1
"B" 1
"C" 1
"A" 2
"B" 2
"C" 2
Jason
I'll generalize everything, if you have a given function foo which will operate on an atom dt with a vector s
q)foo:{[dt;s] dt +\: s}
q)dt:10?10
q)s:100?10
q)dt
8 1 9 5 4 6 6 1 8 5
q)s
4 9 2 7 0 1 9 2 1 8 8 1 7 2 4 5 4 2 7 8 5 6 4 1 3 3 7 8 2 1 4 2 8 0 5 8 5 2 8..
q)foo[;s] each dt
12 17 10 15 8 9 17 10 9 16 16 9 15 10 12 13 12 10 15 16 13 14 12 9 11 11 ..
5 10 3 8 1 2 10 3 2 9 9 2 8 3 5 6 5 3 8 9 6 7 5 2 4 4 ..
13 18 11 16 9 10 18 11 10 17 17 10 16 11 13 14 13 11 16 17 14 15 13 10 12 12 ..
9 14 7 12 5 6 14 7 6 13 13 6 12 7 9 10 9 7 12 13 10 11 9 6 8 8 ..
The solution is to project the symList over the function in question, then use each (or peach) for the date variable.
If your function requires an atomic date and sym, then you can just create a new function to implement this
q)bar:{[x;y] foo[x;] each y};
datelist:`date$10?10
symlist:10?`IBM`MSFT`GOOG
function:{0N!(x;y)}
{.[function;x]} each datelist cross symlist

How to return a cross product of a binary function in KDB

I would like to explore a better way to apply binary function which iterate via each element of the two argument. Let make the question simpler by using below function as an example:
func:{x+y}
a:til 10
q)a
0 1 2 3 4 5 6 7 8 9
b:a
q)b:a
q)b
0 1 2 3 4 5 6 7 8 9
What I what to get is the cross production such that each element of the argument will cross each other and apply the function. My expected result is
0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 3 4 5 6 7 8 9 10 11 12 4 5 6 7 8 9 10 11 12 13 5 6 7 8 9 10 11 12 13 14 6 7 8 9 10 11 12 13 14 15 7 8 9 10 11 12 13 14 15 16 8 9 10 11 12 13 14 15 16 17 9 10 11 12 13 14 15 16 17 18
My current solution is crossing the the list of argument first:
(func/) each (a cross b)
I wonder is there a better way to doing that? simply using func'[a;b] will just get a pairwise result which not what I want.
The following should be what you are looking for:
a +/:\: b
The same can apply for other defined functions too, for example:
a {x mod y}/:\: b
You do not need cross for this just each-right or each-left. Because '+' is a vector function you can just iterate over one list and use other list as full vector.
q) a+/:b

How to efficiently compare elements in two vectors in MATLAB without using loops?

Say I have a matrix A whose first column contains item IDs with repetition and second column contains their weights.
A= [1 40
3 33
2 12
4 22
2 10
3 6
1 15
6 29
4 10
1 2
5 18
5 11
2 8
6 25
1 14
2 11
4 28
3 38
5 35
3 9];
I now want to find the difference of each instance of A and its associated minimum weight. For that, I make a matrix B with its first column containing the unique IDs from column 1 of A, and its column 2 containing the associated minimum weight found from column 2 of A.
B=[1 2
2 8
3 6
4 10
5 11
6 25];
Then, I want to store in column 3 of A the difference of each entry and its associated minimum weight.
A= [1 40 38
3 33 27
2 12 4
4 22 12
2 10 2
3 6 0
1 15 13
6 29 4
4 10 0
1 2 0
5 18 7
5 11 0
2 8 0
6 25 0
1 14 12
2 11 3
4 28 18
3 38 32
5 35 24
3 9 3];
This is the code I wrote to do this:
for i=1:size(A,1)
A(i,3) = A(i,1) - B(B(:,1)==A(i,2),2);
end
But this code takes a long time to execute as it needs to loop through B every time it loops through A. That is, it has a complexity of size(A) x size(B). Is there a better way to do this without using loops, that would execute faster?
You can use accumarray to first compute the minimum value in the second column of A for each unique value in the first column of A. We can then index into the result using the first column of A and compare to the second column of A to create the third column.
% Compute the mins
min_per_group = accumarray(A(:,1), A(:,2), [], #min);
% Compute the difference between the second column and the minima
A(:,3) = A(:,2) - min_per_group(A(:,1));

Change orientation of buffer function

I need a function that splits a vector in smaller frames with an overlap, like buffer, but instead of column-wise, it should be done row-wise.
This is how buffer works:
x = 1:20
x = buffer(x, 10, 5);
x = 0 1 6 11
0 2 7 12
0 3 8 13
0 4 9 14
0 5 10 15
1 6 11 16
2 7 12 17
3 8 13 18
4 9 14 19
5 10 15 20
What I want would be this though:
x = 0 0 1 2
1 2 3 4
3 4 5 6
5 6 7 8
7 8 9 10
9 10 11 12
11 12 13 14
13 14 15 16
15 16 17 18
17 18 19 20
Is there any function or way to achieve that? Maybe combination of buffer + some rearranging?
First figure out the answer in columns, then transpose the resulting matrix:
buffer(x, 4, 2).'

Sorting a vector by the number of time each value occurs

We have the following case:
Q = [idxcell{:,1}];
Sort = sort(Q,'descend')
Sort =
Columns 1 through 13
23 23 22 22 20 19 18 18 18 18 17 17 17
Columns 14 through 26
15 15 14 14 13 13 13 12 12 12 11 10 9
Columns 27 through 39
9 9 8 8 8 8 8 7 7 7 7 7 7
Columns 40 through 52
7 6 6 6 5 4 4 3 3 3 3 2 2
Columns 53 through 64
2 2 2 2 2 2 2 1 1 1 1 1
How can we sort matrix Sort according to how many times its values are repeated?
Awaiting result should be:
repeatedSort = 2(9) 7(7) 1(5) 8(5) 3(4) 18(4) 6(3) 9(3) 12(3) 13(3) 17(3) 4(2) 14(2) 15(2) 22(2) 23(2) 5(1) 10(1) 11(1) 19(1) 20(1)
or
repeatedSort = 2 7 1 8 3 18 6 9 12 13 17 4 14 15 22 23 5 10 11 19 20
Thank you in advance.
You can use the TABULATE function from the Statistics Toolbox, then call SORTROWS to sort by the frequency.
Example:
x = randi(10, [20 1]); %# random values
t = tabulate(x); %# unique values and counts
t = t(find(t(:,2)),1:2); %# get rid of entries with zero count
t = sortrows(t, -2) %# sort according to frequency
the result, where first column are the unique values, second is their count:
t =
2 4 %# value 2 appeared four times
5 4 %# etc...
1 3
8 3
7 2
9 2
4 1
6 1
Here's one way of doing it:
d = randi(10,1,30); %Some fake data
n = histc(d,1:10);
[y,ii] = sort(n,'descend');
disp(ii) % ii is now sorted according to frequency