KDB '.' operator - kdb

The . operator in the simplest form is used to index a list. How would you explain its use in english in this code?
if[x~"last";upd:{[t;x].[t;();,;r::select by sym from x]}]
I also don't understand the empty list and the :: operator in this line, but maybe they will make sense once the . is cleared up.

In plain english I would explain it as:
modify the table t at all () indices by applying the append/comma function with the value r.
First consider a few simpler cases of #:
q)l:3 5 7 9
q)l:1.1 2.2 3.3
q)#[l; 0 2; +; 10]
11.1 2.2 13.3
q)d:`p`o`i!4.4 5.5 6.6
q)#[d; `p`i; -; 10]
p| -5.6
o| 5.5
i| -3.4
As you can see the format is
#[dataStructure; indices; function; y-arg]
means to the dataStructure at indices apply the function with the given y-arguments. Notice for the list l the indices 0 2 meant index both 0 and 2 at the topmost level. There's no way using # to index at depth. e.g. given matrix m:(1 2 3; 4 5 6; 7 8 9) how can we use this format to modify just the values 4 and 6?
q)/ # indexes repeatedly at topmost level
q)/ definitely not what we want
q)#[m;(1;0 2);+;100]
101 102 103
104 105 106
107 108 109
q)/ **. indexes into the data structure**
q).[m;1 2;+;100]
1 2 3
4 5 106
7 8 9
q).[m;(1;0 2);+;100]
1 2 3
104 5 106
7 8 9
Lastly the empty list () is a short way of saying, apply to all indices:
q).[m;();+;100]
101 102 103
104 105 106
107 108 109

. In this case means apply , to t and r. r is global updated on each call and contains last values recieved by sym. :: is assignment to global in most cases.
code.kx.com describe . function in great details

Related

kdb: longList#dictionary behavior

q)(`a`b`c!101 0N 103)~100 101 102 103 104#`a`b`c!1 5 3
1b
I noticed that below two lines are equivalent
100 101 102 103 104#`a`b`c!1 5 3
`a`b`c!100 101 102 103 104#1 5 3
Is it in general true that a#b!c is equivalent to b!a#c?
On the overloaded glyphs page for #, is this usage Index At or Apply At or something else? For x#y is there documentation on the exact behavior when y is a dictionary?
An operation that works on a list will also work on a list that just so happens to be the value of a dictionary (it applies "through" the dictionary and keeps the keys untouched). E.g.:
q)1+`a`b!1 2
a| 2
b| 3
q)2*`a`b!1 2
a| 2
b| 4
Thus an index at would apply the same way:
q)10 20 30#`a`b!1 2
a| 20
b| 30
since it's the same as:
q)10 20 30#1 2
20 30
It's not always true that a#b!c is equivalent to b!a#c if the a#c doesn't make sense. E.g.
q)10 20 30#1.1 2.2
'type
[0] 10 20 30#1.1 2.2
^
q)10 20 30#`a`b!1.1 2.2
'type
[0] 10 20 30#`a`b!1.1 2.2
^
It would only be true if the underlying datatype of the indexes is boolean/short/int/long.
Answered above mainly, it's index at. There's no explicit documentation for when y is a dictionary as it essentially is the same behaviour when y is a list

How can I efficiently convert the output of one KDB function into three table columns?

I have a function that takes as input some of the values in a table and returns a tuple if you will - three separate return values, which I want to transpose into the output of a query. Here's a simplified example of what I want to achieve:
multiplier:{(x*2;x*3;x*3)};
select twoX:multiplier[price][0]; threeX:multiplier[price][1]; fourX:multiplier[price][2] from data;
The above basically works (I think I've got the syntax right for the simplified example - if not then hopefully my intention is clear), but is inefficient because I'm calling the function three times and throwing away most of the output each time. I want to rewrite the query to only call the function once, and I'm struggling.
Update
I think I missed a crucial piece of information in my explanation of the problem which affects the outcome - I need to get other data in the query alongside the output of my function. Here's a hopefully more realistic example:
multiplier:{(x*2;x*3;x*4)};
select average:avg price, total:sum price, twoX:multiplier[sum price][0]; threeX:multiplier[sum price][1]; fourX:multiplier[sum price][2] by category from data;
I'll have a go at adapting your answers to fit this requirement anyway, and apologies for missing this bit of information. The real function if a proprietary and fairly complex algorithm and the real query has about 30 output columns, hence the attempt at simplifying the example :)
If you're just looking for the results themselves you can extract (exec) as lists, create dictionary and then flip the dictionary into a table:
q)exec flip`twoX`threeX`fourX!multiplier[price] from ([]price:til 10)
twoX threeX fourX
-----------------
0 0 0
2 3 4
4 6 8
6 9 12
8 12 16
10 15 20
12 18 24
14 21 28
16 24 32
18 27 36
If you need other columns from the original table too then its trickier but you could join the tables sideways using ,'
q)t:([]price:til 10)
q)t,'exec flip`twoX`threeX`fourX!multiplier[price] from t
An apply # can also achieve what you want. Here data is just a table with 10 random prices. # is then used to apply the multiplier function to the price column while also assigning a column name to each of the three resulting lists:
q)data:([] price:10?100)
q)multiplier:{(x*2;x*3;x*3)}
q)#[data;`twoX`threeX`fourX;:;multiplier data`price]
price twoX threeX fourX
-----------------------
80 160 240 240
24 48 72 72
41 82 123 123
0 0 0 0
81 162 243 243
10 20 30 30
36 72 108 108
36 72 108 108
16 32 48 48
17 34 51 51

How to add some value in the specific range in a matrix?

How to add the value in specific range in a matrix?
I mean if i have a matrix
Columns 1 through 7
4 4 4 4 4 4 4
48 48 48 48 48 48 48
Columns 8 through 14
4 4 4 13 13 13 13
48 48 48 57 57 57 57
Columns 15 through 20
13 13 13 13 13 13
57 57 57 57 57 57
I want to sum all 4 values ,all 13 values,all 48 values,and all 57 values,so the result should be m=[40 130 480 570]
The easiest but stupid method is like this
a=sum(1,(1:10));
b=sum(1,(11:20));
c=sum(2,(1:10));
d=sum(2,(11:20));
m=[a b c d]
If i want to write a code with for-loop or while-loop to show the result i want.how do i write the code?
Or can i use the some method to write a code without loop to show this?
Though the solution of #phnx works fine, you can also use the other outputs of the unique function in combination with accumarray as described in the docs:
[C, ~, ic] = unique(a);
a_counts = accumarray(ic,1);
m = C.*a_counts
This will avoid the warning 'hist' is not recommended...
A simple two-line solution, with A as your original matrix, would be:
[a,b]=hist(A(:),unique(A(:)))
c = a .* b'
with a containing the number of occurances, b the unique elements and c the sums.

Matlab replace consecutive zero value with others value

I have this matrix:
A = [92 92 92 91 91 91 146 146 146 0
0 0 112 112 112 127 127 127 35 35
16 16 121 121 121 55 55 55 148 148
0 0 0 96 96 0 0 0 0 0
0 16 16 16 140 140 140 0 0 0]
How can I replace consecutive zero value with shuffled consecutive value from matrix B?
B = [3 3 3 5 5 6 6 2 2 2 7 7 7]
The required result is some matrix like this:
A = [92 92 92 91 91 91 146 146 146 0
6 6 112 112 112 127 127 127 35 35
16 16 121 121 121 55 55 55 148 148
7 7 7 96 96 5 5 3 3 3
0 16 16 16 140 140 140 2 2 2]
You simply can do it like this:
[M,N]=size(A);
for i=1:M
for j=1:N
if A(i,j)==0
A(i,j)=B(i+j);
end
end
end
If I understand it correctly from what you've described, your solution is going to need the following steps:
Loop over the rows of your matrix, e.g. for row = 1:size(A, 1)
Loop over the elements of each row, identify where each run of zeroes starts and store the indices and the length of the run. For example you might end up with a matrix like: consecutiveZeroes = [ 2 1 2 ; 4 1 3 ; 4 6 5 ; 5 8 3 ] indicating that you have a run at (2, 1) of length 2, a run at (4, 1) of length 3, a run at (4, 6) of length 5, and a run at (5, 8) of length 3.
Now loop over the elements of B counting up how many elements there are of each value. For example you might store this as replacementValues = [ 3 3 ; 2 5 ; 2 6 ; 3 2 ; 3 7 ] meaning three 3's, two 5's, two 6's etc.
Now take a row from your consecutiveZeroes matrix and randomly choose a row of replacementValues that specifies the same number of elements, replace the zeroes in A with the values from replacementValues, and delete the row from replacementValues to show that you've used it.
If there isn't a row in replacementValues that describes a long enough run of values to replace one of your runs of zeroes, find a combination of two or more rows from replacementValues that will work.
You can't do this with just a single pass through the matrix, because presumably you could have a matrix A like [ 15 7 0 0 0 0 0 0 3 ; 2 0 0 0 5 0 0 0 9 ] and a vector B like [ 2 2 2 3 3 3 7 7 5 5 5 5 ], where you can only achieve what you want if you use the four 5's and two 7's and not the three 2's and three 3's to substitute for the run of six zeroes, because you have to leave the 2's and 3's for the two runs of three zeroes in the next row. The easiest approach if efficiency is not critical would probably be to run the algorithm multiple times trying different random combinations until you get one that works - but you'll need to decide how many times to try before giving up in case the input data actually has no solution.
If you get stuck on any of these steps I suggest asking a new, more specific question.

lookup columns and match in matlab

I have an excel_1 with 4 columns (A, B, C, score) with different combinations. I have another excel_2 with 3 columns (A, B, C). I would like according A, B, C and find out score. I'm trying using excel index and match function, but i still can not figure out. without coding, it really make me trouble to match it one by one...May i know how to write code in Matlab?
Example:
excel_1
99 5 35 12
99 2 32 14
97 5 13 94
...
excel_2
97 5 13
99 2 32
...
After execute the code,
Result:
excel_2
97 5 13 94
99 2 32 14
...
Thanks a lot...
A = xlsread('excel_1.xlsx');
B = xlsread('excel_2.xlsx');
[~,J] = ismember(B,A(:,1:size(B,2)),'rows');
if any(J)
result = A(J,:);
end
??? Subscript indices must either be real positive integers or logicals.
Error in ==> Untitled at 6
result = A(J,:);
Solved. can not exist impossible between B and A.
May be you can try this.
[~,J] = ismember(excel_2,excel_1(:,1:size(excel_2,2)),'rows');
if any(J)
result = excel_1(J,:);
end
Then
result =
97 5 13 94
99 2 32 14