I'm playing around with an example on https://code.kx.com/q/ref/amend/#cross-sections
$ q
KDB+ 3.6 2019.04.02 Copyright (C) 1993-2019 Kx Systems
q)d:((1 2 3;4 5 6 7);(8 9;10;11 12);(13 14;15 16 17 18;19 20))
q)i:(2 0; 0 1 0)
q)y:(100 200 300; 400 500 600)
q)r:.[d; i; ,; y]
It all works fine except if I'll try to reference a single element d[1;1]:
q)i:(1 0; 0 1 0)
q)r:.[d; i; ,; y]
'type
[0] r:.[d; i; ,; y]
But if I use join , for lists and for just single element it works as supposed:
q)10,200
10 200
q)10,((),200)
10 200
So why the amend operation breaks on this simple join?
Upd:
One more example:
q)#[(1; 2; 3);1;,;10]
'type
[0] #[(1; 2; 3);1;,;10]
^
but it's ok if one use lists:
q)#[(1; (),2; 3);1;,;10]
1
2 10
3
The answer lies in examining the data you are extracting to join, here the operator -3! is your friend to help reveal the actual structure through multiple layers of enlistment
q)d:((1 2 3;4 5 6 7);(8 9;10;11 12);(13 14;15 16 17 18;19 20))
q)i1:(2 0; 0 1 0)
q)i2:(1 0; 0 1 0)
q)y:(100 200 300; 400 500 600)
q)-3!r1:.[d; i1]
"((13 14;15 16 17 18;13 14);(1 2 3;4 5 6 7;1 2 3))"
q)q){type each x} each .[d; i1]
7 7 7
7 7 7
q)-3!r2:.[d; i2]
"((8 9;10;8 9);(1 2 3;4 5 6 7;1 2 3))"
q){type each x} each .[d; i2]
7 -7 7
7 7 7
No here we can see that in the first case, each element of r1 is a list of lists, but for r2 the first element is 2 lists of longs with an atomic long 10.
From the cross section documentation
The shape of y is 2 3, the same shape as the cross-section selected by d . i
i.e., Shape should be the counts and types matching, the type of each item of y is a 7h, that should match the type of each selection from d.
Essentially when you are using the amendment operators , and # it will be expecting conformity as it is using an amend in place. a:1;a,:1 2 3 will also fail.
We can confirm this with your other examples
q)type #[(1; (),2; 3);1]
7h
Changing that example to adjust the first element
q)#[(1; (),2; 3);0;,;10]
'type
The reason that just using the , operator as 10,((),200) didn't cause any errors, is because you are using it outside the amend overloads, within the amend overload , is expecting to be working with matching shape. When used directly it can promote and adjust shape.
The only explanation I can offer is that kdb+ tries to use amend-in-place (also known as assignment through the operator) when it can.
For example, this works:
q)l:(13 14;15 16 17 18;19 20)
q)l[1],:200
q)l
13 14
15 16 17 18 200
19 20
But this doesn't:
q)l:(8 9;10;11 12)
q)l[1],:200
'type
[0] l[1],:200
^
The latter fails because kdb can't substitute the vector 10 200 for the atom 10 -- changing types is not something that amend-in-place is supposed to do.
If you used your own function {x,y} instead of the plus operator the second
expression from your example would work as expected too because kdb will replace existing values with return values of the function (which, in contrast to the built-in ,, is a black box as far as kdb is concerned):
q)i:(1 0; 0 1 0)
q)r:.[d; i; ,; y]
q).[d; ii; {x,y}; y]
(1 2 3 400 600;4 5 6 7 500)
(8 9 100 300;10 200;11 12)
(13 14;15 16 17 18;19 20)
Related
Let's say I've got a function that defines a matrix in terms of it's i and j coordinates:
f: {y+2*x}
I'm trying to create a square matrix that evaluates this function at all locations.
I know it needs to be something like f ' (til 5) /:\: til 5, but I'm struggling with rest.
Rephrasing your question a bit, you want to create a matrix A = [aij] where aij = f(i, j), i, j = 0..N-1.
In other words you want to evaluate f for all possible combinations of i and j. So:
q)N:5;
q)i:til[N] cross til N; / all combinations of i and j
q)a:f .' i; / evaluate f for all pairs (i;j)
q)A:(N;N)#a; / create a matrix using #: https://code.kx.com/q/ref/take/
0 1 2 3 4
2 3 4 5 6
4 5 6 7 8
6 7 8 9 10
8 9 10 11 12
P.S. No, (til 5) /:\: til 5 is not exactly what you'd need but close. You are generating a list of all pairs i.e. you are pairing or joining the first element of til 5 with every element of (another) til 5 one by one, then the second , etc. So you need the join operator (https://code.kx.com/q/ref/join/):
(til 5),/:\: til 5
You were close. But there is no need to generate all the coordinate pairs and then iterate over them. Each Right Each Left /:\: manages all that for you and returns the matrix you want.
q)(til 5)f/:\:til 5
0 1 2 3 4
2 3 4 5 6
4 5 6 7 8
6 7 8 9 10
8 9 10 11 12
If I have this matrix:
A:
X Y Z
1 1 2
0 3 4
0 5 6
2 7 8
7 9 10
8 11 12
3 13 14
12 14 16
15 17 18
How could I create new matrix B, C, D and E which contains:
B:
0 3 4
0 5 6
C:
X Y Z
1 1 2
2 7 8
3 13 14
D:
7 9 10
8 11 12
E:
12 14 16
15 17 18
The idea is to construct a loop asking if 0<A<1 else 1<A<5 else 6<A<10 else 11<A<15. and create new matrix from that condition. Any idea about how to store the results of the loop?
I suggest you an approach that uses the discretize function in order to group the matrix rows into different categories based on their range. Here is the full implementation:
A = [
1 1 2;
0 3 4;
0 5 6;
2 7 8;
7 9 10;
8 11 12;
3 13 14;
12 14 16;
15 17 18
];
A_range = [0 1 5 10 15];
bin_idx = discretize(A(:,1),A_range);
A_split = arrayfun(#(bin) A(bin_idx == bin,:),1:(numel(A_range) - 1),'UniformOutput',false);
celldisp(A_split);
Since you want to consider 5 different ranges based on the first column values, the arguments passed to discretize must be the first matrix column and a vector containing the group limits (first number inclusive left, second number exclusive right, second number inclusive left, third number exclusive right, and so on...). Since your ranges are a little bit messed up, feel free to adjust them to respect the correct output. The latter is returned in the form of a cell array of double matrices in which every element contains the rows belonging to a distinct group:
A_split{1} =
0 3 4
0 5 6
A_split{2} =
1 1 2
2 7 8
3 13 14
A_split{3} =
7 9 10
8 11 12
A_split{4} =
12 14 16
15 17 18
Instead of using a loop, use logical indexing to achieve what you want. Use the first column of A and check for the ranges that you want to look for, then use this to subset into the final matrix A to get what you want.
For example, to create the matrix C, find all locations in the first column of A that are between 1 and 5, then subset the matrix along the rows using these locations:
m = A(:,1) >= 1 & A(:,1) <= 5;
C = A(m,:);
You can repeat this in a similar way for the rest of the matrices you want to create.
I have a matrix A in Matlab of dimension Nx(N-1), e.g.
N=5;
A=[1 2 3 4;
5 6 7 8;
9 10 11 12;
13 14 15 16;
17 18 19 20];
I want to rearrange the elements of A in a certain way. Specifically I want to create a matrix B of dimension (N-1)xN such that:
for i=1,...,N,
B(:,i) collects
1) the first i-1 elements of the i-1th column of A and
2) the last N-i elements of the ith column of A.
Notice that for i=1 the i-1th column of A does not exist and therefore 1) is skipped; similarly, for i=N theith column of A does not exist and therefore 2) is skipped.
In the example above
B=[5 1 2 3 4
9 10 6 7 8
13 14 15 11 12
17 18 19 20 16];
This code does what I want. I am asking your help to vectorise it in an efficient way.
B=zeros(N-1,N);
for i=1:N
if i>1 && i<N
step1=A(1:i-1,i-1);
step2=A(i+1:N,i);
B(:,i)=[step1;step2];
elseif i==1
B(:,i)=A(i+1:N,i);
elseif i==N
B(:,i)=A(1:i-1,i-1);
end
end
Extract the lower and upper triangular matrices of A. Then reassemble them with a "diagonal shift":
u = triu(A);
l = tril(A,-1);
B = padarray(u(1:end-1,:),[0 1],'pre') + padarray(l(2:end,:),[0 1],'post');
Another valid approach using logical indexing combined with tril and triu:
B = zeros(size(A'));
B(tril(true(size(B)))) = A(tril(true(size(A)), -1));
B(triu(true(size(B)), 1)) = A(triu(true(size(A))));
Result:
>> B
B =
5 1 2 3 4
9 10 6 7 8
13 14 15 11 12
17 18 19 20 16
If I am to split a list into 2 rows, I can use:
q)2 0N#til 10
However, the following syntax does not work:
q)n:2
q)n 0N#til 10
how I can achieve such reshaping?
Need brackets and semi colon
q)2 0N#til 10
0 1 2 3 4
5 6 7 8 9
q)n:2
q)(n;0N)#til 10
0 1 2 3 4
5 6 7 8 9
Here is the general syntax to split a list in matrix form:
(list1)#(list2)
As you can see, left part and right part of '#' is list. So here is one example:
q)list1: (4;3) / or simply (4 3)
q)list2: til 12
q)list1#list2
We can make an integer list in 2 way:
Using semicolon as list1:(2;3;4)
Using spaces as list1:(2 3 4)
But when you have variable, option 2 doesn't work;
q)list1: (n 3) / where n:2
q) `type error
So for your question, solution is to use semicolon to create list:
q) list1:(n;0N)
q) list1#til 10
If
a =
1
2
3
3.5
5
6
6.25
8
9
9.75
11
12
13
14
15
How it is possible to switch numbers which are in a specific constrains, into the smaller part of constrain?
In other word, if b=min(a) & c=b+threshold
b<a<c => a=b.
It means, in this example, let's assume threshold=3. Then "a" should change into a matrix which min(a):min(a)+threshold =[min(a)]. This procedure should continue to the last array of "a"matrix by adding each time the amount of threshold. The result should be like this:
a=
1
1
1
1
5
5
5
5
9
9
9
9
13
13
13
I guess finally I found a solution for that. Sorry if I couldn't explain better my intention and mislead you. Here is the code:
a = [1 2 3 3.5 5 6 6.25 8 9 9.75 11 12 13 14 15]';
b = min(a);
threshold=4;
c=b;
for i=1:size(a,1)
c =c+threshold;
a((a > b & a < c)) = b;
b=b+threshold;
if c==max(a);
break
end
end
P.S. threshold change to "threshold=4" instead of "3"
Logical indexing
Edit, based on the comment thread
I can't think of a way to avoid a loop here, nor am I really sure how to constrain it efficiently, but the general process is the same:
% Initial conditions
threshold = 3;
a = [1 2 3 3.5 5 6 6.25 8 9 9.75 11 12 13 14 15];
b = min(a);
c = b + threshold;
a((a > b & a < c)) = b;
for/while/if % Need some way to constrain the loop
b = c;
c = c+threshold;
a((a > b & a < c)) = b;
end