How do I test a range in KDB? - kdb

Let's say I have a range 3 < x < 5.
What's the cleanest way to get 1 if x > 5, -1 if x < 3 and 0 if it's in the range 3 <x <5?
I'm trying to mix together & and | (trying to adapt the answer from here: In KDB/Q, how do I clip numbers to be in the range -1, 1?). I'm not sure if it's possible this way though.

You could adapt the bin keyword which does something like what you're looking for out of the box:
q)3 5 bin 0 1 2 3 4 5 6
-1 -1 -1 0 0 1 1
If you are looking for 0 to be returned for n where x < n < y then I think you may be able to do something like:
q)f:{[range;x] (range + 1 0) bin x}
q)range35:f[3 5;]
q)range35 0N!til 10
0 1 2 3 4 5 6 7 8 9
-1 -1 -1 -1 0 1 1 1 1 1
Which returns a 0 for the number 4

You could look at a step dictionary which can be used in many places:
q)(`s#-0W 3 6!-1 0 1) til 10
-1 -1 -1 0 0 0 1 1 1 1
Or fill the base case:
q)-1^(`s#3 6!0 1) til 10
-1 -1 -1 0 0 0 1 1 1 1
Even better here for your exact question is to go direct to using bin:
q)3 6 bin til 10
-1 -1 -1 0 0 0 1 1 1 1

Another option:
q){$[x<3;-1;x>5;1;0]}each til 10
-1 -1 -1 0 0 0 1 1 1 1

As an aside Sean and Rian's answers using bin are faster than using conditional evaluation - however in this context its already "fast". For readability's sake I would select the conditional evaluation due to how clear and easy it is to read, understand and modify the conditions and returns.
q)\t:10000 {[range;x] (range + 1 0) bin x}[3 5;-50+til 100]
11
q)\t:10000 3 6 bin -50+til 100
10
q)\t:10000 {$[x<3;-1;x>5;1;0]}'[-50+til 100]
75

Related

Convert Group of Picture structure to the appropriate one for HEVC HM encoder

I'm using HM-16 and Scalable HM 12.3.
I have this GOP order as seen below. As you can see I have QP value for each slice type.
Encode Order Type POC QP
0 I-SLICE 0 23
1 P-SLICE 3 26
2 B-SLICE 2 27
3 b-SLICE 1 28
4 P-SLICE 6 26
5 B-SLICE 5 27
6 b-SLICE 4 28
And I want to convert it to a code like the following where I must define QPoffset CbQPoffset CrQPoffset QPfactor tcOffsetDiv2 betaOffsetDiv2 temporal_id #ref_pics_active #ref_pics reference pictures predict deltaRPS #ref_idcs reference idcs.
# Type POC QPoffset CbQPoffset CrQPoffset QPfactor tcOffsetDiv2 betaOffsetDiv2 temporal_id #ref_pics_active #ref_pics reference pictures predict deltaRPS #ref_idcs reference idcs
Frame1: P 16 1 0 0 0.6 0 0 0 2 3 -16 -24 -32 0
Frame2: B 8 2 0 0 0.2 0 0 1 2 3 -8 -16 8 1 8 4 1 1 0 1
Can you please help me to convert it?
Is there any other way to define the number of B-frames or b-frames in a GOP?
I found the solution considering HM reference manual.
I reorder the frames starting from a B-frame considering only one reference frame. All B frames reference previous I-frame or P-frames.
The new GOP structure is the one below.
It's important to mention here, that a B-frame can't have a P-frame that is encoded later. More explicitly, a B-frame with POC number e.g. 1 can't reference a P-frame with a grater value of POC number e.g. 3.
# Type POC QPoffset CbQPoffset CrQPoffset QPfactor tcOffsetDiv2 betaOffsetDiv2 temporal_id #ref_pics_active #ref_pics reference pictures predict deltaRPS #ref_idcs reference idcs
Frame1: B 1 2 0 0 0.4624 0 0 0 1 1 -1 0
Frame2: B 2 1 0 0 0.4624 0 0 0 1 1 -2 2 1
Frame3: P 3 0 0 0 0.4624 0 0 0 1 1 -3 2 2
Frame4: B 4 2 0 0 0.4624 0 0 0 1 1 -1 2 2
Frame5: B 5 1 0 0 0.4624 0 0 0 1 1 -2 2 3
Frame6: P 6 0 0 0 0.4624 0 0 0 1 1 -3 2 3

How to normalize matrix setting 0 for minimum values and 1 for maximum values?

I need to transform a neural network output matrix with size 2 X N in zeros and ones, where 0 will represent the minimum value of the column and 1 contrariwise. This will be necessary in order to calculate the confusion matrix.
For example, consider this matrix 2 X 8:
2 33 4 5 6 7 8 9
1 44 5 4 7 5 2 1
I need to get this result:
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
How can I do this in MATLAB without for loops? Thanks in advance.
>> d = [ 2 33 4 5 6 7 8 9;
1 44 5 4 7 5 2 1];
>> bsxfun(#rdivide, bsxfun(#minus, d, min(d)), max(d) - min(d))
ans =
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
The bsxfun function is necessary to broadcast the minus and division operations to matrices of different dimensions (min and max have only 1 row each).
Other solution is the following (works only for 2 rows):
>> [d(1,:) > d(2,:); d(1,:) < d(2,:)]
ans =
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
If it's just 2xN, then this will work:
floor(A./[max(A); max(A)])
In general:
floor(A./repmat(max(A),size(A,1),1))

MATLAB - Inserting zero rows and columns into matrix

I have written some code that compresses a matrix to remove zero columns and rows, but I can't work out how to reconstruct the original matrix.
Say I have a matrix:
A = [ 0 3 0 2 1 0 6
3 0 0 4 8 0 5
0 0 0 0 0 0 0
2 4 0 0 2 0 1
1 8 0 2 0 0 7
0 0 0 0 0 0 0
6 5 0 1 7 0 0 ]
Here rows/columns 3 and 6 are empty, so my compression function will give the output:
A_dash = [ 0 3 2 1 6
3 0 4 8 5
2 4 0 2 1
1 8 2 0 7
6 5 1 7 0 ]
A_map = [ 1 2 4 5 7]
Where A_map is a vector mapping the indicies of the rows/columns of A_dash to A. This means that if A_map(3) = 4, then row/column 4 of A is the same as row/column 3 of A_dash - ie. a row/column of zeroes must be inserted between columns/rows 2 and 3 in A_dash
What is the easiest way people can suggest for me to recreate matrix A from A_dash, using the information in A_map?
Here is what I have got so far:
% orig_size is original number of columns/rows
c_count = size(A_dash,1);
A = zeros(c_count, orig_size); % c_count rows to avoid dimension mismatch
for ii = 1:c_count
A(:,A_map(ii)) == A_dash(:,ii);
end
This gives me the right result column-wise:
A = [ 0 3 0 2 1 0 6
3 0 0 4 8 0 5
2 4 0 0 2 0 1
1 8 0 2 0 0 7
6 5 0 1 7 0 0 ]
However, I'm not sure how i should go about inserting the rows, i suppose i could copy the first 1:i rows into one matrix, i:end rows to a second matrix and concatenate those with a zero row in between, but that feels like a bit of a
clunky solution, and probably not very efficient for large sized matrices..
Otherwise, is there a better way that people can suggest I store the map information? I was thinking instead of storing the mapping between column/row indices, that I just store the indices of the zero columns/rows and then insert columns/rows of zeros where appropriate. Would this be a better way?
You've got the indices of the valid rows/columns. Now all you've got to do is put them in a new matrix of zeros the same size as A:
B=zeros(size(A));
B(A_map,A_map)=A_dash
B =
0 3 0 2 1 0 6
3 0 0 4 8 0 5
0 0 0 0 0 0 0
2 4 0 0 2 0 1
1 8 0 2 0 0 7
0 0 0 0 0 0 0
6 5 0 1 7 0 0
Just to check...
>> A==B
ans =
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
A and B are equal everywhere, so we've reconstructed A.

Confusion about fftshift and fft2 in MATLAB

I'm mighty confused about how to correctly apply the FFT transform and its inverse in Matlab. I have a program where I need to
apply FFT2 to a matrix of size 4x32 (corresponding to modes m=-1:2, n=-15:16)
do some processing, which leads to a coefficient matrix for another function, whose Fourier coefficients relate to the first set of data by a simple algebraic (component-wise) formula
Use some properties of the two functions that lets me calculate what I need by just summing up an expression on the form 2*abs(A_n)*cos(phi+n*theta+alpha_n) where A_n is the nth coefficient of the m=1 mode and alpha_n = arg(A_n).
I have experimented a little with the FFT2 function and tried to understand how it arranges its output. From what I understand (from sources in my course literature), the coefficients will be ordered as is illustrated by the following script:
>>m = -1:2; n = -7:8;
>>[N,M] = meshgrid(n,m);
>>MN = M; MN(:,:,2) = N;
>>asfft = #(X) [X(2:4,8:16,:) X(2:4,1:7,:); X(1,8:16,:) X(1,1:7,:)];
>>asfft(MN)
ans(:,:,1) =
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
ans(:,:,2) =
0 1 2 3 4 5 6 7 8 -7 -6 -5 -4 -3 -2 -1
0 1 2 3 4 5 6 7 8 -7 -6 -5 -4 -3 -2 -1
0 1 2 3 4 5 6 7 8 -7 -6 -5 -4 -3 -2 -1
0 1 2 3 4 5 6 7 8 -7 -6 -5 -4 -3 -2 -1
where asfft reorders the indices in the same way I believe fft2 does, but does nothing else. In other words, each index is ordered from 0 up to the max, then from the min to -1. According to the documentation, I should be able to rearrange this so I get 0 in the middle by using fftshift, but it's not giving me the output I expect. Instead, I get this:
>> fftshift(asfft(MN))
ans(:,:,1) =
8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
ans(:,:,2) =
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
As you can see, the max is on the wrong side of the spectrum - instead of -7 -6 ... -1 0 1 ... 8 and -1 0 1 2 I have 8 -7 6 ... and 2 -1 0 1. This is fatal for me, since to be able to do the calculation described in 3 above, I need to know the indices of the respective coefficients. (The two layers are also switched, but that doesn't matter for me since I'm only going to do this on MxN matrices, not on N-d-arrays, later.)
Why is this? What am I doing wrong here?
Considering the simple 1D case first, fft gives you:
[X(0) X(1) X(2) ... X(N/2-1) X(-N/2) X(-N/2+1) ... X(-1)]
where I'm using X(k) to denote elements of the mathematical DFT.
All fftshift does is rotate these by N/2, so you end up with:
[X(-N/2) X(-N/2+1) ... X(-1) X(0) X(1) X(2) ... X(N/2-1)]
i.e. linear order.1
In the multi-dimensional case, fftshift simply applies this rotation in all dimensions.
1. Although since by definition, X(k) == X(N+k) for a DFT, the unrotated vector is also in "linear" order!
Check this newsread entry too
http://www.mathworks.com/matlabcentral/newsreader/view_thread/285244
taking a queue from Oli, the problem is that you're not parsing it in the middle:
so for asfft = #(X) [X(2:4,8:16,:) X(2:4,1:7,:); X(1,8:16,:) X(1,1:7,:)];
size(8:16)
ans =
1 9
size(1:7)
ans =
1 7
so do:
asfft = #(X) [X(3:4,9:16,:) X(3:4,1:8,:); X(1:2,9:16,:) X(1:2,1:8,:)];

MATLAB, what is the best way to trace a boarder in a matrix that is changing each step?

I want to calculate the average slope or gradient at each iteration in such a matrix.
a=[ 10 9 8 7 6 5 5;
9 9 8 7 8 5 5;
8 8 7 7 5 5 5;
7 7 7 6 5 5 5;
6 6 6.6 5 5 5 5;
6 6 6.1 5 5 5 5;
6.3 5 5 5 5 5 5]
Where I am wanting to find the slope or gradient between the a(1,1) position during each step and at each point that boarders a value of 5. Each iteration the position of the 5's changes and so do the other values.
After doing so I will then average the slope. I haven't encountered a problem like this yet and I could not find a Matlab command to simplify.
First you must find out which the coast elements are. From your definition, an element is a coast element if it border (from the right) with a 5. If the sea level is 5, and is the lowest possible value i.e. no element goes beyond sea level, then you must first find all the land elements as,
land=a>5;
This returns,
ans =
1 1 1 1 1 0 0
1 1 1 1 1 0 0
1 1 1 1 0 0 0
1 1 1 1 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 0 0 0 0 0 0
Now, the coast elements are 1s that are followed by a 0. Take the column difference of the land matrix,
coastTmp=diff(land,1,2);
returning,
ans =
0 0 0 0 -1 0
0 0 0 0 -1 0
0 0 0 -1 0 0
0 0 0 -1 0 0
0 0 -1 0 0 0
0 0 -1 0 0 0
-1 0 0 0 0 0
and find the -1s,
coast=find(coastTmp==-1);
which are,
coast =
7
19
20
24
25
29
30
From here it is easy. The gradient is the difference of a(1,1) with all the coast elements, i.e.
slope=a(coast)-a(1,1); % negative slope here
giving,
slope =
-3.700000000000000
-3.400000000000000
-3.900000000000000
-3.000000000000000
-4.000000000000000
-4.000000000000000
-2.000000000000000
and of course the mean is,
mean(slope);