I need to transform a neural network output matrix with size 2 X N in zeros and ones, where 0 will represent the minimum value of the column and 1 contrariwise. This will be necessary in order to calculate the confusion matrix.
For example, consider this matrix 2 X 8:
2 33 4 5 6 7 8 9
1 44 5 4 7 5 2 1
I need to get this result:
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
How can I do this in MATLAB without for loops? Thanks in advance.
>> d = [ 2 33 4 5 6 7 8 9;
1 44 5 4 7 5 2 1];
>> bsxfun(#rdivide, bsxfun(#minus, d, min(d)), max(d) - min(d))
ans =
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
The bsxfun function is necessary to broadcast the minus and division operations to matrices of different dimensions (min and max have only 1 row each).
Other solution is the following (works only for 2 rows):
>> [d(1,:) > d(2,:); d(1,:) < d(2,:)]
ans =
1 0 0 1 0 1 1 1
0 1 1 0 1 0 0 0
If it's just 2xN, then this will work:
floor(A./[max(A); max(A)])
In general:
floor(A./repmat(max(A),size(A,1),1))
Related
I am having trouble creating this matrix in matlab, basically I need to create a matrix that has -1 going across the center diagonal followed be 4s on the diagonal outside of that (example below). All the other values can be zero.
A5 = [-1 4 0 0 0;
4 -1 4 0 0;
0 4 -1 4 0;
0 0 4 -1 4;
0 0 0 4 -1];
I have tried using a command v = [4]; D = diag(v)
but that only works for the center diagonal.
This can also be done using a toeplitz matrix:
function out = tridiag(a,b,c,N)
% TRIDIAG generates a tri-diagonal matrix of size NxN.
% lower diagonal is a
% main diagonal is b
% upper diagonal is c
out = toeplitz([b,a,zeros(1,N-2)],[b,c,zeros(1,N-2)]);
>> tridiag(4,-1,4,5)
ans =
-1 4 0 0 0
4 -1 4 0 0
0 4 -1 4 0
0 0 4 -1 4
0 0 0 4 -1
Note #1: When your desired output is symmetric, you can omit the 2nd input to toeplitz.
Note #2: As the size of the matrix increases, there comes a point where it makes more sense to store it as sparse, as this saves memory and improves performance (assuming your matrix is indeed sparse, i.e. comprised mostly of zeros, as it happens with a tridiagonal matrix). Some useful functions are spdiags, sptoeplitzFEX and blktridiagFEX.
A little hackish, but here it goes:
N = 7; % matrix size
v = [11 22 33]; % row vector containing the diagonal values
w = [0 v(end:-1:1)];
result = w(max(numel(v)+1-abs(bsxfun(#minus, 1:N, (1:N).')),1))
This gives
result =
11 22 33 0 0 0 0
22 11 22 33 0 0 0
33 22 11 22 33 0 0
0 33 22 11 22 33 0
0 0 33 22 11 22 33
0 0 0 33 22 11 22
0 0 0 0 33 22 11
To understand how it works, see some intermediate steps:
>> abs(bsxfun(#minus, 1:N, (1:N).'))
ans =
0 1 2 3 4 5 6
1 0 1 2 3 4 5
2 1 0 1 2 3 4
3 2 1 0 1 2 3
4 3 2 1 0 1 2
5 4 3 2 1 0 1
6 5 4 3 2 1 0
>> max(numel(v)+1-abs(bsxfun(#minus, 1:N, (1:N).')),1)
ans =
4 3 2 1 1 1 1
3 4 3 2 1 1 1
2 3 4 3 2 1 1
1 2 3 4 3 2 1
1 1 2 3 4 3 2
1 1 1 2 3 4 3
1 1 1 1 2 3 4
Use D = diag(u,k) to shift u in k levels above the main diagonal, and D = diag(u,-k) for the opposite direction. Keep in mind that you need u to be in the right length of the k diagonal you want, so if the final matrix is n*n, the k's diagonal will have only n-abs(k) elements.
For you case:
n = 5; % the size of the matrix
v = ones(n,1)-2; % make the vector for the main diagonal
u = ones(n-1,1)*4; % make the vector for +1 and -1 diagonal
A5 = diag(v)+diag(u,1)+diag(u,-1) % combine everything together
Which gives:
A5 =
-1 4 0 0 0
4 -1 4 0 0
0 4 -1 4 0
0 0 4 -1 4
0 0 0 4 -1
I have written some code that compresses a matrix to remove zero columns and rows, but I can't work out how to reconstruct the original matrix.
Say I have a matrix:
A = [ 0 3 0 2 1 0 6
3 0 0 4 8 0 5
0 0 0 0 0 0 0
2 4 0 0 2 0 1
1 8 0 2 0 0 7
0 0 0 0 0 0 0
6 5 0 1 7 0 0 ]
Here rows/columns 3 and 6 are empty, so my compression function will give the output:
A_dash = [ 0 3 2 1 6
3 0 4 8 5
2 4 0 2 1
1 8 2 0 7
6 5 1 7 0 ]
A_map = [ 1 2 4 5 7]
Where A_map is a vector mapping the indicies of the rows/columns of A_dash to A. This means that if A_map(3) = 4, then row/column 4 of A is the same as row/column 3 of A_dash - ie. a row/column of zeroes must be inserted between columns/rows 2 and 3 in A_dash
What is the easiest way people can suggest for me to recreate matrix A from A_dash, using the information in A_map?
Here is what I have got so far:
% orig_size is original number of columns/rows
c_count = size(A_dash,1);
A = zeros(c_count, orig_size); % c_count rows to avoid dimension mismatch
for ii = 1:c_count
A(:,A_map(ii)) == A_dash(:,ii);
end
This gives me the right result column-wise:
A = [ 0 3 0 2 1 0 6
3 0 0 4 8 0 5
2 4 0 0 2 0 1
1 8 0 2 0 0 7
6 5 0 1 7 0 0 ]
However, I'm not sure how i should go about inserting the rows, i suppose i could copy the first 1:i rows into one matrix, i:end rows to a second matrix and concatenate those with a zero row in between, but that feels like a bit of a
clunky solution, and probably not very efficient for large sized matrices..
Otherwise, is there a better way that people can suggest I store the map information? I was thinking instead of storing the mapping between column/row indices, that I just store the indices of the zero columns/rows and then insert columns/rows of zeros where appropriate. Would this be a better way?
You've got the indices of the valid rows/columns. Now all you've got to do is put them in a new matrix of zeros the same size as A:
B=zeros(size(A));
B(A_map,A_map)=A_dash
B =
0 3 0 2 1 0 6
3 0 0 4 8 0 5
0 0 0 0 0 0 0
2 4 0 0 2 0 1
1 8 0 2 0 0 7
0 0 0 0 0 0 0
6 5 0 1 7 0 0
Just to check...
>> A==B
ans =
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
A and B are equal everywhere, so we've reconstructed A.
Is it possible to find the non nan values of a vector but also allowing n number of nans? For example, if I have the following data:
X = [18 3 nan nan 8 10 11 nan 9 14 6 1 4 23 24]; %// input array
thres = 1; % this is the number of nans to allow
and I would like to only keep the longest sequence of values with non nans but allow 'n' number of nans to be kept in the data. So, say that I am willing to keep 1 nan I would have an output of
X_out = [8 10 11 nan 9 14 6 1 4 23 24]; %// output array
Thats is, the two nans at the beginning have been removed becuase they exceed the values in 'thres' above, but the third nan is on its own thus can be kept in the data. I would like to develop a method where thres can be defined as any value.
I can find the non nan values with
Y = ~isnan(X); %// convert to zeros and ones
Any ideas?
In order to find the longest sequence containing at most threshold times NaN we must find the start and the end of said sequence(s).
To generate all possible start points, we can use hankel:
H = hankel(X)
H =
18 3 NaN NaN 8 10 11 NaN 9 14 6 1 4 23 24
3 NaN NaN 8 10 11 NaN 9 14 6 1 4 23 24 0
NaN NaN 8 10 11 NaN 9 14 6 1 4 23 24 0 0
NaN 8 10 11 NaN 9 14 6 1 4 23 24 0 0 0
8 10 11 NaN 9 14 6 1 4 23 24 0 0 0 0
10 11 NaN 9 14 6 1 4 23 24 0 0 0 0 0
11 NaN 9 14 6 1 4 23 24 0 0 0 0 0 0
NaN 9 14 6 1 4 23 24 0 0 0 0 0 0 0
9 14 6 1 4 23 24 0 0 0 0 0 0 0 0
14 6 1 4 23 24 0 0 0 0 0 0 0 0 0
6 1 4 23 24 0 0 0 0 0 0 0 0 0 0
1 4 23 24 0 0 0 0 0 0 0 0 0 0 0
4 23 24 0 0 0 0 0 0 0 0 0 0 0 0
23 24 0 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Now we need to find the last valid element in each row.
To do so, we can use cumsum:
C = cumsum(isnan(H),2)
C =
0 0 1 2 2 2 2 3 3 3 3 3 3 3 3
0 1 2 2 2 2 3 3 3 3 3 3 3 3 3
1 2 2 2 2 3 3 3 3 3 3 3 3 3 3
1 1 1 1 2 2 2 2 2 2 2 2 2 2 2
0 0 0 1 1 1 1 1 1 1 1 1 1 1 1
0 0 1 1 1 1 1 1 1 1 1 1 1 1 1
0 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
The end point for each row is the one, where the corresponding element in C is at most threshold:
threshold = 1;
T = C<=threshold
T =
1 1 1 0 0 0 0 0 0 0 0 0 0 0 0
1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
The last valid element is found using:
[~,idx]=sort(T,2);
lastone=idx(:,end)
lastone =
3 2 1 4 15 15 15 15 15 15 15 15 15 15 15
We must make sure that the actual length of each row is respected:
lengths = length(X):-1:1;
real_length = min(lastone,lengths);
[max_length,max_idx] = max(real_length)
max_length =
11
max_idx =
5
In case there are more sequences of equal maximum length, we just take the first and display it:
selected_max_idx = max_idx(1);
H(selected_max_idx, 1:max_length)
ans =
8 10 11 NaN 9 14 6 1 4 23 24
full script
X = [18 3 nan nan 8 10 11 nan 9 14 6 1 4 23 24];
H = hankel(X);
C = cumsum(isnan(H),2);
threshold = 1;
T = C<=threshold;
[~,idx]=sort(T,2);
lastone=idx(:,end)';
lengths = length(X):-1:1;
real_length = min(lastone,lengths);
[max_length,max_idx] = max(real_length);
selected_max_idx = max_idx(1);
H(selected_max_idx, 1:max_length)
Approach 1: convolution
One possible approach is to convolve Y = double(~isnan(X)); with a window of n ones, where n is decreased by until an acceptable subsequence is found. "Acceptable" means that the subsequence contains at least n-thres ones, that is, the convolution gives at least n-thres.
Y = double(~isnan(X));
for n = numel(Y):-1:1 %// try all possible sequence lengths
w = find(conv(Y,ones(1,n),'valid')>=n-thres); %// is there any acceptable subsequence?
if ~isempty(w)
break
end
end
result = X(w:w+n-1);
Aproach 2: cumulative sum
Convolving Y with a window of n ones (as in approach 1) is equivalent to computing a cumulative sum of Y and then taking differences with n spacing. This is more efficient in terms of number of operations.
Y = double(~isnan(X));
Z = cumsum(Y);
for n = numel(Y):-1:1
w = find([Z(n) Z(n+1:end)-Z(1:end-n)]>=n-thres);
if ~isempty(w)
break
end
end
result = X(w:w+n-1);
Approach 3: 2D convolution
This essentially computes all iterations of the loop in approach 1 at once.
Y = double(~isnan(X));
z = conv2(Y, tril(ones(numel(Y))));
[nn, ww] = find(bsxfun(#ge, z, (1:numel(Y)).'-thres)); %'
[n, ind] = max(nn);
w = ww(ind)-n+1;
result = X(w:w+n-1);
Let's try my favorite tool: RLE. Matlab doesn't have a direct function, so use my "seqle" posted to exchange central. Seqle's default is to return run length encoding. So:
>> foo = [ nan 1 2 3 nan nan 4 5 6 nan 5 5 5 ];
>> seqle(isnan(foo))
ans =
run: [1 3 2 3 1 3]
val: [1 0 1 0 1 0]
The "run" indicates the length of the current run; "val" indicates the value. In this case, val==1 indicates the value is nan and val==0 indicates numeric values. You can see it'll be relatively easy to extract the longest sequence of "run" values meeting the condition val==0 | run < 2 to get no more than one nan in a row. Then just grab the cumulative indices of that run and that's the subset of foo you want.
EDIT:
sadly, what's trivial to find by eye may not be so easy to extract via code. I suspect there's a much faster way to use the indices identified by longrun to get the desired subsequence.
>> foo = [ nan 1 2 3 nan nan 4 5 6 nan nan 5 5 nan 5 nan 4 7 4 nan ];
>> sfoo= seqle(isnan(foo))
sfoo =
run: [1 3 2 3 2 2 1 1 1 3 1]
val: [1 0 1 0 1 0 1 0 1 0 1]
>> longrun = sfoo.run<2 |sfoo.val==0
longlong =
run: [2 1 1 1 6]
val: [1 0 1 0 1]
% longrun identifies which indices might be part of a run
% longlong identifies the longest sequence of valid run
>> longlong = seqle(longrun)
>> lfoo = find(sfoo.run<2 |sfoo.val==0);
>> sbar = seqle(lfoo,1);
>> maxind=find(sbar.run==max(sbar.run),1,'first');
>> getlfoo = lfoo( sum(sbar.run(1:(maxind-1)))+1 );
% first value in longrun , which is part of max run
% getbar finds end of run indices
>> getbar = getlfoo:(getlfoo+sbar.run(maxind)-1);
>> getsbar = sfoo.run(getbar);
% retrieve indices of input vector
>> startit = sum(sfoo.run(1:(getbar(1)-1))) +1;
>> endit = startit+ ((sum(sfoo.run(getbar(1):getbar(end ) ) ) ) )-1;
>> therun = foo( startit:endit )
therun =
5 5 NaN 5 NaN 4 7 4 NaN
Hmmm, who doesn't like challenges, my solution is not as good as m.s.'s, but it is an alternative.
X = [18 3 nan nan 8 10 11 nan 9 14 6 1 4 23 24]; %// input array
thresh =1;
X(isnan(X))= 0 ;
for i = 1:thresh
Y(i,:) = circshift(X',-i); %//circular shift
end
For some reason, the Matlab invert " ' " makes the formatting looks weird.
D = X + sum(Y,1);
Discard = find(D==0)+thresh; %//give you the index of the part that needs to be discarded
chunk = find(X==0); %//Segment the Vector into segments delimited by NaNs
seriesOfZero = circshift(chunk',-1)' - chunk;
bigchunk =[1 chunk( find(seriesOfZero ~= 1)) size(X,2)]; %//Convert series of NaNs into 1 chunk
[values,DiscardChunk] = intersect(bigchunk,Discard);
DiscardChunk = sort(DiscardChunk,'descend')
for t = 1:size(DiscardChunk,2)
X(bigchunk(DiscardChunk(t)-1):bigchunk(DiscardChunk(t))) = []; %//Discard the data
end
X(X == 0) = NaN
%//End of Code
8 10 11 NaN 9 14 6 1 4 23 24
When:
X = [18 3 nan nan nan 8 10 11 nan nan 9 14 6 1 nan nan nan 4 23 24]; %// input array
thresh =2;
8 10 11 NaN 4 23 24
I have one vector of starting points (start) and another vector for end points (end):
start=[1 0 0 0 1 0 0 0 1 0 0 0]
end= [0 0 1 0 0 0 1 0 0 0 1 0]
I want a third vector A all the numbers between each start and endpoint.
A = [1 2 3 4 5 6 7 8 9 1 2 3]
so the result for this example could look like this:
A_result= [1 2 3 5 6 7 9 1 2]
any ideas?
Without loop:
s=[1 0 0 0 1 0 0 0 1 0 0 0];
e=[0 0 1 0 0 0 1 0 0 0 1 0];
mask = cumsum(s)-cumsum([shift(e,1)])
# Will be [1 1 1 0 1 1 1 0 1 1 1 0]
A = [1 2 3 4 5 6 7 8 9 1 2 3];
A(find(mask))
# will be [1 2 3 5 6 7 9 1 2]
Or as The Minion pointed out, simply:
A(mask==1)
# will be [1 2 3 5 6 7 9 1 2]
Check the solution from seb it is way faster for large arrays.
This is a solution using a for-loop
d_start=[1 0 0 0 1 0 0 0 1 0 0 0];
d_end= [0 0 1 0 0 0 1 0 0 0 1 0];
A = [1 2 3 4 5 6 7 8 9 1 2 3];
starting_idx = find(d_start);
ending_idx = find(d_end);
A_result = [];
for l=1:numel(starting_idx)
A_result=horzcat(A_result, A(starting_idx(l):ending_idx(l)));
end
What I am doing is: 1. I get the index of the starting and end points (find). Then i predefine my result. Now i just have to loop over the number of entries in the starting points and add those values from that position to next ending to my result.
Edit: Timing test:
I used my solution as well as the solution from seb (modified to my variable names):
mask = cumsum(d_start)-cumsum([0 d_end(1:end-1)]);
B_res =A(find(mask));
as well as my comment to his solution:
mask = cumsum(d_start)-cumsum([0 d_end(1:end-1)]);
B_res =A(mask==1);
These were the timing results for A:[12888x1]double
t_forloop : 33 sec
t_seb_find: 0.31 sec
t_seb_logical: 0.1964 sec
I have an upper triangular matrix (without the diagonal) given by:
M = [0 3 2 2 0 0; 0 0 8 6 3 2; 0 0 0 3 2 1; 0 0 0 0 2 1; 0 0 0 0 0 0]
The resulting matrix should look like this:
R = [0 0 0 0 0 0; 0 2 0 0 0 0; 2 3 1 0 0 0; 2 6 2 1 0 0; 3 8 3 2 0 0]
Since I couldn't find a simple explanation which describes my goal I tried to visualize it with an image:
I already tried lots of different combinations of rot90, transpose, flipud etc., but I could't find the right transformation that gives me the matrix R
EDIT:
The rows of the matrix M are not always sorted as in the example above. For another matrix M_2:
M_2 = [0 2 3 1 0 0; 0 0 3 6 3 9; 0 0 0 1 2 4; 0 0 0 0 2 6; 0 0 0 0 0 0]
the resulting matrix R_2 need to be the following:
R_2 = [0 0 0 0 0 0; 0 9 0 0 0 0; 1 3 4 0 0 0; 3 6 2 6 0 0; 2 3 1 2 0 0]
Again the visualization below:
EDIT:
Inspired by the tip from #Dan's comment, it can be further simplified to
R = reshape(rot90(M), size(M));
Original Answer:
This should be a simple way to do this
F = rot90(M);
R = F(reshape(1:numel(M), size(M)))
which returns
R =
0 0 0 0 0 0
0 2 0 0 0 0
2 3 1 0 0 0
2 6 2 1 0 0
3 8 3 2 0 0
The idea is that when you rotate the matrix you get
>> F = rot90(M)
F =
0 2 1 1 0
0 3 2 2 0
2 6 3 0 0
2 8 0 0 0
3 0 0 0 0
0 0 0 0 0
which is a 6 by 5 matrix. If you consider the linear indexing over F the corresponding indices are
>> reshape(1:30, size(F))
1 7 13 19 25
2 8 14 20 26
3 9 15 21 27
4 10 16 22 28
5 11 17 23 29
6 12 18 24 30
where elements 6, 11, 12, 16, 17, 18 , and ... are zero now if you reshape this to a 5 by 6 matrix you get
>> reshape(1:30, size(M))
1 6 11 16 21 26
2 7 12 17 22 27
3 8 13 18 23 28
4 9 14 19 24 29
5 10 15 20 25 30
Now those elements corresponding to zero values are on top, exactly what we wanted. So by passing this indexing array to F we get the desired R.
Without relying on order (just rotating the colored strips and pushing them to the bottom).
First solution: note that it doesn't work if there are zeros between the "data" values (for example, if M(1,3) is 0 in the example given). If there may be zeros please see second solution below:
[nRows nCols]= size(M);
R = [flipud(M(:,2:nCols).') zeros(nRows,1)];
[~, rowSubIndex] = sort(~~R);
index = sub2ind([nRows nCols],rowSubIndex,repmat(1:nCols,nRows,1));
R = R(index);
Second solution: works even if there are zeros within the data:
[nRows nCols]= size(M);
S = [flipud(M(:,2:nCols).') zeros(nRows,1)];
mask = 1 + fliplr(tril(NaN*ones(nRows, nCols)));
S = S .* mask;
[~, rowSubIndex] = sort(~isnan(S));
index = sub2ind([nRows nCols],rowSubIndex,repmat(1:nCols,nRows,1));
R = S(index);
R(isnan(R)) = 0;
Alternate option, using loops:
[nRows nCols]= size(M);
R = zeros(nRows,nCols);
for n = 1:nRows
R((n+1):nCols,n)=fliplr(M(n,(n+1):nCols));
end