This post follows up another post: find common value of one matrix in another matrix
As I explained there, I have one matrix MyMatrix 2549x13double
Few example lines from MyMatrix:
-7.80 -4.41 -0.08 2.51 6.31 6.95 4.97 2.91 0.66 -0.92 0.31 1.24 -0.07
4.58 5.87 6.18 6.23 5.20 4.86 5.02 5.33 3.69 1.36 -0.54 0.28 -1.20
-6.22 -3.77 1.18 2.85 -3.55 0.52 3.24 -7.77 -8.43 -9.81 -6.05 -5.88 -7.77
-2.21 -3.21 -4.44 -3.58 -0.89 3.40 6.56 7.20 4.30 -0.77 -5.09 -3.18 0.43
I have identified the maximum value for each row of matrix MyMatrix as following:
[M Ind] = max(MyMatrix, [], 2);
Example lines I obtain in M:
6.95
6.23
3.24
7.20
Now, I would like to select in MyMatrix the 2 values before and after the maximum value as found in M, as I will need to calculate the average of these 5 values. So, in the example, I would like to select:
2.51 6.31 6.95 4.97 2.91
5.87 6.18 6.23 5.20 4.86
-3.55 0.52 3.24 -7.77 -8.43
3.40 6.56 7.20 4.30 -0.77
and to create a new column in MyMatrix with the mean of these 5 values.
Following the code by #Dan, taken from the previous post:
colInd = bsxfun(#plus,PeakInd, -2:2);
MyMatrixT = MyMatrix.';
rowIndT = colInd.';
linIndT = bsxfun(#plus,rowIndT,0:size(MyMatrixT,1):size(MyMatrixT,1)*(size(MyMatrixT,2)-1));
resultT = MyMatrixT(linIndT);
result = resultT.';
mean(result,2)
MyMatrix = [MyMatrix, mean(result,2)];
Here is the new part of the post, regarding the issue when the maximum value is near the edges.
When the maximum is the first or last column of MyMatrix, I would like to have NaN.
Instead, when the maximum is in the second column, I would like to calculate the mean considering one column preceding the maximum, the maximum value, and two columns following the maximum.
While, when the maximum is in the second last column, I would like to consider the two columns preceding the maximum, the maximum value, and only one column following the maximum.
I would be extremely grateful if you could help me. Many thanks!
Instead of creating a 2D array with NaNs plus nanmean, you could use min/max to get the right indexes:
pad = 2;
[~, Ind] = max(MyMatrix, [], 2);
minCol = max(1, Ind-pad);
maxCol = min(size(MyMatrix, 2), Ind+pad);
result = arrayfun(#(row, min_, max_) mean(MyMatrix(row, min_:max_)),...
(1:size(MyMatrix, 1)).', minCol, maxCol);
If you have the Image Processing Toolbox, you can also use padarray, e.g.
B = padarray(magic(5),[0 2],NaN);
B =
NaN NaN 17 24 1 8 15 NaN NaN
NaN NaN 23 5 7 14 16 NaN NaN
NaN NaN 4 6 13 20 22 NaN NaN
NaN NaN 10 12 19 21 3 NaN NaN
NaN NaN 11 18 25 2 9 NaN NaN
(...if you don't have padarray, just manually add 2 NaN columns on either side) then using some bsxfun + sub2ind we get the desired result:
pad_sz = 2;
B = padarray(magic(5),[0 pad_sz],NaN);
[~,I] = nanmax(B,[],2); % by using nanmax we "explicitly" say we ignore NaNs.
colInd = bsxfun(#plus,-pad_sz:pad_sz,I);
linearInd = sub2ind(size(B), repmat((1:5).',[1,size(colInd,2)]), colInd);
picks = B(linearInd);
res = nanmean(picks,2);
% or combine the last 3 lines into:
% res = nanmean(B(sub2ind(size(B), repmat((1:5).',[1,size(colInd,2)]), colInd)),2);
res = res + 0./~(I == pad_sz+1 | I == size(B,2)-pad_sz); %add NaN where needed.
Related
I have a very large (2019x1678 double) DEM (digital elevation model) file put as a matrix in MATLAB. The edges of it contain NaN values. In order to account for edge effects in my code, I have to put a 1 cell buffer (same value as adjacent cell) around my DEM. Where NaNs are present, I need to find the edge of the NaN values in order to build that buffer. I have tried doing this two ways:
In the first I get the row and column coordinates all non-NaN DEM values, and find the first and last row numbers for each column to get the north and south boundaries, then find the first and last column numbers for each row to get the east and west boundaries. I use these in the sub2ind() to create my buffer.
[r, c] = find(~isnan(Zb_ext)); %Zb is my DEM matrix
idx = accumarray(c, r, [], #(x) {[min(x) max(x)]});
idx = vertcat(idx{:});
NorthBoundary_row = transpose(idx(:,1)); % the values to fill my buffer with
NorthBoundary_row_ext = transpose(idx(:,1) - 1); % My buffer cells
columnmax = length(NorthBoundary_row);
column1 = min(c);
Boundary_Colu = linspace(column1,column1+columnmax-1,columnmax);
SouthBoundary_row = (transpose(idx(:,2))); % Repeat for south Boundary
SouthBoundary_row_ext = transpose(idx(:,2) + 1);
SouthB_Ind = sub2ind(size(Zb_ext),SouthBoundary_row,Boundary_Colu);
SouthB_Ind_ext = sub2ind(size(Zb_ext),SouthBoundary_row_ext, Boundary_Colu);
NorthB_Ind = sub2ind(size(Zb_ext),NorthBoundary_row, Boundary_Colu);
NorthB_Ind_ext = sub2ind(size(Zb_ext),NorthBoundary_row_ext, Boundary_Colu);
Zb_ext(NorthB_Ind_ext) = Zb_ext(NorthB_Ind);
Zb_ext(SouthB_Ind_ext) = Zb_ext(SouthB_Ind);
% Repeat above for East and West Boundary by reversing the roles of row and
% column
[r, c] = find(~isnan(Zb_ext));
idx = accumarray(r, c, [], #(x) {[min(x) max(x)]});
idx = vertcat(idx{:});
EastBoundary_colu = transpose(idx(:,1)); % Repeat for east Boundary
EastBoundary_colu_ext = transpose(idx(:,1) - 1);
row1 = min(r);
rowmax = length(EastBoundary_colu);
Boundary_row = linspace(row1,row1+rowmax-1,rowmax);
WestBoundary_colu = transpose(idx(:,2)); % Repeat for west Boundary
WestBoundary_colu_ext = transpose(idx(:,2) + 1);
EastB_Ind = sub2ind(size(Zb_ext),Boundary_row, EastBoundary_colu);
EastB_Ind_ext = sub2ind(size(Zb_ext),Boundary_row, EastBoundary_colu_ext);
WestB_Ind = sub2ind(size(Zb_ext),Boundary_row, WestBoundary_colu);
WestB_Ind_ext = sub2ind(size(Zb_ext),Boundary_row, WestBoundary_colu_ext);
Zb_ext(NorthB_Ind_ext) = Zb_ext(NorthB_Ind);
Zb_ext(SouthB_Ind_ext) = Zb_ext(SouthB_Ind);
Zb_ext(EastB_Ind_ext) = Zb_ext(EastB_Ind);
Zb_ext(WestB_Ind_ext) = Zb_ext(WestB_Ind);
This works well on my small development matrix, but fails on my full sized DEM. I do not understand the behavior of my code, but looking at the data there are gaps in my boundary. I wonder if I need to better control the order of max/min row/column values, though in my test on a smaller dataset, all seemed in order....
The second method I got from a similar question to this and basically uses a dilation method. However, when I transition to my full dataset, it takes hours to calculate ZbDilated. Although my first method does not work, it at least calculates within seconds.
[m, n] = size(Zb); %
Zb_ext = nan(size(Zb)+2);
Zb_ext(2:end-1, 2:end-1) = Zb; % pad Zb with zeroes on each side
ZbNANs = ~isnan(Zb_ext);
ZbDilated = zeros(m + 2, n + 2); % this will hold the dilated shape.
for i = 1:(m+2)
if i == 1 %handling boundary situations during dilation
i_f = i;
i_l = i+1;
elseif i == m+2
i_f = i-1;
i_l = i;
else
i_f = i-1;
i_l = i+1;
end
for j = 1:(n+2)
mask = zeros(size(ZbNANs));
if j == 1 %handling boundary situations again
j_f = j;
j_l = j+1;
elseif j == n+2
j_f = j-1;
j_l = j;
else
j_f = j-1;
j_l = j+1;
end
mask(i_f:i_l, j_f:j_l) = 1; % this places a 3x3 square of 1's around (i, j)
ZbDilated(i, j) = max(ZbNANs(logical(mask)));
end
end
Zb_ext(logical(ZbDilated)) = fillmissing(Zb_ext(logical(ZbDilated)),'nearest');
Does anyone have any ideas on making either of these usable?
Here is what I start out with:
NaN NaN 2 5 39 55 44 8 NaN NaN
NaN NaN NaN 7 33 48 31 66 17 NaN
NaN NaN NaN 28 NaN 89 NaN NaN NaN NaN
Here is the matrix buffered on the limits with NaNs:
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN 2 5 39 55 44 8 NaN NaN NaN
NaN NaN NaN NaN 7 33 48 31 66 17 NaN NaN
NaN NaN NaN NaN 28 NaN 89 NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Here is what I want to get after using fillmissing (though I have noticed some irregularities with how buffer values are filled...):
NaN NaN 2 2 5 39 55 44 8 17 NaN NaN
NaN NaN 2 2 5 39 55 44 8 17 17 NaN
NaN NaN 2 2 7 33 48 31 66 17 17 NaN
NaN NaN NaN 2 28 33 89 31 66 17 17 NaN
NaN NaN NaN 5 28 55 89 8 NaN NaN NaN NaN
To try and clear up any confusion about what I am doing, here is the logical I get from dilation I use for fillmissing
0 0 1 1 1 1 1 1 1 1 0 0
0 0 1 1 1 1 1 1 1 1 1 0
0 0 1 1 1 1 1 1 1 1 1 0
0 0 0 1 1 1 1 1 1 1 1 0
0 0 0 1 1 1 1 1 0 0 0 0
A faster way to apply a 3x3 dilation would be as follows. This does involve some large intermediate matrices, which make it less efficient than, say applying imdilate.
[m, n] = size(Zb); %
Zb_ext = nan(size(Zb)+2);
Zb_ext(2:end-1, 2:end-1) = Zb; % pad A with zeroes on each side
ZbNANs = ~isnan(Zb_ext);
ZbDilated = ZbNANs; % this will hold the dilated shape.
% up and down neighbors
ZbDilated(2:end, :) = max(ZbDilated(2:end, :), ZbNANs(1:end-1, :));
ZbDilated(1:end-1, :) = max(ZbDilated(1:end-1, :), ZbNANs(2:end, :));
% left and right neighbors
ZbDilated(:, 2:end) = max(ZbDilated(:, 2:end), ZbNANs(:, 1:end-1));
ZbDilated(:, 1:end-1) = max(ZbDilated(:, 1:end-1), ZbNANs(:, 2:end));
% and 4 diagonal neighbors
ZbDilated(2:end, 2:end) = max(ZbDilated(2:end, 2:end), ZbNANs(1:end-1, 1:end-1));
ZbDilated(1:end-1, 2:end) = max(ZbDilated(1:end-1, 2:end), ZbNANs(2:end, 1:end-1));
ZbDilated(2:end, 1:end-1) = max(ZbDilated(2:end, 1:end-1), ZbNANs(1:end-1, 2:end));
ZbDilated(1:end-1, 1:end-1) = max(ZbDilated(1:end-1, 1:end-1), ZbNANs(2:end, 2:end));
This is a tedious way to write it, I'm sure there's a loop that can be written that is shorter, but this I think makes the intention clearer.
[Edit: Because we're dealing with a logical array here, instead of max(A,B) we could also do A | B. I'm not sure if there would be any difference in time.]
What #beaker said in a comment was to not use
mask = zeros(size(ZbNANs));
mask(i_f:i_l, j_f:j_l) = 1; % this places a 3x3 square of 1's around (i, j)
ZbDilated(i, j) = max(ZbNANs(logical(mask)));
but rather do
ZbDilated(i, j) = max(ZbNANs(i_f:i_l, j_f:j_l), [], 'all');
[Edit: Because we're dealing with a logical array here, instead of max(A,[],'all') we could also do any(A,'all'), which should be faster. See #beaker's other comment.]
I have the 4x2 matrix A:
A = [2 NaN 5 8; 14 NaN 23 NaN]';
I want to replace the non-NaN values with their associated indices within each column in A. The output looks like this:
out = [1 NaN 3 4; 1 NaN 3 NaN]';
I know how to do it for each column manually, but I would like an automatic solution, as I have much larger matrices to handle. Anyone has any idea?
out = bsxfun(#times, A-A+1, (1:size(A,1)).');
How it works:
A-A+1 replaces actual numbers in A by 1, and keeps NaN as NaN
(1:size(A,1)).' is a column vector of row indices
bsxfun(#times, ...) multiplies both of the above with singleton expansion.
As pointed out by #thewaywewalk, in Matlab R2016 onwards bsxfun(#times...) can be replaced by .*, as singleton expansion is enabled by default:
out = (A-A+1) .* (1:size(A,1)).';
An alternative suggested by #Dev-Il is
out = bsxfun(#plus, A*0, (1:size(A,1)).');
This works because multiplying by 0 replaces actual numbers by 0, and keeps NaN as is.
Applying ind2sub to a mask created with isnan will do.
mask = find(~isnan(A));
[rows,~] = ind2sub(size(A),mask)
A(mask) = rows;
Note that the second output of ind2sub needs to be requested (but neglected with ~) as well [rows,~] to indicate you want the output for a 2D-matrix.
A =
1 1
NaN NaN
3 3
4 NaN
A.' =
1 NaN 3 4
1 NaN 3 NaN
Also be careful the with the two different transpose operators ' and .'.
Alternative
[n,m] = size(A);
B = ndgrid(1:n,1:m);
B(isnan(A)) = NaN;
or even (with a little inspiration by Luis Mendo)
[n,m] = size(A);
B = A-A + ndgrid(1:n,1:m)
or in one line
B = A-A + ndgrid(1:size(A,1),1:size(A,2))
This can be done using repmat and isnan as follows:
A = [ 2 NaN 5 8;
14 NaN 23 NaN];
out=repmat([1:size(A,2)],size(A,1),1); % out contains indexes of all the values
out(isnan(A))= NaN % Replacing the indexes where NaN exists with NaN
Output:
1 NaN 3 4
1 NaN 3 NaN
You can take the transpose if you want.
I'm adding another answer for a couple of reasons:
Because overkill (*ahem* kron *ahem*) is fun.
To demonstrate that A*0 does the same as A-A.
A = [2 NaN 5 8; 14 NaN 23 NaN].';
out = A*0 + kron((1:size(A,1)).', ones(1,size(A,2)))
out =
1 1
NaN NaN
3 3
4 NaN
There is likely a quick little trick for this problem, but I cannot find it. I would like code to produce the matrix in the following image:
Here's a way using logical indexing:
n = 4;
A = nan(n);
A(~eye(n)) = 1:n^2-n; %// Only replace values *not* on diagonal
A = A.'
A =
NaN 1 2 3
4 NaN 5 6
7 8 NaN 9
10 11 12 NaN
Here's one way using triu and tril:
n = 4;
A = reshape(1:n*(n-1),n-1,n).';
z = zeros(n,1);
A = [tril(A,-1) z]+[z triu(A)]+diag(NaN(n,1));
which, in this case for a 4-by-4 matrix, returns
A =
NaN 1 2 3
4 NaN 5 6
7 8 NaN 9
10 11 12 NaN
Here's another way just using reshape:
n = 4;
A = [reshape(1:n*(n-1),n,n-1);NaN(1,n-1)];
A = reshape([NaN;A(:)],n,n).'
Here's another way:
n = 4; %// matrix size
x = 1-eye(n);
x(:) = cumsum(x(:));
x = x.' + diag(NaN(1,n));
You can start from a nan matrix, find the linear indices of the diagonal, then fill up the rest of the elements with an incrementing range:
n=4;
A=nan(n);
inds=setdiff(1:n^2,sub2ind([n,n],1:n,1:n));
A(inds)=1:numel(inds);
A=A.'; %' transpose to get the matrix we need
The transpose in the end is necessary, as linear indexing goes column-first, but your specifics need a row-first assignment of matrix elements.
Result:
>> A
A =
NaN 1 2 3
4 NaN 5 6
7 8 NaN 9
10 11 12 NaN
I am new to MATLAB and I am trying to combine rows with similar values (I have thousands of rows), for example
1 NaN
1 NaN
1 NaN
2 9
2 26.5
2 21.5
2 18
2 24.5
2 12
2 22.5
3 NaN
3 NaN
3 NaN
3 NaN
4 18.5
4 22
4 35.5
...
...
...
to
1 NaN NaN NaN
2 9 26.5 21.5 18 24.5 12 22.5
3 NaN NaN NaN NaN
4 18.5 22 35.5
can any one please help me with this?
This can't be done with normal arrays. Each row has to have same number of columns, but your desired output isn't so. You can work with cell arrays if you wish.
If cell arrays are an option, the best way to tackle this IMHO would be to use an accumarray/sort/cellfun pipeline. First use accumarray to group all of the values together that belong to the same ID, so the first column in your case. Each group would thus be a cell array. However, a consequence with accumarray is that the values that come in per group are unordered. Therefore, what you'd have to group instead are the locations of the values instead. You'd sort these locations and what is output is a cell array where each cell are a list of indices you'd access in the original data.
You'd then call cellfun as the last step to use the indices access the actual data itself.
Something like this comes to mind, assuming your data is stored in X and it's a two-column array.
ind = (1 : size(X,1)).'; %'
out_ind = accumarray(X(:,1), ind, [], #(x) {sort(x)});
out = cellfun(#(x) X(x,2), out_ind, 'uni', 0);
We thus get:
>> celldisp(out)
out{1} =
NaN
NaN
NaN
out{2} =
9.0000
26.5000
21.5000
18.0000
24.5000
12.0000
22.5000
out{3} =
NaN
NaN
NaN
NaN
out{4} =
18.5000
22.0000
35.5000
I have a matrix that looks like the following.
I want to take the column 3 values and put them in another matrix, according to the following rule.
The value in the Column 5 is the row index for the new matrix, and Column 6 is the column index. Therefore 20 (taken from 29,3) should be in Row 1 Column 57 of the new matrix, 30 (from 30,3) should in Row 1 column 4 of the new matrix, and so on.
If the value in column 3 is NaN then I want NaN to be copied over to the new matrix.
Example:
% matrix of values and row/column subscripts
A = [
20 1 57
30 1 4
25 1 16
nan 1 26
nan 1 28
25 1 36
nan 1 53
50 1 56
nan 2 1
nan 2 2
nan 2 3
80 2 5
];
% fill new matrix
B = zeros(5,60);
idx = sub2ind(size(B), A(:,2), A(:,3));
B(idx) = A(:,1);
There are a couple other ways to do this, but I think the above code is easy to understand. It is using linear indexing.
Assuming you don't have duplicate subscripts, you could also use:
B = full(sparse(A(:,2), A(:,3), A(:,1), m, n));
(where m and n are the output matrix size)
Another one:
B = accumarray(A(:,[2 3]), A(:,1), [m,n]);
I am not sure if I understood your question clearly but this might help:
(Assuming your main matrix is A)
nRows = max(A(:,5));
nColumns = max(A(:,6));
FinalMatrix = zeros(nRows,nColumns);
for i=1:size(A,1)
FinalMatrix(A(i,5),A(i,6))=A(i,3);
end
Note that above code sets the rest of the elements equal to zero.