I have a matrix with values in the center and NaNs on the border (imagine a matrix representing a watershed which is never square). I need to pad it with one cell to do some component stress calculations. I am trying to avoid using outside libraries from the core Matlab functionality however what i am trying to do is similar to padarray symmetric but for an irregular border:
padarray(Zb,[1 1],'symmetric','both');
For example:
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN 2 5 39 55 44 8 NaN NaN NaN
NaN NaN NaN NaN 7 33 48 31 66 17 NaN NaN
NaN NaN NaN NaN 28 NaN 89 NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Becomes:
NaN NaN 2 2 5 39 55 44 8 8 NaN NaN
NaN NaN 2 2 5 39 55 44 8 17 17 NaN
NaN NaN 2 2 7 33 48 31 66 17 17 NaN
NaN NaN NaN 28 28 33 89 31 66 17 17 NaN
NaN NaN NaN 28 28 28 89 89 NaN NaN NaN NaN
(Not sure how to handle convex corners with two adjacent values since I need to control edge effects).
This post follows on an earlier question today in which I was able to extract the locations of these padded cells (buffers) into a dilated logical. However using fillmissing with nearest did not create the effect I expected (what padarray does).
Zb_ext(logical(ZbDilated)) = fillmissing(Zb_ext(logical(ZbDilated)),'nearest');
I might be able to reverse what I did to find the padcells to find the adjacent values and use those to replace the pad cell NaNs. But I thought I would first see if there was a simpler solution?
You can use two 2D convolutions to achieve this, where conv2 is within the core MATLAB library so nothing external is needed, and it should be fast.
However, you noted this:
Not sure how to handle convex corners with two adjacent values since I need to control edge effects
I've taken the liberty of defining a "sensible" output for convex corners which is to take the average value, because from your example it seems undefined how these cases, and more complicated ones like cell (5,6), should be handled.
I've added detailed comments to the below code for explanation
% Example matrix
A = [
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN 2 5 39 55 44 8 NaN NaN NaN
NaN NaN NaN NaN 7 33 48 31 66 17 NaN NaN
NaN NaN NaN NaN 28 NaN 89 NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
];
% Track the "inner" indices, where values are defined
inner = (~isnan(A));
B = A; % Copy A so we don't change it
B(~inner) = 0; % Replace NaN with 0 so that convolutions work OK
% First dilate the inner region by one element, taking the average of
% neighbours which are up/down/left/right (no diagonals). This is required
% to avoid including interior points (which only touch diagonally) in the
% averaging. These can be considered the "cardinal neighbours"
kernel = [0 1 0 ; 1 0 1; 0 1 0]; % Cardinal directions in 3x3 stencil
s = conv2(B,kernel,'same'); % 2D convolution to get sum of neighbours
n = conv2(inner,kernel,'same'); % 2D convolution to get count of neighbours
s(inner) = 0; % Zero out the inner region
s = s./n; % Get the mean of neighbours
% Second, dilate the inner region but including the mean from all
% directions. This lets us handle convex corners in the image
s2 = conv2(B,ones(3),'same'); % Sum of neighbours (and self, doesn't matter)
n = conv2(inner,ones(3),'same'); % Count of neighbours (self=0 for dilated elems)
s2 = s2./n; % Get the mean of neighbours
% Finally piece together the 3 matrices:
out = s2; % Start with outmost dilation inc. corners
out(~isnan(s)) = s(~isnan(s)); % Override with inner dilation for cardinal neighbours
out(inner) = A(inner); % Override with original inner data
So for this example, the output would be the same as your example output, except for corners as mentioned:
NaN NaN 2 2 5 39 55 44 8 8 NaN NaN
NaN NaN 2 2 5 39 55 44 8 12.5 17 NaN
NaN NaN 2 4.5 7 33 48 31 66 17 17 NaN
NaN NaN NaN 28 28 50 89 60 66 17 17 NaN
NaN NaN NaN 28 28 58.5 89 89 NaN NaN NaN NaN
Related (and utilised): MATLAB/Octave: Calculate the sum of adjacent/neighboring elements in a matrix
Related
There are two Questions.
---1---
I can successfully assign a smaller matrix to a bigger zero-matrix without problem, like this:
a = zeros(5,5,2);
b = [1 2 3];
a = b;
So I try to load the RawData from file to a 3D-matrix:
G = ["/Users/ripfreeworld/Documents/MATLAB/RawData/G01.txt",
"/Users/ripfreeworld/Documents/MATLAB/RawData/G02.txt",
"/Users/ripfreeworld/Documents/MATLAB/RawData/G03.txt",
"/Users/ripfreeworld/Documents/MATLAB/RawData/G04.txt"];
rawdata = zeros(1500,200,4);
for i = 1 : 4
rawdata(:,:,i) = load(G(i),'ascii');
end
It got stuck just at the first iteration, without getting any data from the load-function.
Then I commented the line with zeros(). And the first iteration succeed. What's the difference with the former simple code?
---2---
The second problem is:
Unable to perform assignment because the size of the left side is
1440-by-152 and the size of the right side is 1440-by-151.
I found that there was an extra column filled with "0". I deleted this column manually this time, but is there any way to keep the 3d form (cube matrix?) , by automatically filling the columns/rows of the smaller matrix with "0" or throwing away the data in the extra columns/rows of the bigger matrix?
Thank you!
Thanks Adriaan, the first question was started from a wrong test by overwriting.
For the second question I found an easy solution which is similar
p=zeros(5,10)*1/0;
for i=1:5
for m=1:10
if isprime((i-1)*10+m-1)
p(i,m)=(i-1)*10+m-1;
end
end
end
>> p
p =
NaN NaN 2 3 NaN 5 NaN 7 NaN NaN
NaN 11 NaN 13 NaN NaN NaN 17 NaN 19
NaN NaN NaN 23 NaN NaN NaN NaN NaN 29
NaN 31 NaN NaN NaN NaN NaN 37 NaN NaN
NaN 41 NaN 43 NaN NaN NaN 47 NaN NaN
Knowing that:
There are a lot of discussion about plotting equal sized matrices in a cell array and it is quite easy to do without a loop.
For example, to plot the 2-by-2 matrices in mycell:
mycell = {[1 1; 2 1], [1 1; 3 1], [1 1; 4 1]};
We can use cellfun to add a row of NaN at the bottom of each matrix and then convert the cell to a matrix:
mycellnaned = cellfun(#(x) {[x;nan(1,2)]}, mycell);
mymat = cell2mat(mycellnaned');
mymat looks like:
1 1 1 1 1
2 1 3 1 4
NaN NaN NaN NaN NaN
Then we can plot it easily:
mymatx = mymat(:,1:2:end);
mymaty = mymat(:,2:2:end);
figure;
plot(mymatx, mymaty,'+-');
The problem:
The problem is now, how do I do something similar with a cell containing non-equal matrices? Such as:
mycell = {
[1:2; ones(1,2)]';
[1:4; ones(1,4)*2]';
[1:6; ones(1,6)*3]';
[1:8; ones(1,8)*4]';
[1:10; ones(1,10)*5]';
[1:12; ones(1,12)*6]';
};
mycell = repmat(mycell,1000,1);
I would not be able to convert them into one matrix like I did before. I could use a loop, as suggested in this answer, but it would be very inefficient if the cell contains thousands of matrices.
Therefore, I'm looking for a more efficient way of plotting non-equal sized matrices in a cell array.
Note that different colours should be used for different matrices in the figure.
Well, while I was writing the question, I figured it out...
I'd like to keep the question open since there might be better solutions.
For everyone else's reference, the solution is simple: add NaN to make the matrices equal sized:
% find out the maximum length of all matrices in the array
cellLengthMax = max(cellfun('length', mycell));
% fill the matrices so they are equal in size.
mycellfilled = cellfun(#(x) {[
x
nan(cellLengthMax-size(x,1), 2)
nan(1, 2)
]}, mycell);
Then convert to a matrix and plot:
mymat = cell2mat(mycellfilled');
mymatx = mymat(:,1:2:end);
mymaty = mymat(:,2:2:end);
figure;
plot(mymatx, mymaty,'+-');
mymat looks like:
1 1 1 2 1 3 1 4 1 5 1 6
2 1 2 2 2 3 2 4 2 5 2 6
NaN NaN 3 2 3 3 3 4 3 5 3 6
NaN NaN 4 2 4 3 4 4 4 5 4 6
NaN NaN NaN NaN 5 3 5 4 5 5 5 6
NaN NaN NaN NaN 6 3 6 4 6 5 6 6
NaN NaN NaN NaN NaN NaN 7 4 7 5 7 6
NaN NaN NaN NaN NaN NaN 8 4 8 5 8 6
NaN NaN NaN NaN NaN NaN NaN NaN 9 5 9 6
NaN NaN NaN NaN NaN NaN NaN NaN 10 5 10 6
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 11 6
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 12 6
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Update:
Time cost for plotting 6000 matrices:
using the solution proposed here: 1.183546 seconds.
using a loop: 3.450423 seconds.
Still not very satisfactory. I really wish to reduce the time to 0.1 seconds, because I'm trying to design an interactive UI, where the user can change a few parameters and the result get plotted instantly.
I don't want to reduce the resolution of the figure.
Update:
I did a profiler and it seems the 99% of the time is wasted on plot(mymatx, mymaty,'+-');. So the conclusion is, there is probably no other way to fasten this.
Essentially, I have a matrix of data with many "holes" represented by NaN, and I want to retrieve the indices of all NaN's that are clustered fewer than 4 times in a single column.
e.g. with the matrix:
A =
23 12 NaN 56 60 21 NaN
60 56 94 22 45 NaN NaN
23 55 19 83 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
84 99 43 32 89 12 NaN
76 92 73 47 22 12 10
23 55 12 93 61 94 20
NaN NaN NaN NaN NaN NaN NaN
41 16 83 39 82 37 43
14 78 92 40 81 29 60
it would return:
ans =
[4; 5; 6; 10; 16; 17; 18; 22; 25; 28; 29; 30; 34; 40; 41; 42; 46; 58; 70; 82]
So far, I have a vector with the indices of all the NaN values from
nan_list=find(isnan(A(:)))
but I don't know how to extract sequential numbers from that vector without using loops, which would be too expensive. I also tried something similar to the answer posted by b3 here, by switching all NaN's to a value that doesn't appear in the matrix, but that code was not as transferable for other data sets.
Thanks for any suggestions!
Code
N = 4; %// Fewer than clusters of N or N+ NaNs are to be detecteed
nan_pos = isnan(A) %// Find NaN positions as a binary array
conv_res = conv2(double(nan_pos),[0 ones(1,N)]')==N %//' Perform convolution
start_ind = find(conv_res(N+1:end,:)) %// Find positions where clusters of N or N+ NaNs start
nan_pos(unique(bsxfun(#plus,start_ind,[0:N-1])))=0 %// Get positions of all those clustered N or N+ NaNs and set them in NaN position array as zeros
out = find(nan_pos) %// Finally the desired output
Example
As an example, let's try this code on a slightly different input that would hopefully test out various aspects of the problem -
A = [
23 12 NaN 56 60 21 NaN
60 56 94 22 45 NaN NaN
23 55 19 83 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN
84 99 43 32 89 12 NaN
76 92 73 47 22 12 10
23 55 12 93 61 94 20
NaN NaN NaN NaN NaN NaN NaN
41 NaN NaN 39 82 37 43
14 78 NaN 40 81 NaN 60]
Now, let's assume that we are looking to find indices of cluster fewer than 3 NaNs. Thus editing N as 3 in the code, the output is -
out =
10 22 23 25 46 58 70 72 82
This makes sense when we look into the input.
This should work:
[rows, ~] = size(A);
maxNansPerCol = 4;
% find which columns have few enough NaNs
Anans = isnan(A);
nansInCols = sum(Anans);
qualifyingCols = nansInCols <= maxNansPerCol;
% zero the other columns
mask = repmat(qualifyingCols,rows,1);
B = Anans .* mask;
% get the NaN locations
indices = find(B(:));
(Apologies if something is slightly off--I don't have MATLAB on this computer to test it)
I have a data :
minval = NaN 7 8 9 9 9 10 10 10 10
NaN NaN 10 10 10 10 10 10 10 10
NaN NaN NaN 10 10 9 10 10 10 9
NaN NaN NaN NaN 9 9 10 9 10 10
NaN NaN NaN NaN NaN 9 10 10 10 10
NaN NaN NaN NaN NaN NaN 10 11 10 10
NaN NaN NaN NaN NaN NaN NaN 10 10 10
NaN NaN NaN NaN NaN NaN NaN NaN 10 10
NaN NaN NaN NaN NaN NaN NaN NaN NaN 10
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
and I do this following :
C=size(minval,2);
D1(1,2:end) = minval(1,2:C);
D2 = bsxfun(#plus,minval(2:C-1,3:C),D1(1,1:C-2)');
D2 = [zeros(1,size(D2,2)) ;D2];
D2(D2==0) = NaN;
D1(2,3:end) = nanmin(D2);
D3 = bsxfun(#plus,minval(3:C-1,4:C),D1(2,2:C-2)');
D3 = [zeros(2,size(D3,2)) ;D3];
D3(D3==0) = NaN;
D1(3,4:end)= nanmin(D3);
Then, I want to backtrack the path which D1(end,end)comes from.
Is there any help? Thank you.
In MATLAB you can index out parts of matrices directly. There's no need for loops here:
C=size(minval,2);
D1(2:C) = minval(1,2:C);
For these ones you are not doing what you hoped, I suspect:
for e=3:C
for b=2:e-1
D2(e)=min(minval(b,e)+D1(b-1));
end
end
In the inner loop, for each value of b (from 2 to e-1), you are overwriting the value of D2 at each step. Only the result for the last value of b will be recorded. There may well be a much simpler way of getting the result you want. min and other functions do not just work on two single values but on entire matrices - e.g. you can do:
min(minval)
ans =
NaN 7 8 9 9 9 10 9 10 9
This is easy in two dimensions, for example:
>> A = NaN(5,4)
>> A(2:4,2:3) = [1 2; 3 4; 5 6]
>> A(2,2) = NaN
>> A(4,3) = NaN
A =
NaN NaN NaN NaN
NaN NaN 2 NaN
NaN 3 4 NaN
NaN 5 NaN NaN
NaN NaN NaN NaN
>> A(~all(isnan(A),2),~all(isnan(A),1))
ans =
NaN 2
3 4
5 NaN
Note that NaN values in rows and columns that are not all NaN are retained.
How to expand this to multiple dimensions? For example if A has three dimensions:
>> A = NaN(5,4,3)
>> A(2:4,2:3,2) = [1 2; 3 4; 5 6]
>> A(2,2,2) = NaN
>> A(4,3,2) = NaN
A(:,:,1) =
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
A(:,:,2) =
NaN NaN NaN NaN
NaN NaN 2 NaN
NaN 3 4 NaN
NaN 5 NaN NaN
NaN NaN NaN NaN
A(:,:,3) =
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
NaN NaN NaN NaN
How do I then get
ans =
NaN 2
3 4
5 NaN
I'd like to do this in four dimensions, and with much larger matrixes than the example matrix A here.
My solution to the problem based on the input A as posted by OP:
>> [i,j,k] = ind2sub(size(A),find(~isnan(A)));
>> l = min([i j k]);
>> u = max([i j k]);
>> B=A(l(1):u(1),l(2):u(2),l(3):u(3))
B =
NaN 2
3 4
5 NaN
>> size(B)
ans =
3 2
Since you stated that you want to do this on much larger matrices I'm not sure about the performance of #ronalchn's solution - that is all the all-calls. But I have no idea to what extend that matters - maybe someone can comment...
Try this:
2 dimensions
A(~all(isnan(A),2),~all(isnan(A),1))
3 dimensions
A(~all(all(isnan(A),2),3),...
~all(all(isnan(A),1),3),...
~all(all(isnan(A),1),2))
4 dimensions
A(~all(all(all(isnan(A),2),3),4),...
~all(all(all(isnan(A),1),3),4),...
~all(all(all(isnan(A),1),2),4),...
~all(all(all(isnan(A),1),2),3))
Basically, the rule is for N dimensions:
on all N dimensions you do the isnan() thing.
Then wrap it in with the all() function N-1 times,
and the 2nd argument each of the all() functions for the ith dimension should be numbers 1 to N in any order, but excluding i.
Since Theodros Zelleke wants to see whose method is faster (nice way of saying he thinks his method is so fast), here's a benchmark. Matrix A defined as:
A = NaN*ones(100,400,3,3);
A(2:4,2:3,2,2) = [1 2; 3 4; 5 6];
A(2,2,2,2) = NaN;A(4,3,2,2) = NaN;
A(5:80,4:200,2,2)=ones(76,197);
His test defined as:
tic;
for i=1:100
[i,j,k,z] = ind2sub(size(A),find(~isnan(A)));
l = min([i j k z]);
u = max([i j k z]);
B=A(l(1):u(1),l(2):u(2),l(3):u(3),l(4):u(4));
end
toc
With results:
Elapsed time is 0.533932 seconds.
Elapsed time is 0.519216 seconds.
Elapsed time is 0.575037 seconds.
Elapsed time is 0.525000 seconds.
My test defined as:
tic;
for i=1:100
isnanA=isnan(A);
ai34=all(all(isnanA,3),4);
ai12=all(all(isnanA,1),2);
B=A(~all(ai34,2),~all(ai34,1),~all(ai12,4),~all(ai12,3));
end
toc
With results:
Elapsed time is 0.224869 seconds.
Elapsed time is 0.225132 seconds.
Elapsed time is 0.246762 seconds.
Elapsed time is 0.236989 seconds.