How do I construct an Esri grid? - matlab

I read a lot of information about this subject but I can't obtain a solution about my problem.
First, I have a file with 3 columns: X Y Z
In MATLAB, I did this:
data = load('data.txt');
X = data(:,1);
Y = data(:,2);
Z = data(:,3);
This file is like this:
7037 6032 3
7036 6028 5
7037 6029 4
7037 6030 3
7038 6031 6
7039 6031 2
7037 6033 7
And I want to obtain the following matrix from the above matrix:
5 NaN NaN NaN NaN NaN
NaN 4 3 NaN 3 7
NaN NaN NaN 6 NaN NaN
NaN NaN NaN 2 NaN NaN
The rules is that the first column Y(1) = min(Y) , the second column Y(2) = Y(1) + 1.
The first line is X(1) = min(X), X(2) = X(1) + 1. Essentially, the first column acts as a row index, the second column acts as a column index, and for each row and column pair, the third column gets mapped to a location in this matrix. As such, the output matrix will be like so: out(1,1)=X(1) Y(1) ; out(1,2) = X(1) Y(2)
At the start, I think about created a matrix out like so:
xr = sort(unique(X));
yr = sort(unique(Y));
a = length(xr);
b = length(yr);
out = NaN(a,b);
After, with a loop, put I place this data onto this out matrix, but this obviously doesn't work.
For more information on an Esri grid, here's a Wikipedia article about it. The example grid in that page is what I desire. http://en.wikipedia.org/wiki/Esri_grid

I now understand what you want. The link that you posted from Wikipedia is very useful. You are trying to build what is known as an Esri grid. Here is a pictorial representation found on Wikipedia:
What you are given is a N x 3 matrix where the first column denotes the row IDs of this matrix, the second row denotes the column IDs of this matrix, and the third column denotes the values at each pair of IDs. So for example, given the example above - specifically looking at the right of the figure, your text file could look like:
275 125 5
275 175 2
...
...
25 75 5
25 125 1
Each row consists of a row index, a column index and a value that maps to this location in the grid. You had the right approach in that you should use unique - specifically the third output. We need to obtain a unique ID for the first two columns of your data independently. Once we do this, I'm going to show you the very powerful accumarray function. We are basically going to use the unique IDs found in the previous step, and we use these to index into our grid and place each value that corresponds to each unique pair of row and column IDs into this grid. Therefore, your code is very simply:
data = load('data.txt');
%// Or you can do this for reproducing the results
%data = [7037 6032 3;
%7036 6028 5;
%7037 6029 4;
%7037 6030 3;
%7038 6031 6;
%7039 6031 2;
%7037 6033 7];
[~,~,id1] = unique(data(:,1));
[~,~,id2] = unique(data(:,2));
out = accumarray([id1 id2], data(:,3), [], [], NaN);
out produces the desired Esri grid, and we get:
out =
5 NaN NaN NaN NaN NaN
NaN 4 3 NaN 3 7
NaN NaN NaN 6 NaN NaN
NaN NaN NaN 2 NaN NaN
So how does this work? accumarray accepts in a matrix of row and column locations that you want to use to access the output. At each of the corresponding row and column locations, you provide a value that gets mapped to this bin. Now, by default accumarray sums up the values that get mapped to each bin, but I'm going to assume that your values in your text file are all unique in that only one value gets mapped to each row and column index. Therefore, we can certainly get away with the default behaviour, and so you'd specify a [] for this behaviour (fourth input). Therefore, we will use the last column of your matrix as the values that get put into this matrix, use the [] input to allow accumarray to infer the size of your matrix (third input), then any values that don't get mapped to anything, we will fill this in with NaN. We aren't going to sum anything.
With the above explanation, the code follows.

Related

first non NaN element value and index in matlab

I want to obtain value and index of first non-NaN element from each column of a matrix in the matlab.
In a separate problem--- There are few columns that does not have NaN. So in that case-- I would like to extract value and index of first non-NaN element from each column and otherwise first element of each column if column does not contain NaN.
Can anybody help regarding these two problems?
The index is easily obtained with the second output of max. The value can be found from that index using sub2ind or computing the corresponding linear index manually.
To return a different index in columns that contain all NaN, use the first output of max to detect that situation and change the result for those columns.
Let x denote your matrix. Then:
[m, index] = max(~isnan(x), [], 1);
value = x(index + (0:size(x,2)-1)*size(x,1));
%// or equivalently x(sub2ind(size(x), index, 1:size(x,2)))
index(~m) = size(x, 1); %// return last index for columns that have all NaN
Example
x = [ 8 NaN 3 NaN
NaN 4 5 NaN];
produces
index =
1 2 1 2
value =
8 4 3 NaN

turn elements to NaN after first negative

I've got a three dimensional array in Matlab. The first dimension is time, the second is Humidity, and the third is Temperature. If a Temperature value is < 0, I want every subsequent temperature value to be turned to NaN.
For example if the array is:
>> sampl = randn(4,3,2)
sampl(:,:,1) =
0.79487 0.71017 -0.39167
0.51754 -1.3068 0.84166
0.49461 0.74159 0.082784
0.66393 1.4677 0.31467
sampl(:,:,2) =
0.78981 1.3096 1.0434
-0.80122 0.16037 -1.0682
-0.32565 -2.1182 -0.31723
0.28468 0.70708 1.4797
What's the most efficient way to turn this into:
sampl(:,:,1) =
0.79487 0.71017 NaN
0.51754 NaN NaN
0.49461 NaN NaN
0.66393 NaN NaN
sampl(:,:,2) =
0.78981 1.3096 1.0434
NaN 0.16037 NaN
NaN NaN NaN
NaN NaN NaN
Specifically, for a particular slice, we want to process along each column, and as soon as we encounter a negative number in one column, we want that location to be NaN as well as all row locations for that same column that follow this NaN value to also be NaN.
Another easy way is to find those locations that are negative in the original matrix, creating another matrix that sets those values toNaN, invoke a cumsum or a cumulative sum along all the rows for each column in each slice of this new matrix, then set the corresponding locations in this cumsum result to NaN in the original matrix to obtain the final result:
>> out = sampl;
>> out(out < 0) = NaN;
>> out = cumsum(out);
>> sampl(isnan(out)) = NaN
sampl(:,:,1) =
0.7949 0.7102 NaN
0.5175 NaN NaN
0.4946 NaN NaN
0.6639 NaN NaN
sampl(:,:,2) =
0.7898 1.3096 1.0434
NaN 0.1604 NaN
NaN NaN NaN
NaN NaN NaN
The reason why cumsum is useful here is because we would essentially examine each column independently along its rows and keep accumulating over all of the rows for each column which has valid entries until we hit a NaN value for a column. After this value, subsequent values in the cumsum would become NaN for each column in each slice independently. As such, after we hit the first NaN in a column, no matter what values we encounter after (NaN or a valid number), the result in the cumsum would still be NaN. This effectively propagates NaN values after we encounter the first negative in a column for your matrix. The last bit is to find those locations in this matrix and set the corresponding locations in the original matrix to NaN, thus giving our result.
Here is a solution using accumarray.
First, get the number of rows and reshape sampl to get a 2D array; it's easier to work with:
NumRow = size(sampl,1);
a = reshape(sampl,NumRow,[])
a looks like this:
a =
0.7949 0.7102 -0.3917 0.7898 1.3096 1.0434
0.5175 -1.3068 0.8417 -0.8012 0.1604 -1.0682
0.4946 0.7416 0.0828 -0.3256 -2.1182 -0.3172
0.6639 1.4677 0.3147 0.2847 0.7071 1.4797
Then find the first row index, for each column, that is negative:
[row,col] = find(a<0);
b = accumarray(col,row,[],#min);
Now b looks like this:
b =
0
2
1
2
3
2
Before inserting NaN's, change the 0 so that whole columns are not filled with NaN's using the colon operator (see next step):
b(b==0) = NumRow+1;
Finally loop through your array and insert NaN's starting from the corresponding index in b until the last row for every column. Also reshape a to get the same size as your initial array:
for k = 1:size(a,2)
a(b(k):NumRow,k) = NaN;
end
out = reshape(a,size(sampl))
Out:
out(:,:,1) =
0.7949 0.7102 NaN
0.5175 NaN NaN
0.4946 NaN NaN
0.6639 NaN NaN
out(:,:,2) =
0.7898 1.3096 1.0434
NaN 0.1604 NaN
NaN NaN NaN
NaN NaN NaN
Here is the whole code that you can copy/paste to run:
clear
clc
NumRow = size(sampl,1);
a = reshape(sampl,NumRow,[])
[row,col] = find(a<0);
b = accumarray(col,row,[],#min)
b(b==0) = NumRow+1;
for k = 1:size(a,2)
a(b(k):NumRow,k) = NaN;
end
out = reshape(a,size(sampl))
Missing a bsxfun-based solution, anyone?
[val, ind] = max(sampl<0); %// ind gives row index of first negative value, if any
ind(~val) = inf; %// if no negative values, set ind to inf so it has no effect
sampl(bsxfun(#ge, (1:size(sampl,1)).', ind)) = NaN; %'// logical indexing to fill NaNs

Mean and Standard Deviation of a column, ignoring zero values - Matlab

I am trying to find the mean of a column however I am having trouble getting an output for a function I created. My code is below, I cannot see what mistake I have made.
for j=1:48;
C_f2 = V(V(:,3) == j,:);
C_f2(C_f2==0)=NaN;
m=mean(C_f2(:,4));
s=std(C_f2(:,4));
row=[j,m,s];
s1=[s1;row];
end
I have checked the matrix, C_f2 and that is full of values so should not be returning NaN. However my output for the matrix s1 is
1 NaN NaN
2 NaN NaN
3 NaN NaN
. ... ...
48 NaN NaN
Can anyone see my issue? Help would me much appreciated!
The matrix C_f2 looks like,
1 185 01 5003
1 185 02 5009
. ... .. ....
1 259 48 5001
On line 3 you set all values which are zero to NaN. The mean function will return NaN as mean if any element is NaN. If you want to ignore the NaN values, you have to use the nanmean function, which comes with the Statistics toolbox. See the following example:
a = [1 NaN 2 3];
mean(a)
ans =
NaN
nanmean(a)
ans =
2
If you don't have the Statistics toolbox, you can exclude NaN elements with logical indexing
mean(a(~isnan(a)))
ans =
2
or it is possibly the easiest, if you directly exlude all elements which are zero instead of replacing them by NaN.
mean(a(a~=0))
Your line C_f2(C_f2==0)=NaN; will put NaNs into C_f2. Then, your mean and std operations will see those NaNs and output NaNs themselves.
To have the mean and std ignore NaN, you need to use the alternate version nanmean and nanstd.
These are part of a toolbox, however, so you might not have them if you just have the base Matlab installation.
Don't set it to NaN, any NaN involved computation without additional rules will return NaN,
use find to correctly index the none zero part of your column
say column n is your input
N = n(find(n~=0))
now do your Mu calculation
To compute the mean and standard deviation of each column excluding zeros:
A = [1 2;
3 0;
4 5;
6 7;
0 0]; %// example data
den = sum(A~=0); %// number of nonzero values in each column
mean_nz = bsxfun(#rdivide, sum(A), den);
mean2_nz = bsxfun(#rdivide, sum(A.^2), den);
std_nz = sqrt(bsxfun(#times, mean2_nz-mean_nz.^2, den./(den-1)));
The results for the example are
mean_nz =
3.5000 4.6667
std_nz =
2.0817 2.5166
The above uses the "corrected" definition of standard deviation (which divides by n-1, where n is the number of values). If you want the "uncorrected" version (i.e. divide by n):
std_nz = sqrt(mean2_nz-mean_nz.^2);

Calculation the elements of different sized matrix in Matlab

Can anybody help me to find out the method to calculate the elements of different sized matrix in Matlab ?
Let say that I have 2 matrices with numbers.
Example:
A=[1 2 3;
4 5 6;
7 8 9]
B=[10 20 30;
40 50 60]
At first,we need to find maximum number in each column.
In this case, Ans=[40 50 60].
And then,we need to find ****coefficient** (k).
Coefficient(k) is equal to 1 divided by quantity of column of matrix A.
In this case, **coefficient (k)=1/3=0.33.
I wanna create matrix C filling with calculation.
Example in MS Excel.
H4 = ABS((C2-C6)/C9)*0.33+ABS((D2-D6)/D9)*0.33+ABS((E2-E6)/E9)*0.33
I4 = ABS((C3-C6)/C9)*0.33+ABS((D3-D6)/D9)*0.33+ABS((E3-E6)/E9)*0.33
J4 = ABS((C4-C6)/C9)*0.33+ABS((D4-D6)/D9)*0.33+ABS((E4-E6)/E9)*0.33
And then (Like above)
H5 = ABS((C2-C7)/C9)*0.33+ABS((D2-D7)/D9)*0.33+ABS((E2-E7)/E9)*0.33
I5 = ABS((C3-C7)/C9)*0.33+ABS((D3-D7)/D9)*0.33+ABS((E3-E7)/E9)*0.33
J5 = ABS((C4-C7)/C9)*0.33+ABS((D4-D7)/D9)*0.33+ABS((E4-E7)/E9)*0.33
C =
0.34 =|(1-10)|/40*0.33+|(2-20)|/50*0.33+|(3-30)|/60*0.33
0.28 =|(4-10)|/40*0.33+|(5-20)|/50*0.33+|(6-30)|/60*0.33
0.22 =|(7-10)|/40*0.33+|(8-20)|/50*0.33+|(9-30)|/60*0.33
0.95 =|(1-40)|/40*0.33+|(2-50)|/50*0.33+|(3-60)|/60*0.33
0.89 =|(4-40)|/40*0.33+|(5-50)|/50*0.33+|(6-60)|/60*0.33
0.83 =|(7-40)|/40*0.33+|(8-50)|/50*0.33+|(9-60)|/60*0.33
Actually A is a 15x4 matrix and B is a 5x4 matrix.
Perhaps,the matrices dimensions are more than this matrices (variables).
How can i write this in Matlab?
Thanks you!
You can do it like so. Let's assume that A and B are defined as you did before:
A = vec2mat(1:9, 3)
B = vec2mat(10:10:60, 3)
A =
1 2 3
4 5 6
7 8 9
B =
10 20 30
40 50 60
vec2mat will transform a vector into a matrix. You simply specify how many columns you want, and it will automatically determine the right amount of rows to transform the vector into a correctly shaped matrix (thanks #LuisMendo!). Let's also define more things based on your post:
maxCol = max(B); %// Finds maximum of each column in B
coefK = 1 / size(A,2); %// 1 divided by number of columns in A
I am going to assuming that coefK is multiplied by every element in A. You would thus compute your desired matrix as so:
cellMat = arrayfun(#(x) sum(coefK*(bsxfun(#rdivide, ...
abs(bsxfun(#minus, A, B(x,:))), maxCol)), 2), 1:size(B,1), ...
'UniformOutput', false);
outputMatrix = cell2mat(cellMat).'
You thus get:
outputMatrix =
0.3450 0.2833 0.2217
0.9617 0.9000 0.8383
Seems like a bit much to chew right? Let's go through this slowly.
Let's start with the bsxfun(#minus, A, B(x,:)) call. What we are doing is taking the A matrix and subtracting with a particular row in B called x. In our case, x is either 1 or 2. This is equal to the number of rows we have in B. What is cool about bsxfun is that this will subtract every row in A by this row called by B(x,:).
Next, what we need to do is divide every single number in this result by the corresponding columns found in our maximum column, defined as maxCol. As such, we will call another bsxfun that will divide every element in the matrix outputted in the first step by their corresponding column elements in maxCol.
Once we do this, we weight all of the values of each row by coefK (or actually every value in the matrix). In our case, this is 1/3.
After, we then sum over all of the columns to give us our corresponding elements for each column of the output matrix for row x.
As we wish to do this for all of the rows, going from 1, 2, 3, ... up to as many rows as we have in B, we apply arrayfun that will substitute values of x going from 1, 2, 3... up to as many rows in B. For each value of x, we will get a numCol x 1 vector where numCol is the total number of columns shared by A and B. This code will only work if A and B share the same number of columns. I have not placed any error checking here. In this case, we have 3 columns shared between both matrices. We need to use UniformOutput and we set this to false because the output of arrayfun is not a single number, but a vector.
After we do this, this returns each row of the output matrix in a cell array. We need to use cell2mat to transform these cell array elements into a single matrix.
You'll notice that this is the result we want, but it is transposed due to summing along the columns in the second step. As such, simply transpose the result and we get our final answer.
Good luck!
Dedication
This post is dedicated to Luis Mendo and Divakar - The bsxfun masters.
Assuming by maximum number in each column, you mean columnwise maximum after vertically concatenating A and B, you can try this one-liner -
sum(abs(bsxfun(#rdivide,bsxfun(#minus,permute(A,[3 1 2]),permute(B,[1 3 2])),permute(max(vertcat(A,B)),[1 3 2]))),3)./size(A,2)
Output -
ans =
0.3450 0.2833 0.2217
0.9617 0.9000 0.8383
If by maximum number in each column, you mean columnwise maximum of B, you can try -
sum(abs(bsxfun(#rdivide,bsxfun(#minus,permute(A,[3 1 2]),permute(B,[1 3 2])),permute(max(B),[1 3 2]))),3)./size(A,2)
The output for this case stays the same as the previous case, owing to the values of A and B.

matrix get min values of a matrix before max values occurred

I was trying to get the min values of a matrix before the max values of the matrix occurred. I have two matrices: matrix data and matrix a. Matrix a is a subset of matrix data and is composed of the max values of matrix data. I have the following code but obviously doing something wrong.
edit:
Matrix a are the max values of matrix data. I derived it from:
for x=1:size(data,1)
a(x)=max(data(x,:));
end
a=a'
clear x
matrix b code:
for x=1:size(data,1)
b(x)=min(data(x,(x<data==a)));
end
b=b'
clear x
matrix data matrix a matrix b
1 2 3 4 4 1
6 5 4 7 7 4
9 6 12 5 12 6
I need all the min values that occurred before to matrix a occurred in matrix data
Short and simple:
[a,idxmax] = max(data,[],2);
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii))), 1:size(data,1));
which is the same as
b=NaN(1,size(data,1)); % preallocation!
for ii=1:size(data,1)
b(ii) = min(data(ii,1:idxmax(ii)));
end
Ignore maximum itself
If you want minimum of everything really before (and not including the maximum), it's possible that the maximum is the first number, and you try taking minimum of an empty matrix. Solution then is to use cell output, which can be empty:
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii)-1)), 1:size(data,1),'uni',false);
Replace empty cells with NaN
If you want to replace empty cells to Nan and then back to a matrix use this:
b(cellfun(#isempty,b))={NaN};
b=cell2mat(b);
or simply use the earlier version and replace b(ii) with NaN when it is equal to a(ii) same outcome:
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii))), 1:size(data,1));
b(b'==a) = NaN
Example:
data=magic(4)
16 2 3 13
5 11 10 8
9 7 6 12
4 14 15 1
outputs:
a' = 16 11 12 15
b =
16 5 6 4
and
b =[1x0 double] [5] [6] [4]
for the 2nd solution using cell output and ignoring the maximum itself also.
And btw:
for x=1:size(data,1)
a(x)=max(data(x,:));
end
a=a'
clear x
can be replaced with
a=max(data,[],2);
It's not pretty but this is the only way I found so far of doing this kind of thing without a loop.
If loops are ok I would recommend Gunther Struyf answer as the most compact use of matlab's in-built array looping function, arrayfun.
Some of the transposition etc may be superfluous if you're wanting column mins instead of row...
[mx, imx] = max(data');
inds = repmat(1:size(data,2), [size(data,1),1]);
imx2 = repmat(imx', [1, size(data,2)]);
data2 = data;
data2(inds >= imx2) = inf;
min(data2');
NOTE: if data is not needed we can remove the additional data2 variable, and reduce the line count.
So to demonstrate what this does, (and see if I understood the question correctly):
for input
>> data = [1,3,-1; 5,2,1]
I get minima:
>> min(data2')
ans = [1, inf]
I.e. it only found the min values before the max values for each row, and anything else was set to inf.
In words:
For each row get index of maximum
Generate matrix of column indices
Use repmat to generate a matrix, same size as data where each row is index of maximum
Set data to infinity where column index > max_index matrix
find min as usual.