I have a 3-dimensial matrix W of size 160x170x18 and I want to compute the difference
between each sucessive matrices inside W.
For example diff1 = W(:,:,1) - W(:,:,2) and diff2 = W(:,:,2) - W(:,:,3), etc ...
Next I want to select some special parts of the resulting matrices, For example:
NewDiff1 = [diff1(20:50,110:140); diff1(60:90,110:140)];
and the same thing for the other matrices.
finally I want to compute the mean of each matrix and the error as follow:
mean1 = mean(mean(NewDiff1));
er1 = 0.1-abs(mean1);
I succeeded to do this for each matrix alone, but prefer to do all at once in a for loop.
The expression
diff1 = diff(W,1,3)
will return, in your example, a 160*170*17 matrix where diffW(:,:,1) = W(:,:,2) - W(:,:,1), which isn't quite what you want. But
diff1 = (-1)*diff(W,1,3)
does, if my arithmetic is good, give you the differences you want. From there on you need something like:
newdiff1 = [diff1(20:50,110:140,:);diff1(60:90,110:140,:)];
and
means = mean(mean(newdiff1));
er1 = 0.1 - abs(mean1);
I haven't tested this thoroughly on matrices of the size you are working with, but it seems to work OK on smaller tests.
Store your matrices into a cell array and then just loop through the contents of the cell array and apply the same differencing logic to each thing. Be careful to use the {} syntax with a cell array to get its contents, rather than () which gives you the cell at a particular location.
Related
I have two cell arrays. One is 'trans_blk' of size <232324x1> consists of cells of size <8x8> and another 'ca' is of size <1024x1> consists of cells of size <8x8>.
I want to compute mean square error (MSE) for each cell of 'ca' with respect to every cell of 'trans_blk'.
I used the following code to compute:
m=0;
for ii=0:7
for jj=0:7
m=m+((trans_blk{:,1}(ii,jj)-ca{:,1}(ii,jj))^2);
end
end
m=m/(size of cell); //size of cell=8*8
disp('MSE=',m);
Its giving an error. Bad cell reference operation in MATLAB.
A couple of ways that I figured you could go:
% First define the MSE function
mse = #(x,y) sum(sum((x-y).^2))./numel(x);
I'm a big fan of using bsxfun for things like this, but unfortunately it doesn't operate on cell arrays. So, I borrowed the singleton expansion form of the answer from here.
% Singleton expansion way:
mask = bsxfun(#or, true(size(A)), true(size(B))');
idx_A = bsxfun(#times, mask, reshape(1:numel(A), size(A)));
idx_B = bsxfun(#times, mask, reshape(1:numel(B), size(B))');
func = #(x,y) cellfun(#(a,b) mse(a,b),x,y);
C = func(A(idx_A), B(idx_B));
Now, if that is a bit too crazy (or if explicitly making the arrays by A(idx_A) is too big), then you could always try a loop approach such as the one below.
% Or a quick loop:
results = zeros(length(A),length(B));
y = B{1};
for iter = 1:length(B)
y = B{iter};
results(:,iter) = cellfun(#(x) mse(x,y) ,A);
end
If you run out of memory: Think of what you are allocating: a matrix of doubles that is (232324 x 1024) elements. (That's a decent chunk of memory. Depending on your system, that could be close to 2GB of memory...)
If you can't hold it all in memory, then you might have to decide what you are going to do with all the MSE's and either do it in batches, or find a machine that you can run the full simulation/code on.
EDIT
If you only want to keep the sum of all the MSEs (as OP states in comments below), then you can save on memory by
% Sum it as you go along:
results = zeros(length(A),1);
y = B{1};
for iter = 1:length(B)
y = B{iter};
results = results + cellfun(#(x) mse(x,y) ,A);
end
results =sum (results);
I have a code that repeatedly calculates a sparse matrix in a loop (it performs this calculation 13472 times to be precise). Each of these sparse matrices is unique.
After each execution, it adds the newly calculated sparse matrix to what was originally a sparse zero matrix.
When all 13742 matrices have been added, the code exits the loop and the program terminates.
The code bottleneck occurs in adding the sparse matrices. I have made a dummy version of the code that exhibits the same behavior as my real code. It consists of a MATLAB function and a script given below.
(1) Function that generates the sparse matrix:
function out = test_evaluate_stiffness(n)
ind = randi([1 n*n],300,1);
val = rand(300,1);
[I,J] = ind2sub([n,n],ind);
out = sparse(I,J,val,n,n);
end
(2) Main script (program)
% Calculate the stiffness matrix
n=1000;
K=sparse([],[],[],n,n,n^2);
tic
for i=1:13472
temp=rand(1)*test_evaluate_stiffness(n);
K=K+temp;
end
fprintf('Stiffness Calculation Complete\nTime taken = %f s\n',toc)
I'm not very familiar with sparse matrix operations so I may be missing a critical point here that may allow my code to be sped up considerably.
Am I handling the updating of my stiffness matrix in a reasonable way in my code? Is there another way that I should be using sparse that will result in a faster solution?
A profiler report is also provided below:
If you only need the sum of those matrices, instead of building all of them individually and then summing them, simply concatenate the vectors I,J and vals and call sparse only once. If there are duplicate rows [i,j] in [I,J] the corresponding values S(i,j) will be summed automatically, so the code is absolutely equivalent. As calling sparse involves an internal call to a sorting algorithm, you save 13742-1 intermediate sorts and can get away with only one.
This involves changing the signature of test_evaluate_stiffness to output [I,J,val]:
function [I,J,val] = test_evaluate_stiffness(n)
and removing the line out = sparse(I,J,val,n,n);.
You will then change your other function to:
n = 1000;
[I,J,V] = deal([]);
tic;
for i = 1:13472
[I_i, J_i, V_i] = test_evaluate_stiffness(n);
nE = numel(I_i);
I(end+(1:nE)) = I_i;
J(end+(1:nE)) = J_i;
V(end+(1:nE)) = rand(1)*V_i;
end
K = sparse(I,J,V,n,n);
fprintf('Stiffness Calculation Complete\nTime taken = %f s\n',toc);
If you know the lengths of the output of test_evaluate_stiffness ahead of time, you can possibly save some time by preallocating the arrays I,J and V with appropriately-sized zeros matrices and set them using something like:
I((i-1)*nE + (1:nE)) = ...
J((i-1)*nE + (1:nE)) = ...
V((i-1)*nE + (1:nE)) = ...
The biggest remaining computation, taking 11s, is the sparse operation
on the final I,J,V vectors so I think we've taken it down to the bare
bones.
Nearly... but one final trick: if you can create the vectors so that J is sorted ascending then you will greatly improve the speed of the sparse call, about a factor 4 in my experience.
(If it's easier to have I sorted, then create the transpose matrix sparse(J,I,V) and un-transpose it afterwards.)
I have two matrices of results, A = 128x631 and B = 128x1014 and I have a function SSD that takes two elements (x,y) as parameters and then calculates the sum of squared differences. I also have a 631x1014 matrix of 0s, called SSDMatrix, ready to put the results of my SSD function into.
What I'm trying to do is compare each element of A with each element of B by passing them into SSD, but I can't figure out how to structure my for loops to get the desired results.
When I try:
SSDMatrix = SSD(A, B);
I get exactly the result I'm looking for, but only for the first cell. How can I repeat this process for each element of A and B?
Currently I have this:
SSDMatrix = zeros(NumFeatures1,NumFeatures2);
for i = 1:631
for j = 1:1014
SSDMatrix(i,j) = SSD(A,B);
end
end
This just results in the first answer being repeated 631*1014 times, so I need a way to index A and B to get the appropriate answer for each (i,j) of SSDMatrix.
It seems you were needed to do something like this -
SSDMatrix = zeros(NumFeatures1,NumFeatures2);
for i = 1:631
for j = 1:1014
SSDMatrix(i,j) = sum( (A(:,i) - B(:,j)).^ 2 );
end
end
This, you can achieve with pdist2 as well that gets us the square root of summed squared distances. Now, please do note that pdist2 is part of the Statistics Toolbox. So, to get the desired output, you can do -
out = pdist2(A.',B.').^2;
Or with bsxfun -
out = squeeze(sum(bsxfun(#minus,A,permute(B,[1 3 2])).^2,1));
I have one array (the "true" cartesian coordinates) which is of size (natoms*3,1) where natoms is the number of atoms. I also have a large number (500,000) of observations stored in an array of size (nobs, natoms*3). Now, I want to create an array of the differences between all observations against the true coordinates. I would like to simply vectorize this by doing something like
for iat = 1:natoms
xyz_dif = xyz_obs(:, 3*iat-2:3*iat) - xyz_true(3*iat-2:3*iat)
end
but this does not work. Instead I am forced to go through each of the observtions like so:
for iat = 1:natoms
for iobs = 1:nobs
xyz_diff(iobs, 3*iat-2:3*iat) = xyzs(iobs, 3*iat-2:3*iat) - xyz_true(3*iat-2:3*iat)
end
end
but this seems quite inefficient. Is there a faster, more efficient way to do this?
Thanks.
use bsxfun
xyz_diff = bsxfun(#minus, xyz_true', xyz_obs)
an alternative solution, which in my view is more readable is to use matrix multiplication:
xyz_diff = xyz_obs-ones(nobs,1)*xyz_true;
I am using interp1 to inteprolate some data:
temp = 4 + (30-4).*rand(365,10);
depth = 1:10;
dz = 0.5; %define new depth interval
bthD = min(depth):dz:max(depth); %new depth vector
for i = 1:length(temp);
i_temp(i,:) = interp1(depth,temp(i,:),bthD);
end
Here, I am increasing the resolution of my measurements by interpolating the measurements from 1 m increments to 0.5 m increments. This code works fine i.e. it gives me the matrix I was looking for. However, when I apply this to my actual data, it takes a long time to run, primarily as I am running an additional loop which runs through various cells. Is there a way of achieving what is described above without using the loop, in other words, is there a faster method?
Replace your for loop with:
i_temp = interp1(depth,temp',bthD)';
You can get rid of the transposes if you change the way that temp is defined, and if you are OK with i_temp being a 19x365 array instead of 365x19.
BTW, the documentation for interp1 is very clear that you can pass in an array as the second argument.