I have two matrices of results, A = 128x631 and B = 128x1014 and I have a function SSD that takes two elements (x,y) as parameters and then calculates the sum of squared differences. I also have a 631x1014 matrix of 0s, called SSDMatrix, ready to put the results of my SSD function into.
What I'm trying to do is compare each element of A with each element of B by passing them into SSD, but I can't figure out how to structure my for loops to get the desired results.
When I try:
SSDMatrix = SSD(A, B);
I get exactly the result I'm looking for, but only for the first cell. How can I repeat this process for each element of A and B?
Currently I have this:
SSDMatrix = zeros(NumFeatures1,NumFeatures2);
for i = 1:631
for j = 1:1014
SSDMatrix(i,j) = SSD(A,B);
end
end
This just results in the first answer being repeated 631*1014 times, so I need a way to index A and B to get the appropriate answer for each (i,j) of SSDMatrix.
It seems you were needed to do something like this -
SSDMatrix = zeros(NumFeatures1,NumFeatures2);
for i = 1:631
for j = 1:1014
SSDMatrix(i,j) = sum( (A(:,i) - B(:,j)).^ 2 );
end
end
This, you can achieve with pdist2 as well that gets us the square root of summed squared distances. Now, please do note that pdist2 is part of the Statistics Toolbox. So, to get the desired output, you can do -
out = pdist2(A.',B.').^2;
Or with bsxfun -
out = squeeze(sum(bsxfun(#minus,A,permute(B,[1 3 2])).^2,1));
Related
I have a vector, v, of N positive integers whose values I do not know ahead of time. I would like to construct another vector, a, where the values in this new vector are determined by the values in v according to the following rules:
- The elements in a are all integers up to and including the value of each element in v
- 0 entries are included only once, but positive integers appear twice in a row
For example, if v is [1,0,2] then a should be: [0,1,1,0,0,1,1,2,2].
Is there a way to do this without just doing a for-loop with lots of if statements?
I've written the code in loop format but would like a vectorized function to handle it.
The classical version of your problem is to create a vector a with the concatenation of 1:n(i) where n(i) is the ith entry in a vector b, e.g.
b = [1,4,2];
gives a vector a
a = [1,1,2,3,4,1,2];
This problem is solved using cumsum on a vector ones(1,sum(b)) but resetting the sum at the points 1+cumsum(b(1:end-1)) corresponding to where the next sequence starts.
To solve your specific problem, we can do something similar. As you need two entries per step, we use a vector 0.5 * ones(1,sum(b*2+1)) together with floor. As you in addition only want the entry 0 to occur once, we will just have to start each sequence at 0.5 instead of at 0 (which would yield floor([0,0.5,...]) = [0,0,...]).
So in total we have something like
% construct the list of 0.5s
a = 0.5*ones(1,sum(b*2+1))
% Reset the sum where a new sequence should start
a(cumsum(b(1:end-1)*2+1)+1) =a(cumsum(b(1:end-1)*2+1)+1)*2 -(b(1:end-1)+1)
% Cumulate it and find the floor
a = floor(cumsum(a))
Note that all operations here are vectorised!
Benchmark:
You can do a benchmark using the following code
function SO()
b =randi([0,100],[1,1000]);
t1 = timeit(#() Nicky(b));
t2 = timeit(#() Recursive(b));
t3 = timeit(#() oneliner(b));
if all(Nicky(b) == Recursive(b)) && all(Recursive(b) == oneliner(b))
disp("All methods give the same result")
else
disp("Something wrong!")
end
disp("Vectorised time: "+t1+"s")
disp("Recursive time: "+t2+"s")
disp("One-Liner time: "+t3+"s")
end
function [a] = Nicky(b)
a = 0.5*ones(1,sum(b*2+1));
a(cumsum(b(1:end-1)*2+1)+1) =a(cumsum(b(1:end-1)*2+1)+1)*2 -(b(1:end-1)+1);
a = floor(cumsum(a));
end
function out=Recursive(arr)
out=myfun(arr);
function local_out=myfun(arr)
if isscalar(arr)
if arr
local_out=sort([0,1:arr,1:arr]); % this is faster
else
local_out=0;
end
else
local_out=[myfun(arr(1:end-1)),myfun(arr(end))];
end
end
end
function b = oneliner(a)
b = cell2mat(arrayfun(#(x)sort([0,1:x,1:x]),a,'UniformOutput',false));
end
Which gives me
All methods give the same result
Vectorised time: 0.00083574s
Recursive time: 0.0074404s
One-Liner time: 0.0099933s
So the vectorised one is indeed the fastest, by a factor approximately 10.
This can be done with a one-liner using eval:
a = eval(['[' sprintf('sort([0 1:%i 1:%i]) ',[v(:) v(:)]') ']']);
Here is another solution that does not use eval. Not sure what is intended by "vectorized function" but the following code is compact and can be easily made into a function:
a = [];
for i = 1:numel(v)
a = [a sort([0 1:v(i) 1:v(i)])];
end
Is there a way to do this without just doing a for loop with lots of if statements?
Sure. How about recursion? Of course, there is no guarantee that Matlab has tail call optimization.
For example, in a file named filename.m
function out=filename(arr)
out=myfun(in);
function local_out=myfun(arr)
if isscalar(arr)
if arr
local_out=sort([0,1:arr,1:arr]); % this is faster
else
local_out=0;
end
else
local_out=[myfun(arr(1:end-1)),myfun(arr(end))];
end
end
end
in cmd, type
input=[1,0,2];
filename(input);
You can take off the parent function. I added it just hoping Matlab can spot the recursion within filename.m and optimize for it.
would like a vectorized function to handle it.
Sure. Although I don't see the point of vectorizing in such a unique puzzle that is not generalizable to other applications. I also don't foresee a performance boost.
For example, assuming input is 1-by-N. In cmd, type
input=[1,0,2];
cell2mat(arrayfun(#(x)sort([0,1:x,1:x]),input,'UniformOutput',false)
Benchmark
In R2018a
>> clear all
>> in=randi([0,100],[1,100]); N=10000;
>> T=zeros(N,1);tic; for i=1:N; filename(in) ;T(i)=toc;end; mean(T),
ans =
1.5647
>> T=zeros(N,1);tic; for i=1:N; cell2mat(arrayfun(#(x)sort([0,1:x,1:x]),in,'UniformOutput',false)); T(i)=toc;end; mean(T),
ans =
3.8699
Ofc, I tested with a few more different inputs. The 'vectorized' method is always about twice as long.
Conclusion: Recursion is faster.
I am currently new to matlab, and I am trying to do a loop over fifty elements at a time instead of one element at a time. For example, I have a list of 1000 elements, and I would like to compute the sum for every fifty elements. Instead of doing a sum function through indexing, it would be much faster with a loop. How would I go about doing this?
I.e. [1,...50th element, 51th element... 100...]
Output would be the the sum values of 1:50, 51:100, 101:150... and so on.
Thanks in advance
I'm not really sure what you mean by "a sum function through indexing", but there are various ways to do this. In general I try to avoid explicit loops in Matlab and let MathWorks functions do their magic.
results = zeros(20,1);
for i = 1:20
results(i) = sum(1 + (50 * (i - 1)):50 + 50 * (i - 1));
end
Another option is to do something like arrayfun.
sIndex = 1:50:951;
eIndex = 50:50:1000;
result = arrayfun(#(x, y) sum(x:y), sIndex, eIndex);
You could also use reshape and sum to do it one shot.
numbers = 1:1000;
numbers2 = reshape(numbers, 50, []);
result = sum(numbers2);
This last method is what I personally would say is a Matlab way of doing it. arrayfun is basically a wrapper around a loop and the loop is...well a loop.
In case you need the sum, you can also use movsum:
array = 1:1000;
win = 50; % window size
msum = movsum(array,win,'Endpoints','discard');
in the same way, you can use:
movmax Moving maximum
movmean Moving mean
movmedian Moving median
movmin Moving minimum
movstd Moving standard deviation
movvar Moving variance
Using cumsum and diff you can obtain the desired result.
C = [0 cumsum(a)];
out = diff(C(1:50:end));
Lets say that I have an array x with some data values.
I performed a clustering algorithm that has produced a label map with the label names - labelMap. Each point in the data now has a unique cluster label associated to it.
I then perform the function foo(subset, secondArg) over each subset. The function foo returns a new array with the result which has the same size as its given the argument (its a map() function which also recieves a second argument).
What follows is the current implementation:
x = rand(1,1000);
numClusters = 3; % specified in advance by the user, for example using a clustering algorithm such as K-Means, this is a given.
fooSecondArg = [1,2,3]; % second argument for foo().
labelMap = kmeans(x,numClusters);
res = zeros(size(x));
%% make me run without a for loop :)
for ind = 1:numClusters
res(labelMap == ind) = foo(x(labelMap == ind), fooSecondArg(ind));
end
My question is as follows:
As there is no overlap between the indices in x foo() acts upon, is there a way to perform foo over x without using a for or a parfor loop? (I don't want to use a parfor loop as It takes quite a while to start the additional matlab processes and I cannot later on deploy it as a stand alone application).
Many thanks!
Edit:
I was asked for an example of foo() because I was told the solution may depend on it. A good viable example you may use in your answer:
function out = foo(x,secondArg)
out = x.^2/secondArg;
Whether the loop can be removed or not depends on the function you have. In your case the function is
function out = foo(x,secondArg)
out = x.^2/secondArg;
where secondArg depends on the cluster.
For this function (and for similar ones) you can indeed eliminate the loop as follows:
res = x.^2 ./ fooSecondArg(labelMap);
Here
labelMap is the same size of x (indicates the cluster label for each entry of x);
fooSecondArg is a vector of length equal to the number of clusters (indicates the second argument for each cluster).
Thus fooSecondArg(labelMap) is an array the same size of x, which for each point gives the second argument corresponding to that point's cluster. The operator ./ performs element-wise division to produce the desired result.
I have a code that repeatedly calculates a sparse matrix in a loop (it performs this calculation 13472 times to be precise). Each of these sparse matrices is unique.
After each execution, it adds the newly calculated sparse matrix to what was originally a sparse zero matrix.
When all 13742 matrices have been added, the code exits the loop and the program terminates.
The code bottleneck occurs in adding the sparse matrices. I have made a dummy version of the code that exhibits the same behavior as my real code. It consists of a MATLAB function and a script given below.
(1) Function that generates the sparse matrix:
function out = test_evaluate_stiffness(n)
ind = randi([1 n*n],300,1);
val = rand(300,1);
[I,J] = ind2sub([n,n],ind);
out = sparse(I,J,val,n,n);
end
(2) Main script (program)
% Calculate the stiffness matrix
n=1000;
K=sparse([],[],[],n,n,n^2);
tic
for i=1:13472
temp=rand(1)*test_evaluate_stiffness(n);
K=K+temp;
end
fprintf('Stiffness Calculation Complete\nTime taken = %f s\n',toc)
I'm not very familiar with sparse matrix operations so I may be missing a critical point here that may allow my code to be sped up considerably.
Am I handling the updating of my stiffness matrix in a reasonable way in my code? Is there another way that I should be using sparse that will result in a faster solution?
A profiler report is also provided below:
If you only need the sum of those matrices, instead of building all of them individually and then summing them, simply concatenate the vectors I,J and vals and call sparse only once. If there are duplicate rows [i,j] in [I,J] the corresponding values S(i,j) will be summed automatically, so the code is absolutely equivalent. As calling sparse involves an internal call to a sorting algorithm, you save 13742-1 intermediate sorts and can get away with only one.
This involves changing the signature of test_evaluate_stiffness to output [I,J,val]:
function [I,J,val] = test_evaluate_stiffness(n)
and removing the line out = sparse(I,J,val,n,n);.
You will then change your other function to:
n = 1000;
[I,J,V] = deal([]);
tic;
for i = 1:13472
[I_i, J_i, V_i] = test_evaluate_stiffness(n);
nE = numel(I_i);
I(end+(1:nE)) = I_i;
J(end+(1:nE)) = J_i;
V(end+(1:nE)) = rand(1)*V_i;
end
K = sparse(I,J,V,n,n);
fprintf('Stiffness Calculation Complete\nTime taken = %f s\n',toc);
If you know the lengths of the output of test_evaluate_stiffness ahead of time, you can possibly save some time by preallocating the arrays I,J and V with appropriately-sized zeros matrices and set them using something like:
I((i-1)*nE + (1:nE)) = ...
J((i-1)*nE + (1:nE)) = ...
V((i-1)*nE + (1:nE)) = ...
The biggest remaining computation, taking 11s, is the sparse operation
on the final I,J,V vectors so I think we've taken it down to the bare
bones.
Nearly... but one final trick: if you can create the vectors so that J is sorted ascending then you will greatly improve the speed of the sparse call, about a factor 4 in my experience.
(If it's easier to have I sorted, then create the transpose matrix sparse(J,I,V) and un-transpose it afterwards.)
I have a 3-dimensial matrix W of size 160x170x18 and I want to compute the difference
between each sucessive matrices inside W.
For example diff1 = W(:,:,1) - W(:,:,2) and diff2 = W(:,:,2) - W(:,:,3), etc ...
Next I want to select some special parts of the resulting matrices, For example:
NewDiff1 = [diff1(20:50,110:140); diff1(60:90,110:140)];
and the same thing for the other matrices.
finally I want to compute the mean of each matrix and the error as follow:
mean1 = mean(mean(NewDiff1));
er1 = 0.1-abs(mean1);
I succeeded to do this for each matrix alone, but prefer to do all at once in a for loop.
The expression
diff1 = diff(W,1,3)
will return, in your example, a 160*170*17 matrix where diffW(:,:,1) = W(:,:,2) - W(:,:,1), which isn't quite what you want. But
diff1 = (-1)*diff(W,1,3)
does, if my arithmetic is good, give you the differences you want. From there on you need something like:
newdiff1 = [diff1(20:50,110:140,:);diff1(60:90,110:140,:)];
and
means = mean(mean(newdiff1));
er1 = 0.1 - abs(mean1);
I haven't tested this thoroughly on matrices of the size you are working with, but it seems to work OK on smaller tests.
Store your matrices into a cell array and then just loop through the contents of the cell array and apply the same differencing logic to each thing. Be careful to use the {} syntax with a cell array to get its contents, rather than () which gives you the cell at a particular location.