Related
I have a matlab program with 5 nested
for
loops and a
if
condition like this:
for x0=1:N
for y0=1:N
for k=1:N
for x1=1:N
for y1=1:N
if ~((y1-x1>N/2)||(x1-y1>N/2)) && ~((y0-x0>N/2)||(x0-y0>N/2))
A(x0,y0)=A(x0,y0)+2^(k*((x0-y0)+(x1-y1)))*B(x1,y1)
end
end
end
end
end
end
where A and B are two matrices. How can I make this program run faster?
I've tried to use meshgrid but it seems doesn't work because there's a
if
condition.
Lets be smart about loops and conditions first, as you are using the loop indices as condition variables.
We start with
~(y1-x1>N/2)||(x1-y1>N/2), or way clearer, abs(y1-x1)<N/2.
Instead of having an if condition, why not enforce y1 to be in range, always?
The last loop can be written as y1=max(x1-N/2,1):min(x1+N/2,N), and thus the entirety of the first part of the if condition is not needed. We can do the same for the other variables, of course:
for x0=1:N
for y0=max(x0-N/2,1):min(x0+N/2,N)
for k=1:N
for x1=1:N
for y1=max(x1-N/2,1):min(x1+N/2,N)
A(x0,y0)=A(x0,y0)+2^(k*((x0-y0)+(x1-y1)))*B(x1,y1)
end
end
end
end
end
Now, for clarity, lets reshuffle and vectorize that k. There is no need for it to be the middle loop, in fact, its only feature as the middle loop is to confuse the person reading the code. But aside from that, there is no need for it to be a loop either.
k=1:N;
for x0=1:N
for y0=max(x0-N/2,1):min(x0+N/2,N)
for x1=1:N
for y1=max(x1-N/2,1):min(x1+N/2,N)
A(x0,y0)=A(x0,y0)+sum(2.^(k*((x0-y0)+(x1-y1))))*B(x1,y1)
end
end
end
end
Now, is this faster?
No. MATLAB is really good at optimizing your code, so it is not faster. But at least its way way clearer, so I guess you got that going for you. But you need it faster! Well.... I am not sure you can. You have a 5 nested loops, that is just super slow. I don't think you can meshgrid this, even without the conditions, because you intermingle all loops. meshgrid is good when well, you do operations on a mesh grid, but in your case you use all x1,y1 for every x0,y0 and thus its not a mesh operation.
Here is a vectorized solution:
x0 = (1:N).';
y0 = 1:N;
x1 = (1:N).';
y1 = 1:N;
k = reshape(1:N, 1, 1, N);
conditiona = ~((y0-x0 > N/2) | (x0-y0 > N/2));
conditionb = ~((y1-x1 > N/2) | (x1-y1 > N/2));
a = 2 .^ (k .* (x0-y0)) .* conditiona;
b = 2 .^ (k .* (x1-y1)) .* B .* conditionb;
bsum = squeeze(sum(sum(b, 1) ,2));
A = A + reshape(reshape(a, [] , N) * bsum ,N ,N);
Note that two 3D arrays a and b are created that may/may not require a lot of memory. In such a case you need to loop over k. For example in the first iteration set k to 1:5. In the second iteration set it to 6:10 and so on. You need to addv the result of each iteration to the previous iteration to form the final A.
Explanation
This function can be vectorized by implicit expansion (that is more efficient than using meshgrid) and using element-wise operators like .^ and .* instead of ^ and * operators. As a result a 5D array is created (because we have 5 loop variables) that should be summed over 3-5th dimensions to produce the final 2D matrix. However that may require a lot of memory. Another point is that functions that contains the sum of products usually can be written as efficient matrix multiplication.
The expression:
2^(k*((x0-y0)+(x1-y1)))*B(x1,y1);
can be written as:
2 .^ (k .* (x0-y0)) .* 2 .^ (k .* (x1-y1)) .* B(x1, y1)
------- a -------- ------------- b ---------------
that is the multiplication of two sub-expressions that each has 3 dimensions, because each contains just 3 loop variables. So the 5D problem is reduced to 3D.
The if condition has also two sub-expressions that each can be multiplied by a and b sub-expressions:
conditiona = ~((y0-x0 > N/2) | (x0-y0 > N/2));
conditionb = ~((y1-x1 > N/2) | (x1-y1 > N/2));
a = 2 .^ (k .* (x0-y0)) .* conditiona;
b = 2 .^ (k .* (x1-y1)) .* B .* conditionb;
A for loop can be formed just by using two loop variables x0 and y0:
for x0=1:N
for y0=1:N
A(x0,y0)=A(x0,y0)+ sum(sum(sum(a(x0,x0, :) .* b, 3), 2), 1);
%or simply A(x0,y0)=A(x0,y0)+ sum(a(x0,x0, :) .* b, "all");
end
end
That can be simplified to the following loop by precomputing sum of b:
bsum = sum(sum(b, 1) ,2);
% bsum = sum(b ,[1, 2]);
for x0=1:N
for y0=1:N
A(x0,y0)=A(x0,y0)+ sum(a(x0,x0, :) .* bsum, 3);
% or as vector x vector multiplication
% A(x0,y0)=A(x0,y0)+ squeeze(a(x0,x0, :)).' * squeeze(bsum);
end
end
Here the loop can be prevented by using the matrix x vector multiplication:
A = A + reshape(reshape(a, [] , N) * bsum ,N ,N);
Update this solution may not be faster under Matlab, because the execution engine can optimise the loops in the original code. (It does provide a speedup under Octave.)
One trick to deal with if statements within loops is to turn the if statement (or part of it) into a logical matrix. You can then multiply the logical matrix elementwise by the matrix of values you are adding in each step. A false value will evaluate to zero and will not change the result.
This only works if each element can be computed independently of the others.
It will generally make the actual calculation line slower, but in Matlab this is often outweighed by the huge speed improvement from the the removal of the for loops.
In your example, we can use this idea with meshgrid to remove the loops over x0 and y0.
The calculation line needs to become an elementwise matrix caluclation, so elementwise operators .*, .^ and | replace *, ^ and |.
% Warning: Y0 and X0 are swapped in this example
[Y0, X0] = meshgrid(1:N,1:N);
% Create a logical matrix which represents part of the if statement
C = ~((Y0-X0>N/2) | (X0-Y0>N/2));
for k=1:N
for x1=1:N
for y1=1:N
if ~((y1-x1>N/2)||(x1-y1>N/2))
% Multiply by C elementwise
A = A + C.*2.^(k*((X0-Y0)+(x1-y1)))*B(x1,y1);
end
end
end
end
You could even take this concept further and remove more loops using multidemensional ndgrids, but it becomes more complex (you have to start summing over dimensions) and the multidimensional arrays become unwieldy if N is large.
Note: be careful with index order. meshgrid defines y as rows and x as columns, so matrix indexing is A(y,x) but you are using A(x,y). So to make your example work I've switched x and y in the output of meshgrid.
I am using an R2019a matlab.
I have a matrix that I want to fill with values at certain positions, based on a calculation done on 2 additional vectors.
Currently I'm doing it with 2 loop, but this doesn't take advantage of matlab's vectorization abilities.
How can the following script be performed in a vectorized manner:
C = zeros(size(vecB), size(vecA));
% Calculate face-vertix connectivity of face area values:
posMat = sparse(A, repmat((1:size(A,1))',1,3), ...
1, size(B,1), size(A,1));
for i = 1:size(B,1)
for j = 1:size(A,1)
if posMat(i,j) == 1
C(i,j) = vecA(j)/vecB(i)/3;
end
end
end
Sizes of variables in the script:
size(A) = 5120 3
size(B) = 2562 3
size(B) = 2562 5120
size(posMat) = 2562 5120
size(vecA) = 5120 1
size(vecB) = 2562 1
Because the condition inside your loop is only satisfied by nonzero elements of posMat, the important thing for efficiency is to take advantage of the fact that this matrix is sparse in a vectorized implementation.
If you use multiplication by a sparse condition as the means of setting the elements that fail the condition to 0 (rather than initializing the matrix using zeros) then only the elements that pass the condition will actually be evaluated.
From R2016b onwards, vecA.'./vecB/3 is implicitly expanded to return the full 2562×5120 double array of vecA(j)/vecB(i)/3 for all values of i and j. But (posMat == 1) .* vecA.'./vecB/3 returns a 2562×5120 sparse double array where the division has only been evaluated for the elements where posMat == 1.
If a sparse value of C if acceptable, then
C = (posMat == 1) .* vecA.'./vecB/3;
will suffice. If the full-storage form is required this output can simply be passed to the full function.
The specific task I'm trying to achieve is hard to describe, so here's an example: given A and x
A = [1 2;
3 0;
3 5;
4 0];
x = [1 2 3];
I want the algorithm to output
output: [1 2]
meaning that all of the nonzero elements in rows 1 and 2 in A are in x.
I have done this using cell arrays and loops; however, A and x are very large and my approach is not at all efficient. Also, I can't seem to figure out how to rework ismember to give me what I want. What is the fastest/least memory intensive method?
EDIT: Apologies, my original example was too simplistic. It is corrected now.
The first answer is good, but I would recommend to not using arrayfun. There are more eloquent ways to do what you ask. Use ismember combined with all, then index into the matrix A when you're done. Basically, your problem is to determine if a row has all of the values found in x and ignoring the zero values. In this case, we can find all of the values in the matrix A that are actually zero, then use this to augment our result.
Using A as the first input and x as the second input will return a matrix of the same size as A that tells you whether an element in A is found in x. If you want to check if all elements in the matrix A for a row can be found in x, check if all elements in a row is 1. On top of this, find all of the elements that are zero, then with the output of ismember set these to 1. This can be done with using a logical OR. After, you can use all and check each row independently by using the output of ismember as the first input into all and setting the second argument to 2. This would then return all of the rows in the matrix A where any column is found in x ignoring any values that are zero for a row in A which is what you're looking for:
A = [1 2; 3 0; 4 0];
x = [1 2 3];
mask = ismember(A, x);
ind = all(mask | A == 0, 2);
I'm also in favour of one-liners. We can consolidate this into one line of code:
ind = all(ismember(A, x) | A == 0, 2);
Even shorter is to simply invert A. All zero elements become true and false otherwise:
ind = all(ismember(A, x) | ~A, 2);
ind would thus be:
>> ind
ind =
3×1 logical array
1
1
0
Since you want the actual row indices, you can just use find on top of this:
>> find(ind)
ans =
1
2
To verify, let's use your second example in your comments:
>> A = [1 2;3 5;4 0];
>> x = [1 2 3];
>> ind = all(ismember(A, x) | ~A, 2)
ind =
3×1 logical array
1
0
0
>> find(ind)
ans =
1
I think the best way to rework ismember is to make sure there are no "no members" by just checking for the nonzero elements in A.
arrayfun can do the work in a fast way. It uses the most efficient parallel computing for your specific machine. The following line should return the correct output:
find(arrayfun(#(a) sum(~ismember(A(a,A(a,:)>0),x)),1:size(A,1))==0)
Is this what you were looking for?
However, if your problem is related to memory, then you may have to break the arrayfun operation into pieces (1:floor(size(A,1)/2), floor(size(A,1)/2):size(A,1) or smaller chunks), since MATLAB puts a bunch of workers to do the task, and may use all your available RAM memory...
This question already has answers here:
Is there a Matlab conditional IF operator that can be placed INLINE like VBA's IIF
(10 answers)
Closed 9 years ago.
I'm looking for Matlab equivalent of c# condition ? true-expression : false-expression conditional assignment. The most I know of is a = 5>2, which is true\false assignment,
but is there any one line conditional assignment for
if condition a=1;else a=2; end?
For numeric arrays, there is another solution --
// C:
A = COND ? X : Y;
becomes
% MATLAB
% A, X and Y are numerics
% COND is a logical condition.
A = COND.*X + (~COND).*Y ;
Advantage:
works wonderfully in parallel for vectors or large arrays - each item in A gets assigned depending on the corresponding condition. The same line works for:
condition is scalar, arrays X and Y are equal in size
condition is an array of any size, X and Y are scalars
condition and X and Y are all arrays of the same size
Warning:
Doesn't work gracefully with NaNs. Beware! If an element of X is nan, or an element of Y is nan, then you'll get a NaN in A, irrespective of the condition.
Really Useful corollary:
you can use bsxfun where COND and X/Y have different sizes.
A = bsxfun( #times, COND', X ) + bsxfun( #times, ~COND', Y );
works for example where COND and X/Y are vectors of different lengths.
neat eh?
One line conditional assignment:
a(a > 5) = 2;
This is an example of logical indexing, a > 5 is a logical (i.e. Boolean or binary) matrix/array the same size as a with a 1 where ever the expression was true. The left side of the above assignment refers to all the positions in a where a>5 has a 1.
b = a > 5; % if a = [9,3,5,6], b = [1,0,0,1]
a(~b) = 3;
c = a > 10;
a(b&c) = ...
Etc...you can do pretty much anything you'd expect with such logical arrays.
Matlab does not have a ternary operator. You though easily write a function that will do such thing for you:
function c = conditional(condition , a , b)
if condition
c = a;
else
c = b;
end
end
You can apply a function to every item in a vector by saying, for example, v + 1, or you can use the function arrayfun. How can I do it for every row/column of a matrix without using a for loop?
Many built-in operations like sum and prod are already able to operate across rows or columns, so you may be able to refactor the function you are applying to take advantage of this.
If that's not a viable option, one way to do it is to collect the rows or columns into cells using mat2cell or num2cell, then use cellfun to operate on the resulting cell array.
As an example, let's say you want to sum the columns of a matrix M. You can do this simply using sum:
M = magic(10); %# A 10-by-10 matrix
columnSums = sum(M, 1); %# A 1-by-10 vector of sums for each column
And here is how you would do this using the more complicated num2cell/cellfun option:
M = magic(10); %# A 10-by-10 matrix
C = num2cell(M, 1); %# Collect the columns into cells
columnSums = cellfun(#sum, C); %# A 1-by-10 vector of sums for each cell
You may want the more obscure Matlab function bsxfun. From the Matlab documentation, bsxfun "applies the element-by-element binary operation specified by the function handle fun to arrays A and B, with singleton expansion enabled."
#gnovice stated above that sum and other basic functions already operate on the first non-singleton dimension (i.e., rows if there's more than one row, columns if there's only one row, or higher dimensions if the lower dimensions all have size==1). However, bsxfun works for any function, including (and especially) user-defined functions.
For example, let's say you have a matrix A and a row vector B. E.g., let's say:
A = [1 2 3;
4 5 6;
7 8 9]
B = [0 1 2]
You want a function power_by_col which returns in a vector C all the elements in A to the power of the corresponding column of B.
From the above example, C is a 3x3 matrix:
C = [1^0 2^1 3^2;
4^0 5^1 6^2;
7^0 8^1 9^2]
i.e.,
C = [1 2 9;
1 5 36;
1 8 81]
You could do this the brute force way using repmat:
C = A.^repmat(B, size(A, 1), 1)
Or you could do this the classy way using bsxfun, which internally takes care of the repmat step:
C = bsxfun(#(x,y) x.^y, A, B)
So bsxfun saves you some steps (you don't need to explicitly calculate the dimensions of A). However, in some informal tests of mine, it turns out that repmat is roughly twice as fast if the function to be applied (like my power function, above) is simple. So you'll need to choose whether you want simplicity or speed.
I can't comment on how efficient this is, but here's a solution:
applyToGivenRow = #(func, matrix) #(row) func(matrix(row, :))
applyToRows = #(func, matrix) arrayfun(applyToGivenRow(func, matrix), 1:size(matrix,1))'
% Example
myMx = [1 2 3; 4 5 6; 7 8 9];
myFunc = #sum;
applyToRows(myFunc, myMx)
Building on Alex's answer, here is a more generic function:
applyToGivenRow = #(func, matrix) #(row) func(matrix(row, :));
newApplyToRows = #(func, matrix) arrayfun(applyToGivenRow(func, matrix), 1:size(matrix,1), 'UniformOutput', false)';
takeAll = #(x) reshape([x{:}], size(x{1},2), size(x,1))';
genericApplyToRows = #(func, matrix) takeAll(newApplyToRows(func, matrix));
Here is a comparison between the two functions:
>> % Example
myMx = [1 2 3; 4 5 6; 7 8 9];
myFunc = #(x) [mean(x), std(x), sum(x), length(x)];
>> genericApplyToRows(myFunc, myMx)
ans =
2 1 6 3
5 1 15 3
8 1 24 3
>> applyToRows(myFunc, myMx)
??? Error using ==> arrayfun
Non-scalar in Uniform output, at index 1, output 1.
Set 'UniformOutput' to false.
Error in ==> #(func,matrix)arrayfun(applyToGivenRow(func,matrix),1:size(matrix,1))'
For completeness/interest I'd like to add that matlab does have a function that allows you to operate on data per-row rather than per-element. It is called rowfun (http://www.mathworks.se/help/matlab/ref/rowfun.html), but the only "problem" is that it operates on tables (http://www.mathworks.se/help/matlab/ref/table.html) rather than matrices.
Adding to the evolving nature of the answer to this question, starting with r2016b, MATLAB will implicitly expand singleton dimensions, removing the need for bsxfun in many cases.
From the r2016b release notes:
Implicit Expansion: Apply element-wise operations and functions to arrays with automatic expansion of dimensions of length 1
Implicit expansion is a generalization of scalar expansion. With
scalar expansion, a scalar expands to be the same size as another
array to facilitate element-wise operations. With implicit expansion,
the element-wise operators and functions listed here can implicitly
expand their inputs to be the same size, as long as the arrays have
compatible sizes. Two arrays have compatible sizes if, for every
dimension, the dimension sizes of the inputs are either the same or
one of them is 1. See Compatible Array Sizes for Basic Operations and
Array vs. Matrix Operations for more information.
Element-wise arithmetic operators — +, -, .*, .^, ./, .\
Relational operators — <, <=, >, >=, ==, ~=
Logical operators — &, |, xor
Bit-wise functions — bitand, bitor, bitxor
Elementary math functions — max, min, mod, rem, hypot, atan2, atan2d
For example, you can calculate the mean of each column in a matrix A,
and then subtract the vector of mean values from each column with A -
mean(A).
Previously, this functionality was available via the bsxfun function.
It is now recommended that you replace most uses of bsxfun with direct
calls to the functions and operators that support implicit expansion.
Compared to using bsxfun, implicit expansion offers faster speed,
better memory usage, and improved readability of code.
None of the above answers worked "out of the box" for me, however, the following function, obtained by copying the ideas of the other answers works:
apply_func_2_cols = #(f,M) cell2mat(cellfun(f,num2cell(M,1), 'UniformOutput',0));
It takes a function f and applies it to every column of the matrix M.
So for example:
f = #(v) [0 1;1 0]*v + [0 0.1]';
apply_func_2_cols(f,[0 0 1 1;0 1 0 1])
ans =
0.00000 1.00000 0.00000 1.00000
0.10000 0.10000 1.10000 1.10000
With recent versions of Matlab, you can use the Table data structure to your advantage. There's even a 'rowfun' operation but I found it easier just to do this:
a = magic(6);
incrementRow = cell2mat(cellfun(#(x) x+1,table2cell(table(a)),'UniformOutput',0))
or here's an older one I had that doesn't require tables, for older Matlab versions.
dataBinner = cell2mat(arrayfun(#(x) Binner(a(x,:),2)',1:size(a,1),'UniformOutput',0)')
The accepted answer seems to be to convert to cells first and then use cellfun to operate over all of the cells. I do not know the specific application, but in general I would think using bsxfun to operate over the matrix would be more efficient. Basically bsxfun applies an operation element-by-element across two arrays. So if you wanted to multiply each item in an n x 1 vector by each item in an m x 1 vector to get an n x m array, you could use:
vec1 = [ stuff ]; % n x 1 vector
vec2 = [ stuff ]; % m x 1 vector
result = bsxfun('times', vec1.', vec2);
This will give you matrix called result wherein the (i, j) entry will be the ith element of vec1 multiplied by the jth element of vec2.
You can use bsxfun for all sorts of built-in functions, and you can declare your own. The documentation has a list of many built-in functions, but basically you can name any function that accepts two arrays (vector or matrix) as arguments and get it to work.
I like splitapply, which allows a function to be applied to the columns of A using splitapply(fun,A,1:size(A,2)).
For example
A = magic(5);
B = splitapply(#(x) x+1, A, 1:size(A,2));
C = splitapply(#std, A, 1:size(A,2));
To apply the function to the rows, you could use
splitapply(fun, A', 1:size(A,1))';
(My source for this solution is here.)
Stumbled upon this question/answer while seeking how to compute the row sums of a matrix.
I would just like to add that Matlab's SUM function actually has support for summing for a given dimension, i.e a standard matrix with two dimensions.
So to calculate the column sums do:
colsum = sum(M) % or sum(M, 1)
and for the row sums, simply do
rowsum = sum(M, 2)
My bet is that this is faster than both programming a for loop and converting to cells :)
All this can be found in the matlab help for SUM.
if you know the length of your rows you can make something like this:
a=rand(9,3);
b=rand(9,3);
arrayfun(#(x1,x2,y1,y2,z1,z2) line([x1,x2],[y1,y2],[z1,z2]) , a(:,1),b(:,1),a(:,2),b(:,2),a(:,3),b(:,3) )