I have two matrices A and B with dimensions [a b c] and [b a c] respectively. I would like to do the following:
C = zeros([a a c]);
for i = 1 : c
C(:,:,i) = A(:,:,i) * B(:,:,i);
end
without using any for loop. In other words, via some in-built function, or maybe some kind of vector manipulation.
Is this possible?
I think your computation should already be reasonably fast. You could however try the mtimesx submission by James Tursa on the File Exchange, that is described as:
MTIMESX is a fast general purpose matrix and scalar multiply routine that has the following features:
Supports multi-dimensional (nD, n>2) arrays directly
[...]
Can meet or beat MATLAB for speed in most cases
If you are dealing with lots of matrices of small size, you might find that vectorizing different dimensions could also speed things up:
C = zeros(a,a,c);
for j = 1:size(A,2)
C = C + bsxfun(#times, A(:,j,:), B(j,:,:));
end
Related
I've been trying to vectorize a for loop in Matlab because it bottlenecks the whole program but I have no clue about how to do it when iterating on different rows or columns of a single matrix inside of the loop. Here's the code :
// X is an n*k matrix, W is n*n
// W(i,j) stores the norm of the vector resulting from subtracting
// the j-th line of X from its i-th line.
for i = 1:n
for j = 1:n
W(i,j) = norm(X(i,:) - X(j,:))
end
end
Edit :
I chose Luis Mendo's answer as it was the most convenient for me and the one that's closer to the mathematical concept behind my algorithm, yet all three answers were correct and I'd advise to use the one that's the most convenient depending on which Matlab Toolboxes you have or the way you want to code.
I also noticed that the common point of all answers was to use a different format, for example : using an array that would store the indices, reshaping current arrays, using one more dimension...
So I think that's the right thing to explore if you have a similar problem.
Using combinatory:
% sample data
X = randi(9,5,4);
n = size(X,1);
% row index combinations
combIdx = combvec(1:n,1:n);
% difference between row combinations
D = X(combIdx(1,:),:)-X(combIdx(2,:),:);
% norm of each row
W = diag(sqrt(D*D'));
% reshape
W = reshape(W,n,[]);
Here is another solution that may be more efficient for big matrices.
For Matlab >=R2016 you can simple use
W = sqrt(sum(abs(reshape(X,n,1,k) - reshape(X,1,n,k)).^2,3))
(if your array is real-valued you can also skip the abs). For earlier versions of Matlab you need to add some repmat magic, namely:
W = sqrt(sum(abs(repmat(reshape(X,n,1,k),[1,n,1]) - repmat(reshape(X,1,n,k),[n,1,1])).^2,3));
P.S.: R2017b might make it even more convenient, at least the release notes mention a function called vecnorm that could replace the somewhat ugly sqrt(sum(abs(.).^2))) thing. But the documentation is not up yet so I don't know what exactly it will do. "
If you have the Statistics Toolbox, use pdist:
W = squareform(pdist(X));
No toolbox: good old bsxfun:
W = sqrt(sum(bsxfun(#minus, permute(X, [1 3 2]), permute(X, [3 1 2])).^2, 3));
Does Matlab do a full matrix multiplication when a matrix multiplication is given as an argument to the trace function?
For example, in the code below, does A*B actually happen, or are the columns of B dotted with the rows of A, then summed? Or does something else happen?
A = [2,2;2,2];
B = eye(2);
f = trace(A*B);
Yes, MATLAB calculates the product, but you can avoid it!
First, let's see what MATLAB does if you do f = trace(A*B):
I think the picture from my Performance monitor says it all really. The first bump is when I created a large A = 2*ones(n), the second, very little bump is for the creation of B = eye(n), and the last bump is where f = trace(A*B) is calculated.
Now, let's see that you get if you do it manually:
If you do it manually, you can save a lot of memory, and it's much faster.
tic
n = 6e3;
A = rand(n);
B = rand(n);
f = trace(A*B);
toc
pause(10)
tic
C(n) = 0;
for ii = 1:n
C(ii) = sum(A(ii,:)*B(:,ii));
end
g = sum(C);
toc
abs(f-g) < 1e-10
Elapsed time is 11.982804 seconds.
Elapsed time is 0.540285 seconds.
ans =
1
Now, as you asked about in the comments: "Is this still true if you use it in a function where optimization can kick in?"
This depends on what you mean here, but as a quick example:
Calculating x = inv(A)*b can be done in a few different ways. If you do:
x = A\b;
MATLAB will chose an algorithm that's best suited for your particular matrix/vector. There are many different alternatives here, depending on the structure of the matrix: is it triangular, hermatian, sparse...? Often it's a upper/lower triangulation. I can pretty much guarantee you that you can't write a code in MATLAB that can outperform MATLABs builtin functions here.
However, if you calculate the same thing this way:
x = inv(A)*b;
MATLAB will actually calculate the inverse of A, then multiply it by b, even though the inverse is not stored in the workspace afterwards. This is much slower, and can also be inaccurate. (In the A\b approach, MATLAB will, if necessary create a permutation matrix to ensure numerical stability.
I have a cell array myBasis of sparse matricies B_1,...,B_n.
I want to evaluate with Matlab the matrix Q(i,j) = trace (B^T_i * B_j).
Therefore, I wrote the following code:
for i=1:n
for j=1:n
B=myBasis{i};
C=myBasis{j};
Q(i,j)=trace(B'*C);
end
end
Which takes already 68 seconds when n=1226 and B_i has 50 rows, and 50 colums.
Is there any chance to speed this up? Usually I exclude for-loops from my matlab code in a c++ file - but I have no experience how to handle a sparse cell array in C++.
As noted by Inox Q is symmetric and therefore you only need to explicitly compute half the entries.
Computing trace( B.'*C ) is equivalent to B(:).'*C(:):
trace(B.'*C) = sum_i [B.'*C]_ii = sum_i sum_j B_ij * C_ij
which is the sum of element-wise products and therefore equivalent to B(:).'*C(:).
When explicitly computing trace( B.'*C ) you are actually pre-computing all k-by-k entries of B.'*C only to use the diagonal later on. AFAIK, Matlab does not optimize its calculation to save it from computing all the entries.
Here's a way
for ii = 1:n
B = myBasis{ii};
for jj = ii:n
C = myBasis{jj};
t = full( B(:).'*C(:) ); % equivalent to trace(B'*C)!
Q(ii,jj) = t;
Q(jj,ii) = t;
end
end
PS,
It is best not to use i and j as variable names in Matlab.
PPS,
You should notice that ' operator in Matlab is not matrix transpose, but hermitian conjugate, for actual transpose you need to use .'. In most cases complex numbers are not involved and there is no difference between the two operators, but once complex data is introduced, confusing between the two operators makes debugging quite a mess...
Well, a couple of thoughts
1) Basic stuff: A'*B = (B'*A)' and trace(A) = trace(A'). Well, only this trick cut your calculations by almost 50%. Your Q(i,j) matrix is symmetric, and you only need to calculate n(n+1)/2 terms (and not n²)
2) To calculate the trace you don't need to calculate every term of B'*C, just the diagonal. Nevertheless, I don't know if it's easy to create a script in Matlab that is actually faster then just calculating B'*C (MatLab is pretty fast with matrix operations).
But I would definitely implement (1)
What's the best way to do the following (in Matlab) if I have two matrices A and B, let'say both of size m-by-n:
C = zeros(m,m);
for t=1:n
C=C+A(:,t)*B(:,t)';
end
This is nothing more than
C = A*B';
where A and B are each m-by-n. I'm not sure that you're going to get more efficient than that unless the matrices have special properties.
One place where you might get a benefit from using bsxfun for matrix multiplication is when the dimensions are sufficiently large (probably 100-by-100 or more) and one matrix is diagonal, e.g.:
A = rand(1e2);
B = diag(rand(1,1e2));
C = bsxfun(#times,A,diag(B).');
This occurs in many matrix transforms – see the code for sqrtm for example (edit sqrtm).
This is the equation I am trying to solve:
h = (X'*X)^-1*X'*y
where X is a matrix and y is a vector ((X'X)^-1 is the inverse of X-transpose times X). I have coded this in Matlab as:
h = (X'*X)\X'*y
which I believe is correct. The problem is that X is around 10000x10000, and trying to calculate that inverse is crashing Matlab on even the most powerful computer I can find (16 cores, 24GB RAM). Is there any way to split this up, or a library designed for doing such large inversions?
Thank you.
That looks like a pseudo inverse. Are you perhaps looking for just
h = X \ y;
I generated a random 10,000 by 10,000 matrix X and a random 10,000 by 1 vector y.
I just broke up my computation step by step. (Code shown below)
Computed the transpose and held it in matrix K
Then I computed Matrix A by multiplying K by X
Computed vector b by multiplying K by vector y
Lastly, I used the backslash operator on A and b to solve
I didn't have a problem with the computation. It took a while, but breaking up the operations into the smallest groups possible helped to prevent the computer from being overwhelmed. However, it could be the composition of the matrix that you are using (ie. Sparse, decimals, etc.).
X = randi(2000, [10000, 10000]);
y = randi(2000, 10000, 1);
K = X';
A = K*X;
b = K*y;
S = A\b;
If you have multiple machines at your disposal, and you can recast your problem into the form h = X\y as proposed by #Ben, then you could use distributed arrays. This demo shows how you can do that.
Jordan,
Your equation is exactly the definition for "Moore-Penrose Matrix Inverse".
Check: http://mathworld.wolfram.com/Moore-PenroseMatrixInverse.html
Directly using h = X \ y; should help.
Or check Matlab pinv(X)*y