Suppose there are two matrices of the same size, and I want to calculate the summation of their column-wise kronecker product. Due to sometimes the column size is quite large so that the speed could be very slow. Thus, is there anyway to vectorize this function or any function may help reducing the complexity in matlab? Thanks in advance.
The corresponding matlab code with a for-loop is provided below, and the answer of d is the interested output:
A = rand(3,7);
B = rand(3,7);
d = zeros(size(A,1)*size(B,1),1);
for i=1:size(A,2)
d = d + kron(A(:,i),B(:,i));
end
Using the rewriting of the Kronecker product given by Daniels answer
e=zeros(size(B,1),size(A,1));
for i=1:size(A,2)
e = e + B(:,i)*A(:,i).';
end
e=reshape(e,[],1);
we say that
C = A'
and thus
for i=1:m
e = e + B(:,i)*C(i,:);
end
which is the definition of the matrix product
B*C.
In conclusion the problem can thus be solved by the simple matrix product
d = reshape(B*A',[],1);
The kronecker product of two vectors is just a reshaped result of the matrix multiplication of both vectors:
e=zeros(size(B,1),size(A,1));
for i=1:size(A,2)
e = e + B(:,i)*A(:,i).';
end
e=reshape(e,[],1);
Now knowing that it's just a sum of products, it can be put into a single line using bsxfun
f=reshape(sum(bsxfun(#times,permute(B,[1,3,2]),permute(A,[3,1,2])),3),[],1);
Depending on the input, the bsxfun-sulution is slightly faster than the matrix multiplication, but that comes with a high memory consumption. The bsxfun-solution uses O(size(A,1)*size(B,1)*size(B,2)) while the for-loop only uses O(size(A,1)*size(B,1)) in addition to the input arguments.
Related
So I have the following matrices:
A = [1 2 3; 4 5 6];
B = [0.5 2 3];
I'm writing a function in MATLAB that will allow me to multiply a vector and a matrix by element as long as the number of elements in the vector matches the number of columns. In A there are 3 columns:
1 2 3
4 5 6
B also has 3 elements so this should work. I'm trying to produce the following output based on A and B:
0.5 4 9
2 10 18
My code is below. Does anyone know what I'm doing wrong?
function C = lab11(mat, vec)
C = zeros(2,3);
[a, b] = size(mat);
[c, d] = size(vec);
for i = 1:a
for k = 1:b
for j = 1
C(i,k) = C(i,k) + A(i,j) * B(j,k);
end
end
end
end
MATLAB already has functionality to do this in the bsxfun function. bsxfun will take two matrices and duplicate singleton dimensions until the matrices are the same size, then perform a binary operation on the two matrices. So, for your example, you would simply do the following:
C = bsxfun(#times,mat,vec);
Referencing MrAzzaman, bsxfun is the way to go with this. However, judging from your function name, this looks like it's homework, and so let's stick with what you have originally. As such, you need to only write two for loops. You would use the second for loop to index into both the vector and the columns of the matrix at the same time. The outer most for loop would access the rows of the matrix. In addition, you are referencing A and B, which are variables that don't exist in your code. You are also initializing the output matrix C to be 2 x 3 always. You want this to be the same size as mat. I also removed your checking of the length of the vector because you weren't doing anything with the result.
As such:
function C = lab11(mat, vec)
[a, b] = size(mat);
C = zeros(a,b);
for i = 1:a
for k = 1:b
C(i,k) = mat(i,k) * vec(k);
end
end
end
Take special note at what I did. The outer-most for loop accesses the rows of mat, while the inner-most loop accesses the columns of mat as well as the elements of vec. Bear in mind that the number of columns of mat need to be the same as the number of elements in vec. You should probably check for this in your code.
If you don't like using the bsxfun approach, one alternative is to take the vector vec and make a matrix out of this that is the same size as mat by stacking the vector vec on top of itself for as many times as we have rows in mat. After this, you can do element-by-element multiplication. You can do this stacking by using repmat which repeats a vector or matrices a given number of times in any dimension(s) you want. As such, your function would be simplified to:
function C = lab11(mat, vec)
rows = size(mat, 1);
vec_mat = repmat(vec, rows, 1);
C = mat .* vec_mat;
end
However, I would personally go with the bsxfun route. bsxfun basically does what the repmat paradigm does under the hood. Internally, it ensures that both of your inputs have the same size. If it doesn't, it replicates the smaller array / matrix until it is the same size as the larger array / matrix, then applies an element-by-element operation to the corresponding elements in both variables. bsxfun stands for Binary Singleton EXpansion FUNction, which is a fancy way of saying exactly what I just talked about.
Therefore, your function is further simplified to:
function C = lab11(mat, vec)
C = bsxfun(#times, mat, vec);
end
Good luck!
I have to calculate Newey-West standard errors for large multiple regression models.
The final step of this calculation is to obtain
nwse = sqrt(diag(N.*inv(X'*X)*Q*inv(X'*X)));
This file exchange contribution implements this as
nwse = sqrt(diag(N.*((X'*X)\Q/(X'*X))));
This looks reasonable, but in my case (5000x5000 sparse Q and X'*X) it's far too slow for my needs (about 30secs, I have to repeat this for about one million different models). Any ideas how to make this line faster?
Please note that I need only the diagonal, not the entire matrix and that both Q and (X'*X) are positive-definite.
I believe you can save a lot of computation time by explicitly doing an LU factorization, [l, u, p, q] = lu(X'*X); and use those factors when doing the calculations. Also, since X are constant for about 100 models, pre-calculating X'*X will most likely save you some time.
Note that in your case, the most time demanding operation might very well be the sqrt-function.
% Constant for every 100 models or so:
M = X'*X;
[l, u, p, q] = lu(M);
% Now, I guess this should be quite a bit faster (I might have messed up the order):
nwse = sqrt(diag(N.*(q * ( u \ (l \ (p * Q))) * q * (u \ (l \ p)))));
The first two terms are commonly used:
l the lower triangular matrix
u the upper triangular matrix
Now, p and q are a bit more uncommon.
p is a row permutation matrix used to obtain numerical stability. There is not much performance gain in doing [l, u, p] = lu(M) compared to [l, u] = lu(M) for sparse matrices.
q however offers a significant performance gain. q is a column permutation matrix that is used to reduce the amount of fill when doing the factorization.
Note that the [l, u, p, q] = lu(M) is only a valid syntax for sparse matrices.
As for why using full pivoting as described above should be faster:
Try the following to see the purpose of the column permutation matrix q. It is easier to work with elements that are aligned around the diagonal.
S = sprand(100,100,0.01);
[l, u, p] = lu(S);
spy(l)
figure
spy(u)
Now, compare it with this:
[ll, uu, pp, qq] = lu(S);
spy(ll);
figure
spy(uu);
Unfortunately, I don't have MATLAB here right now, so I can't guarantee that I put all the arguments in the correct order, but I think it's correct.
Following the helpful answer of Robert_P and the comments of Parag, I found the following to be the fastest for my particular large-scale sparse data:
L=chol(X'*X,'lower'); L=full(L);
invXtX = L'\(L\ speye(size(X,2)));
nwse = sqrt(N.*sum(invXtX.*(Q*invXtX)));
The last line computes the diagonal efficiently, idea taken from here.
I need to multiply a matrix A with n matrices, and get n matrices back. For example, multiply a 2x2 matrix with 3 2x2 matrices stacked as a 2x2x3 Matlab array. bsxfun is what I usually use for such situations, but it only applies for element-wise operations.
I could do something like:
blkdiag(a, a, a) * blkdiag(b(:,:,1), b(:,:,2), b(:,:,3))
but I need a solution for arbitrary n - ?
You can reshape the stacked matrices. Suppose you have k-by-k matrix a and a stack of m k-by-k matrices sb and you want the product a*sb(:,:,ii) for ii = 1..m. Then all you need is
sza = size(a);
b = reshape( b, sza(2), [] ); % concatenate all matrices aloong the second dim
res = a * b;
res = reshape( res, sza(1), [], size(sb,3) ); % stack back to 3d
Your solution can be adapted to arbitrary size using comma-saparated lists obtained from cell arrays:
[k m n] = size(B);
Acell = mat2cell(repmat(A,[1 1 n]),k,m,ones(1,n));
Bcell = mat2cell(B,k,m,ones(1,n));
blkdiag(Acell{:}) * blkdiag(Bcell{:});
You could then stack the blocks on a 3D array using this answer, and keep only the relevant ones.
But in this case a good old loop is probably faster:
C = NaN(size(B));
for nn = 1:n
C(:,:,nn) = A * B(:,:,nn);
end
For large stacks of matrices and/or vectors over which to execute matrix multiplication, speed can start becoming an issue. To avoid re-inventing the wheel, you could simply compile and use the following fast MEX code:
MTIMESX - Mathworks.
As a rule of thumb, MATLAB is often quite inefficient at executing for loops over large numbers of operations which look like they should be vectorizable; I cannot think of a straightforward way of generalising Shai's answer to this case.
I want a code the below code more efficient timewise. preferably without a loop.
arguments:
t % time values vector
t_index = c % one of the possible indices ranging from 1:length(t).
A % a MXN array where M = length(t)
B % a 1XN array
code:
m = 1;
for k = t_index:length(t)
A(k,1:(end-m+1)) = A(k,1:(end-m+1)) + B(m:end);
m = m + 1;
end
Many thanks.
I'd built from B a matrix of size NxM (call it B2), with zeros in the right places and a triangular from according to the conditions and then all you need to do is A+B2.
something like this:
N=size(A,2);
B2=zeros(size(A));
k=c:length(t);
B2(k(1):k(N),:)=hankel(B)
ans=A+B2;
Note, the fact that it is "vectorized" doesn't mean it is faster these days. Matlab's JIT makes for loops comparable and sometimes faster than built-in vectorized options.
I wish to compute the cumulative cosine distance between sets of vectors.
The natural representation of a set of vectors is a matrix...but how do I vectorize the following?
function d = cosdist(P1,P2)
ds = zeros(size(P1,2),1);
for k=1:size(P1,2)
%#used transpose() to avoid SO formatting on '
ds(k)=transpose(P1(:,k))*P2(:,k)/(norm(P1(:,k))*norm(P2(:,k)));
end
d = prod(ds);
end
I can of course write
fz = #(v1,v2) transpose(v1)*v2/(norm(v1)*norm(v2));
ds = cellfun(fz,P1,P2);
...so long as I recast my matrices as cell arrays of vectors. Is there a better / entirely numeric way?
Also, will cellfun, arrayfun, etc. take advantage of vector instructions and/or multithreading?
Note probably superfluous in present company but for column vectors v1'*v2 == dot(v1,v2) and is significantly faster in Matlab.
Since P1 and P2 are of the same size, you can do element-wise operations here. v1'*v equals sum(v1.*v2), by the way.
d = prod(sum(P1.*P2,1)./sqrt(sum(P1.^2,1) .* sum(P2.^2,1)));
#Jonas had the right idea, but the normalizing denominator might be incorrect. Try this instead:
%# matrix of column vectors
P1 = rand(5,8);
P2 = rand(5,8);
d = prod( sum(P1.*P2,1) ./ sqrt(sum(P1.^2,1).*sum(P2.^2,1)) );
You can compare this against the results returned by PDIST2 function:
%# PDIST2 returns one minus cosine distance between all pairs of vectors
d2 = prod( 1-diag(pdist2(P1',P2','cosine')) );