I have N kx1 sparse vectors and I need to multiply each of them by their transpose, creating N square matrices, which I then have to sum over. The desired output is a k by k matrix. I have tried doing this in a loop and using arrayfun, but both solutions are too slow. Perhaps one of you can come up with something faster. Below are specific details about the best solution I've come up with.
mdev_big is k by N sparse matrix, containing each of the N vectors.
fun_sigma_i = #(i) mdev_big(:,i)*mdev_big(:,i)';
sigma_i = arrayfun(fun_sigma_i,1:N,'UniformOutput',false);
sigma = sum(reshape(full([sigma_i{:}]),k,k,N),3);
The slow part of this process is making sigma_i full, but I cannot reshape it into a 3d array otherwise. I've also tried cat instead of reshape (slower), ndSparse instead of full (way slower), and making fun_sigma_i return a full matrix rather than a sparse one (slower).
Thanks for the help! ,
Related
I'd love to know if there is a more efficient way to multiply specific elements of multi-dimensional matrices that doesn't require a 'for' loop.
I have a region * time matrix for an individual (say, 50 regions and 1000 timepoints) and I want to multiply each pair of regions at each timepoint to create a new matrix of the products of each region pair at each time point (50 x 50 x 1000). The way that I'm currently running it is:
for t = 1:1000
for i = 1:50
for j = 1:50
new(i,j,t) = old(i,t) .* old(j,t)
As I'm sure you can imagine, this is super slow. Any ideas on how i can fix it up so that it will run more quickly?
%some example data easy to trace
old=[1:5]'
old(:,2)=old*i
%multiplicatiion
a=permute(old,[1,3,2])
b=permute(old,[3,1,2])
bsxfun(#times,a,b)
permute is used to make 3d-matrices with dimensions n*1*m and 1*n*m out of the n*m input matrix. Changing the dimensions this way, new(i,j,k) can be calculated using new(i,j,k)=a(i,1,k)*b(1,j,k). Applying such operations element-by-element is what bsxfun was designed for.
Regarding bsxfun, try to understand simple 2d-examples like bsxfun(#times,[1:7],[1,10,100]') first
I want to multiply every column of a M × N matrix by corresponding element of a vector of size N.
I know it's possible using a for loop. But I'm seeking a more simple way of doing it.
I think this is what you want:
mat1=randi(10,[4 5]);
vec1=randi(10,[1 5]);
result=mat1.*repmat(vec1,[size(mat1,1),1]);
rempat will replicate vec1 along the rows of mat1. Then we can do element-wise multiplication (.*) to "multiply every column of a M × N matrix by corresponding element of a vector of size N".
Edit: Just to add to the computational aspect. There is an alternative to repmat that I would like you to know. Matrix indexing can achieve the same behavior as repmat and be faster. I have adopted this technique from here.
Observe that you can write the following statement
repmat(vec1,[size(mat1,1),1]);
as
vec1([1:size(vec1,1)]'*ones(1,size(mat1,1)),:);
If you see closely, the expression boils down to vec1([1]'*[1 1 1 1]),:); which is again:
vec1([1 1 1 1]),:);
thereby achieving the same behavior as repmat and be faster. I ran three solutions 100000 times, namely,
Solution using repmat : 0.824518 seconds
Solution using indexing technique explained above : 0.734435 seconds
Solution using bsxfun provided by #LuisMendo : 0.683331 seconds
You can observe that bsxfun is slightly faster.
Although you can do it with repmat (as in #Parag's answer), it's often more efficient to use bsxfun. It also has the advantage that the code (last line) is the same for a row and for a column vector.
%// Example data
M = 4;
N = 5;
matrix = rand(M,N);
vector = rand(1,N); %// or size M,1
%// Computation
result = bsxfun(#times, matrix, vector); %// bsxfun does an "implicit" repmat
I have to perform this operation:
N = A'*P*A
The structure of the P matrix is block diagonal while the A matrix is largely sparse (also in a banded structure). The multiplication is performed in blocks. But the problem is storage.
The N matrix is too huge to store in full (out of memory when trying to allocate). So, I want to store in a sparse fashion. While the sparse command generates only the values in row,column format, can it be applied to store banded matrices with the row column as the index of the block?
I have tried spalloc given in the this question but it hasnt helped storing the row and index of the block.
Thank you.
Image for A P A' formation
The problem lies in the blocks. The blocks are themselves sparse. So is it possible to make blocks as sparse matrices themselves while saving.
So, if a block has a row = 1 and col = 1, then can this be done?
N(row,col) = sparse(A'*P*A)
There may be some additional tricks to play but the first thing to try is to make sure the full matrix N is never created in memory. The immediate problem is that if you call sparse(A'*P*A) then you multiple A'*P then (A'*P)*A and only then do you make it sparse and take out the zeros. Right before making it sparse, the entire non-sparse matrix representation of N is in memory. To force MATLAB to be smarter do the following:
SA = sparse(A);
N = SA'*sparse(P)*SA;
whos N
You should see that N is sparse but, more importantly, each multiplication result is sparse as well because you are multiplying a sparse matrix times a sparse matrix.
I have a vector v of values and a vector r of indices. I want to store the values in a matrix m as follows:
for i = 1:length(v)
m(i, r(i)) = v(i);
end
What is the fastest way to do this in a vectorized way?
I do not know if it is faster, I suppose so but the difference might be very small, but here is one way:
m(sub2ind(size(m),1:length(v),r(1:length(v))))=v;
If r is a column vector then sub2ind will complain about vectors size, you can just take its transpose and it will solve this.
I have to create a matlab matrix that is much bigger that my phisical memory, and i want to take advantage of the sparsity.
This matrix is really really sparse [say N elements in an NxN matrix], and my ram is enought for this. I create the matrix in this way:
A=sparse(zeros(N));
but it goes out of memory.
Do you know the right way to create this matrix?
zeros(N) is creating an NxN matrix, which is not sparse, hence you are running out of memory. Your code is equivalent to
temp = zeros(N)
A = sparse(temp)
Just do sparse(N,N).
Creating an all zeros sparse matrix, and then modifying it is extremely inefficient in matlab.
Instead of doing something like:
A = sparse(N,N) % or even A = sparse([],[],[],N,N,N)
A(1:N,7) = 1:N
It is much more efficient to construct the matrix in triplet form. That is,
construct the column and row indices and the nonzero entries first, then
form the matrix. For example,
i = 1:N;
j = 7*ones(1,N);
x = 1:N;
A = sparse(i,j,x,N,N);
I'd actually recommend the full syntax of sparse([],[],[],N,N,N).
It's useful to preallocate if you know the maximum number of nonzero elements as otherwise you'll get reallocs when you insert new elements.