my Problem is difficult or too simple. So I've been using sparse to creat a matrix from different vectors (example here)
I use sparse(i,j,s,m,n) where i and j are allocations for particles i derived with floor etc.
(it is particle in cell simulation).
this is for 1D. I want to vectorize this problem for 2D and got the problem that i can't creat 3D sparse matrices in octave.
I'm really no pro with octave and i'm searching for a solution that i can put the values from s nicely into a matrix according to the i and j values.
for example: i got particle one in cell x=2 and y=2 than i want to have a matrix with value from particle one in (1,2,2,value). is there a way without much if's and for's?
At the moment is not possible to have N-dimensional sparse matrices in Octave. You can see this with the following example which tries to reshape a 3x9 sparse matrix into 3x3x3:
octave> sp = sparse (2, 6, 1, 3, 9)
sp =
Compressed Column Sparse (rows = 3, cols = 9, nnz = 1 [3.7%])
(2, 6) -> 1
octave> reshape (sp, [3 3 3])
warning: reshape: sparse reshape to N-d array smashes dims
ans =
Compressed Column Sparse (rows = 3, cols = 9, nnz = 1 [3.7%])
(2, 6) -> 1
What you can do is to have a cell array of sparse matrices. So for a particle at KxMxN you could have data{k} = sparse (M, N, value) and access it with data{k}(m,n). It's not ideal but depending on how you get your data organized you may make things more or less readable.
Sparse NDArray is not possible in either Octave nor Matlab. However, there is a class for Matlab which implements Sparse NDArray, although the performance will probably be not optimal, it will still probably be a lot better than using a cell array since it internally represents the sparse NDArray as a simple 2D sparse matrix:
http://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays
Since v4, Octave now supports classdef. I did not try it yet, but even if it does not work yet, it will probably soon. And anyway, you can see how it was done in the script and make a non-class, all functional version by applying the same idea (using a 2D sparse matrix that you manipulate with custom functions to do whatever operations you want in a ND space).
Related
I have a big matrix, 3x3x151.
I was struggling to write a code to find the inverse of each 3x3 matrix and save it in a new matrix.
tried a for loop approach but got no where
This is precisely what pageinv was designed for. If you have R2022a or later, simply:
x = rand(3, 3, 151);
y = pageinv(x);
As the comments above point out, and as is fairly well known, the matrix inverse itself often isn't what you want. There's also pagemldivide if that's what you really want to do.
a quick solution is to split the matrix into cell arrays of 3x3 matrices, and then call cellfun, try this
a=rand(3,3,151);
b=squeeze(num2cell(a,[1,2]));
c=cellfun(#inv, b, 'UniformOutput', false) % inversions of 3x3 matrices
if you want to assemble those back to a matrix, you can call
cmat=reshape(cell2mat(cellfun(#(x) x(:), c', 'UniformOutput', false)), size(a))
I attempted to use the solution from this post: Multiply a 3D matrix with a 2D matrix, to vectorize the multiplication of a one dimensional with a three dimensional, but to no avail. Any help would be greatly appreciated!
for s = 1: s_length
for a = 1: a_length
for g = g_length
two_dim(s,a) = two_dim(s,a) + one_dim(g) * three_dim(s,a,g);
end
end
end
I think this does what you want.
two_dim = reshape(three_dim, [], size(three_dim,3))*one_dim(:);
two_dim = reshape(two_dim, size(three_dim,1), size(three_dim,2));
This works as follows:
First line reshapes your 3D array into a matrix, collapsing the first two dimensions into one. That way the operation you want is standard multiplication of the resulting matrix times a column vector.
Second line reshapes the result into matrix shape.
mtimes does not work with inputs with dimension larger than 2, so you have to find a way to do the multiplication by yourself. The solution of Luis Mendo is a nice solution; here is another one using bsxfun:
two_dim = two_dim + squeeze(sum(bsxfun(#times, three_dim, reshape(one_dim,[1 1 g_length])),3));
Here is how it works:
reshape makes the vector one_dim looking like a 3D array. This must be done because the multiplication between the vector and the 3D array is performed along the 3rd dimension, so Matlab need a hint on sizes.
bsxfun perfoms the element-wise multiplcation, so the result must be sumed up along the 3rd dimension (and squeezed to be compliant with a 2D matrix format)
I need a 3D matrix in matlab, I have another 2D matrix (7570x3) too,
The 3D matrix should have zero number except of all dimensions in 2D matrix that should have 1 value. How can I do that.
i.e. 2D matrix (1,:) = 28,64,27 then 3d(27,64,27) should be 1
how can i do that?
Assuming a is your 2-d matrix, and b is the 3-d one, use sub2ind as follows:
b=false(max(a)); % preallocate memory for a logical zeros matrix b
b(sub2ind(size(b),a(:,1),a(:,2),a(:,3))) = 1;
Check to see what max(a) gives you to see if you can host a 3-d matrix of size(max(a) to begin with. Since you are interested in a logcial matrix (ones and zeros), the size of that matrix in memory is the same as the # of elements, n*m*l, so a 1000x1000x1000 will take 1 GB.
Note, it very well may be that b is very sparse, if that is the case you can refer to this thread to see how to deal with it. Know that at the moment, and to the best of my knowledge, matlab doesn't support 3d sparse matrices. So you may want to check this option from the FEX. When I think of it, you already have a sparse look-up table of the 3D matrix! it is just your 2D matrix you started with...
Thanks a lot #natan
for non-integer matrix also can use:
b=false(floor(max(a))); % preallocate memory for a logical zeros matrix b
b(sub2ind(size(b),floor(a(:,1)),floor(a(:,2)),floor(a(:,3)))) = 1;
or use round function:
b=false(round(max(a))); % preallocate memory for a logical zeros matrix b
b(sub2ind(size(b),round(a(:,1)),round(a(:,2)),round(a(:,3)))) = 1;
I have to perform this operation:
N = A'*P*A
The structure of the P matrix is block diagonal while the A matrix is largely sparse (also in a banded structure). The multiplication is performed in blocks. But the problem is storage.
The N matrix is too huge to store in full (out of memory when trying to allocate). So, I want to store in a sparse fashion. While the sparse command generates only the values in row,column format, can it be applied to store banded matrices with the row column as the index of the block?
I have tried spalloc given in the this question but it hasnt helped storing the row and index of the block.
Thank you.
Image for A P A' formation
The problem lies in the blocks. The blocks are themselves sparse. So is it possible to make blocks as sparse matrices themselves while saving.
So, if a block has a row = 1 and col = 1, then can this be done?
N(row,col) = sparse(A'*P*A)
There may be some additional tricks to play but the first thing to try is to make sure the full matrix N is never created in memory. The immediate problem is that if you call sparse(A'*P*A) then you multiple A'*P then (A'*P)*A and only then do you make it sparse and take out the zeros. Right before making it sparse, the entire non-sparse matrix representation of N is in memory. To force MATLAB to be smarter do the following:
SA = sparse(A);
N = SA'*sparse(P)*SA;
whos N
You should see that N is sparse but, more importantly, each multiplication result is sparse as well because you are multiplying a sparse matrix times a sparse matrix.
I have to create a very big 3D matrix (such as: 500000x60x60). Is there any way to do this in matlab?
When I try
omega = zeros(500000,60,60,'single');
I get an out-of-memory error.
The sparse function is no option since it is only meant for 2D matrices. So is there any alternative to that for higher dimensional matrices?
Matlab only has support for sparse matrices (2D). For 3D tensors/arrays, you'll have to use a workaround. I can think of two:
linear indexing
cell arrays
Linear indexing
You can create a sparse vector like so:
A = spalloc(500000*60*60, 1, 100);
where the last entry (100) refers to the amount of non-zeros eventually to be assigned to A. If you know this amount beforehand it makes memory usage for A more efficient. If you don't know it beforehand just use some number close to it, it'll still work, but A can consume more memory in the end than it strictly needs to.
Then you can refer to elements as if it is a 3D array like so:
A(sub2ind(size(A), i,j,k))
where i, j and k are the indices to the 1st, 2nd and 3rd dimension, respectively.
Cell arrays
Create each 2D page in the 3D tensor/array as a cell array:
a = cellfun(#(x) spalloc(500000, 60, 100), cell(60,1), 'UniformOutput', false);
The same story goes for this last entry into spalloc. Then concatenate in 3D like so:
A = cat(3, a{:});
then you can refer to individual elements like so:
A{i,j,k}
where i, j and k are the indices to the 1st, 2nd and 3rd dimension, respectively.
Since your matrix is sparse, try to use ndsparse (N-dimensional sparse arrays FEX)