I want to convert a sparse matrix in matlab to single precision, however it appears that matlab doesn't have single sparse implemented.
Instead of that, I am just planning on checking it the values are outside of the single precision range and rounding them off to the highest and lowest values of the single precision range.
I'd like to do something like this:
for i = 1:rows
for j = 1:cols
if (abs(A(i,j) < 2^-126))
A(i,j) == 0;
end
end
end
However, ths is extremely slow. Is there another command I can use that will work on sparse matrix class type in MATLAB? I notice most commands don't work for the sparse data type.
EDIT 1:
I also tried the following, but as you can see I run out of memory (the matrix is sparse and is 200K x 200K with ~3 million nonzeros):
A(A < 2^-126) = 0
Error using <
Out of memory. Type HELP MEMORY for your options.
EDIT 2:
Current solution I developed based on input from #rahnema1:
% Convert entries that aren't in single precision
idx = find(A); % find locations of nonzeros
idx2 = find(abs(A(idx)) < double(realmin('single')));
A(idx(idx2)) = sign(A(idx(idx2)))*double(realmin('single'));
idx3 = find(abs(Problem.A(idx)) > double(realmax('single')));
A(idx(idx3)) = sign(A(idx(idx3)))*double(realmax('single'));
You can find indices of non zero elements and use that to change the matrix;
idx = find(A);
Anz = A(idx);
idx = idx(Anz < 2^-126);
A(idx) = 0;
Or more compact:
idx = find(A);
A(idx(A(idx) < 2^-126)) = 0;
However if you want to convert from double to single you can use single function:
idx = find(A);
A(idx) = double(single(full(A(idx))));
or
A(find(A)) = double(single(nonzeros(A)));
To convert Inf to realmax you can write:
A(find(A)) = double(max(-realmax('single'),min(realmax('single'),single(nonzeros(A)))));
If you only want to convert Inf to realmax you can do:
Anz = nonzeros(A);
AInf = isinf(A);
Anz(AInf) = double(realmax('single')) * sign(Anz(AInf));
A(find(A)) = Anz;
As one still reaches this thread when interested in how to convert sparse to single: an easy answer is to use full(). However, this can be problematic if you rely on the memory savings that sparse gives you, as it basically converts to double first.
sparseMatrix = sparse(ones(2, 'double'));
single(full(sparseMatrix)) % works
% ans =
% 2×2 single matrix
% 1 1
% 1 1
single(sparseMatrix) % doesn't work
% Error using single
% Attempt to convert to unimplemented sparse type
Related
I've written a function that generates a sparse matrix of size nxd
and puts in each column 2 non-zero values.
function [M] = generateSparse(n,d)
M = sparse(d,n);
sz = size(M);
nnzs = 2;
val = ceil(rand(nnzs,n));
inds = zeros(nnzs,d);
for i=1:n
ind = randperm(d,nnzs);
inds(:,i) = ind;
end
points = (1:n);
nnzInds = zeros(nnzs,d);
for i=1:nnzs
nnzInd = sub2ind(sz, inds(i,:), points);
nnzInds(i,:) = nnzInd;
end
M(nnzInds) = val;
end
However, I'd like to be able to give the function another parameter num-nnz which will make it choose randomly num-nnz cells and put there 1.
I can't use sprand as it requires density and I need the number of non-zero entries to be in-dependable from the matrix size. And giving a density is basically dependable of the matrix size.
I am a bit confused on how to pick the indices and fill them... I did with a loop which is extremely costly and would appreciate help.
EDIT:
Everything has to be sparse. A big enough matrix will crash in memory if I don't do it in a sparse way.
You seem close!
You could pick num_nnz random (unique) integers between 1 and the number of elements in the matrix, then assign the value 1 to the indices in those elements.
To pick the random unique integers, use randperm. To get the number of elements in the matrix use numel.
M = sparse(d, n); % create dxn sparse matrix
num_nnz = 10; % number of non-zero elements
idx = randperm(numel(M), num_nnz); % get unique random indices
M(idx) = 1; % Assign 1 to those indices
Is there an alternative to randi, I need unique integer values. Using randi the PianoSperimentale matrix may contain repeated integer values.
lover_bound = 10;
upper_bound = 180;
steps = 10;
NumeroCestelli = 8;
livello = [lover_bound:steps:upper_bound];
L = length(livello);
n_c = 500 000
NumeroCestelli = 8
randIdxs = randi([1,L],n_c,NumeroCestelli);
PianoSperimentale = single(livello(randIdxs));
The alternative needs to be fast and support very large matrix. In the past i was using this:
[PianoSperimentale] = combinator(L,NumeroCestelli,'c','r');
for i=1:L
PianoSperimentale(PianoSperimentale==i)=livello(i);
end
but is too slow and painfull. (see Combinator)
Yes, there is:
randsample(10,3)
gives a vector of 3 integers taken from 1 to 10, without replacement.
If you need a matrix instead of a vector:
matrix = NaN(8,12);
matrix(:) = randsample(1000,numel(matrix));
gives an 8x12 matrix of unique integers taken from 1 to 1000.
The function randsample is in the Statistics Toolbox. If you don't have it you can use randperm instead, as noted by #RodyOldenhuis and #Dan (see #Dan's answer).
Also if you don't have the stats toolbox then you could use randperm:
randperm(m, k) + n - 1
This will also give you k random integers between n and n+m without replacement
Suppose I want to find the size of a matrix, but can't use any functions such as size, numel, and length. Are there any neat ways to do this? I can think of a few versions using loops, such as the one below, but is it possible to do this without loops?
function sz = find_size(m)
sz = [0, 0]
for ii = m' %' or m(1,:) (probably faster)
sz(1) = sz(1) + 1;
end
for ii = m %' or m(:,1)'
sz(2) = sz(2) + 1;
end
end
And for the record: This is not a homework, it's out of curiosity. Although the solutions to this question would never be useful in this context, it is possible that they provide new knowledge in terms of how certain functions/techniques can be used.
Here is a more generic solution
function sz = find_size(m)
sz = [];
m(f(end), f(end));
function r = f(e)
r=[];
sz=[sz e];
end
end
Which
Works for arrays, cell arrays and arrays of objects
Its time complexity is constant and independent of matrix size
Does not use any MATLAB functions
Is easy to adapt to higher dimensions
For non-empty matrices you can use:
sz = [sum(m(:,1)|1) sum(m(1,:)|1)];
But to cover empty matrices we need more function calls
sz = sqrt([sum(sum(m*m'|1)) sum(sum(m'*m|1))]);
or more lines
n=m&0;
n(end+1,end+1)=1;
[I,J]=find(n);
sz=[I,J]-1;
Which both work fine for m=zeros(0,0), m=zeros(0,10) and m=zeros(10,0).
Incremental indexing and a try-catch statement works:
function sz = find_size(m)
sz = [0 0];
isError = false;
while ~isError
try
b = m(sz(1) + 1, :);
sz(1) = sz(1) + 1;
catch
isError = true;
end
end
isError = false;
while ~isError
try
b = m(:, sz(2) + 1);
sz(2) = sz(2) + 1;
catch
isError = true;
end
end
end
A quite general solution is:
[ sum(~sum(m(:,[]),2)) sum(~sum(m([],:),1)) ]
It accepts empty matrices (with 0 columns, 0 rows, or both), as well as complex, NaN or inf values.
It is also very fast: for a 1000 × 1000 matrix it takes about 22 microseconds in my old laptop (a for loop with 1e5 repetitions takes 2.2 seconds, measured with tic, toc).
How this works:
The keys to handling empty matrices in a unified way are:
empty indexing (that is, indexing with []);
the fact that summing along an empty dimension gives zeros.
Let r and c be the (possibly zero) numbers of rows and columns of m. m(:,[]) is an r × 0 empty vector. This holds even if r or c are zero. In addition, this empty indexing automatically provides insensitivity to NaN, inf or complex values in m (and probably accounts for the small computation time as well).
Summing that r × 0 vector along its second dimension (sum(m(:,[]),2)) produces a vector of r × 1 zeros. Negating and summing this vector gives r.
The same procedure is applied for the number of columns, c, by empty-indexing in the first dimension and summing along that dimension.
The find command has a neat option to get the last K elements:
I = find(X,K,'last') returns at most the last K indices corresponding to the nonzero entries of the arrayX`.
To get the size, ask for the last k=1 elements. For example,
>> x=zeros(256,4);
>> [numRows,numCols] = find(x|x==0, 1, 'last')
numRows =
256
numCols =
4
>> numRows0 = size(x,1), numCols0 = size(x,2)
numRows0 =
256
numCols0 =
4
You can use find with the single output argument syntax, which will give you numel:
>> numEl = find(x|x==0, 1, 'last')
numEl =
1024
>> numEl0 = numel(x)
numEl0 =
1024
Another straightforward, but less interesting solution uses whos (thanks for the reminder Navan):
s=whos('x'); s.size
Finally, there is format debug.
I have a column vector of data in variable vdata and a list of indeces idx. I want to access vdata at the indeces x before and x after each index in idx. One way I would do it in a for loop is:
x = 10;
accessed_data = [];
for (ii = 1:length(idx))
accessed_data = vdata(idx-x:idx+x);
end
Is there a way to do this in a vectorized function? I found a solution to a very similar question here: Addressing multiple ranges via indices in a vector but I don't understand the code :(.
Assuming min(idx)-x>0 and max(idx)+x<=numel(vdata) then you can simply do
iidx = bsxfun(#plus, idx(:), -x:x); % create all indices
accessed_data = vdata( iidx );
One scheme that uses direct indexing instead of a for loop:
xx = (-x:x).'; % Range of indices
idxx = bsxfun(#plus,xx(:,ones(1,numel(idx))),idx(:).'); % Build array
idxx = idxx(:); % Columnize to interleave columns
idxx = idxx(idxx>=1&idxx<=length(vdata)); % Make sure the idx+/-x is valid index
accessed_data = vdata(idxx); % Indices of data
The second line can be replaced with a form of the first line from #Shai's answer. This scheme checks that all of the resultant indices are valid. Because some might have to be removed, you could end up with a ragged array. One way to solve this is to use cell arrays, but here I just make idxx a vector, and thus accessed_data is as well.
This gives the solution in a matrix, with one row for each value in idx. It assumes that all values in idx are greater than or equal to x, and less than or equal to length(vdata)-x.
% Data
x = 10;
idx = [12 20 15];
vdata = 1:100;
ind = repmat(-x:x,length(idx),1) + repmat(idx(:),1,2*x+1);
vdata(ind)
I have a non-fixed dimensional matrix M, from which I want to access a single element.
The element's indices are contained in a vector J.
So for example:
M = rand(6,4,8,2);
J = [5 2 7 1];
output = M(5,2,7,1)
This time M has 4 dimensions, but this is not known in advance. This is dependent on the setup of the algorithm I'm writing. It could likewise be that
M = rand(6,4);
J = [3 1];
output = M(3,1)
so I can't simply use
output=M(J(1),J(2))
I was thinking of using sub2ind, but this also needs its variables comma separated..
#gnovice
this works, but I intend to use this kind of element extraction from the matrix M quite a lot. So if I have to create a temporary variable cellJ every time I access M, wouldn't this tremendously slow down the computation??
I could also write a separate function
function x= getM(M,J)
x=M(J(1),J(2));
% M doesn't change in this function, so no mem copy needed = passed by reference
end
and adapt this for different configurations of the algorithm. This is of course a speed vs flexibility consideration which I hadn't included in my question..
BUT: this is only available for getting the element, for setting there is no other way than actually using the indices (and preferably the linear index). I still think sub2ind is an option. The final result I had intended was something like:
function idx = getLinearIdx(J, size_M)
idx = ...
end
RESULTS:
function lin_idx = Lidx_ml( J, M )%#eml
%LIDX_ML converts an array of indices J for a multidimensional array M to
%linear indices, directly useable on M
%
% INPUT
% J NxP matrix containing P sets of N indices
% M A example matrix, with same size as on which the indices in J
% will be applicable.
%
% OUTPUT
% lin_idx Px1 array of linear indices
%
% method 1
%lin_idx = zeros(size(J,2),1);
%for ii = 1:size(J,2)
% cellJ = num2cell(J(:,ii));
% lin_idx(ii) = sub2ind(size(M),cellJ{:});
%end
% method 2
sizeM = size(M);
J(2:end,:) = J(2:end,:)-1;
lin_idx = cumprod([1 sizeM(1:end-1)])*J;
end
method 2 is 20 (small number of index sets (=P) to convert) to 80 (large number of index sets (=P)) times faster than method 1. easy choice
For the general case where J can be any length (which I assume always matches the number of dimensions in M), there are a couple options you have:
You can place each entry of J in a cell of a cell array using the num2cell function, then create a comma-separated list from this cell array using the colon operator:
cellJ = num2cell(J);
output = M(cellJ{:});
You can sidestep the sub2ind function and compute the linear index yourself with a little bit of math:
sizeM = size(M);
index = cumprod([1 sizeM(1:end-1)]) * (J(:) - [0; ones(numel(J)-1, 1)]);
output = M(index);
Here is a version of gnovices option 2) which allows to process a whole matrix of subscripts, where each row contains one subscript. E.g for 3 subscripts:
J = [5 2 7 1
1 5 2 7
4 3 9 2];
sizeM = size(M);
idx = cumprod([1 sizeX(1:end-1)])*(J - [zeros(size(J,1),1) ones(size(J,1),size(J,2)-1)]).';