Suppose I want to find the size of a matrix, but can't use any functions such as size, numel, and length. Are there any neat ways to do this? I can think of a few versions using loops, such as the one below, but is it possible to do this without loops?
function sz = find_size(m)
sz = [0, 0]
for ii = m' %' or m(1,:) (probably faster)
sz(1) = sz(1) + 1;
end
for ii = m %' or m(:,1)'
sz(2) = sz(2) + 1;
end
end
And for the record: This is not a homework, it's out of curiosity. Although the solutions to this question would never be useful in this context, it is possible that they provide new knowledge in terms of how certain functions/techniques can be used.
Here is a more generic solution
function sz = find_size(m)
sz = [];
m(f(end), f(end));
function r = f(e)
r=[];
sz=[sz e];
end
end
Which
Works for arrays, cell arrays and arrays of objects
Its time complexity is constant and independent of matrix size
Does not use any MATLAB functions
Is easy to adapt to higher dimensions
For non-empty matrices you can use:
sz = [sum(m(:,1)|1) sum(m(1,:)|1)];
But to cover empty matrices we need more function calls
sz = sqrt([sum(sum(m*m'|1)) sum(sum(m'*m|1))]);
or more lines
n=m&0;
n(end+1,end+1)=1;
[I,J]=find(n);
sz=[I,J]-1;
Which both work fine for m=zeros(0,0), m=zeros(0,10) and m=zeros(10,0).
Incremental indexing and a try-catch statement works:
function sz = find_size(m)
sz = [0 0];
isError = false;
while ~isError
try
b = m(sz(1) + 1, :);
sz(1) = sz(1) + 1;
catch
isError = true;
end
end
isError = false;
while ~isError
try
b = m(:, sz(2) + 1);
sz(2) = sz(2) + 1;
catch
isError = true;
end
end
end
A quite general solution is:
[ sum(~sum(m(:,[]),2)) sum(~sum(m([],:),1)) ]
It accepts empty matrices (with 0 columns, 0 rows, or both), as well as complex, NaN or inf values.
It is also very fast: for a 1000 × 1000 matrix it takes about 22 microseconds in my old laptop (a for loop with 1e5 repetitions takes 2.2 seconds, measured with tic, toc).
How this works:
The keys to handling empty matrices in a unified way are:
empty indexing (that is, indexing with []);
the fact that summing along an empty dimension gives zeros.
Let r and c be the (possibly zero) numbers of rows and columns of m. m(:,[]) is an r × 0 empty vector. This holds even if r or c are zero. In addition, this empty indexing automatically provides insensitivity to NaN, inf or complex values in m (and probably accounts for the small computation time as well).
Summing that r × 0 vector along its second dimension (sum(m(:,[]),2)) produces a vector of r × 1 zeros. Negating and summing this vector gives r.
The same procedure is applied for the number of columns, c, by empty-indexing in the first dimension and summing along that dimension.
The find command has a neat option to get the last K elements:
I = find(X,K,'last') returns at most the last K indices corresponding to the nonzero entries of the arrayX`.
To get the size, ask for the last k=1 elements. For example,
>> x=zeros(256,4);
>> [numRows,numCols] = find(x|x==0, 1, 'last')
numRows =
256
numCols =
4
>> numRows0 = size(x,1), numCols0 = size(x,2)
numRows0 =
256
numCols0 =
4
You can use find with the single output argument syntax, which will give you numel:
>> numEl = find(x|x==0, 1, 'last')
numEl =
1024
>> numEl0 = numel(x)
numEl0 =
1024
Another straightforward, but less interesting solution uses whos (thanks for the reminder Navan):
s=whos('x'); s.size
Finally, there is format debug.
Related
I have 2 big arrays A and b:
A: 10.000++ rows, 4 columns, not unique integers
b: vector with 500.000++ elements, unique integers
Due to the uniqueness of the values of b, I need to find the only index of b, where A(i,j) == b.
What I started with is
[rows,columns] = size(A);
B = zeros(rows,columns);
for i = 1 : rows
for j = 1 : columns
B(i,j) = find(A(i,j)==b,1);
end
end
This takes approx 5.5 seconds to compute, which is way to long, since A and b can be significantly bigger... That in mind I tried to speed up the code by using logical indexing and reducing the for-loops
[rows,columns] = size(A);
B = zeros(rows,columns);
for idx = 1 : numel(b)
B(A==b(idx)) = idx;
end
Sadly this takes even longer: 21 seconds
I even tried to do use bsxfun
for i = 1 : columns
[I,J] = find(bsxfun(#eq,A(:,i),b))
... stitch B together ...
end
but with a bigger arrays the maximum array size is quickly exceeded (102,9GB...).
Can you help me find a faster solution to this? Thanks in advance!
EDIT: I extended find(A(i,j)==b,1), which speeds up the algorithm by factor 2! Thank you, but overall still too slow... ;)
The function ismember is the right tool for this:
[~,B] = ismember(A,b);
Test code:
function so
A = rand(1000,4);
b = unique([A(:);rand(2000,1)]);
B1 = op1(A,b);
B2 = op2(A,b);
isequal(B1,B2)
tic;op1(A,b);op1(A,b);op1(A,b);op1(A,b);toc
tic;op2(A,b);op2(A,b);op2(A,b);op2(A,b);toc
end
function B = op1(A,b)
B = zeros(size(A));
for i = 1:numel(A)
B(i) = find(A(i)==b,1);
end
end
function B = op2(A,b)
[~,B] = ismember(A,b);
end
I ran this on Octave, which is not as fast with loops as MATLAB. It also doesn't have the timeit function, hence the crappy timing using tic/toc (sorry for that). In Octave, op2 is more than 100 times faster than op1. Timings will be different in MATLAB, but ismember should still be the fastest option. (Note I also replaced your double loop with a single loop, this is the same but simpler and probably faster.)
If you want to repeatedly do the search in b, it is worthwhile to sort b first, and implement your own binary search. This will avoid the checks and sorting that ismember does. See this other question.
Assuming that you have positive integers you can use array indexing:
mm = max(max(A(:)),max(b(:)));
idxs = sparse(b,1,1:numel(b),mm,1);
result = full(idxs(A));
If the range of values is small you can use dense matrix instead of sparse matrix:
mm = max(max(A(:)),max(b(:)));
idx = zeros(mm,1);
idx(b)=1:numel(b);
result = idx(A);
In accumarray() the first note about 'subs', first appeared in the MATLAB R14sp3 docs, says:
Note If the subscripts in subs are not sorted, fun should not depend on the order of the values in its input data.
It is not clear to me what is considered to be sorted. Suppose:
subs = [1 1
2 1
1 2
2 2];
shall subs be sorted in the sense issorted(subs,'rows'), or ...
in the linear indexing sense, i.e. issorted(sub2ind([2 2],subs(:,1), subs(:,2)))
I would like to rely on:
accumarray(subs,val,[], #(x) x(end))
If somebody could also provide examples/tests from older releases (to check for backward compatibility) where e.g. 1) is false while 2) is true, that would be great.
PS. I am mostly not interested in alternatives to accumarray, unless very succinct and using the same subs and val.
Ok I did some tests, and I think that "sorted" in the quote is meant in the linear indexing sense. (I'm on R2013a if that matters)
To understand how accumarray calls the specified function, I'll use the trick of grouping the values into a cellarray by specifying fun = #(x) {x} as the function to be applied.
1) 1D indices
First lets create some subscripts and values
N = 10; sz = 4;
subs = randi([1 sz], [N 1]);
vals = (1:N)'*100;
Now we call ACCUMARRAY on the non-sorted indices (multiple times)
C = cell(5,1);
for i=1:5
C{i} = accumarray(subs, vals, [], #(x){x});
end
The order of the values passed to the function is arbitrary but still consistent across the multiple runs:
>> assert(isequal(C{:}))
>> celldisp(C{1})
ans{1} =
800
900
700
ans{2} =
300
ans{3} =
1000
200
100
ans{4} =
400
600
500
This is why the documentation warns you that fun should not depend on the order of the values passed to it.
Now if we sort the subscripts beforehand:
[~,ord] = sort(subs);
C = cell(5,1);
for i=1:5
C{i} = accumarray(subs(ord), vals(ord), [], #(x){x});
end
assert(isequal(C{:}))
celldisp(C{1})
we will see that the values are passed to the function sorted themselves:
ans{1} =
700
800
900
ans{2} =
300
ans{3} =
100
200
1000
ans{4} =
400
500
600
2) 2D indices
I've tried the same thing in the case of 2D subscript indices. First we start with random data:
%# some 2d subscripts and corresponding values
N = 10; sz = 2;
subs = randi([1 sz], [N 2]);
vals = (1:N)*100;
Here is the case for non-sorted indices:
C = cell(5,1);
for i=1:5
C{i} = accumarray(subs, vals, [], #(x){x});
end
assert(isequal(C{:}))
celldisp(C{1})
Here is when I tried "sorting by rows":
[~,ord] = sortrows(subs, [1 2]);
C = cell(5,1);
for i=1:5
C{i} = accumarray(subs(ord,:), vals(ord), [], #(x){x});
end
assert(isequal(C{:}))
celldisp(C{1})
And finally here is when we sort by "linear index":
[~,ord] = sort(sub2ind([sz sz], subs(:,1), subs(:,2)));
C = cell(5,1);
for i=1:5
C{i} = accumarray(subs(ord,:), vals(ord), [], #(x){x});
end
assert(isequal(C{:}))
celldisp(C{1})
I'll omit the long output, and report that only in the last case that the values were passed to the function ordered. So I conclude that the "sorted" criteria is based on the linear indices.
ans{1,1} =
[]
ans{2,1} =
200
600
700
ans{1,2} =
100
300
400
500
1000
ans{2,2} =
800
900
Bonus:
Try using the following function instead:
function out = fun(x)
out = {x};
disp('[')
disp(x)
disp(']')
end
you'll see that the function is called in an unpredictable way, even with some duplication! You might have to increase the size of the data to see such behavior...
As far as backward compatibility goes, the documentation mentions this note all the way back in MATLAB R14sp3 but not R14sp2. Given the fact that it's been documented since 2005, I'd say it is safe to rely on this feature (I doubt anyone using such old versions expects new code to just work anyway!)
G'day,
I'm trying to find a way to use a vector of [x,y] points to index from a large matrix in MATLAB.
Usually, I would convert the subscript points to the linear index of the matrix.(for eg. Use a vector as an index to a matrix) However, the matrix is 4-dimensional, and I want to take all of the elements of the 3rd and 4th dimensions that have the same 1st and 2nd dimension. Let me hopefully demonstrate with an example:
Matrix = nan(4,4,2,2); % where the dimensions are (x,y,depth,time)
Matrix(1,2,:,:) = 999; % note that this value could change in depth (3rd dim) and time (4th time)
Matrix(3,4,:,:) = 888; % note that this value could change in depth (3rd dim) and time (4th time)
Matrix(4,4,:,:) = 124;
Now, I want to be able to index with the subscripts (1,2) and (3,4), etc and return not only the 999 and 888 which exist in Matrix(:,:,1,1) but the contents which exist at Matrix(:,:,1,2),Matrix(:,:,2,1) and Matrix(:,:,2,2), and so on (IRL, the dimensions of Matrix might be more like size(Matrix) = (300 250 30 200)
I don't want to use linear indices because I would like the results to be in a similar vector fashion. For example, I would like a result which is something like:
ans(time=1)
999 888 124
999 888 124
ans(time=2)
etc etc etc
etc etc etc
I'd also like to add that due to the size of the matrix I'm dealing with, speed is an issue here - thus why I'd like to use subscript indices to index to the data.
I should also mention that (unlike this question: Accessing values using subscripts without using sub2ind) since I want all the information stored in the extra dimensions, 3 and 4, of the i and jth indices, I don't think that a slightly faster version of sub2ind still would not cut it..
I can think of three ways to go about this
Simple loop
Just loop over all the 2D indices you have, and use colons to access the remaining dimensions:
for jj = 1:size(twoDinds,1)
M(twoDinds(jj,1),twoDinds(jj,2),:,:) = rand;
end
Vectorized calculation of Linear indices
Skip sub2ind and vectorize the computation of linear indices:
% generalized for arbitrary dimensions of M
sz = size(M);
nd = ndims(M);
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
% the linear indices
inds = twoDinds(:,1) + ([twoDinds(:,2) [argout{:}]]-1) * cumprod(sz(1:3)).';
Sub2ind
Just use the ready-made tool that ships with Matlab:
inds = sub2ind(size(M), twoDinds(:,1), twoDinds(:,2), argout{:});
Speed
So which one's the fastest? Let's find out:
clc
M = nan(4,4,2,2);
sz = size(M);
nd = ndims(M);
twoDinds = [...
1 2
4 3
3 4
4 4
2 1];
tic
for ii = 1:1e3
for jj = 1:size(twoDinds,1)
M(twoDinds(jj,1),twoDinds(jj,2),:,:) = rand;
end
end
toc
tic
twoDinds_prev = twoDinds;
for ii = 1:1e3
twoDinds = twoDinds_prev;
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
inds = twoDinds(:,1) + ([twoDinds(:,2) [argout{:}]]-1) * cumprod(sz(1:3)).';
M(inds) = rand;
end
toc
tic
for ii = 1:1e3
twoDinds = twoDinds_prev;
arg = arrayfun(#(x)1:x, sz(3:nd), 'UniformOutput', false);
[argout{1:nd-2}] = ndgrid(arg{:});
argout = cellfun(...
#(x) repmat(x(:), size(twoDinds,1),1), ...
argout, 'Uniformoutput', false);
twoDinds = kron(twoDinds, ones(prod(sz(3:nd)),1));
inds = sub2ind(size(M), twoDinds(:,1), twoDinds(:,2), argout{:});
M(inds) = rand;
end
toc
Results:
Elapsed time is 0.004778 seconds. % loop
Elapsed time is 0.807236 seconds. % vectorized linear inds
Elapsed time is 0.839970 seconds. % linear inds with sub2ind
Conclusion: use the loop.
Granted, the tests above are largely influenced by JIT's failure to compile the two last loops, and the non-specificity to 4D arrays (the last two method also work on ND arrays). Making a specialized version for 4D will undoubtedly be much faster.
Nevertheless, the indexing with simple loop is, well, simplest to do, easiest on the eyes and very fast too, thanks to JIT.
So, here is a possible answer... but it is messy. I suspect it would more computationally expensive then a more direct method... And this would definitely not be my preferred answer. It would be great if we could get the answer without any for loops!
Matrix = rand(100,200,30,400);
grabthese_x = (1 30 50 90);
grabthese_y = (61 9 180 189);
result=nan(size(length(grabthese_x),size(Matrix,3),size(Matrix,4));
for tt = 1:size(Matrix,4)
subset = squeeze(Matrix(grabthese_x,grabthese_y,:,tt));
for NN=1:size(Matrix,3)
result(:,NN,tt) = diag(subset(:,:,NN));
end
end
The resulting matrix, result should have size size(result) = (4 N tt).
I think this should work, even if Matrix isn't square. However, it is not ideal, as I said above.
I have a non-fixed dimensional matrix M, from which I want to access a single element.
The element's indices are contained in a vector J.
So for example:
M = rand(6,4,8,2);
J = [5 2 7 1];
output = M(5,2,7,1)
This time M has 4 dimensions, but this is not known in advance. This is dependent on the setup of the algorithm I'm writing. It could likewise be that
M = rand(6,4);
J = [3 1];
output = M(3,1)
so I can't simply use
output=M(J(1),J(2))
I was thinking of using sub2ind, but this also needs its variables comma separated..
#gnovice
this works, but I intend to use this kind of element extraction from the matrix M quite a lot. So if I have to create a temporary variable cellJ every time I access M, wouldn't this tremendously slow down the computation??
I could also write a separate function
function x= getM(M,J)
x=M(J(1),J(2));
% M doesn't change in this function, so no mem copy needed = passed by reference
end
and adapt this for different configurations of the algorithm. This is of course a speed vs flexibility consideration which I hadn't included in my question..
BUT: this is only available for getting the element, for setting there is no other way than actually using the indices (and preferably the linear index). I still think sub2ind is an option. The final result I had intended was something like:
function idx = getLinearIdx(J, size_M)
idx = ...
end
RESULTS:
function lin_idx = Lidx_ml( J, M )%#eml
%LIDX_ML converts an array of indices J for a multidimensional array M to
%linear indices, directly useable on M
%
% INPUT
% J NxP matrix containing P sets of N indices
% M A example matrix, with same size as on which the indices in J
% will be applicable.
%
% OUTPUT
% lin_idx Px1 array of linear indices
%
% method 1
%lin_idx = zeros(size(J,2),1);
%for ii = 1:size(J,2)
% cellJ = num2cell(J(:,ii));
% lin_idx(ii) = sub2ind(size(M),cellJ{:});
%end
% method 2
sizeM = size(M);
J(2:end,:) = J(2:end,:)-1;
lin_idx = cumprod([1 sizeM(1:end-1)])*J;
end
method 2 is 20 (small number of index sets (=P) to convert) to 80 (large number of index sets (=P)) times faster than method 1. easy choice
For the general case where J can be any length (which I assume always matches the number of dimensions in M), there are a couple options you have:
You can place each entry of J in a cell of a cell array using the num2cell function, then create a comma-separated list from this cell array using the colon operator:
cellJ = num2cell(J);
output = M(cellJ{:});
You can sidestep the sub2ind function and compute the linear index yourself with a little bit of math:
sizeM = size(M);
index = cumprod([1 sizeM(1:end-1)]) * (J(:) - [0; ones(numel(J)-1, 1)]);
output = M(index);
Here is a version of gnovices option 2) which allows to process a whole matrix of subscripts, where each row contains one subscript. E.g for 3 subscripts:
J = [5 2 7 1
1 5 2 7
4 3 9 2];
sizeM = size(M);
idx = cumprod([1 sizeX(1:end-1)])*(J - [zeros(size(J,1),1) ones(size(J,1),size(J,2)-1)]).';
I have a function, ranker, that takes a vector and assigns numerical ranks to it in ascending order. For example,
ranker([5 1 3 600]) = [3 1 2 4] or
ranker([42 300 42 42 1 42] = [3.5 6 3.5 3.5 1 3.5] .
I am using a matrix, variable_data and I want to apply the ranker function to each row for all rows in variable data. This is my current solution, but I feel there is a way to vectorize it and have it as equally fast :p
variable_ranks = nan(size(variable_data));
for i=1:1:numel(nmac_ids)
variable_ranks(i,:) = ranker(abs(variable_data(i,:)));
end
If you place the matrix rows into a cell array, you can then apply a function to each cell.
Consider this simple example of applying the SORT function to each row
a = rand(10,3);
b = cell2mat( cellfun(#sort, num2cell(a,2), 'UniformOutput',false) );
%# same as: b = sort(a,2);
You can even do this:
b = cell2mat( arrayfun(#(i) sort(a(i,:)), 1:size(a,1), 'UniformOutput',false)' );
Again, you version with the for loop is probably faster..
With collaboration from Amro and Jonas
variable_ranks = tiedrank(variable_data')';
Ranker has been replaced by the Matlab function in the Stat toolbox (sorry for those who don't have it),
[R,TIEADJ] = tiedrank(X) computes the
ranks of the values in the vector X.
If any X values are tied, tiedrank
computes their average rank. The
return value TIEADJ is an adjustment
for ties required by the nonparametric
tests signrank and ranksum, and for
the computation of Spearman's rank
correlation.
TIEDRANK will compute along columns in Matlab 7.9.0 (R2009b), however it is undocumented. So by transposing the input matrix, rows turn into columns and will rank them. The second transpose is then used to organize the data in the same manner as the input. There in essence is a very classy hack :p
One way would be to rewrite ranker to take array input
sizeData = size(variable_data);
[sortedData,almostRanks] = sort(abs(variable_data),2);
[rowIdx,colIdx] = ndgrid(1:sizeData(1),1:sizeData(2));
linIdx = sub2ind(sizeData,rowIdx,almostRanks);
variable_ranks = variable_data;
variable_ranks(linIdx) = colIdx;
%# break ties by finding subsequent equal entries in sorted data
[rr,cc] = find(diff(sortedData,1,2) == 0);
ii = sub2ind(sizeData,rr,cc);
ii2 = sub2ind(sizeData,rr,cc+1);
ii = sub2ind(sizeData,rr,almostRanks(ii));
ii2 = sub2ind(sizeData,rr,almostRanks(ii2));
variable_ranks(ii) = variable_ranks(ii2);
EDIT
Instead, you can just use TIEDRANK from TMW (thanks, #Amro):
variable_rank = tiedrank(variable_data')';
I wrote a function that does this, it's on the FileExchange tiedrank_(X,dim). And it looks like this...
%[Step 0a]: force dim to be 1, and compress everything else into a single
%dimension. We will reverse this process at the end.
if dim > 1
otherDims = 1:length(size(X));
otherDims(dim) = [];
perm = [dim otherDims];
X = permute(X,perm);
end
originalSiz = size(X);
X = reshape(X,originalSiz(1),[]);
siz = size(X);
%[Step 1]: sort and get sorting indicies
[X,Ind] = sort(X,1);
%[Step 2]: create matrix [D], which has +1 at the start of consecutive runs
% and -1 at the end, with zeros elsewhere.
D = zeros(siz,'int8');
D(2:end-1,:) = diff(X(1:end-1,:) == X(2:end,:));
D(1,:) = X(1,:) == X(2,:);
D(end,:) = -( X(end,:) == X(end-1,:) );
clear X
%[Step 3]: calculate the averaged rank for each consecutive run
[a,~] = find(D);
a = reshape(a,2,[]);
h = sum(a,1)/2;
%[Step 4]: insert the troublseome ranks in the relevant places
L = zeros(siz);
L(D==1) = h;
L(D==-1) = -h;
L = cumsum(L);
L(D==-1) = h; %cumsum set these ranks to zero, but we wanted them to be h
clear D h
%[Step 5]: insert the simple ranks (i.e. the ones that didn't clash)
[L(~L),~] = find(~L);
%[Step 6]: assign the ranks to the relevant position in the matrix
Ind = bsxfun(#plus,Ind,(0:siz(2)-1)*siz(1)); %equivalent to using sub2ind + repmat
r(Ind) = L;
%[Step 0b]: As promissed, we reinstate the correct dimensional shape and order
r = reshape(r,originalSiz);
if dim > 1
r = ipermute(r,perm);
end
I hope that helps someone.