Related
I am working with Matlab.
I have a binary square matrix. For each row, there is one or more entries of 1. I want to go through each row of this matrix and return the index of those 1s and store them in the entry of a cell.
I was wondering if there is a way to do this without looping over all the rows of this matrix, as for loop is really slow in Matlab.
For example, my matrix
M = 0 1 0
1 0 1
1 1 1
Then eventually, I want something like
A = [2]
[1,3]
[1,2,3]
So A is a cell.
Is there a way to achieve this goal without using for loop, with the aim of calculating the result more quickly?
At the bottom of this answer is some benchmarking code, since you clarified that you're interested in performance rather than arbitrarily avoiding for loops.
In fact, I think for loops are probably the most performant option here. Since the "new" (2015b) JIT engine was introduced (source) for loops are not inherently slow - in fact they are optimised internally.
You can see from the benchmark that the mat2cell option offered by ThomasIsCoding here is very slow...
If we get rid of that line to make the scale clearer, then my splitapply method is fairly slow, obchardon's accumarray option is a bit better, but the fastest (and comparable) options are either using arrayfun (as also suggested by Thomas) or a for loop. Note that arrayfun is basically a for loop in disguise for most use-cases, so this isn't a surprising tie!
I would recommend you use a for loop for increased code readability and the best performance.
Edit:
If we assume that looping is the fastest approach, we can make some optimisations around the find command.
Specifically
Make M logical. As the below plot shows, this can be faster for relatively small M, but slower with the trade-off of type conversion for large M.
Use a logical M to index an array 1:size(M,2) instead of using find. This avoids the slowest part of the loop (the find command) and outweighs the type conversion overhead, making it the quickest option.
Here is my recommendation for best performance:
function A = f_forlooplogicalindexing( M )
M = logical(M);
k = 1:size(M,2);
N = size(M,1);
A = cell(N,1);
for r = 1:N
A{r} = k(M(r,:));
end
end
I've added this to the benchmark below, here is the comparison of loop-style approaches:
Benchmarking code:
rng(904); % Gives OP example for randi([0,1],3)
p = 2:12;
T = NaN( numel(p), 7 );
for ii = p
N = 2^ii;
M = randi([0,1],N);
fprintf( 'N = 2^%.0f = %.0f\n', log2(N), N );
f1 = #()f_arrayfun( M );
f2 = #()f_mat2cell( M );
f3 = #()f_accumarray( M );
f4 = #()f_splitapply( M );
f5 = #()f_forloop( M );
f6 = #()f_forlooplogical( M );
f7 = #()f_forlooplogicalindexing( M );
T(ii, 1) = timeit( f1 );
T(ii, 2) = timeit( f2 );
T(ii, 3) = timeit( f3 );
T(ii, 4) = timeit( f4 );
T(ii, 5) = timeit( f5 );
T(ii, 6) = timeit( f6 );
T(ii, 7) = timeit( f7 );
end
plot( (2.^p).', T(2:end,:) );
legend( {'arrayfun','mat2cell','accumarray','splitapply','for loop',...
'for loop logical', 'for loop logical + indexing'} );
grid on;
xlabel( 'N, where M = random N*N matrix of 1 or 0' );
ylabel( 'Execution time (s)' );
disp( 'Done' );
function A = f_arrayfun( M )
A = arrayfun(#(r) find(M(r,:)),1:size(M,1),'UniformOutput',false);
end
function A = f_mat2cell( M )
[i,j] = find(M.');
A = mat2cell(i,arrayfun(#(r) sum(j==r),min(j):max(j)));
end
function A = f_accumarray( M )
[val,ind] = ind2sub(size(M),find(M.'));
A = accumarray(ind,val,[],#(x) {x});
end
function A = f_splitapply( M )
[r,c] = find(M);
A = splitapply( #(x) {x}, c, r );
end
function A = f_forloop( M )
N = size(M,1);
A = cell(N,1);
for r = 1:N
A{r} = find(M(r,:));
end
end
function A = f_forlooplogical( M )
M = logical(M);
N = size(M,1);
A = cell(N,1);
for r = 1:N
A{r} = find(M(r,:));
end
end
function A = f_forlooplogicalindexing( M )
M = logical(M);
k = 1:size(M,2);
N = size(M,1);
A = cell(N,1);
for r = 1:N
A{r} = k(M(r,:));
end
end
You can try arrayfun like below, which sweep through rows of M
A = arrayfun(#(r) find(M(r,:)),1:size(M,1),'UniformOutput',false)
A =
{
[1,1] = 2
[1,2] =
1 3
[1,3] =
1 2 3
}
or (a slower approach by mat2cell)
[i,j] = find(M.');
A = mat2cell(i,arrayfun(#(r) sum(j==r),min(j):max(j)))
A =
{
[1,1] = 2
[2,1] =
1
3
[3,1] =
1
2
3
}
Using accumarray:
M = [0 1 0
1 0 1
1 1 1];
[val,ind] = find(M.');
A = accumarray(ind,val,[],#(x) {x});
You can use strfind :
A = strfind(cellstr(char(M)), char(1));
Edit: I added a benchmark, the results show that a for loop is more efficient than accumarray.
You can usefind and accumarray:
[c, r] = find(A');
C = accumarray(r, c, [], #(v) {v'});
The matrix is transposed (A') because find groups by column.
Example:
A = [1 0 0 1 0
0 1 0 0 0
0 0 1 1 0
1 0 1 0 1];
% Find nonzero rows and colums
[c, r] = find(A');
% Group row indices for each columns
C = accumarray(r, c, [], #(v) {v'});
% Display cell array contents
celldisp(C)
Output:
C{1} =
1 4
C{2} =
2
C{3} =
3 4
C{4} =
1 3 5
Benchmark:
m = 10000;
n = 10000;
A = randi([0 1], m,n);
disp('accumarray:')
tic
[c, r] = find(A');
C = accumarray(r, c, [], #(v) {v'});
toc
disp(' ')
disp('For loop:')
tic
C = cell([size(A,1) 1]);
for i = 1:size(A,1)
C{i} = find(A(i,:));
end
toc
Result:
accumarray:
Elapsed time is 2.407773 seconds.
For loop:
Elapsed time is 1.671387 seconds.
A for loop is more efficient than accumarray...
I am a bit confused and would greatly appreciate some help.
I have read many posts about finding neighboring pixels, with this being extremely helpful:
http://blogs.mathworks.com/steve/2008/02/25/neighbor-indexing-2/
However I have trouble applying it on a 4D matrix (A) with size(A)=[8 340 340 15]. It represents 8 groups of 3D images (15 slices each) of which I want to get the neighbors.
I am not sure which size to use in order to calculate the offsets. This is the code I tried, but I think it is not working because the offsets should be adapted for 4 dimensions? How can I do it without a loop?
%A is a 4D matrix with 0 or 1 values
Aidx = find(A);
% loop here?
[~,M,~,~] =size(A);
neighbor_offsets = [-1, M, 1, -M]';
neighbors_idx = bsxfun(#plus, Aidx', neighbor_offsets(:));
neighbors = B(neighbors_idx);
Thanks,
ziggy
Have you considered using convn?
msk = [0 1 0; 1 0 1; 0 1 0];
msk4d = permute( msk, [3 1 2 4] ); % make it 1-3-3-1 mask
neighbors_idx = find( convn( A, msk4d, 'same' ) > 0 );
You might find conndef useful for defining the basic msk in a general way.
Not sure if I've understood your question but what about this sort of approach:
if you matrix is 1D:
M = rand(10,1);
N = M(k-1:k+1); %//immediate neighbours of k
However this could error if k is at the boundary. This is easy to fix using max and min:
N = M(max(k-1,1):min(k+1,size(M,1))
Now lets add a dimenion:
M = rand(10,10);
N = M(max(k1-1,1):min(k1+1,size(M,1), max(k2-1,1):min(k2+1,size(M,2))
That was easy, all you had to do was repeat the same index making the minor change of using size(M,2) for the boundary (and also I changed k to k1 and k2, you might find using an array for k instead of separate k1 and k2 variables works better i.e. k(1) and k(2))
OK so now lets skip to 4 dimensions:
M = rand(10,10,10,10);
N = M(max(k(1)-1,1):min(k(1)+1,size(M,1)), ...
max(k(2)-1,1):min(k(2)+1,size(M,2)), ...
max(k(3)-1,1):min(k(3)+1,size(M,3)), ...
max(k(4)-1,1):min(k(4)+1,size(M,4))); %// Also you can replace all the `size(M,i)` with `end` if you like
I know you said you didn't want a loop, but what about a really short loop just to refactor a bit and also make it generalized:
n=ndims(M);
ind{n} = 0;
for dim = 1:n
ind{dim} = max(k(dim)-1,1):min(k(dim)+1,size(M,dim));
end
N = M(ind{:});
Here's how to get the neighbors along the second dimension
sz = size( A );
ndims = numel(sz); % number of dimensions
[d{1:ndims}] = ind2sub( sz, find( A ) );
alongD = 2; % work along this dim
np = d{alongD} + 1;
sel = np <= sz( alongD ); % discard neighbors that fall outside image boundary
nm = d{alongD} - 1;
sel = sel & nm > 0; % discard neighbors that fall outside image boundary
d = cellfun( #(x) x(sel), d, 'uni', 0 );
neighbors = cat( 1, ...
ind2sub( sz, d{1:alongD-1}, np(sel), d{alongD+1:end} ),...
ind2sub( sz, d{1:alongD-1}, nm(sel), d{alongD+1:end} ) );
I have two matrices of big sizes, which are something similar to the following matrices.
m; with size 1000 by 10
n; with size 1 by 10.
I would like to subtract each element of n from all elements of m to get ten different matrices, each has size of 1000 by 10.
I started as follows
clc;clear;
nrow = 10000;
ncol = 10;
t = length(n)
for i = 1:nrow;
for j = 1:ncol;
for t = 1:length(n);
m1(i,j) = m(i,j)-n(1);
m2(i,j) = m(i,j)-n(2);
m3(i,j) = m(i,j)-n(3);
m4(i,j) = m(i,j)-n(4);
m5(i,j) = m(i,j)-n(5);
m6(i,j) = m(i,j)-n(6);
m7(i,j) = m(i,j)-n(7);
m8(i,j) = m(i,j)-n(8);
m9(i,j) = m(i,j)-n(9);
m10(i,j) = m(i,j)-n(10);
end
end
end
can any one help me how can I do it without writing the ten equations inside the loop? Or can suggest me any convenient way especially when the two matrices has many columns.
Why can't you just do this:
m01 = m - n(1);
...
m10 = m - n(10);
What do you need the loop for?
Even better:
N = length(n);
m2 = cell(N, 1);
for k = 1:N
m2{k} = m - n(k);
end
Here we go loopless:
nrow = 10000;
ncol = 10;
%example data
m = ones(nrow,ncol);
n = 1:ncol;
M = repmat(m,1,1,ncol);
N = permute( repmat(n,nrow,1,ncol) , [1 3 2] );
result = bsxfun(#minus, M, N );
%or just
result = M-N;
Elapsed time is 0.018499 seconds.
or as recommended by Luis Mendo:
M = repmat(m,1,1,ncol);
result = bsxfun(#minus, m, permute(n, [1 3 2]) );
Elapsed time is 0.000094 seconds.
please make sure that your input vectors have the same orientation like in my example, otherwise you could get in trouble. You should be able to obtain that by transposements or you have to modify this line:
permute( repmat(n,nrow,1,ncol) , [1 3 2] )
according to your needs.
You mentioned in a comment that you want to count the negative elements in each of the obtained columns:
A = result; %backup results
A(A > 0) = 0; %set non-negative elements to zero
D = sum( logical(A),3 );
which will return the desired 10000x10 matrix with quantities of negative elements. (Please verify it, I may got a little confused with the dimensions ;))
Create the three dimensional result matrix. Store your results, for example, in third dimension.
clc;clear;
nrow = 10000;
ncol = 10;
N = length(n);
resultMatrix = zeros(nrow, ncol, N);
neg = zeros(ncol, N); % amount of negative values
for j = 1:ncol
for i = 1:nrow
for t = 1:N
resultMatrix(i,j,t) = m(i,j) - n(t);
end
end
for t = 1:N
neg(j,t) = length( find(resultMatrix(:,j,t) < 0) );
end
end
The formula I have to translate to Octave/Matlab goes something like this:
\sum (v_i - m) (v_i - m)^T
I have a matrix, and I need to take each row, subtract m from it and then multiply it with its own transpose. I wrote the inner part as a function:
function w = str(v, m)
y = v - m
w = y * transpose(y)
end
My matrix is like this
xx = [1 2 3 4 5; 1 2 3 4 5; 1 2 3 4 5]
Now I have no idea how to apply this function to each row in a matrix and then sum them up to a new matrix. Maybe someone can help me here.
EDIT: The result is not the dot product. I'm looking for v * v^T, which has a matrix as result!
Probably you need this
X = bsxfun( #minus, A, m );
Y = X'* X;
Suppose the matrix is A, then the solution is
total = sum(sum((A-m).*(A-m),2));
A.*A is an element wise multiplication, hence sum(A.*A,2) returns a column vector, with each element being the self dot product of each row in A.
If m is a vector then, it is slightly more complicated.
[p,~]=size(A);
total = sum(sum((A-repmat(m,p,1)).*(A-repmat(m,p,1)),2));
Cheers.
In the end, I wrote this:
function w = str(v, m)
y = v - m;
w = y' * y;
end
y = zeros(5,5);
for i=1:12
y = y + str(A(i,:), m);
end
Surely not the most elegant way to do this, but it seems to work.
You can subtract the mean using bsxfun
>> v_m = bsxfun( #minus, v, m );
For the sum of outer product of all vectors you can use bsxfun again
>> op = bsxfun( #times, permute( v, [3 1 2]), permute( v, [1 3 2] ) );
>> op = sum( op, 3 );
There are two ways to solve this issue:
Assume A is your matrix:
sum(drag(A' * A))
will do the job. However, it is slightly more efficient with the following:
sum((A .* A)(:))
Assuming
A=[32512199.30 5401000.29 347.33
32512199.69 5401000.45 347.39
32512199.67 5401001.32 353.58
32512199.96 5401001.50 346.99
32512196.71 5401001.69 346.62 ]
and
B=[32512199.30 5401000.29 347.33
32512199.69 5401000.45 347.39
32512199.67 5401001.32 347.00
32512198.85 5401000.91 347.25
32512196.71 5401001.69 346.87 ]
I want using ismember extract the rows that have same X and Y and different Z. X is first column, Y is the second and Z is third.
in A and B I want extract from A 32512199.67 5401001.32 353.58 and 32512196.71 5401001.69 346.62 OR from B 32512199.67 5401001.32 347.00 and 32512196.71 5401001.69 346.87
How can I do it?
inds = find(~ismember(A, B, 'rows'));
new_inds = find(ismember(A(inds, 1:2), B(:, 1:2), 'rows'));
inds(new_inds)
First find the row indices of desired match using ismember and find. Then extract those rows from A/B to XA/XB
row_idx = find ( ismember( ismember(A,B), [1 1 0], 'rows') )
XA = A(row_idx,:)
XB = B(row_idx,:)
Output:
row_idx =
3
5
XA =
3.2512e+07 5.4010e+06 3.5358e+02
3.2512e+07 5.4010e+06 3.4662e+02
XB =
3.2512e+07 5.4010e+06 3.4700e+02
3.2512e+07 5.4010e+06 3.4687e+02