I have a main matrix, say
A=magic(5);
and also a vector
v=[1;3;5;2;2];
I want to add up row-wise the elements of A in this way: add first row from the v(1)st element to the end, second row from the v(2)rd element to the end, third row from the v(3)th element to the end, and so on.
I know that I can do this using for-loop. But I want to know if there is a vectorized way to do it.
edit:
Let me clarify my question with an example:
Assume A and v as above.
A =
17 24 1 8 15
23 5 7 14 16
4 6 13 20 22
10 12 19 21 3
11 18 25 2 9
and
v =
1
3
5
2
2
Now I want a way to get the following results:
answer =
65 % 17+24+1+8+15
37 % 7+14+16
22 % 22
55 % 12+19+21+3
54 % 18+25+2+9
You can use cumsum along the rows. The solution is a bit complex, so I'll start with a simpler example:
Suppose you want to sum all elements of the i-th row of A till (including) the v(i)-th place: res_i = \sum_{k=1..v(i)} A_ik
m = size(A,1); % num of rows
csA = cumsum(A, 2); % cumsum along rows
res = csA( sub2ind( size(A), 1:m, v ) ); % pick the vi-th column for the i-th row
Now, for your question, since you want the sum of all elements from v(i) to the end, we need to flip A and change v accordingly
[m n] = size(A);
fA = fliplr(A);
fv = n + 1 - v; % flip the meaning of v
csA = cumsum( fA, 2 );
res = csA( sub2ind( [m n], 1:m, fv ) ); % should do the trick...
I know it's kinda cheating, but how about:
S = arrayfun(#(ii) sum(A(ii, v(ii):end)), 1:size(A,1)).';
Usually I'm a bit weary when using arrayfun, but when comparing:
% sample data
N = 5000;
A = magic(N);
v = randi(N, N,1);
% simple loop
tic
S = zeros(N,1);
for ii = 1:N
S(ii) = sum(A(ii, v(ii):end));
end
toc
% arrayfun solution
tic
S = arrayfun(#(ii) sum(A(ii, v(ii):end)), 1:N).';
toc
% Shai's solution
tic
[m n] = size(A);
fA = fliplr(A);
fv = n + 1 - v;
csA = cumsum( fA, 2 );
res = csA( sub2ind( [m n], 1:m, fv.' ) ).';
toc
Results:
Elapsed time is 0.386280 seconds. % simple loop
Elapsed time is 0.473916 seconds. % arrayfun
Elapsed time is 0.495794 seconds. % Shai's solution
So, arrayfun's not too bad after all.
But there is an important point here: Just look at the implementations from a distance. How easy to read/understand was the loop solution? How about the vectorized solutions?
With this in mind, also look at the performances. That's for a 5000x5000 matrix...that's 25 million elements there...
Now, would you really want to avoid that loop?
Related
I have a big matrix M (nxm). I am going to sum some elements which are specified by index stored in vector as cell elements. There are many groups of indices so the cell has more than one element. For example
M = rand(2103, 2030);
index{1} = [1 3 2 4 53 5 23 3];
index{2} = [2 3 1 3 23 10234 2032];
% ...
index{2032} = ...;
I am going to sum up all elements at index{1}, sum up all elements at index{2} ..., now I am using a loop
sums = zeros(1, 2032);
for n=1:2032
sums(n) = sum(M(index{n}));
end
I am wondering if there is any way to use one-line command instead of a loop to do that. Using a loop is pretty slow.
Probably a classic use of cellfun
sums = cellfun(#(idx) sum(M(idx)), index);
EDIT: here is a benchmarking for a large case that shows that this approach is slightly slower than a for loop but faster than Eitan T's method
M = rand(2103, 2030);
index = cell(1, 2032);
index{1} = [1 3 2 4 53 5 23 3];
index{2} = [2 3 1 3 23 10234 2032];
for n=3:2032
index{n} = randi(numel(M), 1, randi(10000));
end
N = 1e1;
sums = zeros(1, 2032);
tic
for kk = 1:N
for n=1:2032
sums(n) = sum(M(index{n}));
end
end
toc
tic
for kk = 1:N
sums = cellfun(#(idx) sum(M(idx)), index);
end
toc
tic
for kk = 1:N
sums = cumsum(M([index{:}]));
sums = diff([0, sums(cumsum(cellfun('length', index)))]);
end
toc
results in
Elapsed time is 2.072292 seconds.
Elapsed time is 2.139882 seconds.
Elapsed time is 2.669894 seconds.
Perhaps not as elegant as a cellfun one-liner, but runs more than an order of magnitude faster:
sums = cumsum(M([index{:}]));
sums = diff([0, sums(cumsum(cellfun('length', index)))]);
It even runs approximately 4 or 5 times faster than a JIT-accelerated loop for large inputs. Note that when each cell in index contains a vector with more than ~2000 elements, the performance of this approach begins to deteriorate in comparison with a loop (and cellfun).
Benchmark
M = rand(2103, 2030);
I = ceil(numel(M) * rand(2032, 10));
index = mat2cell(I, ones(size(I, 1), 1), size(I, 2));
N = 100;
tic
for k = 1:N
sums = zeros(1, numel(index));
for n = 1:numel(sums)
sums(n) = sum(M(index{n}));
end
end
toc
tic
for k = 1:N
sums = cellfun(#(idx) sum(M(idx)), index);
end
toc
tic
for k = 1:N
sums = cumsum(M([index{:}]));
sums2 = diff([0, sums(cumsum(cellfun('length', index)))]);
end
toc
When executing this in MATLAB 2012a (Windows Server 2008 R2 running on a 2.27GHz 16-core Intel Xeon processor), I got:
Elapsed time is 0.579783 seconds.
Elapsed time is 1.789809 seconds.
Elapsed time is 0.111455 seconds.
When I answered this question, my answer started with: "For starters, you can get rid of all the if-statements by storing all the names in a cell."
The "For starters" bit was because I thought that I could add an edit with a vectorized solution. But when I tried to do so I ran into trouble vectorizing the use of mrdivide (b/a).
My question (marked below in the code) is if it is possible to solve b(z,:)/a(z,:) without using a loop. In other words, to solve b/a independently for each row of the matrices.
person = [98 206 35 114;
60 206 28 52;
100 210 31 116;
69 217 26 35;
88 213 42 100];
person1 = [93 208 34 107];
allNames = {'Cameron'; 'David'; 'Mike'; 'Bill'; 'Joe'};
n = 5;
a = max(person,repmat(person1,n,1));
b = min(person,repmat(person1,n,1));
for z = 1:5
percent_error = b(z,:)/a(z,:); %// Here is my question
if percent_error >= 0.85
disp(['Match, its ', allNames{z} ,'!'])
end
end
You can indeed eliminate the loop by vectorizing the operation. The trick is working with diagonal block matrices. Each block is matrix with only one row (each time a different row). After you create such a block matrix for a and for b, you can use mrdivide:
% # Without loop
tic
A = zeros(size(a) * size(a, 1));
B = zeros(size(b) * size(b, 1));
V = ones(size(a, 2), 1) * (1:size(a, 1));
idx = (0:size(A, 1):numel(A) - 1) + (V(:)' - 1) * size(a, 1) + 1;
A(idx) = a';
B(idx) = b';
X = diag(B / A);
percent_error1 = X(1:size(a, 1):end);
toc
% # With loop
tic
percent_error2 = zeros(5, 1);
for z = 1:5
percent_error2(z) = b(z,:) / a(z,:);
end
toc
The result is:
Elapsed time is 0.000160 seconds.
Elapsed time is 0.000048 seconds.
percent_error1 =
0.9741
0.8516
0.9670
0.8221
0.9611
percent_error2 =
0.9741
0.8516
0.9670
0.8221
0.9611
Note that this is one of those cases where matrix division of large arrays takes longer than a for loop.
I was thinking about this:
person/person1
But this would only give good result when every index in person is bigger than the corresponding index in person1.
Given the matrix:
a =
1 1 2 2
1 1 2 2
3 3 4 4
3 3 4 4
I would like to get the following four 2x2 matrices:
a1 =
1 1
1 1
a2 =
2 2
2 2
a3 =
3 3
3 3
a4 =
4 4
4 4
From there, I would like to take the max of each matrix and then reshape the result into a 2x2 result matrix, like so:
r =
1 2
3 4
The location of the result max values relative to their original position in the initial matrix is important.
Currently, I'm using the following code to accomplish this:
w = 2
S = zeros(size(A, 1)/w);
for i = 1:size(S)
for j = 1:size(S)
Window = A(i*w-1:i*w, j*w-1:j*w);
S(i, j) = max(max(Window));
end
end
This works but it seems like there must be a way that doesn't involve iteration (vectorization).
I tried using reshape like so:
reshape(max(max(reshape(A, w, w, []))), w, w, [])
however that takes the max of the wrong values and returns:
ans =
3 4
3 4
Is there any way to accomplish this without iteration or otherwise improve my iterative method?
UPDATE: I'm not sure how I've ended up with the most votes (as of 2012-10-28). For anyone reading this, please see angainor's or Rody's answers for better solutions that don't require any additional toolboxes.
Here is a horse race of every answer thus far (excluding Nates - sorry, don't have the requisite toolbox):
Z = 1000;
A = [1 1 2 2; 1 1 2 2; 3 3 4 4; 3 3 4 4];
w = 2;
%Method 1 (OP method)
tic
for z = 1:Z
S = zeros(size(A, 1)/w);
for i = 1:size(S)
for j = 1:size(S)
Window = A(i*w-1:i*w, j*w-1:j*w);
S(i, j) = max(max(Window));
end
end
end
toc
%Method 2 (My double loop with improved indexing)
tic
for z = 1:Z
wm = w - 1;
Soln2 = NaN(w, w);
for m = 1:w:size(A, 2)
for n = 1:w:size(A, 1)
Soln2((m+1)/2, (n+1)/2) = max(max(A(n:n+wm, m:m+wm)));
end
end
Soln2 = Soln2';
end
toc
%Method 3 (My one line method)
tic
for z = 1:Z
Soln = cell2mat(cellfun(#max, cellfun(#max, mat2cell(A, [w w], [w w]), 'UniformOutput', false), 'UniformOutput', false));
end
toc
%Method 4 (Rody's method)
tic
for z = 1:Z
b = [A(1:2,:) A(3:4,:)];
reshape(max(reshape(b, 4,[])), 2,2);
end
toc
The results of the speed test (the loop over z) are:
Elapsed time is 0.042246 seconds.
Elapsed time is 0.019071 seconds.
Elapsed time is 0.165239 seconds.
Elapsed time is 0.011743 seconds.
Drat! It appears that Rody (+1) is the winner. :-)
UPDATE: New entrant to the race angainor (+1) takes the lead!
Not very general, but it works for a:
b = [a(1:2,:) a(3:4,:)];
reshape(max(reshape(b, 4,[])), 2,2).'
The general version of this is a bit *ahum* fuglier:
% window size
W = [2 2];
% number of blocks (rows, cols)
nW = size(a)./W;
% indices to first block
ids = bsxfun(#plus, (1:W(1)).', (0:W(2)-1)*size(a,1));
% indices to all blocks in first block-column
ids = bsxfun(#plus, ids(:), (0:nW(1)-1)*W(1));
% indices to all blocks
ids = reshape(bsxfun(#plus, ids(:), 0:nW(1)*prod(W):numel(a)-1), size(ids,1),[]);
% maxima
M = reshape(max(a(ids)), nW)
It can be done a bit more elegantly:
b = kron(reshape(1:prod(nW), nW), ones(W));
C = arrayfun(#(x) find(b==x), 1:prod(nW), 'uni', false);
M = reshape(max(a([C{:}])), nW)
but I doubt that's gonna be faster...
Another option: slower than the cell2mat(cellfun...) code, but gives the intermediate step:
fun = #(block_struct) reshape((block_struct.data), [],1);
B = reshape(blockproc(A,[2 2],fun),2,2,[])
r=reshape(max(max(B)) ,2,[])
B(:,:,1) =
1 1
1 1
B(:,:,2) =
3 3
3 3
B(:,:,3) =
2 2
2 2
B(:,:,4) =
4 4
4 4
r =
1 2
3 4
I'll join the horse-race with another non-general (yet;) solution, based on linear indices
idx = [1 2 5 6; 3 4 7 8]';
splita = [A(idx) A(idx+8)];
reshape(max(splita), 2, 2);
The times obtained by Colins code, my method last:
Elapsed time is 0.039565 seconds.
Elapsed time is 0.021723 seconds.
Elapsed time is 0.168946 seconds.
Elapsed time is 0.011688 seconds.
Elapsed time is 0.006255 seconds.
The idx array can be easily generalized to larger windows and system sizes.
Note: Nate's solution uses the Image Processing Toolbox function |blockproc|. I would rewrite that:
fun = #(x) max(max(x.data));
r = blockproc(A,[2 2],fun)
Comparing timing across different computers is fraught with difficulties, as is timing things once that are happening in a fraction of a second. TIMEIT would be useful here:
http://www.mathworks.com/matlabcentral/fileexchange/18798
But timing this on my computer with tic/toc took 0.008 seconds.
Cheers,
Brett
I have a very large matrix (216 rows, 31286 cols) of doubles. For reasons specific to the data, I want to average every 9 rows to produce one new row. So, the new matrix will have 216/9=24 rows.
I am a Matlab beginner so I was wondering if this solution I came up with can be improved upon. Basically, it loops over every group, sums up the rows, and then divides the new row by 9. Here's a simplified version of what I wrote:
matrix_avg = []
for group = 1:216/9
new_row = zeros(1, 31286);
idx_low = (group - 1) * 9 + 1;
idx_high = idx_low + 9 - 1;
% Add the 9 rows to new_row
for j = idx_low:idx_high
new_row = new_row + M(j,:);
end
% Compute the mean
new_row = new_row ./ 9
matrix_avg = [matrix_avg; new_row];
end
You can reshape your big matrix from 216 x 31286 to 9 x (216/9 * 31286).
Then you can use mean, which operates on each column. Since your matrix only has 9 rows per column, this takes the 9-row average.
Then you can just reshape your matrix back.
% generate big matrix
M = rand([216 31286]);
n = 9 % want 9-row average.
% reshape
tmp = reshape(M, [n prod(size(M))/n]);
% mean column-wise (and only 9 rows per col)
tmp = mean(tmp);
% reshape back
matrix_avg = reshape(tmp, [ size(M,1)/n size(M,2) ]);
In a one-liner (but why would you?):
matrix_avg = reshape(mean(reshape(M,[n prod(size(M))/n])), [size(M,1)/n size(M,2)]);
Note - this will have problems if the number of rows in M isn't exactly divisible by 9, but so will your original code.
I measured the 4 solutions and here are the results:
reshape: Elapsed time is 0.017242 seconds.
blockproc [9 31286]: Elapsed time is 0.242044 seconds.
blockproc [9 1]: Elapsed time is 44.477094 seconds.
accumarray: Elapsed time is 103.274071 seconds.
This is the code I used:
M = rand(216,31286);
fprintf('reshape: ');
tic;
n = 9;
matrix_avg1 = reshape(mean(reshape(M,[n prod(size(M))/n])), [size(M,1)/n size(M,2)]);
toc
fprintf('blockproc [9 31286]: ');
tic;
fun = #(block_struct) mean(block_struct.data);
matrix_avg2 = blockproc(M,[9 31286],fun);
toc
fprintf('blockproc [9 1]: ');
tic;
fun = #(block_struct) mean(block_struct.data);
matrix_avg3 = blockproc(M,[9 1],fun);
toc
fprintf('accumarray: ');
tic;
[nR,nC] = size(M);
n2average = 9;
[xx,yy] = ndgrid(1:nR,1:nC);
x = ceil(xx/n2average); %# makes xx 1 1 1 1 2 2 2 2 etc
matrix_avg4 = accumarray([xx(:),yy(:)],M(:),[],#mean);
toc
Here's an alternative based on accumarray. You create an array with row and column indices into matrix_avg that tells you which element in matrix_avg a given element in M contributes to, then you use accumarray to average the elements that contribute to the same element in matrix_avg. This solution works even if the number of rows in M is not divisible by 9.
M = rand(216,31286);
[nR,nC] = size(M);
n2average = 9;
[xx,yy] = ndgrid(1:nR,1:nC);
x = ceil(xx/n2average); %# makes xx 1 1 1 1 2 2 2 2 etc
matrix_avg = accumarray([xx(:),yy(:)],M(:),[],#mean);
I want to vectorize the following MATLAB code. I think it must be simple but I'm finding it confusing nevertheless.
r = some constant less than m or n
[m,n] = size(C);
S = zeros(m-r,n-r);
for i=1:m-r+1
for j=1:n-r+1
S(i,j) = sum(diag(C(i:i+r-1,j:j+r-1)));
end
end
The code calculates a table of scores, S, for a dynamic programming algorithm, from another score table, C.
The diagonal summing is to generate scores for individual pieces of the data used to generate C, for all possible pieces (of size r).
Thanks in advance for any answers! Sorry if this one should be obvious...
Note
The built-in conv2 turned out to be faster than convnfft, because my eye(r) is quite small ( 5 <= r <= 20 ). convnfft.m states that r should be > 20 for any benefit to manifest.
If I understand correctly, you're trying to calculate the diagonal sum of every subarray of C, where you have removed the last row and column of C (if you should not remove the row/col, you need to loop to m-r+1, and you need to pass the entire array C to the function in my solution below).
You can do this operation via a convolution, like so:
S = conv2(C(1:end-1,1:end-1),eye(r),'valid');
If C and r are large, you may want to have a look at CONVNFFT from the Matlab File Exchange to speed up calculations.
Based on the idea of JS, and as Jonas pointed out in the comments, this can be done in two lines using IM2COL with some array manipulation:
B = im2col(C, [r r], 'sliding');
S = reshape( sum(B(1:r+1:end,:)), size(C)-r+1 );
Basically B contains the elements of all sliding blocks of size r-by-r over the matrix C. Then we take the elements on the diagonal of each of these blocks B(1:r+1:end,:), compute their sum, and reshape the result to the expected size.
Comparing this to the convolution-based solution by Jonas, this does not perform any matrix multiplication, only indexing...
I would think you might need to rearrange C into a 3D matrix before summing it along one of the dimensions. I'll post with an answer shortly.
EDIT
I didn't manage to find a way to vectorise it cleanly, but I did find the function accumarray, which might be of some help. I'll look at it in more detail when I am home.
EDIT#2
Found a simpler solution by using linear indexing, but this could be memory-intensive.
At C(1,1), the indexes we want to sum are 1+[0, m+1, 2*m+2, 3*m+3, 4*m+4, ... ], or (0:r-1)+(0:m:(r-1)*m)
sum_ind = (0:r-1)+(0:m:(r-1)*m);
create S_offset, an (m-r) by (n-r) by r matrix, such that S_offset(:,:,1) = 0, S_offset(:,:,2) = m+1, S_offset(:,:,3) = 2*m+2, and so on.
S_offset = permute(repmat( sum_ind, [m-r, 1, n-r] ), [1, 3, 2]);
create S_base, a matrix of base array addresses from which the offset will be calculated.
S_base = reshape(1:m*n,[m n]);
S_base = repmat(S_base(1:m-r,1:n-r), [1, 1, r]);
Finally, use S_base+S_offset to address the values of C.
S = sum(C(S_base+S_offset), 3);
You can, of course, use bsxfun and other methods to make it more efficient; here I chose to lay it out for clarity. I have yet to benchmark this to see how it compares with the double-loop method though; I need to head home for dinner first!
Is this what you're looking for? This function adds the diagonals and puts them into a vector similar to how the function 'sum' adds up all of the columns in a matrix and puts them into a vector.
function [diagSum] = diagSumCalc(squareMatrix, LLUR0_ULLR1)
%
% Input: squareMatrix: A square matrix.
% LLUR0_ULLR1: LowerLeft to UpperRight addition = 0
% UpperLeft to LowerRight addition = 1
%
% Output: diagSum: A vector of the sum of the diagnols of the matrix.
%
% Example:
%
% >> squareMatrix = [1 2 3;
% 4 5 6;
% 7 8 9];
%
% >> diagSum = diagSumCalc(squareMatrix, 0);
%
% diagSum =
%
% 1 6 15 14 9
%
% >> diagSum = diagSumCalc(squareMatrix, 1);
%
% diagSum =
%
% 7 12 15 8 3
%
% Written by M. Phillips
% Oct. 16th, 2013
% MIT Open Source Copywrite
% Contact mphillips#hmc.edu fmi.
%
if (nargin < 2)
disp('Error on input. Needs two inputs.');
return;
end
if (LLUR0_ULLR1 ~= 0 && LLUR0_ULLR1~= 1)
disp('Error on input. Only accepts 0 or 1 as input for second condition.');
return;
end
[M, N] = size(squareMatrix);
if (M ~= N)
disp('Error on input. Only accepts a square matrix as input.');
return;
end
diagSum = zeros(1, M+N-1);
if LLUR0_ULLR1 == 1
squareMatrix = rot90(squareMatrix, -1);
end
for i = 1:length(diagSum)
if i <= M
countUp = 1;
countDown = i;
while countDown ~= 0
diagSum(i) = squareMatrix(countUp, countDown) + diagSum(i);
countUp = countUp+1;
countDown = countDown-1;
end
end
if i > M
countUp = i-M+1;
countDown = M;
while countUp ~= M+1
diagSum(i) = squareMatrix(countUp, countDown) + diagSum(i);
countUp = countUp+1;
countDown = countDown-1;
end
end
end
Cheers