Given the nxN matrix A. I want to find the running mean of the rows of the matrix. For this I have done:
mean = cumsum(A, 2);
for k = 1:N
mean(:, k) = mean(:, k)/k;
end
but for large N this takes a while. Is there a more efficient way to do this in MATLAB?
Note: zeeMonkeez's solution is fastest according to some rough benchmarks at the end of my post.
How about
N = 1000;
A = rand(N, N);
m = cumsum(A, 2);
m1 = zeros(size(m));
tic
for j = 1:1000;
for k = 1:N
m1(:, k) = m(:, k)/k;
end
end
toc
Elapsed time is 6.971112 seconds.
tic
for j = 1:1000
n = repmat(1:N, N, 1);
m2 = m./n;
end
toc
Elapsed time is 2.471035 seconds.
Here, you transform your problem into a matrix multiplication (instead of dividing element-wise, divide one matrix by the other point-wise). The matrix you would like to divide by looks like this:
[1, 2, 3, ..., N;
1, 2, .....
.
.
1, 2, .... ]
which you can get using repmat.
EDIT: BENCHMARK
bsxfun as used by #zeeMonkeez is even faster. Both, for the above case (10% difference on my system) and also for a larger matrix (N = 10000) for which case my version actually performs worst (35 sec, vs 30 from OP and 23 from zeeMonkeez's solution).
On my machine, bsxfun is even faster:
N = 1000;
A = rand(N, N);
m = cumsum(A, 2);
tic
for j = 1:1000;
m2 = bsxfun(#rdivide, m, 1:N);
end
toc
Elapsed time: 1.555507 seconds.
bsxfun avoids having to allocate memory for the divisor as repmat does.
Related
Following is the octave codes(part of kmeans)
centroidSum = zeros(K);
valueSum = zeros(K, n);
for i = 1 : m
for j = 1 : K
if(idx(i) == j)
centroidSum(j) = centroidSum(j) + 1;
valueSum(j, :) = valueSum(j, :) + X(i, :);
end
end
end
The codes work, is it possible to vectorize the codes?
It is easy to vectorize the codes without if statement,
but how could we vectorize the codes with if statement?
I assume the purpose of the code is to compute the centroids of subsets of a set of m data points in an n-dimensional space, where the points are stored in a matrix X (points x coordinates) and the vector idx specifies for each data point the subset (1 ... K) the point belongs to. Then a partial vectorization is:
centroid = zeros(K, n)
for j = 1 : K
centroid(j, :) = mean(X(idx == j, :));
end
The if is eliminated by indexing, in particular logical indexing: idx == j gives a boolean array which indicates which data points belong to subset j.
I think it might be possible to get rid of the second for-loop, too, but this would result in very convoluted, unintelligible code.
Brief introduction and solution code
This could be one fully vectorized approach based on -
accumarray: For accumulating summations as done for calulating valueSum. This also introduces a technique how one can use accumarray on a 2D matrix along a certain direction, which isn't possible in a straight-forward manner with it.
bsxfun: For calculating linear indices across all columns for matching row indices from idx.
Here's the implementation -
%// Store no. of columns in X for frequent usage later on
ncols = size(X,2);
%// Find indices in idx that are within [1:k] range, call them as labels
%// Also, find their locations in that range array, call those as pos
[pos,id] = ismember(idx,1:K);
labels = id(pos);
%// OR with bsxfun: [pos,labels] = find(bsxfun(#eq,idx(:),1:K));
%// Find all labels, i.e. across all columns of X
all_labels = bsxfun(#plus,labels(:),[0:ncols-1]*K);
%// Get truncated X corresponding to all indices matches across all columns
X_cut = X(pos,:);
%// Accumulate summations within each column based on the labels.
%// Note that accumarray doesn't accept matrices, so we were required
%// to create all_labels that had same labels within each column and
%// offsetted at constant intervals from consecutive columns
acc1 = accumarray(all_labels(:),X_cut(:));
%// Regularise accumulated array and reshape back to a 2D array version
acc1_reg2D = [acc1 ; zeros(K*ncols - numel(acc1),1)];
valueSum = reshape(acc1_reg2D,[],ncols);
centroidSum = histc(labels,1:K); %// Get labels counts as centroid sums
Benchmarking code
%// Datasize parameters
K = 5000;
n = 5000;
m = 5000;
idx = randi(9,1,m);
X = rand(m,n);
disp('----------------------------- With Original Approach')
tic
centroidSum1 = zeros(K,1);
valueSum1 = zeros(K, n);
for i = 1 : m
for j = 1 : K
if(idx(i) == j)
centroidSum1(j) = centroidSum1(j) + 1;
valueSum1(j, :) = valueSum1(j, :) + X(i, :);
end
end
end
toc, clear valueSum1 centroidSum1
disp('----------------------------- With Proposed Approach')
tic
%// ... Code from earlied mentioned section
toc
Runtime results
----------------------------- With Original Approach
Elapsed time is 1.235412 seconds.
----------------------------- With Proposed Approach
Elapsed time is 0.379133 seconds.
Not sure about its runtime performance but here's a non-convoluted vectorized implementation:
b = idx == 1:K;
centroids = (b' * X) ./ sum(b)';
Vectorizing the calculation makes a huge difference in performance. Benchmarking
The original code,
The partial vectorization from A. Donda and
The full vectorization from Tom,
gave me the following results:
Original Code: Elapsed time is 1.327877 seconds.
Partial Vectorization: Elapsed time is 0.630767 seconds.
Full Vectorization: Elapsed time is 0.021129 seconds.
Benchmarking code here:
%// Datasize parameters
K = 5000;
n = 5000;
m = 5000;
idx = randi(9,1,m);
X = rand(m,n);
fprintf('\nOriginal Code: ')
tic
centroidSum1 = zeros(K,1);
valueSum1 = zeros(K, n);
for i = 1 : m
for j = 1 : K
if(idx(i) == j)
centroidSum1(j) = centroidSum1(j) + 1;
valueSum1(j, :) = valueSum1(j, :) + X(i, :);
end
end
end
centroids = valueSum1 ./ centroidSum1;
toc, clear valueSum1 centroidSum1 centroids
fprintf('\nPartial Vectorization: ')
tic
centroids = zeros(K,n);
for k = 1:K
centroids(k,:) = mean( X(idx == k, :) );
end
toc, clear centroids
fprintf('\nFull Vectorization: ')
tic
centroids = zeros(K,n);
b = idx == 1:K;
centroids = (b * X) ./ sum(b)';
toc
Note, I added an extra line to the original code to element-wise divide valueSum1 by centroidSum1 to make the output of each type of code the same.
Finally, I know this isn't strictly an "answer", however I don't have enough reputation to add a comment, and I thought the benchmarking figures were useful to anyone who is learning MATLAB (like myself) and needs some extra motivation to master vectorization.
I have many points and I want to build distance matrix i.e. distance of every point with all of other points but I want to don't use from loop because take too time...
Is a better way for building this matrix?
this is my loop: for a setl with size: 10000x3 this method take a lot of my time :(
for i=1:size(setl,1)
for j=1:size(setl,1)
dist = sqrt((xl(i)-xl(j))^2+(yl(i)-yl(j))^2+...
(zl(i)-zl(j))^2);
distanceMatrix(i,j) = dist;
end
end
How about using some linear algebra? The distance of two points can be computed from the inner product of their position vectors,
D(x, y) = ∥y – x∥ = √ (
xT x + yT y – 2 xT y ),
and the inner product for all pairs of points can be obtained through a simple matrix operation.
x = [xl(:)'; yl(:)'; zl(:)'];
IP = x' * x;
d = sqrt(bsxfun(#plus, diag(IP), diag(IP)') - 2 * IP);
For 10000 points, I get the following timing results:
ahmad's loop + shoelzer's preallocation: 7.8 seconds
Dan's vectorized indices: 5.3 seconds
Mohsen's bsxfun: 1.5 seconds
my solution: 1.3 seconds
You can use bsxfun which is generally a faster solution:
s = [xl(:) yl(:) zl(:)];
d = sqrt(sum(bsxfun(#minus, permute(s, [1 3 2]), permute(s, [3 1 2])).^2,3));
You can do this fully vectorized like so:
n = numel(xl);
[X, Y] = meshgrid(1:n,1:n);
Ix = X(:)
Iy = Y(:)
reshape(sqrt((xl(Ix)-xl(Iy)).^2+(yl(Ix)-yl(Iy)).^2+(zl(Ix)-zl(Iy)).^2), n, n);
If you look at Ix and Iy (try it for like a 3x3 dataset), they make every combination of linear indexes possible for each of your matrices. Now you can just do each subtraction in one shot!
However mixing the suggestions of shoelzer and Jost will give you an almost identical performance performance boost:
n = 50;
xl = rand(n,1);
yl = rand(n,1);
zl = rand(n,1);
tic
for t = 1:100
distanceMatrix = zeros(n); %// Preallocation
for i=1:n
for j=min(i+1,n):n %// Taking advantge of symmetry
distanceMatrix(i,j) = sqrt((xl(i)-xl(j))^2+(yl(i)-yl(j))^2+(zl(i)-zl(j))^2);
end
end
d1 = distanceMatrix + distanceMatrix'; %'
end
toc
%// Vectorized solution that creates linear indices using meshgrid
tic
for t = 1:100
[X, Y] = meshgrid(1:n,1:n);
Ix = X(:);
Iy = Y(:);
d2 = reshape(sqrt((xl(Ix)-xl(Iy)).^2+(yl(Ix)-yl(Iy)).^2+(zl(Ix)-zl(Iy)).^2), n, n);
end
toc
Returns:
Elapsed time is 0.023332 seconds.
Elapsed time is 0.024454 seconds.
But if I change n to 500 then I get
Elapsed time is 1.227956 seconds.
Elapsed time is 2.030925 seconds.
Which just goes to show that you should always bench mark solutions in Matlab before writing off loops as slow! In this case, depending on the scale of your solution, loops could be significantly faster.
Be sure to preallocate distanceMatrix. Your loops will run much, much faster and vectorization probably isn't needed. Even if you do it, there may not be any further speed increase.
The latest versions (Since R2016b) of MATLAB support Implicit Broadcasting (See also noted on bsxfun()).
Hence the fastest way for distance matrix is:
function [ mDistMat ] = CalcDistanceMatrix( mA, mB )
mDistMat = sum(mA .^ 2).' - (2 * mA.' * mB) + sum(mB .^ 2);
end
Where the points are along the columns of the set.
In your case mA = mB.
Have a look on my Calculate Distance Matrix Project.
I want to generate a binary matrix, let say (8,1). with equal probability means four 1's and four 0s in a matrix. with different permutation of these elements total 70 combination are possible (e.g 8C4). I want these all possible combination one by one.
please help.
The straighforward answer is:
unique(perms([true(1, N / 2), false(1, N / 2)]), 'rows')
or in a fancier form:
unique(perms(sparse(1, 1:N / 2, true, 1, N)), 'rows')
where N is the length of your vector (N = 8 in your example). However, expect this solution to be very slow for large arrays.
Surprisingly, in this case a faster method is to generate all possible permutations (see here) and eliminate those that do not satisfy the desired criterion:
C = cell(N, 1); %// Preallocate memory
[C{:}] = ndgrid([true, false]); %// Generate N grids of binary values
p = cellfun(#(x){x(:)}, C); %// Convert grids to column vectors
p = [p{:}]; %// Obtain all combinations
p = p(sum(p, 2) == N / 2, :); %// Keep only desired combinations
Benchmark
N = 8;
%// Method #1 (one-liner)
tic
for k = 1:1e3
p = unique(perms(sparse(1, 1:N / 2, true, 1, N)), 'rows');
end
toc
%// Method #2
tic
for k = 1:1e3
C = cell(N, 1);
[C{:}] = ndgrid([true, false]);
p = cellfun(#(x){x(:)}, C);
p = [p{:}];
p = p(sum(p, 2) == N / 2, :);
end
toc
The results I got were:
Elapsed time is 0.858539 seconds. %// Method #1
Elapsed time is 0.803826 seconds. %// Method #2
... and for N = 10:
Elapsed time is 55.3068 seconds. %// Method #1
Elapsed time is 1.03664 seconds. %// Method #2
Not only that nchoosek fails for large values of N, it's also slower.
Here is an even faster way to do it, using the fact that you are looking for a subset of binary number representations:
b = dec2bin(1:2^N-1);
x = b-'0';
x = x(sum(x,2)==N/2,:);
The performance comparison:
N = 8;
% Dennis solution
tic
b = dec2bin(1:2^N-1);
x = b-'0';
x=x(sum(x,2)==N/2,:);
toc
% Eitan Method 2
tic
for k = 1:1e3
C = cell(N, 1);
[C{:}] = ndgrid([true, false]);
p = cellfun(#(x){x(:)}, C);
p = [p{:}];
p = p(sum(p, 2) == N / 2, :);
end
toc
Gives these timings:
Elapsed time is 0.002200 seconds.
Elapsed time is 0.594309 seconds.
Note that the resulting lines will be in different orders for both solutions.
I have double summation over m = 1:M and n = 1:N for polar point with coordinates rho, phi, z:
I have written vectorized notation of it:
N = 10;
M = 10;
n = 1:N;
m = 1:M;
rho = 1;
phi = 1;
z = 1;
summ = cos (n*z) * besselj(m'-1, n*rho) * cos(m*phi)';
Now I need to rewrite this function for accepting vectors (columns) of coordinates rho, phi, z. I tried arrayfun, cellfun, simple for loop - they work too slow for me. I know about "MATLAB array manipulation tips and tricks", but as MATLAB beginner I can't understand repmat and other functions.
Can anybody suggest vectorized solution?
I think your code is already well vectorized (for n and m). If you want this function to also accept an array of rho/phi/z values, I suggest you simply process the values in a for-loop, as I doubt any further vectorization will bring significant improvements (plus the code will be harder to read).
Having said that, in the code below, I tried to vectorize the part where you compute the various components being multiplied {row N} * { matrix N*M } * {col M} = {scalar}, by making a single call to the BESSELJ and COS functions (I place each of the row/matrix/column in the third dimension). Their multiplication is still done in a loop (ARRAYFUN to be exact):
%# parameters
N = 10; M = 10;
n = 1:N; m = 1:M;
num = 50;
rho = 1:num; phi = 1:num; z = 1:num;
%# straightforward FOR-loop
tic
result1 = zeros(1,num);
for i=1:num
result1(i) = cos(n*z(i)) * besselj(m'-1, n*rho(i)) * cos(m*phi(i))';
end
toc
%# vectorized computation of the components
tic
a = cos( bsxfun(#times, n, permute(z(:),[3 2 1])) );
b = besselj(m'-1, reshape(bsxfun(#times,n,rho(:))',[],1)'); %'
b = permute(reshape(b',[length(m) length(n) length(rho)]), [2 1 3]); %'
c = cos( bsxfun(#times, m, permute(phi(:),[3 2 1])) );
result2 = arrayfun(#(i) a(:,:,i)*b(:,:,i)*c(:,:,i)', 1:num); %'
toc
%# make sure the two results are the same
assert( isequal(result1,result2) )
I did another benchmark test using the TIMEIT function (gives more fair timings). The result agrees with the previous:
0.0062407 # elapsed time (seconds) for the my solution
0.015677 # elapsed time (seconds) for the FOR-loop solution
Note that as you increase the size of the input vectors, the two methods will start to have similar timings (the FOR-loop even wins on some occasions)
You need to create two matrices, say m_ and n_ so that by selecting element i,j of each matrix you get the desired index for both m and n.
Most MATLAB functions accept matrices and vectors and compute their results element by element. So to produce a double sum, you compute all elements of the sum in parallel by f(m_, n_) and sum them.
In your case (note that the .* operator performs element-wise multiplication of matrices)
N = 10;
M = 10;
n = 1:N;
m = 1:M;
rho = 1;
phi = 1;
z = 1;
% N rows x M columns for each matrix
% n_ - all columns are identical
% m_ - all rows are identical
n_ = repmat(n', 1, M);
m_ = repmat(m , N, 1);
element_nm = cos (n_*z) .* besselj(m_-1, n_*rho) .* cos(m_*phi);
sum_all = sum( element_nm(:) );
z - matrix of doubles, size Nx2;
x - matrix of doubles, size Nx2;
sup = x(i, :);
phi(1, i) = {#(z) exp(-g * sum((z - sup(ones([size(z, 1) 1]),:)) .^ 2, 2))};
this is a Radial Basis Function (RBF) for logistic regression. Here is the formula:
I need your advice, can i optimize this formula? coz it calls millions times, and it takes a lot of time...
It seems in your recent edits, you introduced some syntax errors, but I think I understood what you were trying to do (from the first version).
Instead of using REPMAT or indexing to repeat the vector x(i,:) to match the rows of z, consider using the efficient BSXFUN function:
rbf(:,i) = exp( -g .* sum(bsxfun(#minus,z,x(i,:)).^2,2) );
The above obviously loops over every row of x
You can go one step further, and use the PDIST2 to compute the euclidean distance between every pair of rows in z and x:
%# some random data
X = rand(10,2);
Z = rand(10,2);
g = 0.5;
%# one-line solution
rbf = exp(-g .* pdist2(Z,X,'euclidean').^2);
Now every value in the matrix: rbf(i,j) corresponds to the function value between z(i,:) and x(j,:)
EDIT:
I timed the different methods, here is the code I used:
%# some random data
N = 5000;
X = rand(N,2);
Z = rand(N,2);
g = 0.5;
%# PDIST2
tic
rbf1 = exp(-g .* pdist2(Z,X,'euclidean').^2);
toc
%# BSXFUN+loop
tic
rbf2 = zeros(N,N);
for j=1:N
rbf2(:,j) = exp( -g .* sum(bsxfun(#minus,Z,X(j,:)).^2,2) );
end
toc
%# REPMAT+loop
tic
rbf3 = zeros(N,N);
for j=1:N
rbf3(:,j) = exp( -g .* sum((Z-repmat(X(j,:),[N 1])).^2,2) );
end
toc
%# check if results are equal
all( abs(rbf1(:)-rbf2(:)) < 1e-15 )
all( abs(rbf2(:)-rbf3(:)) < 1e-15 )
The results:
Elapsed time is 2.108313 seconds. # PDIST2
Elapsed time is 1.975865 seconds. # BSXFUN
Elapsed time is 2.706201 seconds. # REPMAT
Amro has mentioned some really good methods. But the bsxfun can be further exploited by reshaping one of the matrices.
>> type r.m
N = 5000;
X = rand(N,2);
Z = rand(N,2);
g = 0.5;
%BSXFUN+loop
tic
rbf2 = zeros(N,N);
for j=1:N
rbf2(:,j) = exp( -g .* sum(bsxfun(#minus,Z,X(j,:)).^2,2) );
end
toc
tic
diffs = bsxfun(#minus, reshape(X', [1, 2, N]), Z);
dist = reshape(sum(diffs.^2, 2), [N, N]);
rbf3 = exp(-g .* dist);
toc
>> r
Elapsed time is 2.235527 seconds.
Elapsed time is 0.877833 seconds.
>> r
Elapsed time is 2.253943 seconds.
Elapsed time is 1.047295 seconds.
>> r
Elapsed time is 2.234132 seconds.
Elapsed time is 0.856302 seconds.
>> max(abs(rbf2(:) - rbf3(:)))
ans =
0
You want to subtract every row of X from every row of Z. This usually is straight forward when one of them is a vector and the other is a matrix. But if both of them are matrices, we can do this by making sure that each matrix in the volume contains just one vector. Here I chose X, but Z can be used interchangeably with X.