I need to do function that works like this :
N1 = size(X,1);
N2 = size(Xtrain,1);
Dist = zeros(N1,N2);
for i=1:N1
for j=1:N2
Dist(i,j)=D-sum(X(i,:)==Xtrain(j,:));
end
end
(X and Xtrain are sparse logical matrixes)
It works fine and passes the tests, but I believe it's not very optimal and well-written solution.
How can I improve that function using some built Matlab functions? I'm absolutely new to Matlab, so I don't know if there really is an opportunity to make it better somehow.
You wanted to learn about vectorization, here some code to study comparing different implementations of this pair-wise distance.
First we build two binary matrices as input (where each row is an instance):
m = 5;
n = 4;
p = 3;
A = double(rand(m,p) > 0.5);
B = double(rand(n,p) > 0.5);
1. double-loop over each pair of instances
D0 = zeros(m,n);
for i=1:m
for j=1:n
D0(i,j) = sum(A(i,:) ~= B(j,:)) / p;
end
end
2. PDIST2
D1 = pdist2(A, B, 'hamming');
3. single-loop over each instance against all other instances
D2 = zeros(m,n);
for i=1:n
D2(:,i) = sum(bsxfun(#ne, A, B(i,:)), 2) ./ p;
end
4. vectorized with grid indexing, all against all
D3 = zeros(m,n);
[x,y] = ndgrid(1:m,1:n);
D3(:) = sum(A(x(:),:) ~= B(y(:),:), 2) ./ p;
5. vectorized in third dimension, all against all
D4 = sum(bsxfun(#ne, A, reshape(B.',[1 p n])), 2) ./ p;
D4 = permute(D4, [1 3 2]);
Finally we compare all methods are equal
assert(isequal(D0,D1,D2,D3,D4))
Related
I'm running a Matlab code in the HPC of my university. I have two versions of the code. The second version, despite generating a smaller array, seems to require more memory. I would like your help to understand if this is in fact the case and why.
Let me start from some preliminary lines:
clear
rng default
%Some useful components
n=7^4;
vectors{1}=[1,20,20,20,-1,Inf,-Inf];
vectors{2}=[-19,19,19,19,-20,Inf,-Inf];
vectors{3}=[-19,0,0,0,-20,Inf,-Inf];
vectors{4}=[-19,0,0,0,-20,Inf,-Inf];
T_temp = cell(1,4);
[T_temp{:}] = ndgrid(vectors{:});
T_temp = cat(4+1, T_temp{:});
T = reshape(T_temp,[],4); %all the possible 4-tuples from vectors{1}, ..., vectors{4}
This is the first version 1 of the code: I construct the matrix D1 listing all possible pairs of unordered rows from T
indices_pairs=pairIndices(n);
D1=[T(indices_pairs(:,1),:) T(indices_pairs(:,2),:)];
This is the second version of the code: I construct the matrix D2 listing a random draw of m=10^6 unordered pairs of rows from T
m=10^6;
p=n*(n-1)/2;
random_indices_pairs = randperm(p, m).';
[C1, C2] = myind2ind (random_indices_pairs, n);
indices_pairs=[C1 C2];
D2=[T(indices_pairs(:,1),:) T(indices_pairs(:,2),:)];
My question: when generating D2 the HPC goes out of memory. When generating D1 the HPC works fine, despite D1 being a larger array than D2. Why is that the case?
These are complementary functions used above:
function indices = pairIndices(n)
[y, x] = find(tril(logical(ones(n)), -1)); %#ok<LOGL>
indices = [x, y];
end
function [R , C] = myind2ind(ii, N)
jj = N * (N - 1) / 2 + 1 - ii;
r = (1 + sqrt(8 * jj)) / 2;
R = N -floor(r);
idx_first = (floor(r + 1) .* floor(r)) / 2;
C = idx_first-jj + R + 1;
end
I have two vectors, say A of size nx1, and B of size 1xm. I want to create a result matrix C (nxm) from a non-linear formula so that
C(i,j) = A(i)/(A(i)+B(j)).
I know I can do this with a loop such as:
for i=1:n,
C(i,1:m)=A(i)./(A(i)+B(1:m));
end
but is it possible in some way to do this without using a loop?
EDIT: Thanks for your answers! As a small addition, before I saw them a friend suggested the following solution:
C = A*ones(1,m)./(ones(n,1)*B+A*ones(1,m))
If you are on MATLAB R2016a or earlier you'll want to use bsxfun to accomplish this
result = bsxfun(#rdivide, A, bsxfun(#plus, A, B));
If you are on R2016b or newer, then there is implicit expansion which allows you to remove bsxfun and just apply the element-operators directly
result = A ./ (A + B);
Benchmark
Here is a real benchmark using timeit to compare the execution speed of using bsxfun, repmat, implicit broadcasting and a for loop. As you can see from the results, the bsxfun and implicit broadcasting methods yield the fastest execution time.
function comparision()
sz = round(linspace(10, 5000, 30));
times1 = nan(size(sz));
times2 = nan(size(sz));
times3 = nan(size(sz));
times4 = nan(size(sz));
for k = 1:numel(sz)
A = rand(sz(k), 1);
B = rand(1, sz(k));
times1(k) = timeit(#()option1(A,B));
A = rand(sz(k), 1);
B = rand(1, sz(k));
times2(k) = timeit(#()option2(A,B));
A = rand(sz(k), 1);
B = rand(1, sz(k));
times3(k) = timeit(#()option3(A,B));
A = rand(sz(k), 1);
B = rand(1, sz(k));
times4(k) = timeit(#()option4(A,B));
end
figure
p(1) = plot(sz, times1 * 1000, 'DisplayName', 'bsxfun');
hold on
p(2) = plot(sz, times2 * 1000, 'DisplayName', 'repmat');
p(3) = plot(sz, times3 * 1000, 'DisplayName', 'implicit');
p(4) = plot(sz, times4 * 1000, 'DisplayName', 'for loop');
ylabel('Execution time (ms)')
xlabel('Elements in A')
legend(p)
end
function C = option1(A,B)
C = bsxfun(#rdivide, A, bsxfun(#plus, A, B));
end
function C = option2(A,B)
m = numel(B);
n = numel(A);
C = repmat(A,1,m) ./ (repmat(A,1,m) + repmat(B,n,1));
end
function C = option3(A, B)
C = A ./ (A + B);
end
function C = option4(A, B)
m = numel(B);
n = numel(A);
C = zeros(n, m);
for i=1:n,
C(i,1:m)=A(i)./(A(i)+B(1:m));
end
end
See this answer for more information comparing implicit expansion and bsxfun.
Implicit expansion is the way to go if you have 2016b or newer, as mentioned by Suever. Another approach without that is to do element-wise operations after making A and B the same size as C, using repmat...
A1 = repmat(A,1,m);
B1 = repmat(B,n,1);
C = A1 ./ (A1 + B1);
Or in 1 line...
C = repmat(A,1,m) ./ (repmat(A,1,m) + repmat(B,n,1));
As a benchmark, I ran your loop method, the above repmat method, and Suever's bsxfun method for m = 1000, n = 100, averaging over 1,000 runs each:
Using for loop 0.00290520 sec
Using repmat 0.00014693 sec
Using bsxfun 0.00016402 sec
So for large matrices, repmat and bsxfun are comparable with repmat edging it. For small matrices though, just looping can be quicker than both, especially with a small n as that's your loop variable!
It's worth testing the given methods for your specific use case, as the timing results seem fairly variable depending on the inputs.
I am trying to calculate Pearson coefficients between all pair combinations of my variables of all my samples.
Say i have an m*n matrix where m are the variables and n are the samples
i want to calculate for each variable of my data what is the correlation to every other variable.
So, i managed to do that with nested loops:
X = rand[1000 100];
for i = 1:1000
base = X(i, :);
for j = 1:1000
target = X(j, :);
correlation = corrcoef(base, target);
correlation = correlation(2, 1);
corData(1, j) = correlation
end
totalCor(i, :) = corData
end
and it works, but takes too much time to run
I am trying to find a way to run the corrcoef function on a row basis, meaning maybe to create an additional matrix with repmat of the base values and correlate to the X data using some FUN function.
Could not figure out how to use the fun with inputs from to arrays, running between individuals lines/columns
help will be appreciated
This post involves a bit of hacking, so bear with it!
Stage #0 To start off, we have -
for i = 1:N
base = X(i, :);
for j = 1:N
target = X(j, :);
correlation = corrcoef(base, target);
correlation = correlation(2, 1)
corData(1, j) = correlation;
end
end
Stage #1 From the documentation of corrcoef in its source code :
If C is the covariance matrix, C = COV(X), then CORRCOEF(X) is the
matrix whose (i,j)'th element is : C(i,j)/SQRT(C(i,i)*C(j,j)).
After hacking into the code of covariance, we see that for the default case of one input, the covariance formula is simply -
[m,n] = size(x);
xc = bsxfun(#minus,x,sum(x,1)/m);
xy = (xc' * xc) / (m-1);
Thus, mixing the two definitions and putting them into the problem at hand, we have -
m = size(X,2);
for i = 1:N
base = X(i, :);
for j = 1:N
target = X(j, :);
BT = [base(:) target(:)];
xc = bsxfun(#minus,BT,sum(BT,1)/m);
C = (xc' * xc) / (m-1); %//'
corData = C(2,1)/sqrt(C(2,2)*C(1,1))
end
end
Stage #2 This is the final stage where we use the real fun aka bsxfun to kill all loops, like so -
%// Broadcasted subtract of each row by the average of it.
%// This corresponds to "xc = bsxfun(#minus,BT,sum(BT,1)/m)"
p1 = bsxfun(#minus,X,mean(X,2));
%// Get pairs of rows from X and get the dot product.
%// Thus, a total of "N x N" such products would be obtained.
p2 = sum(bsxfun(#times,permute(p1,[1 3 2]),permute(p1,[3 1 2])),3);
%// Scale them down by "size(X,2)-1".
%// This was for the part : "C = (xc' * xc) / (m-1)".
p3 = p2/(size(X,2)-1);
%// "C(2,2)" and "C(1,1)" are diagonal elements from "p3", so store them.
dp3 = diag(p3);
%// Get "sqrt(C(2,2)*C(1,1))" by broadcasting elementwise multiplication
%// of "dp3". Finally do elementwise division of "p3" by it.
totalCor_out = p3./sqrt(bsxfun(#times,dp3,dp3.'));
Benchmarking
This section compares the original approach against the proposed one and also verifies the output. Here's the benchmarking code -
disp('---------- With original approach')
tic
X = rand(1000,100);
corData = zeros(1,1000);
totalCor = zeros(1000,1000);
for i = 1:1000
base = X(i, :);
for j = 1:1000
target = X(j, :);
correlation = corrcoef(base, target);
correlation = correlation(2, 1);
corData(1, j) = correlation;
end
totalCor(i, :) = corData;
end
toc
disp('---------- With the real fun aka BSXFUN')
tic
p1 = bsxfun(#minus,X,mean(X,2));
p2 = sum(bsxfun(#times,permute(p1,[1 3 2]),permute(p1,[3 1 2])),3);
p3 = p2/(size(X,2)-1);
dp3 = diag(p3);
totalCor_out = p3./sqrt(bsxfun(#times,dp3,dp3.')); %//'
toc
error_val = max(abs(totalCor(:)-totalCor_out(:)))
Output -
---------- With original approach
Elapsed time is 186.501746 seconds.
---------- With the real fun aka BSXFUN
Elapsed time is 1.423448 seconds.
error_val =
4.996e-16
I have an n x p matrix - mX which is composed of n points in R^p.
I have another m x p matrix - mY which is composed of m reference points in R^p.
I would like to create an n x m matrix - mD which is the Mahalanobis Distance matrix.
D(i, j) means the Mahalanobis Distance between point i in mX, mX(i, :) and point j in mY, mY(j, :).
Namely, is computes the following:
mD(i, j) = (mX(i, :) - mY(j, :)) * inv(mC) * (mX(i, :) - mY(j, :)).';
Where mC is the given Mahalanobis Distance PSD Matrix.
It is easy to be done in a loop, is there a way to vectorize it?
Namely, is the a function which its inputs are mX, mY and mC and its output is mD and fully vectorized without using any MATLAB toolbox?
Thank You.
Approach #1
Assuming infinite resources, here's one vectorized solution using bsxfun and matrix-multiplication -
A = reshape(bsxfun(#minus,permute(mX,[1 3 2]),permute(mY,[3 1 2])),[],p);
out = reshape(diag(A*inv(mC)*A.'),n,m);
Approach #2
Here's a comprise solution trying to reduce the loop complexity -
A = reshape(bsxfun(#minus,permute(mX,[1 3 2]),permute(mY,[3 1 2])),[],p);
imC = inv(mC);
out = zeros(n*m,1);
for ii = 1:n*m
out(ii) = A(ii,:)*imC*A(ii,:).';
end
out = reshape(out,n,m);
Sample run -
>> n = 3; m = 4; p = 5;
mX = rand(n,p);
mY = rand(m,p);
mC = rand(p,p);
imC = inv(mC);
>> %// Original solution
for i = 1:n
for j = 1:m
mD(i, j) = (mX(i, :) - mY(j, :)) * inv(mC) * (mX(i, :) - mY(j, :)).'; %//'
end
end
>> mD
mD =
-8.4256 10.032 2.8929 7.1762
-44.748 -4.3851 -13.645 -9.6702
-4.5297 3.2928 0.11132 2.5998
>> %// Approach #1
A = reshape(bsxfun(#minus,permute(mX,[1 3 2]),permute(mY,[3 1 2])),[],p);
out = reshape(diag(A*inv(mC)*A.'),n,m); %//'
>> out
out =
-8.4256 10.032 2.8929 7.1762
-44.748 -4.3851 -13.645 -9.6702
-4.5297 3.2928 0.11132 2.5998
>> %// Approach #2
A = reshape(bsxfun(#minus,permute(mX,[1 3 2]),permute(mY,[3 1 2])),[],p);
imC = inv(mC);
out1 = zeros(n*m,1);
for ii = 1:n*m
out1(ii) = A(ii,:)*imC*A(ii,:).'; %//'
end
out1 = reshape(out1,n,m);
>> out1
out1 =
-8.4256 10.032 2.8929 7.1762
-44.748 -4.3851 -13.645 -9.6702
-4.5297 3.2928 0.11132 2.5998
Instead if you had :
mD(j, i) = (mX(i, :) - mY(j, :)) * inv(mC) * (mX(i, :) - mY(j, :)).';
The solutions would translate to the versions listed next.
Approach #1
A = reshape(bsxfun(#minus,permute(mY,[1 3 2]),permute(mX,[3 1 2])),[],p);
out = reshape(diag(A*inv(mC)*A.'),m,n);
Approach #2
A = reshape(bsxfun(#minus,permute(mY,[1 3 2]),permute(mX,[3 1 2])),[],p);
imC = inv(mC);
out1 = zeros(m*n,1);
for i = 1:n*m
out(i) = A(i,:)*imC*A(i,:).'; %//'
end
out = reshape(out,m,n);
Sample run -
>> n = 3; m = 4; p = 5;
mX = rand(n,p); mY = rand(m,p); mC = rand(p,p); imC = inv(mC);
>> %// Original solution
for i = 1:n
for j = 1:m
mD(j, i) = (mX(i, :) - mY(j, :)) * inv(mC) * (mX(i, :) - mY(j, :)).'; %//'
end
end
>> mD
mD =
0.81755 0.33205 0.82254
1.7086 1.3363 2.4209
0.36495 0.78394 -0.33097
0.17359 0.3889 -1.0624
>> %// Approach #1
A = reshape(bsxfun(#minus,permute(mY,[1 3 2]),permute(mX,[3 1 2])),[],p);
out = reshape(diag(A*inv(mC)*A.'),m,n); %//'
>> out
out =
0.81755 0.33205 0.82254
1.7086 1.3363 2.4209
0.36495 0.78394 -0.33097
0.17359 0.3889 -1.0624
>> %// Approach #2
A = reshape(bsxfun(#minus,permute(mY,[1 3 2]),permute(mX,[3 1 2])),[],p);
imC = inv(mC);
out1 = zeros(m*n,1);
for i = 1:n*m
out1(i) = A(i,:)*imC*A(i,:).'; %//'
end
out1 = reshape(out1,m,n);
>> out1
out1 =
0.81755 0.33205 0.82254
1.7086 1.3363 2.4209
0.36495 0.78394 -0.33097
0.17359 0.3889 -1.0624
Here is one solution that eliminates one loop
function d = mahalanobis(mX, mY)
n = size(mX, 2);
m = size(mY, 2);
data = [mX, mY];
mc = cov(transpose(data));
dist = zeros(n,m);
for i = 1 : n
diff = repmat(mX(:,i), 1, m) - mY;
dist(i,:) = sum((mc\diff).*diff , 1);
end
d = sqrt(dist);
end
You would invoke it as:
d = mahalanobis(transpose(X),transpose(Y))
Reduce to L2
It seems that Mahalanobis Distance can be reduced to ordinary L2 distance if you are allowed to preprocess matrix mC and you are not afraid of numerical differences.
First of all, compute Cholesky decomposition of mC:
mR = chol(mC) % C = R^t * R, where R is upper-triangular
Now we can use these factors to reformulate Mahalanobis Distance:
(Xi-Yj) * inv(C) * (Xi-Yj)^t = || (Xi-Yj) inv(R) ||^2 = ||TXi - TYj||^2
where: TXi = Xi * inv(R)
TYj = Yj * inv(R)
So the idea is to transform points Xi, Yj to TXi, TYj first, and then compute euclidean distances between them. Here is the algorithm outline:
Compute mR - Cholesky factor of covariance matrix mC (takes O(p^3) time).
Invert triangular matrix mR (takes O(p^3) time).
Multiply both mX and mY by inv(mR) on the right (takes O(p^2 (m+n)) time).
Compute squared L2 distances between pairs of points (takes O(m n p) time).
Total time is O(m n p + (m + n) p^2 + p^3) versus original O(m n p^2). It should work faster when 1 << p << n,m. In such case step 4 would takes most of the time and should be vectorized.
Vectorization
I have little experience of MATLAB, but quite a lot of SIMD vectorization on x86 CPUs. In raw computations, it would be enough to vectorize along one sufficiently large array dimension, and make trivial loops for the other dimensions.
If you expect p to be large enough, it may probably be OK to vectorize along coordinates of points, and make two nested loops for i <= n and j <= m. That's similar to what #Daniel posted.
If p is not sufficiently large, you can vectorize along one of the point sequences instead. This would be similar to solution posted by #dpmcmlxxvi: you have to subtract single row of one matrix from all the rows of the second matrix, then compute squared norms of the resulting rows. Repeat n times (
or m times).
As for me, full vectorization (which means rewriting with matrix operations instead of loops in MATLAB) does not sound like a clever performance goal. Most likely partially vectorized solutions would be optimally fast.
I came to the conclusion that vectorizing this problem is not efficient. My best idea for vectorizing this problem would require m x n x p x p working memory, at least if everything is processed at once. This means with n=m=p=152 the code would already require 4GB Ram. At these dimensions, my system can run the loop in less than a second:
mD=zeros(size(mX,1),size(mY,1));
ImC=inv(mC);
for i=1:size(mX,1)
for j=1:size(mY,1)
d=mX(i, :) - mY(j, :);
mD(i, j) = (d) * ImC * (d).';
end
end
I write a code for linear feedback shift register.My code is in below:
X=5712;
D1(1)=0;
D2(1)=0;
D3(1)=0;
D4(1)=0;
D5(1)=0;
D6(1)=1;
for i=1:X-1
D6(i+1)=D1(i);
D5(i+1)=xor(D1(i),D6(i));
D4(i+1)=D5(i);
D3(i+1)=D4(i);
D2(i+1)=D3(i);
D1(i+1)=D2(i);
end
In my code i can only use 6 shift register.I know for degree,n=2,3,4,6,7,15,22, the polynomial is x^n+x+1.As the polynomial is same for those degrees so i want to write a common code for all.
Matlab experts Please need your help.
Your problem is that you are making separate vectors for each register. Rather make a single matrix (i.e. D replaces all of your D1, D2, ..., Dn) so that you can loop:
X = 20;
n = 6;
D = zeros(X, n);
D(1,n) = 1;
for ii = 1:X-1
D(ii+1, 1:n-2) = D(ii, 2:n-1);
D(ii+1, n-1) = xor(D(ii,1), D(ii,n));
D(ii+1, n) = D(ii, 1);
end
E = D(:, end:-1:1)