MATLAB: efficient way to flip the sign of eigenvectors - matlab

[V D] = eig(A)
gives eigenvectors with non-consistent sign, sometimes the first entry is positive, sometimes negative. It's OK for general purpose, but unfortunately for my job I need the signs to be consistent. For example in a series of such evaluations for different A. For example, I hope the first entries of all eigenvectors to be positive. What are some efficient ways to achieve this?
Here is what I think: An if-else statement to flip the sign (if 1st entry is negative, flip). But it seems not efficient as I have to evaluate eigenvectors many times.

First of all, in general eigenvalues and eigenvectors can be complex. This should be taken into account when we talk about sign. Here I assume you want the first element of all the eigenvectors to be real and positive.
This could be vectorized using bsxfun in this way:
[V, D] = eig(A);
% get the sign of the first row:
signs = sign(V(1, :));
% multiply all columns by the complex conjugate of sign of the first element:
V = bsxfun(#times, V, conj(signs));
Benchmarking:
If you compare the speed of this method with a loop of if statement, you will see that my suggestion is a bit slower. But to be fare, this method should be compared with an equivalent loop which is capable of processing complex values. This is the results of my test:
% the loop solution:
for ii = 1:size(V, 2)
V(:, ii) = V(:, ii) * conj(sign(V(1, ii)));
end
% A = rand(2);
------------------- With BSXFUN
Elapsed time is 0.744195 seconds.
------------------- With LOOP
Elapsed time is 0.500803 seconds.
% A = rand(10);
------------------- With BSXFUN
Elapsed time is 0.828464 seconds.
------------------- With LOOP
Elapsed time is 0.835429 seconds.
% A = rand(100);
------------------- With BSXFUN
Elapsed time is 1.421716 seconds.
------------------- With LOOP
Elapsed time is 4.286256 seconds.
As you see, it depends on your application. If you have many many small matrices, the loop solution looks more convenient. On the other hand, if you are dealing with bigger matrices, definitely a vectorized solution does the job more efficiently.

Only timing will tell what performs better, but complexity wise it is very efficient to look at the first element, and only operate on the whole vector if you find that it has the wrong sign.
So if you would have
E = rand(n)-0.5;
Then this solution:
if E(1)<0
E = -E;
end
would operate on 1+n/2 elements on average
Whilst something like
E = E * sign(E(1))
would operate on 1+n elements.
That being said, I would be surprised if you find a speed difference that is worth the optimization, so feel free to go for the most intuitive solution.

You can use the first element to determine the sign and flip the matrix:
[V, D] = eig(A * sign(A(1)));
Hope this helps.

Related

Which of the two sum calculations in Matlab / Octave is optimal on a row vector?

I'm fairly to Matlab / Octave and Machine learning, but so far I've learned you want to avoid iterative loops for summation and vectorize as much as possible.
Given a row vector like: x = [ 1,2,3,4,5]
You can calculate the sum with these two methods:
sum(x)
x * ones(length(x),1)
While my gut tells me to us the built in functions, the second options feels more 'vectorized'.
Which of the is more optimal and why? Are there tradeoffs between the two in performance vs. memory use, etc...?
In general, it seems like sum is better: the time/memory overhead of allocating the "all ones" vector does not worth it.
However, when one needs to repeatedly sum over vectors of the same length, the allocation of the vector can be done only once, thus reducing the average overhead.
On my machine:
"Caching" the all ones vector:
N=100;T=500000; x=rand(T,N);
tic;o=ones(N,1);for ii=1:T, x(ii,:)*o;end;toc
Elapsed time is 0.480388 seconds.
While using sum:
tic;for ii=1:T, sum(x(ii,:));end;toc
Elapsed time is 0.488517 seconds.
So, it's slightly faster to use the "all ones" vector method in case of repeated sums.
If you take the allocation of the "all ones" vector out of the time calculation, you end up with:
N=100;T=500000; x=rand(T,N);o=ones(N,1);
tic;for ii=1:T, x(ii,:)*o;end;toc
Elapsed time is 0.477762 seconds.
But again, you'll have to allocate it at some point...
Ok, did some more digging:
From a performance standpoint the built in sum() is much better:
x = rand(1,100000000);
%slowwwww
t = cputime; x * ones(length(x),1); e= cputime - t; e
% Faster
t = cputime; sum(x); e= cputime - t; e
I guessing using an extra vector of ones is also needless memory use. Since there is no performance gain over sum() the non-native method is far less optimal.

Multiply each sub-block with a matrix in MATLAB

I would like to multiply each sub-block of a matrix A mxn with a matrix B pxq. For example A can be divided into k sub blocks each one of size mxp.
A = [A_1 A_2 ... A_k]
The resulting matrix will be C = [A_1*B A_2*B ... A_k*B] and I would like to do it efficiently.
What I have tried until now is:
C = A*kron(eye(k),B)
Edited: Daniel I think you are right. I tried 3 different ways. Computing a kronecker product seems to be a bad idea. Even the solution with the reshape works faster than the more compact kron solution.
tic
for i=1:k
C1(:,(i-1)*q+1:i*q) = A(:,(i-1)*p+1:i*p)*B;
end
toc
tic
C2 = A*kron(eye(k),B);
toc
tic
A = reshape(permute(reshape(A,m,p,[]),[1 3 2]),m*k,[]);
C3 = A*B;
C3 = reshape(permute(reshape(C3,m,k,[]),[1 3 2]),m,[]);
toc
When I look at your matrix multiplication code, you have perfectly optimized code within the loop. You can't beat matrix multiplication. Everything you could cut down is the overhead for the iteration, but compared to the long runtime of a matrix multiplication the overhead has absolutely no influence.
What you attempted to do would be the right strategy when the operation within the loop is trivial but the loop is iterated many times. If you take the following parameters, you will notice that your permute solution has actually it's strength, but not for your problem dimensions:
q=1;p=1;n=1;m=1;
k=10^6
Kron totally fails. Your permute solution takes 0.006s while the loop takes 1.512s

Do inner workings of Matlab do Full Matrix Multiplication when Matrix Multiplication is Argument for Trace?

Does Matlab do a full matrix multiplication when a matrix multiplication is given as an argument to the trace function?
For example, in the code below, does A*B actually happen, or are the columns of B dotted with the rows of A, then summed? Or does something else happen?
A = [2,2;2,2];
B = eye(2);
f = trace(A*B);
Yes, MATLAB calculates the product, but you can avoid it!
First, let's see what MATLAB does if you do f = trace(A*B):
I think the picture from my Performance monitor says it all really. The first bump is when I created a large A = 2*ones(n), the second, very little bump is for the creation of B = eye(n), and the last bump is where f = trace(A*B) is calculated.
Now, let's see that you get if you do it manually:
If you do it manually, you can save a lot of memory, and it's much faster.
tic
n = 6e3;
A = rand(n);
B = rand(n);
f = trace(A*B);
toc
pause(10)
tic
C(n) = 0;
for ii = 1:n
C(ii) = sum(A(ii,:)*B(:,ii));
end
g = sum(C);
toc
abs(f-g) < 1e-10
Elapsed time is 11.982804 seconds.
Elapsed time is 0.540285 seconds.
ans =
1
Now, as you asked about in the comments: "Is this still true if you use it in a function where optimization can kick in?"
This depends on what you mean here, but as a quick example:
Calculating x = inv(A)*b can be done in a few different ways. If you do:
x = A\b;
MATLAB will chose an algorithm that's best suited for your particular matrix/vector. There are many different alternatives here, depending on the structure of the matrix: is it triangular, hermatian, sparse...? Often it's a upper/lower triangulation. I can pretty much guarantee you that you can't write a code in MATLAB that can outperform MATLABs builtin functions here.
However, if you calculate the same thing this way:
x = inv(A)*b;
MATLAB will actually calculate the inverse of A, then multiply it by b, even though the inverse is not stored in the workspace afterwards. This is much slower, and can also be inaccurate. (In the A\b approach, MATLAB will, if necessary create a permutation matrix to ensure numerical stability.

MATLAB - Efficient row-vector*Matrix*column-vector

I'm working on a piece of software in MATLAB and I believe I've reached the limit of my knowledge when it comes to optimisation and efficiency. Here's where the expertise of the people on StackOverflow might be helpful.
Using MATLAB's profiler, I've found that the last inefficient line of code is a multiplication of the following form:
function [energy] = getEnergy(S,W)
energy = -(S*W*S');
end
S is a 1xN row vector, W is an NxN matrix (it's not just a diagonal matrix though), and S' is a Nx1 column vector, whose multiplication returns a number.
I understand that this is a primitive operation, but I was wondering whether there is any way to speed this up.
I tried searching Google etc, but unfortunately I do not know the right keywords to search for. I apologise if this is a duplicate.
Thanks in advance.
Your implementation is correct, and the fastest.
You can save ~20-30% of computation time by performing it inside the main code, without call to the function.
>> S = randn(1, 500);
>> W = randn(500);
>> tic; for k = 1 : 10000, e = -(S * W * S'); end; toc
Elapsed time is 0.321595 seconds.
If the bottleneck stems from the fact that you need to repeat this computation for a LOT of different vectors S, then you can do the following vectorization:
% s is k-by-N matrix of k row vectors
energy = sum( ( s * W ) .* s, 2 ); % note the .* in the middle!

Octave/Matlab: Efficient calc of Frobenius inner product?

I have two matrices A and B and what I want to get is:
trace(A*B)
If I'm not mistaken this is called Frobenius inner product.
My concern here is about efficiency. I'm just afraid that this strait-forward approach will first do the whole multiplication (my matrices are thousands of rows/cols) and only then take the trace of the product, while the operation I really need is much simplier. Is there a function or a syntax to do this efficiently?
Correct...summing the element-wise products will be quicker:
n = 1000
A = randn(n);
B = randn(n);
tic
sum(sum(A .* B));
toc
tic
sum(diag(A * B'));
toc
Elapsed time is 0.010015 seconds.
Elapsed time is 0.130514 seconds.
sum(sum(A.*B)) avoids doing the full matrix multiplication
How about using vector multiplication?
(A(:)')*B(:)
Run time check
Comparing four options with A and B of size 1000-by-1000:
1. vector inner product: A(:)'*B(:) (this answer) took only 0.0011 sec.
2. Using element wise multiplication sum(sum(A.*B)) (John's answer) took 0.0035 sec.
3. Trace trace(A*B') (proposed by OP) took 0.054 sec.
4. Sum of diagonal sum(diag(A*B')) (option rejected by John) took 0.055 sec.
Take home message: Matlab is extremely efficient when it comes to matrix/vector product. Using vector inner product is x3 times faster even than the efficient element-wise multiplication solution.
Benchmark code
Code used to provide the run time checks
t=zeros(1,4);
n=1000; % size of matrices
it=100; % average results over XX trails
for ii=1:it,
% random inputs
A=rand(n);
B=rand(n);
% John's rejected solution
tic;
n1=sum(diag(A*B'));
t(1)=t(1)+toc;
% element-wise solution
tic;
n2=sum(sum(A.*B));
t(2)=t(2)+toc;
% MOST efficient solution - using vector product
tic;
n3=A(:)'*B(:);
t(3)=t(3)+toc;
% using trace
tic;
n4=trace(A*B');
t(4)=t(4)+toc;
% make sure everything is correct
assert(abs(n1-n2)<1e-8 && abs(n3-n4)<1e-8 && abs(n1-n4)<1e-8);
end;
t./it
You can now run this benchmark in a click.