In Matlab, I have to study the eventual existence of common eigenvectors basis between 2 Fisher matrices FISH_sp and FISH_xc of size 7x7 and diagonalisable.
I get from my computation the following result:
>> x=null(FISH_sp*FISH_xc-FISH_xc*FISH_sp)
x =
-0.0085
-0.0048
-0.2098
0.9776
-0.0089
-0.0026
0.0109
In this result, It appears that condition to get a common eigenvectors basis on commutator is true. But I need to further examine the mathematics. If one gets a single column vector, then nullspace of the commutator is 1-dimensional as far as Matlab can tell. With that result, one can think about how to verify that vector is indeed an eigenvector of FISH_sp and FISH_xc down to a small tolerance.
But I don't know how to introduce this tolerance in a small Matlab script.
All I have done for instant is :
x=null(FISH_sp*FISH_xc-FISH_xc*FISH_sp)
How can I introduce tolerance in the checking of eigenvector x as being really an eigenvector given a tolerance tol.
And what about the eigenvalues ? : normally, they should not equal to D1 in [V1, D1] =eig(FISH_sp)and not equal to D2 in [V2, D2] =eig(FISH_xc) ? I said they shouldn't since we have to express them in a new and different basis of eigenvectors : then I call these 2 news diagonal matrices D1_new and D2_new. So, I could write :
If I have a passing matrix of all the common eigen vectors basis called P, then one has :
F = P (D1_new + D2_new) p^-1
This endomorphism F is wanted with this expression (to respect the Maximum Likelihood Estimator = MLE).
the problem for instant is that I have only one eigen vector x and not the entire passing matrix P of new eigenvectors. How can I build this passing matrix P from only the single x values of common eigen vector mentioned above ?
Related
I am looking for solving a generalized eigenvectors and eigen value problem in Matlab. For this, I have tested 2 methods.
if Generalized problem is formulated as :
Then, we could multiply by B^(-1) on each side, such as :
So, from a theorical point of view, it is the simple and classical eigen values problem.
Finally, in Matlab, I simply did, with A=FISH_sp and B=FISH_xc :
[Phi, Lambda] = eig(inv(FISH_xc)*FISH_sp);
But results are not correct when I make after a simple Fisher synthesis (constraints are too bad and also making appear nan values. I don't know why I don't get the same results than the second ones below.
the second method comes from the following paper.
To summarize, the algorithm used is described on page 7. I have followed all the steps of this algorithm and it seems to give better results when I make the Fisher synthesis.
Here the interested part (sorry, I think Latex is not available on stakoverflow) :
Here my little Matlab script for this method :
% Diagonalize A = FISH_sp and B = Fish_xc
[V1,D1] = eig(FISH_sp);
[V2,D2] = eig(FISH_xc);
% Applying each step of algorithm 1 on page 7
phiB_bar = V2*(D2.^(0.5)+1e-10*eye(7))^(-1);
barA = inv(phiB_bar)*FISH_sp*phiB_bar;
[phiA, vA] = eig(barA);
Phi = phiB_bar*phiA;
So at the end, I find phi eigenvectors matrix (phi) and lambda diagonal matrix (D1).
Now, I would like to do the link between this generalized problem and the eventual common eigenvectors between A and B matrices (respectively Fish_sp and Fish_xc). Is there a way to perform this?
Indeed, what I have done up to now is to to find a parallel relation between A*Phi and B*Phi, linked by Lambda diagonal matrix. Maybe, we could arrange this relation such that :
A*Phi'=Phi'*Lambda_A'
and
B*Phi'=Phi'*Lambda_B'
From a numerical point of view, why don't I get the same results between the method in 1) and the method in 3) ? I mean about the Phi eigen vectors matrix and the Lambda diagonal matrix.
However, this is the same formulation.
EDIT :
I have wrong results if I want to say that phi diagonalizes both A=FISH_sp and B=FISH_xc matrices.
Indeed, by doing :
% Marginalizing over uncommon parameters between the two matrices
COV_GCsp_first = inv(FISH_GCsp);
COV_XC_first = inv(FISH_XC);
COV_GCsp = COV_GCsp_first(1:N,1:N);
COV_XC = COV_XC_first(1:N,1:N);
% Invert to get Fisher matrix
FISH_sp = inv(COV_GCsp);
FISH_xc = inv(COV_XC);
% Diagonalize
[V1,D1] = eig(FISH_sp);
[V2,D2] = eig(FISH_xc);
% Build phi matrix
% V2 corresponds to eigen vectors of FISH_xc
phiB_bar = V2*diag(diag(D2.^(-0.5)));
% DEBUG : check identity matrix => OK, Identity matrix found !
id = (phiB_bar')*FISH_xc*phiB_bar
% phi matrix
barA = (phiB_bar')*FISH_sp*phiB_bar
[phiA, vA] = eig(barA);
phi = phiB_bar*phiA;
% Check eigen values : OK, columns of eigenvalues found !
FISH_sp*V1./V1
% Check eigen values : OK, columns of eigenvalues found !
FISH_xc*V2./V2
% Check if phi diagolize FISH_sp : NOT OK, not identical eigenvalues
FISH_sp*phi./phi
% Check if phi diagolize FISH_sp : NOT OK, not identical eigenvalues
FISH_xc*phi./phi
So, I don't find that matrix of eigenvectors Phi diagonalizes A and B since the eigenvalues expected are not columns of identical values.
By the way, I find the eigenvalues D1 and D2 coming from :
[V1,D1] = eig(FISH_sp);
[V2,D2] = eig(FISH_xc);
% Check eigen values : OK, columns of eigenvalues D1 found !
FISH_sp*V1./V1
% Check eigen values : OK, columns of eigenvalues D2 found !
FISH_xc*V2./V2
How could I fix this wrong result (I am talking about the ratios :
FISH_sp*phi./phi
FISH_xc*phi./phi
which don't give same values for a given column of FISH_sp and FISH_xc)
)
?? In the paper, they say that phi diagonalizes A=FISH_sp and B=FISH_xc but I can't reproduce it.
If someone could see where is my error ...
You defined barA = inv(phiB_bar)*FISH_sp*phiB_bar. From Eq. (39) in the manuscript it looks like it should be barA = transpose(phiB_bar)*FISH_sp*phiB_bar instead.
Also, your mehtod 1 fails when B is singular (an inverse does not exist). MATLAB's eig(A,B) however should handle also singular Bs if memory serves me well.
I have a 1500x1500 covariance matrix of which I am trying to calculate the determinant for EM-ML method. The covariance matrix is obtained by finding the SIGMA matrix and then passing it into the nearestSPD library (Link) to make the matrix positive definite . In this case the matrix is always singular. Another method I tried was of manually generating a positive definite matrix using A'*A technique. (A was taken as a 1600x1500 matrix). This always gives me the determinant as infinite. Any idea on how I can get a positive definite matrix with a finite determinant?
Do you actually need the determinant, or the log of the determinant?
For example if you are computing a log likelihood of gaussians then what enters into the log likelihood is the log of the determinant. In high dimensions determinants mey not fit in a double, but its log most likely will.
If you perform a cholesky factorisation of the covariance C, with (lower triangular) factor L say so that
C = L*L'
then
det C = det(L) * det( L') = det(L) * det(L)
But the determinant of a lower triangular matrix is the product of its diagonal elements, so, taking logs above we get:
log det C = 2*Sum{ i | log( L[i,i])}
(In response to a comment)
Even if you need to calculate a gaussian pdf, it is better to calculate the log of that and exponentiate only when you need to. For example a d dimenions gaussian with covariance C (which has a cholesky factor L) and mean 0 (purely to save typing) is:
p(x) = exp( -0.5*x'*inv(C)*x) /( sqrt( pow(2pi,d) * det(C))
so
log p(x) = -0.5*x'*inv(C)*x - 0.5*d*log(2pi) - 0.5*log(det(C))
which can also be written
log p(x) = -0.5*y'*y - 0.5*d*log(2pi) - log(det(L))
where
y = inv(L)*x
I have got a problem like A*x=lambda*x, where A is of order d*d, x is of order d*c and lambda is a constant. A and lambda are known and the matrix x is unknown.
Is there any way to solve this problem in matlab?? (Like eigen values but x is a d*c matrix instead of being a vector).
If I've understood you correctly, there will not necessarily be any solutions for x. If A*x=lambda*x, then any column y of x satisfies A*y=lambda*y, so the columns of x are simply eigenvectors of A corresponding to the eigenvalue lambda, and there will only be any solutions if lambda is in fact an eigenvalue.
From the documentation:
[V,D] = eig(A) produces matrices of eigenvalues (D) and eigenvectors
(V) of matrix A, so that A*V = V*D. Matrix D is the canonical form of
A — a diagonal matrix with A's eigenvalues on the main diagonal.
Matrix V is the modal matrix — its columns are the eigenvectors of A.
You can use this to check if lambda is an eigenvalue, and find any corresponding eigenvectors.
You can transform this problem. Write x as vector by by using x(:) (has size d*c x 1). Then A can be rewritten to a d*c x d*c matrix which has c versions of A along the diagonal.
Now it's a simple eigenvalue problem.
Its actually trivial. Your requirement is that A*X = lambda*X, where X is an array. Effectively, look at what happens for a single column of X. If An array X exists, then it is true that
A*X(:,i) = lambda*X(:,i)
And this must be true for the SAME value of lambda for all columns of X. Essentially, this means that X(:,i) is an eigenvector of A, with corresponding eigenvalue lambda. More importantly, it means that EVERY column of X has the same eigenvalue as every other column.
So a trivial solution to this problem is to simply have a matrix X with identical columns, as long as that column is an eigenvector of A. If an eigenvalue has multiplicity greater than one (therefore there are multiple eigenvectors with the same eigenvalue) then the columns of X may be any linear combination of those eigenvectors.
Try it in practice. I'll pick some simple matrix A.
>> A = [2 3;3 2];
>> [V,D] = eig(A)
V =
-0.70711 0.70711
0.70711 0.70711
D =
-1 0
0 5
The second column of V is an eigenvector, with eigenvalue of 5. We can arbitrarily scale an eigenvector by any constant. So now pick the vector vec, and create a matrix with replicated columns.
>> vec = [1;1];
>> A*[vec,vec,vec]
ans =
5 5 5
5 5 5
This should surprise nobody.
I have a high dimensional Gaussian with mean M and covariance matrix V. I would like to calculate the distance from point p to M, taking V into consideration (I guess it's the distance in standard deviations of p from M?).
Phrased differentially, I take an ellipse one sigma away from M, and would like to check whether p is inside that ellipse.
If V is a valid covariance matrix of a gaussian, it then is symmetric positive definite and therefore defines a valid scalar product. By the way inv(V) also does.
Therefore, assuming that M and p are column vectors, you could define distances as:
d1 = sqrt((M-p)'*V*(M-p));
d2 = sqrt((M-p)'*inv(V)*(M-p));
the Matlab way one would rewrite d2as (probably some unnecessary parentheses):
d2 = sqrt((M-p)'*(V\(M-p)));
The nice thing is that when V is the unit matrix, then d1==d2and it correspond to the classical euclidian distance. To find wether you have to use d1 or d2is left as an exercise (sorry, part of my job is teaching). Write the multi-dimensional gaussian formula and compare it to the 1D case, since the multidimensional case is only a particular case of the 1D (or perform some numerical experiment).
NB: in very high dimensional spaces or for very many points to test, you might find a clever / faster way from the eigenvectors and eigenvalues of V (i.e. the principal axes of the ellipsoid and their corresponding variance).
Hope this helps.
A.
Consider computing the probability of the point given the normal distribution:
M = [1 -1]; %# mean vector
V = [.9 .4; .4 .3]; %# covariance matrix
p = [0.5 -1.5]; %# 2d-point
prob = mvnpdf(p,M,V); %# probability P(p|mu,cov)
The function MVNPDF is provided by the Statistics Toolbox
Maybe I'm totally off, but isn't this the same as just asking for each dimension: Am I inside the sigma?
PSEUDOCODE:
foreach(dimension d)
(M(d) - sigma(d) < p(d) < M(d) + sigma(d)) ?
Because you want to know if p is inside every dimension of your gaussian. So actually, this is just a space problem and your Gaussian hasn't have to do anything with it (except for M and sigma which are just distances).
In MATLAB you could try something like:
all(M - sigma < p < M + sigma)
A distance to that place could be, where I don't know the function for the Euclidean distance. Maybe dist works:
dist(M, p)
Because M is just a point in space and p as well. Just 2 vectors.
And now the final one. You want to know the distance in a form of sigma's:
% create a distance vector and divide it by sigma
M - p ./ sigma
I think that will do the trick.
In MATLAB, when I run the command [V,D] = eig(a) for a symmetric matrix, the largest eigenvalue (and its associated vector) is located in last column. However, when I run it with a non-symmetric matrix, the largest eigenvalue is in the first column.
I am trying to calculate eigenvector centrality which requires that I take the compute the eigenvector associated with the largest eigenvalue. So the fact that the largest eigenvalue appears in two separate places it makes it difficult for me to find the solution.
What I usually do is:
[V D] = eig(a);
[D order] = sort(diag(D),'descend'); %# sort eigenvalues in descending order
V = V(:,order);
You just have to find the index of the largest eigenvalue in D, which can easily be done using the function DIAG to extract the main diagonal and the function MAX to get the maximum eigenvalue and the index where it occurs:
[V,D] = eig(a);
[maxValue,index] = max(diag(D)); %# The maximum eigenvalue and its index
maxVector = V(:,index); %# The associated eigenvector in V
NOTE: As woodchips points out, you can have complex eigenvalues for non-symmetric matrices. When operating on a complex input X, the MAX function uses the magnitude of the complex number max(abs(X)). In the case of equal magnitude elements, the phase angle max(angle(X)) is used.
Note that non-symmetric matrices tend to have complex eigenvalues.
eig(rand(7))
ans =
3.2957
-0.22966 + 0.58374i
-0.22966 - 0.58374i
-0.38576
0.49064
0.17144 + 0.27968i
0.17144 - 0.27968i
Also note that eig does not explicitly return sorted eigenvalues (although the underlying algorithm tends to produce them in a nearly sorted order, based on the magnitude of the eigenvalue), but even if you do do a sort, you need to understand how sort works on complex vectors.
sort(rand(5,1) + i*rand(5,1))
ans =
0.42343 + 0.51539i
0.0098208 + 0.76145i
0.20348 + 0.88695i
0.43595 + 0.83893i
0.8225 + 0.91264i
Sort, when applied to complex inputs, works on the magnitude of the complex number.
If you only care for the eigenvector associated with the largest eigenvalue, isn't it better to use eigs?
[V, D] = eigs( a, 1, 'lm' ); %// get first eigenvector with largest eigenvalue magnitude.