I want to make similar graphs to this given on the picture:
I am using Fisher Iris data and employ PCA to reduce dimensionality.
this is code:
load fisheriris
[pc,score,latent,tsquare,explained,mu] = princomp(meas);
I guess the eigenvalues are given in Latent, that shows me only four features and is about reduced data.
My question is how to show all eigenvalues of original matrix, which is not quadratic (150x4)? Please help! Thank you very much in advance!
The short (and useless) answer is that the [V, D] eig(_) function gives you the eigenvectors and the eigenvalues. However, I'm afraid I have bad news for you. Eigenvalues and eigenvectors only exist for square matrices, so there are no eigenvectors for your 150x4 matrix.
All is not lost. PCA actually uses the eigenvalues of the covariance matrix, not of the original matrix, and the covariance matrix is always square. That is, if you have a matrix A, the covariance matrix is AAT.
The covariance matrix is not only square, it is symmetric. This is good, because the singular values of a matrix are related to the eigenvalues of it's covariance matrix. Check the following Matlab code:
A = [10 20 35; 5 7 9]; % A rectangular matrix
X = A*A'; % The covariance matrix of A
[V, D] = eig(X); % Get the eigenvectors and eigenvalues of the covariance matrix
[U,S,W] = svd(A); % Get the singular values of the original matrix
V is a matrix containing the eigenvectors, and D contains the eigenvalues. Now, the relationship:
SST ~ D
U ~ V
I use '~' to indicate that while they are "equal", the sign and order may vary. There is no "correct" order or sign for the eigenvectors, so either is valid. Unfortunately, though, you will only have four features (unless your array is meant to be the other way around).
Related
If X is a multivariate t random variable with mean=[1,2,3,4,5] and a covariance matrix C, how to simulate points in matlab? I try mvtrnd in matlab, but clearly the sample mean does not give mean close to [1,2,3,4,5]. Also, when I test three simple examples, say X1 with mean 0 and C1=[1,0.3;0.3,1], X2 with mean 0 and C2=[0.5,0.15;0.15,0.5] and X3 with mean 0 and C3=[0.4,0.12;0.12,0.4] and use mvtrnd(C1,3,1000000), mvtrnd(C2,3,1000000) amd mvtrnd(C2,3,1000000) respectively, I find the sample points in each case give nearly the correlation matrix [1,0.3;0.3,1] but the sample covariance computed all give near [3,1;1,3]. Why and how to fix it?
The Mean
The t distribution has a zero mean unless you shift it. In the documentation for mvtrnd:
the distribution of t is that of a vector having a multivariate normal
distribution with mean 0, variance 1, and covariance matrix C, divided
by an independent chi-square random value having df degrees of
freedom.
Indeed, mean(X) will approach [0 0] for X = mvtrnd(C,df,n); as n gets larger.
The Correlation
Matching the correlation is straightforward as it addresses a part of the relationship between the two dimensions of X.
% MATLAB 2018b
df = 5; % degrees of freedom
C = [0.44 0.25; 0.25 0.44]; % covariance matrix
numSamples = 1000;
R = corrcov(C); % Convert covariance to correlation matrix
X = mvtrnd(R,df,numSamples); % X ~ multivariate t distribution
You can compare how well you matched the correlation matrix R using corrcoef or corr().
corrcoef(X) % Alternatively, use corr(X)
The Covariance
Matching the covariance is another matter. Admittedly, calling cov(X) will reveal that this is lacking. Recall that the diagonal of the covariance is the variance for the two components of X. My intuition is that we fixed the degrees of freedom df, so there is no way to match the desired variance (& covariance).
A useful function is corrcov which converts a covariance matrix into a correlation matrix.
Notice that this is unnecessary as the documentation for mvtrnd indicates
C must be a square, symmetric and positive definite matrix. If its
diagonal elements are not all 1 (that is, if C is a covariance matrix
rather than a correlation matrix), mvtrnd rescales C to transform it
to a correlation matrix before generating the random numbers.
i have (256*1) vectors of feature come from (16*16) of gray images. number of vectors is 550
when i compute Sample covariance of this vectors and compute covariance matrix determinant
answer is inf
it is possible determinant of finite matrix with finite range (0:255) value be infinite or i mistake some where?
in fact i want classification with bayesian estimation , my distribution is gaussian and when
i compute determinant be inf and ultimate Answer(likelihood) is zero .
some part of my code:
Mean = mean(dataSet,2);
MeanMatrix = Mean*ones(1,NoC);
Xc = double(dataSet)-MeanMatrix; % transform data to the origine
Sigma = (1/NoC) *Xc*Xc'; % calculate sample covariance matrix
Parameters(i).M = Mean';
Parameters(i).C = Sigma;
likelihoods(i) = (1/(2*pi*sqrt(det(params(i).C)))) * (exp(-0.5 * (double(X)-params(i).M)' * inv(params(i).C) * (double(X)-params(i).M)));
variable i show my classes;
variable X show my feature vector;
Can the determinant of such matrix be infinite? No it cannot.
Can it evaluate as infinite? Yes definitely.
Here is an example of a matrix with a finite amount of elements, that are not too big, yet the determinant will rarely evaluate as a finite number:
det(rand(255)*255)
In your case, probably what is happening is that you have too few datapoints to produce a full-rank covariance matrix.
For instance, if you have N examples, each with dimension d, and N<d, then your d x d covariance matrix will not be full rank and will have a determinant of zero.
In this case, a matrix inverse (precision matrix) does not exist. However, attempting to compute the determinant of the inverse (by taking 1/|X'*X|=1/0 -> \infty) will produce an infinite value.
One way to get around this problem is to set the covariance to X'*X+eps*eye(d), where eps is a small value. This technique corresponds to placing a weak prior distribution on elements of X.
no it is not possible. it may be singular but taking elements a large value has will have a determinant value.
I want to compute the determinant of a matrix from its LUP decomposition in MATLAB. The determinant can be found from the formula:
P is a permutation matrix and S is the number of exchanges of rows needed to transform P into an identity matrix. How can I find S in the above formula in MATLAB? Does it have any pre-defined functions, etc.?
If you interpret P as an adjacency matrix, and the vector cycles contains the length of all cycles in the graph described by P, then S=sum(cycles) - length(cycles).
Now all is left is to find the length of all the cycles, for which there are several functions on the File Exchange, like this one.
BTW: [L, U, P] = lu(A), and det(A) = det(inv(P))*det(L)*det(U)
I am wondering how to draw samples in matlab, where I have precision matrix and mean as the input argument.
I know mvnrnd is a typical way to do so, but it requires the covariance matrix (i.e inverse of precision)) as the argument.
I only have precision matrix, and due to the computational issue, I can't invert my precision matrix, since it will take too long (my dimension is about 2000*2000)
Good question. Note that you can generate samples from a multivariant normal distribution using samples from the standard normal distribution by way of the procedure described in the relevant Wikipedia article.
Basically, this boils down to evaluating A*z + mu where z is a vector of independent random variables sampled from the standard normal distribution, mu is a vector of means, and A*A' = Sigma is the covariance matrix. Since you have the inverse of the latter quantity, i.e. inv(Sigma), you can probably do a Cholesky decomposition (see chol) to determine the inverse of A. You then need to evaluate A * z. If you only know inv(A) this can still be done without performing a matrix inverse by instead solving a linear system (e.g. via the backslash operator).
The Cholesky decomposition might still be problematic for you, but I hope this helps.
If you want to sample from N(μ,Q-1) and only Q is available, you can take the Cholesky factorization of Q, L, such that LLT=Q. Next take the inverse of LT, L-T, and sample Z from a standard normal distribution N(0, I).
Considering that L-T is an upper triangular dxd matrix and Z is a d-dimensional column vector,
μ + L-TZ will be distributed as N(μ, Q-1).
If you wish to avoid taking the inverse of L, you can instead solve the triangular system of equations LTv=Z by back substitution. μ+v will then be distributed as N(μ, Q-1).
Some illustrative matlab code:
% make a 2x2 covariance matrix and a mean vector
covm = [3 0.4*(sqrt(3*7)); 0.4*(sqrt(3*7)) 7];
mu = [100; 2];
% Get the precision matrix
Q = inv(covm);
%take the Cholesky decomposition of Q (chol in matlab already returns the upper triangular factor)
L = chol(Q);
%draw 2000 samples from a standard bivariate normal distribution
Z = normrnd(0,1, [2, 2000]);
%solve the system and add the mean
X = repmat(mu, 1, 2000)+L\Z;
%check the result
mean(X')
var(X')
corrcoef(X')
% compare to the sampling from the covariance matrix
Y=mvnrnd(mu,covm, 2000)';
mean(Y')
var(Y')
corrcoef(Y')
scatter(X(1,:), X(2,:),'b')
hold on
scatter(Y(1,:), Y(2,:), 'r')
For more efficiency, I guess you can search for some package that efficiently solves triangular systems.
Is it possible to reverse the following in matlab:
[U,S,V]=svds(fulldata,columns);
Quoting MathWorks:
[U,S,V] = svd(X) produces a diagonal matrix S of the same dimension as X, with nonnegative diagonal elements in decreasing order, and unitary matrices U and V so that X = U*S*V'.
In the case of svds, one will lose some information unless columns is equal to the size of the square matrix fulldata. In this case, I believe the original matrix cannot be reconstructed uniquely.