I am trying to implement PCA in pytorch, given a covariance matrix, its eigenvalues, eigenvectors and a proportion of variance to be captured from a data matrix, is there a formulae to calculate the number of dimensions or principal components that corresponds to the variance?
Yes, there is. The percentage of variance explained by the first N principal components is the sum of the first N eigenvalues divided by the sum of all eigenvalues!
Basically it's the magnitude of the eigenvalue that tells you the importance of the feature.
Related
I have two gridded matrix having latitude,longitude and time(180x360x12). I have calculated correlation coefficient between both matrices using following: http://in.mathworks.com/matlabcentral/answers/15884-correlation-for-multi-dimensional-arrays
now I want to find p-value (0.05) for each grid cell. than I want to set correlation values in matrix in three part: one will show positively significant (<0.05), second will show positively insignificant (>0.05) and third will show negatively significant (<0.05) correlation. Can anyone help me in this regard ?
If you use the scipy pearsonr function to calculate your correlations then this will give you the p values as well.
If cor are the correlations and p are the p-values then retrieving the significant values is then as simple as:
significant_correlations = corr[p<0.05]
My data matrix is X which is 4999*37152. Then I use this command in Matlab:
[coeff, score, latent, tsquared1, explained1] = pca(X);
The output: coeff is 37152*4998, score is 4999*4998, latent is 4998*1. According to http://www.mathworks.com/help/stats/pca.html, the coeff should be p*p. So what is wrong with my code ?
As Matlab documentation says, "Rows of X correspond to observations and columns correspond to variables". So you are feeding in a matrix with only 4999 observations for 37152 observations. Geometrically, you have 4999 points in a 37152-dimensional space. These points are contained in a 4998-dimensional affine subspace, so Matlab gets you 4998 directions there (each expressed as a vector with 37152 components).
For more, see the Statistics site:
Why are there only n-1 principal components for n data points if the number of dimensions is larger than n?
PCA when the dimensionality is greater than the number of samples
The MATLAB documentation is written under assumption that you have at least as many observations as variables, which is how people normally use PCA.
Of course, it's possible that your data actually has 37152 observations for 4999 variables, in which case you need to transpose X.
So I read the documentation on pca and it stated that the columns are organized in descending order of their variance. However, whenever I take the PCA of an example and I take the variance of the PCA matrix I get no specific order. A simple example of this is example:
pc = pca(x)
Which returns
pc =
0.0036 -0.0004
0.0474 -0.0155
0.3149 0.3803
0.3969 -0.1930
0.3794 0.3280
0.5816 -0.2482
0.3188 0.1690
-0.1343 0.7835
0.3719 0.0785
0.0310 -0.0110
Meaning column one should be PC1 and column two should be PC2 meaning var(PC1) > var(PC2), but when I get the variance this is clearly not the case.
var(pc)
ans =
0.0518 0.0932
Can anyone shed light into why the variance of PC1 is not the largest?
The docs state that calling
COEFF = pca(x)
will return a p-by-p matrix, so your result is rather surprising (EDIT: this is because your x data set has so few rows compared to columns (i.e. similar to having 10 unknowns and only 3 equations)). Either way when they talk about variance They don't mean the variance of the coefficients of each component but rather the variance of the x data columns after being projected on to each principal component. The docs state that the output score holds these projections and so to see the descending variance you should be doing:
[COEFF, score, latent] = pca(x)
var(score)
and you will see that var(score) equals the third output latent and is indeed in descending order.
Your misunderstanding is that you are trying to calculate the variance of the coefficients of the principal component vectors. These are just unit vectors describing the direction of the hyperplane on which to project your data such that the resulting projected data has maximum variance. These vectors ARE arranged in an order such that your original data projected onto the hyperplane that each describes will be in descending order of variance, but variance of the projected data (score) and NOT of the coefficients of the principal component vectors (COEFF or in your code pc).
i have (256*1) vectors of feature come from (16*16) of gray images. number of vectors is 550
when i compute Sample covariance of this vectors and compute covariance matrix determinant
answer is inf
it is possible determinant of finite matrix with finite range (0:255) value be infinite or i mistake some where?
in fact i want classification with bayesian estimation , my distribution is gaussian and when
i compute determinant be inf and ultimate Answer(likelihood) is zero .
some part of my code:
Mean = mean(dataSet,2);
MeanMatrix = Mean*ones(1,NoC);
Xc = double(dataSet)-MeanMatrix; % transform data to the origine
Sigma = (1/NoC) *Xc*Xc'; % calculate sample covariance matrix
Parameters(i).M = Mean';
Parameters(i).C = Sigma;
likelihoods(i) = (1/(2*pi*sqrt(det(params(i).C)))) * (exp(-0.5 * (double(X)-params(i).M)' * inv(params(i).C) * (double(X)-params(i).M)));
variable i show my classes;
variable X show my feature vector;
Can the determinant of such matrix be infinite? No it cannot.
Can it evaluate as infinite? Yes definitely.
Here is an example of a matrix with a finite amount of elements, that are not too big, yet the determinant will rarely evaluate as a finite number:
det(rand(255)*255)
In your case, probably what is happening is that you have too few datapoints to produce a full-rank covariance matrix.
For instance, if you have N examples, each with dimension d, and N<d, then your d x d covariance matrix will not be full rank and will have a determinant of zero.
In this case, a matrix inverse (precision matrix) does not exist. However, attempting to compute the determinant of the inverse (by taking 1/|X'*X|=1/0 -> \infty) will produce an infinite value.
One way to get around this problem is to set the covariance to X'*X+eps*eye(d), where eps is a small value. This technique corresponds to placing a weak prior distribution on elements of X.
no it is not possible. it may be singular but taking elements a large value has will have a determinant value.
In opencv, the result of Expectation Max is given as a co-variance matrix. In my work, I have 2 distributions where I need to compute the similarities. I want to use the variance but its given only co-variance of the number of clusters:
I have 2 question that I need help on:
If my cluster is 1, than I assume that the co-variance = variance. Is it correct.?
How to compute variance from co-variance for cluster > 1 (if 1 is correct)
Every cluster in your case is gaussian probability distribution density function. In onedimentional case its parameters are mean and variance.
In multidimentional case variance becomes covariance matrix.It describes ellipsoid axis directions and size.
You can reduce covariance to variance but you'll get circle or sphere instead ellipse or ellipsoid.
So ellipse axis directions will be eigenvectors of covariance matrix, and their halflengths will be square root of eigenvalues. Once you know ellipse axis you should deside the way you will convert it to circle. The radius you accept will be square root of variance.
But if you'll need compute probability you'll should compute covariance matrix from your variance by scaling identity matrix by factor equal to variance.