I'm working with 6x6 matrices which have varying precisions of data. When I try to inverse that matrix in MATLAB I usually get Inf or NaN as all the data and MATLAB throws a warning:
Matrix is singular to working precision.
Is there anyway to avoid it and get proper results?
Your matrix seems to be rank deficient. Only full rank matrices can be robustly inverted.
You may circumvent your problem by adding a small identity matrix to the original one.
A = rand(6,5);
A = A*A'; %' symmetric rank 5 matrix
iA = inv(A); % results with NaNs and infs A is singular
iAs = inv( A + eye(6)*1e-3 ); % add small (1e-3) elements to diagonal - this should help
Related
I have a 943x1682 matrix in which I want to calculate the two most similar vectors in this matrix. So I want see the cosine distance of each vector in the matrix to each vector in the matrix, of course not including the vector with itself, if one cannot do that I can just ignore those.
I made this loop to try to calculate this, so I can get a 1682x1682 matrix, with each cell corresponding to the similarity between i and j. However when I run this, it takes forever to run, and when I try to open the resulting matrix in my workspace, it says:
Cannot display summaries of variables with more than 524288 elements.
Is there an easier way to do this or am I doing something wrong?
Cross posted on MATLAB Answers. Repeating answer here:
Use a standard matrix multiply to get the dot products. MATLAB is very fast at standard matrix multiplies. And then normalize the result. E.g.,
AA = A' * A; % the column dot products via a standard matrix multiply
Anorm = sqrt(diag(AA)); % the norms of the columns
Adist = AA ./ (Anorm .* Anorm.'); % normalize the column dot products into cosine distances
Then pick off the maximum value for your answer, disregarding the diagonal. E.g.,
n = size(A,2); % the number of columns
Adist(1:n+1:end) = -inf; % disregard the diagonal (column compared to itself)
[~,x] = max(Adist(:)); % find the max cosine distance linear index
[col1,col2] = ind2sub(size(Adist),x); % convert linear index into the original columns
Then col1 and col2 are the column numbers of the most similiar columns using cosine distance as a measure.
You can normalise the columns of the matrix first, then the cosine similarity equation simplifies to a matrix multiplication:
aNorm = normc(A);
cosSim = aNorm' * aNorm;
Generally, matrix multiplication is more performant than looping. In a quick test, with N = 1000, the looping code takes ~7 seconds and the matrix multiplication code ~0.5 seconds.
The resultant matrix may still be too large to open in your workspace, you could copy any individual rows or columns into a temporary and view those, or do a contour plot (heat-map) of the matrix to get a visual representation.
I'm trying to build a Gaussian Mixture Model using random initializations and compare the results with one using Kmeans initializations. However, I have difficulty creating the initial covariance matrix. I randomly selected 10 data points from my data set of 2500 data points (each "point" is actually an image), and used them as the means. Then I'm trying to create the covariance matrix from each of these random points.
Here's what I have.
% Randomly initialize GMM parameters
rng(1);
rand_index = randperm(2500);
Mu = data(:,rand_index(1:10));
for i = 1 : 10
Sigma(:,:,i) = cov(Mu);
Pxi(:,i) = mvnpdf(data', Mu(:,i)', Sigma(:,:,i));
end
data is a 50x2500 matrix. I keep getting an error because my Sigma is of the wrong size, or it's not positive definite, etc.
For example, the code above gave the error
Error using mvnpdf (line 116)
SIGMA must be a square matrix with size equal to the number of columns in X, or a row vector with length equal to the number of
columns in X.
If I use
Sigma(:,:,i) = cov([Mu(:,i) Mu(:,i)]');
I get the error
Error using mvnpdf (line 129)
SIGMA must be a square, symmetric, positive definite matrix.
How should I create this covariance matrix?
I assume what you are experiencing is not happening at every run. This is a numerical instability that you can avoid using a simple technique:
%Add a tiny variance to avoid numerical instability
Sigma(:,:,i) = cov([Mu(:,i) Mu(:,i)]');
D = size(Sigma,1);
Sigma(:,:,i) = Sigma(:,:,i) + 1E-5.*diag(ones(D,1));
I want to make similar graphs to this given on the picture:
I am using Fisher Iris data and employ PCA to reduce dimensionality.
this is code:
load fisheriris
[pc,score,latent,tsquare,explained,mu] = princomp(meas);
I guess the eigenvalues are given in Latent, that shows me only four features and is about reduced data.
My question is how to show all eigenvalues of original matrix, which is not quadratic (150x4)? Please help! Thank you very much in advance!
The short (and useless) answer is that the [V, D] eig(_) function gives you the eigenvectors and the eigenvalues. However, I'm afraid I have bad news for you. Eigenvalues and eigenvectors only exist for square matrices, so there are no eigenvectors for your 150x4 matrix.
All is not lost. PCA actually uses the eigenvalues of the covariance matrix, not of the original matrix, and the covariance matrix is always square. That is, if you have a matrix A, the covariance matrix is AAT.
The covariance matrix is not only square, it is symmetric. This is good, because the singular values of a matrix are related to the eigenvalues of it's covariance matrix. Check the following Matlab code:
A = [10 20 35; 5 7 9]; % A rectangular matrix
X = A*A'; % The covariance matrix of A
[V, D] = eig(X); % Get the eigenvectors and eigenvalues of the covariance matrix
[U,S,W] = svd(A); % Get the singular values of the original matrix
V is a matrix containing the eigenvectors, and D contains the eigenvalues. Now, the relationship:
SST ~ D
U ~ V
I use '~' to indicate that while they are "equal", the sign and order may vary. There is no "correct" order or sign for the eigenvectors, so either is valid. Unfortunately, though, you will only have four features (unless your array is meant to be the other way around).
i have (256*1) vectors of feature come from (16*16) of gray images. number of vectors is 550
when i compute Sample covariance of this vectors and compute covariance matrix determinant
answer is inf
it is possible determinant of finite matrix with finite range (0:255) value be infinite or i mistake some where?
in fact i want classification with bayesian estimation , my distribution is gaussian and when
i compute determinant be inf and ultimate Answer(likelihood) is zero .
some part of my code:
Mean = mean(dataSet,2);
MeanMatrix = Mean*ones(1,NoC);
Xc = double(dataSet)-MeanMatrix; % transform data to the origine
Sigma = (1/NoC) *Xc*Xc'; % calculate sample covariance matrix
Parameters(i).M = Mean';
Parameters(i).C = Sigma;
likelihoods(i) = (1/(2*pi*sqrt(det(params(i).C)))) * (exp(-0.5 * (double(X)-params(i).M)' * inv(params(i).C) * (double(X)-params(i).M)));
variable i show my classes;
variable X show my feature vector;
Can the determinant of such matrix be infinite? No it cannot.
Can it evaluate as infinite? Yes definitely.
Here is an example of a matrix with a finite amount of elements, that are not too big, yet the determinant will rarely evaluate as a finite number:
det(rand(255)*255)
In your case, probably what is happening is that you have too few datapoints to produce a full-rank covariance matrix.
For instance, if you have N examples, each with dimension d, and N<d, then your d x d covariance matrix will not be full rank and will have a determinant of zero.
In this case, a matrix inverse (precision matrix) does not exist. However, attempting to compute the determinant of the inverse (by taking 1/|X'*X|=1/0 -> \infty) will produce an infinite value.
One way to get around this problem is to set the covariance to X'*X+eps*eye(d), where eps is a small value. This technique corresponds to placing a weak prior distribution on elements of X.
no it is not possible. it may be singular but taking elements a large value has will have a determinant value.
I have a following problem - I have a matrix A of size 16x22440.
What I need to do is to normalize each row of this matrix, so that the norm of each of them is equal to 1 (for n=1:16 norm(A(n,:))==1)
How can I achieve that in matlab?
Edit:
Each row in this matrix is a vector created of an 160x140 image and thus must be considered separately. The values need to be normalised to create an eigenfaces matrix.
First, compute the norm (I assume Eucleadian norm here)
n = sqrt( sum( A.^2, 2 ) );
% patch to overcome rows with zero norm
n( n == 0 ) = 1;
nA = bsxfun( #rdivide, A, n ); % divide by norm
Does your install of Matlab include the Neural Network Toolbox? If so, then try normr:
nA = normr(A);
Otherwise, #Shai's solution is good except that it won't handle infinite or NaN inputs – it's much safer to check undefined norm cases afterwards:
nA = bsxfun(#rdivide,A,sqrt(sum(A.^2,2)));
nA(~isfinite(nA)) = 1; % Use 0 to match output of #Shai's solution, Matlab's norm()
Note that normalizing of a zero length (all zero components) or infinite length vector (one or more components +Inf or -Inf) or one with a NaN component is not really well-defined. The solution above returns all ones, just as does Matlab's normr function. Matlab's norm function, however, exhibits different behavior. You may wish to specify a different behavior, e.g., a warning or an error, all zeros, NaNs, components scaled by the vector length, etc. This thread discusses the issue for zero-length vectors to some extent: How do you normalize a zero vector?.