The following is an excerpt from a program of mine:
function [P] = abc(M,f);
if det(M) ~= 1, disp(['Matrix M should have determinant 1'])
I allow the option for the user not to enter a value for 'f'.
When I run abc([2 1; 1 1]), the program works fine and it does what it's supposed to. But when I run abc([6 13; 5 11]) I am told "Matrix M should have determinant 1".
What on Earth is going on?
EDIT: In the command window, I entered the following:
M = [6 13; 5 11];
if det(M) ~= 1, disp('Im broken');
end
Matlab then told me itself that it's broken.
Thanks
Welcome to the wonderfully wacky world of floating point arithmetic. MATLAB computes the determinant using an LU decomposition, i.e., linear algebra. It does so since determinant is wildly inefficient for arrays of even mild size unless it did.
A consequence of that LU decomposition, is the determinant is computed as a floating point number. This is not an issue, UNLESS you have entered a problem as trivially simple as you have - the determinant of a 2x2 matrix composed only of small integers. In that case, the determinant itself will also be a (reasonably) small integer. So you could resolve the issue by simply computing the determinant of the 2x2 matrix yourself, using the textbook formula.
D = A(1,1)*A(2,2) - A(1,2)*A(2,1);
This will be exactly correct for small integer matrices A, although even this may show some loss of precision for SOME matrices. For example, consider the simple, 2x2 matrix A:
>> A = [1e8 1;1 1e8];
We know that the determinant of this matrix is 1e16-1.
>> det(A)
ans =
1e+16
Of course, MATLAB displays this as 1e16. But in fact, the number generated by the det function in MATLAB is actually 9999999999999998, so 1e16-2. As bad, had I used the formula I gave above for the 2x2 determinant, it would have returned a result that is still incorrect, 10000000000000000. Both results were off by 1. You can learn more about these issues by looking at the help for eps.
My point is, there are some 2x2 matrices where computation of the determinant will simply be problematic, even though they are integer matrices.
Once your matrices become non-integer, then things really do become true floating point numbers, not integers. This means you simply MUST use comparisons with tolerances on them rather than a test for exact unity. This is a good rule anyway. Always use a tolerance when you make a test for equality, at least until you have learned enough to know when to disobey that rule!
So, you might choose a test like this:
if abs(det(A) - 1) < (10*eps(1))
warning('The sky is falling! det has failed me.')
end
Note that I've used eps(1), since we are comparing things to 1. The fact that I multiplied eps by 10 allows a wee bit of slop in the computation of the determinant.
Finally, you should know that whatever test you are using the determinant for here, it is often a BBBBBBBBBBAAAAAAAAAADDDDDDDD thing to do! Yes, maybe your teacher told you to do this, or you found something in a textbook. But the determinant is just a bad thing to use for numerical computations. There are almost always alternatives to the determinant. Again, this is called judgement, knowing when that which you are told to use is actually the wrong thing to do.
You are running into the standard problems that occur due to the limitations of floating-point numbers. The result of the det function is probably something like 1.000000001.
General rule-of-thumb: Never test floating-point values for equality.
To give you an insight: det is not computed using the old formula you studied in linear algebra, but using more efficient algorithms.
For example, using Gaussian elimination you can transform M in the equivalent upper triangular matrix and then compute the determinant as product of the main diagonal (being the lower triangle all zeros).
M = [6 13; 5 11]
G = M - [0 0; M(2,1)/M(1,1) * M(1,:)];
Theoretically det(M) is equal to det(G), which is 6 * 1/6 = 1, but being G a floating point and not a real number matrix, G(1,1)*G(2,2)~=1!
In fact G(1,1) and G(2,2) are not exactly 1 and 1/6, but they have a very small relative error (see eps, which on most machines is around 2.22e-16). Their real value will be around 6*(1+eps) and 1/6*(1+eps), thus their product will have a small error too.
I'm not sure if Matlab uses the Gaussian elimination or the similar LU decomposition.
Related
This is my matlab code I wrote for a problem I got as homework. after multiplication of A and its transpose the resulting square matrix should have determinant zero according all classmates as their codes (different one) gave them so. Why is my code not giving the determinant of c and d to be infinity
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
detc = det(c);
cinv = inv((A.')*A);
d = A*(A.');
detd = det(d);
dinv = inv(A*(A.'));
x1 = (inv((A.')*A))*((A.')*b);
x2 = A.'*((inv(A*(A.')))*b);
This behavior is explained in the Limitations section of the det's documentation and exemplified in the Find Determinant of Singular Matrix subsection where it is stated:
The determinant of A is quite large despite the fact that A is singular. In fact, the determinant of A should be exactly zero! The inaccuracy of d is due to an aggregation of round-off errors in the MATLAB® implementation of the LU decomposition, which det uses to calculate the determinant.
That said, in this instance, you can produce your desired result by using the m-code implementation given on that same page but sorting the diagonal elements of U in an ascending matter. Consider the sample script:
clc();
clear();
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
[L,U] = lu(c);
% Since det(L) is always (+/-)1, it doesn't impact anything
diagU = diag(U);
detU1 = prod(diagU);
detU2 = prod(sort(diagU,'descend'));
detU3 = prod(sort(diagU,'ascend'));
fprintf('Minimum: %+9.5e\n',min(abs(diagU)));
fprintf('Maximum: %+9.5e\n',max(abs(diagU)));
fprintf('Determinant:\n');
fprintf('\tNo Sort: %g\n' ,detU1);
fprintf('\tDescending Sort: %g\n' ,detU2);
fprintf('\tAscending Sort: %g\n\n',detU3);
This produces the output:
Minimum: +1.53111e-13
Maximum: +1.72592e+02
Determinant:
No Sort: Inf
Descending Sort: Inf
Ascending Sort: 0
Notice that the direction of the sort matters, and that no-sorting gives Inf since a true 0 doesn't exist on the diagonal. The descending sort sees the largest values multiplied first, and apparently, they exceed realmax and are never multiplied by a true 0, which would generate a NaN. The ascending sort clumps together all of the near-zero diagonal values with very few large negative values (in truth, a more robust method would sort based on magnitude, but that was not done here), and their multiplication generates a true 0 (meaning that the value falls below the smallest denormalized number available in IEEE-754 arithmetic) that produces the "correct" result.
All that written, and as others have implied, I'll quote original Matlab developer and Mathworks co-founder Cleve Moler:
[The determinant] is useful in theoretical considerations and hand calculations, but does not provide a sound basis for robust numerical software.
Ok. So the fact that det(A'*A) is not zero is not a good indication of the (non-)singularity of A'*A.
The determinant depends on the scaling, and matrix clearly non-singular can have very small determinant. For instance, the matrix
1/2 * I_n
where I_n is the nxn identity has a determinant of (1/2)^n which is converging (quickly) to 0 as n goes to infinity. But 1/2 * I_n is not, at all, singular.
For this reason, a best idea to check the singularity of a matrix is the condition number.
In you case, after doing some tests
>> A = rand(500, 1500) ;
>> det(A'*A)
ans =
Inf
You can see that the (computed) determinant is clearly non-zero. But this is actually not surprising, and it should not really bother you. The determinant is fairly hard to compute, so yes, it is just rounding errors. If you want a better approximation, you can do the following
>> s = eig(A'*A) ;
>> prod(s)
ans =
0
There, you see it is closer to zero.
The condition number, on the other hand, is a much better estimator of the (non-)singularity of a matrix. Here, it is
>> cond(A'*A)
ans =
1.4853e+20
And, since it is much larger than 1e+16, the matrix is clearly singular. The reason for 1e+16 is a bit tedious, but is mostly due to the computer precision when doing floating point computations.
I think this is pretty much just a rounding problem, the Inf does not mean you are getting Infinity as an answer, it's just that your determinant is really big and exceeded realmax. As Adiel said, A*A.' generates a symmetric matrix, and should have a numerical value for its determinant. for example, set:
A=rand(5,15)
and you should find that the det of A*A.' is just a numerical value.
SO how did your friends get a ZERO, well it's easy to get 0 or inf for det of large matrices (why are you doing this in the first place I have no clue). So I think they are just getting the same/similar rounding issue.
I have a linear equation such as
Ax=b
where A is full rank matrix which its size is 512x512. b is a vector of 512x1. x is unknown vector. I want to find x, hence, I have some options for doing that
1.Using the normal way
inv(A)*b
2.Using SVD ( Singular value decomposition)
[U S V]=svd(A);
x = V*(diag(diag(S).^-1)*(U.'*b))
Both methods give the same result. So, what is benefit of using SVD to solve Ax=b, especially in the case A is a 2D matrix?
Welcome to the world of numerical methods, let me be your guide.
You, as a new person in this world wonders, "Why would I do something this difficult with this SVD stuff instead of the so commonly known inverse?! Im going to try it in Matlab!"
And no answer was found. That is, because you are not looking at the problem itself! The problems arise when you have an ill-conditioned matrix. Then the computing of the inverse is not possible numerically.
example:
A=[1 1 -1;
1 -2 3;
2 -1 2];
try to invert this matrix using inv(A). Youll get infinite.
That is, because the condition number of the matrix is very high (cond(A)).
However, if you try to solve it using SVD method (b=[1;-2;3]) you will get a result. This is still a hot research topic. Solving Ax=b systems with ill condition numbers.
As #Stewie Griffin suggested, the best way to go is mldivide, as it does a couple of things behind it.
(yeah, my example is not very good because the only solution of X is INF, but there is a way better example in this youtube video)
inv(A)*b has several negative sides. The main one is that it explicitly calculates the inverse of A, which is both time demanding, and may result in inaccuracies if values vary by many orders of magnitude.
Although it might be better than inv(A)*b, using svd is not the "correct" approach here. The MATLAB-way to do this is using mldivide, \. Using this, MATLAB chooses the best algorithm to solve the linear system based on its properties (Hermation, upper Hessenberg, real and positive diagonal, symmetric, diagonal, sparse etc.). Often, the solution will be a LU-triangulation with partial permutation, but it varies. You'll have a hard time beating MATLABs implementation of mldivide, but using svd might give you some more insight of the properties of the system if you actually investigates U, S, V. If you don't want to do that, do with mldivide.
I use the function below to generate the betas for a given set of guess lambdas from my optimiser.
When running I often get the following warning message:
Warning: Matrix is singular to working precision.
In NSS_betas at 9
In DElambda at 19
In Individual_Lambdas at 36
I'd like to be able to exclude any betas that form a singular matrix form the solution set, however I don't know how to test for it?
I've been trying to use rcond() but I don't know where to make the cut off between singular and non singular?
Surely if Matlab is generating the warning message it already knows if the matrix is singular or not so if I could just find where that variable was stored I could use that?
function betas=NSS_betas(lambda,data)
mats=data.mats2';
lambda=lambda;
yM=data.y2';
nObs=size(yM,1);
G= [ones(nObs,1) (1-exp(-mats./lambda(1)))./(mats./lambda(1)) ((1-exp(-mats./lambda(1)))./(mats./lambda(1))-exp(-mats./lambda(1))) ((1-exp(-mats./lambda(2)))./(mats./lambda(2))-exp(-mats./lambda(2)))];
betas=G\yM;
r=rcond(G);
end
Thanks for the advice:
I tested all three examples below after setting the lambda values to be equal so guiving a singular matrix
if (~isinf(G))
r=rank(G);
r2=rcond(G);
r3=min(svd(G));
end
r=3, r2 =2.602085213965190e-16; r3= 1.075949299504113e-15;
So in this test rank() and rcond () worked assuming I take the benchmark values as given below.
However what happens when I have two values that are close but not exactly equal?
How can I decide what is too close?
rcond is the right way to go here. If it nears the machine precision of zero, your matrix is singular. I usually go with:
if( rcond(A) < 1e-12 )
% This matrix doesn't look good
end
You can experiment with a value that suites your needs, but taking the inverse of a matrix that is even close to singular with MATLAB can produce garbage results.
You could compare the result of rank(G) with the number of columns of G. If the rank is less than the column dimension, you will have a singular matrix.
you can also check this by:
min(svd(A))>eps
and verifying that the smallest singular value is larger than eps, or any other numerical tolerance that is relevant to your needs. (the code will return 1 or 0)
Here's more info about it...
Condition number (Maximal singular value/Minimal singular value) is another good method:
cond(A)
It uses svd. It should be as close to 1 as possible. Very large values mean that the matrix is almost singular. Inf means that it is precisely singular.
Note that almost all of the methods mentioned in other answers use somehow svd :
There are special tools designed for this problem, appropriately called "rank revealing matrix factorizations". To my best (albeit a little old) knowledge, a good enough way to decide whether a n x n matrix A is nonsingular is to go with
det(A) <> 0 <=> rank(A) = n
and use a rank-revealing QR factorization of A:
AP = QR
where Q is orthogonal, P is a permutation matrix and R is an upper triangular matrix with the property that the magnitude of the diagonal elements is decreased along the diagonal.
I have been give a very large matrix (I cannot change the values of the matrix) and I need to calculate the inverse of a (covariance) matrix.
Sometimes I get the error saying
Matrix is close to singular or badly scaled.
Results may be inaccurate
In these situations I see that the value of the det returns 0.
Before calculating inverse (of a covariance matrix) I want to check the value of the det and perform something like this
covarianceFea=cov(fea_class);
covdet=det(covarianceFea);
if(covdet ==0)
covdet=covdet+.00001;
%calculate the covariance using this new det
end
Is there any way to use the new det and then use this to calculate the inverse of the covariance matrix?
Sigh. Computation of the determinant to determine singularity is a ridiculous thing to do, utterly so. Especially so for a large matrix. Sorry, but it is. Why? Yes, some books tell you to do it. Maybe even your instructor.
Analytical singularity is one thing. But how about numerical determination of singularity? Unless you are using a symbolic tool, MATLAB uses floating point arithmetic. This means it stores numbers as floating point, double precision values. Those numbers cannot be smaller in magnitude than
>> realmin
ans =
2.2251e-308
(Actually, MATLAB goes a bit lower than that, in terms of denormalized numbers, which can go down to approximately 1e-323.) See that when I try to store a number smaller than that, MATLAB thinks it is zero.
>> A = 1e-323
A =
9.8813e-324
>> A = 1e-324
A =
0
What happens with a large matrix? For example, is this matrix singular:
M = eye(1000);
Since M is an identity matrix, it is fairly clearly non-singular. In fact, det does suggest that it is non-singular.
>> det(M)
ans =
1
But, multiply it by some constant. Does that make it non-singular? NO!!!!!!!!!!!!!!!!!!!!!!!! Of course not. But try it anyway.
>> det(M*0.1)
ans =
0
Hmm. Thats is odd. MATLAB tells me the determinant is zero. But we know that the determinant is 1e-1000. Oh, yes. Gosh, 1e-1000 is smaller, by a considerable amount than the smallest number that I just showed you that MATLAB can store as a double. So the determinant underflows, even though it is obviously non-zero. Is the matrix singular? Of course not. But does the use of det fail here? Of course it will, and this is completely expected.
Instead, use a good tool for the determination of singularity. Use a tool like cond, or rank. For example, can we fool rank?
>> rank(M)
ans =
1000
>> rank(M*.1)
ans =
1000
See that rank knows this is a full rank matrix, regardless of whether we scale it or not. The same is true of cond, computing the condition number of M.
>> cond(M)
ans =
1
>> cond(M*.1)
ans =
1
Welcome to the world of floating point arithmetic. And oh, by the way, forget about det as a tool for almost any computation using floating point arithmetic. It is a poor choice almost always.
Woodchips has given you a very good explanation for why you shouldn't use the determinant. This seems to be a common misconception and your question is very related to another question on inverting matrices: Is there a fast way to invert a matrix in Matlab?, where the OP decided that because the determinant of his matrix was 1, it was definitely invertible! Here's a snippet from my answer
Rather than det(A)=1, it is the condition number of your matrix that dictates how accurate or stable the inverse will be. Note that det(A)=∏i=1:n λi. So just setting λ1=M, λn=1/M and λi≠1,n=1 will give you det(A)=1. However, as M → ∞, cond(A) = M2 → ∞ and λn → 0, meaning your matrix is approaching singularity and there will be large numerical errors in computing the inverse.
You can test this in MATLAB with the following simple example:
A = eye(10);
A([1 2]) = [1e15 1e-15];
%# calculate determinant
det(A)
ans =
1
%# calculate condition number
cond(A)
ans =
1.0000e+30
In such a scenario, calculating an inverse is not a very good idea. If you just have to do it, I would suggest using this to increase display precision:
format long;
Other suggestion could be to try using an SVD of the matrix and tinker around with singular values there.
A = U∑V'
inv(A) = V*inv(∑)*U'
∑ is a diagonal matrix where you will see one of the diagonal entries close to 0. Try playing around with this number if you want some sort of an approximation.
from a simulation problem, I want to calculate complex square matrices on the order of 1000x1000 in MATLAB. Since the values refer to those of Bessel functions, the matrices are not at all sparse.
Since I am interested in the change of the determinant with respect to some parameter (the energy of a searched eigenfunction in my case), I overcome the problem at the moment by first searching a rescaling factor for the studied range and then calculate the determinants,
result(k) = det(pre_factor*Matrix{k});
Now this is a very awkward solution and only works for matrix dimensions of, say, maximum 500x500.
Does anybody know a nice solution to the problem? Interfacing to Mathematica might work in principle but I have my doubts concerning feasibility.
Thank you in advance
Robert
Edit: I did not find a convient solution to the calculation problem since this would require changing to a higher precision. Instead, I used that
ln det M = trace ln M
which is, when I derive it with respect to k
A = trace(inv(M(k))*dM/dk)
So I at least had the change of the logarithm of the determinant with respect to k. From the physical background of the problem I could derive constraints on A which in the end gave me a workaround valid for my problem. Unfortunately I do not know if such a workaround could be generalized.
You should realize that when you multiply a matrix by a constant k, then you scale the determinant of the matrix by k^n, where n is the dimension of the matrix. So for n = 1000, and k = 2, you scale the determinant by
>> 2^1000
ans =
1.07150860718627e+301
This is of course a huge number, so you might expect that it should fail, since in double precision, MATLAB will only represent floating point numbers as large as realmax.
>> realmax
ans =
1.79769313486232e+308
There is no need to do all the work of recomputing that determinant, not that computing the determinant of a huge matrix like that is a terribly well-posed problem anyway.
If speed is not a concern, you may want to use det(e^A) = e^(tr A) and take as A some scaling constant times your matrix (so that A - I has spectral radius less than one).
EDIT: In MatLab, the log of a matrix (logm) is calculated via trigonalization. So it is better for you to compute the eigenvalues of your matrix and multiply them (or better, add their logarithm). You did not specify whether your matrix was symmetric or not: if it is, finding eigenvalues are easier than if it is not.
You said the current value of the determinant is about 10^-300.
Are you trying to get the determinant at a certain value, say 1? If so, rescaling is awkward: the matrix you are considering is ill-conditioned, and, considering the precision of the machine, you should consider the output determinant to be zero. It is impossible to get a reliable inverse in other words.
I would suggest to modify the columns or lines of the matrix rather than rescale it.
I used R to make a small test with a random matrix (random normal values), it seems the determinant should be clearly non-zero.
> n=100
> M=matrix(rnorm(n**2),n,n)
> det(M)
[1] -1.977380e+77
> kappa(M)
[1] 2318.188
This is not strictly a matlab solution, but you might want to consider using Mahout. It's specifically designed for large-scale linear algebra. (1000x1000 is no problem for the scales it's used to.)
You would call into java to pass data to/from Mahout.