This question already has answers here:
Why is Matlab's inv slow and inaccurate?
(3 answers)
Closed 7 years ago.
The title explains it already. If I need to find an inverse of a matrix, is there any reason I should use A\eye(size(A)) instead of inv(A)?
And before you ask: Yes, I really need the inverse, not only for calculations.
PS:
isequal(inv(A), A\eye(size(A)))
ans =
0
So which one is more precise?
UPDATE: This question was closed as it appeard to be a duplicate of the question "why is inv in MATLAB so slow and inaccurate". This question here differs significantly by not addressing the speed, nor the accuarcy of the function inv but the difference of inv and .\eye to calculate the true inverse of a matrix.
Let's disregard performance (speed) and best practice for a bit.
eps(n) is a command that returns the distance to the next larger double precision number from n in MATLAB. So, eps(1) = 2.2204e-16 means that the first number after 1 is 1 + 2.2204e-16. Similarly, eps(3000) = 4.5475e-13. Now, let's look at the precision of you calculations:
n = 100;
A = rand(n);
inv_A_1 = inv(A);
inv_A_2 = A \ eye(n);
max(max(abs(inv_A_1-inv_A_2)))
ans =
1.6431e-14
eps(127) = 1.4211e-14
eps(128) = 2.8422e-14
For integers, the largest number you can use that has an accuracy higher than the max difference between your two matrices is 127.
Now, let's check how the accuracy when we try to recreate the identity matrix from the two inverse matrices.
error_1 = max(max(abs((A\eye(size(A))*A) - eye(size(A)))))
error_1 =
3.1114e-14
error_2 = max(max(abs((inv(A)*A) - eye(size(A)))))
error_2 =
2.3176e-14
The highest integer with a higher accuracy than the maximum difference between the two approaches is 255.
In summary, inv(A) is more accurate, but once you start using the inverse matrices, they are for all intended purposes identical.
Now, let's have a look at the performance of the two approaches:
n = fix(logspace(1,3,40));
for i = 1:numel(n)
A = rand(round(n(i)));
t1(i) = timeit(#()inv(A));
t2(i) = timeit(#()A\eye(n(i)));
end
loglog(n,[t1;t2])
It appears that which of the two approaches is fastest is dependent on the matrix size. For instance, using inv is slower for n = 255, but faster for n = 256.
In summary, choose approach based on what's important to you. For most intended purposes, the two approaches are identical.
Note that svd and pinv may be of interest if you're working with badly scaled matrices. If it's really really important you should consider the Symbolic toolbox.
I know you said that you "actually need the inverse", but I can't let this go unsaid: Using inv(A)*b is never the best approach for solving a linear equation! I won't explain further as I think you know this already.
If you need the inverse, you should use inv.
The inverse is calculated via LU decomposition, whereas the backslash operator mldivide calculates the solution to your linear system using different methods depending on the properties of your matrix A (see https://scicomp.stackexchange.com/a/1004), which can yield less accurate results for the inverse.
It should be noted that if you want to solve a linear system, the calculation is likely going to be much faster and more accurate using mldivide(\). The MATLAB documentation of inv is basically one big warning not to use inv to solve linear systems.
Just a way trying to check this, not sure if it's completely helpful though: multiply your inverse matrix result back with it's original version and check the deviation from the identity matrix:
A = rand( 111 );
E1 = abs( (A\eye(size(A) ) * A ) - eye( size(A) ) );
E2 = abs( ( inv(A) * A ) - eye( size(A) ) );
mean(E1(:))
mean(E2(:))
inv seems to be more accurate as I would have expected. Maybe somebody can re-evaluate this. ;)
Related
This is my matlab code I wrote for a problem I got as homework. after multiplication of A and its transpose the resulting square matrix should have determinant zero according all classmates as their codes (different one) gave them so. Why is my code not giving the determinant of c and d to be infinity
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
detc = det(c);
cinv = inv((A.')*A);
d = A*(A.');
detd = det(d);
dinv = inv(A*(A.'));
x1 = (inv((A.')*A))*((A.')*b);
x2 = A.'*((inv(A*(A.')))*b);
This behavior is explained in the Limitations section of the det's documentation and exemplified in the Find Determinant of Singular Matrix subsection where it is stated:
The determinant of A is quite large despite the fact that A is singular. In fact, the determinant of A should be exactly zero! The inaccuracy of d is due to an aggregation of round-off errors in the MATLAB® implementation of the LU decomposition, which det uses to calculate the determinant.
That said, in this instance, you can produce your desired result by using the m-code implementation given on that same page but sorting the diagonal elements of U in an ascending matter. Consider the sample script:
clc();
clear();
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
[L,U] = lu(c);
% Since det(L) is always (+/-)1, it doesn't impact anything
diagU = diag(U);
detU1 = prod(diagU);
detU2 = prod(sort(diagU,'descend'));
detU3 = prod(sort(diagU,'ascend'));
fprintf('Minimum: %+9.5e\n',min(abs(diagU)));
fprintf('Maximum: %+9.5e\n',max(abs(diagU)));
fprintf('Determinant:\n');
fprintf('\tNo Sort: %g\n' ,detU1);
fprintf('\tDescending Sort: %g\n' ,detU2);
fprintf('\tAscending Sort: %g\n\n',detU3);
This produces the output:
Minimum: +1.53111e-13
Maximum: +1.72592e+02
Determinant:
No Sort: Inf
Descending Sort: Inf
Ascending Sort: 0
Notice that the direction of the sort matters, and that no-sorting gives Inf since a true 0 doesn't exist on the diagonal. The descending sort sees the largest values multiplied first, and apparently, they exceed realmax and are never multiplied by a true 0, which would generate a NaN. The ascending sort clumps together all of the near-zero diagonal values with very few large negative values (in truth, a more robust method would sort based on magnitude, but that was not done here), and their multiplication generates a true 0 (meaning that the value falls below the smallest denormalized number available in IEEE-754 arithmetic) that produces the "correct" result.
All that written, and as others have implied, I'll quote original Matlab developer and Mathworks co-founder Cleve Moler:
[The determinant] is useful in theoretical considerations and hand calculations, but does not provide a sound basis for robust numerical software.
Ok. So the fact that det(A'*A) is not zero is not a good indication of the (non-)singularity of A'*A.
The determinant depends on the scaling, and matrix clearly non-singular can have very small determinant. For instance, the matrix
1/2 * I_n
where I_n is the nxn identity has a determinant of (1/2)^n which is converging (quickly) to 0 as n goes to infinity. But 1/2 * I_n is not, at all, singular.
For this reason, a best idea to check the singularity of a matrix is the condition number.
In you case, after doing some tests
>> A = rand(500, 1500) ;
>> det(A'*A)
ans =
Inf
You can see that the (computed) determinant is clearly non-zero. But this is actually not surprising, and it should not really bother you. The determinant is fairly hard to compute, so yes, it is just rounding errors. If you want a better approximation, you can do the following
>> s = eig(A'*A) ;
>> prod(s)
ans =
0
There, you see it is closer to zero.
The condition number, on the other hand, is a much better estimator of the (non-)singularity of a matrix. Here, it is
>> cond(A'*A)
ans =
1.4853e+20
And, since it is much larger than 1e+16, the matrix is clearly singular. The reason for 1e+16 is a bit tedious, but is mostly due to the computer precision when doing floating point computations.
I think this is pretty much just a rounding problem, the Inf does not mean you are getting Infinity as an answer, it's just that your determinant is really big and exceeded realmax. As Adiel said, A*A.' generates a symmetric matrix, and should have a numerical value for its determinant. for example, set:
A=rand(5,15)
and you should find that the det of A*A.' is just a numerical value.
SO how did your friends get a ZERO, well it's easy to get 0 or inf for det of large matrices (why are you doing this in the first place I have no clue). So I think they are just getting the same/similar rounding issue.
When I try to calculate a matrix inverse using Matlab's inv() operation:
A = rand(10,10);
b = rand(10,1);
C = inv(A);
D = C*b;
I get the following warning at the last row: INV is slow and inaccurate. Use A\b for INV(A)*b , and b/A for b*INV(A).
I can change the code above into:
A = rand(10,10);
b = rand(10,1);
C = inv(A);
D = A\b;
Now I don't get the warning, but I don't think this solution is better.
Note: I need to store both the inverse of matrix A as well as inv(A)*c. Also, in my real file the size of matrix A can be 5000 x 5000 or even bigger.
Are there any better solutions in terms of efficiency and accuracy or is the first method fine?
You should listen to Matlab and use the second option. inv(A)*b and A\b are computed with different algorithms under the hood, and \ is indeed more accurate.
The documentation for inv states:
In practice, it is seldom necessary to form the explicit inverse of a matrix. A frequent misuse of inv arises when solving the system of linear equations Ax = b. One way to solve this is with x = inv(A)*b. A better way, from both an execution time and numerical accuracy standpoint, is to use the matrix division operator x = A\b. This produces the solution using Gaussian elimination, without forming the inverse. See mldivide () for further information.
Some additional information:
If you are to calculate
Ax = b
For many different b's, but with a constant A, you might want to pre-factorize A. That is:
[L U P] = lu(A);
x = (U \ (L \ ( P * b)));
Don't know about other fields, but this occurs frequently in power system engineering at least.
If you absolutely need the inverse later on, then you have to compute it. If you can use the backslash operator (\) instead of an inverse again later on, I would stay away from the inverse and listen to MATLAB's suggestion. For numerical reasons, it is always better to use a slash operator when you can, so the second approach is better even though it is slower.
I have a cell array myBasis of sparse matricies B_1,...,B_n.
I want to evaluate with Matlab the matrix Q(i,j) = trace (B^T_i * B_j).
Therefore, I wrote the following code:
for i=1:n
for j=1:n
B=myBasis{i};
C=myBasis{j};
Q(i,j)=trace(B'*C);
end
end
Which takes already 68 seconds when n=1226 and B_i has 50 rows, and 50 colums.
Is there any chance to speed this up? Usually I exclude for-loops from my matlab code in a c++ file - but I have no experience how to handle a sparse cell array in C++.
As noted by Inox Q is symmetric and therefore you only need to explicitly compute half the entries.
Computing trace( B.'*C ) is equivalent to B(:).'*C(:):
trace(B.'*C) = sum_i [B.'*C]_ii = sum_i sum_j B_ij * C_ij
which is the sum of element-wise products and therefore equivalent to B(:).'*C(:).
When explicitly computing trace( B.'*C ) you are actually pre-computing all k-by-k entries of B.'*C only to use the diagonal later on. AFAIK, Matlab does not optimize its calculation to save it from computing all the entries.
Here's a way
for ii = 1:n
B = myBasis{ii};
for jj = ii:n
C = myBasis{jj};
t = full( B(:).'*C(:) ); % equivalent to trace(B'*C)!
Q(ii,jj) = t;
Q(jj,ii) = t;
end
end
PS,
It is best not to use i and j as variable names in Matlab.
PPS,
You should notice that ' operator in Matlab is not matrix transpose, but hermitian conjugate, for actual transpose you need to use .'. In most cases complex numbers are not involved and there is no difference between the two operators, but once complex data is introduced, confusing between the two operators makes debugging quite a mess...
Well, a couple of thoughts
1) Basic stuff: A'*B = (B'*A)' and trace(A) = trace(A'). Well, only this trick cut your calculations by almost 50%. Your Q(i,j) matrix is symmetric, and you only need to calculate n(n+1)/2 terms (and not n²)
2) To calculate the trace you don't need to calculate every term of B'*C, just the diagonal. Nevertheless, I don't know if it's easy to create a script in Matlab that is actually faster then just calculating B'*C (MatLab is pretty fast with matrix operations).
But I would definitely implement (1)
I have been give a very large matrix (I cannot change the values of the matrix) and I need to calculate the inverse of a (covariance) matrix.
Sometimes I get the error saying
Matrix is close to singular or badly scaled.
Results may be inaccurate
In these situations I see that the value of the det returns 0.
Before calculating inverse (of a covariance matrix) I want to check the value of the det and perform something like this
covarianceFea=cov(fea_class);
covdet=det(covarianceFea);
if(covdet ==0)
covdet=covdet+.00001;
%calculate the covariance using this new det
end
Is there any way to use the new det and then use this to calculate the inverse of the covariance matrix?
Sigh. Computation of the determinant to determine singularity is a ridiculous thing to do, utterly so. Especially so for a large matrix. Sorry, but it is. Why? Yes, some books tell you to do it. Maybe even your instructor.
Analytical singularity is one thing. But how about numerical determination of singularity? Unless you are using a symbolic tool, MATLAB uses floating point arithmetic. This means it stores numbers as floating point, double precision values. Those numbers cannot be smaller in magnitude than
>> realmin
ans =
2.2251e-308
(Actually, MATLAB goes a bit lower than that, in terms of denormalized numbers, which can go down to approximately 1e-323.) See that when I try to store a number smaller than that, MATLAB thinks it is zero.
>> A = 1e-323
A =
9.8813e-324
>> A = 1e-324
A =
0
What happens with a large matrix? For example, is this matrix singular:
M = eye(1000);
Since M is an identity matrix, it is fairly clearly non-singular. In fact, det does suggest that it is non-singular.
>> det(M)
ans =
1
But, multiply it by some constant. Does that make it non-singular? NO!!!!!!!!!!!!!!!!!!!!!!!! Of course not. But try it anyway.
>> det(M*0.1)
ans =
0
Hmm. Thats is odd. MATLAB tells me the determinant is zero. But we know that the determinant is 1e-1000. Oh, yes. Gosh, 1e-1000 is smaller, by a considerable amount than the smallest number that I just showed you that MATLAB can store as a double. So the determinant underflows, even though it is obviously non-zero. Is the matrix singular? Of course not. But does the use of det fail here? Of course it will, and this is completely expected.
Instead, use a good tool for the determination of singularity. Use a tool like cond, or rank. For example, can we fool rank?
>> rank(M)
ans =
1000
>> rank(M*.1)
ans =
1000
See that rank knows this is a full rank matrix, regardless of whether we scale it or not. The same is true of cond, computing the condition number of M.
>> cond(M)
ans =
1
>> cond(M*.1)
ans =
1
Welcome to the world of floating point arithmetic. And oh, by the way, forget about det as a tool for almost any computation using floating point arithmetic. It is a poor choice almost always.
Woodchips has given you a very good explanation for why you shouldn't use the determinant. This seems to be a common misconception and your question is very related to another question on inverting matrices: Is there a fast way to invert a matrix in Matlab?, where the OP decided that because the determinant of his matrix was 1, it was definitely invertible! Here's a snippet from my answer
Rather than det(A)=1, it is the condition number of your matrix that dictates how accurate or stable the inverse will be. Note that det(A)=∏i=1:n λi. So just setting λ1=M, λn=1/M and λi≠1,n=1 will give you det(A)=1. However, as M → ∞, cond(A) = M2 → ∞ and λn → 0, meaning your matrix is approaching singularity and there will be large numerical errors in computing the inverse.
You can test this in MATLAB with the following simple example:
A = eye(10);
A([1 2]) = [1e15 1e-15];
%# calculate determinant
det(A)
ans =
1
%# calculate condition number
cond(A)
ans =
1.0000e+30
In such a scenario, calculating an inverse is not a very good idea. If you just have to do it, I would suggest using this to increase display precision:
format long;
Other suggestion could be to try using an SVD of the matrix and tinker around with singular values there.
A = U∑V'
inv(A) = V*inv(∑)*U'
∑ is a diagonal matrix where you will see one of the diagonal entries close to 0. Try playing around with this number if you want some sort of an approximation.
Suppose I have an arbitrary transformation matrix A such as,
A =
0.9966 0.0007 -6.5625
0.0027 0.9938 1.0598
0 0 1.0000
And a set of points such that their x and y coordinates are represented by X and Y respectively.
And suppose,
[Xf Yf] = tformfwd(maketform('projective',A),X,Y);
Now,
[Xff Yff] = tformfwd(maketform('projective',inv(A)),Xf,Yf);
[Xfi Yfi] = tforminv(maketform('projective',A),Xf,Yf);
[Xff Yff] and [Xfi Yfi] seem to be exactly the same (and they should).
Is tforminv just there for convenience or am I missing something here?
I'll preface this by saying it is my best guess...
It's possible that tforminv may perform the transformation without actually forming the inverse matrix. For example, you can solve a system of linear equations Ax = b in two ways:
x = inv(A)*b;
x = A\b;
According to the documentation for inv, the second option (using the matrix division operator) can perform better "from both an execution time and numerical accuracy standpoint" since it "produces the solution using Gaussian elimination, without forming the inverse". tforminv may do something similar and thus show better overall behavior compared with passing the inverse matrix to tformfwd.
If you were so inclined, you could probably try a number of different transformation matrices and test the two approaches (tforminv or tformfwd and inv) to see how accurate the results are and how fast they are each computed.