I have a matrix that is a function of some parameter A=A(x). I would like to find the points x where this matrix becomes singular. Example (I have a large matrix though):
syms x
A=[x sin(x); cos(x^2) 2.5];
So far I have been symbolically computing the determinant of the matrix and then used fzero or newtzero to find the roots of that characteristic equation. I.e.
detA = det(A);
fzero(matlabFunction(detA),startingGuess)
Then I found this: How to find out if a matrix is singular?, where it is advocated to not use the determinant under any circumstances.
Indeed the symbolic determinant calculation is terribly slow. However I tried to use rank(A) instead as suggested in the link and it does not seem to work for symbolic matrices.
Is there any way to implement the suggestions in the link for finding the roots of a characteristic equation of a matrix that is given symbolically?
A possible approach would be the following: a square matrix A is singular if and only if the homogeneous linear (with respect to the vector y) system A*y = 0 has nontrivial solutions y <> 0 (which is equivalent to det(A) = 0 and rank(A) = 0 among others. So a more or less standard, as I recall from the past, technique to compute such points x is to solve the nonlinear system
A(x)*y = 0 (1)
||y|| = 1 (2)
This way you can compute a point x* and a vector y* such that A(x*) is singular and y* is an eigenvector corresponding to the zero eigenvalue of A(x*).
If I remember correctly, you can also solve the somewhat easier system
A(x)*y = 0 (1)
<y,c> = 1 (2a)
where c is "almost" any nonzero random vector (normalize it to 1 to avoid numerical problems).
As a matter of fact there is an enormous bibliography on the subject - you can look for saddle-node bifurcation computations (in case A(x) is the Jacobian of a vector field), or for "distance to instability".
From a discussion with Ander Biguri it seems that the determinant is actually a perfectly fine method of approaching this problem. The problem seems to be to solve the final equation in a stable manner, which would be a different question.
Related
I have the following matrix
R=(A-C)*inv(A+B-C-C')*(A-C');
where A and B are n by n matrices. I want to find n*n matrix C such that the determinant of R is minimized, SO:
C=arg min (det(R));
Is there any function in MATLAB that can handle this problem?
It seems like you are trying to find the minimum of an unconstrained multivariable function. This can probably be achieved with fminunc
fun = #(x)x(1)*exp(-(x(1)^2 + x(2)^2)) + (x(1)^2 + x(2)^2)/20;
x0 = [1,2];
[x,fval] = fminunc(fun,x0)
Note that there are no examples in the documentation where a matrix is used, this is probably because horrendous performance could be expected when trying to solve this problem for a matrix of any nontiny size. (This is not because of matlab, but because of the nature of the problem).
It is also good to realize that this method does not (cannot) guarantee an optimum, only a local optimum.
So I have these three vectors:
And I have to find out for what value of k these three vectors are linearly dependent. I have tried using rref and linsolve with syms for this but that did not work out. I'm relatively new to MatLab and matrices so please keep that in mind.
I know in order to check if vectors are linearly dependent that c1...cn have to be non-zero.
I also want to know how you can use variables in general when solving these types of equations in MatLab.
A set of vectors (at least if you have n vectors in n dimensions) is linearly dependent if the matrix constructed from them is singular, i.e. if its determinant is 0. If you have the Symbolic Math Toolbox, you can construct a symbolic matrix:
syms k;
M = [1 k 0; -1 1 2; 0 0 3];
det(M)
This will tell you that det(M)==3*k+3, which you can solve by hand. But generally, you can ask matlab to solve it:
solve(det(M)==0,k);
which will tell you the answer is -1. So unless k==-1, these vectors are linearly independent (i.e. they comprise a basis of the Euclidean space R^3).
Update: If you don't have the Symbolic Math Toolbox, you could still try to find a numerical solution. First define a function
detfun=#(k) det([1 k 0; -1 1 2; 0 0 3]);
that for any value of k will give you the determinant of your matrix, for instance detfun(3) gives 12. Then you can use fsolve to find a numerical solution to the equation detfun(k)==0, by calling
fsolve(detfun,0)
in which the second argument, 0, refers to the starting point of the search performed by fsolve. This will tell you that the answer is k==-1, but a single call to fsolve will only give you a single solution. If your function has multiple roots, you have to play around with the starting points to find more of them. In this case, you can know that your function (i.e. det(M(k)) is linear in k, so it has a unique root.
I have to solve in MATLAB a linear system of equations A*x=B where A is symmetric and its elements depend on the difference of the indices: Aij=f(i-j).
I use iterative solvers because the size of A is say 40000x40000. The iterative solvers require to determine the product A*x where x is the test solution. The evaluation of this product turns out to be a convolution and therefore can be done dy means of fast fourier transforms (cputime ~ Nlog(N) instead of N^2). I have the following questions to this problem:
is this convolution circular? Because if it is circular I think that I have to use a specific indexing for the new matrices to take the fft. Is that right?
I find difficult to program the routine for the fft because I cannot understand the indexing I should use. Is there any ready routine which I can use to evaluate by fft directly the product A*x and not the convolution? Actually, the matrix A is constructed of 3x3 blocks and is symmetric. A ready routine for the product A*x would be the best solution for me.
In case that there is no ready routine, could you give me an idea by example how I could construct this routine to evaluate a matrix-vector product by fft?
Thank you in advance,
Panos
Very good and interesting question! :)
For certain special matrix structures, the Ax = b problem can be solved very quickly.
Circulant matrices.
Matrices corresponding to cyclic convolution Ax = h*x (* - is convolution symbol) are diagonalized in
the Fourier domain, and can be solved by:
x = ifft(fft(b)./fft(h));
Triangular and banded.
Triangular matrices and diagonally-dominant banded matrices are solved
efficiently by sparse LU factorization:
[L,U] = lu(sparse(A)); x = U\(L\b);
Poisson problem.
If A is a finite difference approximation of the Laplacian, the problem is efficiently solved by multigrid methods (e.g., web search for "matlab multigrid").
Interesting question!
The convolution is not circular in your case, unless you impose additional conditions. For example, A(1,3) should equal A(2,1), etc.
You could do it with conv (retaining only the non-zero-padded part with option valid), which probably is also N*log(N). For example, let
A = [a b c d
e a b c
f e a b
g f e a];
Then A*x is the same as
conv(fliplr([g f e a b c d]),x,'valid').'
Or more generally, A*x is the same as
conv(fliplr([A(end,1:end-1) A(1,:)]),x,'valid').'
I'd like to add some comments on Pio_Koon's answer.
First of all, I wouldn't advise to follow the suggestion for triangular and banded matrices. The time taken by a call to Matlab's lu() procedure on a large sparse matrix massively overshadows any benefits gained by solving the linear system as x=U\(L\b).
Second, in the Poisson problem you end up with a circulant matrix, therefore you can solve it using the FFT as described. In this specific case, your convolution mask h is a Laplacian, i.e., h=[0 -0.25 0; -0.25 1 -0.25; 0 -0.25 0].
The following is an excerpt from a program of mine:
function [P] = abc(M,f);
if det(M) ~= 1, disp(['Matrix M should have determinant 1'])
I allow the option for the user not to enter a value for 'f'.
When I run abc([2 1; 1 1]), the program works fine and it does what it's supposed to. But when I run abc([6 13; 5 11]) I am told "Matrix M should have determinant 1".
What on Earth is going on?
EDIT: In the command window, I entered the following:
M = [6 13; 5 11];
if det(M) ~= 1, disp('Im broken');
end
Matlab then told me itself that it's broken.
Thanks
Welcome to the wonderfully wacky world of floating point arithmetic. MATLAB computes the determinant using an LU decomposition, i.e., linear algebra. It does so since determinant is wildly inefficient for arrays of even mild size unless it did.
A consequence of that LU decomposition, is the determinant is computed as a floating point number. This is not an issue, UNLESS you have entered a problem as trivially simple as you have - the determinant of a 2x2 matrix composed only of small integers. In that case, the determinant itself will also be a (reasonably) small integer. So you could resolve the issue by simply computing the determinant of the 2x2 matrix yourself, using the textbook formula.
D = A(1,1)*A(2,2) - A(1,2)*A(2,1);
This will be exactly correct for small integer matrices A, although even this may show some loss of precision for SOME matrices. For example, consider the simple, 2x2 matrix A:
>> A = [1e8 1;1 1e8];
We know that the determinant of this matrix is 1e16-1.
>> det(A)
ans =
1e+16
Of course, MATLAB displays this as 1e16. But in fact, the number generated by the det function in MATLAB is actually 9999999999999998, so 1e16-2. As bad, had I used the formula I gave above for the 2x2 determinant, it would have returned a result that is still incorrect, 10000000000000000. Both results were off by 1. You can learn more about these issues by looking at the help for eps.
My point is, there are some 2x2 matrices where computation of the determinant will simply be problematic, even though they are integer matrices.
Once your matrices become non-integer, then things really do become true floating point numbers, not integers. This means you simply MUST use comparisons with tolerances on them rather than a test for exact unity. This is a good rule anyway. Always use a tolerance when you make a test for equality, at least until you have learned enough to know when to disobey that rule!
So, you might choose a test like this:
if abs(det(A) - 1) < (10*eps(1))
warning('The sky is falling! det has failed me.')
end
Note that I've used eps(1), since we are comparing things to 1. The fact that I multiplied eps by 10 allows a wee bit of slop in the computation of the determinant.
Finally, you should know that whatever test you are using the determinant for here, it is often a BBBBBBBBBBAAAAAAAAAADDDDDDDD thing to do! Yes, maybe your teacher told you to do this, or you found something in a textbook. But the determinant is just a bad thing to use for numerical computations. There are almost always alternatives to the determinant. Again, this is called judgement, knowing when that which you are told to use is actually the wrong thing to do.
You are running into the standard problems that occur due to the limitations of floating-point numbers. The result of the det function is probably something like 1.000000001.
General rule-of-thumb: Never test floating-point values for equality.
To give you an insight: det is not computed using the old formula you studied in linear algebra, but using more efficient algorithms.
For example, using Gaussian elimination you can transform M in the equivalent upper triangular matrix and then compute the determinant as product of the main diagonal (being the lower triangle all zeros).
M = [6 13; 5 11]
G = M - [0 0; M(2,1)/M(1,1) * M(1,:)];
Theoretically det(M) is equal to det(G), which is 6 * 1/6 = 1, but being G a floating point and not a real number matrix, G(1,1)*G(2,2)~=1!
In fact G(1,1) and G(2,2) are not exactly 1 and 1/6, but they have a very small relative error (see eps, which on most machines is around 2.22e-16). Their real value will be around 6*(1+eps) and 1/6*(1+eps), thus their product will have a small error too.
I'm not sure if Matlab uses the Gaussian elimination or the similar LU decomposition.
I have been give a very large matrix (I cannot change the values of the matrix) and I need to calculate the inverse of a (covariance) matrix.
Sometimes I get the error saying
Matrix is close to singular or badly scaled.
Results may be inaccurate
In these situations I see that the value of the det returns 0.
Before calculating inverse (of a covariance matrix) I want to check the value of the det and perform something like this
covarianceFea=cov(fea_class);
covdet=det(covarianceFea);
if(covdet ==0)
covdet=covdet+.00001;
%calculate the covariance using this new det
end
Is there any way to use the new det and then use this to calculate the inverse of the covariance matrix?
Sigh. Computation of the determinant to determine singularity is a ridiculous thing to do, utterly so. Especially so for a large matrix. Sorry, but it is. Why? Yes, some books tell you to do it. Maybe even your instructor.
Analytical singularity is one thing. But how about numerical determination of singularity? Unless you are using a symbolic tool, MATLAB uses floating point arithmetic. This means it stores numbers as floating point, double precision values. Those numbers cannot be smaller in magnitude than
>> realmin
ans =
2.2251e-308
(Actually, MATLAB goes a bit lower than that, in terms of denormalized numbers, which can go down to approximately 1e-323.) See that when I try to store a number smaller than that, MATLAB thinks it is zero.
>> A = 1e-323
A =
9.8813e-324
>> A = 1e-324
A =
0
What happens with a large matrix? For example, is this matrix singular:
M = eye(1000);
Since M is an identity matrix, it is fairly clearly non-singular. In fact, det does suggest that it is non-singular.
>> det(M)
ans =
1
But, multiply it by some constant. Does that make it non-singular? NO!!!!!!!!!!!!!!!!!!!!!!!! Of course not. But try it anyway.
>> det(M*0.1)
ans =
0
Hmm. Thats is odd. MATLAB tells me the determinant is zero. But we know that the determinant is 1e-1000. Oh, yes. Gosh, 1e-1000 is smaller, by a considerable amount than the smallest number that I just showed you that MATLAB can store as a double. So the determinant underflows, even though it is obviously non-zero. Is the matrix singular? Of course not. But does the use of det fail here? Of course it will, and this is completely expected.
Instead, use a good tool for the determination of singularity. Use a tool like cond, or rank. For example, can we fool rank?
>> rank(M)
ans =
1000
>> rank(M*.1)
ans =
1000
See that rank knows this is a full rank matrix, regardless of whether we scale it or not. The same is true of cond, computing the condition number of M.
>> cond(M)
ans =
1
>> cond(M*.1)
ans =
1
Welcome to the world of floating point arithmetic. And oh, by the way, forget about det as a tool for almost any computation using floating point arithmetic. It is a poor choice almost always.
Woodchips has given you a very good explanation for why you shouldn't use the determinant. This seems to be a common misconception and your question is very related to another question on inverting matrices: Is there a fast way to invert a matrix in Matlab?, where the OP decided that because the determinant of his matrix was 1, it was definitely invertible! Here's a snippet from my answer
Rather than det(A)=1, it is the condition number of your matrix that dictates how accurate or stable the inverse will be. Note that det(A)=∏i=1:n λi. So just setting λ1=M, λn=1/M and λi≠1,n=1 will give you det(A)=1. However, as M → ∞, cond(A) = M2 → ∞ and λn → 0, meaning your matrix is approaching singularity and there will be large numerical errors in computing the inverse.
You can test this in MATLAB with the following simple example:
A = eye(10);
A([1 2]) = [1e15 1e-15];
%# calculate determinant
det(A)
ans =
1
%# calculate condition number
cond(A)
ans =
1.0000e+30
In such a scenario, calculating an inverse is not a very good idea. If you just have to do it, I would suggest using this to increase display precision:
format long;
Other suggestion could be to try using an SVD of the matrix and tinker around with singular values there.
A = U∑V'
inv(A) = V*inv(∑)*U'
∑ is a diagonal matrix where you will see one of the diagonal entries close to 0. Try playing around with this number if you want some sort of an approximation.