I observed a simple, yet particular behavior when using the determinant function in MATLAB and I would like to get some explanations as I didn't find anything about it in the function help documentation.
I'm generating a random unitary matrix Q with the following code:
[Q, R] = qr(randn(3));
After that, I evaluate the determinant of Q with the det function:
det(Q)
I would expect the result to be -1.000 or 1.000. However, the format doesn't seem to be constant. So when I do something like this:
detResults = zeros(100,1);
for ii = 1:100
[Q, R] = qr(randn(3));
detResults(ii,1) = det(Q);
end
the detResults vector contains 1.000 and sometime 1. Is it just a print format issue or is it caused by something else?
It's related to floating-point precision. Each time you iterate through that loop, even though you will theoretically get a determinant of 1 for the Q matrix, the numbers in the matrices themselves are irrational so theoretically the only way you will get the value of 1 is when your numbers are represented with infinite precision. Sometimes there are enough digits so that MATLAB can safely round to 1 with assurance. Also, you're not getting the full picture. The reason why you see 1.0000 and 1 is also related to the print format. The default print format only shows up to five decimal places but it may be prudent to show more decimal places to appreciate the bigger picture.
Here's a small example with using only 10 iterations instead of 100.
Using the default print format:
>> detResults
detResults =
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
Using a format of increased precision (just for display purposes) with format long g:
>> format long g;
>> detResults
detResults =
1
0.999999999999999
1
1
0.999999999999999
1
1
0.999999999999999
1
0.999999999999999
Internally, it really depends on what the Q matrix is and what you get out of the bag when you generate the random matrices. However, as far as precision goes for using these for further calculations, 0.999... is very close to 1 so for all intents and purposes you should consider it equal to 1.
I believe that you are observing the effect of the finite precision of the floating point number representation. By default, MATLAB uses 64-bit floating point numbers. So only a finite set of number, with at most 2^64 unique elements, can be represented by this system exactly. All other numbers, resulting during intermediate computations, are rounded to the nearest representable values. These rouding operations result in errors, which are negligible for most, but not all, applications.
You can estimate the errors in your results by appending this line to your code:
err = detResults - 1;
A simple example to observe the finite-precision artifact is:
2-(sqrt(2))^2
Obviously, this should be exactly 0. But MATLAB would return a non-zero small number because of rounding errors in the square-root and squared steps.
Related
Basically I wanted to ask two things:
Why does this happen? (negative zero in Matlab)
When does this happen?
I came up with this. Octave has some similarities with Matlab, so the usefulness of this feature is clear, but one of the things they said, is that it does not appear in the default output. and I just tackled it right now. So, maybe a new insight on this?
For the second question, in the answered question I referred to, they just said it could happen in some calculations, and in the following calculation which I just did, it doesn't seem really necessary to use (or get) that negative zero.
The code where I encountered this is:
xcorr([1 0 1 1], [0 1 1 0 0])
where it's output is:
-0.0000 -0.0000 1.0000 1.0000 1.0000 2.0000 1.0000 0.0000 0.0000
The xcorr is actually a cross corelation function, which does only some simple operations like summing and multiplications, where it's exact function details can be found here. Anyway, nothing like "complex branch cuts and transformations of the complex plane"
Thanks
These values do not represent zeros. Instead, they are negative values which are very close to zero.
The reason for getting these values and not simply zeros is due to approximations which are performed in the function implementation. According to Matlab documentation: "xcorr estimates the cross-correlation sequence of a random process".
In other words - the values which are displayed on the screen are just approximations for negative values.
In order to test this, you can change the display format of Matlab.
code:
format shortE;
xcorr([1 0 1 1], [0 1 1 0 0])
Result:
ans =
Columns 1 through 5
-6.2450e-017 -5.5511e-017 1.0000e+000 1.0000e+000 1.0000e+000
Columns 6 through 9
2.0000e+000 1.0000e+000 1.1102e-016 1.1796e-016
As you can see, the values in coordinates 1,2,8 and 9 are actually negative.
In the specific sample case, the -0.0000 turned out to actually be tiny non-zero negative numbers. However, in an effort to make printout human readable, Matlab and Octave tries to avoid using scientific notation in printing. As a result, when mixed with big numbers, small number are reduced to 0.0000 or -0.000. This can be changed by setting the default preferences in Matlab or Octave.
But this is NOT the only answer to the asked questions:
Why does this happen? (negative zero in Matlab),
When does this happen?
In fact, in Matlab and Octave, and any computing environment that works with floating points, -0. really a thing. This is not a bug in the computing environment, but rather an allowance in the IEEE-754 standard for binary representation of a floating point number, and it is a product of the floating point processor in the CPU (not the programming language).
Whereas binary representation of integer does not have a special sign bit, IEEE-754 reserves a bit for the sign, separate from the number. So while the number part might mean 0, the sign bit is left to mean negative.
It happens wherever your CPU (Intel or AMD) does a product with 0. and any negative number (including -0.). I don't know if it is required by IEEE-754 for that to happen or if it simply the result of CPU design optimization (maximize speed, minimize size).
In either case, this -0. is a non-issue in that IEEE-754 requires it to be comparatively and arithmetically exactly the same as 0.. That is:
-0. < 0 --> FALSE
-0. == 0 --> TRUE
1+ -0. == 1 --> TRUE
etc...
This is my matlab code I wrote for a problem I got as homework. after multiplication of A and its transpose the resulting square matrix should have determinant zero according all classmates as their codes (different one) gave them so. Why is my code not giving the determinant of c and d to be infinity
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
detc = det(c);
cinv = inv((A.')*A);
d = A*(A.');
detd = det(d);
dinv = inv(A*(A.'));
x1 = (inv((A.')*A))*((A.')*b);
x2 = A.'*((inv(A*(A.')))*b);
This behavior is explained in the Limitations section of the det's documentation and exemplified in the Find Determinant of Singular Matrix subsection where it is stated:
The determinant of A is quite large despite the fact that A is singular. In fact, the determinant of A should be exactly zero! The inaccuracy of d is due to an aggregation of round-off errors in the MATLAB® implementation of the LU decomposition, which det uses to calculate the determinant.
That said, in this instance, you can produce your desired result by using the m-code implementation given on that same page but sorting the diagonal elements of U in an ascending matter. Consider the sample script:
clc();
clear();
A = rand(500,1500);
b = rand(500,1);
c = (A.')*A;
[L,U] = lu(c);
% Since det(L) is always (+/-)1, it doesn't impact anything
diagU = diag(U);
detU1 = prod(diagU);
detU2 = prod(sort(diagU,'descend'));
detU3 = prod(sort(diagU,'ascend'));
fprintf('Minimum: %+9.5e\n',min(abs(diagU)));
fprintf('Maximum: %+9.5e\n',max(abs(diagU)));
fprintf('Determinant:\n');
fprintf('\tNo Sort: %g\n' ,detU1);
fprintf('\tDescending Sort: %g\n' ,detU2);
fprintf('\tAscending Sort: %g\n\n',detU3);
This produces the output:
Minimum: +1.53111e-13
Maximum: +1.72592e+02
Determinant:
No Sort: Inf
Descending Sort: Inf
Ascending Sort: 0
Notice that the direction of the sort matters, and that no-sorting gives Inf since a true 0 doesn't exist on the diagonal. The descending sort sees the largest values multiplied first, and apparently, they exceed realmax and are never multiplied by a true 0, which would generate a NaN. The ascending sort clumps together all of the near-zero diagonal values with very few large negative values (in truth, a more robust method would sort based on magnitude, but that was not done here), and their multiplication generates a true 0 (meaning that the value falls below the smallest denormalized number available in IEEE-754 arithmetic) that produces the "correct" result.
All that written, and as others have implied, I'll quote original Matlab developer and Mathworks co-founder Cleve Moler:
[The determinant] is useful in theoretical considerations and hand calculations, but does not provide a sound basis for robust numerical software.
Ok. So the fact that det(A'*A) is not zero is not a good indication of the (non-)singularity of A'*A.
The determinant depends on the scaling, and matrix clearly non-singular can have very small determinant. For instance, the matrix
1/2 * I_n
where I_n is the nxn identity has a determinant of (1/2)^n which is converging (quickly) to 0 as n goes to infinity. But 1/2 * I_n is not, at all, singular.
For this reason, a best idea to check the singularity of a matrix is the condition number.
In you case, after doing some tests
>> A = rand(500, 1500) ;
>> det(A'*A)
ans =
Inf
You can see that the (computed) determinant is clearly non-zero. But this is actually not surprising, and it should not really bother you. The determinant is fairly hard to compute, so yes, it is just rounding errors. If you want a better approximation, you can do the following
>> s = eig(A'*A) ;
>> prod(s)
ans =
0
There, you see it is closer to zero.
The condition number, on the other hand, is a much better estimator of the (non-)singularity of a matrix. Here, it is
>> cond(A'*A)
ans =
1.4853e+20
And, since it is much larger than 1e+16, the matrix is clearly singular. The reason for 1e+16 is a bit tedious, but is mostly due to the computer precision when doing floating point computations.
I think this is pretty much just a rounding problem, the Inf does not mean you are getting Infinity as an answer, it's just that your determinant is really big and exceeded realmax. As Adiel said, A*A.' generates a symmetric matrix, and should have a numerical value for its determinant. for example, set:
A=rand(5,15)
and you should find that the det of A*A.' is just a numerical value.
SO how did your friends get a ZERO, well it's easy to get 0 or inf for det of large matrices (why are you doing this in the first place I have no clue). So I think they are just getting the same/similar rounding issue.
I have observations from 2 variables Zz = [x y]; where rows are the observations and columns are the variables. I wanted to obtain statistical measures like if they are uncorrelated, i.i.d, mean and variance would like to know if they are independent indentical distribution and uncorrelated.
For uncorrelated I used corrcoef(Zz)
ans =
1.0000 -0.0300
-0.0300 1.0000
To check which pdf they follow, I used the command hist(Zz)
To check for covariance:
cov(Zz)
ans =
211.5004 -6.2984
-6.2984 208.3215
What should be the values to determine if the random varaibles are uncorrelated i.i.d and what can I conclude from these results?
You need to do some statistical tests on the null hypothesis that the true correlation coefficient ρ is 0. You can check the details here http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Inference. Matlab provides you with the p-values on this hypothesis, and the upper and lower bounds
[R,P,RLO,RUP]=corrcoef(Zz)
An example of bootstrap analysis is shown here http://www.mathworks.nl/help/stats/bootstrp.html
Is it possible to find the covariance of a matrix without using any built-in functions or loops in MATLAB? I'm completely clueless about the idea of solving this problem.
I was thinking of something like:
cov(x,y) = 1/(n-1) .* (x*y)
However, I don't think this will work. Any ideas?
Here's a great example of how to numerically compute the covariance matrix. http://www.itl.nist.gov/div898/handbook/pmc/section5/pmc541.htm. However, let's put this in this post for the sake of completeness. I'm a bit confused with what you mean by "built-in" functions because the covariance requires that you sum over columns of a matrix. If you can't use any built-in functions to sum up these elements, then I don't see how you can do this without using for loops. Edit: I figured out how to do it without using built-in functions or loops, but you need to use size to determine how many rows in the matrix you have... unless you specify this as a constant in your function.
Numerically, you compute the covariance matrix like so:
Essentially, the ith row and the jth column of your covariance matrix is such that you take the sum of products of the column i minus the mean of column i with column j minus the mean of column j. Now, add these up, then divide by n - 1. This is known as the unbiased estimator. You'll also notice that this matrix is symmetric because even if you flip the order around (i.e. looking at column j then column i after), the answer should still be the same. I'm assuming you can't use mean from MATLAB either so let's do this from first principles.
First, compute a row vector that computes the mean of every column. What you can do to compute the sum over all of the columns without using sum, as it is also a built-in function, is multiply this row vector of 1s with your matrix A, The output will be a row vector that contains the sum over all of the columns. As such, do this:
one_vector(1:size(A,1)) = 1;
mu = (one_vector * A) / size(A,1);
The trick with the first line of code is that we are dynamically creating an array that is of the same length as the number of rows in your matrix A. We fill this completely full of 1s. Note that you could have used ones, but you said you can't use any built-in functions. mu will contain our vector over all columns.
Now, let's pre-process the data by subtracting every column with the mean, since that's what the definition says we do. To do this without any built-in functions, what you can do is to subtract all of the columns with their own respective means, duplicate mu for as many times as we have 1s in the one_vector. Therefore:
A_mean_subtract = A - mu(one_vector, :);
Here's where it gets a bit tricky (and cool). If we transpose the matrix A, you'll see that the rows become the columns and the columns become the rows. If we took this transpose and multiplied by the original matrix, we would actually get the sum of products between column i and column j of our matrix A. That's the first part of our covariance calculation. We then divide by n - 1. Therefore, our covariance is simply:
covA = (A_mean_subtract.' * A_mean_subtract) / (size(A,1) - 1);
Here's a quick example, as well as what is seen on that website I showed you above. Supposing A was this:
A = [4 2 0.5; 4.2 2.1 0.59; 3.9 2.0 0.58; 4.3 2.1 0.62; 4.1 2.2 0.63]
A =
4.0000 2.0000 0.5000
4.2000 2.1000 0.5900
3.9000 2.0000 0.5800
4.3000 2.1000 0.6200
4.1000 2.2000 0.6300
Running through the above code, this is what we get:
covA =
0.0250 0.0075 0.0042
0.0075 0.0070 0.0034
0.0042 0.0034 0.0026
You'll see that this also matches with the cov function in MATLAB too:
>> cov(A)
ans =
0.0250 0.0075 0.0042
0.0075 0.0070 0.0034
0.0042 0.0034 0.0026
Bit of a hint
If you type in edit cov in your MATLAB command prompt, you can actually see how they compute the covariance matrix without any for loops.... and this is essentially the same answer I gave you :)
If you want to do this more efficiently
Assuming you can use sum and bsxfun, we can do this in fewer (and more efficiently..) lines of code. First, compute your mean vector like we did above using sum:
mu = sum(A) / size(A,1);
Now, to subtract your matrix A with each column's corresponding mean, you can use bsxfun to help you facilitate this subtraction:
A_mean_subtract = bsxfun(#minus, A, mu);
Now, compute your covariance matrix like you did before:
covA = (A_mean_subtract.' * A_mean_subtract) / (size(A,1) - 1);
You should get exactly the same result as we saw before.
Minor note on stability
We are using the straight up definition of calculating the covariance between two columns using the definition. However, it has been shown that using the straight up definition can lend to numerical instability if you provide certain types of data. Consult this Wikipedia page that goes through various algorithms on computing the covariance between two n length vectors that are more stable.
I have been give a very large matrix (I cannot change the values of the matrix) and I need to calculate the inverse of a (covariance) matrix.
Sometimes I get the error saying
Matrix is close to singular or badly scaled.
Results may be inaccurate
In these situations I see that the value of the det returns 0.
Before calculating inverse (of a covariance matrix) I want to check the value of the det and perform something like this
covarianceFea=cov(fea_class);
covdet=det(covarianceFea);
if(covdet ==0)
covdet=covdet+.00001;
%calculate the covariance using this new det
end
Is there any way to use the new det and then use this to calculate the inverse of the covariance matrix?
Sigh. Computation of the determinant to determine singularity is a ridiculous thing to do, utterly so. Especially so for a large matrix. Sorry, but it is. Why? Yes, some books tell you to do it. Maybe even your instructor.
Analytical singularity is one thing. But how about numerical determination of singularity? Unless you are using a symbolic tool, MATLAB uses floating point arithmetic. This means it stores numbers as floating point, double precision values. Those numbers cannot be smaller in magnitude than
>> realmin
ans =
2.2251e-308
(Actually, MATLAB goes a bit lower than that, in terms of denormalized numbers, which can go down to approximately 1e-323.) See that when I try to store a number smaller than that, MATLAB thinks it is zero.
>> A = 1e-323
A =
9.8813e-324
>> A = 1e-324
A =
0
What happens with a large matrix? For example, is this matrix singular:
M = eye(1000);
Since M is an identity matrix, it is fairly clearly non-singular. In fact, det does suggest that it is non-singular.
>> det(M)
ans =
1
But, multiply it by some constant. Does that make it non-singular? NO!!!!!!!!!!!!!!!!!!!!!!!! Of course not. But try it anyway.
>> det(M*0.1)
ans =
0
Hmm. Thats is odd. MATLAB tells me the determinant is zero. But we know that the determinant is 1e-1000. Oh, yes. Gosh, 1e-1000 is smaller, by a considerable amount than the smallest number that I just showed you that MATLAB can store as a double. So the determinant underflows, even though it is obviously non-zero. Is the matrix singular? Of course not. But does the use of det fail here? Of course it will, and this is completely expected.
Instead, use a good tool for the determination of singularity. Use a tool like cond, or rank. For example, can we fool rank?
>> rank(M)
ans =
1000
>> rank(M*.1)
ans =
1000
See that rank knows this is a full rank matrix, regardless of whether we scale it or not. The same is true of cond, computing the condition number of M.
>> cond(M)
ans =
1
>> cond(M*.1)
ans =
1
Welcome to the world of floating point arithmetic. And oh, by the way, forget about det as a tool for almost any computation using floating point arithmetic. It is a poor choice almost always.
Woodchips has given you a very good explanation for why you shouldn't use the determinant. This seems to be a common misconception and your question is very related to another question on inverting matrices: Is there a fast way to invert a matrix in Matlab?, where the OP decided that because the determinant of his matrix was 1, it was definitely invertible! Here's a snippet from my answer
Rather than det(A)=1, it is the condition number of your matrix that dictates how accurate or stable the inverse will be. Note that det(A)=∏i=1:n λi. So just setting λ1=M, λn=1/M and λi≠1,n=1 will give you det(A)=1. However, as M → ∞, cond(A) = M2 → ∞ and λn → 0, meaning your matrix is approaching singularity and there will be large numerical errors in computing the inverse.
You can test this in MATLAB with the following simple example:
A = eye(10);
A([1 2]) = [1e15 1e-15];
%# calculate determinant
det(A)
ans =
1
%# calculate condition number
cond(A)
ans =
1.0000e+30
In such a scenario, calculating an inverse is not a very good idea. If you just have to do it, I would suggest using this to increase display precision:
format long;
Other suggestion could be to try using an SVD of the matrix and tinker around with singular values there.
A = U∑V'
inv(A) = V*inv(∑)*U'
∑ is a diagonal matrix where you will see one of the diagonal entries close to 0. Try playing around with this number if you want some sort of an approximation.