I read at a few places (in the doc and in this blog post : http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/ ) that the use of inv in Matlab is not recommended because it is slow and inaccurate.
I am trying to find the reason of this inaccuracy. As of now, Google did not give m interesting result, so I thought someone here could guide me.
Thanks !
The inaccuracy I mentioned is with the method INV, not MATLAB's implementation of it. You should be using QR, LU, or other methods to solve systems of equations since these methods don't typically require squaring the condition number of the system in question. Using inv typically requires an operation that loses accuracy by squaring the condition number of the original system.
--Loren
I think the point of Loren's blog is not that MATLAB's inv function is particularly slower or more inaccurate than any other numerical implementation of computing a matrix inverse; rather, that in most cases the inverse itself is not needed, and you can proceed by other means (such as solving a linear system using \ - the backslash operator - rather than computing an inverse).
inv() is certainly slower than \ unless you have multiple right hand side vectors to solve for. However, the advice from MathWorks regarding inaccuracy is due to a overly conservative bound in a numerical linear algebra result. In other words, inv() is NOT inaccurate. The link elaborates further : http://arxiv.org/abs/1201.6035
Several widely-used textbooks lead the reader to believe that solving a linear system of equations Ax = b by multiplying the vector b by a computed inverse inv(A) is inaccurate. Virtually all other textbooks on numerical analysis and numerical linear algebra advise against using computed inverses without stating whether this is accurate or not. In fact, under reasonable assumptions on how the inverse is computed, x = inv(A)*b is as accurate as the solution computed by the best backward-stable solvers.
Related
I am studying Stochastic calculus, and occasionally we need to compute an integral (from -infinity to +infinity) for some complex distribution. In this case, it was
with the answer on the right. This is the code I put into Matlab (and I have the symbolic math toolbox), which Matlab simply cannot process:
>> syms x t
>> f = exp(1+2*x)*(1/((2*pi*t)^0.5))*exp(-(x^2)/(2*t))
>> int(f,-inf,inf)
ans =
-((2^(1/2)*pi^(1/2)*exp(2*t + 1)*limit(erf((2^(1/2)*((x*1i)/t - 2i))/(2*(-1/t)^(1/2))), x, -Inf)*1i)/(2*(-1/t)^(1/2)) - (2^(1/2)*pi^(1/2)*exp(2*t + 1)*limit(erf((2^(1/2)*((x*1i)/t - 2i))/(2*(-1/t)^(1/2))), x, Inf)*1i)/(2*(-1/t)^(1/2)))/(2*pi*t)^(1/2)
This answer at the end looks like nonsense, while Wolfram (via their free tool), gives me the answer that the picture above has. Am I missing something fundamental about doing such integrations in Matlab that the basic Mathworks pages don't cover? Where am I proceeding incorrectly?
In order to explain what is happening, we need some theory:
Symbolic systems such as Matlab or Mathematica calculate integrals symbolically by the Risch algorithm (yes, there is a method to mechanically calculate integrals, just like derivatives).
However, the Risch algorithms works differently than using derivation rules. Strictly spoken, it is not an algorithm but a semi-algorithm. This is, it is not deterministic one (as algorithms are).
This (semi) algorithm makes a series of transformations on the input expression (the one to be integrated), and in a specific point, it requires to ask if the transformed expression is equals to zero, because if it were zero, it cannot continue (the input is not integrable using a finite set of terms).
The problem (and the reason of the "semi-algoritmicity") is that, the (apparently simple) equation:
E = 0
Is indecidable (it is also called the constant problem). It means that there cannot exist a formal method to solve the constant problem, for any expression E. Of course, we know to solve the constant problem for specific forms of the expression E (i.e. polynomials), but it is impossible to solve the problem for the general case.
It also means that the Risch algorithm cannot be perfect (being able to solve any integral -integrable in finite terms-). In other words, the Risch algorithm will be as powerful as our ability to solve the constant problem for as many forms of the expression E as we can, but losing any hope of solving for the general case.
Different symbolic systems have similar, but different methods to try to solve equations (and therefore the constant problem), it explains why some of them can "solve" different sets of integrals than others.
And generalizing, because no symbolic system will never be able to solve the constant problem for the general case, it will neither be able to solve any integral (integrable in finite terms).
The second parameter of int() needs to be the variable you're integrating over (which looks like t in this case):
syms x t
f = exp(1+2*x)*(1/((2*pi*t)^0.5))*exp(-(x^2)/(2*t))
int(f,'t',-inf,inf) % <- integrate over t
I am doing a comparison of some alternate linear regression techniques.
Clearly these will be bench-marked relative to OLS (Ordinary Least Squares).
But I just want a pure OLS method, no preconditioning of the data to uncover ill-conditioning in the data as you find when you use regress().
I had hoped to simply use the classic (XX)^-1XY expression? However this would necessitate using the inv() function, but in the MATLAB guide page for inv() it recommends that you use mldivide when doing least squares estimation as it is superior in terms of execution time and numerical accuracy.
However, I'm concerned as to whether it's okay to use mldivide to find the OLS estimates? As an operator it seems I can't see what the function is doing by "stepping-in" in the debugger.
Can I be assume that mldivide will produce the same answers as theoretical OLS under all conditions (including in the presence of) singular/i-ll conditioned matrices)?
If not what is the best way to compute pure OLS answers in MATLAB without any preconditioning of the data?
The short answer is:
When the system A*x = b is overdetermined, both algorithms provide the same answer. When the system is underdetermined, PINV will return the solution x, that has the minimum norm (min NORM(x)). MLDIVIDE will pick the solution with least number of non-zero elements.
As for how mldivide works, MathWorks also posted a description of how the function operates.
However, you might also want to have a look at this answer for the first part of the discussion about mldivide vs. other methods when the matrix A is square.
Depending on the shape and composition of the matrix you would use either Cholesky decomposition for symmetric positive definite, LU decomposition for other square matrix or QR otherwise. Then you can can hold onto the factorization and use linsolve to essentially just do back-substitution for you.
As to whether mldivide is preferable to pinv when A is either not square (overspecified) or is square but singular, the two options will give you two of the infinitely many solutions. According to those docs, both solutions will give you exact solutions:
Both of these are exact solutions in the sense that norm(A*x-b) and norm(A*y-b)are on the order of roundoff error.
According to the help page pinv gives a least squares solution to a system of equations, and so to solve the system Ax=b, just do x=pinv(A)*b.
This question could refer to any computer algebra system which has the ability to compute the Groebner Basis from a set of polynomials (Mathematica, Singular, GAP, Macaulay2, MatLab, etc.).
I am working with an overdetermined system of polynomials for which the full groebner basis is too difficult to compute, however it would be valuable for me to be able to print out the groebner basis elements as they are found so that I may know if a particular polynomial is in the groebner basis. Is there any way to do this?
If you implement Buchberger's algorithm on your own, then you can simply print out the elements as the are found.
If you have Mathematica, you can use this code as your starting point.
https://www.msu.edu/course/mth/496/snapshot.afs/groebner.m
See the function BuchbergerSteps.
Due to the way the Buchberger algorithm works (see, for instance, Wikipedia or IVA), the partial results that you could obtain by printing intermediate results are not guaranteed to constitute a Gröbner basis.
Depending on your ultimate goal, you may want to try instead an algorithm for triangularization of ideals, such as Ritt-Wu's algorithm (see IVA or Shang-Ching Chou's book). This is somewhat similar to reduction to row echelon form in Linear Algebra, and you may interrupt the algorithm at any point to get a partially reduced system of polynomial equations.
I read at a few places (in the doc and in this blog post : http://blogs.mathworks.com/loren/2007/05/16/purpose-of-inv/ ) that the use of inv in Matlab is not recommended because it is slow and inaccurate.
I am trying to find the reason of this inaccuracy. As of now, Google did not give m interesting result, so I thought someone here could guide me.
Thanks !
The inaccuracy I mentioned is with the method INV, not MATLAB's implementation of it. You should be using QR, LU, or other methods to solve systems of equations since these methods don't typically require squaring the condition number of the system in question. Using inv typically requires an operation that loses accuracy by squaring the condition number of the original system.
--Loren
I think the point of Loren's blog is not that MATLAB's inv function is particularly slower or more inaccurate than any other numerical implementation of computing a matrix inverse; rather, that in most cases the inverse itself is not needed, and you can proceed by other means (such as solving a linear system using \ - the backslash operator - rather than computing an inverse).
inv() is certainly slower than \ unless you have multiple right hand side vectors to solve for. However, the advice from MathWorks regarding inaccuracy is due to a overly conservative bound in a numerical linear algebra result. In other words, inv() is NOT inaccurate. The link elaborates further : http://arxiv.org/abs/1201.6035
Several widely-used textbooks lead the reader to believe that solving a linear system of equations Ax = b by multiplying the vector b by a computed inverse inv(A) is inaccurate. Virtually all other textbooks on numerical analysis and numerical linear algebra advise against using computed inverses without stating whether this is accurate or not. In fact, under reasonable assumptions on how the inverse is computed, x = inv(A)*b is as accurate as the solution computed by the best backward-stable solvers.
I have a system of (first order) ODEs with fairly expensive to compute derivatives.
However, the derivatives can be computed considerably cheaper to within given error bounds, either because the derivatives are computed from a convergent series and bounds can be placed on the maximum contribution from dropped terms, or through use of precomputed range information stored in kd-tree/octree lookup tables.
Unfortunately, I haven't been able to find any general ODE solvers which can benefit from this; they all seem to just give you coordinates and want an exact result back. (Mind you, I'm no expert on ODEs; I'm familiar with Runge-Kutta, the material in the Numerical Recipies book, LSODE and the Gnu Scientific Library's solver).
ie for all the solvers I've seen, you provide a derivs callback function accepting a t and an array of x, and returning an array of dx/dt back; but ideally I'm looking for one which gives the callback t, xs, and an array of acceptable errors, and receives dx/dt_min and dx/dt_max arrays back, with the derivative range guaranteed to be within the required precision. (There are probably numerous equally useful variations possible).
Any pointers to solvers which are designed with this sort of thing in mind, or alternative approaches to the problem (I can't believe I'm the first person wanting something like this) would be greatly appreciated.
Roughly speaking, if you know f' up to absolute error eps, and integrate from x0 to x1, the error of the integral coming from the error in the derivative is going to be <= eps*(x1 - x0). There is also discretization error, coming from your ODE solver. Consider how big eps*(x1 - x0) can be for you and feed the ODE solver with f' values computed with error <= eps.
I'm not sure this is a well-posed question.
In many algorithms, e.g, nonlinear equation solving, f(x) = 0, an estimate of a derivative f'(x) is all that's required for use in something like Newton's method since you only need to go in the "general direction" of the answer.
However, in this case, the derivative is a primary part of the (ODE) equation you're solving - get the derivative wrong, and you'll just get the wrong answer; it's like trying to solve f(x) = 0 with only an approximation for f(x).
As another answer has suggested, if you set up your ODE as applied f(x) + g(x) where g(x) is an error term, you should be able to relate errors in your derivatives to errors in your inputs.
Having thought about this some more, it occurred to me that interval arithmetic is probably key. My derivs function basically returns intervals. An integrator using interval arithmetic would maintain x's as intervals. All I'm interested in is obtaining a sufficiently small error bound on the xs at a final t. An obvious approach would be to iteratively re-integrate, improving the quality of the sample introducing the most error each iteration until we finally get a result with acceptable bounds (although that sounds like it could be a "cure worse than the disease" with regards to overall efficiency). I suspect adaptive step size control could fit in nicely in such a scheme, with step size chosen to keep the "implicit" discretization error comparable with the "explicit error" ie the interval range).
Anyway, googling "ode solver interval arithmetic" or just "interval ode" turns up a load of interesting new and relevant stuff (VNODE and its references in particular).
If you have a stiff system, you will be using some form of implicit method in which case the derivatives are only used within the Newton iteration. Using an approximate Jacobian will cost you strict quadratic convergence on the Newton iterations, but that is often acceptable. Alternatively (mostly if the system is large) you can use a Jacobian-free Newton-Krylov method to solve the stages, in which case your approximate Jacobian becomes merely a preconditioner and you retain quadratic convergence in the Newton iteration.
Have you looked into using odeset? It allows you to set options for an ODE solver, then you pass the options structure as the fourth argument to whichever solver you call. The error control properties (RelTol, AbsTol, NormControl) may be of most interest to you. Not sure if this is exactly the sort of help you need, but it's the best suggestion I could come up with, having last used the MATLAB ODE functions years ago.
In addition: For the user-defined derivative function, could you just hard-code tolerances into the computation of the derivatives, or do you really need error limits to be passed from the solver?
Not sure I'm contributing much, but in the pharma modeling world, we use LSODE, DVERK, and DGPADM. DVERK is a nice fast simple order 5/6 Runge-Kutta solver. DGPADM is a good matrix-exponent solver. If your ODEs are linear, matrix exponent is best by far. But your problem is a little different.
BTW, the T argument is only in there for generality. I've never seen an actual system that depended on T.
You may be breaking into new theoretical territory. Good luck!
Added: If you're doing orbital simulations, seems to me I heard of special methods used for that, based on conic-section curves.
Check into a finite element method with linear basis functions and midpoint quadrature. Solving the following ODE requires only one evaluation each of f(x), k(x), and b(x) per element:
-k(x)u''(x) + b(x)u'(x) = f(x)
The answer will have pointwise error proportional to the error in your evaluations.
If you need smoother results, you can use quadratic basis functions with 2 evaluation of each of the above functions per element.