I am implementing the (dual) simplex algorithm in Matlab/Octave.
My algorithm works just fine for a small test problem, but as soon as I try a bigger problem as afiro.mps (from http://www.netlib.org/lp/data/) I get the warning "matrix singular to machine precision, rcond=0" and octave throws an error (or does not terminate). The exact command is:
x = zeros(n,1);
x(B) = A(:,B) \ b; % Compute primal variables
y = A(:,B)' \ c(B); % Compute dual variables
The problem is in standard form
min c*x
s.t. Ax=b
x>=0
A is a m-x-n matrix and B is the index vector of the inactive constraints (base variables).
As I am doing a two phased simplex I choose 1:size(A,1) as an initial base for the dual simplex.
The problem is read from a mps file via a self coded reader. I expect the mps file is read correctly, as the glpk function solves the problem correctly, when it has A,b,c as input parameters.
Is there some way to avoid the warning or do I have an error in my coding?
The basis matrices of linear programming problems usually have very bad condition numbers. This is one the difficulties when implementing a stable simplex algorithm.
You should have a look at this paper that explains this phenomenon:
On the Factorization of Simplex Basis Matrices
Related
I'm trying to compute an inverse of a matrix P, but if I multiply inv(P)*P, the MATLAB does not return the identity matrix. It's almost the identity (non diagonal values in the order of 10^(-12)). However, in my application I need more precision.
What can I do in this situation?
Only if you explicitly need the inverse of a matrix you use inv(), otherwise you just use the backslash operator \.
The documentation on inv() explicitly states:
x = A\b is computed differently than x = inv(A)*b and is recommended for solving systems of linear equations.
This is because the backslash operator, or mldivide() uses whatever method is most suited for your specific matrix:
x = A\B solves the system of linear equations A*x = B. The matrices A and B must have the same number of rows. MATLAB® displays a warning message if A is badly scaled or nearly singular, but performs the calculation regardless.
Just so you know what algorithm MATLAB chooses depending on your input matrices, here's the full algorithm flowchart as provided in their documentation
The versatility of mldivide in solving linear systems stems from its ability to take advantage of symmetries in the problem by dispatching to an appropriate solver. This approach aims to minimize computation time. The first distinction the function makes is between full (also called "dense") and sparse input arrays.
As a side-note about error of order of magnitude 10^(-12), besides the above mentioned inaccuracy of the inv() function, there's floating point accuracy. This post on MATLAB issues on it is rather insightful, with a more general computer science post on it here. Basically, if you are computing numerics, don't worry (too much at least) about errors 12 orders of magnitude smaller.
You have what's called an ill-conditioned matrix. It's risky to try to take the inverse of such a matrix. In general, taking the inverse of anything but the smallest matrices (such as those you see in an introduction to linear algebra textbook) is risky. If you must, you could try taking the Moore-Penrose pseudoinverse (see Wikipedia), but even that is not foolproof.
For a spectrum estimation algorithm I need to find the best fitting linear combination of vectors to fit a target spectral distribution. So far, this works relatively well using the lsqlin optimizer in MATLAB.
However, for the final application I would like to approximate/solve this problem for exclusively zeros and ones, meaning Ax=b solved for Boolean x.
Is there any way to parametrize lsqlin or another optimizer function for this purpose?
If the problem is just:
Solve Ax=b for x in {0,1}
then you can use a MIP solver (e.g. Matlab intlinprog). If the problem is over-constrained and you want a least squares solution:
Min w'w
S.t. Ax - b = w
x in {0,1} (binary variable)
w free variable
then you have a MIQP (Mixed Integer Quadratic Programming) problem. There are good solvers for this such as Cplex and Gurobi (callable from Matlab). Also Matlab has a discussion about an approximation scheme using intlinprog. Another idea is to replace the quadratic objective by a sum of absolute values. This can be formulated as linear MIP model.
I am trying to solve two equations with complex coefficients using ode45.
But iam getting an error message as "Inputs must be floats, namely single or
double."
X = sym(['[',sprintf('X(%d) ',1:2),']']);
Eqns=[-(X(1)*23788605396486326904946699391889*1i)/38685626227668133590597632 + (X(2)*23788605396486326904946699391889*1i)/38685626227668133590597632; (X(2)*23788605396486326904946699391889*1i)/38685626227668133590597632 + X(1)*(- 2500000 + (5223289665997855453060886952725538686654593059791*1i)/324518553658426726783156020576256)] ;
f=#(t,X)[Eqns];
[t,Xabc]=ode45(f,[0 300*10^-6],[0 1])
How can i fix this ? Can somebody can help me ?
Per the MathWorks Support Team, the "ODE solvers in MATLAB 5 (R12) and later releases properly handle complex valued systems." So the complex numbers are the not the issue.
The error "Inputs must be floats, namely single or double." stems from your definition of f using Symbolic Variables that are, unlike complex numbers, not floats. The easiest way to get around this is to not use the Symbolic Toolbox at all; just makes Eqns an anonymous function:
Eqns= #(t,X) [-(X(1)*23788605396486326904946699391889*1i)/38685626227668133590597632 + (X(2)*23788605396486326904946699391889*1i)/38685626227668133590597632; (X(2)*23788605396486326904946699391889*1i)/38685626227668133590597632 + X(1)*(- 2500000 + (5223289665997855453060886952725538686654593059791*1i)/324518553658426726783156020576256)] ;
[t,Xabc]=ode45(Eqns,[0 300*10^-6],[0 1]);
That being said, I'd like to point out that numerically time integrating this system over 300 microseconds (I assume without units given) will take a long time since your coefficient matrix has imaginary eigenvalues on the order of 10E+10. The extremely short wavelength of those oscillations will more than likely be resolved by Matlab's adaptive methods, and that will take a while to solve for a time span just a few orders greater than the wavelength.
I'd, therefore, suggest an analytical approach to this problem; unless it is a stepping stone another problem that is non-analytically solvable.
Systems of ordinary differential equations of the form
,
which is a linear, homogenous system with a constant coefficient matrix, has the general solution
,
where the m-subscripted exponential function is the matrix exponential.
Therefore, the analytical solution to the system can be calculated exactly assuming the matrix exponential can be calculated.
In Matlab, the matrix exponential is calculate via the expm function.
The following code computes the analytical solution and compares it to the numerical one for a short time span:
% Set-up
A = [-23788605396486326904946699391889i/38685626227668133590597632,23788605396486326904946699391889i/38685626227668133590597632;...
-2500000+5223289665997855453060886952725538686654593059791i/324518553658426726783156020576256,23788605396486326904946699391889i/38685626227668133590597632];
Eqns = #(t,X) A*X;
X0 = [0;1];
% Numerical
options = odeset('RelTol',1E-8,'AbsTol',1E-8);
[t,Xabc]=ode45(Eqns,[0 1E-9],X0,options);
% Analytical
Xana = cell2mat(arrayfun(#(tk) expm(A*tk)*X0,t,'UniformOutput',false)')';
k = 1;
% Plots
figure(1);
subplot(3,1,1)
plot(t,abs(Xana(:,k)),t,abs(Xabc(:,k)),'--');
title('Magnitude');
subplot(3,1,2)
plot(t,real(Xana(:,k)),t,real(Xabc(:,k)),'--');
title('Real');
ylabel('Values');
subplot(3,1,3)
plot(t,imag(Xana(:,k)),t,imag(Xabc(:,k)),'--');
title('Imaginary');
xlabel('Time');
The comparison plot is:
The output of ode45 matches the magnitude and real parts of the solution very well, but the imaginary portion is out-of-phase by exactly π.
However, since ode45's error estimator only looks at norms, the phase difference is not noticed which may lead to problems depending on the application.
It will be noted that while the matrix exponential solution is far more costly than ode45 for the same number of time vector elements, the analytical solution will produce the exact solution for any time vector of any density given to it. So for long time solutions, the matrix exponential can be viewed as an improvement in some sense.
I wanna solve #n linear equations AX=bi(for #n b's) in Matlab which b changes in a loop and A is constant.
One way which is fast, is to compute the inverse of A before the loop and in the loop body just get X from inv(A)*b, but because the matrix A is singular, I get an awful answer!
Of course, the numerical solution A/b gives a good answer, but the point is that it takes a long time to compute #n different X's in #n loops.
What I want is a solution which can be both accurate and fast.
I actually think this is a good question, typos and issues of matrix singularity aside. There are a few good ways to handle this, and Tim Davis' factorize submission on MATLAB Central covers all the angles.
However, just for reference, let's do it on our own in native MATLAB, starting with the case where A is square. First, there are the two methods you suggested (inv and \,mldivide):
% inv, slow and inacurate
xinvsol = inv(A)*b;
norm(A*xinvsol - b ,'fro')
% mldivide, faster and accurate
xref = A\b;
norm(A*xref - b ,'fro')
But if like you said A does not change, just factorize A and solve for new b! Say A is symmetric positive definite:
L = chol(A,'lower'); % Cholesky factorization
% mldivide, much faster (not counting the chol factorization) and most accurate
xcholbs= L'\(L\b); %'
norm(A*xcholbs - b ,'fro')
% linsolve, fastest (omits checks for matrix configuration) and most accurate
sol1 = linsolve(L, b, struct('LT',true));
xcholsolv = linsolve(L, sol1, struct('LT',true,'TRANSA',true));
norm(A*xcholsolv - b ,'fro')
If A is not symmetric positive definite, then you'd use LU decomposition for a square matrix or QR otherwise. Again, you can do it all yourself, or you can just use Tim Davis' awesome factorize functions.
The following is an excerpt from a program of mine:
function [P] = abc(M,f);
if det(M) ~= 1, disp(['Matrix M should have determinant 1'])
I allow the option for the user not to enter a value for 'f'.
When I run abc([2 1; 1 1]), the program works fine and it does what it's supposed to. But when I run abc([6 13; 5 11]) I am told "Matrix M should have determinant 1".
What on Earth is going on?
EDIT: In the command window, I entered the following:
M = [6 13; 5 11];
if det(M) ~= 1, disp('Im broken');
end
Matlab then told me itself that it's broken.
Thanks
Welcome to the wonderfully wacky world of floating point arithmetic. MATLAB computes the determinant using an LU decomposition, i.e., linear algebra. It does so since determinant is wildly inefficient for arrays of even mild size unless it did.
A consequence of that LU decomposition, is the determinant is computed as a floating point number. This is not an issue, UNLESS you have entered a problem as trivially simple as you have - the determinant of a 2x2 matrix composed only of small integers. In that case, the determinant itself will also be a (reasonably) small integer. So you could resolve the issue by simply computing the determinant of the 2x2 matrix yourself, using the textbook formula.
D = A(1,1)*A(2,2) - A(1,2)*A(2,1);
This will be exactly correct for small integer matrices A, although even this may show some loss of precision for SOME matrices. For example, consider the simple, 2x2 matrix A:
>> A = [1e8 1;1 1e8];
We know that the determinant of this matrix is 1e16-1.
>> det(A)
ans =
1e+16
Of course, MATLAB displays this as 1e16. But in fact, the number generated by the det function in MATLAB is actually 9999999999999998, so 1e16-2. As bad, had I used the formula I gave above for the 2x2 determinant, it would have returned a result that is still incorrect, 10000000000000000. Both results were off by 1. You can learn more about these issues by looking at the help for eps.
My point is, there are some 2x2 matrices where computation of the determinant will simply be problematic, even though they are integer matrices.
Once your matrices become non-integer, then things really do become true floating point numbers, not integers. This means you simply MUST use comparisons with tolerances on them rather than a test for exact unity. This is a good rule anyway. Always use a tolerance when you make a test for equality, at least until you have learned enough to know when to disobey that rule!
So, you might choose a test like this:
if abs(det(A) - 1) < (10*eps(1))
warning('The sky is falling! det has failed me.')
end
Note that I've used eps(1), since we are comparing things to 1. The fact that I multiplied eps by 10 allows a wee bit of slop in the computation of the determinant.
Finally, you should know that whatever test you are using the determinant for here, it is often a BBBBBBBBBBAAAAAAAAAADDDDDDDD thing to do! Yes, maybe your teacher told you to do this, or you found something in a textbook. But the determinant is just a bad thing to use for numerical computations. There are almost always alternatives to the determinant. Again, this is called judgement, knowing when that which you are told to use is actually the wrong thing to do.
You are running into the standard problems that occur due to the limitations of floating-point numbers. The result of the det function is probably something like 1.000000001.
General rule-of-thumb: Never test floating-point values for equality.
To give you an insight: det is not computed using the old formula you studied in linear algebra, but using more efficient algorithms.
For example, using Gaussian elimination you can transform M in the equivalent upper triangular matrix and then compute the determinant as product of the main diagonal (being the lower triangle all zeros).
M = [6 13; 5 11]
G = M - [0 0; M(2,1)/M(1,1) * M(1,:)];
Theoretically det(M) is equal to det(G), which is 6 * 1/6 = 1, but being G a floating point and not a real number matrix, G(1,1)*G(2,2)~=1!
In fact G(1,1) and G(2,2) are not exactly 1 and 1/6, but they have a very small relative error (see eps, which on most machines is around 2.22e-16). Their real value will be around 6*(1+eps) and 1/6*(1+eps), thus their product will have a small error too.
I'm not sure if Matlab uses the Gaussian elimination or the similar LU decomposition.