I'm using MATLAB 2010b 64bit and its cplex integration to solve an engineering problem. However, because of the memory leak of cplex, memory usage exceeds acceptable limits with cplex (100+GBs including virtual memory) hence I am not able to solve my problem. You can see a similar post here.
Then I tried to use MATLAB linprog from the optimization toolbox but the result was disappointing. The running time of the algorithm for a small problem instance was increased from 80 cpu sec to 2600 cpu sec.
Now, I need an LP solver integration to MATLAB which is similar to CPLEX or linprog. By "similar" I mean the way it accepts data input in the form (F, A, B, Aeq, Beq, ...etc).
I must be able to use it in loops. Do you have any suggestions for that?
I would be very surprised if there was a memory leak in cplex. If you have a large problem then the memory will grow with any sensible solver. Is there perhaps a memory leak in the interface to cplex? How big is your problem? Are you running multi threaded as each thread will take a copy of the problem and hence will eat a lot more memory.
You should not be surprised to find that other solvers take a lot longer than cplex to solve your problem. Certainly the free solvers will be very much slower than cplex for any large problem.
After some trials to fix MATLAB/CPLex API's memory usage problem (memory leak) and after referring to some studies I decided to switch to Gurobi solver. For pure LP problems, it seems to be slightly slower compared to CPlex but this can be due to the way I use Gurobi. Someone may find Gurobi faster compared to CPlex. I suggested that on my previous posts under different questions. Here are some academic studies[Analysis of commercial and free and open source solvers for linear optimization problems][1]
[1] : http://www.statistik.tuwien.ac.at/forschung/CS/CS-2012-1complete.pdf
Related
In Dymola, I often meet a nonlinear system initialization failure or maybe a stiff system that is hard to solve in the large thermo-fluid system, but for a simple system, there wouldn't be this kind of problem. My questions are:
So I am wondering how much is the largest capability of solving a nonlinear system model? For example, how many nonlinear equations I could include in my model at most?
Is there any setting in Dymola which allows increasing the capability of solving nonlinear system?
How could I decrease the number of nonlinear equations in the model without damage to the accuracy of the model?
These are pretty difficult questions to be answered in a generally valid fashion. Still I'll try to share some of my experience with Dymola and non-linear systems.
There is no hard number which will limit the size. It depends more on how strongly non-linear the equations are than on their number. I have simulated models with non-linear systems of size 150, which are pretty stable while others of size 10 can brake...
There are multiple perspectives to this
I have worked on some models that made the C-Compiler run out of memory during compilation. If you have this sort of problem, forcing 64Bit compilation by setting Advanced.CompileWith64=2 can help. Then you shouldn't run out of memory any more. This only refers to the size only.
Performance for non-linear systems can be improved by activating DAE-mode by setting Advanced.Define.DAEsolver=true. This does not work with all solvers though.
Additionally to the above it can help to set Advanced.MoveEquationsToDynamics=true, for which the manual states: "It forces the integrator to solve the nonlinear
equations each integrator step and thereby it also updates the initial guesses more often."
As mentioned by Erik, the homotopy()-operator can be very important as it helps the solver converging in case of difficult initialization.
This is very specific to the model. Decoupling can help, e.g. by splitting the system in smaller systems by adding energy storing elements/states. This can be done based on physics of the system and is the preferable solution if possible. As an (more artificial) alternative filter/delays can be added. Usually this has a negative effect on accuracy.
I very much agree with Markus's advice but would also like to remind you about Modelica's homotopy operator. A well-chosen simplified model can greatly help Dymola to initialize a model with a large and difficult non-linear system.
In general good initial guesses are very important when solving non-linear systems. Using homotopy is simply an implicit way to provide these good guesses.
I am new to LP and have only briefly used PuLP in Python.
Why is there a speed difference between SCIP 3.2.1 - CPLEX 12.63 and CPLEX 12.6.3? Doesn't SCIP still use CPLEX for solving?
Why will someone use SCIP with CPLEX solver, instead of using CPLEX directly?
What is this difference about
This plot is not showing a LP-benchmark, but a Mixed-integer programming benchmark.
Mixed-integer programming solvers typically use a branch-and-cut-based algorithm (including heuristics and co.), where a lot of relaxations are solved (in sequence; treating binary-/integer-variables as continuous resulting in an LP-problem).
One decision then is to choose how to solve these relaxed subproblems. The most simple decision (there are many more; e.g. tuning the Simplex-algorithm's parameters; it get's even more complex when solving problems with nonlinear-conic objectives) is to choose the LP-solver.
SoPlex is a LP-solver implementation by the SCIP-team. Meaning:
SCIP - SoPlex will use SCIP's algorithm for MIP (handling branching, cut-generation and co.) using SoPlex as solver for the internal LP-subproblems
SCIP - CPLEX will use SCIP's algorithm for MIP using CPLEX as solver for the internal LP-subproblems
Why using SCIP with CPLEX (instead of using a pure CPLEX approach)
The why is not that easy to explain.
Keep in mind, that all MIP-solvers are heuristics-based and on some problems SCIP will be faster than CPLEX (despite the underlying LP-solver selected).
Keywords for some theory: NP-hardness (of MIP) and the No free lunch theorem
Faster could mean: faster due to the MIP-based strategies, not the speed of the underlying LP-solver so that you may even gain an overall speedup using CPLEX on the subproblems!
The two solvers (MIP-solvers) are probably also much different in regards to parameters & accessibility (of internal algorithmic components). It's obvious, that you can tune SCIP in a much more general way than CPLEX (because it's open source)
As mattmilten mentioned in the comments: SCIP and CPLEX are also different in regards to the support of problem-classes which can be solved. One example of this might be the possibility for some special nonlinear-constraints (resulting in a MINLP). Using SCIP for these kind of problems, can still use CPLEX' LP-solver internally (same arguments as above)
I am fairly new to Mixed Integer Linear Programming and I was hoping someone could clarify a performance question for me. Basically I am performing a calculation with about 34 decision variables and my calculation time is around 5 seconds. I would like to ideally get the calculation time down into the sub 1 second range.
Currently I am using the CBC solver & MATLAB, but as I understand it this is a single-threaded solver. Most of the MILP solvers I've seen seem to pride themselves on large project performance with 1k+ variables and knocking compute time down from days to hours but only a few of the expensive ones are even multi-threaded. Processor speed would seem to only go so far with such a problem so there has to be something that could be done on the software side.
In a situation like I have what factors play a role in the calculation time? In theory would a solution like Gurobi be able to ramp up and make a discernible difference over CBC on such a small problem?
When solving MIPs, multi-threading usually doesn't help that much, compared to other applications. One thing more sophisticated solvers do better than CBC is presolving. It may very well happen, that all decision variables can be eliminated/fixed in the root node. In general, it is not possible to predict the solving time of a MIP based on its size. The solving process is simply too complex.
To answer your question: I do suppose that you will see a significant speedup when trying another solver (Gurobi, Xpress, CPLEX, SCIP, etc.). It is not guaranteed though, and you might also witness a slowdown.
Most of the commercial MILP solvers would probably consider most problems with only 1000 variables to be tiny. It is very common to solve problems with hundreds of thousands or millions of variables. Note however, that there are also some really small problems that are considered to be very hard or still open in that nobody has solved them to a proven optimal solution. Basically, the time taken to solve these problems is enormously problem dependent.
If you want to see how fast another solver can solve your problem, try submitting it as e.g. an MPS file to the NEOS solver service at http://www.neos-server.org/neos/
In the context of a finite element problem, I have a 12800x12800 sparse matrix. I'm trying to solve the linear system just using MATLAB's \ operator to solve and I get an out of memory error using mldivide. So I'm just wondering if there's a way to speed this up.
I mean, will something like LU factorization actually help here in terms of not getting the memory error anymore? I increased the heap size to 256 GB in preferences, which is the max I can get it to, and I still get the out of memory error.
Also, just a general question. I have 8GB of RAM on my laptop right now. Will upgrading to 16GB help at all? Or maybe something I can do to allocate more memory to MATLAB? I'm pretty unfamiliar with this stuff.
According to this and this you have some options to avoid out of memory problem in matlab:
Increase operating system's virtual memory
Give Higher priority to MATLAB process in task manager
Use 64-bit version of MATLAB
Few months ago, I was working on integer programming in matlab. I faced "out of memory" problem, so I used sparse matrices and followed the mentioned tips, finally the problem is solved!
Are you locked in to using mldivide? Sounds like the perfect situation for an iterative method - bicg, gmres etc?
While backslash takes advantage of the sparsity of A, the qr method it uses produces full matrices that require (number_occupied_elements)^3 memory to be allocated. A few things you can try
If you're dividing sparse matrices with a few diagonals, you can try try to solve the system with forward/backwards substitution
Try breaking the problem into a smaller you break up the problem into a smaller
Run whos to see what elements are occupying your memory before you start the matrix division, can any of these be cleared beforehand?
Not applicable to your problem as you've stated it here, but if your system is defined (A has more rows than columns) than using the pseudo-inverse (A.'*A)\(A.'*b) produces a result using the smaller columns dimension
As for adding additional memory; Matlab32 uses 2^32 bytes of memory (4 Gb) so increasing the physical RAM on your computer won't help unless you're using the the 64 bit version.
MATLAB \ usually tries several methods to solve a problem. First, if it sees that if the structure of your matrix is symmetric it tries a Cholesky factorization. After several steps if it can not find a suitable answer current version of Matlab uses UMFPACK Suitsparse package.
UMFPack is a specific LU implemenation, and it is known for its speed and good usage of memory in practice. It also tries to reduce fill-in and keep matrix as sparse as possible. It is why MATLAB uses this code.
(I am working on UMFPACK for my PhD under supervision of Dr Tim Davis, its creator)
Therefor, using another LU factorization won't help. It is an LU factorization already.
One of the easiest way to solve your problem is testing your problem on another device with a better memory to see if it works.
I guess matlab do some garbage collection and waste some memory, so if you use the UMFPACK directly it might help you. You can either implement it in C/C++ or use MATLAB interface for it. Take a look at the SuitSparse package.
Based on the structure of your matrix I think MATLAB tries to use Cholesky; I don't know what is the strategy of MATLAB if Cholesky fails in memory management. Take into account that Cholesky is easier to manage in terms of memory.
There are other packages that might help you as well. CSparse is a lightweight package and it might help. There are other famouse packages that might be helpful; search for superLU.
Are there any faster and more efficient solvers other than fmincon? I'm using fmincon for a specific problem and I run out of memory for modest sized vector variable. I don't have any supercomputers or cloud computing options at my disposal, either. I know that any alternate solution will still run out of memory but I'm just trying to see where the problem is.
P.S. I don't want a solution that would change the way I'm approaching the actual problem. I know convex optimization is the way to go and I have already done enough work to get up until here.
P.P.S I saw the other question regarding the open source alternatives. That's not what I'm looking for. I'm looking for more efficient ones, if someone faced the same problem adn shifted to a better solver.
Hmmm...
Without further information, I'd guess that fmincon runs out of memory because it needs the Hessian (which, given that your decision variable is 10^4, will be 10^4 x numel(f(x1,x2,x3,....)) large).
It also takes a lot of time to determine the values of the Hessian, because fmincon normally uses finite differences for that if you don't specify derivatives explicitly.
There's a couple of things you can do to speed things up here.
If you know beforehand that there will be a lot of zeros in your Hessian, you can pass sparsity patterns of the Hessian matrix via HessPattern. This saves a lot of memory and computation time.
If it is fairly easy to come up with explicit formulae for the Hessian of your objective function, create a function that computes the Hessian and pass it on to fmincon via the HessFcn option in optimset.
The same holds for the gradients. The GradConstr (for your non-linear constraint functions) and/or GradObj (for your objective function) apply here.
There's probably a few options I forgot here, that could also help you. Just go through all the options in the optimization toolbox' optimset and see if they could help you.
If all this doesn't help, you'll really have to switch optimizers. Given that fmincon is the pride and joy of MATLAB's optimization toolbox, there really isn't anything much better readily available, and you'll have to search elsewhere.
TOMLAB is a very good commercial solution for MATLAB. If you don't mind going to C or C++...There's SNOPT (which is what TOMLAB/SNOPT is based on). And there's a bunch of things you could try in the GSL (although I haven't seen anything quite as advanced as SNOPT in there...).
I don't know on what version of MATLAB you have, but I know for a fact that in R2009b (and possibly also later), fmincon has a few real weaknesses for certain types of problems. I know this very well, because I once lost a very prestigious competition (the GTOC) because of it. Our approach turned out to be exactly the same as that of the winners, except that they had access to SNOPT which made their few-million variable optimization problem converge in a couple of iterations, whereas fmincon could not be brought to converge at all, whatever we tried (and trust me, WE TRIED). To this day I still don't know exactly why this happens, but I verified it myself when I had access to SNOPT. Once, when I have an infinite amount of time, I'll find this out and report this to the MathWorks. But until then...I lost a bit of trust in fmincon :)