I'm solving an optimization problem in python with OR-Tools / CP-sat solver. I'm using a file that takes some hours to reach optimal solution. Is there any way of seeing in the terminal how the process is going, like the best solution found so far, the elapsed time, etc...? I know that with cplex solver we can see this.
Thank you
First, you need to add log_search_progress:true to the parameters.
Second, a good way to speed solving it to use multiple workers. This is done by using the num_search_workers:XXX parameter. If you have a decent machine, XXX=8 is good. If you have a beefier machine, you can try XXX=12 or 16 (or more).
Related
I am using the CP-SAT solver on a JSP.
I am iterating so the solver runs many times (basically simulating each day for a year), I do not need to find the optimal solution, just a reasonably good one, so I would like to be a bit smarter on ending the solver than simply allowing it to run for X seconds each time. For example, i would like to take the 5th solution each time, or even to stop once the current solution makespan is only 5% (for example) shorter than the previous solution.
Is this possible? I am only aware of solver.parameters.max_time_in_seconds as a way of limiting the calculation time. Intermediate solutions are printed by SolutionPrinter but i think this is output only and there is no way to break the solver during a run?
wrong, you can stop the search in a callback, see this recipe:
https://github.com/google/or-tools/blob/stable/ortools/sat/docs/solver.md#stopping-search-early
I'm trying to solve a complex variant of a min-SAT problem. So far in the process I have two subproblems, both giving solution values that need to be considered in the objective function. However, only one of the two problems do I solve with the OR-tools cp_model module. The other is solved by an external algorithm. Now, ideally I would do the following:
cp-solver findes a solution to the first subproblem,
pause the solver,
solve the second subproblem with an external algorithm, taking as argument the solution found by the cp-solver,
feed the result of the external algorithm back to the cp-solver,
cp-solver now considers as the objective value the sum of the solution it itself found to first subproblem and the solution that was found by the external algorithm,
cp-solver goes to the next iteration and repeats steps 1-6 for a new assignment
So my question is: is there a functionality for Google OR-tools that lets me do something like steps 1-6 where the solver runs in cooperation with external algorithms and is fed values accordingly? I'm new to using this module so I'm unaware of what terms I could search for on Google to find what I need. Thanks a lot my friends. Best regards, 30centimeter.
In the cp-sat solver, solve() is stateless and a black box.
The only thing you can do is modify the model and resolve.
I am using Gurobi to run a MIQP (Mixed Integer Quadratic Programming) with linear constraints in Matlab. The solver is very slow and I would like your help to understand whether I can do something about it.
These are the lines which I use to launch the problem
clear model;
clear params;
model.A=[Aineq; Aeq];
model.rhs=[bineq; beq];
model.sense=[repmat('<', size(Aineq,1),1); repmat('=', size(Aeq,1),1)];
model.Q=Q;
model.obj=c;
model.vtype=type;
model.lb=total_lb;
model.ub=total_ub;
params.MIPGap=10^(-1);
result=gurobi(model,params);
This is a screenshot of the output in the Matlab window.
Question 1: It is the first time I am trying to run a MIQP and I would like to have your advice to understand what I can do to improve performance. Let me tell what I have tried so far:
I cheated by imposing params.MIPGap=10^(-1). In this way the phase of node exploration is made shorter. What are the cons of doing this?
I have big-M coefficients and I have tied them to the smallest possible values.
I have tried setting params.ScaleFlag=2; params.ObjScale=2 but it makes things slower
I have changed params.method but it does not seem to help (unless you have some specific recommendation)
I have increase params.Threads but it does not seem to help
Question 2 (minor): Why do I get a negative objective in the root simplex log? How can the objective function be negative?
Without having the full model here, there is not much on advise to give. Tight Big-M formulations are important, but you said, you checked them already. Sometimes splitting them up might help, but this is a complex field.
What might give great benefits for some problems is using the Gurobi parameter tuning tool. So try to export your model and feed the tuning tool with it. It automatically tries different of the hundreds of tuning parameters and might give some nice results.
Regarding the question about negative objectives in the simplex logs, I can think of a couple of possible explanations. First, note that the negative objective values occur in the presence of dual infeasibilities in the dual simplex run. In such a case, I'm not sure exactly what the primal objective values correspond to. Second, if you have a MIQP with products of binaries in the objective, Gurobi may convexify the objective in a way that makes it possible for a negative objective to appear in the reformulated model even when the original model must have a nonnegative objective in any feasible solution.
I'm working on an adaptive and Fully automatic segmentation algorithm under varying light condition , the core of this algorithm uses Particle swarm Optimization(PSO) to tune the fuzzy system and believe me it's very time consuming :| for only 5 particles and 100 iterations I have to wait 2 to 3 hours ! and it's just processing one image from my data set containing over 100 photos !
I'm using matlab R2013 ,with a intel coer i7-2670Qm # 2.2GHz //8.00GB RAM//64-bit operating system
the problem is : when starting the program it uses only 12%-16% of my CPU and only one core is working !!
I've searched a lot and came into matlabpool so I added this line to my code :
matlabpool open 8
now when I start the program the task manger shows 98% CPU usage, but it's just for a few seconds ! after that it came back to 12-13% CPU usage :|
Do you have any idea how can I get this code run faster ?!
12 Percent sounds like Matlab is using only one Thread/Core and this one with with full load, which is normal.
matlabpool open 8 is not enough, this simply opens workers. You have to use commands like parfor, to assign work to them.
Further to Daniel's suggestion, ideally to apply PARFOR you'd find a time-consuming FOR loop in you algorithm where the iterations are independent and convert that to PARFOR. Generally, PARFOR works best when applied at the outermost level possible. It's also definitely worth using the MATLAB profiler to help you optimise your serial code before you start adding parallelism.
With my own simulations I find that I cannot recode them using Parfor, the for loops I have are too intertwined to take advantage of multiple cores.
HOWEVER:
You can open a second (and third, and fourth etc) instance of Matlab and tell this additional instance to run another job. Each instance of matlab open will use a different core. So if you have a quadcore, you can have 4 instances open and get 100% efficiency by running code in all 4.
So, I gained efficiency by having multiple instances of matlab open at the same time and running a job. My jobs took 8 to 27 hours at a time, and as one might imagine without liquid cooling I burnt out my cpu fan and had to replace it.
Also do look into optimizing your matlab code, I recently optimized my code and now it runs 40% faster.
I'm playing around with the travelling salesman example provided with GLPK, and trying to get a feel for what problem size I can reasonably expect to solve. I've managed to solve a 50 node graph, but 100 nodes doesn't seem to be converging in a reasonable timescale (30 minutes or so on modern hardware).
GLPK has lots of options for the MIP solver. I've tried various combinations but I'm not at all clear what options might help. This page has some discussion but is somewhat out of date and the advice is rather general.
Is it reasonable to expect GLPK to solve a 100 node tour or greater in a practical time frame (say, less than 4 hours)? When does the problem size become intractable? Are any of the many command-line options likely to help?
Are any of the many command-line options likely to help?
The tspsol command line applicaton (based on the GLPK C library) or the C API in glptsp.h are tailor-made for solving TSP.
Is it reasonable to expect GLPK to solve a 100 node tour or greater in a practical time frame (say, less than 4 hours)? When does the problem size become intractable?
My guess is that it also greatly depends on the problem instance as well. If you look into the C code, you will see that heuristics are used to generate an initial tour. How well a heuristic works, well...
I assume you know that the TSP is famous for being difficult to solve, see computational complexity.
I was able to solve eil51 in 2 seconds and rat99 in about 50 seconds by starting with a relaxation of the TSP and then eliminating sub-tours until a feasible solution is found. Doing a better modelling is going to achieve more than fiddling with the solver options. GLPK should be able to deal with bigger instances if the modelling is good enough. eil51 and rat99 are instances documented on TSPLIB.
I recommend reading some material on tsp modelling, there are a lot of articles and books out there. A nice starting point maybe 'Applied Integer Programming' by Der-San Chen et al.