I'm using the SCIP solver in the OPTI toolbox in matlab to solve a quadratic optimization problem with integer constraints. I ran it with the following specs and it's been running for a day and has already taken up 55GB of ram in my system and still counting. I'm new to optimization in matlab, am I doing something wrong or is this usual? I tried with less maxnodes and maxtime, but the program stops with the 'Node limit reached' error in those cases. Here's the code (H, Aeq etc. have been defined earlier in the code) -
X = sym('X%d%d', [104 1]);
fun = #(X) 1/2*X'*H*X;
options = optiset('solver', 'SCIP', 'maxnodes', 20000000, 'maxtime', 100000);
Opt = opti('fun', fun, 'eq', Aeq, Beq, 'xtype', xtype, 'options', options);
[xval,fval,exitflag,info] = solve(Opt)
This is not unusual if the quadratic function(s) are nonconvex. This easily leads to hard problems that cannot be solved to proven optimality with today's algorithms in any reasonably finite amount of time. Note that this does not only depend on the size of the problem, but in general smaller problems (of a similar type) will be easier.
This being said, SCIP might already have found a near-optimal solution that is accessible even when the time or node limit is exceeded.
Related
I have an integer programming optimization problem, that I solve in matlab using yalmip and xpress as the solver. For the solver I want to set two stopping criteria - a time limit and an optimal gap limit.
I have tried to use the xpress functions MAXTIME and MIPRELSTOP, the matlab code compiles and runs the optimization but the stopping criteria are not transferred to the solver.
The relevant code part looks as follows:
Cons = [sum(sum((dVar_mat.*(x_mat.*y_vec))')) >= a]; %constraint
obj = sum(sum(dVar_mat.*z_mat)); %objective
ops = sdpsettings('solver', 'xpress', 'verbose', 2); %solver options
ops.xpress.MAXTIME = 10000; %set timelimit
ops.xpress.MIPRELSTOP = 0.05; %set relative gap as stop limit
solIP = optimize(Cons, obj, ops); % Solve
When I run the optimzation, a solution is found but significantly later than I would like it to stop. The report says:
STOPPING - MIPRELSTOP target reached (MIPRELSTOP=0.0001)
meaning the MIPRELSTOP target is still set at the default, which is 0.0001. Similarily, the optimization runs over the time limit, disregarding that stopping criterion as well.
How can I correctly set stopping criteria in matlab/yalmip/xpress?
Are you sure you are using the correct name and that it is exposed in the MATLAB interface, i.e. is that options visible in ops.xpress. I don't have xpress installed so I cannot test it.
(btw, YALMIP question are much better asked on the YALMIP Google groups forum)
The 'MAXTIME' control of Xpress Optimizer can be used with positive and negative values: with positive values for 'MAXTIME' when solving MIP problems the limit is only applied once a solution has been found, otherwise the solving continues until the first solution is found; a negative value means a hard stop, so for your case I would recommend that you try a value like -10000 as the time limit.
(See the documentation in the Xpress Optimizer Reference Manual, eg: https://www.fico.com/fico-xpress-optimization/docs/latest/solver/optimizer/HTML/MAXTIME.html)
Actually I have to calculate values of 3 variables from probably 8 or 9 non-linear equations(may be more for accuracy).
I was using lsqnonlin and fsolve.
Using lsqnonlin, it says solver stopped prematurely (mainly due to value of iteration, funEvals and tolerance) and the output is far away from exact solution. I tried but I don't know on what basis I should set those parameters.
Using fsolve, it says no solution found.
I also used LMFnlsq and LMFsolve but it gives the output nowhere near the exact solution? I tried to change other parameters too but I could not bring those solutions to my desired values.
Is there any other way to solve these overdetermined non-linear equations?
My code till now:
x0 = [20 40 275];
eqn = #(x)[((((x(1)-Sat(1,1))^2+(x(2)-Sat(1,2))^2+(x(3)-Sat(1,3))^2))-dis(1)^2);
((((x(1)-Sat(2,1))^2+(x(2)-Sat(2,2))^2+(x(3)-Sat(2,3))^2))-dis(2)^2);
((((x(1)-Sat(3,1))^2+(x(2)-Sat(3,2))^2+(x(3)-Sat(3,3))^2))- dis(3)^2);
((((x(1)-Sat(4,1))^2+(x(2)-Sat(4,2))^2+(x(3)-Sat(4,3))^2))- dis(4))^2;
((((x(1)-Sat(5,1))^2+(x(2)-Sat(5,2))^2+(x(3)-Sat(5,3))^2))- dis(5))^2;
((((x(1)-Sat(6,1))^2+(x(2)-Sat(6,2))^2+(x(3)-Sat(6,3))^2))- dis(6))^2;
((((x(1)-Sat(7,1))^2+(x(2)-Sat(7,2))^2+(x(3)-Sat(7,3))^2))- dis(7))^2;
((((x(1)-Sat(8,1))^2+(x(2)-Sat(8,2))^2+(x(3)-Sat(8,3))^2))- dis(8))^2;
((((x(1)-Sat(9,1))^2+(x(2)-Sat(9,2))^2+(x(3)-Sat(9,3))^2))- dis(9))^2;
((((x(1)-Sat(10,1))^2+(x(2)-Sat(10,2))^2+(x(3)-Sat(10,3))^2))- dis(10))^2];
lb = [0 0 0];
ub = [100 100 10000];
options = optimoptions('lsqnonlin','MaxFunEvals',3000,'MaxIter',700,'TolFun',1e-18);%,'TolX',1);
x= lsqnonlin(eqn,x0,lb,ub,options)
**Error:**
**Solver stopped prematurely.**
lsqnonlin stopped because it exceeded the iteration limit,
options.MaxIter = 700 (the selected value).
x = 20.349 46.633 9561.5
Hoping for some suggestions!
Thanks in advance!
I usually model this explicitly:
min w'w
f_i(x) = w_i
w is a free variable
L<=x<=U
It should be easy to calculate a feasible (but non-optimal) solution in advance. If you can find a "good" initial solution that would be even better. Then use a general purpose NLP solver (e.g. fmincon) and pass on your initial feasible solution (both x and w). The best thing is to use a modeling system that allows automatic differentiation. Otherwise you should provide correct and precise gradients (and if needed second derivatives). See also the advice here.
I am having difficulty achieving sufficient accuracy in a root-finding problem on Matlab. I have a function, Lik(k), and want to find the value of k where Lik(k)=L0. Basically, the problem is that various built-in Matlab solvers (fzero, fminbnd, fmincon) are not getting as close to the solution as I would like or expect.
Lik() is a user-defined function which involves extensive coding to compute a numerical inverse Laplace transform, etc., and I therefore do not include the full code. However, I have used this function extensively and it appears to work properly. Lik() actually takes several input parameters, but for the current step, all of these are fixed except k. So it is really a one-dimensional root-finding problem.
I want to find the value of k >= 165.95 for which Lik(k)-L0 = 0. Lik(165.95) is less than L0 and I expect Lik(k) to increase monotonically from here. In fact, I can evaluate Lik(k)-L0 in the range of interest and it appears to smoothly cross zero: e.g. Lik(165.95)-L0 = -0.7465, ..., Lik(170.5)-L0 = -0.1594, Lik(171)-L0 = -0.0344, Lik(171.5)-L0 = 0.1015, ... Lik(173)-L0 = 0.5730, ..., Lik(200)-L0 = 19.80. So it appears that the function is behaving nicely.
However, I have tried to "automatically" find the root with several different methods and the accuracy is not as good as I would expect...
Using fzero(#(k) Lik(k)-L0): If constrained to the interval (165.95,173), fzero returns k=170.96 with Lik(k)-L0=-0.045. Okay, although not great. And for practical purposes, I would not know such a precise upper bound without a lot of manual trial and error. If I use the interval (165.95,200), fzero returns k=167.19 where Lik(k)-L0 = -0.65, which is rather poor. I have been running these tests with Display set to iter so I can see what's going on, and it appears that fzero hits 167.19 on the 4th iteration and then stays there on the 5th iteration, meaning that the change in k from one iteration to the next is less than TolX (set to 0.001) and thus the procedure ends. The exit flag indicates that it successfully converged to a solution.
I also tried minimizing abs(Lik(k)-L0) using fminbnd (giving upper and lower bounds on k) and fmincon (giving a starting point for k) and ran into similar accuracy issues. In particular, with fmincon one can set both TolX and TolFun, but playing around with these (down to 10^-6, much higher precision than I need) did not make any difference. Confusingly, sometimes the optimizer even finds a k-value on an earlier iteration that is closer to making the objective function zero than the final k-value it returns.
So, it appears that the algorithm is iterating to a certain point, then failing to take any further step of sufficient size to find a better solution. Does anyone know why the algorithm does not take another, larger step? Is there anything I can adjust to change this? (I have looked at the list under optimset but did not come up with anything useful.)
Thanks a lot!
As you seem to have a 'wild' function that does appear to be monotone in the region, a fairly small range of interest, and not a very high requirement in precision I think all criteria are met for recommending the brute force approach.
Assuming it does not take too much time to evaluate the function in a point, please try this:
Find an upperbound xmax and a lower bound xmin, choose a preferred stepsize and evaluate your function at
xmin:stepsize:xmax
If required (and monotonicity really applies) you can get another upper and lower bound by doing this and repeat the process for better accuracy.
I also encountered this problem while using fmincon. Here is how I fixed it.
I needed to find the solution of a function (single variable) within an optimization loop (multiple variables). Because of this, I needed to provide a large interval for the solution of the single variable function. The problem is that fmincon (or fzero) does not converge to a solution if the search interval is too large. To get past this, I solve the problem inside a while loop, with a huge starting upperbound (1e200) with the constraint made on the fval value resulting from the solver. If the resulting fval is not small enough, I decrease the upperbound by a factor. The code looks something like this:
fval = 1;
factor = 1;
while fval>1e-7
UB = factor*1e200;
[x,fval,exitflag] = fminbnd(#(x)function(x,...),LB,UB,options);
factor = factor * 0.001;
end
The solver exits the while when a good solution is found. You can of course play also with the LB by introducing another factor and/or increase the factor step.
My 1st language isn't English so I apologize for any mistakes made.
Cheers,
Cristian
Why not use a simple bisection method? You always evaluate the middle of a certain interval and then reduce this to the right or left part so that you always have one bound giving a negative and the other bound giving a positive value. You can reduce to arbitrary precision very quickly. Since you reduce the interval in half each time it should converge very quickly.
I would suspect however there is some other problem with that function in that it has discontinuities. It seems strange that fzero would work so badly. It's a deterministic function right?
I have access to a 12 core machine and some matlab code that relies heavily on fftn. I would like to speed up my code.
Since the fft can be parallelized I would think that more cores would help but I'm seeing the opposite.
Here's an example:
X = peaks(1028);
ncores = feature('numcores');
ntrials = 20;
mtx_power_times = zeros(ncores,ntrials);
fft_times = zeros(ncores, ntrials);
for i=1:ncores
for j=1:ntrials
maxNumCompThreads(i);
tic;
X^2;
mtx_power_times(i,j) = toc;
tic
fftn(X);
fft_times(i,j) = toc;
end
end
subplot(1,2,1);
plot(mtx_power_times,'x-')
title('mtx power time vs number of cores');
subplot(1,2,2);
plot(fft_times,'x-');
title('fftn time vs num of cores');
Which gives me this:
The speedup for matrix multiplication is great but it looks like my ffts go almost 3x slower when I use all my cores. What's going on?
For reference my version is 7.12.0.635 (R2011a)
Edit: On large 2D arrays taking 1D transforms I get the same problem:
Edit: The problem appears to be that fftw is not seeing the thread limiting that maxNumCompThreads enforces. I'm getting all the cpus going full speed no matter what I set maxNumCompThreads at.
So... is there a way I can specify how many processors I want to use for an fft in Matlab?
Edit: Looks like I can't do this without some careful work in .mex files. http://www.mathworks.com/matlabcentral/answers/35088-how-to-control-number-of-threads-in-fft has an answer. It would be nice if someone has an easy fix...
Looks like I can't do this without some careful work in .mex files. http://www.mathworks.com/matlabcentral/answers/35088-how-to-control-number-of-threads-in-fft has an answer. It would be nice if someone has an easy fix...
To use different cores, you should use the Parallel Computing Toolbox. For instance, you could use a parfor loop, and you have to pass the functions as a list of handles:
function x = f(n, i)
...
end
m = ones(8);
parfor i=1:8
m(i,:) = f(m(i,:), i);
end
More info is available at:
High performance computing
Multithreaded computation
Multithreading
I have just profiled my MATLAB code and there is a bottle-neck in this for loop:
for vert=-down:up
for horz=-lhs:rhs
y = y + x(k+vert.*length+horz).*DM(abs(vert).*nu+abs(horz)+1);
end
end
where y, x and DM are vectors I have already defined. I vectorised the loop by writing,
B=(-down:up)'*ones(1,lhs+rhs+1);
C=ones(up+down+1,1)*(-lhs:rhs);
y = sum(sum(x(k+length.*B+C).*DM(abs(B).*nu+abs(C)+1)));
But this ended up being sufficiently slower.
Are there any suggestions on how I can speed up this for loop?
Thanks in advance.
What you've done is not really vectorization. It's very difficult, if not impossible, to write proper vectorization procedures for image processing (I assume that's what you're doing) in Matlab. When we use the term vectorized, we really mean "vectorized with no additional computation". For example, this code
a = 1:1000000;
for i = a
n = n+i;
end
would run much slower then this code
a = 1:1000000;
sum(a)
Update: code above has been modified, thanks to #Rasman's keen suggestion. The reason is that Matlab does not compile your code into machine language before running it, and that's what causes it to be slower. Built-in functions like sum, mean and the .* operator run pre-compiled C code behind the scenes. For loops are a great example of code that runs slowly when not optimized for you CPU's registers.
What you have done, and please ignore my first comment, is rewriting your procedure with a vector operation and some additional operations. Those are the operations that take extra CPU simply because you're telling your computer to do more computations, even though each computation separately may (or may not) take less time.
If you are really after speeding up you code, take a look at MEX files. They allow you to write and compile C and C++ code, compile it and run as Matlab functions, just like those fast built-in ones. In any case, Matlab is not meant to be a fast general-purpose programming platform, but rather a computer simulation environment, though this approach has been changing in the recent years. My advise (from experience) is that if you do image processing, you will write for loops, and there's rarely a way around it. Vector operations were written for a more intuitive approach to linear algebra problems, and we rarely treat digital images as regular rectangular matrices in terms of what we do with them.
I hope this helps.
I would use matrices when handling images... you could then try to extract submatrices like so:
X = reshape(x,height,length);
kx = mod(k,length);
ky = floor(k/length);
xstamp = X( [kx-down:kx+up], [ky-lhs:ky+rhs]);
xstamp = xstamp.*getDMMMask(width, height);
y = sum(xstamp);
...
function mask = getDMMask(width, height, nu)
% I don't get what you're doing there .. return an appropriate sized mask here.
return mask;
end