Could you help me on how to set options for the fminunc or lsqnonlin optimizers, in such a way as to force them to do more iterations regardless of what internal tolerances they have ?
It seem that my loss is diminishing but the functions stop prematurely.
I have a code like:
options = optimset('Display','iter','PlotFcns',#optimplotfval)
options.MaxIterations=1e6;
options.CheckGradients=true;
options.FunctionTolerance= 1e-100;
options.OptimalityTolerance=1e-100;
options.StepTolerance =1e-100;
[xsol,fval]=fminunc(#myFun,x0,options);
I tried to put extremely low tol values... but this is strange... ideallly is there a way to say: "do your 1 million iterations regardless of anything else!"
The optimization will terminate if any of the stopping criteria are sasified. Thus, you will have to increase the tolerances, e.g. to Inf to make them obsolete and leave the MaxIterations to be the single remaining criterion.
Related
I have an integer programming optimization problem, that I solve in matlab using yalmip and xpress as the solver. For the solver I want to set two stopping criteria - a time limit and an optimal gap limit.
I have tried to use the xpress functions MAXTIME and MIPRELSTOP, the matlab code compiles and runs the optimization but the stopping criteria are not transferred to the solver.
The relevant code part looks as follows:
Cons = [sum(sum((dVar_mat.*(x_mat.*y_vec))')) >= a]; %constraint
obj = sum(sum(dVar_mat.*z_mat)); %objective
ops = sdpsettings('solver', 'xpress', 'verbose', 2); %solver options
ops.xpress.MAXTIME = 10000; %set timelimit
ops.xpress.MIPRELSTOP = 0.05; %set relative gap as stop limit
solIP = optimize(Cons, obj, ops); % Solve
When I run the optimzation, a solution is found but significantly later than I would like it to stop. The report says:
STOPPING - MIPRELSTOP target reached (MIPRELSTOP=0.0001)
meaning the MIPRELSTOP target is still set at the default, which is 0.0001. Similarily, the optimization runs over the time limit, disregarding that stopping criterion as well.
How can I correctly set stopping criteria in matlab/yalmip/xpress?
Are you sure you are using the correct name and that it is exposed in the MATLAB interface, i.e. is that options visible in ops.xpress. I don't have xpress installed so I cannot test it.
(btw, YALMIP question are much better asked on the YALMIP Google groups forum)
The 'MAXTIME' control of Xpress Optimizer can be used with positive and negative values: with positive values for 'MAXTIME' when solving MIP problems the limit is only applied once a solution has been found, otherwise the solving continues until the first solution is found; a negative value means a hard stop, so for your case I would recommend that you try a value like -10000 as the time limit.
(See the documentation in the Xpress Optimizer Reference Manual, eg: https://www.fico.com/fico-xpress-optimization/docs/latest/solver/optimizer/HTML/MAXTIME.html)
I'm new to Matlab, and I'm trying to solve the differential equation y'=-y/n for a constant n. I define the function in a script like this:
function dv = lc(v1)
dv = -v1/(0.0000047*0.000001);
And then try and solve it like this:
[t,v] = ode23('lc',[0 5],1)
But the operation never finishes executing. It just eats up my RAM and says "Busy" in the corner, until I press ctrl+c to terminate it. What am I doing wrong here?
You're a victim of underflow. What happens is that there is not an infinite number of floating point numbers (see Is floating point math broken?), so the results are not fully precise. When the numbers are small enough there is the possibility of the number being treated as 0 by the computer. The process that matlab uses for ode23 is based on finite differencing, which involves division. Considering the underflow error, there either will be a division by 0 issue, or possible overflow with an incredibly small divisor trending the result toward infinity and not fulfilling the conditions for solution.
How can the output.stepsize be used as a stopping criterion for fmincon when using the 'active-set' algorithm?
When I tried to solve a nonlinear constrained non-convex optimization problem, I observed that the output.stepsize option within 'PlotFcns' would be a better stopping criterion than the tolerances or maximum evaluations. But in the options argument structure, there is no such option.
I also noticed that fmincon uses nonlcon to solve the problem where output.stepsize is calcualted.
If I don't want to change the original code of either fmincon or nonlcon, how can I set an upper limit for output.stepsize to use as a stopping criterion for my optimization run?
Here's one way that you might try to accomplish this. I'm using one of the examples from the doc (there's also discussion of how to do exactly this there as well). I've simply added an output function that returns stop = true when the optimValues.stepsize field reaches a threshold. I also had to make sure that this field wasn't empty, as it appears to be at initialization.
function fmincontest
A = [-1 -2 -2;
1 2 2];
b = [0;72];
x0 = [10;10;10];
options = optimoptions('fmincon','Algorithm','active-set','OutputFcn',#outfun);
[x,fval,exitflag] = fmincon(#myfun,x0,A,b,[],[],[],[],[],options)
function f = myfun(x)
f = -x(1)*x(2)*x(3);
function stop = outfun(x, optimValues, state)
stop = ~isempty(optimValues.stepsize) && optimValues.stepsize < 0.05;
If you check the exit_flag output from fmincon, you'll see that it returns -1 now because the output function is stopping the optimization, again as indicated in the documentation. If you're already using a plot function, you can use that instead, as they have the same format.
You'll need to tailor this to your problem, adjust the threshold, and of course confirm that it works. I can't comment on how good of an idea this is. You should confirm that your error tolerances are still met satisfactorily for all cases. And I'd still want to ask why you can't specify appropriate 'TolCon' and 'TolX' tolerances to achieve what you need.
I am having difficulty achieving sufficient accuracy in a root-finding problem on Matlab. I have a function, Lik(k), and want to find the value of k where Lik(k)=L0. Basically, the problem is that various built-in Matlab solvers (fzero, fminbnd, fmincon) are not getting as close to the solution as I would like or expect.
Lik() is a user-defined function which involves extensive coding to compute a numerical inverse Laplace transform, etc., and I therefore do not include the full code. However, I have used this function extensively and it appears to work properly. Lik() actually takes several input parameters, but for the current step, all of these are fixed except k. So it is really a one-dimensional root-finding problem.
I want to find the value of k >= 165.95 for which Lik(k)-L0 = 0. Lik(165.95) is less than L0 and I expect Lik(k) to increase monotonically from here. In fact, I can evaluate Lik(k)-L0 in the range of interest and it appears to smoothly cross zero: e.g. Lik(165.95)-L0 = -0.7465, ..., Lik(170.5)-L0 = -0.1594, Lik(171)-L0 = -0.0344, Lik(171.5)-L0 = 0.1015, ... Lik(173)-L0 = 0.5730, ..., Lik(200)-L0 = 19.80. So it appears that the function is behaving nicely.
However, I have tried to "automatically" find the root with several different methods and the accuracy is not as good as I would expect...
Using fzero(#(k) Lik(k)-L0): If constrained to the interval (165.95,173), fzero returns k=170.96 with Lik(k)-L0=-0.045. Okay, although not great. And for practical purposes, I would not know such a precise upper bound without a lot of manual trial and error. If I use the interval (165.95,200), fzero returns k=167.19 where Lik(k)-L0 = -0.65, which is rather poor. I have been running these tests with Display set to iter so I can see what's going on, and it appears that fzero hits 167.19 on the 4th iteration and then stays there on the 5th iteration, meaning that the change in k from one iteration to the next is less than TolX (set to 0.001) and thus the procedure ends. The exit flag indicates that it successfully converged to a solution.
I also tried minimizing abs(Lik(k)-L0) using fminbnd (giving upper and lower bounds on k) and fmincon (giving a starting point for k) and ran into similar accuracy issues. In particular, with fmincon one can set both TolX and TolFun, but playing around with these (down to 10^-6, much higher precision than I need) did not make any difference. Confusingly, sometimes the optimizer even finds a k-value on an earlier iteration that is closer to making the objective function zero than the final k-value it returns.
So, it appears that the algorithm is iterating to a certain point, then failing to take any further step of sufficient size to find a better solution. Does anyone know why the algorithm does not take another, larger step? Is there anything I can adjust to change this? (I have looked at the list under optimset but did not come up with anything useful.)
Thanks a lot!
As you seem to have a 'wild' function that does appear to be monotone in the region, a fairly small range of interest, and not a very high requirement in precision I think all criteria are met for recommending the brute force approach.
Assuming it does not take too much time to evaluate the function in a point, please try this:
Find an upperbound xmax and a lower bound xmin, choose a preferred stepsize and evaluate your function at
xmin:stepsize:xmax
If required (and monotonicity really applies) you can get another upper and lower bound by doing this and repeat the process for better accuracy.
I also encountered this problem while using fmincon. Here is how I fixed it.
I needed to find the solution of a function (single variable) within an optimization loop (multiple variables). Because of this, I needed to provide a large interval for the solution of the single variable function. The problem is that fmincon (or fzero) does not converge to a solution if the search interval is too large. To get past this, I solve the problem inside a while loop, with a huge starting upperbound (1e200) with the constraint made on the fval value resulting from the solver. If the resulting fval is not small enough, I decrease the upperbound by a factor. The code looks something like this:
fval = 1;
factor = 1;
while fval>1e-7
UB = factor*1e200;
[x,fval,exitflag] = fminbnd(#(x)function(x,...),LB,UB,options);
factor = factor * 0.001;
end
The solver exits the while when a good solution is found. You can of course play also with the LB by introducing another factor and/or increase the factor step.
My 1st language isn't English so I apologize for any mistakes made.
Cheers,
Cristian
Why not use a simple bisection method? You always evaluate the middle of a certain interval and then reduce this to the right or left part so that you always have one bound giving a negative and the other bound giving a positive value. You can reduce to arbitrary precision very quickly. Since you reduce the interval in half each time it should converge very quickly.
I would suspect however there is some other problem with that function in that it has discontinuities. It seems strange that fzero would work so badly. It's a deterministic function right?
I am having some issues with fminsearch of matlab. I have defined the TolX and TolFun as following
options = optimset('TolFun',1e-8, 'TolX', 1e-8)
Then I tried to estimate the parameters of my functions using
[estimates val] = fminsearch(model, start_point,options)
However, the val is around 3.3032e-04. Even though I specified the TolFun to be 1e-8, it still terminates before that with value around 3.3032e-04. Actually, the desired value of the parameter is obtained at something around 1.268e-04. So I tried to set the TolFun. Why is it not working, it should have converged to the least value of the function isn't it?
There are other reasons for termination of the search, for example, max number of function evaluations, max number of iterations, etc. fminsearch provides additional output arguments that give you information about the reason for termination. You especially want the full OUTPUT argument, which provides number of iterations, termination message, etc.
[X,FVAL,EXITFLAG,OUTPUT] = fminsearch(...) returns a structure
OUTPUT with the number of iterations taken in OUTPUT.iterations, the
number of function evaluations in OUTPUT.funcCount, the algorithm name
in OUTPUT.algorithm, and the exit message in OUTPUT.message.
Another possibility is that you've gotten stuck in local minimum. There's not much to be done for that problem, except to choose a different start point, or a different optimizer.