I have quite a large program, that essentially solve an optimization problem. I'm using fminunc for such purpose. For some reason however when the max amount of iterations is reached and the function is supposed to return it literally get stuck. I've tried to follow some of the suggestions which essentially where use nested function in order to avoid dynamic allocation etc (I was loading from a file everytime the cost function was called).
But still that doesn't seem to solve the problem.
Is there anything a bit more specific I should be aware of? Like some known issue that maybe I currently don't know.
Thank you.
(Let me know what kind of detail I can post).
Some more info:
The output is supposed to be an array of 15876 double, RAM is 32 GB. The actual set up is:
option = optimoptions(#fminunc,...
'Display','iter','GradObj','on','MaxIter',10,...
'ObjectiveLimit',10e-10,'Algorithm','quasi-newton');
I set the number of iterations low just to check if maybe it was the case the number of iterations was the issue, but it doesn't seem to be so. The output I'm getting is:
Solver stopped prematurely.
fminunc stopped because it exceeded the iteration limit,
options.MaxIterations = 10 (the selected value)
But it doesn't return.
Related
I have a MIP (BP, maximization) that takes too long to compute and I'd like to have MPSolver return the first feasible solution it finds, also, I'd like to know if I use RELATIVE_MIP_GAP solver parameter correctly.
I have tried two things:
Callback
I have searched the docs and have not found a callback possibility for MPSolver's solution iteration process (only for CpSolver) with which one could implement stopping on the first feasible solution found.
Relative gap as termination criterion
I tried using RELATIVE_MIP_GAP like so (this is Kotlin language):
val mpSolverParameters = MPSolverParameters().apply {
setDoubleParam(MPSolverParameters.DoubleParam.RELATIVE_MIP_GAP, 1.0)
}
solver.solve(mpSolverParameters)
I've seen as a documentation comment somewhere that a 0.05 value for RELATIVE_MIP_GAP means a 5% gap, so 1.0 should denote a 100% gap.
But it did not work. I know it because when I have set a time limit, it returned a solution at the end of the time limit, but when I ran the same problem without a time limit, it just went on and did not return anything even after much more time than when it stopped at the time limit previously.
If I understand relative gaps correctly, if I set a value of 1.0 for this relative gap parameter, the solver should stop at any feasible solution found immediately, because the objective value of any integer solution is inside a 100% relative difference of the objective value for any continuous state of the variables. I should add that my objective function is always positive, so there's no problem of these two having different signs.
Solution remarks
Both of Laurent Perron's suggestions work for my case.
If using SCIP as the solver, we may call solver.setSolverSpecificParametersAsString("limits/solutions = 1") to get the first feasible solution, but that will be a poor quality one. We may increase this value passed as we see fit.
Check out time limit too: call setTimeLimit(timeInMs) on your solver object. It will return the best feasible solution found so far, or the unsolved state if no solution has been found at all.
Still not sure why RELATIVE_MIP_GAP didn't work, it is part of the API, not a solver specific parameter.
can you try CP-SAT ? Does it fit there ? meaning no continuous variables.
Can you remove the objective function ?
I am doing parameter estimation in matlab using lsqnonlin function.
In my work, I need to plot a graph to show the error in terms of lsqnonlin iteration. So, I need to know which iteration is running at each point of time in lsqnonlin. Could anybody help me how I can extract the iteration number while lsqnonlin is running?
Thanks,
You want to pass it an options parameter setting 'display' to either 'iter' or 'iter-detailed'
http://www.mathworks.com/help/optim/ug/lsqnonlin.html#f265106
Never used it myself, but looking at the help of lsqnonlin, it seems that there is an option to set a custom output function, which gets called during every iteration of the solver. Looking at the specification, it seems that the values optimValues.iteration and optimValues.fval get passed into the function, which is probably the things you are interested in.
You should thus define your own function with the right signature, and depending on your wishes, this function prints it on the command line, makes a plot, saves the intermediate results in a vector, etc. Finally, you need to pass this function as a function handle to the solver: lsqnonlin(..., 'OutputFcn', #your_outputfun).
The simple way to do this would be:
Start with a low number of (maximum) iterations
Get the result
Increase the number of iterations
Get the result
If the maximum iterations is used Go to step 3
This is what I would recommend in most cases when performance is not a big issue.
However, if you cannot afford to do it like this, try edit lsqnonlin and go digging untill you find the point where the number of iterations is found. Then change the function to make sure you store the results you need at that point. (don't forget to change it back afterwards).
The good news is that all relevant files seem to be editable, the bad news is that it is not so clear where you can find the current number of iterations. A quick search led me to fminbnd, but I did not manage to confirm that this is actually used by lsqnonlin.
I am working on the development of an Iterative Learning Controller for a simple transfer function.
The iterations are controlled by the external matlab loop.
But the error e(k) (k is trial number) is not updating ... as the trials increases.
Please detect the error I've commited.
Thanks and Regards.
You might have solved the problem. But as the question is still open, I would like to add something here.
First of all, you might want to check the usage of "memory" block. "The Memory block holds and delays its input by one major integration time step." The reason why the error wasn't updating is that the output of your plant produced was the same in each iteration(you defined external loop). The memory block only delayed one step of your U(K), not the whole iteration.
You might want to store the error of each iteration to workspace, and use it for the next iteration.
The memory should be a vector with a lenght of the single iteration. Not just single value. Delay block can store multiple past samples.
This guy did probably what you were looking for: https://github.com/arthurrichards77/iterative-learning-control
I have a problem with ode45. I've defined a function and trying to solve it by ode, but when i run it, it takes so long. I tried to display the "t" input in my function and it showed time step was 10^-8 ! [I do not get any error from ode45]
So i put a breakpoint at the end of my function, and after I Step once, it goes to sym.m file and calls Function delet(h)
function dxr=Dynfun(t,x)
...
dxr=[A;B]
after Step it goes to
function delete(h)
if builtin('numel',h)==1 && inmem('-isloaded','mupadmex') && builtin('numel',h.s)==1 && ~isa(h.s,'maplesym')
mupadmex(h.s,1);
end
end
and that's what makes it too long, because it goes in a loop in there.
what's the problem?! Thanks
Sounds like it's a "stiff" problem to me. I would recommend using a solver that is designed for stiff problems. I would also recommend trying a fixed step solver at a small step size ~ 0.001 and see what the output looks like. If you are breaking in sym.m, sounds like you've some some symbolic logic going on in there. Is there a way you could take your symbolic expression and convert it to a matlab script?
As indicated by macduff, your problem could be stiff. Try ode15s (which is designed for stiff problems) and see if the stepsize still decreases to unacceptably low values.
If that is indeed the case, then your problem might contain a singularity for the initial values you give it. If your problem has dimensions lower than 3, you can define a small event function to get insight into the values at each step, and plot them to see if there is indeed something problematic going on.
Then -- do you really need symbolic math? The philosophy behind that is that it's easier to read for humans, which makes it terrible to deal with for computers :) If you can transform it into something non-symbolic, please do -- this will noticeably increase performance.
Also, more a word of advice, delete is also Matlab builtin function. It is generally a bad idea to name your functions after Matlab buitins -- it's confusing, and can cause a lot of overhead while Matlab is deciding which one to use.
I used Matlab-fminsearch for a negativ max likelihood model for a binomial distributed function. I don't get any error notice, but the parameter which I want to estimate, take always the start value. Apparently, there is a mistake. I know that I ask a totally general question. But is it possible that anybody had the same mistake and know how to deal with it?
Thanks a lot,
#woodchips, thank you a lot. Step by step, I've tried to do what you advised me. First of all, I actually maximized (-log(likelihood)) and this is not the problem. I think I found out the problem but I still have some questions, if I don't bother you. I have a model(param) to maximize in paramstart=p1. This model is built for (-log(likelihood(F))) and my F is a vectorized function like F(t,Z,X,T,param,m2,m3,k,l). I have a data like (tdata,kdata,ldata),X,T are grids and Z is a function on this grid and (m1,m2,m3) are given parameters.When I want to see the value of F(tdata,Z,X,T,m1,m2,m3,kdata,ldata), I get a good output. But I think fminsearch accept that F(tdata,Z,X,T,p,m2,m3,kdata,ldata) like a constant and thatswhy I always have as estimated parameter the start value. I will be happy, if you have any advise to tweak that.
You have some options you can try to tweak. I'd start with algorithm.
When the function value practically doesn't change around your startpoint it's also problematic. Maybe switching to log-likelyhood helps.
I always use fminunc or fmincon. They allow also providing the Hessian (typically better than "estimated") or 'typical values' so the algorithm doesn't spend time in unfeasible regions.
It is virtually always true that you should NEVER maximize a likelihood function, but ALWAYS maximize the log of that function. Floating point issues will almost always corrupt the problem otherwise. That your optimization starts and stops at the same point is a good indicator this is the problem.
You may well need to dig a little deeper than the above, but even so, this next test is the test I recommend that all users of optimization tools do for every one of their problems, BEFORE they throw a function into an optimizer. Evaluate your objective for several points in the vicinity. Does it yield significantly different values? If not, then look to see why not. Are you creating a non-smooth objective to optimize, or a zero objective? I.e., zero to within the supplied tolerances?
If it does yield different values but still not converge, then make sure you know how to call the optimizer correctly. Yeah, right, like nobody has ever made this mistake before. This is actually a very common cause of failure of optimizers.
If it does yield good values that vary, and you ARE calling the optimizer correctly, then think if there are regions into which the optimizer is trying to diverge that yield garbage results. Is the objective generating complex or imaginary results?