Im using fminunc to minimize a function with 450 variables, here are the lines
opts = optimset('Display','Iter','MaxIter', 10000000,'MaxFunEvals', 100000000,'TolX', 1e-12,'TolFun', 1e-12);
[x,fval,exitflag] = fminunc(#function,x0,opts);
Note that I dont provide the gradient, so fminunc uses quasi-newton algorithm.
After some iterations it endes with this final message
fminunc stopped because it cannot decrease the objective function
along the current search direction.
But if I restart the code with the new point obtained, it continued decreasing, so Im not in the minimum.
It seems to me that the algorithm its just looking for the minimum simply along one direccion and when it finds it, it stops instead start searching among the other direccions.
Any clue how to modify the problem so that doesnt happen and find the minimum? or other programm I should use in order to do it faster?
Edit: the function is convex
This type of problem can occur if your have numerical noise in your objective function. Then gradient/hessian become noisy and the step is off, and line searching ends with precisely this error.
Try plotting your criterion in an interval around your starting values but change one of the parameters from x0(i) to
linspace(x0(i)(1-1e07, x(i)(1+1e07), 100), one number at the time.
If the function looks very jittery, that may be your problem.
Alternatively, maybe you are starting in a region where the function is not convex? In that case try fminsearch to begin with.
Otherwise, consider using analytic gradients.
Also, perhaps your tolerance are set too fine. Try reducing tolfun and tolx to 1e-08. Your problem is very high dimensional.
Related
I am solving a multi objective optimisation problem. The pseudo code is as follows:
For h=1:10 % Number of points
1. Update some constraints % Update the constraint to find new points
2. Minimize F(X) % Objective function, is not updated at ittirations
Subject to the Updated constraints.
End
For each iteration , a new solution is obtained. But, for some iteration the solution is infeasible and fmincon returns exitflag of -2. So, they must be discarded from the possible solution set of the problem.
fmincon consumes a lot of time to recognise an infeasible point but for other points which have exit flag >0 it is fine. Thus, the total consumed time of the problem is significantly large due to these infeasible points.
How can I set the fmincon option to get the infeasible points faster considering the fact that I can only set the option for 1 time.
I have some data which I wish to model in order to be able to get relatively accurate values in the same range as the data.
To do this I used polyfit to fit a 6th order polynomial and due to my x-axis values it suggested I centred and scaled it to get a more accurate fit which I did.
However, now I want to find the derivative of this function in order to model the velocity of my model.
But I am not sure how the polyder function interacts with the scaled and fitted polyfit which I have produced. (I don't want to use the unscaled model as this is not very accurate).
Here is some code which reproduces my problem. I attempted to rescale the x values before putting them into the fit for the derivative but this still did no fix the problem.
x = 0:100;
y = 2*x.^2 + x + 1;
Fit = polyfit(x,y,2);
[ScaledFit,s,mu] = polyfit(x,y,2);
Deriv = polyder(Fit);
ScaledDeriv = polyder(ScaledFit);
plot(x,polyval(Deriv,x),'b.');
hold on
plot(x,polyval(ScaledDeriv,(x-mu(1))/mu(2)),'r.');
Here I have chosen a simple polynomial so that I could fit it accurate and produce the actual derivative.
Any help would be greatly appreciated thanks.
I am using Matlab R2014a BTW.
Edit.
Just been playing about with it and by dividing the resulting points for the differential by the standard deviation mu(2) it gave a very close result within the range -3e-13 to about 5e-13.
polyval(ScaledDeriv,(x-mu(1))/mu(2))/mu(2);
Not sure quite why this is the case, is there another more elegant way to solve this?
Edit2. Sorry for another edit but again was mucking around and found that for a large sample x = 1:1000; the deviation became much bigger up to 10. I am not sure if this is due to a bad polyfit even though it is centred and scaled or due to the funny way the derivative is plotted.
Thanks for your time
A simple application of the chain rule gives
Since by definition
it follows that
Which is exactly what you have verified numerically.
The lack of accuracy for large samples is due to the global, rather then local, Lagrange polynomial interpolation which you have done. I would suggest that you try to fit your data with splines, and obtain the derivative with fnder(). Another option is to apply the polyfit() function locally, i.e. to a moving small set of points, and then apply polyder() to all the fitted polynomials.
I need to numerically integrate the following system of ODEs:
dA/dR = f(R,A,B)
dB/dR = g(R,A,B)
I'm solving the ODEs for a Initial-value stability problem. In this problem, the system is initially stable but goes unstable at some radius. However, whilst stable, I don't want the amplitude to decay away from the starting value (to O(10^-5) for example) as this is non-physical since the system's stability is limited to the background noise amplitude. The amplitude should remain at the starting value of 1 until the system destabilises. Hence, I want to overwrite the derivative estimate to zero whenever it is negative.
I've written some 4th order Runge-Kutta code that achieves this, but I'd much prefer to simply pass ODE45 (or any of the built in solvers) a parameter to make it overwrite the derivative whenever it is negative. Is this possible?
A simple, fast, efficient way to implement this is via the max function. For example, if you want to make sure all of your derivatives remain non-negative, in your integration function:
function ydot = f(x,y)
ydot(1) = ...
ydot(2) = ...
...
ydot = max(ydot,0);
Note that this is not the same thing as the output states returned by ode45 remaining non-negative. The above should ensure that your state variables never decay.
Note, however, that that this effectively makes your integration function stiff. You might consider using a solver like ode15s instead, or at least confirming that the results are consistent with those from ode45. Alternatively, you could use a continuous sigmoid function, instead of the discontinuous, step-like max. This is partly a modeling decision.
I'm using fminunc. Specifically, I'm using a quasi-newton method with BFGS and a user supplied gradient to minimize a function. It goes for about 3 iterations, then does a bunch of function evaluations. I have the function I am trying to minimize display its value when it is called, and I can see that after a few calls on the 4th iteration it is significantly smaller than it was on the third iteration, but Matlab continues to evaluate the function and I'm not sure why. Is there something I can adjust to tell it to accept the update and move to the next iteration?
Sounds like you understand the optimizer pretty well, let's see if we can help with the optimset settings. From the mathworks docs it calls out how to relax the tolerances to terminate the optimization with the following parameters:
TolFun Termination tolerance on the function value, a positive scalar. The
default is 1e-6.
TolX Termination tolerance on x, a positive scalar. The default value is
1e-6.
Sounds like you have the Optimization Toolbox, in which case I believe you can use optimset to get the optimization parameters and then use the fminunc to initialize it.(The docs for the regular optimset don't call out that you can pass the fminunc as an argument, but I would think it would work for the toolbox version.
options = optimset('fminunc');
% you can try either of these two or both, but I would
% recommend only changing one at a time :-)
options = optimset(options,'TolFun',myNewTolFun)
options = optimset(options,'TolX',myNewTolX)
% then you can make your call to the optimizer
x = fminunc(fun,x0,options)
Excruciating detail on the optim-settings can be found here.
I am using fsolve to minimise an energy function in MATLAB. The algorithm I am using fits a grid to noisy lattice data, with costs for the distances of the grid from each data point.
The objective function is formulated with squared error terms, to allow for the Gauss–Newton algorithm to be used. However, the program reverts to Levenberg-Marquardt:
Warning: Trust-region-dogleg algorithm of FSOLVE cannot handle non-square systems;
using Levenberg-Marquardt algorithm instead.
I realised this is probably due to the fact that while the costs have squared errors, there is a stage in the objective (cost) function that chooses the nearest grid centre to each data point, thus making the algorithm non-square.
What I would like to do is to perform this assignment update of nearest grid centres separately to the evaluation of the Jacobian of the cost function. I believe this would then allow Gauss-Newton to be used, and significantly improve the speed of the algorithm.
Currently, I believe there is something like this going on:
while i < options.MaxIter && threshold has not been met
Compute Jacobian of cost function (which includes assignment routine)
Move down the slope in the direction of highest gradient
end
What i would like to happen instead:
while i < options.MaxIter && threshold has not been met
Perform assignment routine
Compute Jacobian of cost function (which is now square, as no assignment occurs)
Move down the slope
end
Is there a way to insert a function like this into the iterations without picking apart the whole of the fsolve algorithm? Even if I manually edited fsolve, would the nature of the Gauss-Newton algorithm allow me to add in this extra step?
Thanks
Since you're working with squared errors, anyway, you could use LSQNONLIN instead of fsovle. This allows you to compute the Jacobian (as well as all necessary preparations) in your objective function. The Jacobian is then returned as second output argument.