I'm trying to solve a non-linear constraint optimization problem using MatLab's fmincon function with SQP algorithm. This solver has been successfully applied on my problem, as I found out during my literature research.
I know my problem's solution, but fmincon struggles to find it reliably. When running the optimization a 100 times with randomly generated start values within my boundaries, I got about 40 % good results. 'good' means that the results are that close to the optimum that I would accept it, although those 'good' results correspond with different ExitFlags. Most common are Exit Flags -2 and 2:
ExitFlag = 2
Local minimum possible. Constraints satisfied.
fmincon stopped because the size of the current step is less than the selected value of the step size tolerance and constraints are satisfied to within the selected value of the constraint tolerance.
ExitFlag = -2
No feasible solution found.
fmincon stopped because the size of the current step is less than the selected value of the step size tolerance but constraints are not satisfied to within the selected value of the constraint tolerance.
The 'non-good' results deviate about 2% of the optimal solution and correspond to ExitFlags 2 and -2, as well.
I played around with the tolerances, but without success. When relaxing the constraint tolerance the number of ExitFlag -2 decreases and some ExitFlag 1 cases occur, but consequently the deviation from the optimal solution rises.
A big problem seems to be the step size which violates its tolerance. Often the solver exits after 2 or 3 iterations because of too small step size / norm of step size (relative change in X is below TolX).Is there a way to counteract these problems? I'd like to tune the solver In away to get appropriate results reliably.
For your information, the options used:
options=optimset('fmincon');
options=optimset(options,...
'Algorithm','sqp',...
'ScaleProblem','obj-and-constr',...
'TypicalX',[3, 50, 3, 40, 50, 50],...
'TolX',1e-12,...%12
'TolFun',1e-8,...%6
'TolCon',1e-3,...%6
'MaxFunEvals',1000,... %1000
'DiffMinChange',1e-10);
Related
I have multiple algorithms trying to solve the same problem using lsqnonlin. The last 3 have one parameter fixed. How do I read the output here.
a) What does funCount mean?
b) Does having lower step size mean better result?
c) If firstOrderOpt is closer to 0, is it a better result?
Algo
Iterations
FunCount
StepSize
FirstOrderOpt
1
10
69
4.00E-10
3.00E-07
2
10
68
2.00E-09
2.00E-07
3
12
65
6.00E-11
1.00E-08
4
10
69
4.00E-10
3.00E-07
5
10
68
2.00E-09
2.00E-07
6
12
65
6.00E-11
1.00E-08
From the documentation, the fields of the lsqnonline output are
Field name
Meaning
irstorderopt
Measure of first-order optimality
iterations
Number of iterations taken
funcCount
The number of function evaluations
cgiterations
Total number of PCG iterations (trust-region-reflective algorithm only)
stepsize
Final displacement in x
algorithm
Optimization algorithm used
message
Exit message
Specifically addressing your questions,
a) funcCount is the number of times your input function was evaluated to obtain the result.
b) As you approach the optimal solution, a smaller step size may be needed to avoid "jumping" straight over it. It isn't really a good measure of having a "better result", but you should expect it to be small (otherwise you could be skipping over optima) but not so small that you get into the noise of numerical precision errors within your function.
A smaller stepsize will also slow the solver down, and likely lead to more iterations - you can see this reflected in your results table as the two rows with stepsize of order 1e-11 have more iterations than the others.
The StepSize is somewhat problem dependent, the related MathWorks documentation on Tolerances and Stopping Criteria may be helpful.
c) Please read the MathWorks documentation on the First Order Optimality Measure.
First-order optimality is a measure of how close a point x is to optimal. Most Optimization Toolbox™ solvers use this measure, though it has different definitions for different algorithms. First-order optimality is a necessary condition, but it is not a sufficient condition. In other words:
The first-order optimality measure must be zero at a minimum.
A point with first-order optimality equal to zero is not necessarily a minimum.
So a smaller firstOrderOpt indicates a better result, but does not necessarily give an estimate to the true optimum - if we had that then we would likely already know the true answer without the need for an optimser!
There is an OptimalityTolerance option within lsqnonlin, so you have control over how small the first order optimality must be for the solver to stop. Again, please see the docs.
i am trying to find an optimal point with multistart / fmincon. Matlab finds a local minima, for which the following message is given in the output struct:
Local minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in feasible directions,
to within the value of the optimality tolerance,and constraints are satisfied to within the value of the
constraint tolerance.<stopping criteria details>
Optimization completed: The relative first-order optimality measure, 7.337955e-07,
is less than options.OptimalityTolerance = 1.000000e-06, and the relative maximum constraintviolation, 7.082693e-07, is less than options.ConstraintTolerance = 1.000000e-06.
I have 13 nonlinear constraints, of which one is violated.
This is the constraint:
c(10) = abs(L_1 - (L_2 - L_3)) - 0.001;
I want to achieve:
abs(L_1 - (L_2 - L_3)) <= 0.001;
If i check this constraint by hand, I get
abs(L_1 - (L_2 - L_3)) = 0.0011;
which is larger than the desired value 0.001
The constraint tolerance in optimoptions is set to 1e-6 by default.
How can this happen? The output clearly says, that all the constraints are met, but in reality this one is not.
I think you've misunderstood what fmincon provides. Solutions are guaranteed to satisfy the bounds. There is no assurance that all constraints (linear or non-linear) will be satisfied.
I will admit that you have to read between the lines when looking at the MATLAB documentation.
For the default 'interior-point' algorithm, fmincon sets components of x0 that violate the bounds lb ≤ x ≤ ub, or are equal to a bound, to the interior of the bound region. For the 'trust-region-reflective' algorithm, fmincon sets violating components to the interior of the bound region. For other algorithms, fmincon sets violating components to the closest bound. Components that respect the bounds are not changed. See Iterations Can Violate Constraints.
I am using the differential evolution optimizer in scipy and I don't understand the intuition behind the tol argument. Specifically is say in the documentation:
tol: float, optional
When the mean of the population energies, multiplied by tol, divided
by the standard deviation of the population energies is greater than 1
the solving process terminates:
convergence = mean(pop) * tol / stdev(pop) > 1
What does setting tol represent from a user perspective?
Maybe the formula in the documentation is easier to understand in the following form (see lines 508 and 526 in the code):
std(population_energies) / mean(population_energies) < tol
It means that convergence is reached when the standard deviation of the energies for each individual in the population, normed by the average, is smaller than the given tolerance value.
The optimization algorithm is iterative. At every iteration a better solution is found. The tolerance parameters is used to define a stopping condition. The stopping condition is actually that all the individuals (parameter sets) have approximately the same energy, i.e. the same cost function value. Then, the parameter set giving the lowest energy is returned as a solution.
It also implies that all the individuals are relatively close to each other in the parameter space. So, no better solution can be expected on the following generations.
I am solving a multi objective optimisation problem. The pseudo code is as follows:
For h=1:10 % Number of points
1. Update some constraints % Update the constraint to find new points
2. Minimize F(X) % Objective function, is not updated at ittirations
Subject to the Updated constraints.
End
For each iteration , a new solution is obtained. But, for some iteration the solution is infeasible and fmincon returns exitflag of -2. So, they must be discarded from the possible solution set of the problem.
fmincon consumes a lot of time to recognise an infeasible point but for other points which have exit flag >0 it is fine. Thus, the total consumed time of the problem is significantly large due to these infeasible points.
How can I set the fmincon option to get the infeasible points faster considering the fact that I can only set the option for 1 time.
I have several equations and each have their own individual frequencies and amplitudes. I would like to sum the equations together and adjust the individual phases, phase1,phase2, and phase3 to keep the total amplitude value of eq_total under a specific value like 0.8. I know I can normalize the signal or change the vertical offset, but for my purposes I need to have the amplitude controlled by changing/finding the values for just the phases in phase1,phase2, and phase3 that will limit the maximum amplitude when the equations are summed.
Note: I'm using constructive and destructive phase interference to adjust the maximum amplitude of the summed equations.
Example:
eq1=0.2*cos(2pi*t*3+phase1)+vertical offset1
eq2=0.7*cos(2pi*t*9+phase2)+vertical offset2
eq3=0.8*cos(2pi*t*5+phase3)+vertical offset3
eq_total=eq1+eq2+eq3
Is there a way to solve for phase1,phase2, and phase3 so that the amplitude of the summed signals in eq_total never goes over 0.8 by just adjusting/finding the values of phase1,phase2,and phase3?
Here's a picture of a geogebra applet I tested this idea with.
Here's the geogebra ggb file I used to edit/test idea with. (I used this to see if my idea would work) Java is required if you want to dynamically interact with the applet
http://dl.dropbox.com/u/6576402/questions/ggb/sin_find_phases_example.ggb
I'm using matlab/octave
Thanks
Your example
eq1=0.2*cos(2pi*t*3+phase1)+vertical offset1
eq2=0.7*cos(2pi*t*9+phase2)+vertical offset2
eq3=0.8*cos(2pi*t*5+phase3)+vertical offset3
eq_total=eq1+eq2+eq3
where the maximum amplitude should be less than 0.8, has infinitely many solutions. Unless you have some additional objective you'd like to achieve, I suggest that you modify the problem such that you find the combination of phase angles that has a maximum amplitude of exactly 0.8 (or 0.79, such that you're guaranteed to be below).
Furthermore only two out of three phase angles are independent; if you increase all by, say, pi/3, the solution still holds. Thus, you have only two unknowns in eq_total.
You can solve the nonlinear optimization problem using e.g. FMINSEARCH. You formulate the problem such that max(abs(eq_total(phase1,phase2))) should equal 0.79.
Thus:
%# define the vector t, verticalOffset here
%# objectiveFunction is (eq_total-0.79)^2, so the phase shifts 1 and 2 that
%# satisfy this (approximately) should guarantee that signal never exceeds 0.8
objectiveFunction = #(phase)(max(abs(0.2*cos(2*pi*t+phase(1))+0.7*cos(2*pi*t*9+phase(2))+0.8*cos(2*pi*t*5)+verticalOffset)) - 0.79)^2;
%# search for optimal phase shift, starting at no shift
solution = fminsearch(objectiveFunction,[0;0]);
EDIT
Unfortunately when I Try this code and plot the results the maximum amplitude is not 0.79 it's over 1. Am I doing something wrong? see code below t=linspace(0,1,8000); verticalOffset=0; objectiveFunction = #(phase)(max(abs(0.2*cos(2*pi*t+phase(1))+0.7*cos(2*pi*t*9+phase(2))+0.8*cos(2*pi*t*5)+verticalOffset)) - 0.79)^2; s1 = fminsearch(objectiveFunction,[0;0]) eqt=0.2*cos(2*pi*t+s1(1))+0.7*cos(2*pi*t*9+s1(2))+0.8*cos(2*pi*t*5)+verticalOffset; plot(eqt)
fminsearch will find a minimum of the objective function. Whether this solution satisfies all your conditions is something you have to test. In this case, the solution given by fminsearch with the starting value [0;0] gives a maximum of ~1.3, which is obviously not good enough. However, when you plot the maximum for a range of phase angles from 0 to 2pi, you'll see that `fminsearch didn't get stuck in a bad local minimum. Rather, there is no good solution at all (z-axis is the maximum).
If I understand you correctly, you are trying to find a phase to vary the amplitude of a signal. To my knowledge, this is not possible.
For a signal
s = A * cos (w*t + phi)
only A allows you to change the amplitude. With w you change the frequency of the signal and phi regulates the "horizontal shift".
Furthermore, I think you are missing a "moving variable" like the time t in the equation above.
Maybe this article clarifies things a little.
If you set all the vertical offsets to be equal to -1, then it solves your problem because each eq# will never be > 0, so the sum can never be >0.8.
I know that this isn't that helpful, but I'm hoping that this will help you understand your problem better.