fmincon doesn't find a global minimum for a convex function - matlab

In my opinion, fmincon is a built-in function for local minimum in matlab. If the objective function is a convex problem, there is only one basin and the local minimum is the global minimum. While starting from different initial points in my experiment, the algorithm got different minimums function. I wonder if fmincon guarantees to be converged to a global minimum for convex problem. If not, is there any other techiniques I can use for convex opimization as fast as possible? Thanks.
P.S. fmincon use interior-point-method for searching minimum in default. Is this a normal problem for interior-point method, that is ,starting from different intial point, the method can get different global minimum for convex problem?
EDIT:
The objective is to minimize the sum of energy consumption by a group of users in a communication process, while the allocation of bandwidth is search. The transmission rate is
$r_k = x_k * log_2(1+\frac{g_k*p_k}{x_k})$
The optimization problem is as follow
$min_{x} sum_k \frac{p_k*b_k}{r_k}$
s.t. $sum_k x_k \leq X_{max}$
The objective and constraints are all convex, thus this should be a convex optimization problem.
For programming code, it is just as follow,
options = optimoptions('fmincon');
problem.options = options;
problem.solver = 'fmincon';
problem.objective = #(x) langBW(x, in_s, in_e, C1, a, p_ul);
problem.Aineq = ones(1,user_num);
problem.bineq = BW2;
problem.nonlcon = #(x) nonlConstr_bw(x,a,p_ul,T1,in_s,in_e,BW2);
problem.x0 = ones(user_num,1)
[b_ul,fval] = fmincon(problem);
langBW is the objective function, which is a convex function of x, the code of langBW is as follow,
function fmin = langBW(x, in_s, in_e, C1, a, p_ul)
if size(x,1)<size(x,2)
x = x';
end
b_ul = x;
r_ul = b_ul .* log2(1 + a.*p_ul./b_ul);
fmin = sum((in_s+in_e).*p_ul./r_ul) + sum(C1);
end
The nonlConstr_bw is the function of nonlinear constraints. It is shown as follow,
function [c,ceq] = nonlConstr_bw(x,a,p_ul,T1,in_s,in_e)
user_num = size(p_ul,1);
if size(x,1)<size(x,2)
x = x';
end
b_ul = x;
r_ul = b_ul .* log2(1 + a.*p_ul./b_ul);
c1 = max(in_s./r_ul) + in_e./r_ul - T1;
c = c1;
ceq = zeros(user_num,1);
end
Except x, all other variables are supplied. The problem is that when I set different problem.x0, for example, when problem.x0=ones(user_num,1);, the solution of [b_ul,fval] = fmincon(problem); is different from that when problem.x0=2*ones(user_num,1);. That is what I am confused about.

fmincon uses the following algorithms:
'interior-point' (default)
'trust-region-reflective'
'sqp' (Sequential Quadratic Programming)
'sqp-legacy'
'active-set'
These methods will converge to a local minimia but not necessarily a global minimum. Further minima may not be unique. The only way to guarantee a global minima is to search the whole solution space.
From your comment, there appears to be only a signal minima? (For example, a shifted parabola?) Then it should converge.
edit--
Even if your function appears convex, the constraints can lead to multiple local minima. Sometimes this is called a "loosely" convex function

Related

Solving system of equations on MATLAB, when a constant exists in variable matrix?

How do I solve the following system of equations on MATLAB when one of the elements of the variable vector is a constant? Please do give the code if possible.
More generally, if the solution is to use symbolic math, how will I go about generating large number of variables, say 12 (rather than just two) even before solving them?
For example, create a number of symbolic variables using syms, and then make the system of equations like below.
syms a1 a2
A = [matrix]
x = [1;a1;a2];
y = [1;0;0];
eqs = A*x == y
sol = solve(eqs,[a1, a2])
sol.a1
sol.a2
In case you have a system with many variables, you could define all the symbols using syms, and solve it like above.
You could also perform a parameter optimization with fminsearch. First you have to define a cost function, in a separate function file, in this example called cost_fcn.m.
function J = cost_fcn(p)
% make sure p is a vector
p = reshape(p, [length(p) 1]);
% system of equations, can be linear or nonlinear
A = magic(12); % your system, I took some arbitrary matrix
sol = A*p;
% the goal of the system of equations to reach, can be zero, or some other
% vector
goal = zeros(12,1);
% calculate the error
error = goal - sol;
% Use a cost criterion, e.g. sum of squares
J = sum(error.^2);
end
This cost function will contain your system of equations, and goal solution. This can be any kind of system. The vector p will contain the parameters that are being estimated, which will be optimized, starting from some initial guess. To do the optimization, you will have to create a script:
% initial guess, can be zeros, or some other starting point
p0 = zeros(12,1);
% do the parameter optimization
p = fminsearch(#cost_fcn, p0);
In this case p0 is the initial guess, which you provide to fminsearch. Then the values of this initial guess will be incremented, until a minimum to the cost function is found. When the parameter optimization is finished, p will contain the parameters that will result in the lowest error for your system of equations. It is however possible that this is a local minimum, if there is no exact solution to the problem.
Your system is over-constrained, meaning you have more equations than unknown, so you can't solve it. What you can do is find a least square solution, using mldivide. First re-arrange your equations so that you have all the constant terms on the right side of the equal sign, then use mldivide:
>> A = [0.0297 -1.7796; 2.2749 0.0297; 0.0297 2.2749]
A =
0.029700 -1.779600
2.274900 0.029700
0.029700 2.274900
>> b = [1-2.2749; -0.0297; 1.7796]
b =
-1.274900
-0.029700
1.779600
>> A\b
ans =
-0.022191
0.757299

How do I optimize constrained integral expressions in MATLAB using anonymous functions?

I have an integrated error expression E = int[ abs(x-p)^2 ]dx with limits x|0 to x|L. The variable p is a polynomial of the form 2*(a*sin(x)+b(a)*sin(2*x)+c(a)*sin(3*x)). In other words, both coefficients b and c are known expressions of a. An additional equation is given as dE/da = 0. If the upper limit L is defined, the system of equations is closed and I can solve for a, giving the three coefficients.
I managed to get an optimization routine to solve for a purely based on maximizing L. This is confirmed by setting optimize=0 in the code below. It gives the same solution as if I solved the problem analytically. Therefore, I know the equations to solve for the coefficent a are correct.
I know the example I presented can be solved with pencil and paper, but I'm trying to build an optimization function that is generalized for this type of problem (I have a lot to evaluate). Ideally, polynomial is given as an input argument to a function which then outputs xsol. Obviously, I need to get the optimization to work for the polynomial I presented here before I can worry about generalizations.
Anyway, I now need to further optimize the problem with some constraints. To start, L is chosen. This allows me to calculate a. Once a is know, the polynomial is a known function of x only i.e p(x). I need to then determine the largest INTERVAL from 0->x over which the following constraint is satisfied: |dp(x)/dx - 1| < tol. This gives me a measure of the performance of the polynomial with the coefficient a. The interval is what I call the "bandwidth". I would like to emphasis two things: 1) The "bandwidth" is NOT the same as L. 2) All values of x within the "bandwidth" must meet the constraint. The function dp(x)/dx does oscillate in and out of the tolerance criteria, so testing the criteria for a single value of x does not work. It must be tested over an interval. The first instance of violation defines the bandwidth. I need to maximize this "bandwidth"/interval. For output, I also need to know which L lead to such an optimization, hence I know the correct a to choose for the given constraints. That is the formal problem statement. (I hope I got it right this time)
Now my problem is setting this whole thing up with MATLAB's optimization tools. I tried to follow ideas from the following articles:
Tutorial for the Optimization Toolbox™
Setting optimize=1 for the if statement will work with the constrained optimization. I thought some how nested optimization is involved, but I couldn't get anything to work. I provided known solutions to the problem from the IMSL optimization library to compare/check with. They are written below the optimization routine. Anyway, here is the code I've put together so far:
function [history] = testing()
% History
history.fval = [];
history.x = [];
history.a = [];
%----------------
% Equations
polynomial = #(x,a) 2*sin(x)*a + 2*sin(2*x)*(9/20 -(4*a)/5) + 2*sin(3*x)*(a/5 - 2/15);
dpdx = #(x,a) 2*cos(x)*a + 4*cos(2*x)*(9/20 -(4*a)/5) + 6*cos(3*x)*(a/5 - 2/15);
% Upper limit of integration
IC = 0.8; % initial
LB = 0; % lower
UB = pi/2; % upper
% Optimization
tol = 0.003;
% Coefficient
% --------------------------------------------------------------------------------------------
dpda = #(x,a) 2*sin(x) + 2*sin(2*x)*(-4/5) + 2*sin(3*x)*1/5;
dEda = #(L,a) -2*integral(#(x) (x-polynomial(x,a)).*dpda(x,a),0,L);
a_of_L = #(L) fzero(#(a)dEda(L,a),0); % Calculate the value of "a" for a given "L"
EXITFLAG = #(L) get_outputs(#()a_of_L(L),3); % Be sure a zero is actually calculated
% NL Constraints
% --------------------------------------------------------------------------------------------
% Equality constraint (No inequality constraints for parent optimization)
ceq = #(L) EXITFLAG(L) - 1; % Just make sure fzero finds unique solution
confun = #(L) deal([],ceq(L));
% Objective function
% --------------------------------------------------------------------------------------------
% (Set optimize=0 to test coefficent equations and proper maximization of L )
optimize = 1;
if optimize
%%%% Plug in solution below
else
% Optimization options
options = optimset('Algorithm','interior-point','Display','iter','MaxIter',500,'OutputFcn',#outfun);
% Optimize objective
objective = #(L) -L;
xsol = fmincon(objective,IC,[],[],[],[],LB,UB,confun,options);
% Known optimized solution from IMSL library
% a = 0.799266;
% lim = pi/2;
disp(['IMSL coeff (a): 0.799266 Upper bound (L): ',num2str(pi/2)])
disp(['code coeff (a): ',num2str(history.a(end)),' Upper bound: ',num2str(xsol)])
end
% http://stackoverflow.com/questions/7921133/anonymous-functions-calling-functions-with-multiple-output-forms
function varargout = get_outputs(fn, ixsOutputs)
output_cell = cell(1,max(ixsOutputs));
[output_cell{:}] = (fn());
varargout = output_cell(ixsOutputs);
end
function stop = outfun(x,optimValues,state)
stop = false;
switch state
case 'init'
case 'iter'
% Concatenate current point and objective function
% value with history. x must be a row vector.
history.fval = [history.fval; optimValues.fval];
history.x = [history.x; x(1)];
history.a = [history.a; a_of_L(x(1))];
case 'done'
otherwise
end
end
end
I could really use some help setting up the constrained optimization. I'm not only new to optimizations, I've never used MATLAB to do so. I should also note that what I have above does not work and is incorrect for the constrained optimization.
UPDATE: I added a for loop in the section if optimizeto show what I'm trying to achieve with the optimization. Obviously, I could just use this, but it seems very inefficient, especially if I increase the resolution of range and have to run this optimization many times. If you uncomment the plots, it will show how the bandwidth behaves. By looping over the full range, I'm basically testing every L but surely there's got to be a more efficient way to do this??
UPDATE: Solved
So it seems fmincon is not the only tool for this job. In fact I couldn't even get it to work. Below, fmincon gets "stuck" on the IC and refuses to do anything...why...that's for a different post! Using the same layout and formulation, fminbnd finds the correct solution. The only difference, as far as I know, is that the former was using a conditional. But my conditional is nothing fancy, and really unneeded. So it's got to have something to do with the algorithm. I guess that's what you get when using a "black box". Anyway, after a long, drawn out, painful, learning experience, here is a solution:
options = optimset('Display','iter','MaxIter',500,'OutputFcn',#outfun);
% Conditional
index = #(L) min(find(abs([dpdx(range(range<=L),a_of_L(L)),inf] - 1) - tol > 0,1,'first'),length(range));
% Optimize
%xsol = fmincon(#(L) -range(index(L)),IC,[],[],[],[],LB,UB,confun,options);
xsol = fminbnd(#(L) -range(index(L)),LB,UB,options);
I would like to especially thank #AndrasDeak for all their support. I wouldn't have figured it out without the assistance!

Finding unknown limit of integration in MATLAB

I have an equation of the form c = integral of f(t)dt limiting from a constant to a variable (I don't want to show the full equation because it is very long and complex). Is there any way to calculate in MATLAB what the value of that variable is (there are no other variables and the equation is too difficult to solve by hand)?
Assume your limit is from cons to t and g(t) as your function with variable t. Now,
syms t
f(t) = int(g(t),t);
This will give you the indefinite integral. Now f(t) will be
f(t) = f(t)+f(cons);
You have the value of f(t)=c. So just solve the equation
S = solve(f(t)==c,t,'Real',true);
eval(S) will give the answer i think
This is an extremely unclear question - if you do not want to post the full equation, post an example instead
I am assuming this is what you intend: you have an integrand f(x), which you know, and has been integrated to give some constant c which you know, over the limits of x = 0, to x = y, for example, where y may change, and you desire to find y
My advice would be to integrate f(x) manually, fill in the first limit, and subtract that portion from c. Next you could employ some technique such as the Newton-Ralphson method to iteratively search for the root to your equation, which should be in x only
You could use a function handle and the quad function for the integral
myFunc = #(t) exp(t*3); % or whatever
t0 = 0;
t1 = 3;
L = 50;
f = #(b) quad(#(t) myFunc(t,b),t0,t1);
bsolve = fzero(f,2);
Hope it help !

MATLAB Optimization Error

My Genetic Algorithm optimization script on Matlab runs, but at the end produces the following message: "Optimization terminated: average change in the penalty fitness value less than options.TolFun but constraints are not satisfied."
Why would it say that? I replaced my fitness function with one that does nothing but returns a constant number, and nothing changed. It may be the case that my constraints are not defined correctly, though I can't find the mistake. Here is the relevant part of the code:
nGenerators = 9;
monthlyHours = 24*daysInJanuary;
(some irrelevant code here)
steamCapacities = [31.46*ones(1,2) 5.5*ones(1,3) 4*ones(1,4)];
nVars = xSize;
IntCon = 1:nVars;
LB = zeros(1, nVars);
UB = ones(1, nVars);
b = -1*steamLoad; % Ax <= b
A = zeros(monthlyHours, xSize);
for p = 1:monthlyHours
A(p, 9*p-8:9*p) = -1*steamCapacities;
% disp(p);
end
(some more code here)
anonFitness = #(x)mosb_test(x, fitnessData);
gaOptions = gaoptimset('Vectorized', 'off', 'UseParallel', 'always', ...
'Display', 'diagnose', 'PlotFcn', #gaplotbestf, 'Generations', 300, ...
'TolFun', 1e-15, 'StallGenLimit', 200);
[x, fval, exitFlag] = ga(anonFitness, nVars, A, b,[],[], LB, UB, [], IntCon, gaOptions);
Here, x represents the on/off states for a set of 9 generators for each hour of the month. Their steam production capacities are as in the variable steamCapacities, and there is a constant steam load that has to be met at every hour. This is represented by the inequality constraint.
Any help appreciated.
Interesting problem! Too bad it can't be solved deterministically. :-( Anyway, your TolFun is defined by the mathworks as being:
Positive scalar. The algorithm stops if the weighted average relative
change in the best fitness function value over StallGenLimit
generations is less than or equal to TolFun
and StallGenLimit has the same description. It makes perfect sense that a constant will give you the same error. It appears to me that the GA is unable to converge to a solution in the StallGenLimit, here 200, generations. You could try making the TolFun more restrictive, like 1e-12, or try and develop a far simpler model for the GA, one that will just allow you to find a solution. Then build complexity on this model until it matches your real system. Likely, it will fail at some point and you will be able to see why the problem can't be solved like too many degrees of freedom, too tight of constraints, etc. HTH!

Matlab, integration of conservation laws

I was wondering if there are established robust libraries or FEX-like packages to deal with scalar conservation laws (say 1D) in matlab.
I am currently dealing with 1D non-linear, non-local, conservation laws and the diffusive error of first order schemes is killing me, moreover a lot of physics is missed. Thus, I am wondering if there is some robust tool already there so to avoid cooking some code myself (ideally, something like boost::odeint for scheme agnostic high order ODE integration in C++).
Any help appreciated.
EDIT: Apologies for the lack of clarity. Here for conservation laws I mean general hyberbolic partial derivative equations in the form
u_t(t,x) + F_x(t,x) = 0
where u=u(t,x) is an intensive conserved variable (say scalar, 1D, e.g. mass density, energy density,...) and F = F(t,x) is its flux. Therefore, I am not interested in the kind of conservation properties Hamiltonian systems feature (energy, currents...) (thanks to #headmyshoulder for his comment).
I cited boost::odeint for a conceptual reference of a robust and generic library addressing a mathematical issue (integration of ODEs). Therefore I am looking for some package implementing Godunov-type methods and so on.
I am currently working on new methods for shock-turbulence simulations and doing lots of code testing/validation in MATLAB. Unfortunately, I haven't found a general library that does what you're hoping, but a basic Godunov or MUSCL code is relatively straightforward to implement. This paper has a good overview of some useful methods:
[1] Kurganov, Alexander and Eitan Tadmor (2000), New High-Resolution Central Schemes for Nonlinear Conservation Laws and Convection-Diffusion Equations, J. Comp. Phys., 160, 214–282. PDF
Here are a few examples from that paper for a 1D equally spaced grid on a periodic domain for solving inviscid Burgers equation. The methods easily generalize to systems of equations, dissipative (viscous) systems, and higher dimension as outlined in [1]. These methods rely on the following functions:
Flux term:
function f = flux(u)
%flux term for Burgers equation: F(u) = u^2/2;
f = u.^2/2;
Minmod function:
function m = minmod(a,b)
%minmod function:
m = (sign(a)+sign(b))/2.*min(abs(a),abs(b));
Methods
Nessyahu-Tadmor scheme:
A 2nd order scheme
function unew = step_u(dx,dt,u)
%%% Nessyahu-Tadmor scheme
ux = minmod((u-circshift(u,[0 1]))/dx,(circshift(u,[0 -1])-u)/dx);
f = flux(u);
fx = minmod((f-circshift(f,[0 1]))/dx,(circshift(f,[0 -1])-f)/dx);
umid = u-dt/2*fx;
fmid = flux(umid);
unew = (u + circshift(u,[0 -1]))/2 + (dx)/8*(ux-circshift(ux,[0 -1])) ...
-dt/dx*( circshift(fmid,[0 -1])-fmid );
This method computes a new u value at xj+1/2 grid points so it also requires a grid shift at each step. The main function should be something like:
clear all
% Set up grid
nx = 256;
xmin=0; xmax=2*pi;
x=linspace(xmin,xmax,nx);
dx = x(2)-x(1);
%initialize
u = exp(-4*(x-pi*1/2).^2)-exp(-4*(x-pi*3/2).^2);
%CFL number:
CFL = 0.25;
t = 0;
dt = CFL*dx/max(abs(u(:)));
while (t<2)
u = step_u(dx,dt,u);
x=x+dx/2;
% handle grid shifts
if x(end)>=xmax+dx
x(end)=0;
x=circshift(x,[0 1]);
u=circshift(u,[0 1]);
end
t = t+dt;
%plot
figure(1)
clf
plot(x,u,'k')
title(sprintf('time, t = %1.2f',t))
if ~exist('YY','var')
YY=ylim;
end
axis([xmin xmax YY])
drawnow
end
Kurganov-Tadmor scheme
The Kurganov-Tadmor scheme of [1] has several advantages over the NT scheme including lower numerical dissipation and a semi-discrete form that allows the use of any time integration method you choose. Using the same spatial discretization as above, it can be formulated as an ODE for du/dt = (stuff). The right hand side of this ODE can be computed by the function:
function RHS = KTrhs(dx,u)
%%% Kurganov-Tadmor scheme
ux = minmod((u-circshift(u,[0 1]))/dx,(circshift(u,[0 -1])-u)/dx);
uplus = u-dx/2*ux;
uminus = circshift(u+dx/2*ux,[0 1]);
a = max(abs(rhodF(uminus)),abs(rhodF(uplus)));
RHS = -( flux(circshift(uplus,[0 -1]))+flux(circshift(uminus,[0 -1])) ...
-flux(uplus)-flux(uminus) )/(2*dx) ...
+( circshift(a,[0 -1]).*(circshift(uplus,[0 -1])-circshift(uminus,[0 -1])) ...
-a.*(uplus-uminus) )/(2*dx);
This function also relies on knowing the spectral radius of the Jacobian of F(u) (rhodF in the code above). For inviscid Burgers this is just
function rho = rhodF(u)
dFdu=abs(u);
The main program of the KT scheme could be something like:
clear all
nx = 256;
xmin=0; xmax=2*pi;
x=linspace(xmin,xmax,nx);
dx = x(2)-x(1);
%initialize
u = exp(-4*(x-pi*1/2).^2)-exp(-4*(x-pi*3/2).^2);
%CFL number:
CFL = 0.25;
t = 0;
dt = CFL*dx/max(abs(u(:)));
while (t<3)
% 4th order Runge-Kutta time stepping
k1 = KTrhs(dx,u);
k2 = KTrhs(dx,u+dt/2*k1);
k3 = KTrhs(dx,u+dt/2*k2);
k4 = KTrhs(dx,u+dt*k3);
u = u+dt/6*(k1+2*k2+2*k3+k4);
t = t+dt;
%plot
figure(1)
clf
plot(x,u,'k')
title(sprintf('time, t = %1.2f',t))
if ~exist('YY','var')
YY=ylim;
end
axis([xmin xmax YY])
drawnow
end