I have an integrated error expression E = int[ abs(x-p)^2 ]dx with limits x|0 to x|L. The variable p is a polynomial of the form 2*(a*sin(x)+b(a)*sin(2*x)+c(a)*sin(3*x)). In other words, both coefficients b and c are known expressions of a. An additional equation is given as dE/da = 0. If the upper limit L is defined, the system of equations is closed and I can solve for a, giving the three coefficients.
I managed to get an optimization routine to solve for a purely based on maximizing L. This is confirmed by setting optimize=0 in the code below. It gives the same solution as if I solved the problem analytically. Therefore, I know the equations to solve for the coefficent a are correct.
I know the example I presented can be solved with pencil and paper, but I'm trying to build an optimization function that is generalized for this type of problem (I have a lot to evaluate). Ideally, polynomial is given as an input argument to a function which then outputs xsol. Obviously, I need to get the optimization to work for the polynomial I presented here before I can worry about generalizations.
Anyway, I now need to further optimize the problem with some constraints. To start, L is chosen. This allows me to calculate a. Once a is know, the polynomial is a known function of x only i.e p(x). I need to then determine the largest INTERVAL from 0->x over which the following constraint is satisfied: |dp(x)/dx - 1| < tol. This gives me a measure of the performance of the polynomial with the coefficient a. The interval is what I call the "bandwidth". I would like to emphasis two things: 1) The "bandwidth" is NOT the same as L. 2) All values of x within the "bandwidth" must meet the constraint. The function dp(x)/dx does oscillate in and out of the tolerance criteria, so testing the criteria for a single value of x does not work. It must be tested over an interval. The first instance of violation defines the bandwidth. I need to maximize this "bandwidth"/interval. For output, I also need to know which L lead to such an optimization, hence I know the correct a to choose for the given constraints. That is the formal problem statement. (I hope I got it right this time)
Now my problem is setting this whole thing up with MATLAB's optimization tools. I tried to follow ideas from the following articles:
Tutorial for the Optimization Toolbox™
Setting optimize=1 for the if statement will work with the constrained optimization. I thought some how nested optimization is involved, but I couldn't get anything to work. I provided known solutions to the problem from the IMSL optimization library to compare/check with. They are written below the optimization routine. Anyway, here is the code I've put together so far:
function [history] = testing()
% History
history.fval = [];
history.x = [];
history.a = [];
%----------------
% Equations
polynomial = #(x,a) 2*sin(x)*a + 2*sin(2*x)*(9/20 -(4*a)/5) + 2*sin(3*x)*(a/5 - 2/15);
dpdx = #(x,a) 2*cos(x)*a + 4*cos(2*x)*(9/20 -(4*a)/5) + 6*cos(3*x)*(a/5 - 2/15);
% Upper limit of integration
IC = 0.8; % initial
LB = 0; % lower
UB = pi/2; % upper
% Optimization
tol = 0.003;
% Coefficient
% --------------------------------------------------------------------------------------------
dpda = #(x,a) 2*sin(x) + 2*sin(2*x)*(-4/5) + 2*sin(3*x)*1/5;
dEda = #(L,a) -2*integral(#(x) (x-polynomial(x,a)).*dpda(x,a),0,L);
a_of_L = #(L) fzero(#(a)dEda(L,a),0); % Calculate the value of "a" for a given "L"
EXITFLAG = #(L) get_outputs(#()a_of_L(L),3); % Be sure a zero is actually calculated
% NL Constraints
% --------------------------------------------------------------------------------------------
% Equality constraint (No inequality constraints for parent optimization)
ceq = #(L) EXITFLAG(L) - 1; % Just make sure fzero finds unique solution
confun = #(L) deal([],ceq(L));
% Objective function
% --------------------------------------------------------------------------------------------
% (Set optimize=0 to test coefficent equations and proper maximization of L )
optimize = 1;
if optimize
%%%% Plug in solution below
else
% Optimization options
options = optimset('Algorithm','interior-point','Display','iter','MaxIter',500,'OutputFcn',#outfun);
% Optimize objective
objective = #(L) -L;
xsol = fmincon(objective,IC,[],[],[],[],LB,UB,confun,options);
% Known optimized solution from IMSL library
% a = 0.799266;
% lim = pi/2;
disp(['IMSL coeff (a): 0.799266 Upper bound (L): ',num2str(pi/2)])
disp(['code coeff (a): ',num2str(history.a(end)),' Upper bound: ',num2str(xsol)])
end
% http://stackoverflow.com/questions/7921133/anonymous-functions-calling-functions-with-multiple-output-forms
function varargout = get_outputs(fn, ixsOutputs)
output_cell = cell(1,max(ixsOutputs));
[output_cell{:}] = (fn());
varargout = output_cell(ixsOutputs);
end
function stop = outfun(x,optimValues,state)
stop = false;
switch state
case 'init'
case 'iter'
% Concatenate current point and objective function
% value with history. x must be a row vector.
history.fval = [history.fval; optimValues.fval];
history.x = [history.x; x(1)];
history.a = [history.a; a_of_L(x(1))];
case 'done'
otherwise
end
end
end
I could really use some help setting up the constrained optimization. I'm not only new to optimizations, I've never used MATLAB to do so. I should also note that what I have above does not work and is incorrect for the constrained optimization.
UPDATE: I added a for loop in the section if optimizeto show what I'm trying to achieve with the optimization. Obviously, I could just use this, but it seems very inefficient, especially if I increase the resolution of range and have to run this optimization many times. If you uncomment the plots, it will show how the bandwidth behaves. By looping over the full range, I'm basically testing every L but surely there's got to be a more efficient way to do this??
UPDATE: Solved
So it seems fmincon is not the only tool for this job. In fact I couldn't even get it to work. Below, fmincon gets "stuck" on the IC and refuses to do anything...why...that's for a different post! Using the same layout and formulation, fminbnd finds the correct solution. The only difference, as far as I know, is that the former was using a conditional. But my conditional is nothing fancy, and really unneeded. So it's got to have something to do with the algorithm. I guess that's what you get when using a "black box". Anyway, after a long, drawn out, painful, learning experience, here is a solution:
options = optimset('Display','iter','MaxIter',500,'OutputFcn',#outfun);
% Conditional
index = #(L) min(find(abs([dpdx(range(range<=L),a_of_L(L)),inf] - 1) - tol > 0,1,'first'),length(range));
% Optimize
%xsol = fmincon(#(L) -range(index(L)),IC,[],[],[],[],LB,UB,confun,options);
xsol = fminbnd(#(L) -range(index(L)),LB,UB,options);
I would like to especially thank #AndrasDeak for all their support. I wouldn't have figured it out without the assistance!
Related
I have been stuck trying trying to write a MATLAB algorithm that computes a recursion in reverse, or that is what it seems like to me.
y_n = (1/n)−10*y_n−1 for n = 1,...,30 works in MATLAB, but because of the (*10), the round-off error makes the algorithm unstable and it is useless. Just by manipulating the recursion, y_n-1 = (1/10)(1/n - y_n) will work and the round-off errors will be reduced 10 fold at each step, potentially making this a stable algorithm.
After a couple days, I still cannot understand the logic needed to code this. Evaluating at y_n-1 is really throwing me in a loop. I was able to tackle the unstable algorithm, but I cannot think of the logic to manipulate the code to make it work. My question lies with how do you code this in MATLAB? I am truly stumped.
% Evaluate the integral yn = integral from 0 to 1 of x^n/(x+10).
% Unstable algorithm:
y(1) = log(11) - log(10);
k = 30;
for n = 1:k
y(n+1) = (1/n) - 10*y(n);
end
n_vector = 0:k;
[n_vector;y]
By manipulating the recursion, the results will be close to true values because of the bound on the error. The current output:
0.0953101798043248
0.0468982019567523
0.0310179804324768
0.0231535290085650
0.0184647099143501
0.0153529008564988
0.0131376581016785
0.0114805618403582
0.0101943815964183
0.00916729514692832
0.00832704853071678
0.00763860560192312
0.00694727731410218
0.00745030378205516
-0.00307446639198020
0.0974113305864686
-0.911613305864686
9.17495658805863
-91.6940103250307
916.992734829255
-9169.87734829255
91698.8211019731
-916988.165565185
9169881.69913012
-91698816.9496345
916988169.536345
-9169881695.32499
91698816953.2869
-916988169532.833
9169881695328.37
-91698816953283.7
What is expected, with the round-off errors taken care of is the results to stay between 0and1.
This output you are getting is correct, and as pointed out in the comments by Mad Physicist, the recursive function you have should behave this way.
If you look at the behavior of the two terms, as n gets bigger the initial subtraction will have less of an effect on the 10*y(n) term. So for large n, we can ignore 1/n.
At large n we then expect each step will increase our value by roughly a factor of 10. This is what you see in your output.
As far as writing a backward recursion. By definition you need a starting value, so you would need to assume y(30) and run the recursion backward as suggested in the comments.
So, I was able to answer by own question. The code needed would look like this:
% This function calculates the value of y20 with a guarantee to have an
% absolute error less than 10^-5
% The yn1 chosen to be high enough to guarantee this is n1 = 25
% Returns the value of y(20)
function [x]= formula(k)
% RECURSION APPROXIMATION
y(k) = 0;
n = k:-1:20;
y(n-1) = (1./10)*(1./n - y(n));
x = y(20);
% FURTHER: I needed to guarantee y20 to have <= 10^-5 magnitude error
% I determined n=25 would be my starting point, approximating y25=0 and working
% backwards to n=20 as I did above.
% y(n-1)=1/10(1/n-yn) “exact solution”
% (yn-1)*=1/10(1/n-(yn)*) “approximate solution with error”
% y(n-1)-(y(n-1))*=1/10(1/n-yn)-1/10(1/n-(yn)*) calculating the error
% = 1/10((yn)*-yn)
% So,
% E(n-1)=1/10(En)
% E(n-2)=1/100(E(n-1))
% E(n-3)=1/1000(E(n-2))
% E(n-4)=1/10000(E(n-3))
% E(n-5)=1/100000(E(n-4)) ⇒ 10^(-5)
% En20=(10^-5)En25
% Therefore, if we start with n1=25, it guarantees that y20 will have 10^-5 magnitude of % the initial propagating error.
How do I solve the following system of equations on MATLAB when one of the elements of the variable vector is a constant? Please do give the code if possible.
More generally, if the solution is to use symbolic math, how will I go about generating large number of variables, say 12 (rather than just two) even before solving them?
For example, create a number of symbolic variables using syms, and then make the system of equations like below.
syms a1 a2
A = [matrix]
x = [1;a1;a2];
y = [1;0;0];
eqs = A*x == y
sol = solve(eqs,[a1, a2])
sol.a1
sol.a2
In case you have a system with many variables, you could define all the symbols using syms, and solve it like above.
You could also perform a parameter optimization with fminsearch. First you have to define a cost function, in a separate function file, in this example called cost_fcn.m.
function J = cost_fcn(p)
% make sure p is a vector
p = reshape(p, [length(p) 1]);
% system of equations, can be linear or nonlinear
A = magic(12); % your system, I took some arbitrary matrix
sol = A*p;
% the goal of the system of equations to reach, can be zero, or some other
% vector
goal = zeros(12,1);
% calculate the error
error = goal - sol;
% Use a cost criterion, e.g. sum of squares
J = sum(error.^2);
end
This cost function will contain your system of equations, and goal solution. This can be any kind of system. The vector p will contain the parameters that are being estimated, which will be optimized, starting from some initial guess. To do the optimization, you will have to create a script:
% initial guess, can be zeros, or some other starting point
p0 = zeros(12,1);
% do the parameter optimization
p = fminsearch(#cost_fcn, p0);
In this case p0 is the initial guess, which you provide to fminsearch. Then the values of this initial guess will be incremented, until a minimum to the cost function is found. When the parameter optimization is finished, p will contain the parameters that will result in the lowest error for your system of equations. It is however possible that this is a local minimum, if there is no exact solution to the problem.
Your system is over-constrained, meaning you have more equations than unknown, so you can't solve it. What you can do is find a least square solution, using mldivide. First re-arrange your equations so that you have all the constant terms on the right side of the equal sign, then use mldivide:
>> A = [0.0297 -1.7796; 2.2749 0.0297; 0.0297 2.2749]
A =
0.029700 -1.779600
2.274900 0.029700
0.029700 2.274900
>> b = [1-2.2749; -0.0297; 1.7796]
b =
-1.274900
-0.029700
1.779600
>> A\b
ans =
-0.022191
0.757299
In my opinion, fmincon is a built-in function for local minimum in matlab. If the objective function is a convex problem, there is only one basin and the local minimum is the global minimum. While starting from different initial points in my experiment, the algorithm got different minimums function. I wonder if fmincon guarantees to be converged to a global minimum for convex problem. If not, is there any other techiniques I can use for convex opimization as fast as possible? Thanks.
P.S. fmincon use interior-point-method for searching minimum in default. Is this a normal problem for interior-point method, that is ,starting from different intial point, the method can get different global minimum for convex problem?
EDIT:
The objective is to minimize the sum of energy consumption by a group of users in a communication process, while the allocation of bandwidth is search. The transmission rate is
$r_k = x_k * log_2(1+\frac{g_k*p_k}{x_k})$
The optimization problem is as follow
$min_{x} sum_k \frac{p_k*b_k}{r_k}$
s.t. $sum_k x_k \leq X_{max}$
The objective and constraints are all convex, thus this should be a convex optimization problem.
For programming code, it is just as follow,
options = optimoptions('fmincon');
problem.options = options;
problem.solver = 'fmincon';
problem.objective = #(x) langBW(x, in_s, in_e, C1, a, p_ul);
problem.Aineq = ones(1,user_num);
problem.bineq = BW2;
problem.nonlcon = #(x) nonlConstr_bw(x,a,p_ul,T1,in_s,in_e,BW2);
problem.x0 = ones(user_num,1)
[b_ul,fval] = fmincon(problem);
langBW is the objective function, which is a convex function of x, the code of langBW is as follow,
function fmin = langBW(x, in_s, in_e, C1, a, p_ul)
if size(x,1)<size(x,2)
x = x';
end
b_ul = x;
r_ul = b_ul .* log2(1 + a.*p_ul./b_ul);
fmin = sum((in_s+in_e).*p_ul./r_ul) + sum(C1);
end
The nonlConstr_bw is the function of nonlinear constraints. It is shown as follow,
function [c,ceq] = nonlConstr_bw(x,a,p_ul,T1,in_s,in_e)
user_num = size(p_ul,1);
if size(x,1)<size(x,2)
x = x';
end
b_ul = x;
r_ul = b_ul .* log2(1 + a.*p_ul./b_ul);
c1 = max(in_s./r_ul) + in_e./r_ul - T1;
c = c1;
ceq = zeros(user_num,1);
end
Except x, all other variables are supplied. The problem is that when I set different problem.x0, for example, when problem.x0=ones(user_num,1);, the solution of [b_ul,fval] = fmincon(problem); is different from that when problem.x0=2*ones(user_num,1);. That is what I am confused about.
fmincon uses the following algorithms:
'interior-point' (default)
'trust-region-reflective'
'sqp' (Sequential Quadratic Programming)
'sqp-legacy'
'active-set'
These methods will converge to a local minimia but not necessarily a global minimum. Further minima may not be unique. The only way to guarantee a global minima is to search the whole solution space.
From your comment, there appears to be only a signal minima? (For example, a shifted parabola?) Then it should converge.
edit--
Even if your function appears convex, the constraints can lead to multiple local minima. Sometimes this is called a "loosely" convex function
I have to construct the following function in MATLAB and am having trouble.
Consider the function s(t) defined for t in [0,4) by
{ sin(pi*t/2) , for t in [0,1)
s(t) = { -(t-2)^3 , for t in [1,3)*
{ sin(pi*t/2) , for t in [3,4)
(i) Generate a column vector s consisting of 512 uniform
samples of this function over the interval [0,4). (This
is best done by concatenating three vectors.)
I know it has to be something of the form.
N = 512;
s = sin(5*t/N).' ;
But I need s to be the piecewise function, can someone provide assistance with this?
If I understand correctly, you're trying to create 3 vectors which calculate the specific function outputs for all t, then take slices of each and concatenate them depending on the actual value of t. This is inefficient as you're initialising 3 times as many vectors as you actually want (memory), and also making 3 times as many calculations (CPU), most of which will just be thrown away. To top it off, it'll be a bit tricky to use concatenate if your t is ever not as you expect (i.e. monotonically increasing). It might be an unlikely situation, but better to be general.
Here are two alternatives, the first is imho the nice Matlab way, the second is the more conventional way (you might be more used to that if you're coming from C++ or something, I was for a long time).
function example()
t = linspace(0,4,513); % generate your time-trajectory
t = t(1:end-1); % exclude final value which is 4
tic
traj1 = myFunc(t);
toc
tic
traj2 = classicStyle(t);
toc
end
function trajectory = myFunc(t)
trajectory = zeros(size(t)); % since you know the size of your output, generate it at the beginning. More efficient than dynamically growing this.
% you could put an assert for t>0 and t<3, otherwise you could end up with 0s wherever t is outside your expected range
% find the indices for each piecewise segment you care about
idx1 = find(t<1);
idx2 = find(t>=1 & t<3);
idx3 = find(t>=3 & t<4);
% now calculate each entry apprioriately
trajectory(idx1) = sin(pi.*t(idx1)./2);
trajectory(idx2) = -(t(idx2)-2).^3;
trajectory(idx3) = sin(pi.*t(idx3)./2);
end
function trajectory = classicStyle(t)
trajectory = zeros(size(t));
% conventional way: loop over each t, and differentiate with if-else
% works, but a lot more code and ugly
for i=1:numel(t)
if t(i)<1
trajectory(i) = sin(pi*t(i)/2);
elseif t(i)>=1 & t(i)<3
trajectory(i) = -(t(i)-2)^3;
elseif t(i)>=3 & t(i)<4
trajectory(i) = sin(pi*t(i)/2);
else
error('t is beyond bounds!')
end
end
end
Note that when I tried it, the 'conventional way' is sometimes faster for the sampling size you're working on, although the first way (myFunc) is definitely faster as you scale up really a lot. In anycase I recommend the first approach, as it is much easier to read.
I have a model with linear constraints and a nonlinear objective function, and I'm trying to use "fmincon" toolbox of MATLAB to solve it. Actually, the Aineq is a 24*13 matrix, and the Aeq is a 24*13 matrix as well. But when I insert this command:
>> [x , lambda] = fmincon(#MP_ObjF,Aineq,bineq,Aeq,beq);
I encounter this error:
Warning: Trust-region-reflective method does not currently solve this type of
problem, using active-set (line search) instead.
In fmincon at 439??? Error using ==> fmincon at 692
Aeq must have 312 column(s).
What is probably wrong with it? Why should Aeq have 312 columns?!? I will appreciate any help. Thanks.
If you look at the documentation for fmincon (doc fmincon ) you'll see an input called opt.In this you can set the algorithm used by matlab to solve your minimization problem. If you run
Opt=optimset('fmincon');
Then you can modify the algorithm option using
Opt.algorithm="active-set";
Just send Opt to fmincon and then matlab wont have this problem anymore. Take a look inside Opt and you'll find a ton of options you can change to modify the optimization routine.
As for the number of columns. If you're using linear constraints then you input argument for MPobjF must be a column vector with n rows and 1 column. Then A must be m X n. Where M is the number of constraints and n is the number of variables. This is so that matrix multiplication is well defined.
I'm sorry if my first answer was ambiguous. Maybe it will help if I do an example, as I saw several suspicious things in your comments. Lets say we want to minimize x^2 + y^2 + (z-1)^2 subject to x + y + z = 1, 2x + 3y - 4z <= 5, x,y,z>=-5. The solution is obviously (0,0,1)...
We first have to make our objective function,
fun = #(vec) vec[1]^2 + vec[2]^2 + (vec[3]-1)^2;
For fmincon to work, there can only be one input to the function, but that input can be a vector. So here x = vec[1] and so on...I think your comments are indicating that your objective function has multiple inputs. If you need to pass some parameters that aren't being optimized there is documentation for this on Matlab's site (http://www.mathworks.com/help/optim/ug/passing-extra-parameters.html)
Then we can set the optimization settings
opt = optimset('fmincon');
opt.algorithm = 'active-set';
You may also have to modify the large-scale setting for the algorithm warning to go away, I can't remember...
Then we can set
Aeq = [1,1,1]; % equality constraint, if you had another eq constraint, it would be another row to Aeq
beq = 1; % equality constraint
A = [2,3,-4]; % inequality
b = 5; % inequality
lb = [-5;-5;-5]; % lower bound
x0 = [0.5;0.5;0]; % initial feasible guess, needs to be a column vector
[x,fval] = fmincon(fun,x0,A,b,Aeq,beq,lb,[],[],opt);
Then hopefully this finds x = [0;0;1]