I am trying to convert my quadprog linear quadratic problem over to fmincon so that later I can add nonlinear constraints. I am having difficulty when I compare my solutions using the two methods (for the same problem). The odd thing is that I get very different cost output when I get almost the same x values. Below is a simplified case of my code without constraints.
Here, my objective function is
%objective function
% cost = a + b*x(1) + c*x(1)^2 + d + e*x(2) + e*x(2)^2
param = [1;2;3;4;5;6];
H = [2*param(3) 0; 0 2*param(6)];
f = [param(2); param(5)];
x0 = [0,0];
[x1,fval1] = quadprog(H,f);
[x2,fval2] = fmincon(#(x) funclinear(x,param), x0);
fval1
fval2
%% defining cost objective function
function cost = funclinear(x, param);
cost=(param(1) + param(2)*x(1) + param(3)*(x(1))^2+ param(4) +param(5)*x(2)+param(6)*(x(2))^2);
end
My resulting x1 and x2 are
x1 =[-3.333333333305555e-01;-4.166666666649305e-01];
x2 =[-3.333333299126037e-01;-4.166666593362859e-01];
which makes sense that they are slightly different since they are different solvers.
However my optimized costs are
fval1 =-1.375000000000000e+00;
fval2 =3.625000000000001e+00;
Does this mean that my objective function is different than my H and f? Any help would be appreciated.
In the quadprog formulation, the constant terms a and d are not considered.
param(1)+param(4) = 1 + 4 = 5
The difference of your results is also 5.
Related
Finding m and c for an equation y = mx + c, with the help of math and plots.
y is data_model_1, x is time.
Avoid other MATLAB functions like fitlm as it defeats the purpose.
I am having trouble finding the constants m and c. I am trying to find both m and c by limiting them to a range (based on smart guess) and I need to deduce the m and c values based on the mean error range. The point where mean error range is closest to 0 should be my m and c values.
load(file)
figure
plot(time,data_model_1,'bo')
hold on
for a = 0.11:0.01:0.13
c = -13:0.1:-10
data_a = a * time + c ;
plot(time,data_a,'r');
end
figure
hold on
for a = 0.11:0.01:0.13
c = -13:0.1:-10
data_a = a * time + c ;
mean_range = mean(abs(data_a - data_model_1));
plot(a,mean_range,'b.')
end
A quick & dirty approach
You can quickly get m and c using fminsearch(). In the first example below, the error function is the sum of squared error (SSE). The second example uses the sum of absolute error. The key here is ensuring the error function is convex.
Note that c = Beta(1) and m = Beta(2).
Reproducible example (MATLAB code[1]):
% Generate some example data
N = 50;
X = 2 + 13*random(makedist('Beta',.7,.8),N,1);
Y = 5 + 1.5.*X + randn(N,1);
% Example 1
SSEh =#(Beta) sum((Y - (Beta(1) + (Beta(2).*X))).^2);
Beta0 = [0.5 0.5]; % Initial Guess
[Beta SSE] = fminsearch(SSEh,Beta0)
% Example 2
SAEh =#(Beta) sum(abs(Y-(Beta(1) + Beta(2).*X)));
[Beta SumAbsErr] = fminsearch(SAEh,Beta0)
This is a quick & dirty approach that can work for many applications.
#Wolfie's comment directs you to the analytical approach to solve a system of linear equations with the \ operator or mldivide(). This is the more correct approach (though it will get a similar answer). One caveat is this approach gets the SSE answer.
[1] Tested with MATLAB R2018a
I'm beginner in optimization and welcome any guide in this field.
I have 15 matrices (i.e., Di of size (n*m)) and want to find best weights (i.e., wi) for weighted averaging them and make a better matrix that is more similar to one given matrix (i.e., Dt).
In fact my objective function is like it:
min [norm2(sum(wi * Di) - Dt) + norm2(W)]
for i=1 ... 15 s.t. sum(wi) = 1 , wi >= 0
How can I optimize this function in Matlab?
You are describing a simple Quadratic programming, that can be easily optimized using Matlab's quadprog.
Here how it goes:
You objective function is [norm2(sum(wi * Di) - Dt) + norm2(W)] subject to some linear constraints on w. Let's re-write it using some simplified notations. Let w be a 15-by-1 vector of unknowns. Let D be an n*m-by-15 matrix (each column is one of the Di matrices you have - written as a single column), and Dt is a n*m-by-1 vector (same as your Dt but written as a column vector). Now some linear algebra (using the fact that ||x||^2 = x'*x and that argmin x is equivalent to argmin x^2)
[norm2(sum(wi * Di) - Dt)^2 + norm2(W)^2] =
(D*w-Dt)'*(D*w-Dt) + w'*w =
w'D'Dw - 2w'D'Dt + Dt'Dt + w'w =
w'(D'D+I)w - 2w'D'Dt + Dt'Dt
The last term Dt'Dt is constant w.r.t w and therefore can be discarded during minimization, leaving you with
H = 2*(D'*D+eye(15));
f = -2*Dt'*D;
As for the constraint sum(w)=1, this can easily be defined by
Aeq = ones(1,15);
beq = 1;
And a lower bound lb = zeros(15,1) will ensure that all w_i>=0.
And the quadratic optimization:
w = quadprog( H, f, [], [], Aeq, beq, lb );
Should do the trick for you!
We have an equation similar to the Fredholm integral equation of second kind.
To solve this equation we have been given an iterative solution that is guaranteed to converge for our specific equation. Now our only problem consists in implementing this iterative prodedure in MATLAB.
For now, the problematic part of our code looks like this:
function delta = delta(x,a,P,H,E,c,c0,w)
delt = #(x)delta_a(x,a,P,H,E,c0,w);
for i=1:500
delt = #(x)delt(x) - 1/E.*integral(#(xi)((c(1)-c(2)*delt(xi))*ms(xi,x,a,P,H,w)),0,a-0.001);
end
delta=delt;
end
delta_a is a function of x, and represent the initial value of the iteration. ms is a function of x and xi.
As you might see we want delt to depend on both x (before the integral) and xi (inside of the integral) in the iteration. Unfortunately this way of writing the code (with the function handle) does not give us a numerical value, as we wish. We can't either write delt as two different functions, one of x and one of xi, since xi is not defined (until integral defines it). So, how can we make sure that delt depends on xi inside of the integral, and still get a numerical value out of the iteration?
Do any of you have any suggestions to how we might solve this?
Using numerical integration
Explanation of the input parameters: x is a vector of numerical values, all the rest are constants. A problem with my code is that the input parameter x is not being used (I guess this means that x is being treated as a symbol).
It looks like you can do a nesting of anonymous functions in MATLAB:
f =
#(x)2*x
>> ff = #(x) f(f(x))
ff =
#(x)f(f(x))
>> ff(2)
ans =
8
>> f = ff;
>> f(2)
ans =
8
Also it is possible to rebind the pointers to the functions.
Thus, you can set up your iteration like
delta_old = #(x) delta_a(x)
for i=1:500
delta_new = #(x) delta_old(x) - integral(#(xi),delta_old(xi))
delta_old = delta_new
end
plus the inclusion of your parameters...
You may want to consider to solve a discretized version of your problem.
Let K be the matrix which discretizes your Fredholm kernel k(t,s), e.g.
K(i,j) = int_a^b K(x_i, s) l_j(s) ds
where l_j(s) is, for instance, the j-th lagrange interpolant associated to the interpolation nodes (x_i) = x_1,x_2,...,x_n.
Then, solving your Picard iterations is as simple as doing
phi_n+1 = f + K*phi_n
i.e.
for i = 1:N
phi = f + K*phi
end
where phi_n and f are the nodal values of phi and f on the (x_i).
I have all the data and an ODE system of three equations which has 9 unknown coefficients (a1, a2,..., a9).
dS/dt = a1*S+a2*D+a3*F
dD/dt = a4*S+a5*D+a6*F
dF/dt = a7*S+a8*D+a9*F
t = [1 2 3 4 5]
S = [17710 18445 20298 22369 24221]
D = [1357.33 1431.92 1448.94 1388.33 1468.95]
F = [104188 104792 112097 123492 140051]
How to find these coefficients (a1,..., a9) of an ODE using Matlab?
I can't spend too much time on this, but basically you need to use math to reduce the equation to something more meaningful:
your equation is of the order
dx/dt = A*x
ergo the solution is
x(t-t0) = exp(A*(t-t0)) * x(t0)
Thus
exp(A*(t-t0)) = x(t-t0) * Pseudo(x(t0))
Pseudo is the Moore-Penrose Pseudo-Inverse.
EDIT: Had a second look at my solution, and I didn't calculate the pseudo-inverse properly.
Basically, Pseudo(x(t0)) = x(t0)'*inv(x(t0)*x(t0)'), as x(t0) * Pseudo(x(t0)) equals the identity matrix
Now what you need to do is assume each time step (1 to 2, 2 to 3, 3 to 4) is an experiment (therefore t-t0=1), so the solution would be to:
1- Build your pseudo inverse:
xt = [S;D;F];
xt0 = xt(:,1:4);
xInv = xt0'*inv(xt0*xt0');
2- Get exponential result
xt1 = xt(:,2:5);
expA = xt1 * xInv;
3- Get the logarithm of the matrix:
A = logm(expA);
And since t-t0= 1, A is our solution.
And a simple proof to check
[t, y] = ode45(#(t,x) A*x,[1 5], xt(1:3,1));
plot (t,y,1:5, xt,'x')
You have a linear, coupled system of ordinary differential equations,
y' = Ay with y = [S(t); D(t); F(t)]
and you're trying to solve the inverse problem,
A = unknown
Interesting!
First line of attack
For given A, it is possible to solve such systems analytically (read the wiki for example).
The general solution for 3x3 design matrices A take the form
[S(t) D(t) T(t)].' = c1*V1*exp(r1*t) + c2*V2*exp(r2*t) + c3*V3*exp(r3*t)
with V and r the eigenvectors and eigenvalues of A, respectively, and c scalars that are usually determined by the problem's initial values.
Therefore, there would seem to be two steps to solve this problem:
Find vectors c*V and scalars r that best-fit your data
reconstruct A from the eigenvalues and eigenvectors.
However, going down this road is treaturous. You'd have to solve the non-linear least-squares problem for the sum-of-exponentials equation you have (using lsqcurvefit, for example). That would give you vectors c*V and scalars r. You'd then have to unravel the constants c somehow, and reconstruct the matrix A with V and r.
So, you'd have to solve for c (3 values), V (9 values), and r (3 values) to build the 3x3 matrix A (9 values) -- that seems too complicated to me.
Simpler method
There is a simpler way; use brute-force:
function test
% find
[A, fval] = fminsearch(#objFcn, 10*randn(3))
end
function objVal = objFcn(A)
% time span to be integrated over
tspan = [1 2 3 4 5];
% your desired data
S = [17710 18445 20298 22369 24221 ];
D = [1357.33 1431.92 1448.94 1388.33 1468.95 ];
F = [104188 104792 112097 123492 140051 ];
y_desired = [S; D; F].';
% solve the ODE
y0 = y_desired(1,:);
[~,y_real] = ode45(#(~,y) A*y, tspan, y0);
% objective function value: sum of squared quotients
objVal = sum((1 - y_real(:)./y_desired(:)).^2);
end
So far so good.
However, I tried both the complicated way and the brute-force approach above, but I found it very difficult to get the squared error anywhere near satisfyingly small.
The best solution I could find, after numerous attempts:
A =
1.216731997197118e+000 2.298119167536851e-001 -2.050312097914556e-001
-1.357306715497143e-001 -1.395572220988427e-001 2.607184719979916e-002
5.837808840775175e+000 -2.885686207763313e+001 -6.048741083713445e-001
fval =
3.868360951628554e-004
Which isn't bad at all :) But I would've liked a solution that was less difficult to find...
I have a summation objective function (non-linear portfolio optimization) which looks like:
minimize w(i)*w(j)*cv(i,j) for i = 1 to 10 and j = 1 to 10
w is the decision vector
cv is a known 10 by 10 matrix
I have the formulation done for the constraints (a separate .m file for the project constraints) and for the execution of the fmincon (a separate .m file for the lower/upper bounds, initial value, and calling fmincon with the arguments).
I just can't figure out how to do the objective function. I'm used to linear programming in GLPK rather than matlab so I'm not doing so good.
I'm currently got:
ObjectiveFunction.m
function f = obj(w)
cv = [all the constants are in here]
i = 1;
j = 1;
n = 10;
var = 0;
while i <= n
while j<=n
var = var + abs(w(i)*w(j)*cv(i, j));
j = j + 1;
end
i = i + 1;
end
f = var
...but this isn't working.
Any help would be appreciated! Thanks in advance :)
So this is from a class I took a few years ago, but it addresses a very similar problem to your own with respect to use of fminsearch to optimize some values. The problem is essentially that you have a t, y, and you want a continuous exponential function to represent t, y in terms of c1*t.*exp(c2*t). The textbook I lifted the values from is called Numerical Analysis by Timothy Sauer. Unfortunately, I don’t remember the exact problem or chapter, but it’s in there somewhere.
c1 and c2 are found recursively by fminsearch minimizing a residual y - ((c1) * t .* exp((c2) * t)). Try copying and running my code below to get a feel for things:
%% Main code
clear all;
t = [1,2,3,4,5,6,7,8];
y = [8,12.3,15.5,16.8,17.1,15.8,15.2,14];
lambda0=[1 -.5];
lambda=fminunc(#expdecayfun,lambda0, ...
optimset('LargeScale','off','Display','iter','TolX',1.e-6),t,y);
c1=lambda(1);
c2=lambda(2);
fprintf('Using the BFGS method through fminunc, c1 = %e\n',c1);
fprintf('and c2 = %e. Since these values match textbook values for\n', c2);
fprintf('c1 and c2, I will stop here.\n');
%% Index of functions:
% expdecayfun
function res=expdecayfun(lambda,t,y) c1=lambda(1);
c2=lambda(2);
r=y-((c1)*t.*exp((c2)*t));
res=norm(r);
Hope this helps!