x + y ≤ 44,
2x + y ≤ 64,
9,000x + 5,000y ≤ 300,000
objective function: 30,000x + 20,000y
I would like to find the Optimal solution in Matlab
But there are error message Problem is unbounded.
Here is my code
A = [1 1;2 1;9 5];
b = [44 64 300];
f = [3 2];
x = linprog(f,A,b)
suppose ans: x=20,y=24
Reference
https://www.mathworks.com/help/optim/ug/linprog.html
It's not a direct answer to your question, but one reason you're having difficulties is that you're using Matlab. Other languages would offer more expressive and easier-to-understand ways of expressing your problem.
Python for instance, coupled with cvxpy gives us:
#!/usr/bin/env python3
import cvxpy as cp
x = cp.Variable()
y = cp.Variable()
constraints = [
x + y <= 44,
2 * x + y <= 64
]
objective = cp.Maximize(9000*x + 5000*y)
prob = cp.Problem(objective, constraints)
N
optimum_value = prob.solve()
print("optimum value", optimum_value)
print("x", x.value)
print("y", y.value)
The code is very readable and the model is easily extended even to non-linear regimes such as quadratic programs and second-order cones.
This returns
optimum value 300000.00000076543
x 20.00000000075882
y 23.999999998787203
I am testing MATLAB capabilities in solving equations for a project that I intend to do, so I gave it a test run with something simple, but the results that it gives me are incorrect. I tried to solve two non-linear equations with two unknowns, one of the solutions is correct the other is not.
syms theta d x y
eq1 = d * cos(theta) == x;
eq2 = d * sin(theta) == y;
sol = solve(eq1, eq2, theta, d)
sol.theta
sol.d
The solutions for d are correct, but for theta I get:
-2*atan((x - (x^2 + y^2)^(1/2))/y)
-2*atan((x + (x^2 + y^2)^(1/2))/y)
And the correct answer for theta is simply atan(y/x)
Then when I evaluate these solutions with x = 1, y = 0, I get:
eval(sol.d)
eval(sol.theta)
d = 1, -1
theta = NaN, -3.1416
Solutions for d are correct, but theta in that scenario should be 0.
What am I doing wrong?
EDIT: solving it by hand it looks like this: Divide the y equation by the x equation
y/x = (d * sin(theta)) / (d * cos(theta))
y/x = sin(theta)/cos(theta)
y/x = tan(theta)
theta = atan(y/x)
Even if matlab solves it in some other way and gets a different expression, it should still yield the same final result when I use numbers and it PARTIALLY does.
For x = 1 and y = 0, theta should be 0, => this doesnt work, it gives NaN (explanation bellow)
for x = 1 and y = 1, theta should be 45 degrees => this works
for x = 0 and y = 1 theta should be 90 degrees => this works
And I just checked it again with the 45 and 90 degree values for x and y and it works, but for x = 1 and y = 0 it still gives NaN as one of the answers and that is because it gets a 0/0 from the way it is expressing it
-2*atan((x - (x^2 + y^2)^(1/2))/y)
-2*(1 - (1^2 + 0^2))^(1/2)/0
-2*(1 - 1)^(1/2)/0
0/0
but if its in the form of atan(y/x) the result is
theta = atan(0/1)
theta = atan(0)
theta = 0
Did you mean to solve this:
syms a b theta d real
eq1 = a==d * cos(theta) ;
eq2 = b==d * sin(theta) ;
[sol] = solve([eq1 eq2],[d theta] ,'IgnoreAnalyticConstraints', true,'Real',true,'ReturnConditions',true);
When solving the equations with symbolic x and y, the solver will find a solution with a certain condition, which can be obtained using the argument 'ReturnCondition':
syms x y theta d real
eq1 = d*cos(theta) == x;
eq2 = d*sin(theta) == y;
sol = solve([eq1; eq2],[d theta],'ReturnConditions',true);
This gives the following result for sol
>> sol.d
(x^2 + y^2)^(1/2)
-(x^2 + y^2)^(1/2)
>> sol.theta
2*pi*k - 2*atan((x - (x^2 + y^2)^(1/2))/y)
2*pi*k - 2*atan((x + (x^2 + y^2)^(1/2))/y)
>> sol.parameters
k
>> sol.conditions
y ~= 0 & in(k, 'integer')
y ~= 0 & in(k, 'integer')
As you can see, y = 0 does not fulfill this general solution given by the solver, resulting in your problem for y = 0. You can find solutions for y = 0 by either making y numeric instead of symbolic, or by adding an assumption:
syms x y theta d real
assume(y==0)
sol = solve([eq1; eq2],[d theta],'ReturnConditions',true);
I guess its easier to just set y=0 numeric, for this one condition, since there are already 4 possible solutions and conditions for the three lines above.
I want to perform some symbolic computation with the discrete-time unit step function, which I cannot seem to find a built-in definition of. I've tried heaviside(n + 0.5) and also sym('piecewise([n < 0 , 0], [n >= 0, 1])'). Here is some sample code:
syms n k
unitStep = #(n) subs(sym('piecewise([k < 0 , 0], [k >= 0, 1])'), n);
x1 = #(n) unitStep(n);
h1 = #(n) 2 .^ (-n) .* unitStep(n);
y1 = #(n) symsum(x1(k) .* h1(n - k), k, -inf, inf);
simplify(y1(n))
Which gives the output
piecewise([in(n, 'real'), symsum(piecewise([n < k, 0], [k <= n, 2^(k - n)]), k, 0, Inf)])
(Not at all helpful) I cannot seem to get satisfactory results whatever I do. Anyone have tips for evaluating symbolic sums with the unit step function in them? I do not want to use heaviside(n) because heaviside(0) evaluates to 0.5.
I have a portfolio of weights I am using quadprog in matlab.
I have all the inputs for the quadprog optimizer. I am just having some trouble formulating
the constraints
I would like my constraints to have a lower bound of either 0 or 1%, is there a way to do that while maintainng my objective function
Thanks!
I am not sure if I understand your question well.
If your weights are already defined in terms of percentage, it is directly the definition of quadprog:
x = quadprog(H, f, [], [], [], [], lb, [])
So H, e, and f should be provided by the matlab description of:
quadprog(H,f) - returns a vector x that minimizes 1/2 * x' * H * x + f' * x. H must be positive definite for the problem to have a finite minimum."
And lb is a vector of the constraints. For instance if x is a vector 3 x 1, then lb = [0.01; 0.01; 0.01] in the case of the desired percentage is 0.01 (1%)
On the other hand, lets assume the sum_{i=1}^{n} w_i is not equal to 1. Therefore, w_i is not defined in terms of percentage.
Therefore, the constraint that you need is p_i (percentage)= w_i / (sum w_i) >= 0.01 (in the case of the lower bound be 1%).
Note that the constraint in this case is
w_i >= 0.01 * (sum w_i)
Or
-0.01 * (sum_{j=1}^{i-1} w_j) + 0.99 * w_i - 0.01 * (sum_{j=i+1}^{n} w_j) >= 0
Or
0.01 * (sum_{j=1}^{i-1} w_j) - 0.99 w_i + 0.01 * (sum_{j=i+1}^{n} w_j) <= 0
Therefore, this is a constraint of the type Ax <= b.
So
A_ij = 0.01 when i is different from j
A_ij = -0.99 when i = j and b = zeros(n, 1)
In this case you are using
x = quadprog(H, f, A, b)
I hope I helped you!
Daniel
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);