According to my previous question, I want to optimize an objective function using binary integer linear programming (all of variables are binary) as follows:
Minimize f = (c1*x1) + (c2*x2) + MAX((c3*x3),(c4*x4)) + (c5*x5)
Subject to: some equality and inequality constraints
For MAX operator, I used auxiliary variable x6 and added x6>=(c3*x3) and x6>=(c4*x4) constraints to the problem so the problem turn into:
Minimize f = (c1*x1) + (c2*x2) + x6 + (c5*x5), with added constraints.
I used CPLEX API for MATLAB to optimize the objective function.
Because all of variables are binary except one ((x6) which is continuous) and coefficients have double values, the problem turn into mixed integer linear programming so I used cplexmilp function with this configuration:
Variable types: ctype='BBBBBC' ( B:Binary, C:Continuous)
Lower Bounds: lb=[0 0 0 0 0 0]
Upper Bounds: ub=[0 0 0 0 0 inf]
Function Call:
[fval] = cplexmilp(f, Aineq, bineq, Aeq, beq,[],[],[],lb,ub,ctype)
but sometimes in the results I see x3 and x4 have continuous values(between 0 and 1) and x3+x4=1.
So my questions are:
Can any one tell me what's wrong with x3 and x4?
Is there a solution for not using auxiliary variable and solving this optimization problem with cplexbilp ?
Thanks in advance
[UPDATE]:
One parts of my code had logical errors which I fixed them, now in all cases which x3 and x4 are not binary we have x3*(1-x3)< 10^(-5), x4*(1-x4)< 10^(-5) and x3+x4=1, so #David Nehme were right (according to his useful comment), but my second question still remains here!
David's solution shows you why your formulation has become linearized but non-binary. You could also try printing out the problem in LP or MPS format to see all the resulting constraints.
You asked about a formulation that continues to be purely binary. Here's one way to do that:
Transforming this to a purely binary formulation
Here's a way to keep the problem with Max() also binary. It involves additional aux. variables, but it is relatively straight-forward once you apply the standard if-then IP tricks.
First, let's list out the four possible cases in a simple table, and see what values the max() term can take. These cases are mutually exclusive.
x3 | x4 | max (c3.x4, c4.x3)
-------------------------------
0 | 0 | 0
1 | 0 | c3
0 | 1 | c4
1 | 1 | max(c3, c4) - a constant
Now, let C34 be the max(c3, c4). Note that C34 is a number, not a variable in the problem. We need this for the new Objective function.
Introducing new binary variables
For each of the four cases above, let's introduce an auxiliary BINARY variable. For clarity, call them y0, y3, y4, y34.
Only one of the cases in the table above can hold, so we add:
y0 + y3 + y4 + y34 = 1
yi are BINARY
Now, all that remains is to add linking constraints that ensure:
If x3=0 AND x4=0 then y0=1
If x3=1 AND x4=0 then y3=1
If x3=0 AND x4=1 then y4=1
If x3=1 AND x4=1 then y34=1
We can ensure that by adding a pair of linear constraints for each of the conditions above.
2 y0 <= (1- x3) + (1 -x4)
(1-x3) + (1-x4) <= y0 + 1
2 y3 <= x3 + (1-x4)
x3+(1-x4) <= y3 + 1
2 y4 <= x4 + (1-x3)
x4+(1-x3) <= y4 + 1
2 y34 <= x3 + x4
x3+x4 <= y34 + 1
The new objective function now becomes:
Minimize f = (c1*x1) + (c2*x2) + (c5*x5) + 0*Y0 + C3*Y3 + C4*Y4 + C34*Y34
Notice that we don't have the Max() term anymore in the objective function. And all x and y variables are binary. All your original constraints plus the new ones above (8+1 = 9 of them) should be included. Once you do that, you can use cplexbilp because it is a pure BILP problem.
Hope that helps.
In your case, the auxiliary variable x6 is needed because the "MAX" function is not linear (it has a discontinuous gradient where c3*x3 == c4*x4). By adding the additional variable and constraints you are creating a linear version of the problem with a solution that is equivalent to your original nonlinear problem. The trade-off is that if c3 or c4 have a value other than 0 or 1, then your problem is not a pure binary problem. That is a very good trade-off, especially if you are using a good MIP solver.
Related
When I run this code I get empty strings for when setting a range for vpasolve, when I do not set the range I only get one solution, even with random on.
The range is set so that it does include the one solution matlab gives me, x=2, y=0 and b=1.5.
for the range I've tried -Inf Inf and NaN NaN, to have it try all numbers.
So please do not give me the answer of saying that my system has no solution, it clearly does.
It also won't solve it symbolicly (same issue with solve), while I can give 1 possible solution.
0.5*(x+y)=1, b*(x+y)=3 ---> x+y=2 and b=1.5
So something else must be wrong, I would appreciate it if you let me know what I'm doing wrong here.
clear all; %just to be safe
syms x y b
a=0.5;
somevalue=1;
someothervalue=3;
eq1= a*x+a*y == somevalue; %this is your first equation
eq2= b*x+b*y == someothervalue; %this is your 2nd equation
eqs=[eq1,eq2]; %use this for vpasolve and set range in range
vars=[x,y,b]; %these are the variable you want to solve for
range = [-1 3; -2 5; -Inf Inf]; %NaN means you set no range
%you can use solve or vpasolve, second one being numeric, which is the one you'll probably want
sol=zeros(5,3);
for i = 1:5
temp = vpasolve(eqs, vars, range, 'random', true);
temp = solve(eqs2, vars);
sol(i,1) = temp.x;
sol(i,2) = temp.y;
sol(i,3) = temp.b;
end
sol
temp1.x
temp1.y
temp1.b
Now I have another clear problem/error when using the solve option, obviously the answer here should be 9:
syms x
eq12 = -3 == sqrt(x);
solve(eq12)
ans =
Empty sym: 0-by-1
as well as:
syms x
eq12 = -3 == (x)^(1/2);
solve(eq12)
ans =
Empty sym: 0-by-1
Obviously the answer here would be 9, so what do I change to let matlab solve this and before you tell me to change the equation, this is the equation I would have to change by hand and not just once but a 100 times or so.
More specifically I need to solve something way more complicated which has a solution; negative and positive, the negative value is physically unrealistic but it's the only Matlab can give me.
93659574124777211691008 H
H + ---------------------------------------------
1208925819614629174706176 H + 762832192176831
1
== -----------------
100000000000000 H
29250045579840375 #1
- --------------------------------------------------------------------
H (922337203685477580800 H + 927343445063259249) 4611686018427387904
2
12879770070323045125 #1
+ ----------------------------------------------------------------------
2
55340232221128654848 H (922337203685477580800 H + 927343445063259249)
where
/ 2
| 181939 H 852912375078609598437 H
#1 == 9223372036854775808 H - sqrt| --------- + -----------------------
\ 5539 25544128856069301600256
\
39917248404619332215368770561441 |
+ -------------------------------------- | 9223372036854775808 + 6318009845245521
85070591730234615865843651857942052864 /
The trouble maker is the minus in front of the sqrt I assume
So I want to determine the alpha, beta and gamma values of my SVM classifier, I trained my SVM classifier and determined the alpha and support vectors. My support vectors have 3 variables per row, below are just ten rows from my support vector:
0.0904235536887480 -0.269325475875919 -0.678528701392414
-0.321039098061280 -0.507618180664821 -1.42365662798284
-0.0737761304021185 -0.269641311369441 -0.647521877863172
0.00105779420640393 -0.311226557946309 -0.667506146498475
0.0913098589312967 -0.289462325547514 -0.391261050348894
0.00622693949845773 -0.166248587146820 -0.149546793127464
-0.292302915842567 -0.564676268888150 -1.60153093563523
0.112997393643248 -0.310512134534035 -0.725281274142312
-0.135361511770186 -0.456321702624641 -1.26973221898260
-0.173160731078767 -0.434439033384469 -1.22687774941370
and similarly below are just ten rows from my alpha:
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
So what I basically doing is to determine the coefficients like so:
c = (-1)*[0.0904*x1 + (-0.2693)*x2 + (-0.6785)*x3] + (-1)*[(-0.321)*x1 + (-0.5076)*x2 + (-1.4236)*x3].....
and so on until the size of my alpha.
So I coded the following in Matlab
syms x1, syms x2, syms x3;
alpha = SVMStruct.Alpha;
svm_vec = SVMStruct.SupportVectors;
for i = 1:size(alpha,1)
c(i) = alpha(i)*(svm_vec(i,1)*x1 + svm_vec(i,2)*x2 + svm_vec(i,3)*x3);
end
sum_it = sum(c);
But this created a very strange output:
(107845064549358722206080751595348329973204613074833920445585562521882937008164658045489239834546021458299139*x1)/50534761550197893278639420198779799540396107395587434771118149413836407509953624874438129483687080755200 + (95720990302914087945142311872326568914380675701489099929103269189530321664249312169660240242394455632803627*x2)/134178504805697854567421908803656709124500009291732154392279224305703564767807900528680550698065697177600 + (90626366614084720573448362168042659133754200934323766866906741825007634289583081638045482944881264585156183*x3)/125521827076297992982426946945356276277758073208394596044390242092432367040852552107475353878835652198400
is this the expected output? Why am I getting it as fractions?
Looking by the output, did I implement my equation correctly?
Your inputs to your calculation are symbolic variables, so any calculations using those variables are going to yield an exact value that is also a symbolic variable. If you want the approximate floating point representation of the result, you'll want to cast the sum as a double
double(sum_it)
In order to do this though, you'll need to specify actual values for x1, x2, and x3
double(subs(sum(c), [x1, x2, x3], [1, 2, 3]))
That being said, I think what you actually want is to solve the system of equations. You can use the \ operator to do this
variables = svm_vec \ alpha;
I'm trying to solve a ILP problem using binary variables in MATLAB (bintprog). I need to define a constraint similar as below :
I want to sum binary variables x<sub>i, j</sub> until one of them equals to zero and after that stop this summation. How can I define this constraint in ILP?
The way I read this (somewhat difficult as there is little explanation; I think the question could have been phrased more clearly), this is essentially counting the first x(k)'s that are 1. Here I assume x(i,j) is mapped to x(k) so we have a better defined ordering. The counting can be done using a new binary variable y(k):
y(k) = x(k)*y(k-1) if k>1
y(k) = x(k) if k=1
y(k) in {0,1}
For example:
x = [1 1 1 0 1 0]
y = [1 1 1 0 0 0]
The nonlinear expression x(k)*y(k-1) can easily be linearized (it is a multiplication of two binary variables; see here; note that with this formulation we can even relax y(k) to continuous between 0 and 1). An optimized version can look like:
y(k) >= x(k)+y(k-1)-1 if k>1
y(k) = x(k) if k=1
0 <= y(k) <= 1
Now you just can do
sum(k, y(k)) <= c
This formulation would also allow to say: number of leading 1s should be between c1 and c2.
We can also just write immediately:
x(1) + ... + x(c+1) <= c
which also forbids more than c leading 1s and is even simpler.
A different problem we see more often (e.g. in generator scheduling) is the condition that we cannot have more than c consecutive x(k)'s turned on (i.e. the generator can not be on for more than c consecutive periods: after c periods on, the generator must be turned off for at least 1 period). This means we don't allow sequences of more than c 1's.
This is quite a non-typical MIP-constraint in my opinion. Here is my attempt in formulating this:
You have to think about:
the necessity of handling Case B (dependent on your objective and other constraints) and the downfalls of the bigM-approach nice read
how to implement the product of two binary-variables (which is easy; see here)
EDIT
At the cost of the double amount of binary-auxiliary variables, it is also possible to formulate this without bigM-constants.
Sketch:
Instead of using a one-sided implication to set the indicators, use equality == both-implications -> no aux-var is free; all are 0 or 1 depending on x(z) only
Now build a second layer of binary-aux variables (b_i) with the following idea: b_i >= smallConst * b_i-1 + smallConst * a_i (smallConst in (0, 0.5]; should not be close 0 )
Use b-vector for your final sum-constraint (like describes above; but not using a-vector anymore!)
I have a linear equation:
vt = v1*x1 + v2*x2 + v3*x3
vt, v1, v2, v3 are scalars with values between 0 and 1. What is the best way to generate one set (any set will be fine) of x1, x2 and x3 that satisfy the equation above. and also satisfy
x1>0
x2>0
x3>0
I have couple thousand sets of vt,v1,v2 and v3, therefore I need to be able to generate x1, x2 and x3 programmatically.
There are two ways you could approach this:
From the method that you have devised in your post. Randomly generate x1 and x2 and ensure that vt < v1*x1 + v2*x2, then go ahead and solve for x3.
Formulate this into linear program. A linear program is basically solving a system of equations that are subject to inequality or equality constraints. In other words:
As such, we can translate your problem to be of a linear programming problem. The "maximize" statement is what is known as the objective function - the overall goal of what you are trying to accomplish. In linear programming problems, we are trying to minimize or maximize this objective. To do this, we must satisfy the inequalities seen in the subject to condition. Usually, this program is represented in canonical form, and so the constraints on each variable should be positive.
The maximize condition can be arbitrary as you don't care about the objective. You just care about any solution. This whole paradigm can be achieved by linprog in MATLAB. What you should be careful with is how linprog is specified. In fact, the objective is minimized instead of maximized. The conditions, however, are the same with the exception of ensuring that all of the variables are positive. We will have to code that in ourselves.
In terms of the arbitrary objective, we can simply do x1 + x2 + x3. As such, c = [1 1 1]. Our equality constraint is: v1*x1 + v2*x2 + v3*x3 = vt. We also must make sure that x is positive. In order to code this in, what we can do is choose a small constant so that all values of x are greater than this value. Right now, linprog does not support strict inequalities (i.e. x > 0) and so we have to circumvent this by doing this trick. Also, to ensure that the values are positive, linprog assumes that the Ax <= b. Therefore, a common trick that is used is to negate the inequality of x >= 0, and so this is equivalent to -x <= 0. To ensure the values are non-zero, we would actually do: -x <= -eps, where eps is a small constant. However, when I was doing experiments, by doing it this way, two of the variables end up to be the same solution. As such, what I would recommend we do is to generate good solutions that are random each time, let's draw b to be from a uniform random distribution as you said. This will then give us a starting point every time we want to solve this problem.
Therefore, our inequalities are:
-x1 <= -rand1
-x2 <= -rand2
-x3 <= -rand3
rand1, rand2, rand3 are three randomly generated numbers that are between 0 and 1. In matrix form, this is:
[-1 0 0][x1] [-rand1]
[0 -1 0][x2] <= [-rand2]
[0 0 -1][x3] [-rand3]
Finally, our equality constraint from before is:
[v1 v2 v3][x1] [vt]
[x2] =
[x3]
Now, to use linprog, you would do this:
X = linprog(c, A, b, Aeq, beq);
c is a coefficient array that is defined for the objective. In this case, it would be defined as [1 1 1], A and b is the matrix and column vector defined for the inequality constraints and Aeq and beq is the matrix and column vector defined for the equality constraints. X would thus give us the solution after linprog converges (i.e. x1, x2, x3). As such, you would do this:
A = -eye(3,3);
b = -rand(3,1);
Aeq = [v1 v2 v3];
beq = vt;
c = [1 1 1];
X = linprog(c, A, b, Aeq, beq);
As an example, suppose v1 = 0.33, v2 = 0.5, v3 = 0.2, and vt = 2.5. Therefore:
rng(123); %// Set seed for reproducibility
v1 = 0.33; v2 = 0.5; v3 = 0.2;
vt = 2.5;
A = -eye(3,3);
b = -rand(3,1);
Aeq = [v1 v2 v3];
beq = vt;
c = [1 1 1];
X = linprog(c, A, b, Aeq, beq);
I get:
X =
0.6964
4.4495
0.2268
To verify that this equals vt, we would do:
s = Aeq*X
s = 2.5000
The above simply does v1*x1 + v2*x2 + v3*x3. This is computed in a dot product form to make things easy as X is a column vector and v1, v2, v3 are already set in Aeq and is a row vector.
As such, either way is good, but at least with linprog, you don't have to keep looping until you get that condition to be satisfied!
Small Caveat
One small caveat that I forgot to mention in the above approach is that you need to make sure that vt >= v1*rand1 + v2*rand2 + v3*rand3 to ensure convergence. Since you said that v1,v2,v3 are bounded between 0 and 1, the worst case is when v1,v2,v3 are all equal to 1. As such, we really need to make sure that vt > rand1 + rand2 + rand3. If this is not the case, then simply take each value of rand1, rand2, rand3, and divide by (rand1 + rand2 + rand3) / vt. As such, this will ensure that the total summation will equal vt assuming that all of the weights are 1, and this will allow the linear program to converge properly.
If you don't, then the solution will not converge due to the inequality conditions placed in for b, and you won't get the right answer. Just some food for thought! As such, do this for b before you run linprog
if sum(-b) > vt
b = b ./ (sum(-b) / vt);
end
Good luck!
I am trying to solve a problem using linear optimization on matlab
sum=zeros(5,1);
for i=1:5
min 2x1+3x2-5x3
s.t.
x1 <= 4;
x2+3x3 <= 2;
sum(1:i) >= 3
sol=linprog(fill them)
sum(i) = sum(i)+sol(2)-sol(3)
end
How can I represent sum(i) in the matrices we use in linprog function?
The fact that x4 and x5 do not appear explicitly in the objective function does not necessarily mean they do not exist.
Think about this objective function
min 2*x1 + 3*x2 - 5*x3 + 0*x4 + 0*x5
Now you have a linear program with 5 variables.
Bottom line: just add two zeros to your objective function and you are done.