Dears, I'm using CP-SAT in python.
I have two variables:
pippo = model.NewIntVar(-93372036854775808,9123372036854775807, 'pippo')
test = model.NewIntVar(-93372036854775808,9123372036854775807, 'test')
and the following decision variables:
for r in self.all_records:
memory = {}
for p in self.all_decisions:
# create bool decision variables
memory[p] = model.NewBoolVar('c' + str(r) + self.scenario.decisions[p].name)
self.decision_var[r] = memory
I have two constraints:
1) test==sum(self.scenario.dataset['is_fraud'][r]*self.decision_var[r][0] for r in self.all_records)
2) pippo == test
If I remove constraint 2) everything is ok, but with constraint 2 I get this error:
Possible integer overflow in constraint: linear {
vars: 126660
vars: 126661
coeffs: -1
coeffs: 1
domain: 0
domain: 0
}
and Status is MODEL_INVALID.
I really do not understand why. Could you help me please?
The solver pro-actively checks for possible overflow.
If the model has a potential integer overflow, it is deemed invalid.
You are seeing this message. Please reduce the bounds of the integer variables (int32 min/max is most likely enough).
Related
I am stuck with a cumulative constraint I seem not to use properly, I seek for help ! :)
I have tasks with precedence and, for some of them, required & forbidden resources.
I need to decide when to start a task & who to assign to it.
To do so, I'm using an array of decision variable resource_allocation:
array[Tasks, Resources] of var 0..1: resource_allocation; %Selection of resources per task.
To manage the required/forbidden resources, I used the following:
% Mandatory & Forbidden constraint allocation
constraint forall(t in Tasks, r in resource_required[t])(resource_allocation[t,r]=1);
constraint forall(t in Tasks, r in resource_forbidden[t])(resource_allocation[t,r]=0);
resource_required being set of int storing the resources number that are required/forbidden.
Each resource represents 1 worker, and each worker can only perform one task at a time, so I am trying to state that, for every resources, the cumsum of allocation can at max be 1.
It might be important to note that start is also a decision variable.
% Constraint allocated to only one task at a time
constraint forall(t in Resources)(
cumulative(start, duration, [resource_allocation[t, r] | t in Tasks], 1)
);
Doing so, I always end up with the following error
JC:70.12-82
in call 'cumulative'
cumulative:21-3.49-7
in binary '/\' operator expression
cumulative:25-3.49-7
in if-then-else expression
cumulative:26-5.48-9
in binary '/\' operator expression
cumulative:30-5.48-9
in if-then-else expression
cumulative:47.7-32
in call 'fzn_cumulative'
fzn_cumulative:4-9.20-17
in let expression
fzn_cumulative:8-13.20-17
in if-then-else expression
fzn_cumulative:10-17.19-17
in let expression
fzn_cumulative:12.21-74
in variable declaration for 'late'
in call 'max'
with i = <expression>
MiniZinc: evaluation error: arithmetic operation on infinite value
Process finished with non-zero exit code 1.
I need a little guidance, I looked in the source code of fzn_cumulative, but I don't get what is going on.
Thanks !
You might consider to limit the domains of your decisions variables.
int is unlimited (disregarding the limited number of bits per int) and may lead to overflow situations or complaints about infinite values.
include "globals.mzn";
set of int: Tasks = 1..3;
set of int: Resources = 1..3;
set of int: Durations = 1..10;
set of int: Times = 1..1000;
% Use this editor as a MiniZinc scratch book
array[Tasks, Resources] of var 0..1: resource_allocation; %Selection of resources per task.
array [Tasks] of var Times: start;
array [Tasks] of var Durations: duration;
array [Tasks] of set of Resources: resource_required = [{1,2},{2,3},{3}];
array [Tasks] of set of Resources: resource_forbidden = [{},{1},{1}];
% Mandatory & Forbidden constraint allocation
constraint forall(t in Tasks, r in resource_required[t])(resource_allocation[t,r]=1);
constraint forall(t in Tasks, r in resource_forbidden[t])(resource_allocation[t,r]=0);
% Constraint allocated to only one task at a time
% Changed "t in Resources" to "r in Resources"
constraint forall(r in Resources)(
cumulative(start, duration, [resource_allocation[t, r] | t in Tasks], 1)
);
A little bit of background. I'm trying to make a model for clustering a Design Structure Matrix(DSM). I made a draft model and have a couple of questions. Most of them are not directly related to DSM per se.
include "globals.mzn";
int: dsmSize = 7;
int: maxClusterSize = 7;
int: maxClusters = 4;
int: powcc = 2;
enum dsmElements = {A, B, C, D, E, F,G};
array[dsmElements, dsmElements] of int: dsm =
[|1,1,0,0,1,1,0
|0,1,0,1,0,0,1
|0,1,1,1,0,0,1
|0,1,1,1,1,0,1
|0,0,0,1,1,1,0
|1,0,0,0,1,1,0
|0,1,1,1,0,0,1|];
array[1..maxClusters] of var set of dsmElements: clusters;
array[1..maxClusters] of var int: clusterCard;
constraint forall(i in 1..maxClusters)(
clusterCard[i] = pow(card(clusters[i]), powcc)
);
% #1
% constraint forall(i, j in clusters where i != j)(card(i intersect j) == 0);
% #2
constraint forall(i, j in 1..maxClusters where i != j)(
card(clusters[i] intersect clusters[j]) == 0
);
% #3
% constraint all_different([i | i in clusters]);
constraint (clusters[1] union clusters[2] union clusters[3] union clusters[4]) = dsmElements;
var int: intraCost = sum(i in 1..maxClusters, j, k in clusters[i] where k != j)(
(dsm[j,k] + dsm[k,j]) * clusterCard[i]
) ;
var int: extraCost = sum(el in dsmElements,
c in clusters where card(c intersect {el}) = 0,
k,j in c)(
(dsm[j,k] + dsm[k,j]) * pow(card(dsmElements), powcc)
);
var int: TCC = trace("\(intraCost), \(extraCost)\n", intraCost+extraCost);
solve maximize TCC;
Question 1
I was under the impression, that constraints #1 and #2 are the same. However, seems like they are not. The question here is why? What is the difference?
Question 2
How can I replace constraint #2 with all_different? Does it make sense?
Question 3
Why the trace("\(intraCost), \(extraCost)\n", intraCost+extraCost); shows nothing in the output? The output I see using gecode is:
Running dsm.mzn
intraCost, extraCost
clusters = array1d(1..4, [{A, B, C, D, E, F, G}, {}, {}, {}]);
clusterCard = array1d(1..4, [49, 0, 0, 0]);
----------
<sipped to save space>
----------
clusters = array1d(1..4, [{B, C, D, G}, {A, E, F}, {}, {}]);
clusterCard = array1d(1..4, [16, 9, 0, 0]);
----------
==========
Finished in 5s 419msec
Question 4
The expression constraint (clusters[1] union clusters[2] union clusters[3] union clusters[4]) = dsmElements;, here I wanted to say that the union of all clusters should match the set of all nodes. Unfortunately, I did not find a way to make this big union more dynamic, so for now I just manually provide all clusters. Is there a way to make this expression return union of all sets from the array of sets?
Question 5
Basically, if I understand it correctly, for example from here, the Intra-cluster cost is the sum of all interactions within a cluster multiplied by the size of the cluster in some power, basically the cardinality of the set of nodes, that represents the cluster.
The Extra-cluster cost is a sum of interactions between some random element that does not belong to a cluster and all elements of that cluster multiplied by the cardinality of the whole space of nodes to some power.
The main question here is are the intraCost and extraCost I the model correct (they seem to be but still), and is there a better way to express these sums?
Thanks!
(Perhaps you would get more answers if you separate this into multiple questions.)
Question 3:
Here's an answer on the trace question:
When running the model, the trace actually shows this:
intraCost, extraCost
which is not what you expect, of course. Trace is in effect when creating the model, but at that stage there is no value of these two decision values and MiniZinc shows only the variable names. They got some values to show after the (first) solution is reached, and can then be shown in the output section.
trace is mostly used to see what's happening in loops where one can trace the (fixed) loop variables etc.
If you trace an array of decision variables then they will be represented in a different fashion, the array x will be shown as X_INTRODUCED_0_ etc.
And you can also use trace for domain reflection, e.g. using lb and ub to get the lower/upper value of the domain of a variable ("safe approximation of the bounds" as the documentation states it: https://www.minizinc.org/doc-2.5.5/en/predicates.html?highlight=ub_array). Here's an example which shows the domain of the intraCost variable:
constraint
trace("intraCost: \(lb(intraCost))..\(ub(intraCost))\n")
;
which shows
intraCost: -infinity..infinity
You can read a little more about trace here https://www.minizinc.org/doc-2.5.5/en/efficient.html?highlight=trace .
Update Answer to question 1, 2 and 4.
The constraint #1 and #2 means the same thing, i.e. that the elements in clusters should be disjoint. The #1 constraint is a little different in that it loops over decision variables while the #2 constraint use plain indices. One can guess that #2 is faster since #1 use the where i != j which must be translated to some extra constraints. (And using i < j instead should be a little faster.)
The all_different constraint states about the same and depending on the underlying solver it might be faster if it's translated to an efficient algorithm in the solver.
In the model there is also the following constraint which states that all elements must be used:
constraint (clusters[1] union clusters[2] union clusters[3] union clusters[4]) = dsmElements;
Apart from efficiency, all these constraints above can be replaced with one single constraint: partition_set which ensure that all elements in dsmElements must be used in clusters.
constraint partition_set(clusters,dsmElements);
It might be faster to also combine with the all_different constraint, but that has to be tested.
I am looking to minimize an objective function subject to certain constraints.
The function that I am looking to minimize is:
def distance_function(choice_matrix, distance_matrix, factory_distance):
hub_to_demand_distance = distance_matrix.dot(choice_matrix)
hub_factory_distance = pd.concat([hub_to_demand_distance, factory_distance],axis=1)
min_dist_to_demand = pd.DataFrame(hub_factory_distance.min(axis=1))
transposed_choice = choice_matrix.T
factory_to_hub = transposed_choice.dot(factory_distance)
total_distance = min_dist_to_demand.sum(axis=0)+factory_to_hub.sum(axis=0)
return total_distance
These are the constraints that I have defined:
cons = (
{'type':'ineq','fun': lambda f: 1-choice_matrix[0][0]-choice_matrix[1][0]},
{'type':'ineq','fun': lambda f: 1-choice_matrix[0][1]-choice_matrix[1][1]},
{'type':'ineq','fun': lambda f: 1-choice_matrix[0][2]-choice_matrix[1][2]},
{'type':'ineq','fun': lambda f: 1-choice_matrix[0][3]-choice_matrix[1][3]},
{'type':'eq','fun': lambda f: choice_matrix[0][0]+choice_matrix[0][1]+choice_matrix[0][2]+choice_matrix[0][3]-1},
{'type':'eq','fun': lambda f: choice_matrix[1][0]+choice_matrix[1][1]+choice_matrix[1][2]+choice_matrix[1][3]-1}
)
I have tried using Scipy Optimize to minimize the function as shown:
optimize.minimize(distance_function, choice_matrix, args=(distance_matrix, factory_distance),method='SLSQP',jac=None,constraints=cons)
When I run this, I get the following error:
ValueError: Dot product shape mismatch, (4, 4) vs (8,)
Could you please tell me:
Why this is happening and what needs to be done?
In the code that I shown, I have taken Choice Matrix to have 4 rows and 2 columns and hence I have manually defined 6 constraints (The constraint is the sum of the elements in each row should be lesser than or equal to 1. The other constraint is the sum of the elements in each column should be equal to 1)
My question is if my Choice Matrix has 40 rows and 5 columns, is there a better way to define the constraints than manually entering 45 lines?
Thank you in advance for your help!
I want to implement a constraint depending on the change of values in my binary decision variable, x, over "time".
I am trying to implement a minimum operating time constraint for a unit commitment optimization problem for power systems. x is representing the unit activation where 0 and 1 show that a power unit, n, at a certain time, t, respectively is shut off or turned on.
For this, indicator constraints seem to be a promising solution and with the inspiration of a similar problem the implementation seemed quite straightforward.
So, since boolean operators are introduced (! and ¬), I prematurely wanted to express the change in a boolean way:
#constraint(m, xx1[n=1:N,t=2:T], (!x[n,t-1] && x[n,t]) => {next(t, 1) + next(t, 2) == 2})
Saying: if unit was deactivated before but now is on, then demand the unit to be active for the next 2 times.
Where next(t, i) = x[((t - 1 + i) % T) + 1].
I got the following error:
LoadError: MethodError: no method matching !(::VariableRef)
Closest candidates are:
!(!Matched::Missing) at missing.jl:100
!(!Matched::Bool) at bool.jl:33
!(!Matched::Function) at operators.jl:896
I checked that the indicator constraint is working properly with a single term only.
Question: Is this possible or is there another obvious solution?
Troubleshooting and workarounds: I have tried the following (please correct me if my diagnosis is wrong):
Implement change as an expression: indicator constraints only work with binary integer variables.
Implement change as another variable relating to x. I have found a solution but it is quite sketchy, which is documented in a Julia discourse. The immediate problem, found from the solution, is that indicator constraints do not work as bi-implication but only one way, LHS->RHS. Please see the proper approach given by #Oscar Dowson.
You can get the working code from github.
The trick is to find constraint(s) that have an equivalent truth-table:
# Like
(!x[1] && x[2]) => {z == 1}
# Is equivalent to:
z >= -x[1] + x[2]
# Proof
-x[1] + x[2] = sum <= z
--------------------------
- 0 + 0 = 0 <= 0
- 1 + 0 = -1 <= 0
- 0 + 1 = 1 <= 1
- 1 + 1 = 0 <= 0
I was recommended MOSEK Modeling Cookbook to help working out the correct formulation of constraints.
See eventually the thread here from where I got the answer for further details.
I am solving a serie of linear programing problems using linprog by indexing each problem in a for-loop:
from scipy.optimize import linprog
for i in range(1,N):
sol[i] = linprog(coa3[N][0], A_ub = coa4[i], b_ub = chvneg[i], options= {"disp": True})
I would like to save in a list (still indexed over i) the function minimization result and the array displaying the values of the variables. I guess I need to add in the for-loop something like minfval[i] = ??? and xval[i] = ???, but actually I don't know how to extract these values from the results provided by linprog. Any suggestions? Thanks a lot.
Consider reading the docs as it's pretty clearly explained what exactly linprog returns.
The good thing is, that you are actually storing these values with your code already because you are storing the whole scipy.optimize.OptimizeResult.
It's now just a matter of accessing it:
from scipy.optimize import linprog
for i in range(1,N):
sol[i] = linprog(coa3[N][0], A_ub = coa4[i], b_ub = chvneg[i], options= {"disp": True})
# iterate over solution-vectors
for i in range(1,N):
print(sol[i].x)
# iterate over objectives
for i in range(1,N):
print(sol[i].fun)
You should also check out the field success just to be safe!
Extraction of the docs (link above) on what is contained in linprog's result:
x : ndarray
The independent variable vector which optimizes the linear
programming problem.
fun : float
Value of the objective function.
slack : ndarray
The values of the slack variables. Each slack variable corresponds
to an inequality constraint. If the slack is zero, then the
corresponding constraint is active.
success : bool
Returns True if the algorithm succeeded in finding an optimal
solution.
status : int
An integer representing the exit status of the optimization::
0 : Optimization terminated successfully
1 : Iteration limit reached
2 : Problem appears to be infeasible
3 : Problem appears to be unbounded
nit : int
The number of iterations performed.
message : str
A string descriptor of the exit status of the optimization.