Integer Programming
Integer Programming
LECTURE NOTES
“INTEGER PROGRAMMING”
Acknowledgements:
I would like to acknowledge Prof. W.L. Winston's "Operations Research: Applications and Algorithms“
(slides submitted by Brooks/Cole, a division of Thomson Learning, Inc.) and Prof. J.E. Beasley's
lecture notes which greatly influence these notes...
I retain responsibility for all errors and would love to hear from readers...
www.isl.itu.edu.tr/ya
INTEGER PROGRAMMING
1. FORMULATING IP.............................................................................................. 2
1.1 Budgeting Problems................................................................................................ 2
1.2 Knapsack Problems ................................................................................................ 4
1.3 Fixed Charge Problems.......................................................................................... 5
1.4 Membership in Specified Subsets......................................................................... 8
1.5 Either-Or Constraints ............................................................................................ 10
1.6 If-Then Constraints................................................................................................ 11
1.7 Traveling Salesperson Problems ........................................................................ 12
1.8 Piecewise Linear Function ................................................................................... 13
2. SOLVING IP...................................................................................................... 21
2.1 Categorization ........................................................................................................ 21
2.2 LP Relaxation ......................................................................................................... 22
2.3 Enumeration ........................................................................................................... 23
2.4 The Branch-and-Bound Method.......................................................................... 24
2.5 Implicit Enumeration.............................................................................................. 41
2.6 Cutting Planes........................................................................................................ 42
2.7 Branch & Cut .......................................................................................................... 48
2.8 Branch & Price ....................................................................................................... 49
When formulating LP’s we often found that, strictly, certain variables should have been
regarded as taking integer values but, for the sake of convenience, we let them take
fractional values reasoning that the variables were likely to be so large that any
fractional part could be neglected.
While this is acceptable in some situations, in many cases it is not, and in such cases
we must find a numeric solution in which the variables take integer values.
Problems in which this is the case are called integer programs (IP's) and the subject of
solving such programs is called integer programming (also referred to by the initials
IP).
IP's occur frequently because many decisions are essentially discrete (such as yes/no,
do/do not) in that one or more options must be chosen from a finite set of alternatives.
An IP in which all variables are required to be integers is called a pure IP problem.
If some variables are restricted to be integer and some are not then the problem is a
mixed IP problem.
The case where the integer variables are restricted to be 0 or 1 comes up surprising
often. Such problems are called pure (mixed) 0-1 programming problems or pure
(mixed) binary IP problems.
For any IP we can generate an LP by taking the same objective function and same
constraints but with the requirement that variables are integer replaced by appropriate
continuous constraints:
“xi ≥ 0 and integer” can be replaced by xi ≥ 0
“xi = 0 or 1” can be replaced by xi ≥ 0 and xi ≤ 1
The LP obtained by omitting all integer or 0-1 constraints on variables is called LP
Relaxation of the IP (LR).
Answer
We will use a 0-1 variable xj for each project:
xj is 1 if we decide to do project j;
xj is 0 otherwise (i.e. not do project j).
This leads to the 0-1 programming problem:
max 0.2 x1 + 0.3 x2 + 0.5 x3 + 0.1 x4
s.t. 0.5 x1 + 1 x2 + 1.5 x3 + 0.1 x4 ≤ 3.1
0.3 x1 + 0.8 x2 + 1.5 x3 + 0.4 x4 ≤ 2.5
0.2 x1 + 0.2 x2 + 0.3 x3 + 0.1 x4 ≤ 0.4
xj = 0 or 1 j = 1, … 4
Example 2. Knapsack
For instance, the following is a knapsack problem:
Maximize 8 x1 + 11 x2 + 6 x3 + 4 x4
Subject to 5 x1 + 7 x2 + 4 x3 + 3 x4 ≤ 14
xj = 0 or 1 j = 1, … 4
There is a cost associated with performing an activity at a nonzero level that does not
depend on the level of the activity.
An important trick can be used to formulate many production and location problems
involving the idea of a fixed charge as IP.
Answer
Let xj be number of clothing produced.
Let yj be 1 if any clothing j is manufactured and 0 otherwise.
Profit = Sales revenue – Variable Cost – Costs of renting machinery
For example the profit from shirts is
z1 = ( 12 – 6 ) x1 – 200 y1
Since supply of labor and cloth is limited, Gandhi faces two constraints.
To ensure xj > 0 forces yj = 1, we include the additional constraints
xj ≤ Mj yj
From the cloth constraint at most 40 shirts can be produced (M1=40), so the additional
constraint for shirts is not an additional limit on x1 (If M1 were not chosen large (say
M1=10), then the additional constraint for shirts would unnecessarily restrict the value of
x1).
From the cloth constraint at most 53 shorts can be produced (M2=53)
From the labor constraint at most 25 pants can be produced (M3=25)
(∑j should be read as "sum over all integer values of j from 1 to n inclusive")
A region can only be assigned to an open lockbox:
x1j + x2j + x3j + x4j ≤ M yj
M is any number that should be at least 4 as there are four regions.
(Suppose we do not open LA lockbox; then y1 is 0, so all of x11, x21, x31, and x41 must
also be 0. If y1 is 1, then there is no restriction on the x values.)
min 28 x11 + 84 x12 + 112 x13 + 112 x14
+ 60 x21 + 20 x22 + 50 x23 + 50 x24
+ 96 x31 + 60 x32 + 24 x33 + 60 x34
+ 64 x41 + 40 x42 + 40 x43 + 16 x44
+ 50 y1 + 50 y2 + 50 y3 + 50 y4
Using decision variables that equal 1 if an object is part of a solution and 0 otherwise,
set covering, set packing, and set partitioning models formulate problems where the
core issue is membership in specifed subsets.
Many applications in areas such as location problems (fire/police station, warehouse,
facility), scheduling (crew, airline, truck, bus), political districting
• Set covering constraints
Require that at least one member of subcollection J belongs to a solution:
∑j∈J xj ≥ 1
• Set packing constraints
Require that at most one member of subcollection J belongs to a solution:
∑j∈J xj ≤ 1
• Set partitioning constraints
Require that exactly one member of subcollection J belongs to a solution:
∑j∈J xj = 1
“Given a number of cities and the costs of traveling from any city to any other city, what
is the cheapest round-trip route (tour) that visits each city once and then returns to the
starting city?”
This problem is called the traveling salesperson problem (TSP), not surprisingly.
An itinerary that begins and ends at the same city and visits each city once is called a
tour.
Suppose there are N cities.
Let cij = Distance from city i to city j (for i≠j) and
Let cii = M (a very large number relative to actual distances)
Also define xij as a 0-1 variable as follows:
xij = 1 if s/he goes from city i to city j;
xij = 0 otherwise
The formulation of the TSP is:
0-1 variables can be used to model optimization problems involving piecewise linear
functions.
A piecewise linear function consists of several straight line segments.
The points where the slope of the piecewise linear function changes are called the break
points of the function.
If a piecewise linear function f(x) involved in a formulation has the property that the slope
of the f(x) becomes less favorable to the decision maker as x increases, then the tedious
IP formulation is unnecessary.
2.1 Categorization
2.2 LP Relaxation
For any IP we can generate an LP by taking the same objective function and same
constraints but with the requirement that variables are integer replaced by appropriate
continuous constraints:
“xi = 0 or 1” Æ xi >= 0 and xi <= 1
“xi > 0 and integer” Æ xi >= 0
The LP obtained by omitting all integer and 0-1 constraints on variables is called the LP
Relaxation of the IP (LR). We can then solve this LR of the original IP.
LR – IP Relation
Since LR is less constrained than IP:
• If IP is a maximization problem, the optimal objective value for LR is greater than or
equal to that of IP.
• If IP is a minimization problem, the optimal objective value for LR is less than or
equal to that of IP.
• If LR is infeasible, then so is IP.
So solving LR does give some information. It gives a bound on the optimal value, and, if
we are lucky, may give the optimal solution to IP.
2.3 Enumeration
Unlike LP (where variables took continuous values) in IP's (where all variables are
integers) each variable can only take a finite number of discrete (integer) values.
Hence the obvious solution approach is simply to enumerate all these possibilities -
calculating the value of the objective function at each one and choosing the (feasible)
one with the optimal value.
0 0 1 1 do two projects
0 1 0 1
1 0 0 1
0 1 1 0
1 0 1 0
1 1 0 0
Review
Hence for our example, we merely have to examine 16 possibilities before we know
precisely what the best possible solution is. This example illustrates a general truth
about integer programming.
What makes solving the problem easy when it is small is precisely what makes it hard
very quickly as the problem size increases.
This is simply illustrated: suppose we have 100 integer variables each with two possible
integer values then there are 2x2x2x ... x2 = 2100 (approximately 1030) possibilities
which we have to enumerate (obviously many of these possibilities will be infeasible, but
until we generate one we cannot check it against the constraints to see if it is feasible).
For 100 integer variable - conceptually there is not a problem - simply enumerate all
possibilities and choose the best one. But computationally (numerically) this is just
impossible.
The most effective general purpose optimal algorithm is an LP-based tree search
approach called as branch and bound (B&B).
The method was first put forward in the early 1960's by Land and Doig.
Example 3. Mixed IP
(Winston 9.4., p. 523)
max z = 2 x1 + x2
s.t. 5 x1 + 2 x2 ≤ 8
x1 + x2 ≤ 3
x1, x2 ≥ 0; x1 integer
Answer
We solve the LR (subp.1) of the problem
[replace “x1≥ 0 and integer” by “x1 ≥ 0”]
Then using any LP package or utilizing simplex or graphical solution method we get
z = 11/3, x1 = 2/3, x2=7/3
Because x2 is allowed to be fractional, we do not branch on x2.
We branch on x1 and create two new subproblems:
Subp.2: LR + x1 ≥ 1
Subp.3: LR + x1 ≤ 0
We see that the optimal solution to subp.2 is
z = 7/2, x1 = 1, x2 = 3/2
As only x1 assume integer value, this solution is feasible for the original MIP Æ
Candidate solution; LB = 7/2
Example 4. Binary IP
max z = 0.2 x1 + 0.3 x2 + 0.5 x3 + 0.1 x4
s.t. 0.5 x1 + 1 x2 + 1.5 x3 + 0.1 x4 ≤ 3.1
0.3 x1 + 0.8 x2 + 1.5 x3 + 0.4 x4 ≤ 2.5
0.2 x1 + 0.2 x2 + 0.3 x3 + 0.1 x4 ≤ 0.4
xj = 0 or 1 j = 1, … 4
Answer
Replace “xj = 0 or 1 (j=1,...,4)” by “0 ≤ xj ≤ 1 (j=1,...,4)” Æ LR of the problem
Optimal solution to the LR:
z=0.65, x2=0.5, x3=1, x1=x4=0
The variable x2 is fractional. To resolve this we can generate two new problems:
P1: LR + x2=0
P2: LR + x2=1
We now have two new subproblem to solve (jumptracking).
If we do this we get
P1 solution: z=0.6, x1=0.5, x3=1, x2=x4=0
P2 solution: z=0.63, x2=1, x3=0.67, x1=x4=0
Review
Note here that B&B, like complete enumeration, also involves powers of 2 as we
progress down the (binary) tree.
However also note that we did not enumerate all possible integer solutions (of which
there are 16). Instead here we solved 7 LP's.
This is an important point, and indeed why tree search works at all. We do not need to
examine as many LP's as there are possible solutions.
Inspection
Recall that
ci is the benefit obtained if item i is chosen
b is the total amount of an available resource
ai is the amount of the available resource used by item i
Observe that ratio ri (ci/ai) may be interpreted as the benefit item i earns for each unit of
the resource used by item i.
Thus, the best items have the largest value of r and the worst items have the smallest
values of r.
To solve any subproblem resulting from a knapsack problem, compute all the ratios.
Then put the best item in the knapsack.
Then put the second best item in the knapsack.
Continue in this fashion until the best remaining item will overfill the knapsack.
Then fill the knapsack with as much of this item as possible.
TSP
Please recall that
We define xij as a 0-1 variable:
xij = 1 if TS goes from city i to city j;
xij = 0 otherwise
cij = distance form city i to city j (for i≠j)
cii = M (a very large number relative to actual distances)
An itinerary that begins and ends at the same city and visits each city once is
called a tour.
It seems reasonable that we might be able to find the answer to TSP by solving an
assignment problem having a cost matrix whose ijth is cij.
If the optimal solution to the assignment problem yields a tour, it is the optimal solution
to the TSP.
Unfortunately, the optimal solution to the assignment problem need not be a tour (may
yield subtours).
If we could exclude all feasible solutions that contain subtours and then solve the
assignment problem, we would obtain the optimal solution to TSP Æ Not easy to do...
Several B&B approaches have been developed for solving TSPs.
Example 7: TSP
(Winston 9.6., p. 530)
Joe State lives in Gary, Indiana and owns insurance agencies in Gary, Fort Wayne,
Evansville, Terre Haute, and South Bend.
Each December, he visits each of his insurance agencies.
The distance between each agency:
miles G FW E TH SB
G 0 132 217 164 58
FW 132 0 290 201 79
E 217 290 0 113 303
TH 164 201 113 0 196
SB 58 79 303 196 0
What order of visiting his agencies will minimize the total distance traveled?
Answer
We first solve the assignment problem (subp.1) applying the Hungarian method to the
cost matrix shown:
COSTS G FW E TH SB
G 1000 132 217 164 58
FW 132 1000 290 201 79
E 217 290 1000 113 303
TH 164 201 113 1000 196
SB 58 79 303 196 1000
The optimal solution will be:
x15=x21=x34=x43=x52=1, z=495
Evaluation of Heuristics
• Performance guarantees
Gives a worse-case bound on how far away from optimality a tour constructed by the
heuristic can be
• Probabilistic analysis
A heuristic is evaluated by assuming that the location of cities follows some known
probability distribution
• Empirical analysis
Heuristics are compared to the optimal solution for a number of problems for which
the optimal tour is known
A linear inequality is a valid inequality for a given IP problem if it holds for all integer
feasible solutions to the model.
Relaxations can often be strengthened dramatically by including valid inequalities that
are not needed by a correct discrete model.
To strengthen a relaxation, a valid inequality must cut off (render infeasible) some
feasible solutions to current LR that are not feasible in the IP model.
This need to cut off noninteger relaxation solutions is why valid inequalities are
sometimes called cutting planes.
Optimum
(integer)
solution
P Optimum
fractional
solution
x
Example: min. x + 10y
s.t. x, y are in P
x, y integer
add “y ≥1”
P add “y ≤ x –1”
These constraints
x were obtained by
inspection. We will
Example: min. x + 10y develop techniques
s.t. x, y are in P later.
x, y integer
If we add exactly
the right
inequalities, then
y every corner point
of the LP will be
integer, and the IP
can be solved by
solving the LP
Cut Classification
• General purpose
A fractional extreme point can always be separated (LP-based approach, that works
for IP)
o Disjunctive cuts
o Gomory cutting planes
• Problem specific
Derived from problem structure, generally facets. (Capital Budgeting (Knapsack), Set
Packing... )
The LP relaxation
A variation of B&B termed Branch & Cut (B&C) takes advantage of valid inequalities to
speed computation.
B&C algorithms modify the basic B&B strategy by attempting to strengthen relaxations
with new inequalities before branching a partial solution.
Added constraints should hold for all feasible solutions to IP model, but they should cut
off (render infeasible) the last relaxation optimum.