Logic-Based Benders Decomposition
∗
J. N. Hooker
Graduate School of Industrial Administration
Carnegie Mellon University, Pittsburgh, PA 15213 USA
G. Ottosson
Department of Information Technology
Computing Science, Uppsala University
Box 311, S-751 05 Uppsala , Sweden
November 1995, Revised November 2000
Abstract
Benders decomposition uses a strategy of “learning from o ne’s mis-
takes.” The aim of this paper is to extend this strategy to a much larger
class of problems. The key is to generalize the linear programming dual
used in the classical method to an “inference dual.” Solution of the
inference dua l takes the form of a logical deduction that yields Ben-
ders cuts. The dual is therefore very different from other generalized
duals that have been proposed. The approach is illustrated by work-
ing out the details for propositional satisfiability and 0-1 programming
problems. Computational tests are carried out for the latter, but the
most promising contribution of logic-based Benders may be to provide
a framework for combining optimization and constraint programming
methods.
∗
This research was partially supported by U.S. Office of Naval Research Grant N00014-
95-1-0517 and by the Engineering Design Research Center at Carnegie Mellon University,
an Engineering Research Center of the National Science Foundation, under grant EEC-
8943164.
1
Benders decomposition [7, 17] uses a problem-solving strategy that can
be g eneralized to a larger context. It assigns some of the variables trial va lues
and finds the best solut ion consistent with these values. In the process it
learns something about the quality of other trial solut ions. It uses this
information to reduce the number of solutions it must enumerate to find
an optimal solution. The strategy might described as “learning from one’s
mistakes.”
The central element of Benders decomposition is the derivation of Ben-
ders cuts that exclude superfluous sol utions. Classical Benders cuts are
formulated by solving the dual of the subproblem that remains when the
trial values are fixed. The subproblem must therefore be one for which dual
mult iplers are defined, such as a linear or nonlinear programming probl em.
The key to generalizing Benders decomposi tion is to extend the class
of problems for which a suitable dual can be formulat ed. We depart from
previous generalizations of duality by defining an inference dual for any
optimization problem. The solution of the inference dual is a proof of op-
timality within an appropriate logical formalism. Generalized Benders cuts
are obtained by determining under what conditions the proof r emains valid.
Classical Benders decomposition can be seen to be a special case of this
approach if it i s viewed in a different light than the usual.
Logic-based Benders decomposition can be applied to any class of opt i-
mization problems, but a proof scheme and a method of generating Benders
cuts must be devised for each class. We illustrate the method by applying
it to propositiona l satisfiability, 0-1 programming problems, and a machine
scheduling problem. The subproblems in these cases are not the traditional
linear or nonlinear programming problems. One can therefore take advan-
tage of special structure in the subproblems that is inaccessible to the tra-
ditional Benders m ethod.
For example, a sat isfiability or 0-1 programming problem may decou-
ple into several smaller problems when certain variables are fixed. The
subproblem can therefore be solved rapidly by solving its indivi dual compo-
nents, even though the comp onent s are themselves general satisfiability or
0-1 problems.
We address the satisfiability problem because it illustrates logic-based
Benders in a lucid way and is an i mport ant problem in its own right. We
examine the 0-1 progr amming problem because of the attention it has his-
torically received. None of this should sugg est, however, that logic-based
Benders is applicable only to these problem classes.
For i nstance, Jain and Gr ossmann [ 31] recently s olved machine schedul-
2
ing problems using a technique that is in effect logic-based Benders de-
coposition. Because the subproblems are one-machine scheduling problems,
classical Benders cuts are unavailable. Jain and Grossmann achieved dra-
matic speedups in computation by solving the subproblems with constr aint
technology. This experience suggests that logic-based Benders can provide
a natural medium for combining optimization and constraint programming.
This idea is dis cussed in [27, 29 ].
The first section below reviews related work. Section 2 presents the basic
idea of logic-based Benders decomposition. Section 3 intro duces inference
duality, and Section 4 shows how linear programming duality is a special
case. The next two sections present logic-based Benders decomposition in
the abstract, followed by its classica l realization. Sections 7 and 8 apply
logic-based Benders to the propositional satisfiability problem and 0-1 pro-
gramming, respectively. Section 9 presents computational results for 0-1
program ming, and Section 10 describes Jain and G rossmann’s work. The
final section is reserved for concluding remarks.
1 Related Work
Various types of generalized duality have been proposed over the years. Tind
and Wolsey provided a survey i n 1981 [43]. Much of this and subsequent
work [2, 3, 8, 9, 13, 48, 49] is related to the superadditive duality of Johnson
[35]. In 1981 Wolsey [50] used such a notion of duality as the basis for a
generalized Benders algorithm in which the classi ca l dual prices are replaced
by a price function. Several other duals have been suggested for integer
program ming [4, 6, 15, 16, 40, 41]. A recent paper of Williams [47] examines
a wide var iety of duality concepts. Sti ll more recently, Lagrangean and
surrogate duals are interpreted i n [27, 29] as forms of a relaxation dual.
The inference dual proposed here is fundamentally different from these
earlier duals, because it regards the dual as an inference problem. Its so-
lution is in general a proof, rather than a set of prices or a pri ce function.
The resulting generalizatio n of Benders decomposition is therefore unlike
Wolsey’s .
The idea of i nference duality might be traced ultimately to Jeroslow and
Wang [33]. They showed that when li near programming demonstrates the
unsatisfiability of a set of Horn clauses in propositional logic, the dual so-
lution contains information about a unit resolution proof of unsatisfiability.
(Unit resolution is defined in Section 7 below.) This introduces the key idea
3
that the dual solution can be seen as encoding (or partially encoding) a
proof. It does not, however, show how to generalize the idea beyond a linear
program ming co ntext, and a subsequent generalization focused on another
type of linear programming problem, namely gain-free Leontief flows [32].
Hooker and Yan [30] introduced the logic-based Benders scheme de-
scribed here, or a special case of it, in the context of logic circuit verification,
and they presented computational results. Their m ethod is that of Section 5
below specialized to logic circuits. After the first draft of the present paper
was written (1995), Hooker [25] pro posed inference duality as a basis for
postoptimality analysis, and Dawande and Hooker [14] specialized the ap-
proach to sensitivity analysis for mixed integer programming. These ideas
are present ed in [27] as part of a general theoreti cal framework.
2 The Basic Idea
As already noted, Benders decomposition learns from its mistakes. A similar
idea has been developed in a general way in the constraint programming
lit erature under the rubric of nogoods ([45], Section 5.4; [34]). When in
the process of solving a problem one deduces that a certain partial solution
cannot be completed t o obtain a feasible solution, one can examine the
reasons for this. Often the same reasons lead to a constraint that excludes
a number of partial solutions. Such a constraint is a nogood.
Benders decomposition uses a more specific strategy. It begins by par-
titioning the variables of a problem into two vectors x and y. It fixes y
to a trial value so as to define a subproblem that conta ins only x. If the
solution of the subproblem reveals that the trial value of y is unacceptable,
the solution of the subproblem’s dual is used to identify a number of other
values of y that are likewise unacceptable. The next trial value must be one
that has not been excluded. Eventually only a cceptable values remain, and
if all goes well, the algorithm terminates after enumerating only a few of the
possible values of y. (The method is presented in more detail below.)
Because Benders decomposition sea rches for an optimal as well as a
feasible soluti on, it actually enumerates trial values of (z, y ), where z is the
objective function value. Its “nogoods” have the form z ≥ β(y), where
β(y) is a bound on the optimal value that depends on y. The constraints
z ≥ β(y) are known as Benders cuts and can rule out a large number of
values of (z, y).
The specialized context of Benders decomposition enhances the general
4
strategy in two ways.
• The pre-arranged partition of variables can exploit problem structure.
When y is fixed, t he resulting subproblem may simplify substantially
or decouple into a number of small subproblems.
• The lineari ty of the subproblem allows one to obtain a Benders cut in
an easy and systematic way, namely by solvi ng the linear progra mming
dual.
The intent here is to generalize a Benders-like strategy while retaining these
two advantages to a large degree. T he key to doing so is to generalize the
notion of a dual. The dual must be definable for any type of subproblem,
not just linear ones, and must provide an appropriate bound on the optimal
value.
Such a dual can be formulated simply by observing that a valid bound
β on the optimal value is obtained by inferring it from the constra ints.
A generalized dual can t herefore defined as an inf erence dual, which is the
problem of inferr ing a strongest possible bound from the constrai nt set. The
classical linear program ming dual is the inference dual probl em for linear
optimization. A solution of the inference dual takes the form of a proof that
β is in fact a bo und on the optimal value.
In the context of Benders decompo s ition, the proof that solves the sub-
problem dual provides a valid bound β on the assumption that y is fixed
to some particular value. But the same reasoning may deliver valid bounds
when y takes other values. A constraint that imposes these bounds as a
function of y becomes the log ic-based Benders cut. It plays the same ro le as
the cl assical Benders cut, although it may not take the form of an inequal ity
constrai nt.
3 Inference Duality
Consider a general optimization problem,
min f(x)
subj ect to x ∈ S
x ∈ D
(1)
where f is a real-valued function. The domain D is distinguished from the
feasible set S. The domain might be the set of real vectors, 0-1 vectors, et c.
5
