Deciding Linear Inequalities by Computing Loop Residues
ROBERT SHOSTAK
SRI International, Menlo Park, Cahforma
ABSTRACT V R Pratt has shown that the real and integer feastbdlty of sets of linear mequallUes of the
form x _< y + c can be decided quickly by examining the loops m certain graphs Pratt's method is
generahzed, first to real feaslbdlty of mequahues m two variables and arbitrary coefficients, and ultimately
to real feaslbdlty of arbitrary sets of hnear mequahtles The method is well suited to apphcatlons m
program verification
KEY WORDS AND PHRASES theorem proving, decision procedures, program venficauon, linear
programmmg
CRCATEGORIES 3 15,369,521,532,541
1. lntroductton
Procedures for deciding whether a given set of linear inequalities has solutions often
play an important role in deductive systems for program verification. Array bounds
checks and tests on index variables are but two of the many common programming
constructs that give rise to formulas involving inequalities. A number of approaches
have been used to decide the feasibdity of sets of inequalities [3, 8, 9, 16, 22], ranging
from goal-driven rewriting mechanisms [27] to the powerful simplex techniques [8]
of linear programming. Some simple methods are well suited to the small, trivial
problems that most often arise, but are insufficiently general. Full-scale simplex
techniques, on the other hand, are general and fast for medium to large problems,
but do not take advantage of the trivial structure of the small problems (revolving
only a few variables and equations) encountered most frequently in program verifi-
cation and related applications.
The algorithm presented here retains the generality needed in the exceptional case,
without sacrifice of speed and simplicity in the more typical small problem case. It
builds on V. R. Pratt's observation [18, 20] that most of the inequalities that arise
from verification conditions are of the form x _< y + c, where x and y are variables
and c is a constant. Pratt showed that a conjunction of such inequalities can be
decided quickly by examining the loops of a graph constructed from the inequalities
of the conjunction. We generalize this approach, first to inequalities with no more
Permission to copy without fee all or part of this material ts granted provided that the copies are not made
or distributed for direct commercial advantage, the ACM copyrtght notice and the title of the publication
and its date appear, and notice is gwen that copying is by permission of the Association for Computing
Machinery To copy otherwise, or to republish, reqmres a fee and/or spectfic permission
This work was supported m part by the National Science Foundation under Grant MCS 76-81425, the Air
Force Office of Scientific Research under Contract F44620-73-C-0068, and the Rome Air Development
Center under Contract F30602-78-C-0031 A condensed version appeared in the Proceedings of the Fourth
Workshop on Automatic Deduction, Austin, Texas, February 1979
Author's address SR1 International, 333 Ravenswood Avenue, Menlo Park, CA 94025
© 1981 ACM 0004-5411/81/1000-0769 $00 75
Journal of the Association for Computing Machinery, Vol 28. No 4. October 1981. pp 769-779
770
ROBERT SHOSTAK
than two variables and with arbitrary coefficients, and then to arbitrary linear
inequalities. Our generalization reduces to Pratt's test for inputs having the simple
structure he describes.
The algorithm has recently been used by Aspvall and Shiloach [1] as the basis for
a polynomial-time algorithm for the two-variable case.
It should be remarked that the usefulness of the algorithm for verification-type
applications is not affected by the recent work of L. G. Khachiyan showing that
linear programming requires polynomial time m the worst case [16]. While Khachi-
yan's algorithm has not yet been thoroughly evaluated (at least in the West), a
number of researchers have expressed doubt as to its usefulness in practice. In any
case, Khachiyan's algorithm is clearly not suited to very small problems of the kind
that arise in program verification. For the examples gwen in this paper, the imtiali-
zation step alone of Khachiyan's algorithm requires more computation than does
their complete solution using the method of loop residues.
The discussion is presented in six sections. Sections 2 and 3 are concerned with
preliminary definitions and a statement of the method for inequahtles with two
variables and arbitrary coefficients. Section 4 discusses issues of complexity and
usefulness for integer problems and relates the method to Pratt's. Sections 5 and 6
deal with the extension of the method to sets having strict inequalities and sets with
arbitrary linear inequalities. The last section presents a proof of the theorem that
underlies the method.
2. Defimtions
Let S be a set of hnear inequalities each of whose members can be written in the
form
ax + by _< c,
where x, y are real variables and a, b, c are reals. Without loss of
generality we require that all variables appearing in S other than a special variable
v0, called the
zero variable,
have nonzero coefficients. We also assume that v0 appears
only with coefficient zero.
Construct an undirected multigraph G from S as follows. Gwe G a vertex for each
variable occurring in S and an edge for each inequality. Let the edge associated with
an inequality
ax + by _< c
connect the vertex for x with the vertex for y. Label each
vertex with its associated variable I and each edge with Its associated inequality. G is
said to be the
graph for S.
Now let P be a path through G, given by a sequence
va, vz ..... vn+a
of vertices and
a sequence ea,
ez
..... en of edges, n _> 1. The
triple sequence
for P is given by
(al, bx, ca), (az, bz, c2) ..... (an, bn, cn),
where for each
i, 1 <_ i <_ n, a,v, + b,v,+a <- c,
is the inequahty labehng e,. 2 P is
admtssible
if, for 1 _< t _< n - 1, b~ and a,+~ have opposite signs, that is, one is strictly
positwe and the other is negative.
Intuitively, admissible paths correspond to sequences of inequalities that form
transmvity chains. For example, the sequence x _< y, y _ z, z _< 3 gives rise to an
admissible path, as does
2x_>3y-4,
2y>_4-z, -z>_x.
In what follows It Is notat~onally convenient to write v for both the varmble v and the vertex assooated
with that variable
z In the case where v, and v,+~ happen to be identical 0 e, e, is a self-loop), an arbitrary choice is made as
to the ordering of the first two components of the assooated triple
Deciding Linear Inequalities by Computing Loop Residues
771
Note that the sequence
x_< y, y_< z, -z_< r
cannot label an admissible path, since the coefficients of z have the wrong relative
signs.
A path is a
loop
if its first and last vertices are identical. A loop is
simple
if its
intermediate vertices are distinct.
Note that the reverse of an admissible loop is always admissible, and that the cyclic
permutations of a loop P are admissible if and only if al and bn are of opposite sign,
where (aa, bl, Cl) ..... (an, bn, cn) is the triple sequence for P. In this case, we say P
is
permutable.
Note also that since Vo appears in S o~y with coefficient 0, no
admissible loop with initial vertex Vo is permutable.
Now define, for a given admissible path P, the
residue inequality of P as
the
inequality obtained from P by applying transitivity to the inequalities labeling its
edges. For example, if the inequalities along P are
x _< 2y + l, y_< 2 - 3z, -z_< w,
we have
x_<2y+l_<2(2-3z)+l_<2(2+3w)+l--6w+5.
The residue inequality of P is thus x - 6w <_ 5.
More formally, define the
residue r,
of P as the triple
(ap, bp, cp)
given by
(ap, bp,
Cp) = (al, bx,
Cl)
*
(as, b2, c2) * -.. * (an, bn, cn),
where (ax, bl, Cl) ..... (an, bn, Cn) is the triple sequence for P and * is the binary
operation on triples defined by
a ~
(a, b, c) * (a', b', c') =
(kaa', -kbb', k(ca' - c'b))
and k
-
la'l"
The
residue inequality
of P is then given by
apx + bpy _< cp,
where x and y are the
first and last vertices, respectively, of P.
It is straightforward to show that • is associative, so that rR is in fact uniquely
defined. The idea that the residue inequality of a path is implied by the inequalities
labeling the path is expressed in the following lemma.
LEMMA 1.
Any point (i.e., assignment of reals to variables) that satisfies the
inequalities labeling an admissible path P also satisfies the residue inequality of P.
PROOF. Straightforward by induction on the length of P. Q.E.D.
3. Procedure for Inequalities with Two Variables
In the case where P is a loop with initial vertex, say, x, Lemma 1 asserts that any
point satisfying the inequalities along P must also satisfy
apx + bex _< cp.
If it
happens that
ae + bp
= 0 and
c,
< 0, the residue inequality of P is false, and we say
that P is an
infeasible loop.
It follows that a set S of inequalities is unsatisfiable if the graph G for S has an
infeasible loop. The converse, however, does not hold in general. Figure 1, for
example, shows the graph for S = {x _<y, 2x + y _< 1, z _< x, w_< z, z _< 1 + w,
z _> ½}. Although S is unsatisfiable, the graph has no infeasible loops, simple or
otherwise.
The gist of our main theorem is that G can be modified to obtain a graph G' that
has an infeasible simple loop if and only if S is unsatisfiable.
772
ROBERT SHOSTAK
w~z x~y
w
2x+y~
vo
FIG. 1.
Graph G for S = {x .~ y, 2x + y - < l, z -< x, w _< z, z -< w +
1, z_>½}.
w~z x~Y
z~½ x.~½
vo
FIG 2. Closure of G
Definition.
Let G be the graph for S. Obtain a
closure G'
of G by adding, for each
simple admissible loop P (modulo permutation and reversal) of G a new edge labeled
with the residue inequality of P.
Note that closures are not necessarily unique, since the initial vertex of each
permutable loop can be chosen arbitrarily.
THEOREM.
S is unsatisfiable if and only if G' has an infeasible simple loop.
Figure 2 shows the unique closure of the graph of Figure 1. Note that the only
loop of G contributing an edge to G' is the
xyx
loop. The
VoXZVo
loop of G' is
infeasible (having residue (0, 0, -~)); hence the example S, according to the theorem,
must be unsatisfiable.
We show later that any cyclic permutation of an infeasible permutable loop is
itself infeasible, and that the reverse of an infeasible loop is also infeasible. We thus
have the following decision procedure for satisfiability of S:
(1) The simple admissible loops of G are enumerated modulo cyclic permutation
and reversal, and their residues are computed. If any loops are found to be
infeasible, S is unsatisfiable.
(2) Otherwise, the closure of G is formed by adding a new edge for each residue
inequality. The residues of all newly formed simple admissible loops are now
computed. If any are found to be infeasible, S is unsatisfiable. Otherwise S has
solutions.
Note that this procedure, as stated, does not actually construct a solution if S is
feasible. The proof of the main theorem, given in Section 7, provides such a
Deciding Linear Inequalities by Computing Loop Residues 773
construction. Note also that the new admissible loops formed in (2) must have initial
vertex Vo.
4. Efficiency and Other Issues
Any implementation of the procedure must, of course, incorporate some means of
generating the simple loops of a graph. For this purpose, several algorithms exist
(Johnson [141, Read and Tarjan [21], Szwarcfiter and Lauer [251) that operate in
time order l(I V I +
IEI),
and space order I VI +
IEI,
where l is the number of loops
generated. These algorithms are easily modified to generate only admissible loops
without adversely affecting efficiency. Since each loop has length on the order of
I VI, these algorithms require little more time than that needed for output, A graph
may, of course, have quite a few simple loops--exponentially many (in I E I), in fact,
in the worst case. One can show that the procedure we have described, like the
simplex method, exhibits exponential worst case asymptotic behavior.
In practice, however, one does not encounter such behavior. The sets of inequalities
that arise from verification conditions usually have the form of transitivity chains.
The corresponding graphs are treelike, seldom having more than a few loops. Most
of the loops that do occur are 2-loops, which are easily tested at the time the graph
is constructed.
Pratt [20] has noted that these sets often fall within what he has termed separation
theory. All the inequalities of such sets are of the form x _< y + c. The residue of a
loop whose labeling inequalities are of this form is given by one of <1, -1, m>,
(- 1, 1, m ), where m is the sum of the constants c around the loop. The graph for a
set S in separation theory is thus its own closure, so the main theorem of the last
section reduces, in this case, to Pratt's observation that such a set S is infeasible if
and only if the sum of the constants around some simple loop is negative. Pratt notes
that this condition can be tested in order (1 V] +
IEI) 3
time by taking a max/+
transitive closure of the incidence matrix of the graph. In practice, however, it may
be more efficient to generate loops using one of the algorithms mentioned earlier.
Note that a set of inequalities in separation theory with integer constants is integer
feasible if and only if it is real feasible. While the main theorem therefore decides
integer feasibility in this case, it cannot decide integer feasibility in general. It has
been observed [23], however, that the transformations Bledsoe [4] describes for
reducing formulas in integer arithmetic to sets of inequalities tends to produce sets
that are integer feasible if and only if they are real feasible. The main theorem thus
provides a useful, but not complete, test for integer feasibility.
5. Strict Inequalities
The procedure is trivially generalized to handle strict inequalities (i.e., inequalities of
the form ax + by < c). Let an admissible loop be strict if one or more of its edges is
labeled with a strict inequality. A strict loop P with residue (ap, bp, cp) is infeasible
if ap
+ bp =
0 and Cp _< O. If the definition of closure is now modified in such a way
that new edges arising from strict loops are labeled with strict inequalities, the main
theorem still holds.
6. Extension to Arbitrary Sets of Inequalities
The method can be further generalized to sets of inequalities with arbitrary coeffi-
cients and arbitrary numbers of variables. The basic idea is illustrated by the
following example. Consider the set
S= (x_<y,y_<z,z<_y-x+ 1, x_> 2).