scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Exploiting resolution proofs to speed up LTL vacuity detection for BMC

TL;DR: The vacuity detection tool, VaqTree, uses a characteristic of resolution proofs— peripherality—and proves that if a variable is a source of vacuity, then there exists a resolution proof in which this variable is peripheral.
Abstract: When model-checking reports that a property holds on a model, vacuity detection increases user confidence in this result by checking that the property is satisfied in the intended way. While vacuity detection is effective, it is a relatively expensive technique requiring many additional model-checking runs. We address the problem of efficient vacuity detection for Bounded Model Checking (BMC) of linear temporal logic properties, presenting three partial vacuity detection methods based on the efficient analysis of the resolution proof produced by a successful BMC run. In particular, we define a characteristic of resolution proofs— peripherality—and prove that if a variable is a source of vacuity, then there exists a resolution proof in which this variable is peripheral. Our vacuity detection tool, VaqTree, uses these methods to detect vacuous variables, decreasing the total number of model-checking runs required to detect all sources of vacuity.

Summary (3 min read)

1 Introduction

  • Model-checking [7] is a widely-used automated technique for verification of both hardware and software artifacts that checks whether a temporal logic property is satisfied by a finite-state model of the artifact.
  • If the model does not satisfy the property, a counterexample, which can aid in debugging, is produced.
  • Vacuity detection [2,18,21,1] is an automatic sanity check that can be applied after a positive model-checking run in order to gain confidence that the model and the property capture the desired behaviours.
  • The peripherality algorithm examines the structure of the resolution proof, identifying as vacuous those variables that are not necessary or central to the derivation of false.

2.1 Bounded Model-Checking

  • Bounded model-checking (BMC) [4] is a method for determining whether a linear temporal logic (LTL) formulaϕ holds on a finite state system represented by a Kripke structure K up to a finite number of steps.
  • Below, the authors give an informal overview of Kripke structures, LTL formulas and BMC.
  • A run of K is a sequence of states starting with s0 that obeys R. Each run has an associated trace π, where πi is simply the set of propositional formulas that label the ith state in the run.
  • The error constraint CLe is encoded according to a recursive procedure which removes the temporal and logical operators from the property [4], e.g., the algorithm encodes ϕ =.
  • When a DPLL-based SAT solver processes an unsatisfiable theory, a resolution derivation of false (or the empty clause) is implicitly constructed [10, 27].

2.2 Resolution Proofs

  • Resolution is an inference rule that is applied to propositional clauses to produce logical consequences.
  • Π represents a tree of resolutions between the clauses labeling its nodes.
  • Its roots are the nodes with no parents; otherwise, all nodes have exactly two parents.
  • Given a non-root node labeled by the clause c, and the labels of its parents, c1 and c2, c is the resolvent since it has been produced by resolving c1 and c2 on some variable v.
  • Figure 2 shows a resolution proof of the unsatisfiability of Roots(Π).

3 Defining Vacuity

  • This article uses the following definition of vacuity.
  • Therefore, their techniques aim to find the k-step vacuous variables of ϕ.
  • This follows from the fact that a subformula is vacuous iff it is mutually vacuous in all of its atomic propositions [13, Th. 9], and that the definitions can be easily extended to mutual vacuity.
  • The authors now review some of the alternative definitions of vacuity and their algorithms.
  • In comparison, the authors use an “existential” definition of vacuity: a formula is vacuous if there exists a proof that does not use a subformula.

4 Exploiting Resolution Proofs

  • In Section 3, the authors discussed the existence of a sound and complete vacuity detection algorithm for BMC, which requires as many model-checking runs as there are propositional variables in the property being checked.
  • The authors propose a new vacuity detection strategy: first detect partial vacuity using inexpensive techniques and then complete the analysis using extra model-checking runs.
  • Consider a model that is composed of two completely disjoint sub-models, running in parallel, i.e.,K = K1 ‖ K2.
  • Suppose that K1 satisfies Gp, K2 satisfies Gq, and that both do so non-vacuously.
  • Any method based on examining only one resolution proof cannot prove the absence of vacuity, since another resolution proof, showing the property to be vacuous, might exist.

4.1 Examining UNSAT cores

  • Given a resolution proof that BMCk(K, ϕ) is unsatisfiable, the authors can sometimes cheaply determine that the similar theory BMCk(K, ϕ[p← x]) is also unsatisfiable, and therefore, that the property is p-vacuous.
  • Definition 2 provides an algorithm to detect some vacuous variables.
  • 1.2 Local Irrelevance Variables which do not appear in the UNSAT core are vacuous.
  • By looking at the UNSAT core, it is possible to anticipate whether a variable will not be involved in resolutions between CLK and CLe using the following definition.
  • Assume that p is locally irrelevant in BMCk(K, ϕ).

4.2 Peripherality

  • In Section 4.1, two vacuity detection methods based on examining the variables in the UNSAT core were found to fall short of completeness.
  • By applying this labeling function to the proof shown in Figure 2, the authors can determine that variable p from EXAMPLE 2 is peripheral.
  • The authors defined three methods of detecting vacuity based on examining the UNSAT core and the resolution proof produced by BMC.
  • Proof-outputting SAT solver generates the resolution proof (Π) for CLK ∪ CLe.
  • This is a new component, written in Java (around 1.3k lines of code).

6 Practical Experience

  • The authors have run VaqTree on two benchmark suites.
  • To evaluate the overall performance of the tool and the effectiveness of their partial vacuity detection methods, the authors have created a benchmark suite SA using various models and properties from the NUSMV distribution.
  • To evaluate the scalability of the tool to industrial models, the authors have created a benchmark suite SB from the models in the IBM Formal Verification Benchmarks Library [14].
  • In SA, this corresponds to a test case from the asynchronous abp4 model (roughly 30 boolean variables, with k = 19).
  • The authors full results are available in Table 1.

6.1 Results obtained with SA

  • This benchmark suite consists of 5 models: abp4, msi wtrans, pci, and prod-cell from the NUSMV distribution (107 properties) and toyFGS04 from [15] (14 properties).
  • Execution times for naive detection include CNF theory generation and satisfiability testing for each variable of the property.
  • The numbers (see Table 1) show that local irrelevance is faster than peripherality in 96% of the cases.
  • VaqTree with local irrelevance was faster than naive detection in 70 (58%) of the test cases, out of which 30 cases were twice as fast, and 20 cases were faster by an order of magnitude.
  • Points below the diagonal indicate where the X-axis method detects more vacuous variables than the Y-axis method.

6.2 Results obtained with SB

  • This benchmark suite consists of 13 models from the IBM Formal Verification Benchmarks Library [26] (18 properties).
  • At this k, some of the models where too large to analyze using VaqTree, and some of the properties did not hold.
  • Naive vacuity detection required eight model-checking runs, taking 115.68 seconds to generate the corresponding CNF theories and 2.36 seconds to test their satisfiability, requiring a total of 118.04 seconds.
  • Irrelevance took 0.36 seconds to find one of the vacuous variables during the partial pass.
  • Graphs in Figure 9 show that their techniques do in fact detect vacuity, as indicated by the points that appear below the diagonal.

6.3 Conclusions

  • In summary, the authors observed that local irrelevance performs best out of their proposed partial methods, finding most vacuity in the least amount of time.
  • On the industrial benchmark SB , the overhead produced by peripherality was negligible.
  • Thus, the authors believe that both local irrelevance and peripherality can be used to speed up naive detection.
  • The authors plan to enhance their methods by developing a heuristic based on the clause/variable ratio and proof size that indicates when naive detection should be applied instead.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Software Tools for Technology Transfer manuscript No.
(will be inserted by the editor)
Exploiting Resolution Proofs to Speed Up LTL Vacuity Detection for
BMC
Jocelyn Simmonds
1
, Jessica Davies
1
, Arie Gurfinkel
2
, Marsha Chechik
1
1
Department of Computer Science, University of Toronto
2
Software Engineering Institute, Carnegie Mellon University
The date of receipt and acceptance will be inserted by the editor
Abstract. When model-checking reportsthat a propertyholds
on a model, vacuity detection increases user confidence in
this result by checking that the property is satisfied in the in-
tended way. While vacuity detection is effective, it is a rela-
tively expensive technique requiring many additional model-
checking runs. We address the problem of efficient vacuity
detection for Bounded Model Checking (BMC) of LTL prop-
erties, presenting three partial vacuity detection methodsbased
on the efficient analysis of the resolution proof produced by
a successful BMC run. In particular, we define a character-
istic of resolution proofs peripherality and prove that if
a variable is a source of vacuity, then there exists a resolu-
tion proof in which this variable is peripheral. Our vacuity
detection tool, VaqTree, uses these methods to detect vacu-
ous variables, decreasing the total number of model-checking
runs required to detect all sources of vacuity.
1 Introduction
Model-checking [7] is a widely-used automated technique
for verification of both hardware and software artifacts that
checks whether a temporal logic property is satisfied by a
finite-state model of the artifact. If the model does not satisfy
the property, a counterexample, which can aid in debugging,
is produced. If the model does satisfy the property, no infor-
mation about why it does so is provided by the model-checker
alone. A positive answer without any additional information
can be misleading, since a property may be satisfied in a way
that was not intended. For instance, a property “every request
is eventually acknowledged” is satisfied in an environment
that never generates requests.
Vacuity detection [2,18,21,1]is an automatic sanity check
that can be applied after a positive model-checking run in or-
der to gain confidence that the model and the property cap-
ture the desired behaviours. Informally, a property is said to
be vacuous if it has a subformula which is not relevant to
its satisfaction, or if the property itself is a tautology. Con-
versely, a property is satisfied non-vacuously if every part of
the formula is important even a slight change to the formula
affects its satisfaction.
In this article, we focus on vacuity detection for SAT-
based Bounded Model Checking (BMC). Given a BMC prob-
lem with a particular bound, we wish to determine if the
property holds vacuously on the model up to this bound. In
this context, a naive method for detecting vacuity is to re-
place subformulas of the temporal logic property with un-
constrained boolean variables and run BMC for each such
substitution. If the property with some substitution still holds
on the model, the property is vacuous. This naive approach is
expensive, since in the worst case it requires as many model-
checking runs as there are subformulas in the property. Our
goal is to reduce the number of model-checking runs required
to detect vacuity. We do this by detecting some vacuity through
novel and inexpensive techniques reported in this article, and
complete the method by running the naive algorithm on the
remaining atomic subformulas. The key to our technique is
that SAT-based BMC can automatically provide useful infor-
mation (a resolution proof) beyond a decision whether the
property holds on the model; we exploit such proofs for par-
tial vacuity detection.
In SAT-based BMC, the property and the behavior of the
model are encoded in a propositional theory, such that the
theory is satisfiable if and only if the formula does not hold.
When the property does hold, a DPLL-based SAT solver can
produce a resolution proof that derives false from a subset of
the clauses in the theory called the UNSAT core. Intuitively,
the resolution proof provides an explanation of why the prop-
erty is not falsified by the model, and the UNSAT core deter-
mines the relevant parts of the model and the property [19].
In this article, we develop three methods of increasing
precision (irrelevance, local irrelevance, and peripherality)
to analyze the resolution proof to achieve partial vacuity de-
tection. These algorithms are used by our vacuity detection

tool, VaqTree,in order to reduce the number of model-checking
runs required to find all sources of vacuity, thus reducing exe-
cution times. Irrelevance and local irrelevance detect vacuity
based on which variables appear in the UNSAT core, and in
which locations. However, as these methods only examine
the UNSAT core, their precision is limited. The periphera-
lity algorithm examines the structure of the resolution proof,
identifying as vacuous those variables that are not necessary
or central to the derivation of false. This method is as pre-
cise as can be achieved through analyzing a single resolution
proof, and its running time is linear in the size of the resolu-
tion proof and the number of variables in the property. Our
experience shows that local irrelevance is the ideal candidate
for speeding up naive vacuity detection.
The remainder of the article is organized as follows. Sec-
tion 2 presents some required background, followed, in Sec-
tion 3 by our definition of vacuity, the naive algorithm for
LTL vacuity detection using BMC, and an overview of work
in the vacuity detection field. Section 4 presents the three al-
gorithms that detect vacuity by analyzing a resolution proof.
Tool support for our approach is described in Section 5. Our
experimental results are presented in Section 6. We conclude
with a summary, additional related work, and suggestions for
future work in Section 7.
2 Background
In this section, we review bounded model-checking and res-
olution proofs.
2.1 Bounded Model-Checking
Bounded model-checking (BMC) [4] is a method for deter-
mining whether a linear temporal logic (LTL) formula ϕ holds
on a finite state system represented by a Kripke structure K
up to a finite number of steps. To solve an instance of the
BMC problem, denoted by BMC
k
(K, ϕ), it is required to de-
termine whether K |=
k
ϕ, where |=
k
is the k-depth satisfac-
tion relation. Below, we give an informal overview of Kripke
structures, LTL formulas and BMC. More detailed definitions
can be found in [7,4].
A Kripke structure K has a finite set of states S, one of
which is considered to be the initial state s
0
. A transition re-
lation R S × S relates states to states. Each state is labeled
by the set of propositional formulas (or variables) that hold in
that state. A run of K is a sequence of states starting with s
0
that obeys R. Each run has an associated trace π, where π
i
is simply the set of propositional formulas that label the i
th
state in the run. We write π
i
to denote the suffix of the trace
beginning at i.
LTL formulas are built from propositional variables, the
usual boolean operators (, , ¬), and the temporal op-
erators G (“always”), F (“eventually”), U (“until”), and X
(“next”). Their semantics are defined on linear traces, such as
those produced by runs of a Kripke structure. π |= ϕ means
s
0
s
1
{p} {q}
Fig. 1: A Kripke structure.
that the trace π satisfies the LTL formula ϕ. For example,
π |= Fϕ if and only if there exists some i such that ϕ holds
on π
i
. The satisfaction relation |= is defined inductively in
a similar way for all operators and propositional variables in
LTL. We refer the reader to [7] for a detailed description of
the semantics of LTL.
A Kripke structure K satisfies an LTL formula ϕ if and
only if π |= ϕ for all traces π of K. The BMC problem
BMC
k
(K, ϕ) is to determine whether K satisfies ϕ for up to
k steps, i.e., whether K |=
k
ϕ. The k-depth satisfaction rela-
tion |=
k
is defined inductively; for example, π |=
k
Gϕ if and
only if π
i
|=
k1
ϕ for all i k.
To determine whether K |=
k
ϕ, the problem is converted
to a propositional formula Φ (see [4,6,5]) which is satisfi-
able if and only if there exists a length-k counterexample to
K |=
k
ϕ. Φ is then given to a SAT solver which decides its
satisfiability. The propositional encoding represents the be-
havior of K up to k steps with a path constraint CL
K
, and
encodes all counterexamples to ϕ of length k in an error con-
straint CL
e
. Therefore, if the theory CL
K
CL
e
is satis-
fiable, there is a path through K which obeys the transition
relation and falsifies ϕ. The value of each variable v of K at
each time step is represented using new boolean variables v
i
(0 i k), called timed variables.
The transition relation of a Kripke structure can be repre-
sented symbolically by a propositional formula over the vari-
ables V and primed variables V
(which represent the vari-
ables in the next state). For example, in the model in Fig-
ure 1, the transition relation is represented by the formula
R = (p ¬q ¬p
q
) (¬p q ¬p
q
). The path
constraint is obtained by substituting the timed variables V
i
for V in R, and replacing V
by the timed variables for the
next step, V
i+1
. This is repeated for each 0 i < k, and the
resulting propositional formulas are conjoined along with a
formula representing the initial state [4]. In Figure 1, if k = 1,
CL
K
= (p
0
¬q
0
) ((p
0
¬q
0
¬p
1
q
1
)
(¬p
0
q
0
¬p
1
q
1
)).
The error constraint CL
e
is encoded according to a recursive
procedure which removes the temporal and logical operators
from the property [4], e.g., the algorithm encodes ϕ = Gp,
where p is a propositional variable, expanded up to k = 2, by
the formula ¬p
0
¬p
1
¬p
2
.
After the boolean formulas for the path and error con-
straints are calculated, they are converted to Conjunctive Nor-
mal Form (CNF) before being passed to a SAT solver. If
the solver reports that CL
K
CL
e
is unsatisfiable, it means
that there is no length-k counterexample to ϕ; otherwise, a

(¬r
0
) (r
0
p
0
) (¬p
0
q
0
) (¬p
0
¬q
0
) (p
0
)
(p
0
)
(q
0
)
(¬p
0
)
()
Fig. 2: A resolution proof for EXAMPLE 2.
satisfying assignment is returned. When a DPLL-based SAT
solver processes an unsatisfiable theory, a resolution deriva-
tion of false (or the empty clause) is implicitly constructed [10,
27]. This resolution proof is used to verify that false can in-
deed be derived from CL
K
CL
e
[28].
2.2 Resolution Proofs
Resolution is an inference rule that is applied to propositional
clauses to produce logical consequences. A clause is a dis-
junction of literals (boolean variables or their negations). For
example, (v
1
¬v
2
v
5
) is a clause stating that at least one
of v
1
, ¬v
2
or v
5
must be true. The resolution rule takes two
clauses, where one contains a literal v and the other its nega-
tion ¬v, and produces a clause containing the union of the
two clauses’ literals minus v and ¬v. For example, resolv-
ing (v
1
¬v
2
v
5
) and (v
2
v
6
) produces the resolvent
(v
1
v
5
v
6
).
A resolution proof Π is a directed acyclic graph whose
nodes are labeled by propositional clauses. Π represents a
tree of resolutions between the clauses labeling its nodes.
Its roots are the nodes with no parents; otherwise, all nodes
have exactly two parents. The nodes with no children are
called the leaves. For example, the roots of resolution proof
Π in Figure 2 are Roots(Π) = {(¬r
0
), (r
0
p
0
), (¬p
0
q
0
), (¬p
0
¬q
0
), (p
0
)}, and the leaf of Π is the empty clause,
i.e., Leaf (Π) = false. Given a non-root node labeled by the
clause c, and the labels of its parents, c
1
and c
2
, c is the re-
solvent since it has been produced by resolving c
1
and c
2
on
some variable v. A resolution proof Π is a proof of unsat-
isfiability of a set of clauses A if and only if all roots of Π
belong to A, and one of the leaves of Π is the empty clause.
For example, Figure 2 shows a resolution proof of the unsat-
isfiability of Roots(Π). If a propositional theory in CNF is
unsatisfiable, an UNSAT core is an unsatisfiable subset of its
clauses.
Given two disjoint sets of clauses A and B, a variable v is
said to be local to A if and only if v appears in A but does not
appear in B, and v is said to be global if it appears in both
A and B. In Figure 2, if Roots(Π) = A B, where A =
{(¬r
0
), (r
0
p
0
), (¬p
0
q
0
)} and B = {(¬p
0
¬q
0
), (p
0
)},
then r
0
is local to A, and the rest of the variables are global.
3 Defining Vacuity
This article uses the following definition of vacuity.
Definition 1. Let K be a Kripke structure, ϕ be a formula
s.t. K |=
k
ϕ, and p be a variable. ϕ is k-step p-vacuous iff
K |=
k
ϕ[p x], where x is a variable not occurring in K or
in ϕ.
If ϕ is k-step p-vacuous, we call p a k-step vacuous vari-
able. A property ϕ is k-step vacuous if and only if ϕ contains
a k-step vacuous variable. Therefore, our techniques aim to
find the k-step vacuous variables of ϕ. The qualifier k-step”
is omitted in the remainder of the article but should be under-
stood implicitly in the BMC context.
Definition 1 can be generalized to vacuity in arbitrary (not
necessarily atomic) subformulas. This follows from the fact
that a subformulais vacuous iff it is mutually vacuous in all of
its atomic propositions [13, Th. 9], and that the definitions can
be easily extended to mutual vacuity. A set of atomic propo-
sitions {p
1
, ..., p
n
} is mutually vacuous if K |=
k
ϕ[p
1
x
1
, ..., p
n
x
n
], where {x
1
, ..., x
n
} are new variables. For
example, if ϕ contains subformula θ = p q, and p and q
are mutually vacuous, then we can deduce that θ is vacuous
as well.
Naive Vacuity Detection. Definition 1 suggests a sound and
complete algorithm for vacuity detection: for each proposi-
tional variable p in ϕ, run BMC on ϕ[p x], where x is a
variable that does not appear in K and ϕ. If K |=
k
ϕ[p x]
for some p, then ϕ is k-step vacuous. We refer to this algo-
rithm as naive. Its drawback is that it may require as many
model-checking runs as there are propositional variables in
ϕ.
We now review some of the alternativedefinitions of vacu-
ity and their algorithms. The first attempt to formulate and
automate vacuity detection is due to Beer et al. [2]. They
consider a property ϕ to be vacuous if ϕ contains a sub-
formula ψ such that replacing ψ by any other formula does
not affect the satisfaction of ϕ. Applying this definition di-
rectly would require an infinite number of subformula re-
placements, precluding a practical implementation. However,
Beer et al. show that to detect vacuity w.r.t. a single occur-
rence of a subformula ψ in w-ACTL, it is sufficient to replace
ψ with only true and false. This was later extended to CTL*
by Kupferman and Vardi [18], and to the modal µcalculus
by Dong et al. [9]. Purandare and Somenzi [21] showed how
to speed up subformula vacuity by analyzing the parse tree of
a CTL property.
Armoni et al. [1] generalized the above syntactic defini-
tion of vacuity by introducing universal quantification, i.e.,
x · ϕ[ψ x]. Based on the domain of x, three notions of
vacuity are obtained, the most robust of which being trace
vacuity. Gurfinkel and Chechik [12] extended Armoni’s defi-
nition of vacuity to CTL*, thus uniformly capturing CTL and
LTL. Armoni et al. also analyzed the syntactic structure of
the property in order to avoid checking the operands of sub-
formulas that are known to be vacuous. Such optimizations

complement our techniques, which focus on detecting vacu-
ous atomic subformulas.
In [20], Namjoshi has introduced a proof-based variant
of vacuity. Although it is called proof vacuity in the original
paper, we refer to it as forall-proof vacuity. This definition is
based on the semantic proofs of K |= ϕ for a Kripke structure
K and a formula ϕ. Informally, a formula ϕ is forall-proof
vacuous in a subformula ψ if ψ is not used in any proof of
K |= ϕ. Of course, a formal definition depends on the exact
interpretation of the notion of “proof”. In comparison, we use
an “existential” definition of vacuity: a formula is vacuous
if there exists a proof that does not use a subformula. Inter-
estingly, we rely on syntactic (and not semantic) resolution
proofs that may include “semantically-useless” resolutions.
As a result, it is possible that a formula ϕ is vacuous in ψ in
a model K, yet there is no resolution proof of bounded satis-
faction of K |= ϕ that does not use ψ. More importantly, our
goal is to develop a method to efficiently detect vacuity for
LTL as it was defined by [2,3,1,12], whereas Namjoshi was
looking for an alternative definition of vacuity for branching
time logic.
Our definition of vacuity is syntactic, and in this respect,
it is similar to the original definition of Beer et al. [2]. How-
ever, Definition 1 is stronger, and is equivalent to the seman-
tic definition of Armoni et al. [1], as shown by Gurfinkel and
Chechik [12].
4 Exploiting Resolution Proofs
In Section 3, we discussed the existence of a sound and com-
plete vacuity detection algorithm for BMC, which requires
as many model-checking runs as there are propositional vari-
ables in the property being checked. We propose a new vacu-
ity detection strategy: first detect partial vacuity using inex-
pensive techniques and then complete the analysis using ex-
tra model-checking runs. Since we are interested in replacing
expensive model-checking runs by inexpensive partial vacu-
ity detection methods, we limit ourselves to considering the
output of the original model-checking run on BMC
k
(K, ϕ),
i.e., CL
K
CL
e
. This run provides us with a single reso-
lution proof to analyze. Of course, in general, there may be
many ways to derive the empty clause from different sub-
sets of BMC
k
(K, ϕ). Any method that only examines one of
these derivations is inherently incomplete, in the sense that
a property may be p-vacuous but there is no way of deter-
mining this based on a given resolution proof. For example,
consider a model that is composed of two completely disjoint
sub-models, running in parallel, i.e., K = K
1
k K
2
. Suppose
that K
1
satisfies Gp, K
2
satisfies Gq, and that both do so
non-vacuously. Then the property ϕ = Gp Gq holds on K
p-vacuously and q-vacuously. However, one of the possible
resolution proofs showing that ϕ holds proves that Gp holds
non-vacuously on K
1
. Thus, it is impossible to determine that
ϕ is vacuous in p from this proof. Any method based on ex-
amining only one resolution proof cannot prove the absence
of vacuity, since another resolution proof, showing the prop-
erty to be vacuous, might exist.
In this section, we introduce three algorithms of increas-
ing precision for partial vacuity detection, based on examin-
ing the UNSAT core (irrelevance and local irrelevance) and
the resolution proof produced by BMC (peripherality).
4.1 Examining UNSAT cores
Given a resolution proof that BMC
k
(K, ϕ) is unsatisfiable,
we can sometimes cheaply determine that the similar theory
BMC
k
(K, ϕ[p x]) is also unsatisfiable, and therefore, that
the property is p-vacuous. In this section, we consider how
to determine that BMC
k
(K, ϕ[p x]) is unsatisfiable given
that BMC
k
(K, ϕ) is unsatisfiable, using only an UNSAT core.
4.1.1 Irrelevance
Intuitively, any variable that does not appear in the UNSAT
core does not contribute to the reason why ϕ holds on K, so
it can be considered irrelevant.
Definition 2. Let K be a model, and ϕ an LTL formula. As-
sume that Π is an UNSAT core of BMC
k
(K, ϕ) witnessing
that K |=
k
ϕ. Then, p is irrelevant with respect to
BMC
k
(K, ϕ) and Π iff p
i
does not appear in Π for any time
instance i.
If a variable is irrelevant, it is also vacuous, as shown by
the following theorem.
Theorem 1. If p is irrelevant with respect to BMC
k
(K, ϕ)
and Π, then ϕ is k-step p-vacuous.
Proof: Let BMC
k
(K, ϕ) = CL
K
CL
e
and U be the UNSAT
core returned by the SAT solver for BMC
k
(K, ϕ). Assume that
p is irrelevant in BMC
k
(K, ϕ). So U does not contain any p
i
by
Definition 2. Therefore, U CL
K
CL
e
implies U CL
K
CL
e
[p
i
x
i
| 0 i < k]. U is also an UNSAT core of
BMC
k
(K, ϕ[p x]) so ϕ[p x] holds on K. Thus, ϕ is p-
vacuous.
Definition 2 provides an algorithm to detect some vacu-
ous variables. However, a variable can appear in the UNSAT
core and still be vacuous, as demonstrated by the following
example.
EXAMPLE 1. Consider a Kripke structure K with variables p
and q given by the constraints Init = p q, R = p q
,
which mean that the initial state is labeled by {p, q}, and
the transition relation is expressed by the propositional for-
mula p q
over unprimed and primed variables. Let ϕ =
X(p q) be the property to check. ϕ is p-vacuous since it is
satisfied simply because q is true in any successor of the ini-
tial state. The CNF encoding of the one-step BMC problem
is CL
K
= {(p
0
q
0
), (p
0
q
1
)} = {(p
0
), (q
0
), (¬p
0
, q
1
)},
CL
e
= {(¬p
1
), (p
1
, ¬q
1
)}. In this case, the unique minimal
UNSAT core contains all of the clauses of the problem except
for (q
0
). Thus, all p
i
appear in the UNSAT core, and p cannot
be determined vacuous using irrelevance.

(p
0
) (¬p
0
, q
1
) (x
1
, ¬q
1
) (¬x
1
)
q
1
¬q
1
()
Fig. 3: A resolution proof for EXAMPLE 1.
This example shows that even if we are to look at every
UNSAT core of a BMC problem, irrelevance is still unable to
detect existing vacuity.
4.1.2 Local Irrelevance
Variables which do not appear in the UNSAT core are vac-
uous. The converse is not true: vacuous variables may also
appear in the UNSAT core. Intuitively, these variables are
not the central reason why ϕ holds on K. For example, the
clauses of CL
K
may resolve against each other, representing
some simplification and unification of parts of the model, be-
fore resolutions with CL
e
clauses are performed. If a variable
is resolved upon using only the CL
K
clauses or only the CL
e
clauses, it is potentially vacuous. By looking at the UNSAT
core, it is possible to anticipate whether a variable will not
be involved in resolutions between CL
K
and CL
e
using the
following definition.
Definition 3. Let K be a model, and ϕ an LTL formula. As-
sume that Π is an UNSAT core of BMC
k
(K, ϕ) witness-
ing K |=
k
ϕ. Then, p is locally irrelevant with respect to
BMC
k
(K, ϕ) and Π iff for each time instance i, either p
i
does
not appear in Π or p
i
is local to either CL
e
Π or CL
K
Π.
In EXAMPLE 1, p is locally irrelevant since p
1
only oc-
curs in the clauses of U taken from CL
e
, while p
0
only ap-
pears in U within CL
K
clauses. Moreover, the UNSAT core
of the original problem can be convertedto an UNSAT core of
the new theory, thus proving that p is vacuous. Specifically,
U = {(p
0
), (¬p
0
, q
1
), (¬p
1
), (p
1
, ¬q
1
)} is the UNSAT core
of the original problem, so substituting x for p in the clauses
of U that came from CL
e
gives
U
= {(p
0
), (¬p
0
, q
1
), (¬x
1
), (x
1
, ¬q
1
)}.
This is a subset of
BMC
1
(K, ϕ[p x]) = {(p
0
), (q
0
), (¬p
0
, q
1
), (¬x
1
),
(x
1
, ¬q
1
)},
so it is a candidate for the new UNSAT core. The substitution
may have prevented the resolutions necessary to derive the
empty clause. However, Figure 3 shows a proof that U
is
also unsatisfiable. In this case, it was possible to substitute x
i
for p
i
in the clauses coming from CL
e
in the original UNSAT
core and create an UNSAT core for BMC
k
(K, ϕ[p x]). In
fact, this observation applies to all cases of local irrelevance
by Theorem 2. Therefore, Definition 3 specifies an algorithm
to detect some vacuous variables.
Theorem 2. If p is locally irrelevant with respect to
BMC
k
(K, ϕ) and Π, then ϕ is k-step p-vacuous.
Proof: Let BMC
k
(K, ϕ) = CL
K
CL
e
and U be the UN-
SAT core returned by the SAT solver for BMC
k
(K, ϕ). Assume
that p is locally irrelevant in BMC
k
(K, ϕ). So for all p
i
, either
p
i
does not appear in U , or p
i
is local to CL
e
U = U
e
or
to CL
K
U = U
K
by Definition 3. Let U
e
be U
e
with each
occurence of p
i
replaced by x
i
. Since each p
i
that has been
replaced is local to U
e
, and U
K
U
e
= U is unsatisfiable, then
U
K
U
e
is also unsatisfiable. Since U
e
CL
e
[p
i
x
i
| 0
i < k], the set of clauses CL
K
CL
e
[p
i
x
i
| 0 i < k] is
unsatisfiable as well. Therefore, K |=
k
ϕ[p x] holds, so ϕ
is p-vacuous.
Unfortunately, if a variable p is not locally irrelevant in
an UNSAT core, the formula can still be p-vacuous, as shown
by the following example.
EXAMPLE 2. Consider a Kripke structure with atomic propo-
sitions r, p and q whose initial state is given by the constraint:
Init = ¬r p q. The formula ϕ = ¬p q is p-vacuous in
the initial state. Let us assume that the zero-step BMC prob-
lem is encoded in CNF as follows:
CL
K
= (¬r
0
)(r
0
p
0
)(¬p
0
q
0
)
CL
e
= (p
0
)(¬p
0
¬q
0
)
There are several resolution proofs that can establish un-
satisfiability of CL
K
CL
e
; one such proof is shown in Fig-
ure 2. In none of the proofs is p locally irrelevant with respect
to CL
e
and CL
K
.
The problem with local irrelevance is that it is impossible
to tell if a variable is going to be used in a resolution joining
CL
K
and CL
e
clauses based on the UNSAT core alone.
4.2 Peripherality
In Section 4.1, two vacuity detection methods based on ex-
amining the variables in the UNSAT core were found to fall
short of completeness. It was seen that even if every possible
resolution proof could be analyzed, irrelevance and local ir-
relevance still might fail to detect existing vacuity. Here, we
extend the analysis to the resolution proofs structure. The
resulting peripherality algorithm is superior, since it guaran-
tees vacuity will be found if all possible resolution proofs are
considered.
The limitations of detecting vacuity based only on the
UNSAT core were demonstrated in EXAMPLE 2. By exam-
ining the resolution proof in Figure 2, we see that although
p
0
appears both in CL
K
clauses and in CL
e
clauses, it is
always resolved “locally”. That is, if we resolve two clauses
c
1
= (..., p
i
, ...) and c
2
= (..., ¬p
i
, ...), p
i
and ¬p
i
must have
been preserved from their original source in some set of root
clauses. If all the originating root clauses belong to CL
K
or
all belong to CL
e
, then p
i
is being resolved on locally. In this
case, we can replace p
i
in either set of clauses without af-
fecting their unsatisfiability. For example, in Figure 2, p
0
can

Citations
More filters
Journal ArticleDOI
01 Feb 2015
TL;DR: This paper defines Temporal Antecedent Failure, an extension of antecedent failure to temporal logic, which refines the notion of vacuity, and shows how vacuity information can be derived from the automaton built for the original property, and introduces the notionof vacuity explanation.
Abstract: Different definitions of vacuity in temporal logic model checking have been suggested along the years. Examining them closely, however, reveals an interesting phenomenon. On the one hand, some of the definitions require high-complexity vacuity detection algorithms. On the other hand, studies in the literature report that not all vacuities detected in practical applications are considered a problem by the system verifier. This brings vacuity detection into an undesirable situation where the user of the model checking tool may find herself waiting a long time for results that are of no interest for her. In this paper we restrict our attention to practical usage of vacuity detection. We define Temporal Antecedent Failure, an extension of antecedent failure to temporal logic, which refines the notion of vacuity. According to our experience, this type of vacuity always indicates a problem in the model, environment or property. We show how vacuity information can be derived from the automaton built for the original property, and we introduce the notion of vacuity explanation. Our experiments demonstrate that this type of vacuity as well as its reasons can be computed with a negligible increase in the overall runtime.

8 citations

Journal ArticleDOI
TL;DR: A depth-firstsearch algorithm and a breadth-first-search algorithm are proposed to compute minimal unsatisfiable cores in SMT, adopting different searching strategy.
Abstract: Explaining the causes of infeasibility of formulas has practical applications in various fields, such as formal verification and electronic design automation. A minimal unsatisfiable subformula provides a succinct explanation of infeasibility and is valuable for applications. The problem of deriving minimal unsatisfiable cores from Boolean formulas has been addressed rather frequently in recent years. However little attention has been concentrated on extraction of unsatisfiable subformulas in Satisfiability Modulo Theories(SMT). In this paper, we propose a depth-firstsearch algorithm and a breadth-first-search algorithm to compute minimal unsatisfiable cores in SMT, adopting different searching strategy. We report and analyze experimental results obtaining from a very extensive test on SMT-LIB benchmarks.

8 citations


Cites background from "Exploiting resolution proofs to spe..."

  • ...Localizing a minimal unsatisfiable subformula is necessary to determine the underlying reasons for the failure, and is used in many practical applications, including model checking on predicate abstraction[1], vacancy detection[2], error localization[3], and synthesizing circuits[4], etc....

    [...]

Journal ArticleDOI
TL;DR: This paper proposes to enhance a UC of an unsatisfiable LTL formula with information about the time points at which the subformulas of the UC are relevant for unsatisfiability from a resolution proof and by transferring that information to a UC in LTL.

8 citations

Journal Article
TL;DR: In this article, a semi-automatic plausibility checking approach for LTL-based specifications is proposed, where the user can specify the desired behavior of the LTL specification in several smaller parts, namely by an initial truth value and one or more temporal queries (TQs).
Abstract: Many approaches for keeping business processes in line with constraints stemming from various sources (such as laws, standards, internal policies, best practices, etc.) are based on Linear Temporal Logic (LTL). Creating LTL specifications is an error-prone task, which entails the risk that the formula does not match the intention of its creator. Manual testing is time-consuming and usually limited to a small amount of test cases. This paper proposes a semi-automatic plausibility checking approach for LTL-based specifications. Additionally to the LTL formula, the user specifies the desired behavior of the LTL specification in several smaller parts, namely by an initial truth value and one or more temporal queries (TQs). TQs change the truth value once a specific pattern of events occurred in an event trace. By this approach, a large set of test cases for the LTL specification can be created automatically.

6 citations

01 Jun 2016
TL;DR: By this approach, a large set of test cases for the LTL specification can be created automatically and a semi-automatic plausibility checking approach for LTL-based specifications is proposed.
Abstract: Many approaches for keeping business processes in line with constraints stemming from various sources (such as laws and regulations) are based on Linear Temporal Logic (LTL). Creating LTL specifications is an error-prone task which entails the risk that the formula does not match the intention of its creator. This paper proposes a plausibility checking approach for LTL-based specifications. The proposed approach can provide confidence in an LTL formula if plausibility checking is passed. If the formula does not pass the plausibility checks, a counterexample trace and the truth values of both the LTL formula and the plausibility specification are generated and can be used as a starting point for correction.

6 citations


Cites background from "Exploiting resolution proofs to spe..."

  • ...Vacuity detection is concerned with avoiding tautologies and subformulas that are not relevant for the satisfaction of the formula (e.g., Simmonds et al. [20])....

    [...]

References
More filters
01 Sep 1996
TL;DR: Model checking tools, created by both academic and industrial teams, have resulted in an entirely novel approach to verification and test case generation that often enables engineers in the electronics industry to design complex systems with considerable assurance regarding the correctness of their initial designs.
Abstract: Turing Lecture from the winners of the 2007 ACM A.M. Turing Award. In 1981, Edmund M. Clarke and E. Allen Emerson, working in the USA, and Joseph Sifakis working independently in France, authored seminal papers that founded what has become the highly successful field of model checking. This verification technology provides an algorithmic means of determining whether an abstract model---representing, for example, a hardware or software design---satisfies a formal specification expressed as a temporal logic (TL) formula. Moreover, if the property does not hold, the method identifies a counterexample execution that shows the source of the problem. The progression of model checking to the point where it can be successfully used for complex systems has required the development of sophisticated means of coping with what is known as the state explosion problem. Great strides have been made on this problem over the past 28 years by what is now a very large international research community. As a result many major hardware and software companies are beginning to use model checking in practice. Examples of its use include the verification of VLSI circuits, communication protocols, software device drivers, real-time embedded systems, and security algorithms. The work of Clarke, Emerson, and Sifakis continues to be central to the success of this research area. Their work over the years has led to the creation of new logics for specification, new verification algorithms, and surprising theoretical results. Model checking tools, created by both academic and industrial teams, have resulted in an entirely novel approach to verification and test case generation. This approach, for example, often enables engineers in the electronics industry to design complex systems with considerable assurance regarding the correctness of their initial designs. Model checking promises to have an even greater impact on the hardware and software industries in the future. ---Moshe Y. Vardi, Editor-in-Chief

7,392 citations

Book
01 Jun 1992
TL;DR: Tutorial introduction background the Z language the mathematical tool-kit sequential systems syntax summary and how to use it to solve sequential systems problems.
Abstract: Tutorial introduction background the Z language the mathematical tool-kit sequential systems syntax summary.

3,547 citations

Book ChapterDOI
22 Mar 1999
TL;DR: This paper shows how boolean decision procedures, like Stalmarck's Method or the Davis & Putnam Procedure, can replace BDDs, and introduces a bounded model checking procedure for LTL which reduces model checking to propositional satisfiability.
Abstract: Symbolic Model Checking [3, 14] has proven to be a powerful technique for the verification of reactive systems. BDDs [2] have traditionally been used as a symbolic representation of the system. In this paper we show how boolean decision procedures, like Stalmarck's Method [16] or the Davis & Putnam Procedure [7], can replace BDDs. This new technique avoids the space blow up of BDDs, generates counterexamples much faster, and sometimes speeds up the verification. In addition, it produces counterexamples of minimal length. We introduce a bounded model checking procedure for LTL which reduces model checking to propositional satisfiability. We show that bounded LTL model checking can be done without a tableau construction. We have implemented a model checker BMC, based on bounded model checking, and preliminary results are presented.

2,424 citations


"Exploiting resolution proofs to spe..." refers background or methods in this paper

  • ...In this section, we review bounded model-checking and resolution proofs....

    [...]

  • ...To determine whether K | k ϕ, the problem is converted to a propositional formula (see [4–6]) which is satisfiable if and only if there exists a length-k counterexample to K | k ϕ. is then given to a SAT solver which decides its satisfiability....

    [...]

  • ...More detailed definitions can be found in [4,7]....

    [...]

  • ...Below, we give an informal overview of Kripke structures, LTL formulas, and BMC....

    [...]

  • ...The BMC problem BMCk(K, ϕ) is to determine whether K satisfies ϕ for up to k steps, i.e., whether K | k ϕ....

    [...]

Book ChapterDOI
27 Jul 2002
TL;DR: This paper describes version 2 of the NuSMV tool, a state-of-the-art symbolic model checker designed to be applicable in technology transfer projects and is robust and close to industrial systems standards.
Abstract: This paper describes version 2 of the NuSMV tool. NuSMV is a symbolic model checker originated from the reengineering, reimplementation and extension of SMV, the original BDD-based model checker developed at CMU [15]. The NuSMV project aims at the development of a state-of-the-art symbolic model checker, designed to be applicable in technology transfer projects: it is a well structured, open, flexible and documented platform for model checking, and is robust and close to industrial systems standards [6].

1,456 citations


"Exploiting resolution proofs to spe..." refers background in this paper

  • ...More detailed definitions can be found in [4,7]....

    [...]

Journal Article
TL;DR: The NuSMV tool as mentioned in this paper is a symbolic model checker developed at CMU and designed to be applicable in technology transfer projects, it is a well structured, open, flexible and documented platform for model checking, and is robust and close to industrial systems standards.
Abstract: This paper describes version 2 of the NuSMV tool. NuSMV is a symbolic model checker originated from the reengineering, reimplementation and extension of SMV, the original BDD-based model checker developed at CMU [15]. The NuSMV project aims at the development of a state-of-the-art symbolic model checker, designed to be applicable in technology transfer projects: it is a well structured, open, flexible and documented platform for model checking, and is robust and close to industrial systems standards [6].

1,377 citations

Frequently Asked Questions (11)
Q1. What contributions have the authors mentioned in the paper "Exploiting resolution proofs to speed up ltl vacuity detection for bmc" ?

The authors address the problem of efficient vacuity detection for Bounded Model Checking ( BMC ) of LTL properties, presenting three partial vacuity detection methods based on the efficient analysis of the resolution proof produced by a successful BMC run. In particular, the authors define a characteristic of resolution proofs – peripherality – and prove that if a variable is a source of vacuity, then there exists a resolution proof in which this variable is peripheral. 

The authors plan to investigate this further in the future. The authors plan to enhance their methods by developing a heuristic based on the clause/variable ratio and proof size that indicates when naive detection should be applied instead. Thus, the authors believe that both local irrelevance and peripherality can be used to speed up naive detection. 

Naive vacuity detection required eight model-checking runs, taking 115.68 seconds to generate the corresponding CNF theories and 2.36 seconds to test their satisfiability, requiring a total of 118.04 seconds. 

The authors conjecture that the poor performance is due to a low clause/variable ratio [22] which favours naive detection in cases where vacuity is not present. 

The error constraint CLe is encoded according to a recursive procedure which removes the temporal and logical operators from the property [4], e.g., the algorithm encodes ϕ = 

Since the authors are interested in replacing expensive model-checking runs by inexpensive partial vacuity detection methods, the authors limit ourselves to considering the output of the original model-checking run on BMCk(K, ϕ), i.e., CLK ∪ CLe. 

After the boolean formulas for the path and error constraints are calculated, they are converted to Conjunctive Normal Form (CNF) before being passed to a SAT solver. 

If Π is a resolution proof whose root clauses are divided into two disjoint sets,A∪B, then the labeling functionL is defined recursively as shown in Figure 4, where c is used to represent a clause. 

Armoni et al. [1] generalized the above syntactic definition of vacuity by introducing universal quantification, i.e., ∀x · ϕ[ψ ← x]. 

It might be possible to make them more effective by modifying the SAT solver to guide it to a particular kind of a proof (e.g., by changing the decision order heuristic), or to output multiple proofs (if possible). 

In this context, a naive method for detecting vacuity is to replace subformulas of the temporal logic property with unconstrained boolean variables and run BMC for each such substitution.