546 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006
Reachability Analysis of Discrete-Time Systems
With Disturbances
Sa
ˇ
sa V. Rakovic
´
, Eric C. Kerrigan, Member, IEEE, David Q. Mayne, Fellow, IEEE, and John Lygeros, Member, IEEE
Abstract—This paper presents new results that allow one to com-
pute the set of states that can be robustly steered in a finite number
of steps, via state feedback control, to a given target set. The as-
sumptions that are made in this paper are that the system is dis-
crete-time, nonlinear and time-invariant and subject to mixed con-
straints on the state and input. A persistent disturbance, depen-
dent on the current state and input, acts on the system. Existing
results are not able to address state- and input-dependent distur-
bances and the results in this paper are, therefore, a generaliza-
tion of previously published results. One of the key aims of this
paper is to present results such that one can perform the relevant
set computations using polyhedral algebra and computational ge-
ometry software, provided the system is piecewise affine and the
constraints are polygonal. Existing methods are only applicable to
piecewise affine systems that either have no control inputs or no
disturbances, whereas the results in this paper remove this limita-
tion. Some simple examples are also given that show that, even if
all the relevant sets are convex and the system is linear, convexity
of the set of controllable states cannot be guaranteed.
Index Terms—Controllability, control systems, nonlinear sys-
tems, Piecewise affine systems, reachability analysis, robustness,
set invariance.
I. INTRODUCTION
T
HE problems of reachability, invariance and control in-
variance for discrete-time systems have been extensively
studied in the literature for over four decades (see [1]–[6] for
some seminal papers on the subject). Recently these problems
have attracted renewed attention, partly because improvements
in computational capabilities have made it possible to imple-
ment the algorithms for systems of practical interest (see, for
instance, an excellent survey paper [7] for more details and a set
of relevant references). Another reason for the renewed interest
in these problems is the emergence of new classes of practically
important systems, such as hybrid systems. These are systems
whose states, inputs and outputs can take on values from both a
countable set (e.g., the set of integers) as well as an uncountable
set (e.g., the set of real numbers). In recent years, invariance and
Manuscript received December 23, 2004; revised August 22, 2005. Recom-
mended by Associate Editor C. T. Abdallah. This research was supported in
part by the Engineering and Physical Sciences Research Council and the Royal
Academy of Engineering, U. K., and in part by the European Commission under
the HYCON Network of Excellence Contract FP6-IST-511368.
S. V. Rakovic
´
and D. Q. Mayne are with the Department of Electrical and
Electronic Engineering, Imperial College London SW7 2BT, U.K. (e-mail:
sasa.rakovic@imperial.ac.uk; d.mayne@imperial.ac.uk).
E. C. Kerrigan is with the Department of Electrical and Electronic Engi-
neering and the Department of Aeronautics, Imperial College London SW7
2BT, U.K.(e-mail: e.kerrigan@imperial.ac.uk).
J. Lygeros is with the Systems and Control Division, Department of Elec-
trical and Computer Engineering, University of Patras, Patras GR26500, Greece
(e-mail: lygeros@ee.upatras.gr).
Digital Object Identifier 10.1109/TAC.2006.872835
reachability problems for classes of hybrid systems have been
studied by a number of authors [8]–[15].
One class of systems that, to the authors’ knowledge, has re-
ceived relatively little attention are systems with mixed con-
straints on the states, control inputs and disturbances. When this
class of systems is treated, it is often with an insufficient amount
of detail and overly conservative approximations. Systems with
mixed state, control and disturbance constraints may arise in
practice for a number of reasons.
1) When modeling systems with physical constraints.
Here the model must reflect the fact that the constraints
will be satisfied by all evolutions of the system, what-
ever the control inputs and disturbances.
2) When designing controllers to meet safety or perfor-
mance specifications, i.e., to ensure that the state of the
system remains in a certain region of the state space.
Safety and performance specifications may be violated
if the inputs are not chosen properly.
A couple of simple examples illustrate the point. Consider the
following discrete-time model for the longitudinal motion of a
car on a highway:
where represents the position of the car, its ve-
locity,
represents the control acceleration applied by
the engine or brakes, and
a disturbance acceleration
due to wind. It is assumed that
and .For
simplicity, all other constants have been normalized to 1.
One would like to capture the situation where the vehicle is
prevented from going backward. This is a reasonable require-
ment in many cases (e.g., on a highway) and is very easy to
implement in practice (assuming that the wind is incapable of
pushing the car backward when the brakes are applied one could
simply disallow the reverse gear). This can be captured by the
hard state constraint
. To enforce this constraint, the model
needs to incorporate the additional state-dependent constraint
on the inputs (control and disturbance).
For another example, consider the following piecewise affine
system:
(1)
which is subject to a bounded disturbance
. The func-
tion
models physical saturation limits on the input. As-
suming that these saturation limits are symmetric and have unit
0018-9286/$20.00 © 2006 IEEE
RAKOVI et al.: REACHABILITY ANALYSIS OF DISCRETE-TIME SYSTEMS WITH DISTURBANCES 547
magnitude, an equivalent way of modeling (1) is to treat it as
linear system with an input-dependent disturbance, i.e., letting
(2)
where the control is constrained to
(3)
and the input-dependent disturbance satisfies
, where
(4)
Another common reason why state- and input-dependent dis-
turbances arise in practice is when it is known that the uncer-
tainty of a model is greater in certain regions of the state-input
space than in other regions. For example, when a nonlinear
model is linearized, the uncertainty gets larger the further one
gets from the point of linearization. This uncertainty can be
modeled as a state- and input-dependent disturbance, where the
size of the disturbance decreases the closer one gets to the point
of linearization. A state- and input-dependent disturbance model
will therefore allow one to obtain less conservative results than
if one were to assume that the disturbance is independent of the
state and input.
Another example when one can model uncertainty as a state-
and input-dependent disturbance is when there is parametric un-
certainty present in the model. For example, if there is uncer-
tainty in the pair
in (2), then one can think of the uncer-
tainty as an additional state- and input-dependent disturbance.
The reader is referred to [16] to see how reachability compu-
tations can be carried out for this specific class of uncertainty
when the system is linear. The results in this paper can, with
some effort, be used to extend the results in [16] to the class of
piecewise affine systems with parametric uncertainty.
More generally, consider state variables
, control variables
and disturbance variables , taking values in the sets , , and
, respectively. Consider dynamic constraints on these vari-
ables of the form
and (5)
where
and . Here, is assumed
to capture the physical, state-dependent constraints on the con-
trol and disturbance inputs. The goal is to develop methods for
designing controllers for this class of dynamical systems.
Though fairly general results exist that can be applied to a
large class of nonlinear discrete-time systems, to our knowl-
edge, none of these control and analysis algorithms are capable
of explicitly dealing with this class of problems. For example,
most authors assume that the disturbance is not dependent on
the state and input—the only paper which addresses state-de-
pendent disturbances directly (for linear systems) is [17].
The key tool that allows one to perform a reachability anal-
ysis (often also called a controllability analysis), is software for
implementing the so-called predecessor operator, which allows
one to compute the set of states that can be robustly steered
(using an admissible control input) to a given target set in a
single step. The predecessor operator is then called in a recursive
fashion in order to compute the set of states that can be robustly
steered to the given target set in a finite number of steps.
A direct way of approximating the computation of the prede-
cessor set is to grid the state-input-disturbance space, effectively
approximating the original system by a finite state-input-distur-
bance system. Clearly, this approach has computational com-
plexity drawbacks, since the computation grows exponentially
with the dimension of the state, input and disturbance spaces.
Moreover, even though results exist guaranteeing asymptotic
convergence to the real set as the grid gets finer, in practice it
is not always clear how fine or coarse the grid needs to be in
order to have sufficiently accurate results.
A more elegant approach is to use symbolic algebra software
and/or quantifier elimination methods [15], [18]–[20]. The idea
here is to encode the predecessor computation in an appropriate
system of logic using quantifiers to capture requirements that
need to hold for some control actions, all disturbance, at some
or for all times, etc. Computational tools [21], [22] can then be
used to eliminate the quantifiers in these formulas and derive
quantifier free formulas that define the set of states where the
requirements hold (e.g., the predecessor set). For many classes
of systems this approach is exact and does not involve any ap-
proximation. Moreover, the quantifier elimination approach is
very general. In addition to linear and piecewise linear/affine
systems (on which the computational methods proposed in this
paper mostly apply), quantifier elimination methods can also be
applied to a considerably more general class of discrete-time
systems, for example systems whose dynamics and constraints
are encoded by piecewise polynomial functions. The limits of
the applicability of this approach to continuous-time systems are
investigated in [23], where methods for using systems amenable
to the quantifier elimination approach to approximate even more
general classes of systems are also discussed.
The main drawback of methods based on quantifier elimina-
tion is their complexity. It is known that general purpose quan-
tifier elimination is worst case doubly exponential in the size of
the input and output data. For the classes of problems consid-
ered here and under some conditions (e.g., absence of control
and/or disturbance variables), one can exploit structure present
in the formulas used to encode the predecessor computation to
get better performance [24], [25]. Worst case bounds are still ex-
ponential, even though the running times observed in practice
are typically much faster. [26] presents the results in this line
of work that are most closely related to our study. In this refer-
ence, the special structure afforded by piecewise linear functions
is exploited to derive algorithms with very reasonable running
times, reasonable enough to allow their application to realistic
problems in network monitoring. For other cases of the applica-
tion of symbolic methods to problems in control theory (equilib-
rium computation, stabilization, tracking) the reader is referred
to [18], [27], and [28].
It is well-known that if the system
is linear or piece-
wise affine and the relevant constraints sets (e.g.,
) are poly-
gons, then standard software for polytope manipulation can be
used for reachability analysis [7], [13], [29]. There are a number
548 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006
of benefits that can be obtained from using computational ge-
ometry software, rather than gridding the state–space or using
quantifier elimination and computer algebra packages.
• Many algorithms for performing fundamental opera-
tions on polyhedra have a computational complexity
that is a polynomial function of the size of the input
and output data [30]. As mentioned above, many quan-
tifier elimination algorithms do not have this property
and their computational complexity is often doubly-ex-
ponential with respect to the size of the input and/or
output data. For numerical methods based on gridding
the computation is typically exponential in the dimen-
sion of the state, input and disturbance spaces for fixed
accuracy.
• Software for manipulating polyhedra exploit the
structure of the problem, whereas gridding and gen-
eral-purpose quantifier elimination packages do not
always do this. See [31] for some results that show
how, by exploiting the structure when computing
projections of polytopes, a geometric approach can
reduce the computational requirements by a number
of orders of magnitude, compared to quantifier elimi-
nation methods such as Fourier elimination.
• It is often easier to visualize, understand and imple-
ment the results and exploit any structure, whereas it is
not always so clear how to proceed with an approach
that is not geometric.
One of the key aims of this paper is to present results such
that one can perform the relevant reachability computations
using polyhedral algebra and computational geometry software,
provided the system is piecewise affine and the constraints are
polygonal. Existing methods for piecewise affine systems are
limited to systems that either have no control input or no
disturbance [13], whereas the results in this paper remove
this limitation. The extension of these results is not trivial;
we will show, via some examples, that even if all the relevant
sets are convex and the system is linear, convexity of the set
of controllable states cannot be guaranteed if there are mixed
constraints on the state, input and disturbance.
This paper is organized as follows. The problem definition is
given in Sections II-A and B, which relates the problem defini-
tion with some well-known results on set invariance. The main
result for the computation of the predecessor set is presented
in Section II-C, topological properties of the predecessor set
are discussed in Section II-D and special cases are discussed
in Section II-E. Section III highlights the fact that the reacha-
bility analysis can be carried out using polyhedral algebra if the
system is piecewise affine and the relevant sets are polygons. To
validate the results, Section IV presents a few simple numerical
examples. The main contributions of this paper are summarized
in Section V. Appendix I contains some results regarding con-
tinuity of set-valued maps and Appendix II gives some new re-
sults that allow one to compute the set difference of (possibly
nonconvex) polygons.
Note that some of the results given in this paper, namely for
the case where the disturbance is independent of the state and
input, were originally reported in the thesis [32, Ch. 4] and the
conference papers [33] and [34]. The conference paper [35] and
the thesis [36] significantly extended these results to cover the
more general case of state- and input-dependent disturbances;
this paper follows a similar line of development. The results in
[32]–[34] are summarized in Section II-E.3.
II. G
ENERAL CASE
To keep the notation as simple as possible and maintain a
large degree of generality, we will adopt a nonlinear approach
for a large part of this paper. Definitions and results for inter-
esting special cases, for example when the system is piecewise
affine or the constraints on the disturbance are independent of
the state, will be introduced where appropriate.
Given two sets
and , the reflection of
through the origin is , the complement
of
in is , the set difference
between
and is ,
the Minkowski set addition of
and is
and the Pontryagin difference be-
tween
and is for all .
Given a set
, the (or-
thogonal) projection of the set
onto is defined as
such that . The set
of nonnegative integers is denoted by
.
A. Definitions
Consider the problem of controlling a nonlinear discrete-time
system in the form
(6)
where
is the current state (assumed to be measured), is the
state at the next time instant,
is the current input, and is an
uncertain parameter, which shall be referred to as the “distur-
bance,” and may change from one sample to the next.
The disturbance takes on values in a set, which is dependent
on the current state and input, i.e.,
(7)
where
denotes the disturbance space. We say that
the disturbance is independent of the state and input if the set
for all and will
use the notation
to denote this fact. A distur-
bance that is dependent only on the state or input is defined
in a similar fashion and the notation
and
, respectively, will be used to denote this. We
define the “nominal/no disturbance” case when
for all . Note that the set does not directly depend
on previous values of the disturbance. However, constraints of
this type (used, for example, to encode rate constraints on the
disturbance or the disturbance dynamics) can be included, in
cases when it is possible to measure them, by appropriately ex-
tending the state to include past disturbance values. A similar
comment extends to the input constraints.
The state and input are required to satisfy a set of mixed con-
straints
(8)
RAKOVI et al.: REACHABILITY ANALYSIS OF DISCRETE-TIME SYSTEMS WITH DISTURBANCES 549
where is the state space and is the input
space. These constraints typically arise due to physical limita-
tions, desired levels of performance or safety considerations.
Combining this constraint with the previous constraint on the
disturbance, let
and (9)
be the subset of the graph of
where the constraints on the
state and input are also satisfied. In order to have a well-defined
problem, we have the standing assumption that
for all , hence
(10)
The state-dependent set of admissible inputs can now be defined
as
(11)
The set of admissible states is then
such that
(12)
If the state and input constraints are not coupled, then we will
use the notation
or to denote this.
Remark 1: Note that for the case when a feedback control
law
is applied to (6), by considering
with and , where
and , the
required reachability analysis follows the procedure outlined in
Section II-E.
Often part of the control objective is to guarantee robust con-
vergence to a given set, either in minimum time, some finite time
or asymptotically. Let
denote this so-called target set (also
often called terminal constraint set) and, without loss of gener-
ality, assume that
(13)
One of the key aims of this paper is to present results that
allow for the computation of the set of initial states for which
a time-varying state feedback control law exists such that the
constraints on the state and input (8) are robustly satisfied (for
all allowable disturbances) over a finite horizon and that the state
is guaranteed to be in
at the end of the horizon.
Let
denote a control policy
(sequence of control laws, i.e.,
, )
over a horizon of length
and let
denote a sequence of disturbances. Also, let denote
the solution of (6) when the state is
at time 0 (since the system
is time-invariant, we can always take the current time to be zero),
the control policy is
and the disturbance sequence is .
For a given current state
and policy , let be the
set of admissible disturbance sequences of length
, i.e.,
(14)
Clearly, if the disturbance is independent of the state and input,
then
for all .
Next, let
be the set of admissible policies of length ,
i.e., those policies that satisfy, for all
, the state
and control constraints (8) over the horizon
,
and the terminal constraint
(15)
In other words, the set of admissible policies is defined as
(16)
The set
is the set of initial states for which an admissible
policy of length
exists (often also called the -step control-
lable set) and is defined as
(17)
B. Reachability Analysis and Invariant Sets
Before proceeding to give our main result, we first recall a few
well-known results that link reachability analysis to the compu-
tation of invariant sets. Central to this discussion is the so-called
predecessor set (or one-step set) of a given set.
Definition 1 (Predecessor Set): Given a set
, the pre-
decessor set
is the set of states for which there exists an
admissible input such that, for all allowable disturbances, the
successor state is in
, i.e.,
such that
for all (18)
An equivalent formulation of (18) is
such that
(19)
where
.
For any integer
, let denote the -step predecessor set
to
, i.e., is the set of states that can be steered, by a
time-varying state feedback control law, to the target set
in
steps, for all allowable disturbance sequences while satisfying,
at all times, the constraint
. In other words, is
given by (17) with
. Following a standard procedure [4],
the sequence of sets
may be calculated recursively as
follows:
(20a)
(20b)
Recall that a given set
is defined to be robust control
invariant [7] if for any
, there exists a such that
for all . A robust control invariant
set
is called maximal in if all other robust control
invariant sets in
are contained in .
550 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006
We are now in a position to state some important, well-known
results that link the recursion in (20) to its use in the computation
of invariant sets. Since it is beyond the scope of this paper to give
a detailed literature review of this subject, we refer the reader to
the surveys [7] and [29] for a detailed discussion. In this paper,
we would like to highlight the following results.
Proposition 1 (Results on Set Invariance):
i)
There exists a unique robust control invariant set
that is maximal in , provided that
is nonempty.
ii)
A given set
is robust control invariant if and only if
.
iii)
is robust control invariant for all if and only
if
is robust control invariant.
iv)
If
, then for all and
the maximal robust control invariant set
satisfies
. Furthermore, for a given
if and only if .
Remark 2: If the system has no input
, i.e., if is a func-
tion only of
, then Proposition 1 still holds with the ap-
propriate modifications to definitions, but with “robust control
invariant” replaced with “robust positively invariant” [7].
Remark 3: Without any additional assumptions on the
system or sets, it is possible to find examples for which
if [6].
Itisclearthatresults that enable onetocomputethepredecessor
set also allow one to compute each of the sets in the sequence
. Furthermore, as will be shown below in Corollary 2,
one can also employ the predecessor operator via the recursion
(20) to compute an arbitrarily close approximation to the max-
imal robust control invariant set
, provided some additional
compactness and continuity assumptions are satisfied. Finally,
the computation of the predecessor set plays a crucial role in al-
lowing one to compute optimal control laws for piecewise affine
discrete-time systems with disturbances [34], [37], [38].
C. Main Result
As discussed in the introduction, the main aim of this paper is
to provide results that allow one to use computational geometry
packages for computing the predecessor set. Due to the fact that
existing computational geometry software do not provide gen-
eral tools for the direct elimination of the universal quantifier in
an expression, one first has to obtain an equivalent expression for
the predecessor set that only contains the existential quantifier.
The elimination of the existential quantifier can then be achieved
by computing the projection of an appropriately defined set. Of
course, any suitable quantifier elimination software may also be
used to compute the projection. However, as mentioned in the
Introduction, we are not aware of quantifier elimination methods
withacomputationalcomplexitybound that isapolynomial func-
tionoftheinputand outputdata,whereas computationalgeometry
methods exist with polynomial complexity bounds.
Before proceeding to state our main result, we define
for all
(21)
Fig. 1. Graphical illustration of Theorem 1.
the set of admissible state-input pairs for which the state of the
system at the next sample instant is in a given set
for all
admissible disturbances, and
(22)
the set of state-input-disturbance triplets for which the state of
the system evolves to a given set
at the next time instant.
Note that the sets
and are also functions of the set
as evident from their definitions; however, in order to simplify
notation in the sequel of this paper we simply write
and but
we bear in mind that
and .
We are now in a position to state our main result, originally
presented in [35] and [36].
Theorem 1 (Predecessor Set):
, the set of states that
are robustly controllable to
in one step, is given by
(23)
where
is given by
(24)
Proof: A graphical interpretation of the proof is given in
Fig. 1.
From the definition of the set difference
(25)
so that
such that (26)
It follows that
for all (27)
The proof is completed by noting that
such that and
for all (28a)
such that
for all (28b)
(28c)
