UC Berkeley
UC Berkeley Previously Published Works
Title
Stochastic model predictive control: An overview and perspectives for future
research
Permalink
https://escholarship.org/uc/item/1wt3d4vr
Journal
IEEE Control Systems, 36(6)
ISSN
1066-033X
Author
Mesbah, A
Publication Date
2016-12-01
DOI
10.1109/MCS.2016.2602087
Peer reviewed
eScholarship.org Powered by the California Digital Library
University of California
1066-033X /16©2016ieee
30 IEEE CONTROL SYSTEMS MAGAZINE » DECEMBER 2016
Digital Object Identifier 10.1109/MCS.2016.2602087
Date of publication: 11 November 2016
AN OVERVIEW AND PERSPECTIVES
FOR FUTURE RESEARCH
Ali MesbAh
Stochastic Model
Predictive Control
M
odel predictive control (MPC) has demon-
strated exceptional success for the high-per-
formance control of complex systems [1], [2].
The conceptual simplicity of MPC as well as
its ability to effectively cope with the complex
dynamics of systems with multiple inputs and outputs, in-
put and state/output constraints, and conicting control
objectives have made it an attractive multivariable con-
strained control approach [1]. MPC (a.k.a. receding-horizon
control) solves an open-loop constrained optimal control
problem (OCP) repeatedly in a receding-horizon manner
[3]. The OCP is solved over a nite sequence of control ac-
tions
{, ,, }uu u
N01 1
f
-
at every sampling time instant that the
current state of the system is measured. The rst element
of the sequence of optimal control actions is applied to the
system, and the computations are then repeated at the next
sampling time. Thus, MPC replaces a feedback control law
(),
$r which can have formidable ofine computation, with
the repeated solution of an open-loop OCP [2]. In fact, re-
peated solution of the OCP
confers an “implicit” feed-
back action to MPC to cope
with system uncertainties
and disturbances. Alterna-
tively, explicit MPC approaches
circumvent the need to solve an
OCP online by deriving relationships
for the optimal control actions in terms of an “explicit”
function of the state and reference vectors. However, explic-
it MPC is not typically intended to replace standard MPC
but, rather, to extend its area of application [4]–[6].
Although MPC offers a certain degree of robustness to
system uncertainties due to its receding-horizon imple-
mentation, its deterministic formulation typically renders
it inherently inadequate for systematically dealing with
uncertainties. Addressing the general OCP for uncertain sys-
tems would involve solving the dynamic programming (DP)
problem over (arbitrary) feedback control laws
()
$r [7], [8].
Solving the DP problem, however, is deemed to be imprac-
tical for real-world control applications since computational
complexity of a DP problem increases exponentially with
IMAGE LICENSED BY INGRAM PUBLISHING
DECEMBER 2016 « IEEE CONTROL SYSTEMS MAGAZINE 31
uncertainties and their interactions with the system
dynamics, constraints, and performance criteria. Param-
eterized feedback control laws allow for using the knowl-
edge of predicted uncertainties in computing
the control law, while reducing the computa-
tions to polynomial dependence on the
state dimension [16].
RMPC approaches rely on bounded, deter-
ministic descriptions of system uncertainties.
In real-world systems, however, uncertainties
are often considered to be of probabilistic nature.
When the stochastic system uncertainties can be
adequately characterized, it is more natural to explicitly
account for the probabilistic occurrence of uncertainties in a
control design method. Hence, stochastic MPC (SMPC) has
recently emerged with the aim of systematically incorpo-
rating the probabilistic descriptions of uncertainties into a
stochastic OCP. In particular, SMPC exploits the probabi-
listic uncertainty descriptions to define chance constraints,
which require the state/output constraints be satisfied
with at least a priori specified probability level—or, alter-
natively, be satisfied in expectation (see, for example, [17]–
[20], and the references therein for
various approaches to chance-con-
strained optimization). Chance
constraints enable the system-
atic use of the stochastic char-
acterization of uncertainties
to allow for an admissible level
of closed-loop constraint vio-
lation in a probabilistic sense.
SMPC allows for systemati-
cally seeking tradeoffs between
fulfilling the control objectives
and guaranteeing a probabilistic con-
straint satisfaction due to uncertainty. The ability to effec-
tively handle constraints in a stochastic setting is
particularly important for MPC of uncertain systems when
high-performance operation is realized in the vicinity of
constraints. In addition, the probabilistic framework of
SMPC enables shaping the probability distribution of
system states/outputs. The ability to regulate the probabil-
ity distribution of system states/outputs is important for
the safe and economic operation of complex systems when
the control cost function is asymmetric, that is, when the
probability distributions have long tails [21].
Stochastic optimal control is rooted in stochastic pro-
gramming and chance-constrained optimization; see, for
example, [22] and [23] for a historical perspective. The pio-
neering work on chance-constrained MPC includes [17],
[18], [24], and [25]. In recent years, interest in SMPC has
been growing from both the theoretical and application
standpoints. SMPC has found applications in many differ-
ent areas, including building climate control, power
the state dimension (known as curse of
dimensionality) [9].
The past two decades have witnessed
significant developments in the area of robust
MPC (RMPC) with the aim to devise computationally
affordable optimal control approaches that allow for the
systematic handling of system uncertainties. RMPC
approaches consider set-membership-type uncertain-
ties—uncertainties are assumed to be deterministic and
lie in a bounded set. Early work on RMPC was primarily
based on min–max OCP formulations, where the control
actions are designed with respect to worst-case evalua-
tions of the cost function and the constraints of the OCP
must be satisfied for all possible uncertainty realizations
(see, for example, [10] and [11]). Min–max MPC approaches
could not, however, contain the “spread” of state trajecto-
ries, rendering the optimal control actions overly conser-
vative or, possibly, infeasible [12]. To address the
shortcomings of min–max OCPs, tube-based MPC has
recently been developed [12]–[15]. Tube-based MPC
approaches use a partially separable feedback control law
parameterization to allow for the direct handling of
32 IEEE CONTROL SYSTEMS MAGAZINE » DECEMBER 2016
generation and distribution, chemical processes, operation
research, networked controlled systems, and vehicle path
planning. Table 1 provides an overview of various emerg-
ing application areas for SMPC; this table by no means pro-
vides an exhaustive list of SMPC applications reported in
the literature. The majority of reported SMPC approaches
have been developed for linear systems (for example, algo-
rithms based on the stochastic tube [26] or affine parame-
terization of control policy [27]). Several SMPC applications
to linear and nonlinear systems have been reported based
on stochastic programming-based approaches [28]–[30]
and Monte Carlo sampling techniques [31], [32]. Stochastic
nonlinear MPC (SNMPC) has received relatively little
attention, with only a few applications reported mainly in
the area of process control [33]–[35].
This article gives an overview of the main developments
in the area of SMPC in the past decade and provides the
reader with an impression of the different SMPC algorithms
and the key theoretical challenges in stochastic predictive
control without undue mathematical complexity. The gen-
eral formulation of a stochastic OCP is first presented, fol-
lowed by an overview of SMPC approaches for linear and
nonlinear systems. Suggestions of some avenues for future
research in this rapidly evolving field concludes the article.
NOTATION
R
n
denotes the n-dimensional Euclidean space.
:
[, ).0R 3=
+
{, ,}12
N f= is the set of natural numbers.
:
{}
.0NN,=
+
:
{,
,,
}aa b1Z
[,]ab
f=+
is the set of integers from
a
to
b
.
P
x
is
the (multivariate) probability distribution of random
variable(s)
x
.
E
denotes expectation and
:
[·]E
x
=
[·|()]xx0E
=
is the conditional expectation.
Pr
denotes probability, and
:
[·][·| () ]Pr Pr
xx
0
x
==
is the conditional probability.
:
x xAx
A
=
<
is the weighted two-norm of
x
, where
A
is a
positive-definite matrix.
GENERAL FORMuLATION OF SMPC
Consider a stochastic, discrete-time system
(, ,),xfxuw
tt
tt
1
=
+
(1a)
(, ,),yhxuv
tt
tt
= (1b)
where
t
N!
+
;
,xuRR
U
t
n
t
n
xu
!!1 , and
y
R
t
n
y
! are the
system states, inputs, and outputs, respectively;
U is a non-
empty measurable set for the inputs;
w
R
t
n
w
! and
v
R
t
n
v
!
are disturbances and measurement noise that are unknown
at the current and future time instants but have known
probability distributions
P
w
and
P
v
, respectively; and
f
and
h
are (possibly nonlinear) Borel-measurable functions that
describe the system dynamics and outputs, respectively.
For simplicity, the formulation of SMPC is presented for
the case of full state-feedback control, in which the system
states are known at each sampling time instant. Let
N
N!
be the prediction horizon, and assume that the control hori-
zon is equal to the prediction horizon. Define an
N
-stage
feedback control policy as
= {(), (),, ( )},
N01 1
$$ $fr rr r
-
: (2)
where the Borel-measurable function
()
:,RU
()
i
in
1
x
"$r
+
for
all
,,iN01
f
=-
is a general (causal) state feedback con-
trol law. At the ith stage of the control policy, the control
input
u
i
is selected as the feedback control law
(),
i
$r that is,
TABLE 1 An overview of applications of stochastic model predictive control for linear (SMPC) and nonlinear systems (SNMPC).
SMPC (Stochastic-Tube and
Affine-Parameterization
Approaches)
Stochastic
Programming-Based
SMPC
Sample-Based
SNMPC SNMPC
Air traffic control [31], [32]
Automotive applications [133] [28], [29] [134] [135]
Building climate control [27], [84]
Microgrids [136] [105]
Networked control systems [137], [138] [139]
Operation research and
finance
[69], [140]–[142] [30], [143]
Process control [17], [24], [54], [122], [138] [33]–[35], [55], [110]
Robot and vehicle path
planning
[89], [144], [145] [93] [64]
Telecommunication network
control
[146]
Wind turbine control [26]
DECEMBER 2016 « IEEE CONTROL SYSTEMS MAGAZINE 33
().u
ii
$r= In SMPC, the value function of a stochastic OCP
is commonly defined as
=
:
(,)(,)
()
,Vx JxuJxE
Nt xcii fN
i
N
0
1
t
r +
=
-
tt
=
G
/
(3)
where
:J
UR R
c
n
x
"#
+
and
:J RR
f
n
x
"
+
are the cost-per-
stage function and the final cost function, respectively, and
x
i
t
denotes the predicted states at time i given the initial states
xx
t0
=
t
, control laws
{}
,()
j
j
i
0
1
$r
=
-
and disturbance realiza-
tions
{} .w
j
j
i
0
1
=
-
The minimization of the value function (3) is commonly
performed subject to chance constraints on system outputs
(or states). Let
y
i
t
denote the predicted outputs at time i
given the initial states
xx
t0
=
t
. In its most general form, a
joint chance constraint over the prediction horizon is defined
by [36], [37]
[(), ,,], ,,,Pr gy js iN01 1for allfor all
xji
t
ff
#$b
==
t
(4)
where
:g RR
j
n
y
" is a (possibly nonlinear) Borel-measur-
able function, s is the total number of inequality constraints,
and
(, )01
!b denotes the lower bound for the probability
that the inequality constraint
()gy 0
ji
#
t
must be satisfied.
The conditional probability
Pr
x
t
in (4) indicates that the
probability that
() ,gy 0
ji
#
t
for all
,,,js1
f= for all
,,Ni 0
f= holds is dependent on the initial states
xx
t0
=
t
;
note that the predicted outputs
y
i
t
depend on disturbances
{}w
i
i
i
0
1
=
-
. A special case of (4) is a collection of individual
chance constraints [38]
[()] ,,,, ,,,Pr gy jsiN011for all
xjij
t
ff
#$b
==
t
(5)
where different probability levels
j
b
are assigned for dif-
ferent inequality constraints. Expressions (4) and (5) can be
simplified to define chance constraints pointwise in time
or in terms of the expectation of the inequality constraints
()gy 0
ji
#
t
(see, for example, [39]). In addition, state chance
constraints can be handled through appropriate choice of
the function
h
in (1b).
Using the value function (3) and joint chance constraint
(4), the stochastic OCP for (1) is formulated as follows. Given
the current system states
x
t
, the centerpiece of an SMPC
algorithm with hard input constraint and joint chance con-
straint is the stochastic OCP
:
() (, )minVx Vx
N
tN
t
r=
)
r
(6)
P
:
(, ,),
(, ),
() ,
[(), ,,],
,
,
,
,
,
,
,
Pr
xfxw
yhx
gy js
w
xx
i
i
i
i
i
01
such that
forall
forall
forall
forall
forall
forall
U
Z
Z
Z
Z
Z
[, ]
[, ]
[, ]
[, ]
[, ]
iiii
iii
i
xji
iw
t
N
N
N
N
N
1
0
01
0
01
1
01
t
$
f
!
#$
+
!
!
!
!
!
r
r
r
b
=
=
=
=
+ -
-
-
tt
tt
t
t
where
()Vx
N
t
)
denotes the optimal value function under the
optimal control policy
*r
. The receding-horizon implemen-
tation of the stochastic OCP (6) involves applying the first
element of the sequence
*r
to the true system at every time
instant that the states
x
t
are measured, that is,
().u
t
0
$r=
*
The key challenges in solving the stochastic OCP (6)
include 1) the arbitrary form of the feedback control laws
(),
i
$r 2) the nonconvexity and general intractability of
chance constraints [40], [41], and 3) the computational
complexity associated with uncertainty propagation
through complex system dynamics (for example, nonlin-
ear systems). In addition, establishing theoretical proper-
ties, such as recursive feasibility and stability, of the
stochastic OCP (6) poses a major challenge. Numerous
SMPC approaches have been developed to obtain tracta-
ble surrogates for the stochastic OCP (6). Table 2 summa-
rizes the key features based on which SMPC approaches
can be broadly categorized. In subsequent sections, vari-
ous SMPC formulations will be analyzed in light of the
distinguishing features given in Table 2. Broadly, SMPC
approaches can be categorized in terms of the type of
system dynamics, that is, linear or nonlinear dynamics.
SMPC approaches for linear systems are further catego-
rized based on three main schools of thought: stochastic-
tube approaches [42]–[46], approaches using affine
parameterization of the control policy [47]–[56], and sto-
chastic programming-based approaches [56]–[62]. There
has been much less development in the area of SMPC for
nonlinear systems. The main contributions in this area
can be categorized in terms of their underlying uncer-
tainty propagation methods, namely sample-based
approaches [31], [32], [63], Gaussian-mixture approxima-
tions [64], generalized polynomial chaos (gPC) [33], [34],
[65], and the Fokker–Planck equation [35], [66], [67]. It is
worth nothing, however, that a unique way to classify the
numerous SMPC approaches reported in the literature
does not exist. It has been attempted throughout the dis-
cussion to contrast the various SMPC approaches in
terms of the key features listed in Table 2.
SMPC FOR LINEAR SYSTEMS
Much of the literature on SMPC deals with stochastic linear
systems. For linear systems with additive uncertainties, the
general stochastic system (1) takes the form
,xAxBuDw
tt
tt
1
=++
+
(7a)
,yCxFv
ttt
=+
(7b)
where
,,,,ABCD
and
F
are the state-space system matri-
ces, and the disturbance
w
and measurement noise
v
are
often (but not always) assumed to be sequences of indepen-
dent, identically distributed (i.i.d.) random variables. For
linear systems with multiplicative uncertainties, the system
matrices in (7a) consist of time-varying uncertain elements
with known probability distributions