1
Foundations of Control and Estimation over Lossy Networks
Luca Schenato
1
, Member, IEEE Bruno Sinopoli
2
, Member, IEEE, Massimo Franceschetti
3
, Member, IEEE
Kameshwar Poolla
2
,and Shankar Sastry
2
, Fellow, IEEE
1
Department of Information Engineering, University of Padova, Italy
2
Department of Electrical Engineering, UC Berkeley, Berkeley, CA, USA
3
Department of Electrical and Computer Engineering UC San Diego, La Jolla, CA, USA
schenato@dei.unipd.it
{sinopoli,poolla,sastry}@eecs.berkeley.edu
massimo@ece.ucsd.edu
Abstract—When data are transmitted to an estimation-control
unit over a network, and control commands are issued to
subsystems over the same network, both observation and control
packets may be lost or delayed. This process can be modeled by
assigning probabilities to successfully receive packets. Determin-
ing the impact of this uncertainty on the feedback-loop requires
a generalization of classical control theory. This paper presents
the foundations of such new theory.
Motivations and overview of the efforts of different research
groups are described first. Then, novel contributions of the
authors are presented. These include showing threshold behav-
iors which are governed by the uncertainty parameters of the
communication network: for network protocols where successful
transmissions of packets is acknowledged at the receiver (e.g.
TCP-like protocols), there exists critical probabilities for the
successful delivery of packets, below which the optimal controller
fails to stabilize the system. Furthermore, for these protocols,
the separation principle holds and the optimal LQG control is
a linear function of the estimated state. In stark contrast, it
is shown that when there is no acknowledgement of successful
delivery of control packets (e.g. UDP-like protocols), the LQG
optimal controller is in general nonlinear.
I. INTRODUCTION
The increasingly fast convergence of sensing, computing
and wireless communication on cost effective, low power,
thumb-size devices, is quickly enabling a surge of new control
applications. In recent years, we have already witnessed the
wireless infrastructure overshadowing its wired counterpart
in all applications where it could be securely and reliably
implemented. Glamorous is the case of cellular telephony,
that is progressively substituting wireline telephony. So has
happened to LAN access, now dominated by WI-FI. Doomed
to fall next is wired broadband access, such as DSL, with the
advent of WIMax and 3G wireless data services. The process
is likely to continue with the advent of sensor technology.
Everything is getting “sensed:” vehicles, roads, buildings,
airspaces, environment, and so on. This ability to collect data
over a network at a very fine temporal and spatial granularity,
and the ability to process such data in real-time and then
perform appropriate control actions, opens to the development
of new applications [1][2][3].
This research is supported in part by DARPA under grant F33615-01-C-
1895, by the European Community Research Information Society Technolo-
gies under Grant No. RECSYS IST-2001-32515, and by the Italian Ministry
of Education, University and Research (MIUR).
Why not have real time alarm systems for catastrophic, yet
predictable events, such as tsunamis, landslides, train crashes?
What we used to regard as unforeseeable events are merely
combinations of other events that we can now observe. Why
not have efficient controllers for electric grids, exchanging load
information between local stations, to optimize delivery and
avoid costly and dangerous blackouts? A main issue that needs
to be addressed to realize this vision is the the development
of theoretical foundations of remote control over unreliable
networks. We can fully benefit from the ability to collect an
enormous amount of data from the physical world only if
we can analyze the behavior of control processes acting over
networks.
The benefits of pervasive networking and sensing are clear.
For example buildings, both residential and commercial, can
greatly benefit from the use of sensor networks, by decreasing
construction and operating costs, while improving comfort
and safety. Today, more than half of the cost of an Heating,
Ventilation, Air Conditioning (HVAC) system in a building
is represented by installation and most of it is wiring. Wire-
less communication could sensibly lower this cost [4][5].
Moreover, combining wireless technology with Micro Electro
Mechanical Systems (MEMS) technology could reduce the
cost further, allowing sensors to be embedded in products such
as ceiling tiles and furniture, and enable improved control
of the indoor environment[6]. On the operating cost, such
systems could dramatically improve energy efficiency. The
United States is the bigger consumer of energy with 8.5
quadrillion British Thermal Units (BTU). Commercial and
residential sectors account for about 40% of total consumption,
according to a study conducted by the Energy Information
Administration in March 2004. With oil and gas prices rising
and not likely to decrease anytime soon, it is imperative to
find ways to decrease consumption by avoiding useless waste.
Another example where pervasive wireless technology will
have a high impact is Supervisory Control And Data Ac-
quisition (SCADA) networks. These networks, were origi-
nally developed in the 1960s, and are used for industrial
measurement, monitoring, and control systems, especially by
electricity and natural gas utilities, water and sewage utilities,
railroads, telecommunications, and other critical infrastructure
organizations. They enable remote monitoring and control of
a large variety of industrial devices, such as water and gas
2
pumps, track switches, and traffic signals.
SCADA systems typically implement a distributed system
whose elements are called points. A point can be a single
input or output value, monitored or controlled by the system. A
variety of host computers allow for “supervisory level” control
of the remote site. Great part of the control takes place at
distributed locations called Remote Terminal Units (RTUs).
RTUs connects to physical equipment such as switches, pumps
and other devices, and monitor and control these devices.
SCADA systems often have Distributed Control System (DCS)
components. In this case smart RTUs are employed, capa-
ble of performing autonomous control and decision without
the intervention of the master computers. The role of host
computers is generally restricted to supervisory level control.
Data acquisition begins at the RTU level and includes meter
readings and equipment statuses that are communicated to the
SCADA as required. Data is then compiled and formatted in
such a way that a control room operator using the SCADA can
make appropriate supervisory decisions that may be required
to over-ride normal RTU controls. SCADA systems have tra-
ditionally used combinations of different infrastructure to meet
communication requirements. The existence of a consolidated
wired legacy infrastructure hinders the development of open
systems based on wireless technology, that would provide
superior performance and lower costs, easier maintenance and
upgradability. Most of the remote monitoring and control
application could run over the wireless infrastructure, while
components could be easily swapped without any service
interruption.
A third example of application of wireless sensor technol-
ogy is represented by in-car networks. Electronics is quickly
becoming a main differentiator in the automotive industry,
with companies offering electronic services, from Global Po-
sitioning System (GPS) in-vehicle safety and security system,
to DVD, to drive-by-wire systems. These enhancements of
course come at a price. Electronic systems now account for a
sizeable part of the cost and weight of a vehicle. Cars have
over 50 embedded computers running a variety of applications,
from safety-critical to pure entertainment. In addition, these
applications consist of sensors, actuators and controllers that
are spatially distributed in the vehicle. These components
communicate using dedicated wires, bringing the length of
wires in high-end luxury cars to amount for more than three
miles and adding over two hundred pounds of weight to the
vehicle. As electronics is only likely to in increase in cars, with
new services and applications, this design scheme will not be
sustainable for long. In vehicle networking will become essen-
tial and a prime application of networked embedded systems
theory. As many applications concur in sharing computing and
communication resources, issues of scheduling, network delay
and data loss will need to be dealt with.
Beyond these examples there is a whole new and unexplored
terrain, where any engineer can exploit his/her imagination.
There is a surge of new startups trying to carve a niche in new
markets, and established companies trying to take advantage of
the new technology to improve their offerings while creating
new products and services.
By looking at all applications mentioned above, a common
modus operandi is revealed, which is typical of networked
control systems. Data is sent from possibly multiple sensors
to one or more computing units, using a communication
network. Such data is then processed to estimate the state
of a dynamical phenomenon, and control inputs are sent to
actuators, again through the network. Both measurements and
inputs have very stringent time constraints, depending on the
system dynamics, that the network needs to be able to satisfy.
Placing a communication network in the control loop raises
many issues. One of the key parameters in digital control
systems design is the selection of a fixed sampling period. This
is mainly a function of the system dynamics, and it places a
hard constraint on the time necessary to receive observations,
estimate the state, compute an input, and transmit it to the
actuators. All of this needs to happen within one sampling
interval. Computing power of modern machines, combined
with usually wired, dedicated interconnection between differ-
ent parts of the system, guarantees that such constraints are
met. When closing the loop around wireless sensor networks,
the assumption of data availability does not hold anymore,
as packets are randomly dropped and delayed. While system
and control theory provide a wealth of analytical results, the
assumptions that the theory is traditionally based upon do not
hold true in this setting, and neglecting these phenomena may
yield to catastrophic system performance. A notion of time,
either global or local, is needed to order and combine possibly
different sensor data for state estimation. The estimator needs
to know what to do when observations are not arriving, and
the controller needs to design an input using uncertain state
estimates, not knowing whether its previous input has been
successfully received by the actuators.
More generally, the use of networks in control systems
imposes a paradigm shift in the engineer’s mentality. Deter-
ministic methods need to be replaced by stochastic ones, as
such is the nature of the network phenomena. This argument
is particularly true in wireless networks, where the use of a
shared channel with random disturbances and noise cannot be
modelled deterministically.
This paper attempts to place the theoretical foundations for
the design of estimation and control systems over networks.
II. CONTROL OVER NETWORKS
A. Foundations
There are a number of basic problems that arise when at-
tempting to realize the vision of pervasive wireless networking
described above. Wireless networks are inherently less reliable
and secure than their wired counterparts. Penetration of wire-
less technology in modern society will be limited by these two
factors. For example, car manufacturers today are reluctant
to put wireless networks in cars, especially if connecting
highly critical systems, e.g. braking, steering, accelerating etc.
Loss of data may have a disastrous effect on the behavior of
the vehicle. Similarly, in SCADA systems, which represent
the standard control infrastructure in industrial processes and
also in some experimental facilities such as nuclear fusion,
communication is ethernet based, and it is likely to remain so
until we can guarantee acceptable performance and security.
3
In short, applications need to be designed robust enough to
cope with unreliability in the network.
Issues of communication delay, data loss, and time-
synchronization play critical roles. In particular, communi-
cation and control are tightly coupled and they cannot be
addressed independently. Specific questions that arise are the
following. What is the amount of data loss that the control loop
can tolerate to reliably perform its task? Can communication
protocols be designed to satisfy this constraint? The goal of
this paper is to provide some first steps in answering such
questions by examining the basic system-theoretic implications
of using unreliable networks for control. This requires a
generalization of classical control techniques that explicitly
takes into account the stochastic nature of the communication
channel.
In order to understand the complex coupling between com-
munication and control it is necessary to place the foun-
dations first. We start by addressing some simple canonical
problems that will shed some light on the real system be-
havior. We shall consider the following abstractions. Packet
networks communication channels typically use one of two
fundamentally different protocols: TCP-like or UDP-like. In
the first case there is acknowledgement of received packets,
while in the second case no-feedback is provided on the
communication link. The well known Transmission Control
(TCP) and User Datagram (UDP) protocols used in the Internet
are specific examples of our more general notion of TCP-like
and UDP-like communication protocol classes. We want to
study the effect of data losses due to the unreliability of the
network links under these two general protocol abstractions.
Accordingly, we model the arrival of both observations and
control packets as random processes whose parameters are
related to the characteristics of the communication channel.
Two independent Bernoulli processes are considered, with
parameters
γ and ν, that govern packet losses between the
sensors and the estimation-control unit, and between the latter
and the actuation points, see Figure 1. We point out that
using Bernoulli processes is clearly an idealization that is
chosen for mathematical tractability. The networking compo-
nent obviously has an additional impact on the performance
of the closed loop systems. Routing and congestion control
mechanisms would affect the packet arrival probability and it
is necessary in practice to estimate this probability to compute
the optimal control law. The presence of correlations in the
packet loss process can be taken into account, in principle,
at the cost of complicating the mathematical analysis. Our
foundations are instead based on simple abstractions which, as
we shall see, already reveal useful design guidelines and can
explain real system behaviors that are observed in practice.
B. Previous Work
Study of stability of dynamical systems where components
are connected asynchronously via communication channels
has received considerable attention in the past few years and
our contribution can be put in the context of the previous
literature. In [7] and [8], the authors proposed to place an
estimator, i.e. a Kalman filter, at the sensor side of the
Plant
Actuators
Sensors
ACK
Estimator
State
feedback
Z
-1
DELAY
COMM. NETWORK
OPTIMAL LQG CONTROLLER
TCP-like
protocols
Plant
Actuators
Sensors
Estimator
State
feedback
COMM. NETWORK
OPTIMAL LQG CONTROLLER
UDP-like
protocols
Fig. 1. Architecture of the closed loop system over a communication
network under TCP-like protocols (top) and UDP-like protocols (bottom). The
binary random variables ν
t
and γ
t
indicates whether packets are transmitted
successfully.
link without assuming any statistical model for the data loss
process. In [9], Smith et al. considered a suboptimal but
computationally efficient estimator that can be applied when
the arrival process is modeled as a Markov chain, which is
more general than a Bernoulli process. Other works include
Nilsson et al. [10][11] who present the LQG optimal regulator
with bounded delays between sensors and controller, and
between the controller and the actuator. In this work, bounds
for the critical probability values are not provided and there is
no analytic solution for the optimal controller. The case where
dropped measurements are replaced by zeros is considered
by Hadjicostis and Touri [12], but only in the scalar case.
Other approaches include using the last received sample for
control [11], or designing a dropout compensator [13], which
combines estimation and control in a single process. However,
the former approach does not consider optimal control and
the latter is limited to scalar systems. Yu et al. [14] studied
the design of an optimal controller with a single control
channel and deterministic dropout rates. Seiler et al. [15]
considered Bernoulli packet losses only between the plant
and the controller and posed the controller design as an H
∞
optimization problem. Other authors [16] [17] [18] [19] model
networked control systems with missing packets as Markovian
jump linear systems (MJLSs), however this approach gives
suboptimal controllers since the estimators are stationary.
4
Finally, Elia [20][21] proposed to model the plant and the
controller as deterministic time invariant discrete-time systems
connected to zero-mean stochastic structured uncertainty. The
variance of the stochastic perturbation is a function of the
Bernoulli parameters, and the controller design is posed an an
optimization problem to maximize mean-square stability of the
closed loop system. This approach allows analysis of Multiple
Input Multiple Output (MIMO) systems with many different
controller and receiver compensation schemes [20], however,
it does not include process and observation noise and the
controller is restricted to be time-invariant, hence sub-optimal.
There is also an extensive literature, inspired by Shannon’s
results on the maximum bit-rate that a channel with noise can
reliably carry, whose goal is to determine the minimum bit-
rate that is needed to stabilize a system through feedback [22]
[23] [24] [25] [26] [27] [28] [29] [30] [31]. This approach
is somewhat different from ours as we consider bits to be
grouped into packets that form single entities which can be
lost. Nonetheless there are several similarities that are not yet
fully explored.
Compared to previous works, this paper considers the
alternative approach where the external compensator feeding
the controller is the optimal time varying Kalman gain. More-
over, this paper considers the general Multiple Input Multiple
Output (MIMO) case, and gives some necessary and sufficient
conditions for closed loop stability. The work of [32] is most
closely related to this paper. However, we consider the more
general case when the matrix C is not the identity and there
is noise in the observation and in the process. In addition,
we also give stronger necessary and sufficient conditions for
existence of the solution in the infinite horizon LQG control.
C. Our Contribution
We study the effect of data losses due to the unreliability
of the network links under two different classes of protocols.
In our analysis, the distinction between the two classes of
protocols will reside exclusively in the availability of packet
acknowledgement. Adopting the framework proposed by Imer
et al. [32], we will refer therefore to TCP-like protocols if
packet acknowledgement is available and to UDP-like proto-
cols otherwise.
We show that, for the TCP-like case, the classic separation
principle holds, and consequently the controller and estimator
can be designed independently. Moreover, the optimal con-
troller is a linear function of the state. In sharp contrast, for
the UDP-like case, a counter-example demonstrates that the
optimal controller is in general non-linear. In the special case
when the state is fully observable and the observation noise
is zero the optimal controller is indeed linear. We explicitly
note that a similar, but slightly less general special case
was previously analyzed in [32], where both observation and
process noise are assumed to be zero and the input coefficient
matrix to be invertible.
Our final set of results relate to convergence in the infinite
horizon. Here, results on estimation with missing observation
packets [33] [34] are extended to the control case. We show
the existence of a critical domain of values for the parameters
of the Bernoulli arrival processes,
ν and γ, outside which a
transition to instability occurs and the optimal controller fails
to stabilize the system.
These results are visually summarized in Figure 2, where
our stability bounds are depicted for a scalar system. The
stability regions are the regions above those bounds. Notice
that for TCP-like protocols there exist critical arrival proba-
bilities for the control and observation packets below which
the system is in the unstable region. These critical values are
independent of each other, which is another consequence of
the fact that the separation principle holds for these protocols.
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
γ
ν
ν
c
TCP−stable
UDP−stable
γ
c
Fig. 2. Stability regions for TCP-like protocols and UDP-like protocols for a
scalar unstable system. These bounds are tight (i.e. necessary and sufficient)
in the scalar case. The dashed line corresponds to the boundary of a weaker
(sufficient) condition on the stability region for UDP-like protocols as recently
reported in [32].
In contrast, for UDP-like protocols the critical arrival prob-
abilities for the control and observation channels are coupled,
and the stability domain boundary assumes a curved form. The
performance of the optimal controller degrades considerably
when compared to TCP-like protocols, as the stability region
of UDP is strictly contained into the one of TCP. Finally, the
figure also reports the boundary of a weaker condition on the
stability region for UDP-like protocols as reported in [32],
which is indicated with a dashed line.
III. PROBLEM FORMULATION
Consider the following linear stochastic system with inter-
mittent observation and control packets:
x
k+1
= Ax
k
+ Bu
k
+ w
k
(1)
u
a
k
= ν
k
u
c
k
(2)
y
k
= γ
k
Cx
k
+ v
k
, (3)
where u
a
k
is the control input to the actuator, u
c
k
is the desired
control input computed by the controller, (x
0
, w
k
, v
k
) are
Gaussian, uncorrelated, white, with mean (¯x
0
, 0, 0) and covari-
ance (P
0
, Q, R) respectively, and (γ
k
, ν
k
) are i.i.d. Bernoulli
random variables with P (γ
k
= 1) = ¯γ and P (ν
k
= 1) = ¯ν.
The stochastic variable ν
k
models the loss packets between
5
the controller and the actuator: if the packet is correctly
delivered then u
a
k
= u
c
k
, otherwise if it is lost then the
actuator does nothing, i.e. u
a
k
= 0. This compensation scheme
is summarized by Equation (2). This modeling choice is
not unique: for example if the control packet u
c
k
is lost,
the actuator could employ the previous control value, i.e.
u
a
k
= u
a
k−1
, as suggested in [11]. The analysis of this scheme
requires a different problem formulation and is not considered
here. However both schemes are sensible compensation, and in
Section VII an empirical comparison seems to suggest that the
zero-input scheme indeed outperforms the hold-input scheme.
The stochastic variable γ
k
models the packet loss between
the sensor and the controller: if the packet is delivered then
y
k
= Cx
k
+ v
k
, while if the packet is lost the controller
reads pure noise, i.e. y
k
= v
k
. This observation model is
summarized by Equation (3). A different observation formal-
ism was proposed in [33], where the missing observation was
modeled as an observation for which the measurement noise
had infinite covariance. It is possible to show that both models
are equivalent, but the one considered in this paper has the
advantage of simpler analysis. This is because at times when
packets are not delivered, the optimal estimator ignores the
observation y
k
, therefore its value is irrelevant.
Let us define the following information sets:
I
k
=
(
F
k
∆
= {y
k
, γ
k
, ν
k−1
}, TCP-like
G
k
∆
= {y
k
, γ
k
}, UDP-like
(4)
where y
k
= (y
k
, y
k−1
, . . . , y
1
), γ
k
= (γ
k
, γ
k−1
, . . . , γ
1
), and
ν
k
= (ν
k
, ν
k−1
, . . . , ν
1
).
Consider also the following cost function:
J
N
(u
N−1
, ¯x
0
, P
0
) = E
x
′
N
W
N
x
N
+
+
P
N−1
k=0
(x
′
k
W
k
x
k
+ν
k
u
′
k
U
k
u
k
) |u
N−1
, ¯x
0
,P
0
(5)
where u
N−1
= (u
N−1
, u
N−2
, . . . , u
1
). Note that we are
weighting the input only if it is successfully received at the
plant. In the event it is not received, the plant applies zero
input and therefore there is no energy expenditure.
We now seek a control input sequence u
∗N−1
as a function
of the admissible information set I
k
, i.e. u
k
= g
k
(I
k
), that
minimizes the functional defined in Equation (5), i.e.
J
∗
N
(¯x
0
, P
0
)
∆
= min
u
k
=g
k
(I
k
)
J
N
(u
N−1
, ¯x
0
, P
0
), (6)
where I
k
= {F
k
, G
k
} is one of the sets defined in Equa-
tion (4). The set F corresponds to the information provided
under an acknowledgement-based communication protocols
(TCP-like) in which successful or unsuccessful packet delivery
at the receiver is acknowledged to the sender within the same
sampling time period. The set G corresponds to the information
available at the controller under communication protocols in
which the sender receives no feedback about the delivery of the
transmitted packet to the receiver (UDP-like). The UDP-like
schemes are simpler to implement than the TCP-like schemes
from a communication standpoint. Moreover UDP-like proto-
cols includes broadcasting which is not feasible under TCP-
like protocols. However, UDP-like protocols provide a leaner
information set. The goal of this paper is to design optimal
LQG controllers and to estimate their closed-loop performance
for both TCP-like and UDP-like protocols.
IV. OPTIMAL ESTIMATION
We start defining the following variables:
ˆx
k|k
∆
= E[x
k
| I
k
],
e
k|k
∆
= x
k
− ˆx
k|k
,
P
k|k
∆
= E[e
k|k
e
′
k|k
| I
k
].
(7)
Derivations below will make use of the following facts:
Lemma 4.1: The following facts are true [35]:
(a) E [(x
k
− ˆx
k
)ˆx
′
k
| I
k
] = E
e
k|k
ˆx
′
k
| I
k
= 0
(b) E [x
′
k
Sx
k
| I
k
] = ˆx
′
k
S ˆx
k
+trace
SP
k|k
, ∀S ≥ 0
(c) E [E[ g(x
k+1
) |I
k+1
] | I
k
] = E [g(x
k+1
) | I
k
] , ∀g(·).
We now make the following computations which we use to
derive the optimal LQG controller.
E[x
′
k+1
Sx
k+1
| I
k
] = E[x
′
k
A
′
SAx
k
| I
k
]+
+¯νu
′
k
B
′
SBu
k
+ 2¯νu
′
k
B
′
SA ˆx
k|k
+ trace(SQ)
(8)
where both the independence of ν
k
, w
k
, x
k
, and the zero-mean
property of w
k
are exploited. The previous expectation holds
true for both the information sets, i.e. I
k
= F
k
or I
k
= G
k
.
Also
E[e
′
k|k
T e
k|k
| I
k
] = trace(T E[e
k|k
e
′
k|k
| I
k
])
= trace(T P
k|k
), ∀T ≥ 0.
The equations for the optimal estimator are different
whether TCP-like or UDP-like communication protocols are
used
A. Estimator design under TCP-like protocols
Equations for optimal estimator are derived using arguments
similar to those used in standard Kalman filtering. The inno-
vation step is given by:
ˆx
k+1|k
∆
= AE[x
k
|F
k
]+ν
k
Bu
k
= Aˆx
k|k
+ν
k
Bu
k
(9)
e
k+1|k
∆
= x
k+1
− ˆx
k+1|k
= Ae
k|k
+ w
k
(10)
P
k+1|k
∆
= E[e
k+1|k
e
′
k+1|k
|ν
k
, F
k
]= AP
k|k
A
′
+Q (11)
where the independence of w
k
and F
k
, and the requirement
that u
k
is a deterministic function of F
k
, are used. Since
y
k+1
, γ
k+1
, w
k
and F
k
are independent, the correction step is
given by:
ˆx
k+1|k+1
= ˆx
k+1|k
+γ
k+1
K
k+1
(y
k+1
−C ˆx
k+1|k
) (12)
e
k+1|k+1
∆
= x
k+1
−ˆx
k+1|k+1
(13)
= (I −γ
k+1
K
k+1
C)e
k+1|k
−γ
k+1
K
k+1
v
k+1
P
k+1|k+1
= P
k+1|k
−γ
k+1
K
k+1
CP
k+1|k
(14)
K
k+1
∆
= P
k+1|k
C
′
(CP
k+1|k
C
′
+ R)
−1
(15)
where we simply applied the standard derivation for the time
varying Kalman filter using the following time varying system
matrices: A
k
= A, C
k
= γ
k
C, and Cov(v
k
) = R.
