1936
IEEE TRANSACTIONS
ON
AUTOMATIC CONTROL, VOL.
31,
NO.
12,
DECEMBER
1992
Stability Properties
of
Constrained Queueing
Systems and Scheduling Policies for
Maximum Throughput in Multihop
Radio Networks
Leandros Tassiulas
and
Anthony Ephremides,
Member,
IEEE
Abstruct-The stability
of
a queueing network with interde-
pendent servers is considered. The dependency
of
servers is
described by the definition
of
their subsets that can be activated
simultaneously. Multihop packet radio networks
(PRN’s)
pro-
vide a motivation for the consideration
of
this system. We study
the problem
of
scheduling the server activation under the con-
straints imposed by the dependency among them. The perfor-
mance criterion
of
a scheduling policy m is its throughput that
is characterized by its stability region C,, that is, the set of
vectors
of
arrival rates
for
which the system is stable.
A
policy
m,, is obtained which is optimal in the sense that its stability
region
Cn0
is a superset of the stability region
of
every other
scheduling policy. The stability region
Cmo
is characterized.
Finally, we study the behavior
of
the network
for
arrival rates
that lie outside the stability region. Implications
of
the results in
certain types
of
concurrent database and parallel processing
systems are discussed.
I. INTRODUCTION
E
consider a queueing network model that is suit-
able for communication networks with interde-
pendent service components. The queueing network has
arbitrary topology and multiple servers. The servers are
interdependent in that they cannot provide service simul-
taneously. The dependency among them is reflected
on
the constraints which specify exactly which subsets of
servers may be active simultaneously. For example, when
the constrained queueing system is used as a model of a
radio network, the servers correspond to the links and the
constraints disallow simultaneous transmissions for neigh-
boring links. We consider slotted time. At each time slot,
routing decisions are taken for the served customers and
eligible sets of servers are selected for activation. We
assume that these decisions are made in a centralized
fashion and are based on global knowledge of the queue
lengths in the entire network. We assume that buffering
at each queue is infinite. We consider the system to be
Manuscript received August
8,
1991; revised April
25,
1092.
Paper
recommended by Associate Editor,
K.
W.
Ross.
L.
Tassiulas is with the Department
of
Electrical Engineering,
Poly-
technic University,
Brooklyn,
NY
11201.
A. Ephremides is with the Department
of
Electrical Engineering and
Systems Research Center, University
of
Maryland, College Park, MD
20142.
IEEE Log Number 9204115.
stable
if
the queues do not tend to increase without
bound. We wish to find control policies under which the
system is stable for given arrival and service rates. Indeed,
we
characterize the region of arrival and service rate
vectors for which there exists some stabilizing policy, and
do find a policy which in fact stabilizes the system for all
arrival and service rate vectors in that region. Such a
policy is in a sense optimal as far as throughput is con-
cerned.
Our main motivation for the consideration of this con-
strained queueing network model is to study the resource
allocation problem in multihop radio networks. We are
interested in scheduled link activation schemes, as op-
posed to random access methods, for sharing a common
channel among neighboring nodes.
In
scheduled link acti-
vation, a sequence
S,,
t
=
1,2;..,
of sets of links which
may transmit simultaneously without conflicts is specified
(the schedule) and at each slot
t
the links of the set
S,
are
allowed to transmit. The link activation scheduling prob-
lem is to determine the sequence
S,
in a fashion that
optimizes some performance index. Most of the schemes
for the scheduling problem have the following form.
A
sequence
SI;..,
S,
of eligible link sets is selected and the
entire schedule consists of periodic repetition of that
sequence. Several approaches have been taken for the
determination of the basic schedule sequence
S,;..,
S,.
In [41,[61,[16],
[18],
and [20] different performance criteria
are adopted and either optimal or suboptimal computa-
tion of
S,;..,
S,
follows. Special emphasis has been given
in obtaining distributedly implementable algorithms for
the design of
S,;..,S,.
In
[19]
the problem of optimal
design of a fixed (state independent) schedule is consid-
ered and results analogous to the golden ratio policy in
a
single-hop network
[lo]
are obtained.
In
151 scheduling
schemes are considered where the set of activated links at
each slot is selected based on the network state in that
slot. In this work, we consider dynamic link activation
scheduling where the activated links at each slot are
selected based on the queue lengths at all network nodes.
The maximum throughput policy that we obtain for the
constrained queueing model provides a link activation
method that stabilizes the network for all arrival rates for
which
it
is stabilizable.
0018-9286/92$03.00
0
1992
IEEE
TASSIULAS
AND
EPHREMIDES: STABILITY PROPERTIES
OF
CONSTRAINED QUEUEING SYSTEMS
1937
In addition to multihop radio networks, the constrained
queueing model is appropriate for other resource alloca-
tion problems as well. A model of a database with concur-
rency control and locking has been considered in [ll],
[141, and
[151;
the constrained queueing system that we
study in this paper captures that database model where
the constraints reflect the locking constraints of the
database and the policy that we propose provides a
concurrency control algorithm that achieves maximum
throughput. In [31, a generalized multiserver queue is
proposed as a model of certain parallel processing sys-
tems; that multiserver queue can also be modeled by an
appropriate constrained queueing system.
This paper is organized as follows. In Section 11, we
describe the constrained queueing model. In Section 111,
we state the stability performance criteria and we present
the optimality results. In Section IV, the behavior of the
system in the instability region is investigated. In Section
V, we demonstrate how the constrained queueing system
appropriately models multihop radio networks and certain
computer systems. A few words about the notation before
we proceed. The random quantities are denoted by upper
case letters; for the nonrandom quantities we reserve the
lower case letters. Vectors are denoted by boldface char-
acters. A random process, that is, a sequence of random
variables indexed by time is denoted by the same symbol
as the random variables without the time index.
11.
THE
CONSTRAINED
QUEUEING
MODEL
We consider a network consisting of
L
nodes and
N
links. The connectivity of the system is represented by the
directed graph
G
=
(V,
E),
where
V
is the set of nodes
and
E
is the set of links (Fig. 1). Each link corresponds to
a server that serves customers residing at the origin node
of the link; after service, the customers are directed to the
destination node of the link. The origin and destination
nodes of link
i
are denoted by
q(i)
and
Mi),
respectively.
The terms servers and links are used interchangeably in
the following. A customer may enter the network at any
node. Its destination is a subset of the network nodes in
the sense that as long as the customer reaches any of
these nodes it leaves the system. Each customer reaches
its destination by appropriate routing through the net-
work. There are
J
customer classes which are distin-
guished by the destinations of the customers. The set of
destination nodes for class
j
is
y.
At each node
1
cus-
tomers of all classes are queued, except of those classes
j
for which node
1
is a destination, that is
1
E
(any
customer of the latter classes leaves the system as long as
it reaches node
1).
We consider slotted time. At each slot
t
certain links originating from node
1
provide service;
those are the active links at slot
t.
Notice that the cus-
tomers are not committed to specific outgoing links of a
node
1
by the time they reach
1
but at the beginning of
each slot a decision is taken which customers (of which
classes) are allocated at which links. This decision corre-
sponds to routing.
Fig.
1.
The connectivity graph
of
a constrained queueing network.
There are constraints in the simultaneous activation of
the serves in the sense that certain servers cannot provide
service at the same time.
An
activation
set
is a set of
servers which can be activated in the same slot.
An
activation set is represented by its
activation vector,
that is
a binary vector with N elements; the ith element corre-
sponds to server
i,
and is equal to
1
if
server
i
belongs to
the activation set and to
0
otherwise. The terms activation
set and activation vector will be used interchangeably in
the rest of the paper. The
constraint set
S
consists of all
activation vectors of the system; this set completely speci-
fies the activation constraints. We make the following
assumption about the structure of the constraint set which
is natural in the systems we consider.
C.l
Every subset of an activation set is an activation set
itself.
At the beginning of each slot an activation set of links
is selected that provide service during the slot. This is
referred as scheduling in the following.
A.
Queue Length Dynamics
The servers are synchronized to start service at the
beginning of a time slot. At each slot, we control the
system through the selection of the activation set and of
the class of the customer assigned to each activated server
for service. The binary variable
E,,(t)
indicates whether
server
i
is activated in slot
t
or not and which customer
class it serves;
if
Eij(t)
=
1
server
i
is activated and serves
a customer of class
j
otherwise it is not. A customer
served by server
i
in slot
t
completes service with some
probability
m,.
More specifically, we consider a binary
variable
M,(t)
and a customer served by server
i
during
slot
t
completes service and moves from queue
q(i)
to
queue
h(i)
if
M,(t)
=
1;
otherwise it remains at queue
q(i). The vector
E(t)
=
(Eij([):
i
=
l;..,
N,
j
=
l;..,J),
indicates which class each server serves at slot
t.
A binary
vector
e
=
(ei,:
i
=
I,...
,
N,
j
=
l;..,
J)
is a
multiclass
actication uector
if
the corresponding vectors
e’
=
(eij:
i
=
l;..,
N),
j
=
l;..,J
are such that
Ci=,e’
E
S.
Let
8
by the collection of all multiclass activation vectors. At
each slot
t
the vector
E(t)
is selected from the set
8.
The
decisions are based
on
the number of customers of each
class in each queue. This information is represented as
1938
IEEE TRANSACITONS ON AUTOMATIC CONTROL, VOL.
37,
NO.
12,
DECEMBER
1992
follows. Let
X,,(t)
be the number of customers of class
j
at queue
I
by the end of slot
t
(or the beginning of slot
t
+
1).
The vector
X(t)
=
(X,,(t):
1
=
1;-+,
L,
j
=
l;..,
J)
consists of the lengths of the queues
of
all customer
classes and is called the multiclass queue length vector at
slot
t.
We denote by
2
the space where the vector
X(t)
lies.
Consider a function
g:2-
8;
if
g(x)
=
e
=
(eI,:
i
=
l;..,
N,
j
=
l;..,
J)
then denote the vector
eJ
by
g’(x).
An
activation rule
is a function
g:Z+
8
with the prop-
erty that no servers are considered activated for nonexist-
ing customers, that is to say, the number of servers of
queue
1
activated
by
the activation vector
gJ(x)
are less
than or equal to
xI,;
where servers of queue
I
are those
servers
i
for which
q(i)
=
1.
An
activation policy
is a
collection of activation rules
g,,
t
=
1,2,
;
at slot
t
we
have
E(t)
=g,(X(t
-
1)).
Until Section
V,
we consider
stationary policies that is policies which use the same
activation rule at each slot. In Section
V,
it will become
clear that we do not gain anything with respect to stability
if we consider nonstationary policies in addition to sta-
tionary. The class of all stationary activation policies is
denoted by
H.
When the network is operated by policy
7~
with activation rule
g,
at slot
t
+
1
we have
EJ(t
+
1)
=
g’(X(t))
where
EJ(t)
=
(EL,(t): i
=
l,,..,
N)
is the activa-
tion vector of class
j
at slot
t.
The state of the system
evolves according to the following equation:
XJ(t
+
1)
=
XJ(t)
+
RJM(t
+
l)EJ(t
+
1)
+
A’(t
+
1)
t
=
0,
l;..,j
=
1;s.
>
J
(2.1)
where
M(t)
is
a
diagonal matrix, the ith diagonal element
of
which is equal to
M,(t),XJ(t)
=
(X/,(t):
1
=
l;..,
L)
is
the vector of the queue lengths of class
j
by the end of
slot
t,
A’(t)
=
(A,,(t):
I
=
l;..,
L)
is a vector with its Ith
element
A,,(t)
being equal to the number
of
customers of
class
j
arriving at queue
I
during slot
t
and
RI
is an
L
x
N
matrix that reflects the connectivities of the queues
among themselves and with the destination node of class
j.
Matrix
RJ
is called the routing matrix of class
j.
The
element of
RI
in its Ith row and ith column is
1,
if
h(
i)
=
1
and queue
1
is not connected
with the destination node of class
1.
r;,
=
[;I,
ifq(i)
=
1
otherwise.
We assume that
(A/,(t))T=
,,
{M,(t)):=,
are i.i.d. sequences
of random variables for all
1
=
l;..,
L,
j
=
l;--,J,
i
=
l;..,
N.
Furthermore, we assume that the above processes
are independent among themselves and the second mo-
ments of the arrival processes
E[A:,(t)]
are finite. Under
those statistical assumptions and for any policy in
H
the
queue length process
{X(t))y=
is a Markov chain. Finally,
we make the following assumption concerning the topol-
ogy
of the network.
C.2
If a customer of class
j,
may reach some queue
I,
then this customer may be forwarded from queue
I,
to
some destination node
of
class
j,
if an appropriate route
is selected. More specifically, if there is a sequence of
servers
i,;..,
i,
such that
E[Aq(i,)j,(t)I
>
0,
h(i,)
=
q(im+,),
m
=
l,-.*,n
-
1
then there exists a sequence of
servers
i;;..,
ik,
such that the queue
q(il)
receives nonzero
traffic of class
j,,
h(ih)
=
q(ih+,), m
=
1;--,n’
-
1
and
there exists a link in
Ed
from
h(i;,)
to the destination
node of class
j,.
111.
STABILITY
CONSIDERATIONS
The system is stable if the queue length process reaches
a
steady state and does not blow to infinity. When the
Markov chain
X
is irreducible, stability of the system is
equivalent to ergodicity of
X.
Under the general assump-
tions we made about the constraint set and the topology
of the queueing system we cannot guarantee irreducibility
of the queue length process. In the general case, the state
space is partitioned in transient and recurrent states. We
consider the system to be stable if all recurrent states are
positive recurrent and the queue length process hits the
recurrent states with probability one; that is,
X
does not
remain in the set of transient states forever. In the follow-
ing, we state our definition of stability after we recall
some basic facts from Markov chain theory
([121).
A
state
x
is
reachable
by some state
y
if
P(X(t
+
n)
=
xlX(t)
=
y)
>
0
for some
n
2
1.
The states
x
and
y
com-
municate
if
they are reachable by each other.
A
set of
states
R
is
closed
if
P(X(t
+
1)
=
xlX(t)
=
y)
=
0
for all
y
E
R,
x
E
R.
The state space of the chain is partitioned
in the sets
T,
R,,
R2;..,
where
Rj,
j
=
1,2;..,
are closed
sets of communicating states and
T
contains all states
which do not belong to any closed set of communicating
states and therefore are transient. For any
x
E
T
assume
that
X(0)
=
x
and consider the time
if
X(t)
E
T, Vt
>
0
otherwise
rx
=
(%7
min
(t
> 0:
X(t)
E
T},
(3.1)
at which the chain hits some of the sets
R’
for the first
time when it starts at
t
=
0
from state
x.
If
U
,
Rj
=
0,
then clearly
rx
=
m.
We can now define stability as fol-
lows.
Definition
3.1:
The system is stable if for the queue
length process
X
we have
P(r,
<
m)
=
1
Vy
E
T
(3.la)
and all states
x
E
U
4=
R,
are positive recurrent.
The next theorem states sufficient conditions for the
stability of the system according to Definition
3.1.
Those
conditions involve the drift of a test (Lyapunov) function
on
the state space of the chain.
Theorem
3.1:
Consider a Markov chain
X(t)
with state
space
F.
If there exists
a
lower bounded real function
TASSIULAS
AND
EPHREMIDES: STABILITY PROPERTIES
OF
CONSTRAINED QUEUEING SYSTEMS
1939
I/:
2
-+
R,
an
E
>
0
and a finite subset
2(,
of
2
such that
E[V(X(t
+
1))
-
V(X(t))(X(t)
=
y]
5
-E
ify
@gl.
(3.2)
E[I/(X(t
+
l))lx(r)
=
y]
<
x
ify
E,x;)
(3.3)
then for the time
T~
as defined in
(3.1)
we have
P(T~
<
M)
=
1
vx
E
T
and all states
x
E
U
;=,
R,
are positive recurrent.
criteria for irreducible chains
([2]).
A.
Scheduling for Maximum Throughput
We would like the system to be stable for a wide range
of arrival rates. The arrival rate of class
j
to queue
1,
E[A,,(t)l
is denoted by
a/,.
The multiclass arrival rate
vector
a
=
(a/,:
1
=
I;--
,
L,
j
=
l;..,J>
consists of the
arrival rates of all classes at all queues. We quantify the
performance of an activation policy by its stability region.
Definition
3.2:
Stability region
C,
of policy
rr
is the set
of multiclass arrival rate vectors
a
for which the system is
stable under
rr.
We wish a policy
rr
to have a large stability region.
Roughly speaking, the largest the stability region the
better the policy is.
Definition
3.3:
A
policy
rrl
dominates
another policy
rr2
if
C,?
c
C,,.
If policy
rr,
dominates policy
rr2
the system is stable
under
rrl
whenever it is stable under
rr2
(Fig.
2).
Two
policies are not always comparable since it may be that
no
one dominates the other. This is the case for policies
rr3
and
rrl
in Fig.
2.
Proo$
The theorem is a trivial extension of Foster's
Definition
3.4:
The stability region of the system is
c=
U
e,.
T€
G
The set
C
contains all arrival rate vectors for which there
exists a policy in
H
that stabilizes the system.
An
optimal
policy,
that is, one which dominates any other policy in
H,
should have stability region that is a superset of the
stability region
of
any other policy in
H;
therefore, it
should have stability region equal to
C.
Such a policy is
called a maximum throughput policy in the rest of the
paper. Notice that since two policies may not have compa-
rable stability regions, it is not clear at all whether a
maximum throughput policy exists or not. One of our
main results is that an optimal policy indeed exists.
B.
Maximum Throughput Policy
The policy
r0
that we specify next achieves maximum
throughput. The activation rule for
rro
is denoted by
go(.);
the vector
E(t)
=
go(X(t
-
1))
is selected in three
stages. Let
us
denote the service rate
E[M,(t>l
by
m,;
the
service rate vector is
m
=
(m,: i
=
l;..,
NI.
Stage
1.
For each server
i
a weight
D,(t)
is selected as
follows. For each class
j
and server
i
consider the follow-
a
f
a
2
Fig.
2.
Stability region diagram.
ing quantity:
Let
D,(t)
=
maxi=,;,,
{D,,(t)}
be the weight of server
i
and D(t>
=
(Di(t>: i
=
l;..,
N)
the weight vector at slot
t.
Stage
2.
A
maximum weighted activation vector
E
is
selected from
S
i.
=
arg max{DT(t)c}.
CtS
If more than one vector
c
achieves the maximum,
i.
is
selected arbitra;ily among them.
Stage
3.
Let
j,
be the class for which
D,(t)
=
DLj;(t>
for
each server
i;
if_more than one class satisfies the above
inequality then
j,
can be any of these classes. The multi-
class activation vector
E(t)
is as follows,
1,
if
c^,
=
1,
j
=;
and
Xq(,,,(t
-
1)
is
greater than the number of servers
that serve queue
q(
i)
0,
otherwise.
E&)
=
Remarks:
1)
If
Djj(t)
is greater than zero and server
i
serves a
customer of class
j
during slot
t
then the quantity
Djj(t>
tends to be reduced. That is, the difference between
XhciIj(t>
and Xqcj,j(t) is diminished. Policy
rro
selects
E(t>
such that the servers
i
and the corresponding classes
j
for
which
Djj(t>
is larger are activated. In other words,
rro
tends for each class to equalize the queue lengths of the
same class in different network nodes, giving priority to
the servers and classes for which this difference is larger.
2)
The implementation of policy
rro
requires the solu-
tion of the following optimization problem at each time
slot
t:
max{D'(t)c}
(3.4)
crS
1940
IEEE
TRANSACTlONS ON AUTOMATIC CONTROL,
VOL.
37,
NO.
12,
DECEMBER
1992
The number of possible activation vectors (the cardinality
of
SI
is usually largely compared to the number of servers;
in fact, it is
of
exponential order with respect to the
number of servers most
of
the times. Therefore, the
solution
of
the above optimization problem by exhaustive
search of all activation vectors is usually out of the ques-
tion.
In
certain cases, the constraint set
S
has a specific
structure that can be utilized for the solution of (3.4).
In
Section
V,
the constraint sets are illustrated for several
communication and computer systems. Finding efficient
algorithms for the solution
of
(3.4) given the constraint set
S
in each particular application is important for the
implementation of
T".
C.
Characterization
of
the Stability Region
We proceed now to characterize the system stability
region
C.
The set
C'
that we specify next plays an essen-
tial role in the characterization
of
C
since, as it will be
shown later
C'
c
C
c
c,
where
F
is the closure of
C';
the closure
of
C'
is well defined since
C'
is a subset of
RL
'.
The definition of
C'
involves deterministic flows in
the graph
G
and the heuristic discussion that precedes its
definition provides some intuition.
Assume that the constrained queueing system is stable
under some scheduling policy
T
and that
it
operates in
steady state. Let
f,,
be the rate with which customers of
class
j
are served by server
i.
Since the system is in steady
state, the rate with which customers
of
class
j
enter some
queue
1
should be equal to the rate with which customers
of
the same class leave the queue
I;
that is, the rates
f,,
should satisfy the flow conservation equations in each
network node. Consider a multicommodity arrival rate
vector
a
and let a]
=
(al,:
1
=
l;..
,
L)
be the vector which
contains the arrival rates
of
class
j
at all network queues
for
j
=
l;..,
J.
The vector
fl
=
(f,,:
i
=
l;..,
N)
that
consists
of
nonnegative numbers and satisfies the flow
conservation equations which are written in a matrix form
as
(3.5)
a1
=
-Rlfl
is called an a-admissible flow vector for class
j.
The vector
f
=
(f,,:
i
=
l;..,
N,
j
=
l;..,
J)
that consists of nonnega-
tive numbers and is such that the corresponding vectors
f'
satisfy
(3.5)
for
j
=
l;-.,J is an a-admissible multicom-
modity flow vector. Let
Fa
be the set of all a-admissible
multicommodity flo? vectors. Associated with a vnector
f
E
Fa
is the vector
f
=
C:=,f'. The component of
f
that
corresponds to server
i
is the total rate with which cus-
tomers are servedA by server
i,
irrespectively
of
their
classes; therefore,
f
is called
total
flow
vector. The set
C'
is defined now as follows:
C'
=
{a: there exists
f
E
F,,
c
E
co(S)
such that for the
corresponding
i
we have
m;
'f:
<
c,
if
f:
>
0
and
f,
=
0
if
c,
=
0
)
where
COW
the convex hull
of
the constraint set
S.
The
closure of
C'
is
characterized in the following lemma.
?
=
{a: there exists an
f
E
F,,
and
a
c
E
co(S),
Lemma
3.1:
The closure
?
of
C'
is as follows:
such that
M-'
i
5
c}
where
M
is the diagonal matrix with ith diagonal element
equal to
m,,
i
=
l;..,
N.
Proot
It appears in the appendix.
D.
Optimality Results
The optimality
of
m0
and the characterization of
C
are
stated in this section. Two lemmas precede the theorem.
In
the following lemma, we show that under
ro
the
system is stable in
C'.
It is shown that a quadratic func-
tion of the queue length vector satisfies the conditions
(3.2) and (3.3) therefore, stability follows from Theorem
3.1.
Lemma
3.2:
Under policy
ro
the system is stable for
every a
E
C'
C'
c
c,,,
.
Proof
It appears in the appendix.
Lemma
3.3:
If a
E
(c)',
then the system is unstable
for any policy in
H.
Proof
It appears in the appendix.
Policy
no
achieves indeed maximum throughput as it is
Theorem
3.2:
The set
C'
characterizes the system stabil-
stated
in
the following theorem.
ity
region in the sense
C'CCCC
and for the stability region
of
policy
T~,
we have
C'
c
C,[)
c
c
c
c,,,.
Proofi
By definition of the system stability region we
have
C,,,
c
C
and from Lemma 3.2
C'
c
CTII
c
c.
c
c
c'
c
c,,,.
(3
4
From Lemmas 3.2 and 3.3 we have the following:
(3.7)
The theorem follows from
(3.6)
and
(3.7).
0
Remarks:
1) From the first part of Theorem 3.2 we have
c
-
C
c
c'
-
C'.
It is argued in the following that
c
-
C'
is
the boundary of
C'
which is a surface (has
no
interior) in
the space where a lies. We claim that for
no
a
E
-
C'
there exists no ball centered in a which belongs to
C'.
If a belongs to
c'
-
C'
then 6a does not belong to
C'
for
any
6
>
1. This is because
if
6a
E
I?'
then from the
definition of
C'
and Lemma 3.1 we have that a belongs to
C'.
In this case a does not belong to
c
-
C'
which is a
contradiction. From the above discussion we see that part