Receiver-driven Layered Multicast
Steven McCanne
University of California, Berkeley and
Lawrence Berkeley National Laboratory
mccanne@ee.lbl.gov
Van Jacobson
Network Research Group
Lawrence Berkeley National Laboratory
van@ee.lbl.gov
Martin Vetterli
University of California, Berkeley
martin@eecs.berkeley.edu
Abstract
State of the art, real-time, rate-adaptive, multimedia applications
adjust their transmission rate to match the available network ca-
pacity. Unfortunately, this source-based rate-adaptation performs
poorly in a heterogeneous multicast environment because there is
no single target rate — the conflicting bandwidth requirements of
all receivers cannot be simultaneously satisfied with one transmis-
sion rate. If the burden of rate-adaption is moved from the source
to the receivers, heterogeneity is accommodated. One approach to
receiver-driven adaptation is to combine a layered source coding
algorithm with a layered transmission system. By selectively for-
warding subsets of layers at constrained network links, each user
receives the best quality signal that the network can deliver. We
and others have proposed that selective-forwarding be carried out
using multiple IP-Multicast groups where each receiver specifies its
level of subscription by joining a subset of the groups. In this pa-
per, we extend the multiple group framework with a rate-adaptation
protocol called Receiver-driven Layered Multicast, or RLM. Under
RLM, multicast receivers adapt to both the static heterogeneity of
link bandwidths as well as dynamic variations in network capacity
(i.e., congestion). We describe the RLM protocol and evaluate its
performance with a preliminary simulation study that characterizes
user-perceived quality by assessing loss rates over multiple time
scales. For the configurations we simulated, RLM results in good
throughput with transient short-term loss rates on the order of a few
percent and long-term loss rates on the order of one percent. Fi-
nally, we discuss our implementation of a software-based Internet
video codec and its integration with RLM.
ACM SIGCOMM '96, August 1996, Stanford, CA.
Copyright
c
1995 by the Association for Computing Machinery,
Inc. Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that new
copies bear this notice and the full citation on the first page. Copyrights
for components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted.
1 Introduction
The Internet's heterogeneity and scale make multipoint com-
munication design a difficult problem. For real-time mul-
timedia, we would like to “broadcast” a live signal from
any particular sender to an arbitrarily large set of receivers
along paths with potentially high variability in bandwidth.
The simplest solution to this problem is to distribute a uni-
form representation of the signal to all interested receivers
using IP Multicast [8]. Unfortunately, this is suboptimal —
low-capacity regions of the network suffer congestion while
high-capacity regions are underutilized.
The problems posed by heterogeneity are not just the-
oretical, they impact our daily use of Internet remote-
conferencing. For example, each week for the past year,
U.C. Berkeley has broadcast a seminar over their campus
network and onto the Internet. As depicted in Figure 1, a
video application is run on a “seminar host” that sources a
single-rate signal at 128 kb/s, the nominal rate for video over
the Internet Multicast Backbone, or MBone [11]. However,
a number of users on the local campus network have high
bandwidth connectivity and would prefer to receive higher-
rate, higher-quality video. At the other bandwidth extreme,
many users have ISDN access and would like to partici-
pate from home, but a 128 kb/s video stream overwhelms
an ISDN line.
In this open-loop approach, the sender broadcasts at some
fixed rate without regard to changing network conditions. A
better approach is to adjust the transmission rate to match
the available capacity in the network, i.e., to react to conges-
tion. Pioneering research in rate-adaptive video [1, 19, 23]
has shown that this is feasible, but unfortunately, in the con-
text of multicast, the notion of network capacity is ill defined.
A control scheme that adjusts the rate of a single stream at
the source simply cannot meet the conflicting requirements
of a set of heterogeneous receivers.
An alternative approach is to combine a layered compres-
sion algorithm with a layered transmission scheme [29, 32].
In this approach, a signal is encoded into a number of lay-
ers that can be incrementally combined to provide progres-
SIGCOMM – August 1996 – Stanford, CA
2
64 kb/s
Gateway
H
Ethernets
H
Router
MBone
ISDN
ISDN
MBone
Router
Gateway
Campus Backbone
Ethernets
UCB
Seminar
? kb/s
(MBone)
H
H
Internet
500 kb/s 10 Mb/s
Seminar
(MBone)
Internet
Campus Backbone
UCB
Figure 1: Network heterogeneity
sive refinement. By dropping layers at choke points in the
network — i.e., selectively forwarding only the number of
layers that any given link can manage — heterogeneity is
managed by locally degrading the quality of the transmitted
signal.
This framework provides an elegant solution to hetero-
geneity but a crucial piece is missing. The system must have
mechanisms for determining, communicating, and executing
the selective forwarding of subflows along all the links in a
distribution. While much of the previous work leaves this
as an implementation detail, a novel mechanism based on IP
Multicast was suggested by Deering [6] and elaborated on
and/or independently reported in [4, 9, 20, 26, 33]. In this
approach, the different layers of the hierarchical signal are
striped across multiple multicast groups and receivers adapt
to congestion by adding and dropping layers (i.e., joining
and leaving multicast groups). Receivers implicitly define
the multicast distribution trees simply by expressing their in-
terest in receiving flows. Thus there is no explicit signaling
between the receivers and routers or between the receivers
and source.
While this general mechanism has been discussed in the
research community, the problem has not been studied in de-
tail, algorithms for adaptation have not been developed, and
systems based on these ideas have not yet emerged. This pa-
per addresses some of the open questions related to layered
multicast transport through the design and simulation of an
experimental network protocol called Receiver-driven Lay-
ered Multicast or RLM. In the following section we describe
the network model assumed by RLM. Next we provide in-
tuition for RLM and present the protocol in detail. We then
explore its performance through simulation. Finally, we dis-
cuss the integration of RLM into a comprehensive systems
framework, report on related work, and describe our future
work.
priority
drop
random
drop
Requested
Rate
Quality
B
Figure 2: Priority-/Random-drop Tradeoff.
2 The Network Model
RLM works within the existing IP model and requires no
new machinery in the network. We assume:
only best-effort, multipoint packet delivery, e.g., with-
out guarantees for packet ordering, minimum band-
width, etc.;
the delivery efficiency of IP Multicast, i.e., that traffic
flows only along links with downstream recipients; and,
group-oriented communication: senders need not know
that receivers exist and receivers can dynamically join
and leave the communication group in an efficient and
timely manner.
These three requirements are sufficient for single source
distribution to arbitrary numbers of receivers under RLM.
To handle multiple, simultaneous sources, RLM assumes
that receivers can specify their group membership on a per-
source basis (i.e., a receiver can ask for packets sent to some
group but exclude packets from one or more sources)
.
We refer to a set of end-systems communicating via a
common set of layered multicast groups as a session. Be-
cause the IP Multicast service model does not export any
of the routing mechanism, we cannot guarantee that all the
groups of a single session follow the same distribution tree.
That is, multicast routing can be carried out on a per-group
basis and differentgroups can be routed along differentspan-
ning trees. Although RLM is most easily conceptualized in
a network where all the groups follow the same route, this is
not a requirement.
The relationship among the information contained across
the set of groups in a session can either be cumulative or
independent. In the cumulative case, each layer provides re-
finement information to the previous layers and the receiver
must subscribe to all groups up to and including the high-
est group. In the independent case, each layer is independent
and the receiver need only subscribe to one group. This latter
scheme is often called simulcast because the source transmits
Source-based pruning is not part of the current IP Multicast specifica-
tion but is included in the next version, IGMP-3, which is under review by
the IETF.
SIGCOMM – August 1996 – Stanford, CA
3
multiples copies of the same signal simultaneously at differ-
ent rates (resulting in different qualities). In this paper, we
focus on the cumulative model because it makes more effec-
tive use of bandwidth but RLM is also compatible with the
simulcast model.
Instead of the best-effort, IP Multicast model described
above,the universallycited approach to layered packet trans-
mission adds a drop-preference packet discard policy to all
the routers in the network. Under drop-preference, when
congestion occurs, routers discard less important informa-
tion (i.e., low-priority packets) before more important infor-
mation (i.e., high-priority packets). Although this approach
provides graceful degradation in the presence of packet loss,
we believe it has scaling problems because it rewards poorly-
behaved users.
This effect is illustrated in Figure 2, which plots the qual-
ity of a received signal vs. the requested bit rate for both
priority-drop and random-drop policies. In both cases, the
quality of the received signal increases with the requested
rate up to the bottleneck capacity
but beyond this, the
quality depends on the drop policy. With random-drop,qual-
ity degrades because packets are dropped uniformly across
all layers, while with priority-drop the quality remains con-
stant because only “enhancement” packets are dropped. The
key distinguishing feature of these two curves is their con-
vexity. Because the random-drop curve is strictly convex,
it has a unique maximum. Thus we can design a control
system that maximizes the quality metric and drives the sys-
tem toward the stable, uncongested bottleneck rate
. The
priority-drop curve has no unique maximum and hence does
not admit a control system that optimizes delivered quality
by converging to a single, stable operating point. In fact, a
greedy or naive user would likely request a rate far above
the bottleneck rate
, driving the network into a persistently
congested state.
3 The RLM Protocol
Building on the best-effort IP-Multicast network model, we
now describe RLM at a high-level to develop intuition for
the protocol before discussing the low-level details. To first
order, the source takes no active role in the protocol. It sim-
ply transmits each layer of its signal on a separate multi-
cast group. The key protocol machinery is run at each re-
ceiver, where adaptation is carried out by joining and leav-
ing groups. Conceptually, each receiver runs the following
simple control loop:
on congestion, drop a layer;
on spare capacity, add a layer.
Under this scheme, a receiver searches for the optimal level
of subscription much as a TCP source searches for the bottle-
neck transmission rate with the slow-start congestion avoid-
ance algorithm [21]. The receiver adds layers until conges-
R
1
R
3
R
S
2
1
R
2
R
3
128 kb/s
10Mb/s
512 kb/s
10Mb/s
10Mb/s
R
S
Figure 3: End-to-end adaptation.
tion occurs and backs off to an operating point below this
bottleneck.
Figure 3 illustrates the RLM scheme. Suppose source
is transmitting three layers of video to receivers
,
, and
. Because the
path has high capacity,
can suc-
cessfully subscribe to all three layers and receive the highest
quality signal. However, if either
or
try to subscribe
to the third layer, the 512 kb/s link becomes congested and
packets will be dropped. Both receivers react to this conges-
tion by dropping layer three, prompting the network to prune
the unwanted layer from the 512 kb/s link. Finally, because
of the limited capacity of the 128 kb/s link,
might have to
drop back all the way to a single layer. The effect is that the
distribution trees for each layer have been implicitly defined
as a side effect of the receiver adaptation.
3.1 Capacity Inference
To drive the adaptation, a receiver must determine if its cur-
rent level of subscription is too high or low. By definition,
the subscription is too high if it causes congestion. This is
easy to detect because congestion is expressed explicitly in
the data stream through lost packets and degraded quality.
On the other hand, when the subscription is too low, there is
no equivalent signal — the system continues to operate at its
current level of performance. We must rely on some other
mechanism to provide this feedback.
One source for this feedback might be to monitor link uti-
lization and explicitly notify end-systems when capacity be-
comes available. However, this requires new mechanism in
the network that renders deployment difficult. The approach
we adopt in RLM is to carry out active experiments by spon-
taneously adding layers at “well chosen” times. We call this
spontaneous subscription to the next layer in the hierarchy
a join-experiment. If a join-experiment causes congestion,
the receiver quickly drops the offending layer. If a join-
experiment is successful (i.e., no congestion occurs), then
SIGCOMM – August 1996 – Stanford, CA
4
Time
A
B
C
D
E
F
Layer #
4
3
2
1
Figure 4: An RLM “sample path”
the receiver is one step closer to the optimal operating point.
3.2 RLM Adaptation
Unfortunately, join-experiments cause transient congestion
that can impact the quality of the delivered signal. There-
fore, we need to minimize the frequencyand duration of join-
experiments without impacting the algorithm's convergence
rate or its ability to track changing network conditions. This
is done through a learning algorithm, where over time, each
receiver determines the level of subscription that causes con-
gestion. By doing join-experiments infrequently when they
are likely to fail, but readily when they are likely to succeed,
we reduce the impact of the experiments. We implement
this learning strategy by managing a separate join-timer for
each level of subscription and applying exponential backoff
to problematic layers.
Figure 4 illustrates the exponential backoff strategy from
the perspective of a single host receiving up to four layers.
Initially, the receiver subscribes to layer 1 and sets a join-
timer (A). At this point, the timer duration is short because
the layer has not yet provenproblematic. Once the join-timer
expires, the receiver subscribes to layer 2 and sets another
join-timer (B). Again, the timer is short and layer 3 is soon
added. The process repeats to layer 4, but at this point, we
will assume congestion occurs (C). A queue will then build
up and cause packet loss. Once the receiver detects these
lost packets, it drops back to layer 3. The layer 3 join-timer is
then multiplicatively increased and another timeout is sched-
uled (D). Again, the process repeats, congestion is encoun-
tered, and the join-timer is further increased (E). Later, unre-
lated transient congestion provokes the receiver to drop down
to layer 2 (F). At this point, because the layer 3 join-timer is
still short, the layer is quickly reinstated.
In order to properly correlate a join-experiment with its
outcome, we must know how long it takes for a local layer
change to be fully established in the network and for the re-
sulting impact to be detected back at the receiver. We call
this time interval the detection-time. If a join-experiment
lasts longer than the detection-time without congestion oc-
curring, then we deem the experiment successful. On the
other hand, if congestion occurs within the detection-time
interval, we assume the experiment failed and increase the
join-timer for that layer. Because the detection-time is un-
R
S
R
L
R
H
R
L
L
L
R
R
L
H
R
R
2
L
L
1
R
L
R
L
R
L
L
L
R
S
join-2
Figure 5: Shared Learning
known and highly variable, we estimate it and its variance
adaptively. We initialize our estimator (mean and deviation)
with a conservative (i.e., large) value, and adapt it using
failed join-experiments. That is, when an experiment fails,
we update our estimator with the time interval between the
start of the experiment and the onset of congestion.
3.3 Scaling RLM
If each receivercarries out the aboveadaptation algorithm in-
dependently, the system scales poorly. As the session mem-
bership grows, the aggregate frequency of join-experiments
increases; hence, the fraction of time the network is con-
gested due to join-experiments increases. Moreover, mea-
surement noise increases because experiments tend to inter-
fere with each other. For example, if one receiver is conduct-
ing an experiment on layer 2 and another begins an experi-
ment on layer 4 that causes congestion, then the first receiver
can misinterpret the congestion and mistakenly back off its
layer 2 join-timer.
We can avoid these problems by scaling down the in-
dividual join-experiment rates in proportion to the overall
group size. In other words, we can fix the aggregate join-
experiment rate independent of session size much as RTCP
scales back its control message rate in proportion to the
group size [28]. However, reducing the experiment rate in
this manner decreases the learning rate. For large groups,
the algorithm will take too long to converge.
Our solution is “shared learning”: Before a receiver con-
ducts a join-experiment, it notifies the entire group by mul-
ticasting a message identifying the experimental layer. Thus
all receivers can learn from other receivers' failed join-
experiments. For example, Figure 5 shows a topology with
a single source, one receiver
situated along a high-
speed path (denoted by the thickened links) and a set re-
ceivers, each labeled
, situated at the far end of a low-rate
link. Suppose a low-rate receiver decides to conduct a join-
experiment on layer 2. It broadcasts a join-2 message to the
group and joins the layer 2 multicast group. As a result, link
becomes oversubscribed and congestion results, causing
packets to be dropped indiscriminately across both layers.
SIGCOMM – August 1996 – Stanford, CA
5
At this point, all of the
receivers detect the congestion
and since they know a layer 2 experiment is in progress, they
all scale back their layer 2 join-timer. Thus all of the low-
bandwidth receivers learn together that layer 2 is problem-
atic. Each receiver need not run individual experiments to
discover this on their own.
This learning process is conservative. Receivers make
their decisions based on failed experimentsnot on successful
experiments. Moreover, the success/failure decision is based
on local observations, not on a global outcome. That is, each
receiver decides whether the experiment succeeds based on
the network conditions on the path from the source to that re-
ceiver, entirely independent of the receiver that instantiated
the join-experiment. Hence, a given experimentmay succeed
for some receivers but fail for others.
Even though the shared learning process enhances the pro-
tocol's scalability by reducing convergence time, overlapped
experiments can still adversely impact the learning rate. But
because receivers explicitly announce the start of each ex-
periment, the probability that an experiment overlaps with
another can be substantially reduced by suppressing the start
of a new experiment when one is outstanding. For example,
if in Figure 5 receiver
decides to carry out a join-4 ex-
periment that causes congestion on link
, then the low-rate
receivers can misinterpret this as a failed join-2 experiment.
But because
sees the explicit join-2 announcement, it
will suppress the join-4 experiment and thereby limit the in-
terference. Note that this exchange of information is merely
an optimization. If the announcement packet is lost, the al-
gorithm still works albeit with potentially reduced perfor-
mance.
Because the shared learning process determines what does
not work rather than what does work, each receiver can
advance its level of subscription only through actual join-
experiments. If the suppression algorithm were completely
exclusionary, then the convergence time could still be very
large because each receiver would have to wait its turn to run
an experiment. Instead, we allow experimental overlap if the
pending level is the same as or less than the level in progress.
This gives newer receivers with lower levels of subscription
an opportunity to conduct experiments in the presence of a
large population of established receivers at higher levels of
subscription. Although this mechanism allows experimental
overlap,a receiver that causes an overlapcan condition its re-
sponse accordingly by reacting more conservatively than in
the non-overlappedcase. The intuition behind this scheme is
that high-layer receivers allow low-layer receivers to quickly
adapt to their stable level of subscription. As the low-layer
receivers adapt, their join-experiment frequency falls off and
the high-layer receivers will again find idle periods in which
to conduct join-experiments.
This technique for sharing information relies on the fact
that the network signals congestion by dropping packets
across all layers of the distribution. Under a priority-drop
policy, receivers not subscribed to the experimental layer
would not see packet loss and would not know the experi-
.
L F R
.
T
D
T
D
T
D
T
J
L F R
. .
T
D
(drop)
.
L F
F = our layer is highest of recently added layers
L < T
L > T
L > T = loss rate exceeds theshold
R = our layer was recently added
L = packet loss
(drop)
(relax) (add)
M
D
S
H
S
M
D
H
Hysteresis
Drop
Steady
Measurement
Figure 6: The receiver protocol state machine.
ment failed. In short, a priority-drop policy interferes with
the scalability of RLM.
3.4 The RLM State Machine
Figure 6 elaborates the protocolsketched in the previous sec-
tion. There are four states: steady-state (S), hysteresis state
(H), measurement state (M), and drop state (D). Each state
transition is labeled with the reason for the transition, either
packet loss or a timeout. Actions associated with a transition
are indicated in parentheses.
Join-timers (
) are randomized to avoid protocol syn-
chronization effects [15], while detection-timers (
) are
set to a scaled value of the detection-time estimator. The
add action implies that we subscribe to the next layer in the
multicast group hierarchy, while the drop action implies that
we drop the current layer and multiplicatively increase the
join-timer for that layer. The relax action implies that we
multiplicatively decrease the join-timer for the current layer.
There are two types of loss actions: a fast reaction to a sin-
gle packet loss (indicated by
) and a slower reaction to a
sustained loss rate. The loss rate is measured with a short-
term estimator and action is taken if the estimator exceeds a
configured threshold (indicated by
).
In the S state, there is always a pending join-timer (unless
the receiver is subscribed to all available layers). When the
join-timer expires, we broadcast an explicit notification mes-
sage to the group and add a layer. Upon reception of the join-
experiment message, a receiver notes the experiment start
time for that layer. In this way, we track the join-experiment
activity at each layer and deem an experiment “in progress”
if the time since the experiment started is less than
