scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Optimal Adaptation Trajectories for Block-Request Adaptive Video Streaming

23 Dec 2013-pp 1-8
TL;DR: This paper evaluates two HAS clients, Microsoft SmoothStreaming and an own streaming client that supports the recently adopted HAS standard Dynamic Adaptive Streaming over HTTP (DASH), in an indoor Wireless Local Area Network (WLAN) emulated with a high degree of precision.
Abstract: Block-Request Adaptive Streaming (BRAS), in form of its most prominent representative HTTP-Based Adaptive Streaming (HAS), is about to become the dominating technology for video delivery over the Internet. One of the challenges in the development of BRAS clients is the design of mechanisms that dynamically adapt the streamed video quality to network conditions, in order to maximize user's Quality of Experience (QoE). The main contribution of this paper is an approach to calculating optimal adaptation trajectories. This approach not only allows to benchmark the performance of any streaming client, it also provides the possibility to study the impact of the networking environment, and of configuration parameters such as the start-up delay, number of available video representations, etc., on the achievable streaming performance. Since, to the best of our knowledge, there exist no widely accepted or standard approach to measure QoE for BRAS, we alternatively maximize the average video bit-rate, minimize the number of quality switches, and impose a hard constraint on the absence of rebuffering events. Further, we evaluate two HAS clients, Microsoft SmoothStreaming and our own streaming client that supports the recently adopted HAS standard Dynamic Adaptive Streaming over HTTP (DASH), in an indoor Wireless Local Area Network (WLAN) emulated with a high degree of precision. We compare their performance with the optimal client, and explore the configuration parameter space of the DASH client. Finally, we evaluate the impact of start-up delays and number of available video representations on achievable streaming performance.

Summary (3 min read)

I. INTRODUCTION

  • Recently, video streaming has become one of the biggest sources of traffic on the Internet [1] .
  • Even in an indoor residential or office WLAN, the static user is typically exposed to interference, cross-traffic, and fading effects.
  • It also offers reliable communication by means of retransmitting lost packets, which enables usage of efficient video compression technologies that are particularly sensitive to packet losses (the loss of an I-frame may result in several seconds of corrupted playback).
  • Neither is any of the existing clients widely accepted as a basis for performance comparison.

III. OPTIMAL CLIENT

  • The authors present the optimization metric that they use, and their approach to calculating optimal adaptation trajectories.
  • Main factors that influence QoE include (i) duration and distribution of frozen frames, (ii) properties of the adaptation trajectory, such as minimum bit-rate, average bit-rate, frequency and magnitude of bit-rate switches, and (iii) start-up delay.
  • Therefore, the authors use the following objectives and constraints for optimization of adaptation trajectories.
  • Obviously, the earliest time when the playback can start is when the MPD file and the first segment of the representation with the lowest bit-rate are downloaded.
  • Therefore, the authors subsequently solve the following optimization problem OP2 in oder to select the optimum solution of OP1 that has the minimum number of quality switches.

A. DASH client

  • The details on the adaptation logic of their DASH client are provided in [18] .
  • A prototype has been implemented as a plugin for the VLC player [19] .
  • If, however, the throughput is high enough, no delays are introduced, the buffer level rises above β max , and the video quality is decreased until the buffer level starts to decrease again.
  • It takes into account the current throughput, its fluctuations, as well as the buffer level, which can be interpreted as the integrated mismatch between the throughput and the video bit-rate.
  • They control its sensitivity to bandwidth fluctuations, speed of convergence, efficiency of bandwidth utilization, and the probability of buffer underruns.

B. Microsoft SmoothStreaming client

  • The MSS streaming solution is supported by most major Internet browsers and is deployed by several popular video on demand websites.
  • The MSS client was used in several studies on BRAS client behavior and was found to outperform several competing approaches.
  • Since the implementation of MSS is closed, the exact operation of its adaptation logic is not known.
  • The MSS client version the authors used (Silverlight 5.1.20125.0) seems to maintain a target buffer level of approximately 30 seconds.
  • In the buffering phase, the player will request the segments as fast as possible to fill the buffer.

A. Setting

  • The emulated part of The video traffic was always routed via the station with the second lowest maximum throughput of 1.7 Mbps.
  • In addition to the video traffic, the authors generated synthetic background traffic mimicking the behavior of 14 HTTP clients, based on the stochastic model from Pries at al. [22] .
  • The authors assured that the differences in segment sizes between DASH and MSS was small enough to be negligible.
  • Depending on the format of the manifest file, the client might not know the actual segment bit-rate (and thus Fig. 2 .
  • Mean, minimum, maximum (the latter two are shown as percentage of the mean), also known as Video bit-rate variation.

B. Experiment 1

  • In this experiment the authors evaluate two metrics.
  • The top figure depicts the increase of the average video bit-rate of optimal trajectories depending on the start-up delay, for two numbers of available video representations: 6 and 14.
  • Maybe somewhat surprisingly, the improvement from a start-up delay of 60 sec is only around 12%.
  • Note that in real deployment scenarios, a high number of available video representations might be quite unrealistic, since it results in higher encoding and storage costs for content providers.
  • Thus, the higher amount of quality switches for 14 representations might partially be explained by the inexact calculations.

C. Experiment 2

  • The authors compare the performance of the MSS client, and of the DASH client with different configurations, with optimal performance.
  • In all configurations of the DASH client, the critical buffer threshold was set to β crit = 2s.
  • The lower threshold β min of the target buffer interval varied between 5s and 20s.
  • The authors observe that the DASH client always outperforms the MSS client w.r.t. the average video bit-rate (78% to 90% of the optimum vs. 62%).
  • It also outperforms the MSS client w.r.t. the total duration of frozen frames, and, for some of the configurations, w.r.t. the number of quality switches.

D. Experiment 3

  • The traffic of the two clients were routed over the same wireless station.
  • Figure 4b shows various performance metrics as Figure 4a did for the case of a single client, except that the values are now averages over the runs and over the two clients.
  • In fact, the authors observe that the conclusions from experiment 2 still hold for the two clients setting.
  • The authors observe that optimal trajectories have almost perfect fairness since all values are approximately 0.
  • Further, the authors observe that the fairness of their DASH client is comparable or better than that of the MSS client.

VI. CONCLUSION

  • Further, the authors evaluated the behavior of the widely deployed MSS video streaming client and their own client supporting the recently adopted DASH standard in a WLAN, and compared their performance with the optimum.
  • It also includes performance evaluation under a broader spectrum of network conditions.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Optimal Adaptation Trajectories for Block-Request
Adaptive Video Streaming
Konstantin Miller
, Nicola Corda
, Savvas Argyropoulos
, Alexander Raake
and Adam Wolisz
Technische Universit
¨
at Berlin, Germany
Email: {konstantin.miller, adam.wolisz}@tu-berlin.de
Telekom Innovation Laboratories (T-Labs), Berlin, Germany
Email: {nicola.corda, savvas.argyropoulos, alexander.raake}@telekom.de
Abstract—Block-Request Adaptive Streaming (BRAS), in form
of its most prominent representative HTTP-Based Adaptive
Streaming (HAS), is about to become the dominating technology
for video delivery over the Internet. One of the challenges in
the development of BRAS clients is the design of mechanisms
that dynamically adapt the streamed video quality to network
conditions, in order to maximize user’s Quality of Experience
(QoE).
The main contribution of this paper is an approach to
calculating optimal adaptation trajectories. This approach not
only allows to benchmark the performance of any streaming
client, it also provides the possibility to study the impact of the
networking environment, and of configuration parameters such
as the start-up delay, number of available video representations,
etc., on the achievable streaming performance. Since, to the best
of our knowledge, there exist no widely accepted or standard
approach to measure QoE for BRAS, we alternatively maximize
the average video bit-rate, minimize the number of quality
switches, and impose a hard constraint on the absence of re-
buffering events.
Further, we evaluate two HAS clients, Microsoft Smooth-
Streaming and our own streaming client that supports the
recently adopted HAS standard Dynamic Adaptive Streaming
over HTTP (DASH), in an indoor Wireless Local Area Network
(WLAN) emulated with a high degree of precision. We compare
their performance with the optimal client, and explore the
configuration parameter space of the DASH client.
Finally, we evaluate the impact of start-up delays and num-
ber of available video representations on achievable streaming
performance.
I. INTRODUCTION
Recently, video streaming has become one of the biggest
sources of traffic on the Internet [1]. Fast home broadband con-
nections, connected TV sets, and the spread of WiFi/3G/4G-
enabled mobile terminals are among the key drivers/enablers
for its growing popularity. In addition to faster connections,
technological improvements in video coding and video de-
livery have made it possible for more users to watch video
content online.
Due to the extreme heterogeneity of end-user devices and
types of network connections, it is not possible to use the same
representation of a video for each streaming session. It must
be adapted to capabilities of the device, such as processing
power and display properties, and to network conditions such
as available bandwidth, latency, jitter, and packet loss rate on
network path(s) from the content source(s) to the video client.
Moreover, it is not sufficient to perform the configuration once
for each streaming session. A user might, e.g., experience con-
tinuous throughput fluctuations ranging from tens of kilobits
to tens of megabits per second. This effect is especially visible
in the more and more deployed case of wirelessly connected
users. Even in an indoor residential or office WLAN, the static
user is typically exposed to interference, cross-traffic, and
fading effects. The link quality fluctuations are even stronger in
the case of mobile users. Thus, it is necessary to continuously
adapt the representation of the video in order to achieve a
satisfactory QoE.
Recently, HAS, a variant of BRAS, has become one of the
dominating technologies for adaptive video streaming. With
BRAS, the video is segmented in chunks of several seconds
duration, and each segment is available at the content source(s)
in several representations, each representation providing a
different encoding bit-rate. Further, each segment starts with a
random access point of the stream, thus allowing a video client
to concatenate segments from different representations during
the playback. With HAS, a video client issues HTTP GET or
GET RANGE requests to download individual segments. The
meta information about available segments and representations
is downloaded by the client prior to starting the streaming
session in form of an XML file, called Manifest or Media
Presentation Description (MPD) file.
One of the benefits of HAS is that it is leveraging on the
ubiquitous and highly optimized HTTP delivery infrastructure,
including Content Delivery Networks (CDNs), caches, proxies,
etc. Also, HTTP is usually allowed to traverse Network
Address Translation (NAT) devices and firewalls, in contrast
to other application layer protocols. Further, HAS has good
scalability properties since the streaming logic resides within
the client, thus relieving the server from keeping extensive
state, performing adaptation tasks, etc.
Another important feature of HTTP is its deployment on top
of the Transmission Control Protocol (TCP). From the point
of view of adaptive streaming, this carries both advantages
and disadvantages. On the one hand, TCP offers built-in
congestion control and congestion avoidance mechanisms, that
are necessary to maintain the stability of the network, as well
as to ensure some sort of fairness of resource allocation among
competing flows. It also offers reliable communication by
means of retransmitting lost packets, which enables usage of
efficient video compression technologies that are particularly

sensitive to packet losses (the loss of an I-frame may result
in several seconds of corrupted playback). On the other hand,
retransmission of lost packets results in delaying further pack-
ets, which makes TCP less suitable for low-delay streaming.
Further, TCP reacts to packet losses and to sporadic transmis-
sion delay peaks by reducing its sending rate, unnecessarily
degrading the QoE of a streaming session. Finally, the complex
dynamics of TCP make throughput estimation and prediction,
which are essential for fast and robust adaptation of video
quality, more challenging.
Currently, there are several popular commercial implemen-
tations of the HAS technology, including Microsoft Smooth-
Streaming (MSS), Adobe Dynamic Streaming (ADS), Apple
HTTP Live Streaming (HLS), as well as a number of develop-
ments reported in the research literature. Despite the growing
attention from the research community, however, there exist
several open issues.
One of them involves the methodology for performance
evaluation of streaming clients. In a typical performance
evaluation, we either compare an approach to some predefined
requirements, or we measure its gain w.r.t. some state-of-the-
art solutions, or we compare its performance to the maximum
performance that can be achieved in a certain setting. There
exist, however, no widely accepted benchmarks to measure
the performance of BRAS clients in best-effort networks.
Neither is any of the existing clients widely accepted as a basis
for performance comparison. Finally, little has been done on
developing approaches to calculating optimal performance of
a BRAS client in a given setting. Such an approach, however,
would not only allow to benchmark the performance of any
streaming client, it would also allow to study the impact of the
networking environment and of configuration parameters such
as the start-up delay, number of available video representa-
tions, etc., on the streaming performance. The first contribution
of this paper addresses this issue by presenting an approach
to calculate optimal adaptation trajectories, given the complete
information on the throughput process (that is, the amount of
data that can be downloaded until time t, for each t).
This approach can be used in three different ways. First, we
can calculate an optimal trajectory for a throughput process
that was recorded by a streaming client during a streaming
session. In many cases, however, streaming clients introduce
delays between subsequent requests so that we no longer can
calculate an optimal trajectory from the recorded throughput
since we do not know which throughput the client could
have achieved during the gaps. Thus, instead of using a trace
recorded by a streaming client, we might use a trace recorded
by a continuous TCP flow under the same network conditions.
Finally, optimal trajectories can be calculated for artificial
throughput processes in order to study the impact their features
have on the optimal performance.
As a second contribution, we evaluate two HAS clients:
Microsoft SmoothStreaming and our own streaming client
supporting the recently adopted HAS standard DASH. Due to
the immense heterogeneity of possible deployment scenarios
for video streaming clients, it is challenging to perform a
solid performance evaluation, since it would require various
test runs in very different networking environments. Wireless
networks, however, are among the most challenging environ-
ments for streaming. In addition to cross-traffic, the throughput
in, e.g., a WLAN is impacted by fading effects, packet losses,
Media Access Control (MAC) layer retransmissions, etc. The
usage of TCP as transport protocol makes the resulting net-
work dynamics even more complex. Thus, we selected for the
evaluation an indoor WLAN cell, which we emulated with a
high degree of precision. We compare the trajectories of the
two clients with the optimum and explore the parameter space
of our DASH client w.r.t. its performance in wireless networks.
The structure of the paper is as follows. In Section II,
we review the related work. In Section III, we present the
optimization metric that we use and our approach to cal-
culating optimal adaptation trajectories. In Section IV, we
briefly present the two streaming clients used for performance
evaluation. Section V presents the setting and the results of
the evaluation. Finally, Section VI concludes the paper.
II. R
ELATED WORK
An approach to calculating optimal adaptation trajectories
using a Markov decision process is presented by Jarnikov et
al. [2]. With this approach, an optimal strategy is calculated for
a given distribution function of segment download times. The
objective function is a linear function giving constant penalty
to playback interruptions and changes of video quality, and a
reward proportional to the selected video quality. The authors
perform a numerical evaluation of the approach using fixed,
uniform and normal distributions of the available bandwidth.
A potential limitation of this approach is that temporal corre-
lation of segment download times is not considered.
Several studies present designs of heuristic adaptation strate-
gies. They are usually accompanied by more or less extensive
evaluations that, however, sometimes exhibit one or several
of the following deficiencies. Simulations and emulations are
sometimes too simplified, e.g., TCP and/or MAC layer behav-
ior is not modeled. In many cases, no cross-traffic is present,
throughput fluctuations are achieved by piecewise continuous
throughput limitations using traffic control tools like tc or
DummyNet. Another common drawback is an oversimplified
evaluation metric, such as, e.g., the total re-buffering time,
without taking into account the average video bit-rate, the total
number of quality switches, or the duration of the start-up
delay.
Tappayuthpijarn et al. [3] presented an adaptation heuristic
based on a prediction of layer 2 throughput in an Long-Term
Evolution (LTE) cell. The heuristic is evaluated by comparison
with non-adaptive streaming in an emulated LTE cell with 8
concurrent clients.
Zhou et al. [4] present an adaptation logic based on a
proportional derivative (PD) controller.
Evensen et al. [5] investigate mechanisms that allow to
aggregate bandwidth over multiple network interfaces in het-
erogeneous networks.

M
¨
uller et al. [6] evaluated the MSS, ADS, HLS, and their
own DASH clients in an emulated environment using three
application-layer traces recorded from a 3G connection while
driving on the freeway. The metrics were: (i) average bit-rate
during the streaming session, (i) number of quality switches,
and (iii) time spent in re-buffering.
Akhshabi et al. [7] experimentally evaluated the MSS,
ADS, and the Netflix HAS clients. The authors focus on
three aspects: reaction to persistent or short-term throughput
changes, the ability of two players to properly operate on a
shared network path, and if the player is able to sustain a
short playback delay and thus perform well with live content.
The authors identified significant inefficiencies in each of the
studied players.
Recently, performance and fairness issues with multiple
streaming clients competing for bottleneck bandwidth moved
into the focus of the research community. Several studies
specifically look into client design aspects affecting such
settings.
Jiang et al. [8] argue that the throughput estimation com-
ponent is crucial for the behavior of a streaming client and
compares 4 estimators, of which harmonic mean turns out to
perform best. The study also show that random instead of fixed
delays between subsequent requests improve performance
when multiple clients compete for bottleneck bandwidth.
Huang et al. [9] also analyze the impact, bandwidth esti-
mation techniques have on the behavior of streaming clients.
Further work is presented by Liu et al. [10], and Tian et
al. [11].
Studies that investigate QoE models for adaptive streaming
include Cranley et al. [12], Ninassi et al. [13], and Seufert et
al. [14].
III. O
PTIMAL CLIENT
In this section, we present the optimization metric that
we use, and our approach to calculating optimal adaptation
trajectories.
The main challenge in designing efficient adaptation strate-
gies is that path throughput is a random process, whose value
is difficult to reliably predict for the relevant time horizon,
ranging from several seconds to several tens of seconds. (We
use the term ”throughput process” to denote the total amount
of data V (t) received by a client during the time [0,t].) The
question that we answer in this section is how to calculate an
optimal trajectory having perfect knowledge of V (t).
In order to address this question, we first need to select an
appropriate optimization objective. Main factors that influence
QoE include (i) duration and distribution of frozen frames,
(ii) properties of the adaptation trajectory, such as minimum
bit-rate, average bit-rate, frequency and magnitude of bit-rate
switches, and (iii) start-up delay. To the best of our knowledge,
there exist no widely accepted or standard QoE metric, which
quantifies human perception of all these factors together.
Therefore, we use the following objectives and constraints for
optimization of adaptation trajectories.
We use start-up delay as an independent variable, that is,
an optimal trajectory is calculated for a given start-up delay.
Further, we impose a hard constraint on the absence of buffer
underruns, meaning that an optimal trajectory is not allowed
to have buffer underruns. As for optimization objective, we
first maximize the average video bit-rate over the duration of
the streaming session. This maximization in general results in
a space of optimal solutions that are potentially prone to fre-
quent video quality fluctuations. Therefore, we subsequently
minimize the number of quality switches.
We will use the following notation. We denote by S
MPD
the
size of the MPD file. We denote by R the set of available
representations, by n the number of segments in the video,
and by τ the duration of the segments. (We assume the same
duration for all segments but the approach also works for
segments of different durations.) Further, we denote by S
i,j
the size of segment i from representation j.
Obviously, the earliest time when the playback can start is
when the MPD file and the first segment of the representation
with the lowest bit-rate are downloaded. We denote this time
by T
E
. (A client does not always have to download the first
segment in the lowest representation. Still, given a throughput
process V (t), T
E
is always well-defined, and will be used in
the following as a reference.)
We define T
S
to be the time from the start of the download
(t =0) until the playback is started. It must hold T
S
T
E
.
The important value for optimization, however, is not T
S
but
the time between the earliest possible time when the playback
can start and the actual start of playback,
˜
T
S
= T
S
T
E
,
which we define as the start-up delay. The reason is that the
time T
E
cannot be influenced by the client, while
˜
T
S
is a
configurable parameter of the client’s adaptation strategy.
The resulting playback deadlines for the individual segments
are given by D
i
= T
S
+(i 1) · τ. The maximum amount of
data a video player can download until the playback deadline
of segment i is thus given by V (D
i
).
In order to formulate the optimization problem, we denote
by x
ij
∈{0, 1} the optimization variables stating if the client
downloads segment i from representation j or not.
In the following, we first maximize the average video
bit-rate (which is equivalent to maximization of the total
amount of data downloaded by the video client), demanding
the absence of buffer underruns. We obtain the following
optimization problem.
(OP1) max
n
i=1
m
j=1
S
ij
x
ij
(1)
s.t.
m
j=1
x
ij
1 for all i =1,...,n
(2)
k
i=1
m
j=1
S
ij
x
ij
V (D
k
) for all k =1,...,n
(3)

Here, constraint (2) ensures that each segment is down-
loaded from at least one representation, while constraint (3)
ensures that each segment is downloaded before its playback
deadline. Note that constraint (3) implicitly accounts for
the configured start-up delay (included in the definition of
playback deadlines D
i
).
Problem OP1 has in general a space of optimal solutions
that are more or less prone to video quality fluctuations.
Unnecessary quality switches, however, significantly impact
the QoE. Therefore, we subsequently solve the following
optimization problem OP2 in oder to select the optimum
solution of OP1 that has the minimum number of quality
switches. We denote by V
the optimal objective value of
problem OP1.
(OP2) min
1
2
n1
i=1
m
j=1
(x
ij
x
i+1,j
)
2
(4)
s.t.
m
j=1
x
ij
1 for all i =1,...,n
(5)
k
i=1
m
j=1
S
ij
x
ij
V (D
k
) for all k =1,...,n
(6)
n
i=1
m
j=1
S
ij
x
ij
V
(7)
While constraints (5) and (6) are the same as constraints (2)
and (3) in OP1, constraint (7) ensures that the minimization
of the new objective function (4) (total number of quality
switches) does not result in a sub-optimal value for the average
video bit-rate.
The problem OP1 is known as a Multiple-Choice Nested
Knapsack Problem (MCNKP) [15], [16]. More precisely, it is
a special case, where the values of the items and the weights of
the items are equal. (This variant of the Knapsack Problem is
sometimes referred to as the Subset Sum Problem.) MCNKP
is NP-hard but there exist pseudo-polynomial time algorithms.
Problem OP2 is a Quadratic MCNKP. We solve both problems
with the software Gurobi [17].
IV. S
TREAMING CLIENTS
In the following we briefly present the two HAS clients we
selected for performance evaluation in wireless networks.
A. DASH client
The details on the adaptation logic of our DASH client are
provided in [18]. A prototype has been implemented as a plug-
in for the VLC player [19].
In short, the adaptation logic tries to maintain the buffer
level within a certain target interval [β
min
max
]. If the buffer
level is below β
min
, the video quality is increased step by
step, until the buffer level starts to rise. If the buffer is above
the middle of the target interval 0.5 · (β
min
+ β
max
) but the
throughput is too low to switch to the next representation,
subsequent requests are delayed in order not to let the buffer
level rise. If, however, the throughput is high enough, no
delays are introduced, the buffer level rises above β
max
, and
the video quality is decreased until the buffer level starts to
decrease again. The throughput is averaged over the last Δ
t
seconds. In order to avoid buffer underruns by all means, if
the buffer level falls below a configurable critical threshold
β
crit
, the lowest video quality is selected immediately. An extra
start-up phase provides a fast ramp-up of the video quality to
the available throughput.
The operation of the algorithm is similar to a PID controller
with hysteresis and some additional tweaks. It takes into
account the current throughput, its fluctuations, as well as
the buffer level, which can be interpreted as the integrated
mismatch between the throughput and the video bit-rate.
Further, instead of a target buffer level, it has a target interval.
In total, the algorithm offers 10 parameters that can be
tuned to optimize its behavior. They control its sensitivity to
bandwidth fluctuations, speed of convergence, efficiency of
bandwidth utilization, and the probability of buffer underruns.
See Miller et al. [18] for more details.
B. Microsoft SmoothStreaming client
The MSS streaming solution is supported by most major
Internet browsers and is deployed by several popular video on
demand websites. The MSS client was used in several studies
on BRAS client behavior and was found to outperform several
competing approaches.
Since the implementation of MSS is closed, the exact
operation of its adaptation logic is not known. However,
several aspects of its behavior could be observed in different
studies and also in our own experiments.
MSS client uses a fixed segment duration of 2 seconds. It
typically starts to request the first segment from the lowest
representation. It then switches between the representations
step by step in a smooth manner in order to avoid abrupt
changes of quality that might impact QoE. Further, the quality
is switched only when there is a certain probability that the
throughput can sustain the new video bit-rate.
The MSS client version we used (Silverlight 5.1.20125.0)
seems to maintain a target buffer level of approximately 30
seconds. Its operation can be split in two phases, buffering and
steady-state. In the buffering phase, the player will request the
segments as fast as possible to fill the buffer. In the steady-
state phase, the segments are requested every 2 seconds.
V. E
VA L UAT IO N
In this section, we present the setting and the results of the
performance evaluation of the two selected HAS clients and
the optimum behavior, in a WLAN.
A. Setting
For the evaluation we emulate a typical Internet path starting
in an IEEE 802.11a WLAN in an indoor environment. We
use a site-specific model by Al-Bado et al. [20], implemented
in the NS-3 [21] network simulator. The emulated part of

Fig. 1. Evaluation setup. Clients and servers are interconnected via an
emulated IEEE 802.11a cell that is based on a site-specific model of the
Berlin Open Wireless Lab (BOWL) testbed [20].
the network is connected to PCs hosting client and server
software via two gigabit Ethernet interfaces, as shown in
Figure 1. The model consists of eight nodes and each of
the possible 56 unidirectional links is modeled separately. We
use seven nodes as stations and one as the access point. To
give an impression of the link qualities, a TCP transfer of
100MB of data between the access point and each individual
station achieves an average throughput of approximately (in
[Mbps]): 1.4, 1.7, 19, 19, 21, 21, 21. (These values refer to
TCP throughput of individual links in the absence of cross-
traffic.) The one-way delays on the emulated links connecting
the stations and the access points with Ethernet interfaces of
the emulation host were set to a constant value of 1 ms.
The video traffic was always routed via the station with the
second lowest maximum throughput of 1.7 Mbps. In addition
to the video traffic, we generated synthetic background traffic
mimicking the behavior of 14 HTTP clients, based on the
stochastic model from Pries at al. [22]. Each of the wireless
stations carried traffic of two such clients.
The video sequence that we used was Big Buck Bunny [23].
We encoded the raw video data in 6 and 14 representations,
distributing the target representation bit-rates logarithmically
between 100 kbps and 5 Mbps. We set the Group of Pictures
(GOP) size to 2 seconds, which is the maximum allowed
for MSS. The encoded data was split into segments and
two manifest files were generated, one for DASH and one
for MSS. We assured that the differences in segment sizes
between DASH and MSS was small enough to be negligible.
It constituted on average 0.17% and was bounded from above
by 1.34%. The distribution of segment sizes is illustrated in
Figure 2. The sequence is slightly longer than 596 seconds so
we obtained 298 full segments and one short segment, which
we omitted.
Note that bit-rate fluctuations across segments of the same
representation may significantly affect the performance of the
streaming client. Depending on the format of the manifest file,
the client might not know the actual segment bit-rate (and thus
Fig. 2. Video bit-rate variation: mean, minimum, maximum (the latter two
are shown as percentage of the mean). Mean video bit-rates were configured
to be logarithmically distributed between 100 kbps and 5 Mbps. (Note the
logarithmic scale of the y-axis.)
the segment size) in advance, but only the average bit-rate of
the representation. Due to variable bit-rate encoding, however,
the segment bit-rate may fluctuate by up to a factor of 10
and more. Our dataset was encoded such as to keep these
fluctuations small and thus they are bounded by 95% of the
mean, as shown in Figure 2.
In order to have a fair comparison, we padded the DASH
manifest file with random characters to make it of the same
size as the joint size of the three files that need to be down-
loaded by the MSS client: the Hypertext Markup Language
(HTML) file, the Silverlight Application Package (XAP) file
and the manifest file.
In order to compare the performance of the MSS and the
DASH clients to the optimum, we proceeded as follows.
Each experiment with a video client was followed by an
experiment under the same conditions, where the video client
was replaced by a TCP flow lasting for the duration of the
video sequence. The throughput process of the TCP flow was
then used as input V (t) for optimization problems OP1 and
OP2, to calculate optimal adaptation trajectories. We didn’t use
the throughput process as recorded by the video client since it
may be suboptimal, because video clients typically introduce
gaps between subsequent requests in order to prevent their
buffer level from exceeding a certain threshold.
All experiments were repeated approximately 50 times.
Confidence intervals in some of the figures are omitted to
improve readability.

Citations
More filters
Journal ArticleDOI
TL;DR: The technical development of HAS, existing open standardized solutions, but also proprietary solutions are reviewed in this paper as fundamental to derive the QoE influence factors that emerge as a result of adaptation.
Abstract: Changing network conditions pose severe problems to video streaming in the Internet. HTTP adaptive streaming (HAS) is a technology employed by numerous video services that relieves these issues by adapting the video to the current network conditions. It enables service providers to improve resource utilization and Quality of Experience (QoE) by incorporating information from different layers in order to deliver and adapt a video in its best possible quality. Thereby, it allows taking into account end user device capabilities, available video quality levels, current network conditions, and current server load. For end users, the major benefits of HAS compared to classical HTTP video streaming are reduced interruptions of the video playback and higher bandwidth utilization, which both generally result in a higher QoE. Adaptation is possible by changing the frame rate, resolution, or quantization of the video, which can be done with various adaptation strategies and related client- and server-side actions. The technical development of HAS, existing open standardized solutions, but also proprietary solutions are reviewed in this paper as fundamental to derive the QoE influence factors that emerge as a result of adaptation. The main contribution is a comprehensive survey of QoE related works from human computer interaction and networking domains, which are structured according to the QoE impact of video adaptation. To be more precise, subjective studies that cover QoE aspects of adaptation dimensions and strategies are revisited. As a result, QoE influence factors of HAS and corresponding QoE models are identified, but also open issues and conflicting results are discussed. Furthermore, technical influence factors, which are often ignored in the context of HAS, affect perceptual QoE influence factors and are consequently analyzed. This survey gives the reader an overview of the current state of the art and recent developments. At the same time, it targets networking researchers who develop new solutions for HTTP video streaming or assess video streaming from a user centric point of view. Therefore, this paper is a major step toward truly improving HAS.

746 citations

Proceedings ArticleDOI
15 Dec 2014
TL;DR: In this work, the influence of several adaptation parameters, namely, switch amplitude, switching frequency, and recency effects, on Quality of Experience (QoE) is investigated and a simplified QoE model for HAS is presented, which only relies on the switch amplitude and the playback time of each layer.
Abstract: HTTP Adaptive Streaming (HAS) is employed by more and more video streaming services in the Internet. It allows to adapt the downloaded video quality to the current network conditions, and thus, avoids stalling (i.e., playback interruptions) to the greatest possible extend. The adaptation of video streams is done by switching between different quality representation levels, which influences the user perceived quality of the video stream. In this work, the influence of several adaptation parameters, namely, switch amplitude (i.e., quality level difference), switching frequency, and recency effects, on Quality of Experience (QoE) is investigated. Therefore, crowdsourcing experiments were conducted in order to collect subjective ratings for different adaptation-related test conditions. The results of these subjective studies indicate the influence of the adaptation parameters, and based on these findings a simplified QoE model for HAS is presented, which only relies on the switch amplitude and the playback time of each layer.

116 citations


Cites background from "Optimal Adaptation Trajectories for..."

  • ...The approach presented in [23] can be adapted to incorporate a value function that takes amplitudes and times on individual layers into account....

    [...]

Journal ArticleDOI
TL;DR: Results indicate that the video quality has to be maximized first, and that the number of quality switches is less important, and a method to compute the optimal QoE-optimal adaptation strategy for HAS on a per user basis with mixed-integer linear programming is presented.

69 citations


Cites background or methods from "Optimal Adaptation Trajectories for..."

  • ...A two-step approach for modeling the optimal QoE adaptation for a single user is provided in [30]....

    [...]

  • ...In the first step, the downloaded data volume is maximized, since [30] assumes that larger data volume results into higher video quality....

    [...]

  • ...It has to be noted that [30] suggests to maximize the downloaded volume which leads to a different quality value function to maximize end user’s perception....

    [...]

  • ...The VOLUME value function resembles the approach of [30] and maximizes the downloaded data volume....

    [...]

  • ...Based on [30], we use mixed integer linear programing to find the optimal adaptation strategy, but we investigate different objective functions in the first step for maximizing QoE....

    [...]

Journal ArticleDOI
TL;DR: This work introduces an adaptation algorithm for HTTP-based live streaming called LOLYPOP (short for low-latency prediction-based adaptation), which is designed to operate with a transport latency of a few seconds, and leverages Transmission Control Protocol throughput predictions on multiple time scales.
Abstract: Recently, Hypertext Transfer Protocol (HTTP)-based adaptive streaming has become the de facto standard for video streaming over the Internet. It allows clients to dynamically adapt media characteristics to the varying network conditions to ensure a high quality of experience (QoE)—that is, minimize playback interruptions while maximizing video quality at a reasonable level of quality changes. In the case of live streaming, this task becomes particularly challenging due to the latency constraints. The challenge further increases if a client uses a wireless access network, where the throughput is subject to considerable fluctuations. Consequently, live streams often exhibit latencies of up to 20 to 30 seconds. In the present work, we introduce an adaptation algorithm for HTTP-based live streaming called LOLYPOP (short for low-latency prediction-based adaptation), which is designed to operate with a transport latency of a few seconds. To reach this goal, LOLYPOP leverages Transmission Control Protocol throughput predictions on multiple time scales, from 1 to 10 seconds, along with estimations of the relative prediction error distributions. In addition to satisfying the latency constraint, the algorithm heuristically maximizes the QoE by maximizing the average video quality as a function of the number of skipped segments and quality transitions. To select an efficient prediction method, we studied the performance of several time series prediction methods in IEEE 802.11 wireless access networks. We evaluated LOLYPOP under a large set of experimental conditions, limiting the transport latency to 3 seconds, against a state-of-the-art adaptation algorithm called FESTIVE. We observed that the average selected video representation index is by up to a factor of 3 higher than with the baseline approach. We also observed that LOLYPOP is able to reach points from a broader region in the QoE space, and thus it is better adjustable to the user profile or service provider requirements.

56 citations


Cites methods from "Optimal Adaptation Trajectories for..."

  • ...Several studies assume perfect information about future throughput to compute optimal adaptation trajectories that can be used to benchmark existing algorithms and evaluate the potential for performance increase [Miller et al. 2013; Zou et al. 2015]....

    [...]

Journal ArticleDOI
TL;DR: In this paper, a proportional-integral-derivative (PID) controller is proposed to support unicast streaming sessions in a dense wireless access network, and a control-theoretic approach is used to efficiently utilize available wireless resources, providing high quality of experience (QoE) to a large number of users.
Abstract: Recently, the way people consume video content has been undergoing a dramatic change. Plain TV sets, that have been the center of home entertainment for a long time, are losing ground to hybrid TVs, PCs, game consoles, and, more recently, mobile devices such as tablets and smartphones. The new predominant paradigm is: watch what I want, when I want, and where I want. The challenges of this shift are manifold. On the one hand, broadcast technologies such as DVB-T/C/S need to be extended or replaced by mechanisms supporting asynchronous viewing, such as IPTV and video streaming over best-effort networks, while remaining scalable to millions of users. On the other hand, the dramatic increase of wireless data traffic begins to stretch the capabilities of the existing wireless infrastructure to its limits. Finally, there is a challenge to video streaming technologies to cope with a high heterogeneity of end-user devices and dynamically changing network conditions, in particular in wireless and mobile networks. In the present work, our goal is to design an efficient system that supports a high number of unicast streaming sessions in a dense wireless access network. We address this goal by jointly considering the two problems of wireless transmission scheduling and video quality adaptation, using techniques inspired by the robustness and simplicity of proportional-integral-derivative (PID) controllers. We show that the control-theoretic approach allows to efficiently utilize available wireless resources, providing high quality of experience (QoE) to a large number of users.

47 citations

References
More filters
Book ChapterDOI
01 Jan 2010
TL;DR: With simulation based studies, the approach can be studied in detail at varying scales, with varying data applications, varying field conditions, and will result in reproducible and analyzable results.
Abstract: As networks of computing devices grow larger and more complex, the need for highly accurate and scalable network simulation technologies becomes critical. Despite the emergence of large-scale testbeds for network research, simulation still plays a vital role in terms of scalability (both in size and in experimental speed), reproducibility, rapid prototyping, and education. With simulation based studies, the approach can be studied in detail at varying scales, with varying data applications, varying field conditions, and will result in reproducible and analyzable results.

1,462 citations

Proceedings ArticleDOI
10 Dec 2012
TL;DR: A principled understanding of bit-rate adaptation is presented and a suite of techniques that can systematically guide the tradeoffs between stability, fairness, and efficiency are developed, which lead to a general framework for robust video adaptation.
Abstract: Many commercial video players rely on bitrate adaptation logic to adapt the bitrate in response to changing network conditions. Past measurement studies have identified issues with today's commercial players with respect to three key metrics---efficiency, fairness, and stability---when multiple bitrate-adaptive players share a bottleneck link. Unfortunately, our current understanding of why these effects occur and how they can be mitigated is quite limited.In this paper, we present a principled understanding of bitrate adaptation and analyze several commercial players through the lens of an abstract player model. Through this framework, we identify the root causes of several undesirable interactions that arise as a consequence of overlaying the video bitrate adaptation over HTTP. Building on these insights, we develop a suite of techniques that can systematically guide the tradeoffs between stability, fairness and efficiency and thus lead to a general framework for robust video adaptation. We pick one concrete instance from this design space and show that it significantly outperforms today's commercial players on all three key metrics across a range of experimental scenarios.

806 citations


"Optimal Adaptation Trajectories for..." refers background in this paper

  • ...[8] argue that the throughput estimation component is crucial for the behavior of a streaming client and compares 4 estimators, of which harmonic mean turns out to perform best....

    [...]

Proceedings ArticleDOI
23 Feb 2011
TL;DR: This paper focuses on the rate-adaptation mechanisms of adaptive streaming and experimentally evaluates two major commercial players (Smooth Streaming, Netflix) and one open source player (OSMF).
Abstract: Adaptive (video) streaming over HTTP is gradually being adopted, as it offers significant advantages in terms of both user-perceived quality and resource utilization for content and network service providers. In this paper, we focus on the rate-adaptation mechanisms of adaptive streaming and experimentally evaluate two major commercial players (Smooth Streaming, Netflix) and one open source player (OSMF). Our experiments cover three important operating conditions. First, how does an adaptive video player react to either persistent or short-term changes in the underlying network available bandwidth. Can the player quickly converge to the maximum sustainable bitrate? Second, what happens when two adaptive video players compete for available bandwidth in the bottleneck link? Can they share the resources in a stable and fair manner? And third, how does adaptive streaming perform with live content? Is the player able to sustain a short playback delay? We identify major differences between the three players, and significant inefficiencies in each of them.

729 citations


"Optimal Adaptation Trajectories for..." refers methods in this paper

  • ...[7] experimentally evaluated the MSS, ADS, and the Netflix HAS clients....

    [...]

Proceedings ArticleDOI
14 Nov 2012
TL;DR: This work measures three popular video streaming services -- Hulu, Netflix, and Vudu -- and finds that accurate client-side bandwidth estimation above the HTTP layer is hard, and rate selection based on inaccurate estimates can trigger a feedback loop, leading to undesirably variable and low-quality video.
Abstract: Today's commercial video streaming services use dynamic rate selection to provide a high-quality user experience. Most services host content on standard HTTP servers in CDNs, so rate selection must occur at the client. We measure three popular video streaming services -- Hulu, Netflix, and Vudu -- and find that accurate client-side bandwidth estimation above the HTTP layer is hard. As a result, rate selection based on inaccurate estimates can trigger a feedback loop, leading to undesirably variable and low-quality video. We call this phenomenon the "downward spiral effect", and we measure it on all three services, present insights into its root causes, and validate initial solutions to prevent it.

372 citations


"Optimal Adaptation Trajectories for..." refers background in this paper

  • ...[9] also analyze the impact, bandwidth estimation techniques have on the behavior of streaming clients....

    [...]

Proceedings ArticleDOI
10 Dec 2012
TL;DR: This paper develops a fully-functional DASH system, develops novel video rate control algorithms that balance the needs for video rate smoothness and high bandwidth utilization, and shows that a small video rate margin can lead to much improved smoothness in video rate and buffer size.
Abstract: Dynamic Adaptive Streaming over HTTP (DASH) is widely deployed on the Internet for live and on-demand video streaming services. Video adaptation algorithms in existing DASH systems are either too sluggish to respond to congestion level shifts or too sensitive to short-term network bandwidth variations. Both degrade user video experience. In this paper, we formally study the responsiveness and smoothness trade-off in DASH through analysis and experiments. We show that client-side buffered video time is a good feedback signal to guide video adaptation. We then propose novel video rate control algorithms that balance the needs for video rate smoothness and high bandwidth utilization. We show that a small video rate margin can lead to much improved smoothness in video rate and buffer size. The proposed DASH designs are also extended to work with multiple CDN servers. We develop a fully-functional DASH system and evaluate its performance through extensive experiments on a network testbed and the Internet. We demonstrate that our DASH designs are highly efficient and robust in realistic network environment.

313 citations

Frequently Asked Questions (19)
Q1. What have the authors contributed in "Optimal adaptation trajectories for block-request adaptive video streaming" ?

The main contribution of this paper is an approach to calculating optimal adaptation trajectories. This approach not only allows to benchmark the performance of any streaming client, it also provides the possibility to study the impact of the networking environment, and of configuration parameters such as the start-up delay, number of available video representations, etc., on the achievable streaming performance. Since, to the best of their knowledge, there exist no widely accepted or standard approach to measure QoE for BRAS, the authors alternatively maximize the average video bit-rate, minimize the number of quality switches, and impose a hard constraint on the absence of rebuffering events. The authors compare their performance with the optimal client, and explore the configuration parameter space of the DASH client. Finally, the authors evaluate the impact of start-up delays and number of available video representations on achievable streaming performance. Further, the authors evaluate two HAS clients, Microsoft SmoothStreaming and their own streaming client that supports the recently adopted HAS standard Dynamic Adaptive Streaming over HTTP ( DASH ), in an indoor Wireless Local Area Network ( WLAN ) emulated with a high degree of precision. 

Future work includes a study of adaptation strategies that, based on short-term prediction of TCP throughput and subsequent solutions of presented optimization problems OP1 and OP2 over small and medium time horizons, might increase the client ’ s performance towards the achievable optimum. 

The main challenge in designing efficient adaptation strategies is that path throughput is a random process, whose value is difficult to reliably predict for the relevant time horizon, ranging from several seconds to several tens of seconds. 

The throughput process of the TCP flow was then used as input V (t) for optimization problems OP1 and OP2, to calculate optimal adaptation trajectories. 

performance and fairness issues with multiple streaming clients competing for bottleneck bandwidth moved into the focus of the research community. 

It turned out that their client is able to outperform the MSS client in the evaluated setting by achieving an average video bit-rate that is between 78% and 90% of the optimum, while MSS achieves approximately 62%, with less quality fluctuations, less time spent in re-buffering, comparable fairness when two clients share a bottleneck link, and a lower average buffer level, which is especially beneficial for live content. 

Low values let the video client switch the video quality more often in order to more closely follow throughput fluctuations and thus increase the average video quality by better utilizing the available bandwidth. 

The one-way delays on the emulated links connecting the stations and the access points with Ethernet interfaces of the emulation host were set to a constant value of 1 ms. 

Jiang et al. [8] argue that the throughput estimation component is crucial for the behavior of a streaming client and compares 4 estimators, of which harmonic mean turns out to perform best. 

In order to avoid buffer underruns by all means, if the buffer level falls below a configurable critical threshold βcrit, the lowest video quality is selected immediately. 

They control its sensitivity to bandwidth fluctuations, speed of convergence, efficiency of bandwidth utilization, and the probability of buffer underruns. 

The authors didn’t use the throughput process as recorded by the video client since it may be suboptimal, because video clients typically introduce gaps between subsequent requests in order to prevent their buffer level from exceeding a certain threshold. 

low values increase the risk of buffer underruns but they increase the ”liveness” of the streaming session, which is important for transmission of live content. 

To give an impression of the link qualities, a TCP transfer of 100MB of data between the access point and each individual station achieves an average throughput of approximately (in [Mbps]): 1.4, 1.7, 19, 19, 21, 21, 21. 

It then switches between the representations step by step in a smooth manner in order to avoid abrupt changes of quality that might impact QoE. 

The metrics were: (i) average bit-rate during the streaming session, (i) number of quality switches, and (iii) time spent in re-buffering. 

Also note that the average number of switches required by the optimal trajectory is as low as approximately 2 (and, in fact, might be even lower due to the tolerance gap allowed in the optimization process). 

The authors perform a numerical evaluation of the approach using fixed, uniform and normal distributions of the available bandwidth. 

The reason is that despite the high fluctuation of the wireless link throughput (fading plus cross-traffic), 6 representations are enough to utilize all the available bandwidth, as the authors will also see in experiment 2.