THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

An example system may include one or more collectors, an analyzer, and a presentation module. The one or more collectors receive a plurality of data streams that include operational data for a plurality of application nodes. The plurality of data streams are captured and provided by a plurality of meters deployed on at least one cloud computing platform to respectively meter the plurality application nodes. The analyzer processes the plurality of data streams to generate real-time performance data for an application associated with the plurality of application nodes. The presentation module streams the real-time performance data to at least one stakeholder of the application for display via a dashboard. The real-time performance data includes one or more performance metrics describing the performance of plurality of the application nodes of the application.

Application monitoring for cloud-based architectures

Quality of Experience (QoE) has received much attention over the past years and has become a prominent issue for delivering services and applications. A significant amount of research has been devoted to understanding, measuring, and modelling QoE for a variety of media services. The next logical step is to actively exploit that accumulated knowledge to improve and manage the quality of multimedia services, while at the same time ensuring efficient and cost-effective network operations. Moreover, with many different players involved in the end-to-end service delivery chain, identifying the root causes of QoE impairments and finding effective solutions for meeting the end users’ requirements and expectations in terms of service quality is a challenging and complex problem. In this article, we survey state-of-the-art findings and present emerging concepts and challenges related to managing QoE for networked multimedia services. Going beyond a number of previously published survey articles addressing the topic of QoE management, we address QoE management in the context of ongoing developments, such as the move to softwarized networks, the exploitation of big data analytics and machine learning, and the steady rise of new and immersive services (e.g., augmented and virtual reality). We address the implications of such paradigm shifts in terms of new approaches in QoE modeling and the need for novel QoE monitoring and management infrastructures.

A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services

HTTP adaptive streaming (HAS) is the most recent attempt regarding video quality adaptation. It enables cheap and easy to implement streaming technology without the need for a dedicated infrastructure. By using a combination of TCP and HTTP it has the advantage of reusing all the existing technologies designed for ordinary web. Equally important is that HAS traffic passes through firewalls and works well when NAT is deployed. The rate adaptation controller of HAS, commonly called adaptive bitrate selection (ABR), is currently receiving a lot of attention from both industry and academia. However, most of the research efforts concentrate on a specific aspect or a particular methodology without considering the overall context. This paper presents a comprehensive survey of the most significant research activities in the area of client-side HTTP-based adaptive video streaming. It starts by decomposing the ABR module into three subcomponents, namely: resource estimation function, chunk request scheduling, and adaptation module. Each subcomponent encapsulates a particular function that is vital to the operation of an ABR scheme. A review of each of the subcomponents and how they interact with each other is presented. Furthermore, those external factors that are known to have a direct impact on the performance of an ABR module, such as content nature, CDN, and context, are discussed. In conclusion, this paper provides an extensive reference for further research in the field.

Adaptive Bitrate Selection: A Survey

In the forthcoming deployments of WebRTC systems, we speculate that high quality video conferencing will see wide adoption. It is currently being deployed on Google Chrome and Firefox web-browsers, meanwhile desktop and mobile clients are under development. Without a standardized signaling mechanism, service providers can enable various types of topologies; ranging from full-mesh to centralized video conferencing and everything in between. In this paper, we evaluate the performance of various topologies using endpoints implementing WebRTC. We specifically evaluate the performance of the congestion control currently implemented and deployed in these web-browser, Receive-side Real-Time Congestion Control (RRTCC). We use transport impairments like varying throughput, loss and delay, and varying amounts of cross-traffic to measure the performance. Our results show that RRTCC is performant when by itself, but starves when competing with TCP. When competing with selfsimilar media streams, the late-arriving flow temporarily starves the existing media flows.

/pdf/performance-analysis-of-receive-side-real-time-congestion-gts8othuty.pdf

Performance Analysis of Receive-Side Real-Time Congestion Control for WebRTC

https://csperkins.org/publications/2014/09/ellis2014two-level.pdf

A two-level Markov model for packet loss in UDP/IP-based real-time video applications targeting residential users

Real-time applications used by residential customers, such as streaming video and IPTV, are sensitive to packet losses, whether due to IP-layer congestion, or link-layer problems such as bit errors induced by impulse noise. To achieve acceptable user experience for these applications, numerous application-layer forward error correction (AL-FEC) schemes have been proposed. We evaluate some of the FEC schemes developed as part of the OpenFEC project, using packet loss traces of IPTV-like traffic measured on ADSL and Cable links. We consider the effectiveness of these schemes in correcting the loss patterns present on residential links, explain why performance is different using measured loss traces compared with previous simulations using uniform random packet loss, and give recommendations for the use of FEC in streaming video applications deployed to residential Internet users.

Performance analysis of AL-FEC for RTP-based streaming video traffic to residential users

Little performance data currently exists for streaming high-quality Internet video to residential users. Data on streaming performance will provide valuable input to the design of new protocols and applications, such as congestion control and error-correction schemes, and sizing playout buffers in video receivers. This paper presents measurements of streaming real-time UDP traffic to a number of residential users, and discusses the basic characteristics of the data.

/pdf/end-to-end-and-network-internal-measurements-of-real-time-k5uj3ay7a0.pdf

End-to-end and network-internal measurements of real-time traffic to residential users

Packet loss is a major problem for real-time Internet applications Markov models of packet loss are often used to develop and evaluate the performance of these applications Despite their wide use, these models have not been validated in terms of how well they capture the loss conditions experienced by residential Internet users We evaluate the accuracy of common packet loss models using traces of IPTV-like traffic measured on residential ADSL and Cable links, and find that these models are insufficient to capture the observed packet loss patterns We introduce a new type of model, incorporating packet delay information, and show improved accuracy over previous models

https://csperkins.org/publications/2012/10/loss-modelling-lcn.pdf

Modelling packet loss in RTP-based streaming video for residential users

Real-time multimedia flows comprise a large, and increasing, fraction of the traffic on the Internet. An important subset of that traffic, primarily due to interactive applications, runs over UDP/IP, and requires applications to implement congestion control to ensure the stability of the network. The IETF is developing congestion control algorithms for such uses as part of the new WebRTC standards, but there is no standard algorithm that can be used at this time. We do not propose a congestion control algorithm. Rather, we propose a circuit breaker for RTP sessions that can detect when an application is causing excessive network congestion, and shut down the transmission. This can be used as an envelope within which congestion control algorithms can operate, providing a safety net to prevent congestion collapse. We present the RTP circuit breaker algorithm, and provide an initial performance evaluation to show that it performs as desired.

https://csperkins.org/publications/2013/12/singh2013circuitbreaker.pdf

Martin Ellis

Papers

A two-level Markov model for packet loss in UDP/IP-based real-time video applications targeting residential users

Performance analysis of AL-FEC for RTP-based streaming video traffic to residential users

End-to-end and network-internal measurements of real-time traffic to residential users

Modelling packet loss in RTP-based streaming video for residential users

Circuit Breakers for Multimedia Congestion Control