Quality of experience (QoE) is the perceptual quality of service (QoS) from the users' perspective. For video service, the relationship between QoE and QoS (such as coding parameters and network statistics) is complicated because users' perceptual video quality is subjective and diversified in different environments. Traditionally, QoE is obtained from subjective test, where human viewers evaluate the quality of tested videos under a laboratory environment. To avoid high cost and offline nature of such tests, objective quality models are developed to predict QoE based on objective QoS parameters, but it is still an indirect way to estimate QoE. With the rising popularity of video streaming over the Internet, data-driven QoE analysis models have newly emerged due to availability of large-scale data. In this paper, we give a comprehensive survey of the evolution of video quality assessment methods, analyzing their characteristics, advantages, and drawbacks. We also introduce QoE-based video applications and, finally, identify the future research directions of QoE.

From QoS to QoE: A Tutorial on Video Quality Assessment

The performance evaluation of video transport mechanisms becomes increasingly important as encoded video accounts for growing portions of the network traffic. Compared to the widely studied MPEG-4 encoded video, the recently adopted H.264 video coding standards include novel mechanisms, such as hierarchical B frame prediction structures and highly efficient quality scalable coding, that have important implications for network transport. This tutorial introduces a trace-based evaluation methodology for the network transport of H.264 encoded video. We first give an overview of H.264 video coding, and then present the trace structures for capturing the characteristics of H.264 encoded video. We give an overview of the typical video traffic and quality characteristics of H.264 encoded video. Finally, we explain how to account for the H.264 specific coding mechanisms, such as hierarchical B frames, in networking studies.

/pdf/video-transport-evaluation-with-h-264-video-traces-2ztswbnb2c.pdf

Video Transport Evaluation With H.264 Video Traces

The objective approaches of 3D image quality assessment play a key role for the development of compression standards and various 3D multimedia applications. The quality assessment of 3D images faces more new challenges, such as asymmetric stereo compression, depth perception, and virtual view synthesis, than its 2D counterparts. In addition, the widely used 2D image quality metrics (e.g., PSNR and SSIM) cannot be directly applied to deal with these newly introduced challenges. This statement can be verified by the low correlation between the computed objective measures and the subjectively measured mean opinion scores (MOSs), when 3D images are the tested targets. In order to meet these newly introduced challenges, in this paper, besides traditional 2D image metrics, the binocular integration behaviors-the binocular combination and the binocular frequency integration, are utilized as the bases for measuring the quality of stereoscopic 3D images. The effectiveness of the proposed metrics is verified by conducting subjective evaluations on publicly available stereoscopic image databases. Experimental results show that significant consistency could be reached between the measured MOS and the proposed metrics, in which the correlation coefficient between them can go up to 0.88. Furthermore, we found that the proposed metrics can also address the quality assessment of the synthesized color-plus-depth 3D images well. Therefore, it is our belief that the binocular integration behaviors are important factors in the development of objective quality assessment for 3D images.

Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors

In this paper we will present the concept of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity where industries, research centers and universities have joined forces to design a backwardscompatible, flexible and modular broadcast 3D-TV system, where all parts of the 3D processing chain are optimised to one another. This includes content creation, coding and transmission, display and research in human 3D perception, which be will used to guide the development process. The goals of the project comprise the development of a novel broadcast 3D camera, algorithms to convert existing 2D-video material into 3D, a 2Dcompatible coding and transmission scheme for 3D-video using MPEG2/4/7 technologies and the design of two new autostereoscopic displays.

IST project ATTEST - an evolutionary and optimised approach on 3D-TV

The increasing popularity of video (i.e., audio-visual) applications or services over both wired and wireless links has prompted recent growing interests in the investigations of quality of experience (QoE) in online video transmission. Conventional video quality metrics, such as peak-signal-to-noise-ratio and quality of service, only focus on the reception quality from the systematic perspective. As a result, they cannot represent the true visual experience of an individual user. Instead, the QoE introduces a user experience-driven strategy which puts special emphasis on the contextual and human factors in addition to the transmission system. This advantage has raised the popularity and widespread usage of QoE in video transmission. In this paper, we present an overview of selected issues pertaining to QoE and its recent applications in video transmission, with consideration of the compelling features of QoE (i.e., context and human factors). The selected issues include QoE modeling with influence factors in the end-to-end chain of video transmission, QoE assessment (including subjective test and objective QoE monitoring) and QoE management of video transmission over different types of networks. Through the literature review, we observe that the context and human factors in QoE-aware video transmission have attracted significant attentions since the past two to three years. A vast number of high quality works were published in this area, and will be highlighted in this survey. In addition to a thorough summary of recent progresses, we also present an outlook of future developments on QoE assessment and management in video transmission, especially focusing on the context and human factors that have not been addressed yet and the technical challenges that have not been completely solved so far. We believe that our overview and findings can provide a timely perspective on the related issues and the future research directions in QoE-oriented services over video communications.

QoE in Video Transmission: A User Experience-Driven Strategy

It is well known that the human visual system can perceive high frequencies in 3D stereo video, even if that information is present in only one of the views. Therefore, the best perceived 3D stereo video quality may be achieved by asymmetric coding where the reference and auxiliary (right and left) views are coded at unequal PSNR. However, the questions of what is the best level of asymmetry in order to maximize the perceived quality and whether asymmetry should be achieved by spatial resolution reduction or PSNR (quality) reduction have been open issues. We conducted extensive subjective tests, which indicate that if the reference view is encoded at sufficiently high quality and the auxiliary view is encoded at a lower quality but above a certain PSNR threshold, then the degradation in 3D video quality is unnoticeable. Since asymmetric coding by PSNR reduction gives finer control over achievable PSNR values over spatial resolution reduction, asymmetry by PSNR reduction allows us to encode at a point more close to this just-noticeable asymmetry PSNR threshold; hence will be preferred over the spatial resolution reduction method. Subjective tests also indicate that below this just-noticeable asymmetry threshold, where subtle artifacts start to appear, symmetric coding performs better than asymmetric coding in terms of perceived 3D video quality. Therefore, we show that the choice between asymmetric vs. symmetric coding depends on PSNR; hence, the available total bitrate. This paper also proposes a novel asymmetric scalable stereo video coding framework to enable adaptive stereoscopic video streaming taking full advantage of these observations and subjective test results.

Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming

Three-dimensional (3-D) video is the next natural step in the evolution of digital media technologies. Recent 3-D autostereoscopic displays can display multiview video with up to 200 views. While it is possible to broadcast 3-D stereo video (two views) over digital TV platforms today, streaming over Internet Protocol (IP) provides a more flexible approach for distribution of stereo and free-view 3-D media to home and mobile with different connection bandwidths and different 3-D displays. Here, flexible transport refers to rate-scalable, resolution-scalable, and view-scalable transport over different channels including digital video broadcasting (DVB) and/or IP. In this paper, we first briefly review the state of the art in 3-D video formats, coding methods for different transport options and video formats, IP streaming protocols, and streaming architectures. We then take a look at beyond the state of the art in 3-D video transport research, including asymmetric stereoscopic video streaming, adaptive and peer-to-peer (P2P) streaming of multiview video, view-selective streaming and future directions in broadcast of 3-D media over IP and jointly over DVB and IP.

Flexible Transport of 3-D Video Over Networks

Multiview entertainment is the next step in 3D immersive media networking owing to its improved depth perception and free-viewpoint viewing capability whereby users can observe the scene from the desired viewpoint. This paper outlines a delivery system for multiview plus depth video, combining the broadcast and broadband networks. The digital video broadcast (DVB) network is used along with adaptive peer-to-peer (P2P) distribution over the Internet to deliver high-volume multimedia to users. The DVB network has been used to deliver part of the 3D service, owing to its robustness and wide availability, as a mechanism to guarantee the minimum 3D quality of experience. The developed system brings key contributions in the P2P transport for real-time multimedia delivery, including a user preference-aware adaptation mechanism, adaptive redundant chunk scheduling for robustness, incentives to decrease the load on the content server for improved system scalability, and resynchronization capability with the DVB transmission. The introduced features are compared with those of some other well-known P2P solutions to highlight the quantitative gains. A subjective testing campaign has also been organized on the developed hybrid platform, which proves the effectiveness of user-aware adaptation over network-based adaptation on a mean opinion score scale.

/pdf/adaptive-multiview-video-delivery-using-hybrid-networking-1ymmvsxul0.pdf

Adaptive Multiview Video Delivery Using Hybrid Networking

Inspired by the success of BitTorrent in peer-to-peer (P2P) file sharing, many research groups have proposed extensions of BitTorrent to enable P2P video delivery over the Internet in a time-sensitive manner. However, fixed sized chunks, a feature of the Torrent protocol, leads to inefficiency in application layer framing for error resilient video streaming, since video slices are variable length when encoded at constant quality. This paper proposes two modifications to the Torrent protocol, variable chunk size and adaptive scheduling window, for efficient, error-resilient, adaptive P2P streaming of scalable video. Experimental results show that the proposed modifications yield superior results in terms of number of decoded frames, hence superior quality of experience, in P2P video streaming.

Variable chunk size and adaptive scheduling window for P2P streaming of scalable video

Peer-to-peer (P2P) video streaming has recently drawn significant attention since P2P solutions offer scalability with increasing number of recipients. P2P streaming may be a feasible solution for multi-view video (MVV) delivery, which has significantly higher bandwidth requirements than the legacy 2D video. However, a practical P2P MVV service cannot be achieved without properly addressing the perceived quality of experience (QoE) of MVV. QoE is an important issue especially in adaptive video streaming in which the quality of the content varies to match the available channel capacity. In this study, we first evaluate the perceived quality of MVV, when various adaptation methods are adopted to scale MVV bitrate to a given rate, by using subjective tests. The results of these tests are used to identify the best rate adaptation method. Next, we propose a mesh-based P2P streaming architecture that employs rate adaptation according to the findings to deliver the best QoE under diverse network conditions. We implemented our protocols and ran experiments on PlanetLab testbed to evaluate the performance of our proposed solution. The experimental results validate that the proposed adaptation strategy ensures continuous play-out by prioritizing transmission of the base-layer and augments the video quality when additional bit rate is available.

C. G. Gurler

Papers

Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming

Flexible Transport of 3-D Video Over Networks

Adaptive Multiview Video Delivery Using Hybrid Networking

Variable chunk size and adaptive scheduling window for P2P streaming of scalable video

Adaptive multi-view video streaming over P2P networks considering quality of experience