Scalable video coding is a powerful solution for content delivery in many interactive multimedia services due to its adaptability to varying terminal and network constraints. In order to successfully exploit such adaptability, it is necessary to understand users' preference among various scalability options and consequently develop an optimal bit rate adaptation strategy. In this paper, we present a study of subjective quality assessment of scalable video coding, which investigates the influence of the combination of scalability options on perceived quality with the goal of providing guidelines for an adaptive strategy that selects the optimal combination for a given bandwidth constraint. In particular, the study is based on paired comparison of stimuli that is suitable for our goal due to its simplicity and easiness. We propose a new method, called Paired Evaluation via Analysis of Reliability (PEAR), which analyzes paired comparison results and produces not only quality scores but also intuitive measures of confidence of the scores for significance analysis. Results and analysis of extensive subjective tests for two different scalable video codecs and high definition contents are described, from which general consistent conclusions are drawn. The video and subjective data used in the paper are publicly available to the research community.

Subjective Quality Evaluation via Paired Comparison: Application to Scalable Video Coding

Efficient bit stream adaptation and resilience to packet losses are two critical requirements in scalable video coding for transmission over packet-lossy networks. Various scalable layers have highly distinct importance, measured by their contribution to the overall video quality. This distinction is especially more significant in the scalable H.264/advanced video coding (AVC) video, due to the employed prediction hierarchy and the drift propagation when quality refinements are missing. Therefore, efficient bit stream adaptation and unequal protection of these layers are of special interest in the scalable H.264/AVC video. This paper proposes an algorithm to accurately estimate the overall distortion of decoder reconstructed frames due to enhancement layer truncation, drift/error propagation, and error concealment in the scalable H.264/AVC video. The method recursively computes the total decoder expected distortion at the picture-level for each layer in the prediction hierarchy. This ensures low computational cost since it bypasses highly complex pixel-level motion compensation operations. Simulation results show an accurate distortion estimation at various channel loss rates. The estimate is further integrated into a cross-layer optimization framework for optimized bit extraction and content-aware channel rate allocation. Experimental results demonstrate that precise distortion estimation enables our proposed transmission system to achieve a significantly higher average video peak signal-to-noise ratio compared to a conventional content independent system.

Unequal Error Protection for Robust Streaming of Scalable Video Over Packet Lossy Networks

High Efficiency Video Coding (HEVC) is an emerging international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). Compared to H.264/AVC, HEVC has achieved substantial compression performance improvement. During the HEVC standardization, we proposed several motion vector coding techniques, which were crosschecked by other experts and then adopted into the standard. In this paper, an overview of the motion vector coding techniques in HEVC is firstly provided. Next, the proposed motion vector coding techniques including a priority-based derivation algorithm for spatial motion candidates, a priority-based derivation algorithm for temporal motion candidates, a surrounding-based candidate list, and a parallel derivation of the candidate list, are also presented. Based on HEVC test model 9 (HM9), experimental results show that the combination of the proposed techniques achieves on average 3.1% bit-rate saving under the common test conditions used for HEVC development.

Motion Vector Coding in the HEVC Standard

A decoder-side motion vector derivation algorithm for hybrid video coding is proposed. The algorithm is based on template matching and aims to reduce motion parameter bit-rate by re-estimating the applicable motion parameters at the decoder side. An average bit-rate savings of about 6%-8% is observed compared to the reference H.264/AVC. Decoder-side motion vector derivation was included in multiple proposals for the new High Efficiency Video Coding standard. This paper details and analyzes the algorithm and discusses its relation to other coding tools.

Decoder-Side Motion Vector Derivation for Block-Based Video Coding

https://ir.nctu.edu.tw/bitstream/11536/39701/1/754901.pdf

A rate-distortion optimization model for SVC inter-layer encoding and bitstream extraction

decoding of video bitstreams, and thus allows various viewing devices to adapt their video reception and playback according to devices capability and network performance. This desirable feature, however, comes with a caveat: the parts of a bitstream needed for providing good quality playback at different devices may differ significantly depending on the visual characteristics of video programs, the quantization and dependency settings of SVC encoders as well as the display formats of viewing devices. In this paper, we present the results of our preliminary investigation on the intricate relations among these factors. We discovered a set of constraints on the setting of quantization parameters and the choices of reference layers for inter-layer dependencies that ensures good rate- distortion trade-offs and regular bitstream extraction paths. We called these constraints, the adaptation rules for inter-layer encoding, and their resulting outputs, the well-adapted layers/bitstreams. We further discovered that bitstream extraction by different viewing devices follow predictable paths if the SVC bitstream is well-adapted and the playback process extracts a complete set of interdependent network abstraction layer (NAL) units in every refinement step. These regular extraction paths enabled us to develop an efficient bitstream extraction algorithm based on local optimization of rate-distortion ratios. The experimental results using standard testing sequences confirmed our findings. Index Terms-- Scalable Video Coding, Rate-Distortion Optimization, Bitstream Extraction

https://ir.nctu.edu.tw:443/bitstream/11536/6312/1/000253974200068.pdf

Rate-Distortion Optimized SVC Bitstream Extraction for Heterogeneous Devices: A Preliminary Investigation

This paper addresses the problem of adapting overlapped block motion compensation (OBMC) windows for use with variable blocksize motion estimates. We tackle the problem by using a parametric window design, which expresses, based on a statistical motion model, the optimal weights as a function of the distances between the predicted pixel and its nearby block centers. The formula enables both prediction weights and prediction order to be adapted on a pixelbypixel basis. Extensive experiments have been conducted using JM 12.4. Compared with conventional block motion compensation, our scheme shows a bitrate saving of 18% (5% on average) while maintaining the same or even higher PSNR (0.1dB). It also provides a competitive advantage to variable block size motion compensation. Additionally, a hybrid of the two techniques achieves a further bitrate reduction of 13%. The result, nevertheless, provides only a lower bound on what is achievable since both motion estimation and mode decision were accomplished without considering OBMC. Further improvement is expected by incorporating iterative methods.

A parametric window design for OBMC with variable block size motion estimates

/pdf/rate-distortion-optimized-svc-bitstream-extraction-for-58ooyt31kf.pdf

This paper addresses the problem of improving motion sampling efficiency for motion-compensated prediction (MCP). We provide a theoretical framework for analyzing the effect of motion sampling structure on MCP efficiency. It is shown that the sampling grid induced by the quadtree partition in H.264/AVC is suboptimal. To improve sampling efficiency, we propose a new pattern, which provides sampling points at both the center and the top-left corner of a macroblcok. When contrasted with conventional NxN/2 block partition, the proposed scheme performs consistently and significantly better in subjective and objective quality.

https://ir.nctu.edu.tw:443/bitstream/11536/26243/1/000287728001013.pdf

Tse-Wei Wang

Papers

A rate-distortion optimization model for SVC inter-layer encoding and bitstream extraction

Rate-Distortion Optimized SVC Bitstream Extraction for Heterogeneous Devices: A Preliminary Investigation

A parametric window design for OBMC with variable block size motion estimates

Rate-Distortion Optimized SVC Bitstream Extraction for Heterogeneous Devices: A Preliminary Investigation

On the analysis and design of motion sampling structure for advanced motion-compensated prediction