Elements of Information Theory (2nd ed.). Thomas M. Cover and Joy A. Thomas

Scalable video coding is attractive due to the capability of reconstructing lower resolution or lower quality signals from partial bit streams. This allows for simple solutions in adaptation to network and terminal capabilities. Different modalities of scalability are specified by video coding standards like MPEG-2 and MPEG-4. This paper gives a short overview over these techniques and analyzes in more detail the encoder/decoder drift problem, which is the major reason why scalable coding has been significantly less efficient than single-layer coding in most of these implementations. Only recently, new scalable video coding technology has evolved, which seems to close the gap of compression performance compared to state of the art single-layer video coding. New methods of efficient enhancement layer prediction were developed to improve traditional (motion-compensated hybrid) scalable coders, providing more flexible compromises on the drift problem. As a new technology trend, motion-compensated spatiotemporal wavelet coding has matured which entirely discards the drift and allows most flexible combinations of spatial, temporal, and signal-to-noise ratio (SNR) scalability with fine granularity over a broad range of data rates.

Advances in Scalable Video Coding

Digital coding of waveforms: Principles and applications to speech and video

2017 25th European Signal Processing Conference (EUSIPCO)

High Efficiency Video Coding (HEVC) provides superior coding efficiency than previous video coding standards at the cost of increasing encoding complexity. The complexity increase of motion estimation (ME) procedure is rather significant, especially when considering the complicated partitioning structure of HEVC. To fully exploit the coding efficiency brought by HEVC requires a huge amount of computations. In this paper, we analyze the ME structure in HEVC and propose a parallel framework to decouple ME for different partitions on many-core processors. Based on local parallel method (LPM), we first use the directed acyclic graph (DAG)-based order to parallelize coding tree units (CTUs) and adopt improved LPM (ILPM) within each CTU (DAGILPM), which exploits the CTU-level and prediction unit (PU)-level parallelism. Then, we find that there exist completely independent PUs (CIPUs) and partially independent PUs (PIPUs). When the degree of parallelism (DP) is smaller than the maximum DP of DAGILPM, we process the CIPUs and PIPUs, which further increases the DP. The data dependencies and coding efficiency stay the same as LPM. Experiments show that on a 64-core system, compared with serial execution, our proposed scheme achieves more than 30 and 40 times speedup for 1920 × 1080 and 2560 × 1600 video sequences, respectively.

Efficient Parallel Framework for HEVC Motion Estimation on Many-Core Processors

The 3D video extension of High Efficiency Video Coding (3D-HEVC) exploits texture-depth redundancies in 3D videos using intercomponent coding tools. It also inherits the same quadtree coding structure as HEVC for both components. The current software implementation of 3D-HEVC includes encoder shortcuts that speed up the quadtree construction process, but those are always accompanied by coding losses. Furthermore, since the texture and its associated depth represent the same scene, at the same time instant and view point, their quadtrees are closely linked. In this paper, an intercomponent tool is proposed in which this link is exploited to save both runtime and bits through a joint coding of the quadtrees. If depth is coded before the texture, the texture quadtree is initialized from the coded depth quadtree. Otherwise, the depth quadtree is limited to the coded texture quadtree. A 31% encoder runtime saving, a -0.3% gain for coded and synthesized views and a -1.8% gain for coded views are reported for the second method.

Initialization, Limitation, and Predictive Coding of the Depth and Texture Quadtree in 3D-HEVC

With the expectation of greatly enhanced user experience, 3D video is widely perceived as the next major advancement in video technology. In order to fulfil the expectation of enhanced user experience, 3D video calls for new technologies addressing efficient content creation, representation/coding, transmission and display.Emerging Technologies for 3D Video will deal with all aspects involved in 3D video systems and services, including content acquisition and creation, data representation and coding, transmission, view synthesis, rendering, display technologies, human perception of depth and quality assessment.Key features:Offers an overview of key existing technologies for 3D videoProvides a discussion of advanced research topics and future technologiesReviews relevant standardization effortsAddresses applications and implementation issuesIncludes contributions from leading researchersThe book is a comprehensive guide to 3D video systems and services suitable for all those involved in this field, including engineers, practitioners, researchers as well as professors, graduate and undergraduate students, and managers making technological decisions about 3D video.

Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering

Multiview video (MVV) plus depths formats use view synthesis to build intermediate views from existing adjacent views at the receiver side. Traditional view synthesis exploits the disparity information to interpolate an intermediate view by considered inter-view correlations. However, temporal correlation between different frames of the intermediate view can be used to improve the synthesis. We propose a new coding scheme for 3-D High Efficiency Video Coding (HEVC) that allows us to take full advantage of temporal correlations in the intermediate view and improve the existing synthesis from adjacent views. We use optical flow techniques to derive dense motion vector fields (MVF) from the adjacent views and then warp them at the level of the intermediate view. This allows us to construct multiple temporal predictions of the synthesized frame. A second contribution is an adaptive fusion method that judiciously selects between temporal and inter-view prediction to eliminate artifacts associated with each prediction type. The proposed system is compared against the state-of-the-art view synthesis reference software 1-D Fast technique used in 3-D HEVC standardization. Three intermediary views are synthesized. Gains of up to 1.21-dB Bjontegaard Delta peak SNR are shown when evaluated on several standard MVV test sequences.

http://biblio.telecom-paristech.fr/cgi-bin/download.cgi?id=15041

Multiview Plus Depth Video Coding With Temporal Prediction View Synthesis

We present a new technique for the compression of remote-sensing hyperspectral images based on wavelet transform and zerotree coding of coefficients. In order to improve encoding efficiency, the image is first segmented in a small number of regions with homogeneous texture. Then, a shape-adaptive wavelet transform is carried out on each region and the resulting coefficients are finally encoded by a shape-adaptive version of SPIHT. Thanks to the segmentation map (sent as a side information) region boundaries are faithfully preserved and selective encoding strategies can be easily implemented. In addition, by-now homogeneous region textures can be more efficiently encoded.

Region-oriented compression of multispectral images by shape-adaptive wavelet transform and SPIHT

Super Multi-View (SMV) video content is composed of tens or hundreds of views that provide a light-field representation of a scene. This representation allows a glass-free visualization and eliminates many causes of discomfort existing in current available 3D video technologies. Efficient video compression of SMV content is a key factor for enabling future 3D video services. This paper first compares several coding configurations for SMV content and several inter-view prediction structures are also tested and compared. The experiments mainly suggest that large differences in coding efficiency can be observed from one configuration to another. Several ratios for the number of coded and synthesized views are compared, both objectively and subjectively. It is reported that view synthesis significantly affects the coding scheme. The amount of views to skip highly depends on the sequence and on the quality of the associated depth maps. Reported ranges of bitrates required to obtain a good quality for the tested SMV content are realistic and coherent with future 4K/8K needs. The reliability of the PSNR metric for SMV content is also studied. Objective and subjective results show that PSNR is able to reflect increase or decrease in subjective quality even in the presence of synthesized views. However, depending on the ratio of coded and synthesized views, the order of magnitude of the effective quality variation is biased by PSNR. Results indicate that PSNR is less tolerant to view synthesis artifacts than human viewers. Finally, preliminary observations are initiated. First, the light-field conversion step does not seem to alter the objective results for compression. Secondly, the motion parallax does not seem to be impacted by specific compression artifacts. The perception of the motion parallax is only altered by variations of the typical compression artifacts along the viewing angle, in cases where the subjective image quality is already low. To the best of our knowledge, this paper is the first to carry out subjective experiments and to report results of SMV compression for light-field 3D displays. It provides first results showing that improvement of compression efficiency is required, as well as depth estimation and view synthesis algorithms improvement, but that the use of SMV appears realistic according to next generation compression technology requirements. HighlightsStudy of the impact of compression on subjective quality for lightfield SMV content.To the best of our knowledge, this paper is the first to report results of this kind.Several SMV coding configurations are compared both objectively and subjectively.Compression efficiency, depth estimation and view synthesis require improvements.SMV appears realistic according to next generation compression technology requirements.

Marco Cagnazzo

Papers

Initialization, Limitation, and Predictive Coding of the Depth and Texture Quadtree in 3D-HEVC

Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering

Multiview Plus Depth Video Coding With Temporal Prediction View Synthesis

Region-oriented compression of multispectral images by shape-adaptive wavelet transform and SPIHT

Subjective evaluation of Super Multi-View compressed contents on high-end light-field 3D displays