Current 3-D video (3DV) technology is based on stereo systems. These systems use stereo video coding for pictures delivered by two input cameras. Typically, such stereo systems only reproduce these two camera views at the receiver and stereoscopic displays for multiple viewers require wearing special 3-D glasses. On the other hand, emerging autostereoscopic multiview displays emit a large numbers of views to enable 3-D viewing for multiple users without requiring 3-D glasses. For representing a large number of views, a multiview extension of stereo video coding is used, typically requiring a bit rate that is proportional to the number of views. However, since the quality improvement of multiview displays will be governed by an increase of emitted views, a format is needed that allows the generation of arbitrary numbers of views with the transmission bit rate being constant. Such a format is the combination of video signals and associated depth maps. The depth maps provide disparities associated with every sample of the video signal that can be used to render arbitrary numbers of additional views via view synthesis. This paper describes efficient coding methods for video and depth data. For the generation of views, synthesis methods are presented, which mitigate errors from depth estimation and coding.

3-D Video Representation Using Depth Maps

This paper describes an extension of the high efficiency video coding (HEVC) standard for coding of multi-view video and depth data. In addition to the known concept of disparity-compensated prediction, inter-view motion parameter, and inter-view residual prediction for coding of the dependent video views are developed and integrated. Furthermore, for depth coding, new intra coding modes, a modified motion compensation and motion vector coding as well as the concept of motion parameter inheritance are part of the HEVC extension. A novel encoder control uses view synthesis optimization, which guarantees that high quality intermediate views can be generated based on the decoded data. The bitstream format supports the extraction of partial bitstreams, so that conventional 2D video, stereo video, and the full multi-view video plus depth format can be decoded from a single bitstream. Objective and subjective results are presented, demonstrating that the proposed approach provides 50% bit rate savings in comparison with HEVC simulcast and 20% in comparison with a straightforward multi-view extension of HEVC without the newly developed coding tools.

3D High-Efficiency Video Coding for Multi-View Video and Depth Data

The present disclosure pertains to an image-processing device and method that make it possible to suppress block noise. A βLUT_input calculator and a clip processor determine βLUT_input, which is a value inputted to an existing-β generator and an expanded-β generator. When the value of βLUT_input qp from the clip processor is 51 or less, the existing-β generator determines β using a LUT conforming to the HVEC standard, and supplies same to a filter process determination unit. When the value of βLUT_input qp from the clip processor is larger than 51, the expanded-β generator determines the expanded β and supplies same to the filter process determination part. This disclosure, for example, can be applied to an image processing device.

Image processing device and method

Three-dimensional television (3D-TV) is the next major revolution in television. A successful rollout of 3D-TV will require a backward-compatible transmission/distribution system, inexpensive 3D displays, and an adequate supply of high-quality 3D program material. With respect to the last factor, the conversion of 2D images/videos to 3D will play an important role. This paper provides an overview of automatic 2D-to-3D video conversion with a specific look at a number of approaches for both the extraction of depth information from monoscopic images and the generation of stereoscopic images. Some challenging issues for the success of automatic 2D-to-3D video conversion are pointed out as possible research topics for the future.

3D-TV Content Creation: Automatic 2D-to-3D Video Conversion

There exist a variety of ways to represent 3D content, including stereo and multiview video, as well as frame-compatible and depth-based video formats. There are also a number of compression architectures and techniques that have been introduced in recent years. This paper provides an overview of relevant 3D representation and compression formats. It also analyzes some of the merits and drawbacks of these formats considering the application requirements and constraints imposed by different storage and transmission systems.

/pdf/3d-tv-content-storage-and-transmission-16mf8wi7on.pdf

3D-TV Content Storage and Transmission

The video-plus-depth data representation uses a regular texture video enriched with the so-called depth map, providing the depth distance for each pixel. The compression efficiency is usually higher for smooth, gray level data representing the depth map than for classical video texture. However, improvements of the coding efficiency are still possible, taking into account the fact that the video and the depth map sequences are strongly correlated. Classically, the correlation between the texture motion vectors and the depth map motion vectors is not exploited in the coding process. The aim of this paper is to reduce the amount of information for describing the motion of the texture video and of the depth map sequences by sharing one common motion vector field. Furthermore, in the literature, the bitrate control scheme generally fixes for the depth map sequence a percentage of 20% of the texture stream bitrate. However, this fixed percentage can affect the depth coding efficiency, and it should also depend on the content of each sequence. We propose a new bitrate allocation strategy between the texture and its associated per-pixel depth information.We provide comparative analysis to measure the quality of the resulting 3D + t sequences.

/pdf/motion-vector-sharing-and-bitrate-allocation-for-3d-video-1f5p36z7o5.pdf

Motion vector sharing and bitrate allocation for 3D video-plus-depth coding

Depth image-based rendering (DIBR) is the process of synthesizing some new "virtual" views from one "real" view and the associated per-pixel depth information. The most important problem in this process is to deal with the newly exposed areas (holes) appearing in the virtual images. One common solution to decrease the number of holes is to pre-process the depth map, before the warping. In this paper, we present a new filtering technique for depth image-based rendering. In order to reduce or completely remove the newly exposed areas an efficient smoothing is necessary for the sharp depth changes near object boundaries. In the meantime it is useless to filter the smooth areas in the depth map. Our solution is based on a weighted Gaussian filter taking into account the distance to the contours. By this way, the geometric distortions and the computation time are reduced compared to a uniform filtering of the depth map. We present some results in the context of creation of stereoscopic views for 3D TV.

Distance Dependent Depth Filtering in 3D Warping for 3DTV

Motion-compensated temporal filtering subband video codecs have attracted recently a lot of attention, due to their compression performance comparable with that of state-of-the-art hybrid codecs and due to their additional scalability features. In this paper, we present a scalable video codec based on a 5/3 adaptive temporal lifting decomposition. Different adaptation criteria for coping with the occluded areas are discussed and new criteria for optimizing the temporal prediction are introduced. For our simulations, we use a memory-constraint “on-the-fly” implementation. We also evaluate the temporal scalability properties of this video coding structure.

Motion compensation and scalability in lifting-based video coding

Multi-view video and 3D television are emerging applications raising the problem of efficient encoding of a depth map, in addition to classical texture images. This paper investigates depth image coding via an adaptive wavelet lifting scheme. Switching between long filters in homogeneous areas and short filters over the edges of the depth map is decided based on the contours detected in the texture image. The method takes thus into consideration the correlation existing between the edges in the texture and in the depth image, leading to an improved encoding of the latter one.

Adaptive wavelet coding of the depth map for stereoscopic view synthesis

We present a new multiple-description coding (MDC) method for scalable video, designed for transmission over error-prone networks We employ a redundant motion-compensated scheme derived from the Haar multiresolution analysis, in order to build temporally correlated descriptions in a t +2D video coder Our scheme presents a redundancy which decreases with the resolution level This is achieved by additionally subsampling some of the wavelet temporal subbands We present an equivalent four-band lifting implementation leading to simple central and side decoders as well as a packet-based reconstruction strategy in order to cope with random packet losses

/pdf/a-motion-compensated-overcomplete-temporal-decomposition-for-4addg925z2.pdf

Christophe Tillier

Papers

Motion vector sharing and bitrate allocation for 3D video-plus-depth coding

Distance Dependent Depth Filtering in 3D Warping for 3DTV

Motion compensation and scalability in lifting-based video coding

Adaptive wavelet coding of the depth map for stereoscopic view synthesis

A motion-compensated overcomplete temporal decomposition for multiple description scalable video coding