A high-quality three-dimensional (3-D) broadcast service (3-D TV) is becoming increasingly feasible based on various recent technological developments combined with an enhanced understanding of 3-D perception and human factors issues surrounding 3-D TV. In this paper, 3-D technology and perceptually relevant issues, in particular 3-D image quality and visual comfort, in relation to 3-D TV systems are reviewed. The focus is on near-term displays for broadcast-style single- and multiple-viewer systems. We discuss how an image quality model for conventional two-dimensional images needs to be modified to be suitable for image quality research for 3-D TV. In this respect, studies are reviewed that have focused on the relationship between subjective attributes of 3-D image quality and physical system parameters that induce them (e.g., parameter choices in image acquisition, compression, and display). In particular, artifacts that may arise in 3-D TV systems are addressed, such as keystone distortion, depth-plane curvature, puppet theater effect, cross talk, cardboard effect, shear distortion, picket-fence effect, and image flipping. In conclusion, we summarize the perceptual requirements for 3-D TV that can be extracted from the literature and address issues that require further investigation in order for 3-D TV to be a success.

A survey of perceptual evaluations and requirements of three-dimensional TV

This article investigates the interaction between different techniques for depth compression and view synthesis rendering with multiview video plus scene depth data Two different approaches for depth coding are compared, namely H264/MVC, using temporal and inter-view reference images for efficient prediction, and the novel platelet-based coding algorithm, characterized by being adapted to the special characteristics of depth-images Since depth-images are a 2D representation of the 3D scene geometry, depth-image errors lead to geometry distortions Therefore, the influence of geometry distortions resulting from coding artifacts is evaluated for both coding approaches in two different ways First, the variation of 3D surface meshes is analyzed using the Hausdorff distance and second, the distortion is evaluated for 2D view synthesis rendering, where color and depth information are used together to render virtual intermediate camera views of the scene The results show that-although its rate-distortion (R-D) performance is worse-platelet-based depth coding outperforms H264, due to improved sharp edge preservation Therefore, depth coding needs to be evaluated with respect to geometry distortions

/pdf/the-effects-of-multiview-depth-video-compression-on-30q5jlsi07.pdf

The effects of multiview depth video compression on multiview rendering

Interest in 3D video applications and systems is growing rapidly and technology is maturating. It is expected that multiview autostereoscopic displays will play an important role in home user environments, since they support multiuser 3D sensation and motion parallax impression. The tremendous data rate cannot be handled efficiently by representation and coding formats such as MVC or MPEG-C Part 3. Multiview video plus depth (MVD) is a new format that efficiently supports such advanced 3DV systems, but this requires high-quality intermediate view synthesis. For this, a new approach is presented that separates unreliable image regions along depth discontinuities from reliable image regions, which are treated separately and fused to the final interpolated view. In contrast to previous layered approaches, our algorithm uses two boundary layers and one reliable layer, performs image-based 3D warping only, and was generically implemented, that is, does not necessarily rely on 3D graphics support. Furthermore, different hole-filling and filtering methods are added to provide high-quality intermediate views. As a result, high-quality intermediate views for an existing 9-view auto-stereoscopic display as well as other stereo- and multiscopic displays are presented, which prove the suitability of our approach for advanced 3DV systems.

/pdf/view-synthesis-for-advanced-3d-video-systems-4scduwp8qt.pdf

View Synthesis for Advanced 3D Video Systems

In this paper, we propose a fast disparity and motion estimation for multi-view video coding (MVC). When implementing MVC, one of the most critical problems is heavy computational complexity caused by the large amount of information in multi-view sequences. Hence, a fast algorithm is essential. To reduce this computational complexity, we adoptively controlled a search range considering the reliability of each macroblock. In order to estimate this reliability, we calculated the difference between the predicted vectors that were obtained from different methods. When working with conventional encoders, vectors can be predicted using median filtering from causal blocks. Moreover, we calculated another predicted vector using multi-view camera geometry or the relationship between the disparity and motion vectors. We assumed that this difference indicated the reliability of the current macroblock. By using these properties, we were able to determine new search range and reduce the number of searching points within the limited window. The proposed MVC system was tested with several multi- view sequences to evaluate performance. Experimental results showed that the proposed algorithm was able to reduce processing time by maximumly 70-80% in estimation process.

Fast Disparity and Motion Estimation for Multi-view Video Coding

Hybrid video decoder supporting intermediate view synthesis of an intermediate view video from a first- and a second-view video which are predictively coded into a multi-view data signal with frames of the second-view video being spatially subdivided into sub-regions and the multi-view data signal having a prediction mode is provided, having: an extractor configured to respectively extract, from the multi-view data signal, for sub-regions of the frames of the second-view video, a disparity vector and a prediction residual; a predictive reconstructor configured to reconstruct the sub-regions of the frames of the second-view video, by generating a prediction from a reconstructed version of a portion of frames of the first-view video using the disparity vectors and a prediction residual for the respective sub-regions; and an intermediate view synthesizer configured to reconstruct first portions of the intermediate view video.

Hybrid video coding supporting intermediate view synthesis

A multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed in this paper for 3-D virtual reality applications. In the first part of this paper, we develop a codec that features random access to any encoded view. The codec performance is good and comparable to the MPEG standard. In the second part, we interpolate the disparities and refer to appropriate decoded images for the synthesis of any intermediate views. Our design provides enough information so that each synthesis can be achieved within 0.04 sec for a 320/spl times/240 image (excluding the decoding time). An eye glass viewer with infrared synchronization is also adopted to view the stereo results which are very good.

Multi-view image compression and intermediate view synthesis for stereoscopic applications

A multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed in this paper for 3-D virtual reality applications. In the first part of this paper, we develop a codec that is featured of random access to any encoded view. The codec performance is good and comparable to the MPEG standard. In the second .part, we interpolate the disparities and refer to appropriate decoded images for the synthesis of any intermediate views. Our design provides enough information so that each synthesis can be achieved within 0.04 sec for a 320~ 240 image (excluding the decoding time). An eye glass viewer with infrared synchronization is also adopted to view the stereo results and they really look good.

http://ieeexplore.ieee.org/iel5/6910/18592/00856315.pdf

Gan-Cheih Chang

Papers

Multi-view image compression and intermediate view synthesis for stereoscopic applications

Multi-View Image Compression and Intermediate View Synthesis For Stereoscopic Applications