Research efforts on 3DTV technology have been strengthened worldwide recently, covering the whole media processing chain from capture to display. Different 3DTV systems rely on different 3D scene representations that integrate various types of data. Efficient coding of these data is crucial for the success of 3DTV. Compression of pixel-type data including stereo video, multiview video, and associated depth or disparity maps extends available principles of classical video coding. Powerful algorithms and open international standards for multiview video coding and coding of video plus depth data are available and under development, which will provide the basis for introduction of various 3DTV systems and services in the near future. Compression of 3D mesh models has also reached a high level of maturity. For static geometry, a variety of powerful algorithms are available to efficiently compress vertices and connectivity. Compression of dynamic 3D geometry is currently a more active field of research. Temporal prediction is an important mechanism to remove redundancy from animated 3D mesh sequences. Error resilience is important for transmission of data over error prone channels, and multiple description coding (MDC) is a suitable way to protect data. MDC of still images and 2D video has already been widely studied, whereas multiview video and 3D meshes have been addressed only recently. Intellectual property protection of 3D data by watermarking is a pioneering research area as well. The 3D watermarking methods in the literature are classified into three groups, considering the dimensions of the main components of scene representations and the resulting components after applying the algorithm. In general, 3DTV coding technology is maturating. Systems and services may enter the market in the near future. However, the research area is relatively young compared to coding of other types of media. Therefore, there is still a lot of room for improvement and new development of algorithms.

Coding Algorithms for 3DTV—A Survey

We explored the response of the human visual system to mixed-resolution stereo video-sequences, in which one eye view was spatially or temporally low-pass filtered. It was expected that the perceived quality, depth, and sharpness would be relatively unaffected by low-pass filtering, compared to the case where both eyes viewed a filtered image. Subjects viewed two 10-second stereo video-sequences, in which the right-eye frames were filtered vertically (V) and horizontally (H) at 1/2 H, 1/2 V, 1/4 H, 1/4 V, 1/2 H 1/2 V, 1/2 H 1/4 V, 1/4 H 1/2 V, and 1/4 H 1/4 V resolution. Temporal filtering was implemented for a subset of these conditions at 1/2 temporal resolution, or with drop-and-repeat frames. Subjects rated the overall quality, sharpness, and overall sensation of depth. It was found that spatial filtering produced acceptable results: the overall sensation of depth was unaffected by low-pass filtering, while ratings of quality and of sharpness were strongly weighted towards the eye with the greater spatial resolution. By comparison, temporal filtering produced unacceptable results: field averaging and drop-and-repeat frame conditions yielded images with poor quality and sharpness, even though perceived depth was relatively unaffected. We conclude that spatial filtering of one channel of a stereo video-sequence may be an effective means of reducing the transmission bandwidth.

Stereo image quality: effects of mixed spatio-temporal resolution

A method for decoding a compressed image stream, the image stream having a plurality of frames, each frame consisting of a merged image including pixels from a left image and pixels from a right image. The method involves the steps of receiving each merged image; changing a clock domain from the original input signal to an internal domain; for each merged image, placing at least two adjacent pixels into an input buffer and interpolating an intermediate pixel, for forming a reconstructed left frame and a reconstructed right frame according to provenance of the adjacent pixels; and reconstructing a stereoscopic image stream from the left and right image frames. The invention also teaches a system for decoding a compressed image stream.

Process and system for encoding and playback of stereoscopic video sequences

This paper presents an algorithm for rendering a static scene from multiple perspectives. While most current computer graphics algorithms render scenes as they appear from a single viewpoint (the location of the camera) multiple viewpoint rendering (MVR) renders a scene from a range of spatially-varying viewpoints. By exploiting perspective coherence, MVR can produce a set of images orders of magnitude faster than conventional rendering methods. Images produced by MVR can be used as input to multiple-perspective displays such as holographic stereograms, lenticular sheet displays, and holographic video. MVR can also be used as a geometry-to-image prefilter for image-based rendering algorithms. MVR techniques are adapted from single viewpoint computer graphics algorithms and can be accelerated using existing hardware graphics subsystems. This paper describes the characteristics of MVR algorithms in general, along with the design, implementation, and applications of a particular MVR rendering system.

/pdf/multiple-viewpoint-rendering-1b06fpptr9.pdf

Multiple viewpoint rendering

A system is provided for processing a compressed image stream of a stereoscopic image stream, the compressed image stream having a plurality of frames in a first format, each frame consisting of a merged image comprising pixels sampled from a left image and pixels sampled from a right image. A receiver receives the compressed image stream and a decompressing module in communication with the receiver decompresses the compressed image stream. The left and right images of the decompressed image stream are stored in a frame buffer. A serializing unit reads pixels of the frames stored in the frame buffer and outputs a pixel stream comprising pixels of a left frame and pixels of a right frame. A stereoscopic image processor receives the pixel stream, buffers the pixels, performs interpolation in order to reconstruct pixels of the left and right images and outputs a reconstructed left pixel stream and a reconstructed right pixel stream, the reconstructed streams having a format different from the first format. A display signal generator receives the stereoscopic pixel stream to provide an output display signal.

Apparatus for processing a stereoscopic image stream

The proposed approach to stereo image coding takes advantage of the singleness of vision property of the human visual system. Experiments show that a stereo image pair, in which one of the images is low-pass filtered and subsampled, is perceived as a sharp 3-D image. The depth information is perceived due to the stereopsis effect, and the sharpness is maintained due to the details in the non-filtered image. A methodology for the evaluation of the compression effects on the 3-D perception of stereo images is presented. It is based on measurements of response-time and accuracy of human subjects performing simple 3-D perception tasks.

Compression Of Stereo Images And The Evaluation Of Its Effects On 3-D Perception

The proposed approach to stereo image coding takes advantage of the singleness of vision property of the human visual system. A stereo image pair, in which one of the images is low-pass filtered and subsampled, is perceived stereoscopically as a sharp 3-D image. The depth information is perceived due to the stereopsis effect, and the sharpness is maintained due to the details in the nonfiltered image. Low-pass filtering, subsampling, and discrete cosine transform image coding are used for the compression of the stereo images. A methodology for the evaluation of the compression effects on the 3-D perception of compressed stereo images is presented. It is based on measurements of response time and accuracy of human subjects performing simple 3-D perception tasks.

Joseph Tselgov

Papers

Compression Of Stereo Images And The Evaluation Of Its Effects On 3-D Perception

Compression of stereo images using subsampling and transform coding