Multiview video consists of multiple views of the same scene. They require enormous amount of data to achieve high image quality, which makes it indispensable to compress multiview video. Therefore, data compression is a major issue for multiviews. In this paper, we explore an efficient fractal video codec to compress multiviews. The proposed scheme first compresses a view-dependent geometry of the base view using fractal video encoder with homogeneous region condition. With the extended fractional pel motion estimation algorithm and fast disparity estimation algorithm, it then generates prediction images of other views. The prediction image uses the image-based rendering techniques based on the decoded video. And the residual signals are obtained by the prediction image and the original image. Finally, it encodes residual signals by the fractal video encoder. The idea is also to exploit the statistical dependencies from both temporal and interview reference pictures for motion compensated prediction. Experimental results show that the proposed algorithm is consistently better than JMVC8.5, with 62.25% bit rate decrease and 0.37 dB PSNR increase based on the Bjontegaard metric, and the total encoding time (TET) of the proposed algorithm is reduced by 92%.

/pdf/an-efficient-fractal-video-sequences-codec-with-multiviews-3vqqn307dx.pdf

An Efficient Fractal Video Sequences Codec with Multiviews

This paper presents a method that jointly performs stereo matching and inter-view interpolation to obtain the depth map and virtual view image. A novel view synthesis method based on depth map layers representation of the stereo image pairs is proposed. The main idea of this approach is to separate the depth map into several layers of depth based on the disparity distance of the corresponding points. The novel view synthesis can be interpolated independently to each layer of depth by masking the particular depth layer. The final novel view synthesis obtained with the entire layers flattens into a single layer. Since the image view synthesis is performed in separate layers, the extracted new virtual object can be superimposed onto another 3D scene. The method is useful for free viewpoint video application with a small number of camera configurations. Based on the experimental results, it shows that the algorithm could improve the efficiency of finding the depth map and to synthesis the new virtual view images.

Novel view synthesis based on depth map layers representation

Virtual view rendering is one of the key techniques of interactive 3D video systems, and is analogue expression to the real world in virtual reality systems. This paper proposes a new algorithm of virtual view rendering between two views taken in the same scene at the same time, which is based on self-adaptive block matching disparity estimation. This algorithm outputs any view between the two origin views. In order to gain accurate dense disparity field, a method of self-adaptive block matching is proposed in disparity estimation according to the information of texture and gray scale. Image interpolation is used to render certain virtual view and reverse mapping is adopted to fill up holes formed in interpolation. Satisfying visual effect is shown in experimental result.

Virtual View Rendering Based on Self-adaptive Block Matching Disparity Estimation

A new Depth Image Layers Separation (DILS) algorithm for synthesizing inter-view images based on disparity depth map layers representation is presented. The approach is to separate the depth map into several layers identified through histogram-based clustering. Each layer is extracted using inter-view interpolation to create objects based on location and depth. DILS is a new paradigm in selecting interesting image locations based on depth, but also in producing new image representations that allow objects or parts of an image to be described without the need of segmentation and identification. The image view synthesis can reduce the configuration complexity of multi-camera arrays in 3D imagery and free-viewpoint applications. The simulation results show that depth layer separation is able to create inter-view images that may be integrated with other techniques such as occlusion handling processes. The DILS algorithm can be implemented using both simple as well as sophisticated stereo matching methods to synthesize inter-view images.

Depth Image Layers Separation (DILS) algorithm of image view synthesis based on stereo vision

This paper presents a method that jointly performs stereo matching and inter-view interpolation to obtain the depth map and virtual view image. A novel view synthesis method based on depth map layers representation of the stereo image pairs is proposed. The main idea of this approach is to separate the depth map into several layers of depth based on the disparity distance of the corresponding points. The novel view synthesis can be interpolated independently to each layer of depth by masking the particular depth layer. The final novel view synthesis obtained with the entire layers flattens into a single layer. Since the image view synthesis is performed in separate layers, the extractednew virtual object can be superimposed onto another 3D scene. The method is useful to imply free viewpoint video application with a small number of camera configurations. Based on the experimental results, it shows that the algorithm improve the efficiency of finding the depth map and to synthesis the new virtual view images.

Disparity depth map layers representation for image view synthesis

A novel intermediate view synthesis method based on disparity estimation is presented in this paper. Color distance is adopted to set truncation value adaptively when Belief Propagation (BP) algorithm is used in disparity estimation. In view interpolation, the left and right part of the image are divided into three kinds using strong and weak consistency constraints separately. Then different strategy is applied according to the characteristic of each region. Experimental results show that it can synthesize virtual images with high quality.

Zeng-yi Qi

Papers

A Novel Intermediate View Synthesis Method Based on Disparity Estimation