View synthesis

About: View synthesis is a research topic. Over the lifetime, 1701 publications have been published within this topic receiving 42333 citations.

Multiple reference views for hole reduction in DIBR view synthesis

Abstract: Depth-image-based rendering (DIBR) technique is widely used to synthesize virtual views in many applications including multiview and free viewpoint video systems. An inherent problem of DIBR oriented view synthesis is that holes may appear in the synthesized view. Hole filling techniques generally extrapolates/interpolates the hole regions with the neighboring information based on an assumption that the texture pattern in the hole regions is similar to that of the neighboring background information. However, in some cases especially with complex texture, the assumption may not hold. Moreover, hole filling techniques can only provide an estimation for the hole regions which may not be true. In this paper, based on a thorough examination of the hole generation mechanism in view interpolation with multiple reference views, the problem of emerging holes in a target virtual view can be greatly alleviated by making good use of other neighboring complementary views in addition to its two (commonly used) most neighboring primary views. Furthermore, a scheme of selective warping of the complementary views is developed by efficiently locating a small number of useful pixels in the complementary views for hole reduction. Experimental results show that the hole size based on two primary reference views may be reduced by about 90 percent with the help of complementary reference views, while the developed selective warping scheme can save about 69 percent of rendering time compared with fully warping two complementary reference views in the rendering process.

View synthesis with sparse light field for 6DoF immersive video

Abstract: Virtual view synthesis, which generates novel views similar to the characteristics of actually acquired images, is an essential technical component for delivering an immersive video with realistic binocular disparity and smooth motion parallax. This is typically achieved in sequence by warping the given images to the designated viewing position, blending warped images, and filling the remaining holes. When considering 6DoF use cases with huge motion, the warping method in patch unit is more preferable than other conventional methods running in pixel unit. Regarding the prior case, the quality of synthesized image is highly relevant to the means of blending. Based on such aspect, we proposed a novel blending architecture that exploits the similarity of the directions of rays and the distribution of depth values. By further employing the proposed method, results showed that more enhanced view was synthesized compared with the well‐designed synthesizers used within moving picture expert group (MPEG‐I). Moreover, we explained the GPU‐based implementation synthesizing and rendering views in the level of real time by considering the applicability for immersive video service.

Blind Quality Index of Depth Images Based on Structural Statistics for View Synthesis

Abstract: The quality of depth images is crucial for virtual view synthesis. However, the quality assessment of depth images is still largely unexplored. This letter presents a blind quality metric of Depth image based on Structural Statistics (DSS). The design philosophy is inspired by the fact that structural distortion in the depth images usually leads to geometric distortion, which is the main cause for degraded quality of synthesized views. Specifically, the statistical features for shape and orientation are calculated based on discrete orthogonal moments and gradients, generating two groups of quality-aware features. Then, the quality model is built from the extracted statistical features using a regression module. The experimental results demonstrate the effectiveness of the proposed metric.