View synthesis

About: View synthesis is a research topic. Over the lifetime, 1701 publications have been published within this topic receiving 42333 citations.

DeepVoxels++: Enhancing the Fidelity of Novel View Synthesis from 3D Voxel Embeddings.

Abstract: We present a novel view synthesis method based upon latent voxel embeddings of an object, which encode both shape and appearance information and are learned without explicit 3D occupancy supervision. Our method uses an encoder-decoder architecture to learn such deep volumetric representations from a set of images taken at multiple viewpoints. Compared with DeepVoxels, our DeepVoxels++ applies a series of enhancements: a) a patch-based image feature extraction and neural rendering scheme that learns local shape and texture patterns, and enables neural rendering at high resolution; b) learned view-dependent feature transformation kernels to explicitly model perspective transformations induced by viewpoint changes; c) a recurrent-concurrent aggregation technique to alleviate single-view update bias of the voxel embeddings recurrent learning process. Combined with d) a simple yet effective implementation trick of frustum representation sufficient sampling, we achieve improved visual quality over the prior deep voxel-based methods (33\(\%\) SSIM error reduction and 22\(\%\) PSNR improvement) on \(360^\circ \) novel-view synthesis benchmarks.

An Uncalibrated View-Synthesis Pipeline

Abstract: This paper deals with the process of view synthesis based on the relative affine structure. It describes a complete pipeline that, starting with uncalibrated images, produces a virtual sequence with viewpoint control. Experiments illustrate the approach.

A flexible vehicle surround view camera system by central-around coordinate mapping model

Abstract: The surround view camera system is an emerging driving assistant technology that can assist drivers in parking by providing top-down view of surrounding situations. Such a system usually consists of four wide-angle or fish-eye cameras that mounted around the vehicle, and a bird-eye view is synthesized from images of these cameras. Commonly there are two fundamental problems for the surround view synthesis, geometric alignment and image synthesis. Geometric alignment performs fish-eye calibration and computes the image perspective transformation between the bird-eye view and images from the surrounding cameras. Image synthesis technique dedicates to seamless stitch between adjacent views and color balancing. In this paper, we propose a flexible central-around coordinate mapping (CACM) model for vehicle surround view synthesis. The CACM model calculates perspective transformation between a top-view central camera coordinate and the around camera coordinates by a marker point based method. With the transformation matrices, we could generate the pixel point mapping relationship between the bird-eye view and images of the surrounding cameras. After geometric alignment, an image fusion method based on distance weighting is adopted for seamless stitch, and an effective overlapping region brightness optimization method is proposed for color balancing. Both the seamless stitch and color balancing can be easily operated by using two types of weight coefficient under the framework of the CACM model. Experimental results show that the proposed approaches could provide a high-performance surround view camera system.

Intermediate View Synthesis Based on Adaptive BP Algorithm and View Interpolation

Abstract: This paper presents a novel intermediate view synthesis method based on adaptive belief propagation (BP) algorithm and view interpolation. First, we raise the accuracy in disparity estimation by introducing an adaptive BP algorithm. Then, image pairs are divided into three kinds of regions. Finally, intermediate view is obtained by applying a new interpolation method to each region. Experimental results indicate that the proposed method can synthesize virtual images with high quality and performs better than a latest method.

Fast and high-quality virtual view synthesis from multi-view plus depth videos

Abstract: Depth image based rendering (DIBR) is an effective method for virtual view synthesis from Multi-view Plus Depth(MVD) video. Synthetic images, however, often contain ghost effect and some holes of varying sizes. This paper uses color correction of reference views, and combines depth-based image fusion with direct color image fusion to decrease the ghost effect. Meanwhile, the cracks are filled using depth filtering and inverse warping. What’s more, the image depth-aided inpainting with GPU acceleration is used to fill the remaining big disocclusions. Experimental results show that our proposed method improved the quality of virtual view synthetic images and reduced the processing time sharply.