View synthesis

About: View synthesis is a research topic. Over the lifetime, 1701 publications have been published within this topic receiving 42333 citations.

On the Use of Ray-tracing for Viewpoint Interpolation in Panoramic Imagery

Abstract: To acquire seamless visualization of environments fromdifferent viewing positions and orientations, it is desirableto generate virtual images for an arbitrary position givena set of reference views. In this paper, a simple interpolationmethod based on ray-tracing is proposed for viewpointsynthesis from panoramas taken with multi-sensor cameras.Instead of attempting to recover a dense 3D reconstructionof the scene, the method estimates the pose between eachpanorama and then backward projects the point along theray that exhibits the best colour consistency. We show thatby limiting the search space using sparse depth information,both the speed and the accuracy of the interpolationare improved.

Novel View Synthesis via Depth-guided Skip Connections

Abstract: We introduce a principled approach for synthesizing new views of a scene given a single source image. Previous methods for novel view synthesis can be divided into image-based rendering methods (e.g., flow prediction) or pixel generation methods. Flow predictions enable the target view to re-use pixels directly, but can easily lead to distorted results. Directly regressing pixels can produce structurally consistent results but generally suffer from the lack of low-level details. In this paper, we utilize an encoder–decoder architecture to regress pixels of a target view. In order to maintain details, we couple the decoder aligned feature maps with skip connections, where the alignment is guided by predicted depth map of the target view. Our experimental results show that our method does not suffer from distortions and successfully preserves texture details with aligned skip connections.

Efficient stereo disparity estimation for intermediate view synthesis

Abstract: An efficient algorithm addressing robust disparity estimation for intermediate view synthesis is proposed. In the proposed method, a new adaptive-size window approach based on region information is introduced to stereo matching in order to overcome problems with fixed-size window. Dynamic programming (DP) technique is used to find optimized disparity values. The reliability of disparity estimation is then measured with a criterion based on uniqueness and smoothness constrains. In occluded areas and image points with unreliable disparity assignments, region-based interpolation strategy is applied to compensate the disparity values. After projecting the left to right and right to left disparities onto the intermediate image, an arbitrary intermediate view is synthesized. Experimental results with natural stereo pairs show that the proposed algorithms provide good disparity map and obtain the intermediate views with high quality.

Seamless View Synthesis Through Texture Optimization

Abstract: In this paper, we present a novel view synthesis method named Visto, which uses a reference input view to generate synthesized views in nearby viewpoints. We formulate the problem as a joint optimization of inter-view texture and depth map similarity, a framework that is significantly different from other traditional approaches. As such, Visto tends to implicitly inherit the image characteristics from the reference view without the explicit use of image priors or texture modeling. Visto assumes that each patch is available in both the synthesized and reference views and thus can be applied to the common area between the two views but not the out-of-region area at the border of the synthesized view. Visto uses a Gauss-Seidel-like iterative approach to minimize the energy function. Simulation results suggest that Visto can generate seamless virtual views and outperform other state-of-the-art methods.