Topic
View synthesis
About: View synthesis is a research topic. Over the lifetime, 1701 publications have been published within this topic receiving 42333 citations.
Papers published on a yearly basis
Papers
More filters
02 Sep 2017
TL;DR: This paper presents a depth image based rendering algorithm for view synthesis task that addresses the challenging occlusion filling problem with a hierarchical clustering approach.
Abstract: This paper presents a depth image based rendering algorithm for view synthesis task. We address the challenging occlusion filling problem with a hierarchical clustering approach. Depth distribution of neighboring pixels around each occlusion is explored and from which we determine the number of surrounding depth planes with agglomerative clustering. Pixels in the most distant plane are picked as candidates to restore that occlusion. The proposed algorithm is evaluated on Middlebury stereo dataset and Microsoft Research 3D video dataset. Results show that our method ranks among the best performers.
8 citations
11 Jul 2010
TL;DR: In this paper, an improved algorithm to generate a smooth and accurate depth map for view synthesis and multiview video coding is developed, which can achieve up to 4 dB improvement in view synthesis, while requires much fewer bits to encode the depth map.
Abstract: In this paper, an improved algorithm to generate a smooth and accurate depth map for view synthesis and
multiview video coding is developed. For each block in the target view, the algorithm first uses epipolar geometry
to find its matched block in the reference view, from which an initial depth is obtained using the triangulation
method and depth projection. 3D warping is then applied to refine the depth. In addition, a structural similarity
and maximum likelihood-based approach is developed to fuse the depth estimations from multiple references.
Finally, the depth map is smoothed via segmentation and plane fitting. Compared to existing 3D warping-based
depth estimation, the proposed algorithm can achieve up to 4 dB improvement in view synthesis, while requires
much fewer bits to encode the depth map. Experimental results in multiview video coding show that the proposed
method can outperform the H.264 JMVC software by more than 1 dB.
8 citations
Posted Content•
TL;DR: MVSNeRF as discussed by the authors proposes a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference, leveraging plane-swept cost volumes (widely used in multi-view stereo) for geometry-aware scene reasoning.
Abstract: We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference. Our approach leverages plane-swept cost volumes (widely used in multi-view stereo) for geometry-aware scene reasoning, and combines this with physically based volume rendering for neural radiance field reconstruction. We train our network on real objects in the DTU dataset, and test it on three different datasets to evaluate its effectiveness and generalizability. Our approach can generalize across scenes (even indoor scenes, completely different from our training scenes of objects) and generate realistic view synthesis results using only three input images, significantly outperforming concurrent works on generalizable radiance field reconstruction. Moreover, if dense images are captured, our estimated radiance field representation can be easily fine-tuned; this leads to fast per-scene reconstruction with higher rendering quality and substantially less optimization time than NeRF.
8 citations
TL;DR: A novel three-stage processing algorithm is devised to reconstruct sharp depth transitions, using a disparity map and geometric interpolation based on parametric Bézier curves, which consistently leads to quality improvement of synthesised images in comparison with reference methods.
Abstract: Transmission errors or packet loss in depth maps have great impact on the decoding quality and view synthesis of 3D and multiview video. Thus efficient methods to recover corrupted depth data are critical functions for accurate view rendering. This paper proposes an error concealment method for intra-coded depth maps, based on spatial intra and inter-view methods, which exploit neighbouring data of depth and colour images received error-free. A novel three-stage processing algorithm is devised to reconstruct sharp depth transitions (i.e. lost depth contours), using a disparity map and geometric interpolation based on parametric Bezier curves. The simulation results obtained from different views of various MVD sequences, for different packetisation modes and a wide range of packet loss rates (PLR), show that the proposed method consistently leads to quality improvement of synthesised images in comparison with reference methods.
8 citations
01 Sep 2000
TL;DR: This work presents a different approach based on plane homographies for image based view synthesis, which assumes that two physical planes in the scene can be identified and that a minimum of six matched points across three images is sufficient for view synthesis.
Abstract: The problem of synthesizing novel views consists of generating new views of a scene using at least two reference views. Although novel views might be rendered after the explicit 3D reconstruction of the scene, image based view synthesis is currently the most attractive approach. Image based view synthesis may be achieved through either epipolar geometry or trilinear tensors. We present a different approach based on plane homographies for image based view synthesis. Two cases are investigated here. In the first case, we assume that two physical planes in the scene can be identified and we show that a minimum of six matched points across three images is sufficient for view synthesis. In the second case however, we do not make any assumption and we show that we can calculate two plane homographies and use them for view synthesis. Experimental results on both real and simulated images show the feasibility and accuracy of our approach.
8 citations