View synthesis

About: View synthesis is a research topic. Over the lifetime, 1701 publications have been published within this topic receiving 42333 citations.

View Synthesis Oriented Depth Map Coding Algorithm

Abstract: A view synthesis-oriented depth map coding algorithm is proposed in this paper. Depth map is classified into inner motion regions, background subtracted with edges and edge regions by using edge detection of current depth map and the frame difference between corresponding color images. Then macroblocks in different regions are encoded with different RDO schemes so as to speed up the encoding process while keep the quality of reconstructed depth map. Experimental results show that the proposed method not only improves the encoding speed significantly, but also saves bitrate of depth map sequence while maintaining high quality of synthesized virtual view images.

View synthesis and 3D reconstruction of piecewise planar scenes using intersection lines between the planes

Abstract: A linear algorithm for synthesizing new views of piecewise planar objects without making an explicit 3D reconstruction is proposed, together with a simple algorithm for 3D reconstruction of the scene. It is shown how this can be done using only one image and information about the projection of the intersection lines between the object planes. These could either be estimated manually or more automatically using at least one more image. No calibration information is needed. A main idea in the paper is to work with textured planes. A patch in one image, corresponding to a planar surface in the scene, is transformed to a patch in another image by a homography. The generalized eigenvectors and eigenvalues of the homographies have geometrical interpretations that are used in the algorithms. To generate a new image, we calculate the homographies for each plane to the new image. The textures can then easily be mapped to the new image. The reconstruction is given as the equation of each plane in the corresponding homography to a certain image.

Novel view synthesis with residual error feedback for FTV

Abstract: The availability of multi-view images of a scene makes possible new and exciting applications, including Free-viewpoint TV (FTV). FTV allows us to change viewpoint freely in a 3D world, where the virtual viewpoint images are synthesized by Depth-Image-Based Rendering (DIBR). In this paper, we propose a new method of DIBR using multi-view images acquired in a linear camera arrangement. The proposed method improves virtual viewpoint images by predicting the residual errors. For virtual viewpoint image synthesis, it is necessary to estimate the depth maps with multi-view images. Some algorithms to estimate depth map were proposed, but it is difficult to estimate accurate depth map. As a result, rendered virtual viewpoint images have some errors due to the depth errors. Therefore, our proposed method takes into account those depth errors and improves the quality of the rendered virtual viewpoint images. In the proposed method, the virtual images of each camera position are generated using the real images from each other camera. Then, the residual errors can be calculated between the generated images and the real images acquired by the actual cameras. The residual errors are processed and fed back to predict the residual errors that can be happened to virtual viewpoint images generated by conventional method. In the experiments, PSNR could be improved for few decibels compared with the conventional method.

Virtual View Specification and Synthesis for Free Viewpoint Television

Abstract: Free viewpoint television (FTV) is a new concept that aims at giving viewers the flexibility to select a novel viewpoint by employing multiple video streams as the input. Current proposed solutions for FTV include those based on ray-space resampling which demand at least dozens of cameras and large storage and transmission resources for those video streams. Image-based rendering (IBR) methods that rely on dense depth map estimation also face practical difficulties since accurate depth map estimation remains a challenging problem. This paper proposes a framework for FTV based on IBR that relieves the need for an accurate depth map by introducing a hybrid virtual view synthesis method. The framework also includes an intuitive method for virtual view specification in uncalibrated views. We present both simulation and real data experiments to validate the proposed framework and the component algorithms.

Depth-guided inpainting algorithm for Free-Viewpoint Video

Abstract: Free-Viewpoint Video (FVV) is a novel technique which creates virtual images of multiple direction by view synthesis. In this paper, an exemplar-based depth-guided inpainting algorithm is proposed to fill disocclusions due to uncovered areas after projection. We develop an improved priority function which uses the depth information to impose a desirable inpainting order. We also propose an efficient background-foreground separation technique to enhance the accuracy of hole filling. Furthermore, a gradient-based searching approach is developed to reduce the computational cost and the location distance is incorporated into patch matching criteria to improve the accuracy. The experimental results have shown that the gradient-based search in our algorithm requires a much lower computational cost (factor of 6 compared to global search), while producing significantly improved visual results.