scispace - formally typeset
Search or ask a question

Showing papers on "View synthesis published in 2000"


Journal ArticleDOI
TL;DR: A novel automatic method for view synthesis from a triplet of uncalibrated images based on trinocular edge matching followed by transfer by interpolation, occlusion detection and correction and finally rendering is presented.

38 citations


Proceedings ArticleDOI
28 May 2000
TL;DR: A multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed in this paper for 3-D virtual reality applications.
Abstract: A multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed in this paper for 3-D virtual reality applications. In the first part of this paper, we develop a codec that features random access to any encoded view. The codec performance is good and comparable to the MPEG standard. In the second part, we interpolate the disparities and refer to appropriate decoded images for the synthesis of any intermediate views. Our design provides enough information so that each synthesis can be achieved within 0.04 sec for a 320/spl times/240 image (excluding the decoding time). An eye glass viewer with infrared synchronization is also adopted to view the stereo results which are very good.

16 citations


Book ChapterDOI
26 Jun 2000
TL;DR: This work shows how the 'combination of views' proposed by Ullman and Basri [19] can be appropriately parameterised when a sequence of five or more images is available, and discusses an alternative approach, direct image-interpolation, and argues that this method is preferable when there is a large difference in orientation between the original gaze directions.
Abstract: We present a simple procedure for synthesising novel views, using two or more basis-images as input It is possible for the user to interactively adjust the viewpoint, and for the corresponding image to be computed and rendered in real-time Rather than employing a 3D model, our method is based on the linear relations which exist between images taken with an affine camera We show how the 'combination of views' proposed by Ullman and Basri [19] can be appropriately parameterised when a sequence of five or more images is available This is achieved by fitting polynomial models to the coefficients of the combination, where the latter are functions of the (unknown) camera parameters We discuss an alternative approach, direct image-interpolation, and argue that our method is preferable when there is a large difference in orientation between the original gaze directions We show the results of applying the parameterisation to a fixating camera, using both simulated and real input Our observations are relevant to several applications, including visualisation, animation, and low-bandwidth communication

14 citations


Proceedings ArticleDOI
26 Mar 2000
TL;DR: This paper proposes a new method, called 3D spring-based face model (SBFM), to determine the precise face model of a person with different poses and facial expressions from a single image.
Abstract: It is known that 2D views of a person can be synthesised if the face 3D model of that person is available. This paper proposes a new method, called 3D spring-based face model (SBFM), to determine the precise face model of a person with different poses and facial expressions from a single image. The SBFM combines the concepts of generic 3D face model in computer graphics and deformable template in computer vision. Face image databases from MIT AI laboratory and Yale University are used to test our proposed method and the results are encouraging.

12 citations


Proceedings ArticleDOI
23 Jun 2000
TL;DR: This approach to the problem of image-based view synthesis is based upon the implicit construction of a 3D approximation of the scene, composed of planar triangular patches, which minimizes the distortions in the rendering of the new view.
Abstract: The teleoperation of equipment under impoverished sensing and communication delays, cannot be handled efficiently by conventional remote control techniques. Our approach to this problem is based on an augmented reality control mode in which a graphical model of the equipment is overlaid upon real views from the work-site. A basic capability required in order to produce such an augmented reality mode is the ability to synthesize visual information from new viewpoints based upon existing ones, so as to compensate for the sparsity of real data. Our approach to the problem of image-based view synthesis is based upon the implicit construction of a 3D approximation of the scene, composed of planar triangular patches. New views are then generated by texture-mapping the available real image data onto the reprojected triangles. In order to generate a physically valid joint-triangulation which minimizes the distortions in the rendering of the new view, an iterative approach is utilized. This approach begins with an initial triangulation and refines it iteratively through node-linking alterations and a split and merge process, based upon correlation values between corresponding triangular patches. The paper presents results for both synthetic and real scenes.

12 citations


Proceedings ArticleDOI
30 Jul 2000
TL;DR: The method uses for its part the synthesis of photorealistic views based on the trilinearity theory combined with image mosaicking, which minimizes the algebraic processing needed for view synthesis in order to offer in real time a natural rendering of the scene to the user.
Abstract: Among the new data processing services and applications, virtual navigation in a real 3D world takes more and more importance and requires specific research. With regard to the reactions of the most important part of net surfers for example, we think that for a user of an interactive immersive tool, it is absolutely necessary to obtain a real time rendering of the virtual world, as natural and realistic as possible. That is why we focus our work on an image based approach, avoiding the slow and heavy handling of 3D models of the scene, generally not very realistic and typical in geometry based approaches. Our method uses for its part the synthesis of photorealistic views based on the trilinearity theory combined with image mosaicking. It especially minimizes the algebraic processing needed for view synthesis in order to offer in real time a natural rendering of the scene to the user, according to his/her relative movement from his/her initial position.

9 citations


Proceedings ArticleDOI
01 Sep 2000
TL;DR: This work presents a different approach based on plane homographies for image based view synthesis, which assumes that two physical planes in the scene can be identified and that a minimum of six matched points across three images is sufficient for view synthesis.
Abstract: The problem of synthesizing novel views consists of generating new views of a scene using at least two reference views. Although novel views might be rendered after the explicit 3D reconstruction of the scene, image based view synthesis is currently the most attractive approach. Image based view synthesis may be achieved through either epipolar geometry or trilinear tensors. We present a different approach based on plane homographies for image based view synthesis. Two cases are investigated here. In the first case, we assume that two physical planes in the scene can be identified and we show that a minimum of six matched points across three images is sufficient for view synthesis. In the second case however, we do not make any assumption and we show that we can calculate two plane homographies and use them for view synthesis. Experimental results on both real and simulated images show the feasibility and accuracy of our approach.

8 citations


Journal ArticleDOI
TL;DR: A two-way interaction is built between the analysis and reconstruction stages, which provides the tradeoff between the final image quality and amount of data transmitted, in a low-complexity solution enabling online processing capability while preserving the MPEG-4 compatibility of the I3D representation.
Abstract: A new approach for compact representation, MPEG-4 encoding, and reconstruction of video objects captured by an uncalibrated system of multiple cameras is presented. The method is based on the incomplete 3-D (I3D) technique, which was initially investigated for stereo video objects captured by parallel cameras. Non-overlapping portions of the object are extracted from the reference views, each view having the corresponding portion with the highest resolution. This way, the redundancy of the initial multiview data is reduced. The areas which are extracted from the basis views are denoted as areas of interest. The output of the analysis stage, i.e., the areas of interest and the corresponding parts of the disparity fields are encoded in the MPEG-4 bitstream. Disparity fields define the correspondence relations between the reference views. The view synthesis is performed by disparity-oriented reprojection of the areas of interest into the virtual view plane and can be seen as an intermediate postprocessing stage between the decoder and the scene compositor. This work performs an extension from parallel stereo views to arbitrary configured multi-views with new analysis and synthesis algorithms. Moreover, a two-way interaction is built between the analysis and reconstruction stages, which provides the tradeoff between the final image quality and amount of data transmitted. The focus is on a low-complexity solution enabling online processing capability while preserving the MPEG-4 compatibility of the I3D representation. It is finally shown that our method yields quite convincing results despite the minimal data used and the approximations involved.

6 citations



Proceedings ArticleDOI
18 Mar 2000
TL;DR: A linear algorithm that is useful for realizing geometric registration between the view of a real scene and that of a virtual object in an image-based rendering framework and the recovery of the camera pose that is necessary for the base image selection can be performed.
Abstract: We propose a linear algorithm that is useful for realizing geometric registration between the view of a real scene and that of a virtual object in an image-based rendering framework. In a unified framework, the novel view synthesis of a virtual object based on three views' matching constraints and the recovery of the camera pose that is necessary for the base image selection can be performed. The feasibility of the algorithm is demonstrated by using ground-truth synthesized data and real scene data.

6 citations



Proceedings ArticleDOI
01 Sep 2000
TL;DR: A new algebraic derivation is proposed for the projection matrix of the third camera by using the relationships between the projection matrices, the epipolar geometry and the trilinear tensors to work in the special case where the camera centers are collinear.
Abstract: In the computation of the 3D structure, it has been demonstrated that by introducing a third camera provides improvement over the use of two views only. In this paper, we propose a new algebraic derivation for the projection matrix of the third camera. This derivation is obtained by using the relationships between the projection matrices, the epipolar geometry and the trilinear tensors. Compared to previous methods, our method has the advantage to work in the special case where the camera centers are collinear. In addition, it has the advantage of simplicity and it is very straightforward to implement since it is a linear method. In our experiments, we used the view synthesis to high-light the quality of the computed projective structure. Tests have been performed on both synthetic and real data.

Proceedings ArticleDOI
Paul Bao1, D. Xu1
19 Jul 2000
TL;DR: A new view synthesis technique using the 2D discrete wavelet-based view morphing to generate new views with linear interpolating techniques and Quantization techniques can be embedded here to compress the coefficients to reduce the morphing complexity.
Abstract: This paper presents a new view synthesis technique using the 2D discrete wavelet-based view morphing. View morphing is completely based on pairwise images without camera calibration and depth information of images. First, the fundamental matrix related to any pair of images is estimated. Then, with the fundamental matrix, the pair of image planes is rectified to be parallel and to have their corresponding points lie on the same scanline, which gives an opportunity to generate new views with linear interpolating techniques. The pre-warped images are then decomposed into a hierarchical structure with the wavelet transform. Corresponding coefficients between two decomposed images are therefore linearly interpolated to form the multiresolution representation of an intermediate view. Quantization techniques can be embedded here to compress the coefficients to reduce the morphing complexity. Finally, when displaying, compressed images are decoded and inverse wavelet transformed. A post-warping procedure is employed to transform the interpolated views to its desired position.

Proceedings Article
01 Jan 2000
TL;DR: This paper presents a synthesis method for arbitrary virtual views in a multi-view camera set-up using the recently developed Incomplete 3D technique, a disparity-based multiview representation for a weakly convergent camera setup in combination with the trilinear warping functions to describe new virtual camera positions.
Abstract: The proposed approach is motivated by applications which allow user navigation and individual viewpoint specification in shared virtual environments with telepresence quality. In this context, we present a synthesis method for arbitrary virtual views in a multi-view camera set-up. This method generates a close to realtime, view adaptable reconstruction of a 3dimensional, (3D), object taken from at least two cameras. In this method, we use the recently developed Incomplete 3D, (IC3D), technique, a disparity-based multiview representation for a weakly convergent camera setup in combination with the trilinear warping functions to describe new virtual camera positions. Previously, only virtual views inside the baseline could be described by IC3D. Hence, user movement in virtual environments (VE) was restricted. In order to create a virtual camera position and orientation outside the baseline we apply point correspondences across the two reference images to a trilinear tensor, built from the fundamental matrix between the reference views. The trilinearities provide a general warping function from the reference images to virtual view that is governed directly by the virtual camera parameters.

Proceedings ArticleDOI
01 Sep 2000
TL;DR: The aim of this study is to determine whether needle-maps delivered by a new shape-from-shading (SFS) algorithm can be used as a compact object representation, and to show that the needle maps can be use to generate novel object views.
Abstract: Investigates the use of shape from-shading for view synthesis. The aim of our study is to determine whether needle-maps delivered by a new shape-from-shading (SFS) algorithm can be used as a compact object representation. Specifically, we aim to show that the needle maps can be used to generate novel object views. To this end we conduct two sets of experiments. Firstly, we use the recovered needle maps to re-illuminate objects under varying lighting directions. Here we show that a single input image can be used to construct faithful re-illuminations under radical illumination changes. Secondly, we investigate how the needle map can be used to generate new object poses. Here we show that needle-maps can be used for both view interpolation and view extrapolation.

01 Jan 2000
TL;DR: In this paper, a multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed for 3D virtual reality applications, which can be achieved within 0.04 sec for a 320~ 240 image (excluding the decoding time).
Abstract: A multi-view image compression system that is capable of providing sufficient and reliable disparity information for intermediate view synthesis is proposed in this paper for 3-D virtual reality applications. In the first part of this paper, we develop a codec that is featured of random access to any encoded view. The codec performance is good and comparable to the MPEG standard. In the second .part, we interpolate the disparities and refer to appropriate decoded images for the synthesis of any intermediate views. Our design provides enough information so that each synthesis can be achieved within 0.04 sec for a 320~ 240 image (excluding the decoding time). An eye glass viewer with infrared synchronization is also adopted to view the stereo results and they really look good.

Journal ArticleDOI
TL;DR: In this article, a 2D view cannot be represented by a linear combination of other 2D views under perspective projection, and instead, a solution is presented to solve the problem under the perspective projection.
Abstract: This paper addresses the issue of generating a 2D view of a 3D object from its other 2D views. Linear Combination method is the typical approach to this problem. However, a 2D view cannot be represented by a linear combination of other 2D views under perspective projection. Instead, we have presented a solution under perspective projection. The proposed method also applies to the construction of virtual frontal view face image and the results are encouraging.