View morphing

doi:10.1145/237170.237196

Home
/
Papers
/
View morphing

Proceedings Article•DOI•

View morphing

Steven M. Seitz¹, Charles R. Dyer¹•Institutions (1)

University of Wisconsin-Madison¹

01 Aug 1996-pp 21-30

TL;DR: This paper introduces a simple extension to image morphing that correctly handles 3D projective camera and scene transformations and works by prewarping two images prior to computing a morph and then postwarped the interpolated images.

read less

Abstract: Image morphing techniques can generate compelling 2D transitions between images. However, differences in object pose or viewpoint often cause unnatural distortions in image morphs that are difficult to correct manually. Using basic principles of projective geometry, this paper introduces a simple extension to image morphing that correctly handles 3D projective camera and scene transformations. The technique, called view morphing, works by prewarping two images prior to computing a morph and then postwarping the interpolated images. Because no knowledge of 3D shape is required, the technique may be applied to photographs and drawings, as well as rendered scenes. The ability to synthesize changes both in viewpoint and image structure affords a wide variety of interesting 3D effects via simple image transformations. CR

...read moreread less

Citations

PDF

Open Access

More filters

Book•

Computer Vision: Algorithms and Applications

[...]

Richard Szeliski

30 Sep 2010

TL;DR: Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images and takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene.

...read moreread less

Abstract: Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

...read moreread less

4,146 citations

Cites background from "View morphing"

..., a person smiling in one image and frowning in the other, view morphing, which combines ideas from view interpolation with regular morphing, can be used (Seitz and Dyer 1996)....
[...]
...(Optional) Take a single image, such as the Mona Lisa or a friend’s picture, and create and animate 3D view morph (Seitz and Dyer 1996)....
[...]
...1989); (d) view morphing between two images (Seitz and Dyer 1996); (e–f) 3D face modeling (images courtesy of Frédéric Devernay); (g) z-keying live and computer generated imagery (Kanade et al....
[...]
...5c) (see Section 3.6.3 and Beier and Neely 1992) and was later applied to view interpolation (Chen and Williams 1993; Seitz and Dyer 1996), panoramic image stitching (Figure1....
[...]

Journal Article•DOI•

Photo tourism: exploring photo collections in 3D

[...]

Noah Snavely¹, Steven M. Seitz¹, Richard Szeliski²•Institutions (2)

University of Washington¹, Microsoft²

01 Jul 2006

TL;DR: This work presents a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface that consists of an image-based modeling front end that automatically computes the viewpoint of each photograph and a sparse 3D model of the scene and image to model correspondences.

...read moreread less

Abstract: We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each photograph as well as a sparse 3D model of the scene and image to model correspondences. Our photo explorer uses image-based rendering techniques to smoothly transition between photographs, while also enabling full 3D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. Our system also makes it easy to construct photo tours of scenic or historic locations, and to annotate image details, which are automatically transferred to other relevant images. We demonstrate our system on several large personal photo collections as well as images gathered from Internet photo sharing sites.

...read moreread less

3,398 citations

Cites background or methods from "View morphing"

...…more challenging problems of reconstructing full surface models [Debevec et al. 1996; Teller et al. 2003], light fields [Gortler et al. 1996; Levoy and Hanrahan 1996], or pixel-accurate view interpolations [Chen and Williams 1993; McMillan and Bishop 1995; Seitz and Dyer 1996; Zitnick et al. 2004]....
[...]
...More recent work in image-based rendering has focused on techniques for new view synthesis, e.g., [Chen and Williams 1993; McMillan and Bishop 1995; Gortler et al. 1996; Levoy and Hanrahan 1996; Seitz and Dyer 1996; Aliaga et al. 2003; Zitnick et al. 2004]....
[...]
...1996; Levoy and Hanrahan 1996], or pixel-accurate view interpolations [Chen and Williams 1993; McMillan and Bishop 1995; Seitz and Dyer 1996; Zitnick et al. 2004]....
[...]

Journal Article•DOI•

Modeling the World from Internet Photo Collections

[...]

Noah Snavely¹, Steven M. Seitz¹, Richard Szeliski²•Institutions (2)

University of Washington¹, Microsoft²

01 Nov 2008-International Journal of Computer Vision

TL;DR: This paper presents structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like “Notre Dame” or “Trevi Fountain,” and presents these algorithms and results as a first step towards 3D modeled sites, cities, and landscapes from Internet imagery.

...read moreread less

Abstract: There are billions of photographs on the Internet, comprising the largest and most diverse photo collection ever assembled. How can computer vision researchers exploit this imagery? This paper explores this question from the standpoint of 3D scene modeling and visualization. We present structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like "Notre Dame" or "Trevi Fountain." This approach, which we call Photo Tourism, has enabled reconstructions of numerous well-known world sites. This paper presents these algorithms and results as a first step towards 3D modeling of the world's well-photographed sites, cities, and landscapes from Internet imagery, and discusses key open problems and challenges for the research community.

...read moreread less

2,207 citations

Cites background or methods from "View morphing"

...More recent work in IBR has focused on techniques for new view synthesis, e.g., (Chen and Williams 1993; McMillan and Bishop 1995; Gortler et al. 1996; Levoy and Hanrahan 1996; Seitz and Dyer 1996; Aliaga et al. 2003; Zitnick et al. 2004; Buehler et al. 2001)....
[...]
...4.1 Keypoint Detection and Matching The first step is to find feature points in each image....
[...]
...If more than one feature in I matches the same feature in J , we remove all of these matches, as some of them must be spurious....
[...]
...…more challenging problems of reconstructing full surface models (Debevec et al. 1996; Teller et al. 2003), light fields (Gortler et al. 1996; Levoy and Hanrahan 1996), or pixel-accurate view interpolations (Chen and Williams 1993; McMillan and Bishop 1995; Seitz and Dyer 1996; Zitnick et al. 2004)....
[...]

Proceedings Article•DOI•

Unsupervised Learning of Depth and Ego-Motion from Video

[...]

Tinghui Zhou¹, Matthew Brown², Noah Snavely², David G. Lowe²•Institutions (2)

University of California, Berkeley¹, Google²

25 Apr 2017

TL;DR: In this paper, an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences is presented, which uses single-view depth and multiview pose networks with a loss based on warping nearby views to the target using the computed depth and pose.

...read moreread less

Abstract: We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences. In common with recent work [10, 14, 16], we use an end-to-end learning approach with view synthesis as the supervisory signal. In contrast to the previous work, our method is completely unsupervised, requiring only monocular video sequences for training. Our method uses single-view depth and multiview pose networks, with a loss based on warping nearby views to the target using the computed depth and pose. The networks are thus coupled by the loss during training, but can be applied independently at test time. Empirical evaluation on the KITTI dataset demonstrates the effectiveness of our approach: 1) monocular depth performs comparably with supervised methods that use either ground-truth pose or depth for training, and 2) pose estimation performs favorably compared to established SLAM systems under comparable input settings.

...read moreread less

1,972 citations

Proceedings Article•DOI•

Layered depth images

[...]

Jonathan Shade¹, Steven J. Gortler², Li-wei He³, Richard Szeliski⁴•Institutions (4)

University of Washington¹, Harvard University², Stanford University³, Microsoft⁴

24 Jul 1998

TL;DR: A set of efficient image based rendering methods capable of rendering multiple frames per second on a PC that warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques and splatting an efficient solution to the resampling problem.

...read moreread less

Abstract: In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan’s warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alphacompositing can be done efficiently without depth sorting. This makes splatting an efficient solution to the resampling problem.

...read moreread less

1,362 citations

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175

Collapse

References

PDF

Open Access

More filters

Proceedings Article•

Three-dimensional computer vision: a geometric viewpoint

[...]

Olivier Faugeras¹•Institutions (1)

French Institute for Research in Computer Science and Automation¹

31 Dec 1993

TL;DR: Results from constrained optimization some results from algebraic geometry differential geometry are shown.

...read moreread less

Abstract: Projective geometry modelling and calibrating cameras edge detection representing geometric primitives and their uncertainty stereo vision determining discrete motion from points and lines tracking tokens over time motion fields of curves interpolating and approximating three-dimensional data recognizing and locating objects and places answers to problems. Appendices: constrained optimization some results from algebraic geometry differential geometry.

...read moreread less

2,744 citations

Journal Article•DOI•

Animating rotation with quaternion curves

[...]

Ken Shoemake¹•Institutions (1)

Singer Corporation¹

01 Jul 1985

TL;DR: A new kind of spline curve is presented, created on a sphere, suitable for smoothly in-betweening (i.e. interpolating) sequences of arbitrary rotations, without quirks found in earlier methods.

...read moreread less

Abstract: Solid bodies roll and tumble through space. In computer animation, so do cameras. The rotations of these objects are best described using a four coordinate system, quaternions, as is shown in this paper. Of all quaternions, those on the unit sphere are most suitable for animation, but the question of how to construct curves on spheres has not been much explored. This paper gives one answer by presenting a new kind of spline curve, created on a sphere, suitable for smoothly in-betweening (i.e. interpolating) sequences of arbitrary rotations. Both theory and experiment show that the motion generated is smooth and natural, without quirks found in earlier methods.

...read moreread less

2,006 citations

"View morphing" refers methods in this paper

...Alternatively, if the orientations are expressed using quaternions, the interpolation is computed by spherical linear interpolation [13]....
[...]

Proceedings Article•DOI•

Plenoptic modeling: an image-based rendering system

[...]

Leonard McMillan¹, Gary Bishop¹•Institutions (1)

University of North Carolina at Chapel Hill¹

15 Sep 1995

TL;DR: An image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function is presented and a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equivalent to the epipolar constraint defined for planar projections are introduced.

...read moreread less

Abstract: Image-based rendering is a powerful new approach for generating real-time photorealistic computer graphics. It can provide convincing animations without an explicit geometric representation. We use the “plenoptic function” of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation. The plenoptic function is a parameterized function for describing everything that is visible from a given point in space. We present an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function. In addition, we introduce a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equivalent to the epipolar constraint defined for planar projections.

...read moreread less

1,555 citations

Proceedings Article•DOI•

QuickTime VR: an image-based approach to virtual environment navigation

[...]

Shenchang Eric Chen¹•Institutions (1)

Apple Inc.¹

15 Sep 1995

TL;DR: This paper presents a new approach which uses 360-degree cylindrical panoramic images to compose a virtual environment which includes viewing of an object from different directions and hit-testing through orientation-independent hot spots.

...read moreread less

Abstract: Traditionally, virtual reality systems use 3D computer graphics to model and render virtual environments in real-time. This approach usually requires laborious modeling and expensive special purpose rendering hardware. The rendering quality and scene complexity are often limited because of the real-time constraint. This paper presents a new approach which uses 360-degree cylindrical panoramic images to compose a virtual environment. The panoramic image is digitally warped on-the-fly to simulate camera panning and zooming. The panoramic images can be created with computer rendering, specialized panoramic cameras or by "stitching" together overlapping photographs taken with a regular camera. Walking in a space is currently accomplished by "hopping" to different panoramic points. The image-based approach has been used in the commercial product QuickTime VR, a virtual reality extension to Apple Computer's QuickTime digital multimedia framework. The paper describes the architecture, the file format, the authoring process and the interactive players of the VR system. In addition to panoramic viewing, the system includes viewing of an object from different directions and hit-testing through orientation-independent hot spots. CR

...read moreread less

1,515 citations

"View morphing" refers background in this paper

...Image mosaic techniques [10, 2, 14, 6] also rely heavily on reprojection methods to project images onto a planar, cylindrical, or spherical manifold....
[...]
...In terms of its ability to achieve changes in viewpoint, view morphing is related to previous view-based techniques such as view synthesis [3, 7, 11, 12] and mosaics [10, 2, 14, 6]....
[...]

Book•

Digital Image Warping

[...]

George Wolberg

10 Aug 1990

TL;DR: This book is intended to be a practical guide for eclectic scientists and engineers who find themselves in need of implementing warping algorithms and comprehending the underlying concepts.

...read moreread less

Abstract: From the Publisher: This book is intended to be a practical guide for eclectic scientists and engineers who find themselves in need of implementing warping algorithms and comprehending the underlying concepts.

...read moreread less

1,461 citations

"View morphing" refers background or methods in this paper

...In this paper, the term image morphing refers specifically to methods that use linear interpolation to compute feature positions in in-between images, including [15, 1, 8]....
[...]
...Any of several image morphing techniques, for instance [15, 1, 8], may be used to compute the intermediate image interpolation....
[...]
...Our use of projective transforms to compute reprojections takes advantage of an efficient scanline algorithm [15]....
[...]
...The remaining correspondences are determined automatically by interpolation [15, 1, 8]....
[...]
...The positions of the control points in Is and Îs specify a linear system of equations whose solution yields Hs [15]....
[...]