# A method for 3D reconstruction of piecewise planar objects from single panoramic images

## Summary (3 min read)

### 1 Introduction

- Methods for 3D reconstruction from images abound in the literature.
- On one hand, research is directed towards completely automatic systems; these are relatively difficult to realize and it is not clear yet if they are ready for use by a non expert.
- The guideline of the work described here is to provide an intermediate solution, reconstruction from a single image, that needs relatively little user interaction.
- Most of the existing methods were developed for the use of a pinhole camera (with the exception of [12] where mosaics are used).
- Coplanarity constraints are used to complete the reconstruction, via simultaneous reconstruction of points and planes.

### 2 Camera Model

- The authors use an omnidirectional camera formed by the combination of a parabolic mirror and an orthographic camera whose viewing direction is parallel to the mirror’s axis [10].
- Geometrically speaking, the projection center of the orthographic camera coincides with the infinite one among the two focal points.
- This work is partially supported by the EPSRC funded project GR/K89221 . of the paraboloid.
- Given the image of a point and a small amount of calibration information described below, it is possible to determine the 3D direction of the line joining the original 3D point and the finite focal point of the paraboloid.
- These formulas are well known [10, 14], but presented here for the sake of completeness.

### 2.1 Representation of Mirror and Camera

- The mirror is a rotationally symmetric paraboloid.
- Without loss of generality, the authors may represent the paraboloid in usual quadric notation by the following symmetric matrix: where means equality up to scale, which accounts for the use of homogeneous coordinates.
- The mirror’s axis is the Z-axis and the finite focal point is the coordinate origin, i.e. "! .
- The parameter $ is the magnification factor of the orthographic projection.

### 2.2 Projection of a 3D Point

- Its projection can be computed as follows.
- Let 9 be the line joining / and the mirror’s finite focal point .

### 2.3 Calibration

- The above projection equations show that the mirror’s shape parameter and the magnification $ of the orthographic projectioncan be grouped together in a parameter AB C ED describing the combined system.
- These parameters have a simple geometrical meaning: consider the horizontal circle on the paraboloid at the height of the focal point .
- If the mirror’s top border does not lie at the height of the focal point , then the authors can not directly determine the circle G in the image.
- The calibration procedure has to be done only once for a fixed configuration.
- Another, more flexible calibration method, is described in [2].

### 2.4 Backprojection

- The most important feature of their mirror-camera system is that from a panoramic image, the authors may create correct perspective images of the scene, as if they had been observed by a pinhole camera with optical center at .
- Given the calibration parameters, the projection rays are determined in Euclidean space, which is useful for obtaining metric 3D reconstructions as described later.

### 3 Input

- To prepare the description of the 3D reconstruction method, the authors first explain the (user-provided) input.
- The basic primitives for their method are interest points ).
- Lines are defined by two or more interest points and they are grouped together into sets of mutually parallel lines and (d)).
- The input data are rather easy to provide interactively, which typically takes 10-15 minutes per image.
- 1By ideal points and ideal lines the authors denote points and lines at infinity respectively.

### 4 Basic Idea for 3D Reconstruction

- Sets of coplanar 3D points define polygons onto which texture can be mapped for visualization purposes.
- The authors assume that the image has been calibrated as described in 2.3.
- Planes with known normal and which contain / (known from the input) are then completely defined.
- This alternation scheme allows to reconstruct objects whose parts are sufficiently “interconnected”, i.e. the points on the object have to be linked together via coplanarity or other geometrical constraints.
- This discussion shows that it is possible to obtain a 3D reconstruction from one image and constraints of the types considered.

### 5.2 Computation of the Direction of Parallel Lines

- Given the input that two or more 3D lines are parallel, the authors can compute the lines’ 3D direction as follows.
- For each line, the authors may compute the 3D interpretation plane, i.e. the plane spanned by the focal point and the 3D line.
- This plane is given by the backprojection rays of the image points.
- If more than two points are given, a least squares fit is done to determine the plane: the normal is computed as the right singular vector associated to the least singular value [6] of the following matrix: - The interpretation plane is then given by 4 T .
- If more than two interpretation planes are given, a least squares fit is done as above.

### 5.3 Computation of the Normal Direction of a Set of Parallel Planes

- Planes are depicted by the user by indicating sets of coplanar points in the image and (f)).
- In the following, the authors suppose that the normal vectors have unit norm.

### 6 Simultaneous Reconstruction of Points and Planes

- The coplanarity constraints provided by the user are in general overconstrained, i.e. several points may lie on more than one plane.
- In the following, the authors only consider planes with known normal direction.
- The authors say that two planes and are connected if they share a point, i.e. if the intersection of and is non empty.
- The authors now show how the considered planes and points may be reconstructed simultaneously in a least squares manner.
- The partial derivatives (divided by ) of the cost function are given by: R; <; - Nullifying these equations leads to a homogeneous linear equation system in the unknowns and , giving the least squares solution.

### 7 Complete Algorithm

- Backproject the other points that lie on planes in the actual partition (cf. 5.4).
- Note that this process is done completely automatically.
- From the 3D reconstruction, the authors may create textured VRML models (see an example in 8).

### 8 Example

- The input image was obtained with the CycloVision ParaShot system and an Agfa ePhoto 1680 camera.
- Texture maps were created from the panoramic image using the projection equations in 2.2 and bicubic interpolation [8].
- With other images, similar results were obtained.

### 9 Conclusion

- The authors have presented a method for interactive 3D reconstruction of piecewise planar objects from a single panoramic view.
- The method was developed for a sensor based on a parabolic mirror, but its adaptation to other sensors is straightforward.
- 3D reconstruction is done using geometrical constraints provided by the user, that are simple in nature (coplanarity, perpendicularity and parallelism) and may be easily provided without any computer vision expertise.
- The major drawback of single-view 3D reconstruction is of course that only limited classes of objects may be reconstructed and that the reconstruction is usually incomplete.
- Please contact the author for getting a paper version with color figures.

Did you find this useful? Give us your feedback

##### Citations

760 citations

434 citations

### Cites methods from "A method for 3D reconstruction of p..."

...Furthermore, easily modified multiple view algorithms can be applied for reconstruction (Taylor, 2000; Sturm, 2000)....

[...]

234 citations

### Cites background from "A method for 3D reconstruction of p..."

...perpendicularity or parallelism of lines and coplanarity of points [470]....

[...]

210 citations

### Additional excerpts

...) !...

[...]

200 citations

### Cites background from "A method for 3D reconstruction of p..."

...Interestingly, there is a direct generalization for the para-catadioptric cameras, leading to the quartic polynomial eigenvalue problem....

[...]

##### References

3,280 citations

688 citations

### "A method for 3D reconstruction of p..." refers background or methods in this paper

...These formulas are well known [10, 14], but presented here for the sake of completeness....

[...]

...We use an omnidirectional camera formed by the combination of a parabolic mirror and an orthographic camera whose viewing direction is parallel to the mirror’s axis [10]....

[...]

432 citations

### "A method for 3D reconstruction of p..." refers methods in this paper

...in [4], but for small problems (the size of the matrix is the number of planes plus the number of points, which is usually at most a few dozens for single images) we simply use singular value decomposition [6]....

[...]

415 citations

310 citations

### "A method for 3D reconstruction of p..." refers background in this paper

...One of the two approaches in [9] achieves the reconstruction by measuring heights of points with respect to a ground plane....

[...]

...the approaches described in [9, 12] only allow to reconstruct planar surfaces whose vanishing line can be determined in the image....

[...]

##### Related Papers (5)

##### Frequently Asked Questions (2)

###### Q2. What are the future works in "A method for 3d reconstruction of piecewise planar objects from single panoramic images" ?

The authors have presented a method for interactive 3D reconstruction of piecewise planar objects from a single panoramic view. 3D reconstruction is done using geometrical constraints provided by the user, that are simple in nature ( coplanarity, perpendicularity and parallelism ) and may be easily provided without any computer vision expertise. The major drawback of single-view 3D reconstruction is of course that only limited classes of objects may be reconstructed and that the reconstruction is usually incomplete. One advantage of their method compared to other approaches is that a wider class of objects can be reconstructed ( especially, there is no requirement of disposing of two or more ideal points for each plane ).