Conventional video cameras have limited fields of view which make them restrictive for certain applications in computational vision. A catadioptric sensor uses a combination of lenses and mirrors placed in a carefully arranged configuration to capture a much wider field of view. One important design goal for catadioptric sensors is choosing the shapes of the mirrors in a way that ensures that the complete catadioptric system has a single effective viewpoint. The reason a single viewpoint is so desirable is that it is a requirement for the generation of pure perspective images from the sensed images. In this paper, we derive the complete class of single-lens single-mirror catadioptric sensors that have a single viewpoint. We describe all of the solutions in detail, including the degenerate ones, with reference to many of the catadioptric systems that have been proposed in the literature. In addition, we derive a simple expression for the spatial resolution of a catadioptric sensor in terms of the resolution of the cameras used to construct it. Moreover, we include detailed analysis of the defocus blur caused by the use of a curved mirror in a catadioptric sensor.

A Theory of Single-Viewpoint Catadioptric Image Formation

Omnidirectional vision systems can provide panoramic alertness in surveillance, improve navigational capabilities, and produce panoramic images for multimedia. Catadioptric realizations of omnidirectional vision combine reflective surfaces and lenses. A particular class of them, the central panoramic systems, preserve the uniqueness of the projection viewpoint. In fact, every central projection system including the well known perspective projection on a plane falls into this category.

In this paper, we provide a unifying theory for all central catadioptric systems. We show that all of them are isomorphic to projective mappings from the sphere to a plane with a projection center on the perpendicular to the plane. Subcases are the stereographic projection equivalent to parabolic projection and the central planar projection equivalent to every conventional camera. We define a duality among projections of points and lines as well as among different mappings.

This unification is novel and has a significant impact on the 3D interpretation of images. We present new invariances inherent in parabolic projections and a unifying calibration scheme from one view. We describe the implied advantages of catadioptric systems and explain why images arising in central catadioptric systems contain more information than images from conventional cameras. One example is that intrinsic calibration from a single view is possible for parabolic catadioptric systems given only three lines. Another example is metric rectification using only affine information about the scene.

/pdf/a-unifying-theory-for-central-panoramic-systems-and-k79l8nhczt.pdf

A Unifying Theory for Central Panoramic Systems and Practical Applications

Catadioptric sensors are devices which utilize mirrors and lenses to form a projection onto the image plane of a camera. Central catadioptric sensors are the class of these devices having a single effective viewpoint. In this paper, we propose a unifying model for the projective geometry induced by these devices and we study its properties as well as its practical implications. We show that a central catadioptric projection is equivalent to a two-step mapping via the sphere. The second step is equivalent to a stereographic projection in the case of parabolic mirrors. Conventional lens-based perspective cameras are also central catadioptric devices with a virtual planar mirror and are, thus, covered by the unifying model. We prove that for each catadioptric projection there exists a dual catadioptric projection based on the duality between points and line images (conics). It turns out that planar and parabolic mirrors build a dual catadioptric projection pair. As a practical example we describe a procedure to estimate focal length and image center from a single view of lines in arbitrary position for a parabolic catadioptric system.

/pdf/catadioptric-projective-geometry-34v0kgyfe7.pdf

Catadioptric Projective Geometry

Conventional video cameras have limited fields of view which make them restrictive for certain applications in computational vision. A catadioptric sensor uses a combination of lenses and mirrors placed in a carefully arranged configuration to capture a much wider field of view. When designing a catadioptric sensor, the shape of the mirror(s) should ideally be selected to ensure that the complete catadioptric system has a single effective viewpoint. In this paper, we derive the complete class of single-lens single-mirror catadioptric sensors which have a single viewpoint and an expression for the spatial resolution of a catadioptric sensor in terms of the resolution of the camera used to construct it. We also include a preliminary analysis of the defocus blur caused by the use of a curved mirror.

/pdf/a-theory-of-catadioptric-image-formation-8thl4b2kae.pdf

A theory of catadioptric image formation

Digital video imaging systems and techniques for efficiently transforming warped video images into rectilinear video images, real-time tracking of persons and objects, face recognition of persons, monitoring and tracking head pose of a person and associated perspective view of the person.

Digital processing of video images

Described here is an image sensor with a hyperboloidal mirror for vision based navigation of a mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of a hyperboloidal mirror.

Omnidirectional imaging with hyperboloidal projection

Described here is an image sensor with a hyperboloidal mirror for vision based navigation of a mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of a hyperboloidal mirror. The authors show a prototype of a mobile robot system with HyperOmni Vision and a method for estimating the motion of the robot and finding unknown obstacles.

Obstacle detection with omnidirectional image sensor HyperOmni Vision

Telepresence by Real-Time View-Dependent Image Generation from Omnidirectional Video Streams

We have developed a high-resolution omnidirectional stereo imaging sensor that can take images at video-rate. The sensor system takes an omnidirectional view by a component constructed of six cameras and a hexagonal pyramidal mirror and acquires stereo views by symmetrically connecting two sensor components. The paper describes a method of generating stereo panoramic images by using our sensor. First, the sensor system is calibrated; that is, twelve cameras are correctly aligned with pyramidal mirrors and the Tsai's method restores the radial distortion of each camera image. Stereo panoramic images are then computed by registering the camera images captured at the same time.

Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors

This paper describes a visual surveillance and monitoring system which is based on omnidirectional imaging and view-dependent image generation from omnidirectional video streams. While conventional visual surveillance and monitoring systems usually consist of either a number of fixed regular cameras or a mechanically controlled camera, the proposed system has a single omnidirectional video camera using a hyperboloidal mirror. This approach has an advantage of less latency in looking around a large field of view. In a prototype system developed, the viewing direction is determined by viewers' head tracking, by using a mouse, or by moving object trading in the omnidirectional image.

Kazumasa Yamazawa

Papers

Omnidirectional imaging with hyperboloidal projection

Obstacle detection with omnidirectional image sensor HyperOmni Vision

Telepresence by Real-Time View-Dependent Image Generation from Omnidirectional Video Streams

Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors

Visual surveillance and monitoring system using an omnidirectional video camera