An omnistereo panorama consists of a pair of panoramic images, where one panorama is for the left eye and another panorama is for the right eye. The panoramic stereo pair provides a stereo sensation up to a full 360 degrees. Omnistereo panoramas can be constructed by mosaicing images from a single rotating camera. This approach also enables the control of stereo disparity, giving larger baselines for faraway scenes, and a smaller baseline for closer scenes. Capturing panoramic omnistereo images with a rotating camera makes it impossible to capture dynamic scenes at video rates and limits omnistereo imaging to stationary scenes. We present two possibilities for capturing omnistereo panoramas using optics without any moving parts. A special mirror is introduced such that viewing the scene through this mirror creates the same rays as those used with the rotating cameras. The lens used for omnistereo panorama is also introduced, together with the design of the mirror. Omnistereo panoramas can also be rendered by computer graphics methods to represent virtual environments.

/pdf/omnistereo-panoramic-stereo-imaging-39wj1bbut7.pdf

Omnistereo: panoramic stereo imaging

A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been constrained to planar concentric circles. We resample regular perspective images to produce a set of multiperspective panoramas and then compute depth maps directly from these resampled panoramas. Our panoramas sample uniformly in three dimensions: rotation angle, inverse radial distance, and vertical elevation. The use of multiperspective panoramas eliminates the limited overlap present in the original input images and, thus, problems as in conventional multibaseline stereo can be avoided. Our approach differs from stereo matching of single-perspective panoramic images taken from different locations, where the epipolar constraints are sine curves. For our multiperspective panoramas, the epipolar geometry, to the first order approximation, consists of horizontal lines. Therefore, any traditional stereo algorithm can be applied to multiperspective panoramas with little modification. In this paper, we describe two reconstruction algorithms. The first is a cylinder sweep algorithm that uses a small number of resampled multiperspective panoramas to obtain dense 3D reconstruction. The second algorithm, in contrast, uses a large number of multiperspective panoramas and takes advantage of the approximate horizontal epipolar geometry inherent in multiperspective panoramas. It comprises a novel and efficient 1D multibaseline matching technique, followed by tensor voting to extract the depth surface. Experiments show that our algorithms are capable of producing comparable high quality depth maps which can be used for applications such as view interpolation.

/pdf/stereo-reconstruction-from-multiperspective-panoramas-25dk17r98a.pdf

Stereo reconstruction from multiperspective panoramas

Omnidirectional views of an indoor environment at different locations are integrated into a global map. A single camera swiveling about the vertical axis takes consecutive images and arranges them into a panoramic representation, which provides rich information around the observation point: a precise omnidirectional view of the environment and coarse ranges to objects in it. Using the coarse map, the system autonomously plans consecutive observations at the intersections of lines connecting object points, where the directions of the imaging are estimated easily and precisely. From two panoramic views at the two planned locations, a modified binocular stereo method yields a more precise, but with direction-dependent uncertainties, local map. New observation points are selected to decrease the uncertainty, and another local map is yielded, which is then integrated into a more reliable global representation of the world with the adjacent local maps. >

Omni-directional stereo

Described here is an image sensor with a hyperboloidal mirror for vision based navigation of a mobile robot. Its name is HyperOmni Vision. This sensing system can acquire an omnidirectional view around the robot, in real-time, with use of a hyperboloidal mirror.

Omnidirectional imaging with hyperboloidal projection

We present Jump, a practical system for capturing high resolution, omnidirectional stereo (ODS) video suitable for wide scale consumption in currently available virtual reality (VR) headsets. Our system consists of a video camera built using off-the-shelf components and a fully automatic stitching pipeline capable of capturing video content in the ODS format. We have discovered and analyzed the distortions inherent to ODS when used for VR display as well as those introduced by our capture method and show that they are small enough to make this approach suitable for capturing a wide variety of scenes. Our stitching algorithm produces robust results by reducing the problem to one of pairwise image interpolation followed by compositing. We introduce novel optical flow and compositing methods designed specifically for this task. Our algorithm is temporally coherent and efficient, is currently running at scale on a distributed computing platform, and is capable of processing hours of footage each day.

Jump: virtual reality video

A novel imaging method is presented for acquiring an omni-directional view with range information using a single camera. The difficult correspondence problem is solved by tracking each feature in the image sequence in the same manner as in epipolar plane image analysis. Although the equivalent camera distance is not long, the authors can obtain reliable estimates because of the high resolution of the panoramic views. The authors expect the resolution in locating sharp edges to be very fine, up to the resolution of camera rotation, 0.005 degree. A global map making procedure is also proposed which uses omni-directional views at different locations. Omni-directional binocular stereo is used to acquire a path-centered local map with direction-dependent uncertainty. By combining multiple local maps, it is possible to build a more reliable global map. Also explored is the possibility for using the panoramic stereo for map making by a robot. >

Omni-directional stereo for making global map

Omni-directional views of environment are analyzed to build a world model of an autonomous mobile robot. A panoramic view is yielded by arranging the vertical line at the center of each image taken by a camera attached on the robot. The panoramic view is featured by the cylindrical projection of a 3D line onto a sinusoidal curve. The following three issues are discussed: (1) projection of 3-D space onto the panoramic view; (2) epipolar curves for stereo matching; and (3) optical flow. A method for finding the 3D structure of an environment is proposed. >

Analysis of omni-directional views at different location

A method is proposed for obtaining precise range information by swivelling a conventional TV camera, and the experimental verification of the method is described. To realize the method, the blurring that occurs in the image as a result of aberrations of the TV camera lens is used. The direction to the object is measured with a subpixel-level precision which depends on the angular resolution of the camera controller. The uncertainties of the object locations measured by conventional stereo methods depend on the resolution of the image sensor. It is shown that the method can be used to locate objects with a precision depending on the angular resolution of the camera controller, which is 0 to 20 times better than the conventional imaging method. >

Acquiring precise range information from camera motion

Presents a new sensing method, an omnidirectional stereo, to observe the environment around the robot. The omnidirectional stereo realized by swiveling a camera on the robot precisely yields a 2(1/2)D panoramic representation, a high resolution omnidirectional view with range estimates, of the world around the observation point. The panoramic representation is efficient as the vision of the mobile robot. An application of the panoramic representations in global map making by the mobile robot is discussed. >

http://ieeexplore.ieee.org/iel2/561/4454/00174600.pdf

M. Yamamoto

Papers

Omni-directional stereo

Omni-directional stereo for making global map

Analysis of omni-directional views at different location

Acquiring precise range information from camera motion

Panoramic representations of scenes around a point