Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

This paper presents a method of estimating camera pose in an unknown scene. While this has previously been attempted by adapting SLAM algorithms developed for robotic exploration, we propose a system specifically designed to track a hand-held camera in a small AR workspace. We propose to split tracking and mapping into two separate tasks, processed in parallel threads on a dual-core computer: one thread deals with the task of robustly tracking erratic hand-held motion, while the other produces a 3D map of point features from previously observed video frames. This allows the use of computationally expensive batch optimisation techniques not usually associated with real-time operation: The result is a system that produces detailed maps with thousands of landmarks which can be tracked at frame-rate, with an accuracy and robustness rivalling that of state-of-the-art model-based systems.

/pdf/parallel-tracking-and-mapping-for-small-ar-workspaces-4qm7cjaut2.pdf

Parallel Tracking and Mapping for Small AR Workspaces

We present a real-time algorithm which can recover the 3D trajectory of a monocular camera, moving rapidly through a previously unknown scene. Our system, which we dub MonoSLAM, is the first successful application of the SLAM methodology from mobile robotics to the "pure vision" domain of a single uncontrolled camera, achieving real time but drift-free performance inaccessible to structure from motion approaches. The core of the approach is the online creation of a sparse but persistent map of natural landmarks within a probabilistic framework. Our key novel contributions include an active approach to mapping and measurement, the use of a general motion model for smooth camera movement, and solutions for monocular feature initialization and feature orientation estimation. Together, these add up to an extremely efficient and robust algorithm which runs at 30 Hz with standard PC and camera hardware. This work extends the range of robotic systems in which SLAM can be usefully applied, but also opens up new areas. We present applications of MonoSLAM to real-time 3D localization and mapping for a high-performance full-size humanoid robot and live augmented reality with a hand-held camera

/pdf/monoslam-real-time-single-camera-slam-ydmj8dc1vj.pdf

MonoSLAM: Real-Time Single Camera SLAM

We propose a real-time method for odometry and mapping using range measurements from a 2-axis lidar moving in 6-DOF. The problem is hard because the range measurements are received at different times, and errors in motion estimation can cause mis-registration of the resulting point cloud. To date, coherent 3D maps can be built by off-line batch methods, often using loop closure to correct for drift over time. Our method achieves both low-drift and low-computational complexity without the need for high accuracy ranging or inertial measurements. The key idea in obtaining this level of performance is the division of the complex problem of simultaneous localization and mapping, which seeks to optimize a large number of variables simultaneously, by two algorithms. One algorithm performs odometry at a high frequency but low fidelity to estimate velocity of the lidar. Another algorithm runs at a frequency of an order of magnitude lower for fine matching and registration of the point cloud. Combination of the two algorithms allows the method to map in real-time. The method has been evaluated by a large set of experiments as well as on the KITTI odometry benchmark. The results indicate that the method can achieve accuracy at the level of state of the art offline batch methods.

/pdf/loam-lidar-odometry-and-mapping-in-real-time-43agyngf9i.pdf

LOAM: Lidar Odometry and Mapping in Real-time

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications, and witnessing a steady transition of this technology to industry. We survey the current state of SLAM. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

/pdf/past-present-and-future-of-simultaneous-localization-and-dhljw0akg5.pdf

Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age

A method and apparatus for designing an iris biometrics system that operates in minimally constrained settings. The image acquisition system has fewer constraints on subjects than traditional methods by extending standoff distance and capture volume. The method receives design parameters and provides derived quantities that are useful in designing an image acquisition system having a specific set of performance requirements. Exemplary scenarios of minimally constrained settings are provided, such as a high volume security checkpoint, an office, an aircraft boarding bridge, a wide corridor, and an automobile.

Method and apparatus for designing iris biometric systems for use in minimally

Systems, methods, and computer readable media to track and estimate the accuracy of a visual inertial odometry (VIO) system. Various embodiments are able to receive one or more VIO feature measurements associated with a set of image frames from a VIO system and generate a plurality of feature models to estimate health values for the VIO system. The various embodiments determine a plurality of feature health values with the feature models based on the VIO feature measurements and compare the feature health values with ground truth health scores associated with the set of image frames to determine one or more errors. The feature model parameters are updated based on the comparison with the feature health values with ground truth health scores.

Visual Inertial Odometry Health Fitting

This report presents two efficient solutions to the two-view, relative pose problem from three image point correspondences and one common reference direction. This three-plus-one problem can be used either as a substitute for the classic five-point algorithm using a vanishing point for the reference direction, or to make use of an inertial measurement unit commonly available on robots and mobile devices, where the gravity vector becomes the reference direction. We provide a simple closed-form solution and a solution based on techniques from algebraic geometry and investigate numerical and computational advantages of each approach. In a set of real experiments, we demonstrate the power of our approach by comparing it to the fivepoint method in a hypothesize-and-test visual odometry setting. Comments University of Pennsylvania Department of Computer and Information Science Technical Report No. MSCIS-11-15. This technical report is available at ScholarlyCommons: http://repository.upenn.edu/cis_reports/960 GRASP LABORATORY, UNIVERSITY OF PENNSYLVANIA, MS-CIS-11-15 1 Structure from Motion with Directional Correspondence for Visual Odometry GRASP Laboratory Technical Report MS-CIS-11-15 Oleg Naroditsky, Xun S. Zhou, Jean Gallier, Stergios I. Roumeliotis, and Kostas Daniilidis

/pdf/structure-from-motion-with-directional-correspondence-for-4vd5mz8c9w.pdf

Structure From Motion With Directional Correspondence for Visual Odometry

Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

Mobile device navigation system

Mixed reality training systems using Head Mounted Displays (HMDs) require very high precision knowledge of the 3D
location and 3D orientation of the user's head. This is required by the system to know where to insert the synthetic
actors and objects in the HMD. The inserted objects must appear stable and not jitter or drift. Moreover latency of less
than 5 milliseconds for pose estimation is required for lag-free see-through HMD operation. We describe how to achieve
this performance using a multi-camera based visual navigation system mounted on the HMD. A Kalman filter is used to
integrate high rate estimates from an IMU with a visual odometry system and to predict head motion. Landmark
matching and GPS when available are used to correct any drifts.

Oleg Naroditsky

Papers

Method and apparatus for designing iris biometric systems for use in minimally

Visual Inertial Odometry Health Fitting

Structure From Motion With Directional Correspondence for Visual Odometry

Mobile device navigation system

Infrastructure free 6 DOF location and pose estimation for mixed reality systems