We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

This paper presents ORB-SLAM, a feature-based monocular simultaneous localization and mapping (SLAM) system that operates in real time, in small and large indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

/pdf/orb-slam-a-versatile-and-accurate-monocular-slam-system-t5f1c0a74e.pdf

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

/pdf/orb-slam-a-versatile-and-accurate-monocular-slam-system-rnvdngj54a.pdf

ORB-SLAM: a Versatile and Accurate Monocular SLAM System

We present ORB-SLAM2, a complete simultaneous localization and mapping (SLAM) system for monocular, stereo and RGB-D cameras, including map reuse, loop closing, and relocalization capabilities. The system works in real time on standard central processing units in a wide variety of environments from small hand-held indoors sequences, to drones flying in industrial environments and cars driving around a city. Our back-end, based on bundle adjustment with monocular and stereo observations, allows for accurate trajectory estimation with metric scale. Our system includes a lightweight localization mode that leverages visual odometry tracks for unmapped regions and matches with map points that allow for zero-drift localization. The evaluation on 29 popular public sequences shows that our method achieves state-of-the-art accuracy, being in most cases the most accurate SLAM solution. We publish the source code, not only for the benefit of the SLAM community, but with the aim of being an out-of-the-box SLAM solution for researchers in other fields.

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras

We propose a direct (feature-less) monocular SLAM algorithm which, in contrast to current state-of-the-art regarding direct methods, allows to build large-scale, consistent maps of the environment Along with highly accurate pose estimation based on direct image alignment, the 3D environment is reconstructed in real-time as pose-graph of keyframes with associated semi-dense depth maps These are obtained by filtering over a large number of pixelwise small-baseline stereo comparisons The explicitly scale-drift aware formulation allows the approach to operate on challenging sequences including large variations in scene scale Major enablers are two key novelties: (1) a novel direct tracking method which operates on \(\mathfrak{sim}(3)\), thereby explicitly detecting scale-drift, and (2) an elegant probabilistic solution to include the effect of noisy depth values into tracking The resulting direct monocular SLAM system runs in real-time on a CPU

/pdf/lsd-slam-large-scale-direct-monocular-slam-4eqdolimo8.pdf

LSD-SLAM: Large-Scale Direct Monocular SLAM

Localization approaches typically rely on an already available map to identify the position of the sensor in the environment. Such maps are usually built beforehand and often require the user to record data from the same sensor used for localization. In this paper, we relax this assumption and present a localization approach based on architectural floor plans. In general, floor plans are readily available for most man-made buildings but only represent basic architectural structures. The incomplete knowledge leads to ambiguous pose estimates. To solve this problem, we present W-RGB-D, a new method for indoor global localization based on WiFi and an RGB-D camera. We introduce a sensor model for RGB-D cameras that is suitable to be used with abstract floor plans. To resolve ambiguities during global localization, we estimate a coarse initial distribution about the sensor position using the WiFi signal strength. We evaluate our W-RGB-D localization method in indoor environments and compare its performance with RGBD-based Monte Carlo localization. Our results demonstrate that the use of WiFi information as proposed with our approach improves the localization in terms of convergence speed and quality of the solution.

/pdf/w-rgb-d-floor-plan-based-indoor-global-localization-using-a-2xcfvo2r6b.pdf

W-RGB-D: Floor-plan-based indoor global localization using a depth camera and WiFi

Opening doors is a fundamental skill for mobile robots operating in human environments. In this paper we present an approach to learn a dynamic model of a door from sensor observations and utilize it for effectively swinging the door open to a desired angle. The learned model enables the realization of dynamic door-opening strategies and reduces the complexity of the door opening task. For example, the robot does not need to maintain a grasp of the handle, which would form a closed kinematic chain. Accordingly, it reduces the degrees of freedom required of the manipulator and facilitates motion planning. Additionally, execution is faster, because the robot merely needs to push the door long enough to achieve the right combination of position and speed such that the door stops at the desired state. Our approach applies Gaussian process regression to learn the deceleration of the door with respect to position and velocity of the door. This model of the dynamics can be easily learned from observing a human teacher or by interactive experimentation.

/pdf/learning-the-dynamics-of-doors-for-robotic-manipulation-1uog8yh004.pdf

Learning the dynamics of doors for robotic manipulation

Mobile robots rely on the ability to sense the geometry of their local environment in order to avoid obstacles or to explore the surroundings. For this task, dedicated proximity sensors such as laser range finders or sonars are typically employed. Cameras are a cheap and lightweight alternative to such sensors, but do not directly offer proximity information. In this paper, we present a novel approach to learning the relationship between range measurements and visual features extracted from a single monocular camera image. As the learning engine, we apply Gaussian processes, a non-parametric learning technique that not only yields the most likely range prediction corresponding to a certain visual input but also the predictive uncertainty. This information, in turn, can be utilized in an extended grid-based mapping scheme to more accurately update the map. In practical experiments carried out in different environments with a mobile robot equipped with an omnidirectional camera system, we demonstrate that our system is able to produce proximity estimates with an accuracy comparable to that of dedicated sensors such as sonars or infrared range finders.

/pdf/monocular-range-sensing-a-non-parametric-learning-approach-3dae4ecyni.pdf

Monocular range sensing: A non-parametric learning approach

A nonparametric learning approach to range sensing from omnidirectional vision

The typically restricted field of view of visual sensors often imposes limitations on the performance of localization and simultaneous localization and mapping (SLAM) approaches. In this paper, we propose and analyze the combination of an RGB-D camera with two planar mirrors to split the field of view such that it covers both front and rear view of a mobile robot. We describe how to estimate the extrinsic calibration parameters of the modified sensor using a standard parametrization and a reduced one that exploits the properties of the setup. Our experimental evaluation on real-world data demonstrates the robustness of the calibration procedure. Additionally, we show that our proposed sensor modification substantially improves the accuracy and the robustness in a simultaneous localization and mapping task. I . I N T R O D U C T I O N Performing localization or SLAM with consumer grade RGB-D cameras such as the Microsoft Kinect has been a topic of intensive research in recent years [1], [2], [3], [4]. Many approaches are extensions of algorithms commonly used with stereo cameras or lidars (laser range scanners), which typically have a horizontal field of view of 180◦ to 270◦ and provide accurate depth measurements in a range between a few centimeters and several dozen meters. For RGB-D sensors the distance in which the sensor provides depth measurements is generally limited to a range of about 0.6 m to 8 m with a restricted field of view of 43◦ vertically by 57◦ horizontally. In contrast to lidars, for which one can usually assume some meaningful geometric structure to be in the sensor’s field of view (e.g., more than half of the room for a field of view of 180◦), in the context of RGB-D cameras we often need to deal with visually and geometrically ambiguous structure, e.g., when perceiving only a flat part of a wall or one corner of a room. Accordingly, an extension of the perception capabilities of RGB-D sensors would be highly beneficial. Obviously, an improvement can be achieved by the use of multiple sensors. This, however, comes with the inherent increase of computational and financial costs. In this paper, we propose a novel catadioptric setup for Kinect-style RGB-D cameras that is of extremely low cost (less than 20 EUR), and requires neither significant computational resources nor higher power consumption and substantially relaxes the above-mentioned limitations. Catadioptric sensors have been extensively used in the robotics and computer vision community for localization, visual odometry and SLAM [5]. Various shapes of mirrors have been used to increase the field of view, including All authors are with the Department of Computer Science at the University of Freiburg, Germany. The authors thank Janosch Dobler, Robert Gronsfeld and Pratik Agarwal for their support. Fig. 1. A resulting map and the trajectory estimate from our evaluation experiment, where we show that the proposed catadioptric sensor substantially improves the accuracy in a robot SLAM task. Note that only the RGB-D data was used to generate the map. parabolic, hyperbolic and elliptic. Also mirrors have been used to capture stereo images with a single camera [6]. To the best of our knowledge, however, we are the first to propose to combine an RGB-D camera with mirrors. In this work, we use two planar mirrors to split the field of view of the camera, such that the robot has effectively two fields of view of about 20◦ vertically by 57◦ horizontally in opposite directions. This setup is particularly beneficial for robots moving in the plane, as the sensor data gained by the horizontally extended perception yields much more information relevant to the planar motion than the sacrificed perception in the vertical direction. However, the proposed sensor extension can be also beneficial for robots moving with more degrees of freedom as, e.g., the ambiguity between translation and rotation is alleviated. Figure 1 depicts a Pioneer robot with the proposed catadioptric sensor and a map created only from the obtained RGB-D data. To readily offer the developed device to the research community, we publish the used CAD model and our implementation of the calibration software1. I I . R E L AT E D W O R K

http://ais.informatik.uni-freiburg.de/publications/papers/endres14iros.pdf

Felix Endres

Papers

W-RGB-D: Floor-plan-based indoor global localization using a depth camera and WiFi

Learning the dynamics of doors for robotic manipulation

Monocular range sensing: A non-parametric learning approach

A nonparametric learning approach to range sensing from omnidirectional vision

A catadioptric extension for RGB-D cameras