This paper presents ORB-SLAM, a feature-based monocular simultaneous localization and mapping (SLAM) system that operates in real time, in small and large indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

/pdf/orb-slam-a-versatile-and-accurate-monocular-slam-system-t5f1c0a74e.pdf

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

/pdf/orb-slam-a-versatile-and-accurate-monocular-slam-system-rnvdngj54a.pdf

ORB-SLAM: a Versatile and Accurate Monocular SLAM System

We propose a semi-direct monocular visual odometry algorithm that is precise, robust, and faster than current state-of-the-art methods. The semi-direct approach eliminates the need of costly feature extraction and robust matching techniques for motion estimation. Our algorithm operates directly on pixel intensities, which results in subpixel precision at high frame-rates. A probabilistic mapping method that explicitly models outlier measurements is used to estimate 3D points, which results in fewer outliers and more reliable points. Precise and high frame-rate motion estimation brings increased robustness in scenes of little, repetitive, and high-frequency texture. The algorithm is applied to micro-aerial-vehicle state-estimation in GPS-denied environments and runs at 55 frames per second on the onboard embedded computer and at more than 300 frames per second on a consumer laptop. We call our approach SVO (Semi-direct Visual Odometry) and release our implementation as open-source software.

/pdf/svo-fast-semi-direct-monocular-visual-odometry-46fauoqkah.pdf

SVO: Fast semi-direct monocular visual odometry

With the objective to improve road safety, the automotive industry is moving toward more “intelligent” vehicles. One of the major challenges is to detect dangerous situations and react accordingly in order to avoid or mitigate accidents. This requires predicting the likely evolution of the current traffic situation, and assessing how dangerous that future situation might be. This paper is a survey of existing methods for motion prediction and risk assessment for intelligent vehicles. The proposed classification is based on the semantics used to define motion and risk. We point out the tradeoff between model completeness and real-time constraints, and the fact that the choice of a risk assessment method is influenced by the selected motion model.

/pdf/a-survey-on-motion-prediction-and-risk-assessment-for-39io9df2mm.pdf

A survey on motion prediction and risk assessment for intelligent vehicles

Direct methods for visual odometry (VO) have gained popularity for their capability to exploit information from all intensity gradients in the image. However, low computational speed as well as missing guarantees for optimality and consistency are limiting factors of direct methods, in which established feature-based methods succeed instead. Based on these considerations, we propose a semidirect VO (SVO) that uses direct methods to track and triangulate pixels that are characterized by high image gradients, but relies on proven feature-based methods for joint optimization of structure and motion. Together with a robust probabilistic depth estimation algorithm, this enables us to efficiently track pixels lying on weak corners and edges in environments with little or high-frequency texture. We further demonstrate that the algorithm can easily be extended to multiple cameras, to track edges, to include motion priors, and to enable the use of very large field of view cameras, such as fisheye and catadioptric ones. Experimental evaluation on benchmark datasets shows that the algorithm is significantly faster than the state of the art while achieving highly competitive accuracy.

https://www.zora.uzh.ch/id/eprint/127902/1/TRO16_Forster-SVO.pdf

SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems

Autonomous vehicles rely on their perception systems to acquire information about their immediate surroundings. It is necessary to detect the presence of other vehicles, pedestrians, and other relevant entities. Safety concerns and the need for accurate estimations have led to the introduction of lidar systems to complement camera- or radar-based perception systems. This article presents a review of state-of-the-art automotive lidar technologies and the perception algorithms used with those technologies. Lidar systems are introduced first by analyzing such a system?s main components, from laser transmitter to beamscanning mechanism. The advantages/disadvantages and the current status of various solutions are introduced and compared. Then, the specific perception pipeline for lidar data processing is detailed from an autonomous vehicle perspective. The model-driven approaches and emerging deep learning (DL) solutions are reviewed. Finally, we provide an overview of the limitations, challenges, and trends for automotive lidars and perception systems.

Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems

A control of autonomous vehicles, and a method for controlling at least one autonomous ground vehicle configured to adopt two operating modes, including a manual mode in which the driving depends on driving instructions from the driver of the vehicle, and an autonomous mode, in which the driving depends on data received from sensors configured to provide information on surroundings of the vehicle, and on data received by a navigation system, the method including: authorizing the autonomous mode only when the vehicle is within an area determined by defined boundaries; receiving reservation data from a user interface; generating a set message to be transmitted to the vehicle in accordance with the received reservation data; and transmitting the set message to the vehicle such that the vehicle autonomously performs a movement corresponding to the transmitted set message.

Control of the autonomous mode of bimodal vehicles

Autonomous vehicles rely on their perception systems to acquire information about their immediate surroundings. It is necessary to detect the presence of other vehicles, pedestrians and other relevant entities. Safety concerns and the need for accurate estimations have led to the introduction of Light Detection and Ranging (LiDAR) systems in complement to the camera or radar-based perception systems. This article presents a review of state-of-the-art automotive LiDAR technologies and the perception algorithms used with those technologies. LiDAR systems are introduced first by analyzing the main components, from laser transmitter to its beam scanning mechanism. Advantages/disadvantages and the current status of various solutions are introduced and compared. Then, the specific perception pipeline for LiDAR data processing, from an autonomous vehicle perspective is detailed. The model-driven approaches and the emerging deep learning solutions are reviewed. Finally, we provide an overview of the limitations, challenges and trends for automotive LiDARs and perception systems.

Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems

Safety applications at road intersections require algorithms that can estimate the manoeuvre intention of all the drivers in the scene. In this paper, the use of contextual information extracted from a digital map of the road network is explored. We propose a Bayesian network which combines probabilistically uncertain observations on the vehicle's behaviour and information about the geometrical and topological characteristics of the road intersection in order to infer a driver's manoeuvre intention. The approach is evaluated on trajectories recorded from real traffic, including complex scenarios where the behaviour of the vehicle is inconsistent. We define an evaluation method that accounts for the impossibility to make reliable predictions in some situations, and show that the system is able to reliably combine vehicle state information and map information to infer a driver's intended manoeuvre.

/pdf/exploiting-map-information-for-driver-intention-estimation-1s7wdvc7z7.pdf

Exploiting map information for driver intention estimation at road intersections

Estimating the pose in real-time is a primary function for intelligent vehicle navigation. Whilst different solutions exist, most of them rely on the use of high-end sensors. This paper proposes a solution that exploits an automotive type L1-GPS receiver, features extracted by low-cost perception sensors and vehicle proprioceptive information. A key idea is to use the lane detection function of a video camera to retrieve accurate lateral and orientation information with respect to road lane markings. To this end, lane markings are mobile-mapped by the vehicle itself during a first stage by using an accurate localizer. Then, the resulting map allows for the exploitation of camera-detected features for autonomous real-time localization. The results are then combined with GPS estimates and dead-reckoning sensors in order to provide localization information with high availability. As L1-GPS errors can be large and are time correlated, we study in the paper several GPS error models that are experimentally tested with shaping filters. The approach demonstrates that the use of low-cost sensors with adequate data-fusion algorithms should lead to computer-controlled guidance functions in complex road networks.

/pdf/mapping-and-localization-using-gps-lane-markings-and-3rxv5lrdz9.pdf

Javier Ibanez-Guzman

Papers

Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems

Control of the autonomous mode of bimodal vehicles

Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems

Exploiting map information for driver intention estimation at road intersections

Mapping and localization using GPS, lane markings and proprioceptive sensors