Perception system design is a vital step in the development of an autonomous vehicle (AV). With the vast selection of available off-the-shelf schemes and seemingly endless options of sensor systems implemented in research and commercial vehicles, it can be difficult to identify the optimal system for one’s AV application. This article presents a comprehensive review of the state-of-the-art AV perception technology available today. It provides up-to-date information about the advantages, disadvantages, limits, and ideal applications of specific AV sensors; the most prevalent sensors in current research and commercial AVs; autonomous features currently on the market; and localization and mapping methods currently implemented in AV research. This information is useful for newcomers to the AV field to gain a greater understanding of the current AV solution landscape and to guide experienced researchers towards research areas requiring further development. Furthermore, this paper highlights future research areas and draws conclusions about the most effective methods for AV perception and its effect on localization and mapping. Topics discussed in the Perception and Automotive Sensors section focus on the sensors themselves, whereas topics discussed in the Localization and Mapping section focus on how the vehicle perceives where it is on the road, providing context for the use of the automotive sensors. By improving on current state-of-the-art perception systems, AVs will become more robust, reliable, safe, and accessible, ultimately providing greater efficiency, mobility, and safety benefits to the public.

Autonomous vehicle perception: The technology of today and tomorrow

Self-driving vehicles will soon be a reality, as main automotive companies have announced that they will sell their driving automation modes in the 2020s. This technology raises relevant controversies, especially with recent deadly accidents. Nevertheless, autonomous vehicles are still popular and attractive thanks to the improvement they represent to people’s way of life (safer and quicker transit, more accessible, comfortable, convenient, efficient, and environment-friendly). This paper presents a review of motion planning techniques over the last decade with a focus on highway planning. In the context of this article, motion planning denotes path generation and decision making. Highway situations limit the problem to high speed and small curvature roads, with specific driver rules, under a constrained environment framework. Lane change, obstacle avoidance, car following, and merging are the situations addressed in this paper. After a brief introduction to the context of autonomous ground vehicles, the detailed conditions for motion planning are described. The main algorithms in motion planning, their features, and their applications to highway driving are reviewed, along with current and future challenges and open issues.

/pdf/a-review-of-motion-planning-for-highway-autonomous-driving-56ou2tzah8.pdf

A Review of Motion Planning for Highway Autonomous Driving

An approach for formally verifying the safety of automated vehicles is proposed. Due to the uniqueness of each traffic situation, we verify safety online, i.e., during the operation of the vehicle. The verification is performed by predicting the set of all possible occupancies of the automated vehicle and other traffic participants on the road. In order to capture all possible future scenarios, we apply reachability analysis to consider all possible behaviors of mathematical models considering uncertain inputs (e.g., sensor noise, disturbances) and partially unknown initial states. Safety is guaranteed with respect to the modeled uncertainties and behaviors if the occupancy of the automated vehicle does not intersect that of other traffic participants for all times. The applicability of the approach is demonstrated by test drives with an automated vehicle at the Robotics Institute at Carnegie Mellon University.

/pdf/online-verification-of-automated-road-vehicles-using-5702vmzc1h.pdf

Online Verification of Automated Road Vehicles Using Reachability Analysis

The detection of bad weather conditions is crucial for meteorological centers, specially with demand for air, sea and ground traffic management. In this article, a system based on computer vision is presented which detects the presence of rain or snow. To separate the foreground from the background in image sequences, a classical Gaussian Mixture Model is used. The foreground model serves to detect rain and snow, since these are dynamic weather phenomena. Selection rules based on photometry and size are proposed in order to select the potential rain streaks. Then a Histogram of Orientations of rain or snow Streaks (HOS), estimated with the method of geometric moments, is computed, which is assumed to follow a model of Gaussian-uniform mixture. The Gaussian distribution represents the orientation of the rain or the snow whereas the uniform distribution represents the orientation of the noise. An algorithm of expectation maximization is used to separate these two distributions. Following a goodness-of-fit test, the Gaussian distribution is temporally smoothed and its amplitude allows deciding the presence of rain or snow. When the presence of rain or of snow is detected, the HOS makes it possible to detect the pixels of rain or of snow in the foreground images, and to estimate the intensity of the precipitation of rain or of snow. The applications of the method are numerous and include the detection of critical weather conditions, the observation of weather, the reliability improvement of video-surveillance systems and rain rendering.

/pdf/rain-or-snow-detection-in-image-sequences-through-use-of-a-qjfavabyoc.pdf

Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks

This paper presents the design and first test on a simulator of a vehicle trajectory-planning algorithm that adapts to traffic on a lane-structured infrastructure such as highways. The proposed algorithm is designed to run on a fail-safe embedded environment with low computational power, such as an engine control unit, to be implementable in commercial vehicles of the near future. The target platform has a clock frequency of less than 150 MHz, 150 kB RAM of memory, and a 3-MB program memory. The trajectory planning is performed by a two-step algorithm. The first step defines the feasible maneuvers with respect to the environment, aiming at minimizing the risk of a collision. The output of this step is a target group of maneuvers in the longitudinal direction (accelerating or decelerating), in the lateral direction (changing lanes), and in the combination of both directions. The second step is a more detailed evaluation of several possible trajectories within these maneuvers. The trajectories are optimized to additional performance indicators such as travel time, traffic rules, consumption, and comfort. The output of this module is a trajectory in the vehicle frame that represents the recommended vehicle state (position, heading, speed, and acceleration) for the following seconds.

Maneuver-Based Trajectory Planning for Highly Autonomous Vehicles on Real Road With Traffic and Driver Interaction

https://hal.archives-ouvertes.fr/hal-00506570/document

Driver steering assistance for lane departure avoidance

This paper presents the design and the practical implementation of vehicle steering assistance that helps the driver avoid unintended lane departure. A switching strategy is built to govern the driver-assistance interaction, and the resulting hybrid system is formalized as an input/output (I/O) hybrid automaton. Composite Lyapunov functions, polyhedral-like invariant sets, and linear matrix inequality (LMI) methods constitute the heart of the approach used to design the lane-departure avoidance (LDA) system. The practical implementation of this steering assistance in a prototype vehicle confirms the effectiveness of this approach.

Driver Steering Assistance for Lane-Departure Avoidance Based on Hybrid Automata and Composite Lyapunov Function

This paper discusses driving system design based on traffic rules. This allows fully automated driving in an environment with human drivers, without necessarily changing equipment on other vehicles or infrastructure. It also facilitates cooperation between the driving system and the host driver during highly automated driving. The concept, referred to as legal safety, is illustrated for highly automated driving on highways with distance keeping, intelligent speed adaptation, and lane-changing functionalities. Requirements by legal safety on perception and control components are discussed. This paper presents the actual design of a legal safety decision component, which predicts object trajectories and calculates optimal subject trajectories. System implementation on automotive electronic control units and results on vehicle and simulator are discussed.

/pdf/highly-automated-driving-on-highways-based-on-legal-safety-10e4zdnby4.pdf

Highly Automated Driving on Highways Based on Legal Safety

This paper presents an integrated system to prevent driver from taking a curve with a dangerous speed. The integrated system underlines three functions developed as follows : the first function consists in knowing the geometry characteristics of the coming road; the second function has to compute a safe speed according to the road section, the vehicle and the driver; the last function must impose the speed control of the vehicle in order to achieve a safe speed in the case that the driver does not achieve it.

Speed Limitation Based on an Advanced Curve Warning System

In this work, we have proposed and implemented in our prototype vehicle a complete lateral control system that works with a good performance even for strong curvatures. The proposed system has important new features with respect to the known systems. While a lookahead distance linear with the speed is often proposed, we have proposed here a quadratic lookahead distance of the speed, that improved considerably the tracking of the lane in curves. A strategy to optimize the use of the information coming from the lane detection module is also proposed. The logics is, if we do not have optimal lane detection exactly in the desired lookahead distance, but in the proximities of this distance we can still control the vehicle with high accuracy. Finally, we have proposed a visual interface that alerts the driver in the case of a degradation in the lane detection. The driver can deactivate the lateral control whenever he wants by simply counteracting the system and an automatic reactivation of the system is provided under strict conditions on the vehicle positioning and on the quality of the lane detection

Benoit Lusetti

Papers

Driver steering assistance for lane departure avoidance

Driver Steering Assistance for Lane-Departure Avoidance Based on Hybrid Automata and Composite Lyapunov Function

Highly Automated Driving on Highways Based on Legal Safety

Speed Limitation Based on an Advanced Curve Warning System

A new robust control system with optimized use of the lane detection data for vehicle full lateral control under strong curvatures