Forecasting the motion of surrounding vehicles is a critical ability for an autonomous vehicle deployed in complex traffic. Motion of all vehicles in a scene is governed by the traffic context, i.e., the motion and relative spatial configuration of neighboring vehicles. In this paper we propose an LSTM encoder-decoder model that uses convolutional social pooling as an improvement to social pooling layers for robustly learning interdependencies in vehicle motion. Additionally, our model outputs a multi-modal predictive distribution over future trajectories based on maneuver classes. We evaluate our model using the publicly available NGSIM US-101 and I-80 datasets. Our results show improvement over the state of the art in terms of RMS values of prediction error and negative log-likelihoods of true future trajectories under the model's predictive distribution. We also present a qualitative analysis of the model's predicted distributions for various traffic scenarios.

/pdf/convolutional-social-pooling-for-vehicle-trajectory-4fzlp3b04v.pdf

Convolutional Social Pooling for Vehicle Trajectory Prediction

In order to drive safely and efficiently on public roads, autonomous vehicles will have to understand the intentions of surrounding vehicles, and adapt their own behavior accordingly. If experienced human drivers are generally good at inferring other vehicles' motion up to a few seconds in the future, most current Advanced Driving Assistance Systems (ADAS) are unable to perform such medium-term forecasts, and are usually limited to high-likelihood situations such as emergency braking. In this article, we present a first step towards consistent trajectory prediction by introducing a long short-term memory (LSTM) neural network, which is capable of accurately predicting future longitudinal and lateral trajectories for vehicles on highway. Unlike previous work focusing on a low number of trajectories collected from a few drivers, our network was trained and validated on the NGSIM US-101 dataset, which contains a total of 800 hours of recorded trajectories in various traffic densities, representing more than 6000 individual drivers.

https://hal-mines-paristech.archives-ouvertes.fr/hal-01691832/document

An LSTM network for highway trajectory prediction

To safely and efficiently navigate through complex traffic scenarios, autonomous vehicles need to have the ability to predict the future motion of surrounding vehicles. Multiple interacting agents, the multi-modal nature of driver behavior, and the inherent uncertainty involved in the task make motion prediction of surrounding vehicles a challenging problem. In this paper, we present an LSTM model for interaction aware motion prediction of surrounding vehicles on freeways. Our model assigns confidence values to maneuvers being performed by vehicles and outputs a multi-modal distribution over future motion based on them. We compare our approach with the prior art for vehicle motion prediction on the publicly available NGSIM US-101 and I-80 datasets. Our results show an improvement in terms of RMS values of prediction error. We also present an ablative analysis of the components of our proposed model and analyze the predictions made by the model in complex traffic scenarios.

Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs

In this paper, we propose a neural motion planner for learning to drive autonomously in complex urban scenarios that include traffic-light handling, yielding, and interactions with multiple road-users. Towards this goal, we design a holistic model that takes as input raw LIDAR data and a HD map and produces interpretable intermediate representations in the form of 3D detections and their future trajectories, as well as a cost volume defining the goodness of each position that the self-driving car can take within the planning horizon. We then sample a set of diverse physically possible trajectories and choose the one with the minimum learned cost. Importantly, our cost volume is able to naturally capture multi-modality. We demonstrate the effectiveness of our approach in real-world driving data captured in several cities in North America. Our experiments show that the learned cost volume can generate safer planning than all the baselines.

/pdf/end-to-end-interpretable-neural-motion-planner-1fccr9x1pe.pdf

End-To-End Interpretable Neural Motion Planner

Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate maneuvers 3.5 seconds before they occur with over 80% F1-score in real-time.

/pdf/car-that-knows-before-you-do-anticipating-maneuvers-via-l7upniaxlh.pdf

Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models

Risk estimation for the current traffic situation is crucial for safe autonomous driving systems. One part of the uncertainty in risk estimation is the behavior of the surrounding traffic participants. In this paper we focus on highway scenarios, where possible behaviors consist of a change in acceleration and lane change maneuvers. We present a novel approach for the recognition of lane change intentions of traffic participants. Our novel approach is an extension of the Naive Bayesian approach and results in a generative model. It builds on the relations to the directly surrounding vehicles and to the static traffic environment. We obtain the conditional probabilities of all relevant features using Gaussian mixtures with a flexible number of components. We systematically reduce the number of features by selecting the most powerful ones. Furthermore we investigate the predictive power of each feature with respect to the time before a lane change event. In a large scale experiment on real world data with over 160.781 samples collected on a test drive of 1100km we trained and validated our intention prediction model and achieved a significant improvement in the recognition performance of lane change intentions compared to current state of the art methods.

A lane change detection approach using feature ranking with maximized predictive power

Understanding traffic situations in dynamic traffic environments is an essential requirement for autonomous driving. The prediction of the current traffic scene into the future is one of the main problems in this context. In this publication we focus on highway scenarios, where the maneuver space for traffic participants is limited to a small number of possible behavior classes. Even though there are many publications in the field of maneuver prediction, most of them set the focus on the classification problem, whether a certain maneuver is executed or not. We extend approaches which solve the classification problem of lane-change behavior by introducing the novel aspect of estimating a continuous distribution of possible trajectories. Our novel approach uses the probabilities which are assigned by a Random Decision Forest to each of the maneuvers lane following, lane change left and lane change right. Using measured data of a vehicle and the knowledge of the typical lateral movement of vehicles over time taken from realworlddata, we derive a Gaussian Mixture Regression method. For the final result we combine the predicted probability density functions of the regression method and the computed maneuver probabilities using a Mixture of Experts approach. In a large scale experiment on real world data collected on multiple test drives we trained and validated our prediction model and show the gained high prediction accuracy of the proposed method.

When will it change the lane? A probabilistic regression approach for rarely occurring events

The vision of autonomous driving is piecewise becoming reality. Still the problem of executing the driving task in a safe and comfortable way in all possible environments, for instance highway, city or rural road scenarios is a challenging task. In this paper we present a novel approach to planning trajectories for autonomous vehicles. Hereby we focus on the problem of planning a trajectory, given a specific behavior option, e.g. merging into a specific gap at a highway entrance or a roundabout. Therefore we explicitly take arbitrary road geometry and prediction information of other traffic participants into account. We extend former contributions in this field by providing a flexible problem description and a trajectory planner without specialization to distinct classes of maneuvers beforehand. Using a carefully chosen representation of the dynamic free space, the method is capable of considering multiple lanes including the predicted dynamics of other traffic participants, while being real-time capable at the same time. The combination of those properties in one general planning method represents the novelty of the proposed method. We demonstrate the capability of our algorithm to plan safe trajectories in simulation and in real traffic in real-time.

Wiggling through complex traffic: Planning trajectories constrained by predictions

By observing their environment as well as other traffic participants, humans are enabled to drive road vehicles safely. Vehicle passengers, however, perceive a notable difference between non-experienced and experienced drivers. In particular, they may get the impression that the latter ones anticipate what will happen in the next few moments and consider these foresights in their driving behavior. To make the driving style of automated vehicles comparable to the one of human drivers with respect to comfort and perceived safety, the aforementioned anticipation skills need to become a built-in feature of self-driving vehicles. This article provides a systematic comparison of methods and strategies to generate this intention for self-driving cars using machine learning techniques. To implement and test these algorithms we use a large data set collected over more than 30 000 km of highway driving and containing approximately 40 000 real-world driving situations. We further show that it is possible to classify driving maneuvers upcoming within the next 5 s with an Area Under the ROC Curve ( AUC ) above 0.92 for all defined maneuver classes. This enables us to predict the lateral position with a prediction horizon of 5 s with a median lateral error of less than 0.21 m.

Teaching Vehicles to Anticipate: A Systematic Study on Probabilistic Behavior Prediction Using Large Data Sets

Risk estimation for the current traffic situation is crucial for safe autonomous driving systems. The computation of risk estimates however is always uncertain, especially if the behavior of traffic participants has to be taken into account. Besides risk estimation, knowledge about the future behavior of other traffic participants can be used for Adaptive Cruise Control Applications, helping to choose a driving strategy with more foresight, which is not only desirable under comfort aspects, but can also be used to reduce fuel consumption. In this publication the authors focus on highway scenarios, where possible behaviors consist of changes in acceleration and lane-change maneuvers. Based on this insight, the authors present a novel approach for the prediction of future positions in highway scenarios.

Julian Schlechtriemen

Papers

A lane change detection approach using feature ranking with maximized predictive power

When will it change the lane? A probabilistic regression approach for rarely occurring events

Wiggling through complex traffic: Planning trajectories constrained by predictions

Teaching Vehicles to Anticipate: A Systematic Study on Probabilistic Behavior Prediction Using Large Data Sets

A probabilistic long term prediction approach for highway scenarios