Automated driving systems (ADSs) promise a safe, comfortable and efficient driving experience. However, fatalities involving vehicles equipped with ADSs are on the rise. The full potential of ADSs cannot be realized unless the robustness of state-of-the-art is improved further. This paper discusses unsolved problems and surveys the technical aspect of automated driving. Studies regarding present challenges, high-level system architectures, emerging methodologies and core functions including localization, mapping, perception, planning, and human machine interfaces, were thoroughly reviewed. Furthermore, many state-of-the-art algorithms were implemented and compared on our own platform in a real-world driving setting. The paper concludes with an overview of available datasets and tools for ADS development.

A Survey of Autonomous Driving: Common Practices and Emerging Technologies

We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal filtering of image-based bounding box estimation. This approach has the important advantage of mitigating tracking drift (i.e. drifting away from the target object), which is a common symptom of model-free trackers and is detrimental to sustained convoying in practice. To illustrate our solution, we collected extensive footage of an underwater robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several convolutional neural networks, both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot's independent motion.

Underwater multi-robot convoying using visual tracking by detection

Underwater Multi-Robot Convoying using Visual Tracking by Detection

In this paper, we introduce a new geometric 3D feature combined with a clustering approach. Besides 3D data provided by a LiDAR point cloud, reflectivity information is used to further enhance the descriptivity of the feature. The proposed feature can be extracted and compared in real-time. Similar parts of an object, such as features belonging to an automobile headlight, are automatically clustered in an object model without explicit specification. Additionally, we provide a method for autonomous vehicles to automatically learn the shapes of observed moving objects and use them for real-time classification. The resulting object models consisting of the extracted feature clusters are interpretable by humans.

A new geometric 3D LiDAR feature for model creation and classification of moving objects

We describe a variable-velocity trajectory plan- ning algorithm for navigating car-like robots through unknown, unstructured environments along a series of possibly corrupted GPS waypoints. The trajectories are guaranteed to be kine- matically feasible, i.e., they respect the robot's acceleration and deceleration capabilities as well as its maximum steering angle and steering rate. Their costs are computed using LiDAR and camera data and depend on factors such as proximity to obstacles, curvature, changes of curvature, and slope. In a second step, velocities for the least-cost trajectory are adjusted based on the dynamics of the vehicle. When the robot is faced with an obstacle on its trajectory, the planner is restarted to compute an alternative trajectory. Our algorithm is robust against GPS error and waypoints placed in obstacle-filled areas. It was successfully used at euRathlon 2013 1 , where our autonomous vehicle MuCAR-3 took first place in the "Autonomous Navigation" scenario.

Trajectory planning for car-like robots in unknown, unstructured environments

Previous publications of our institute describe a robust vehicle tracking system for daylight conditions. This paper presents an improved vehicle tracking system which is able to detect and track the convoy leader also by twilight and night. The primary sensor equipment consists of a daytime camera, a LiDAR and an inertial navigation system. An expansion with a thermal and a lowlight camera was necessary to be robust against any illumination conditions. The system is capable of estimating the relative 3D position and orientation, the velocity and the steering angle of a convoy leader precisely in real-time. This makes it possible to follow the convoy leader's track. Another novelty is coupling a Kalman filter with a particle filter for higher stability and accuracy in vehicle tracking. The tracking system shows excellent functionality while driving more than 50km fully autonomously in urban- and unstructured environments at night.

/pdf/autonomous-convoy-driving-by-night-the-vehicle-tracking-21sep7v3db.pdf

Autonomous convoy driving by night: The vehicle tracking system

This paper presents a robust method for vehicle tracking with a monocular camera. A previously published model-based tracking method uses a particle filter which needs an initial vehicle hypothesis both at system start and in case of a tracking loss. We present a template-based solution using different features to estimate a 3D vehicle pose roughly but fast. Combining model- and template-based object tracking keeps the advantages of each algorithm: Precise estimation of the 3D vehicle pose and velocity combined with a fast (re-) initialization approach. The improved tracking system was evaluated while driving autonomously in urban and unstructured environments. The results show that poorly visible vehicles can be tracked during different weather conditions in real-time.

/pdf/combining-model-and-template-based-vehicle-tracking-for-whcq2ypr9q.pdf

Combining model- and template-based vehicle tracking for autonomous convoy driving

This paper presents a vision-based solution for detection and tracking of convoy vehicles. Our approach is able to estimate the 3D vehicle pose, velocity and steering angle of the leader vehicles and needs template images of each vehicle. The improved template-based algorithm refers to a previously publication. First, we present an extension of our dynamic region growing algorithm, which is used to remove unnecessary image information. Thanks to a new preprocessing step, the segmentation algorithm is more stable while finding the vehicle silhouette. Second, we achieve an improved pose estimation by using rotated image features. Third, the extensive comparison of algorithms to train cascade classifiers leads to the best one for vehicle detection. The algorithm was evaluated while driving autonomously in urban and non-urban environments. Experimental validation shows that our approach can detect and track poorly visible vehicles under different weather conditions in real-time.

/pdf/monocular-template-based-vehicle-tracking-for-autonomous-9sqfx1kag8.pdf

Monocular template-based vehicle tracking for autonomous convoy driving

Team MuCAR participated in the convoy scenario within the ELROB 2016 robotics trial and achieved the best score overall. This paper describes the hard- and software system of MuCAR, our new multi-sensor data fusion method as well as the challenging situations during the competition. The competition took place in an unstructured environment without lane markings and with dynamic objects. Autonomous following of a specific convoy leader and detecting hazardous environments in the form of ERICard associated warning signs were the main tasks of the convoy scenario. In order to achieve high robustness, we present a module which fuses the measurement data of different sensors and tracking modules at object level. In comparison to other participants, our team was able to drive the course without any manual interventions. Additionally, we were the only team which could detect all ERICards completely autonomously.

How MuCAR won the convoy scenario at ELROB 2016

This paper describes a real-time method for building 3D feature models of an object of interest, e.g. a vehicle. The model generation works unsupervised and consists of four recursive steps. The first two steps identify the object of interest and extract the object-related sensor data. In the third and fourth step, prominent features are detected and integrated into a common 3D feature model. This method is primarily designed for a model-based vehicle following system. The system requires a model with prominent features for vehicle tracking by day and night. We present an unsupervised algorithm that processes data from different sensor types to extract the object's prominent features over time. Common sensor types are LiDAR, color- and thermal cameras. We generated feature models of different vehicles online and tested them with a vehicle following system afterwards. Using these generated models, the model-based vehicle detection of the following system could detect convoy leaders with high accuracy and without interruptions.

Carsten Fries

Papers

Autonomous convoy driving by night: The vehicle tracking system

Combining model- and template-based vehicle tracking for autonomous convoy driving

Monocular template-based vehicle tracking for autonomous convoy driving

How MuCAR won the convoy scenario at ELROB 2016

Real-time unsupervised feature model generation for a vehicle following system