https://www.jstage.jst.go.jp/article/jsprs1975/37/6/37_6_39/_pdf

1998 IEEE International Conference on SMCに参加して

Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.

/pdf/deep-learning-for-safe-autonomous-driving-current-challenges-3u5n6rw5eh.pdf

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

This paper presents a lightweight stereo vision-based driving lane detection and classification system to achieve the ego-car’s lateral positioning and forward collision warning to aid advanced driver assistance systems (ADAS). For lane detection, we design a self-adaptive traffic lanes model in Hough Space with a maximum likelihood angle and dynamic pole detection region of interests (ROIs), which is robust to road bumpiness, lane structure changing while the ego-car’s driving and interferential markings on the ground. What’s more, this model can be improved with geographic information system or electronic map to achieve more accurate results. Besides, the 3-D information acquired by stereo matching is used to generate an obstacle mask to reduce irrelevant objects’ interfere and detect forward collision distance. For lane classification, a convolutional neural network is trained by using manually labeled ROI from KITTI data set to classify the left/right-side line of host lane so that we can provide significant information for lane changing strategy making in ADAS. Quantitative experimental evaluation shows good true positive rate on lane detection and classification with a real-time (15Hz) working speed. Experimental results also demonstrate a certain level of system robustness on variation of the environment.

Lane Detection and Classification for Forward Collision Warning System Based on Stereo Vision

Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data, however existing autonomy datasets either represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment, which contains annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A&M University, and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. This novel dataset provides the resources needed by researchers to continue to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments. RELLIS-3D will be published at this https URL.

RELLIS-3D Dataset: Data, Benchmarks and Analysis

SLAM; definition and evolution

This paper presents real-time obstacles detection and their status classification method for collision warning in the vehicle active safety system. Specifically, stereo cameras and millimeter wave (mmw)-radar are fused to help the driving ego-vehicle to find “Danger” or “Potential Danger” in a timely way through combining with the vehicle kinematic model. The proposed method makes full use of the unique advantages of stereo cameras and mmw-radar to sense the environment through several modules. Cameras are mainly used to detect the near or lateral dynamic objects and to obtain the obstacles region of interest (ROI) considering its rich information and high sensitivity to the lateral displacement, while far or longitudinal relative dynamic objects are detected by mmw-radar according to its observational ability to make up for the disadvantage of cameras. In detail, a cameras detector utilizes ”error vectors” rather than the optical flow to obtain dynamic classes through two times clustering. Mmw-radar mainly detects relative dynamic objects, whose absolute speed can be computed according to the ego-vehicle’s state. Then, the detected objects of these two detectors are integrated in an obstacles ROI map, which is obtained through an UV-disparity obstacles detection algorithm to get the final dynamic and relative dynamic objects. Finally, they are classified by comparing them with a dangerous area that is acquired according to the vehicle kinematic model in a special vehicle coordinate system, which is fixed to the ground temporarily. This method is tested on our mobile platforms and the results prove that it can work effectively even though the ego-vehicle drives quickly.

Real-Time Obstacles Detection and Status Classification for Collision Warning in a Vehicle Active Safety System

Multi-camera visual SLAM for off-road navigation

Intersections are quite important and complex traffic scenarios, where the future motion of surrounding vehicles is an indispensable reference factor for the decision-making or path planning of autonomous vehicles. Considering that the motion trajectory of a vehicle at an intersection partly obeys the statistical law of historical data once its driving intention is determined, this paper proposes a long short-term memory based (LSTM-based) framework that combines intention prediction and trajectory prediction together. First, we build an intersection prior trajectories model (IPTM) by clustering and statistically analyzing a large number of prior traffic flow trajectories. The prior trajectories model with fitted probabilistic density is used to approximate the distribution of the predicted trajectory, and also serves as a reference for credibility evaluation. Second, we conduct the intention prediction through another LSTM model and regard it as a crucial cue for a trajectory forecast at the early stage. Furthermore, the predicted intention is also a key that is associated with the prior trajectories model. The proposed framework is validated on two publically released datasets, next generation simulation (NGSIM) and INTERACTION. Compared with other prediction methods, our framework is able to sample a trajectory from the estimated distribution, with its accuracy improved by about 20%. Finally, the credibility evaluation, which is based on the prior trajectories model, makes the framework more practical in the real-world applications.

Vehicle Motion Prediction at Intersections Based on the Turning Intention and Prior Trajectories Model

Deep reinforcement learning (DRL) has been successfully applied to end-to-end autonomous driving, especially in simulation environments. However, common DRL approaches used in complex autonomous driving scenarios sometimes are unstable or difficult to converge. This paper proposes two approaches to improve the stability of the policy model training with as few manual data as possible. For the first approach, reinforcement learning is combined with imitation learning to train a feature network with a small amount of manual data for parameters initialization. For the second approach, an auxiliary network is added to the reinforcement learning framework, which can leverage the real-time measurement information to deepen the understanding of environment, without any guide of demonstrators. To verify the effectiveness of these two approaches, simulations in image information-based and lidar information-based end-to-end autonomous driving systems are conducted, respectively. These approaches are not only tested in the virtual game world, but also applied in Gazebo, in which we build a 3D world based on the real vehicle model of Ranger XP900 platform, the real 3D obstacle model, and the real motion constraints with inertial characteristics, so as to ensure that the trained end-to-end autonomous driving model is more suitable for the real world. Experimental results show that the performance is increased by over 45% in the virtual game world, and can converge quickly and stably in Gazebo in which previous methods can hardly converge.

Wenjie Song

Papers

Lane Detection and Classification for Forward Collision Warning System Based on Stereo Vision

Real-Time Obstacles Detection and Status Classification for Collision Warning in a Vehicle Active Safety System

Multi-camera visual SLAM for off-road navigation

Vehicle Motion Prediction at Intersections Based on the Turning Intention and Prior Trajectories Model

Stabilization Approaches for Reinforcement Learning-Based End-to-End Autonomous Driving