With recent developments, the performance of automotive radar has improved significantly. The next generation of 4D radar can achieve imaging capability in the form of high-resolution point clouds. In this context, we believe that the era of deep learning for radar perception has arrived. However, studies on radar deep learning are spread across different tasks, and a holistic overview is lacking. This review paper attempts to provide a big picture of the deep radar perception stack, including signal processing, datasets, labelling, data augmentation, and downstream tasks such as depth and velocity estimation, object detection, and sensor fusion. For these tasks, we focus on explaining how the network structure is adapted to radar domain knowledge. In particular, we summarise three overlooked challenges in deep radar perception, including multi-path effects, uncertainty problems, and adverse weather effects, and present some attempts to solve them.

/pdf/towards-deep-radar-perception-for-autonomous-driving-30plzr3f.pdf

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

In response to severe environmental and energy crises, the world is increasingly focusing on electric vehicles (EVs) and related emerging technologies. Emerging technologies for EVs have great potential to accelerate the development of smart and sustainable transportation and help build future smart cities. This paper reviews new trends and emerging EV technologies, including wireless charging, smart power distribution, vehicle-to-home (V2H) and vehicle-to-grid (V2G) systems, connected vehicles, and autonomous driving. The opportunities, challenges, and prospects for emerging EV technologies are systematically discussed. The successful commercialization development cases of emerging EV technologies worldwide are provided. This review serves as a reference and guide for future technological development and commercialization of EVs and offers perspectives and recommendations on future smart transportation.

/pdf/trends-and-emerging-technologies-for-the-development-of-2qopzj2g.pdf

Trends and Emerging Technologies for the Development of Electric Vehicles

Roadside object detection and classification provide a good understanding of driving scenarios in regard to over-the-horizon perception. However, typical roadside sensors are insufficient when used separately. The fusion of the millimeter-wave (MMW) radar and monovision camera serves as an efficient approach. Unfortunately, the uncertain and conflicting data in extreme light conditions pose challenges to the fusion process. To this end, this study proposed an evidential framework to fuse the radar and camera data. A novel modeling approach for basic belief assignments (BBAs) was proposed, which took the uncertainty of convolutional neural network (CNN) model into consideration. Moreover, the single-scan and multiscan fusion methods were developed based on the enhanced evidence theory, which utilized different weighted coefficients by referring to the reinforced belief (RB) divergence measure and belief entropy (BE). Both numerical and empirical experiments were conducted to investigate the method performance. Specifically, in numerical experiments, the belief value of actual classification increased to 99.01%. For empirical experiments, based on the real datasets collected by roadside devices, the proposed method was demonstrated to outperform the state-of-the-art ones in terms of 71.06% and 87.23% precisions for bright light and low illumination conditions, respectively. The results verify that the proposed method is effective in fusing the conflicting and uncertain data.

Object Classification Based on Enhanced Evidence Theory: Radar–Vision Fusion Approach for Roadside Application

Next-generation automotive radars provide elevation data in addition to range-, azimuth- and Doppler velocity. In this experimental study, we apply a state-of-the-art object detector (PointPillars), previously used for LiDAR 3D data, to such 3+1D radar data (where 1D refers to Doppler). In ablation studies, we first explore the benefits of the additional elevation information, together with that of Doppler, radar cross section and temporal accumulation, in the context of multi-class road user detection. We subsequently compare object detection performance on the radar and LiDAR point clouds, object class-wise and as a function of distance. To facilitate our experimental study, we present the novel View-of-Delft (VoD) automotive dataset. It contains 8693 frames of synchronized and calibrated 64-layer LiDAR-, (stereo) camera-, and 3+1D radar-data acquired in complex, urban traffic. It consists of 123106 3D bounding box annotations of both moving and static objects, including 26587 pedestrian, 10800 cyclist and 26949 car labels. Our results show that object detection on 64-layer LiDAR data still outperforms that on 3+1D radar data, but the addition of elevation information and integration of successive radar scans helps close the gap. The VoD dataset is made freely available for scientific benchmarking at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://intelligent-vehicles.org/datasets/view-of-delft/</uri> . 

/pdf/multi-class-road-user-detection-with-3-1d-radar-in-the-view-3i71v6j1.pdf

Multi-Class Road User Detection With 3+1D Radar in the View-of-Delft Dataset

Multi-target tracking with radars is a highly challenging problem due to detection artifacts, sensor noise, and interference sources. The traditional signal processing chain is, therefore, a complex combination of various algorithms with several tunable tracking-parameters. Usually, these are initially set by engineers and are independent of the scene tracked. For this reason, they are often non-optimal and generate poorly performing tracking. In this context, scene-adaptive radar processing refers to algorithms that can sense, understand and learn information related to detected targets as well as the environment and adapt its tracking-parameters to optimize the desired goal. In this paper, we propose a Deep Reinforcement Learning framework that guides the scene-adaptive choice of radar tracking-parameters towards an improved performance on multi-target tracking. 

Scene-adaptive radar tracking with deep reinforcement learning

Obtaining the accurate and real-time state of surrounding objects is essential for automated vehicle planning and decision-making to ensure safe driving. In complex traffic scenarios, object occlusion, clutter interference, and limited sensor detection capabilities lead to false alarms and missed object detection, making it challenging to ensure the stability of tracking and state prediction. To address these challenges, in this study, we developed a robust multi-object detection and tracking method for moving objects based on radar and camera data fusion. First, the radar and camera perform target detection independently, and the detection results are correlated in the image plane to generate a random finite set with an object type. Then, based on the Gaussian mixture probability hypothesis density algorithm framework, the tracking process is improved using elliptic discriminant thresholds, an attenuation function, and simplified pruning methods. The experimental results demonstrate that the improved algorithm can accurately estimate the number and state of targets in object occlusion, measurement loss scenarios, and achieve robust continuous multi-object tracking. The proposed method could guide the design of safer and more efficient intelligent driving systems.

Robust Detection and Tracking Method for Moving Object Based on Radar and Camera Data Fusion

Automotive millimeter-wave (MMW) radar is essential in autonomous vehicles due to its robustness in all weather conditions. Traditional commercial automotive radars are limited by their resolution, which makes the object classification task difficult. Thus, the concept of a new generation of four-dimensional (4D) imaging radar was proposed. It has high azimuth and elevation resolution and contains Doppler information to produce a high-quality point cloud. In this paper, we propose an object classification network named Radar Transformer. The algorithm takes the attention mechanism as the core and adopts the combination of vector attention and scalar attention to make full use of the spatial information, Doppler information, and reflection intensity information of the radar point cloud to realize the deep fusion of local attention features and global attention features. We generated an imaging radar classification dataset and completed manual annotation. The experimental results show that our proposed method achieved an overall classification accuracy of 94.9%, which is more suitable for processing radar point clouds than the popular deep learning frameworks and shows promising performance.

https://www.mdpi.com/1424-8220/21/11/3854/pdf

Radar Transformer: An Object Classification Network Based on 4D MMW Imaging Radar.

Intelligent transportation systems (ITSs) play an increasingly important role in traffic management and traffic safety. Smart cameras are the most widely used sensors in ITSs. However, cameras suffer from a reduction in detection and positioning accuracy due to target occlusion and external environmental interference, which has become a bottleneck restricting ITS development. This work designs a stable perception system based on a millimeter-wave radar and camera to address these problems. Radar has better ranging accuracy and weather robustness, which is a better complement to camera perception. Based on an improved Gaussian mixture probability hypothesis density (GM-PHD) filter, we also propose an optimal attribute fusion algorithm for target detection and tracking. The algorithm selects the sensors’ optimal measurement attributes to improve the localization accuracy while introducing an adaptive attenuation function and loss tags to ensure the continuity of the target trajectory. The verification experiments of the algorithm and the perception system demonstrate that our scheme can steadily output the classification and high-precision localization information of the target. The proposed framework could guide the design of safer and more efficient ITSs with low costs.

Sen Li

Papers

Robust Detection and Tracking Method for Moving Object Based on Radar and Camera Data Fusion

Radar Transformer: An Object Classification Network Based on 4D MMW Imaging Radar.

Robust Target Detection and Tracking Algorithm Based on Roadside Radar and Camera.

2-D CFAR Procedure of Multiple Target Detection for Automotive Radar

Traffic participants classification based on 3D radio detection and ranging point clouds