Radar has been a key enabler of advanced driver assistance systems in automotive for over two decades. Being an inexpensive, all-weather and long-range sensor that simultaneously provides velocity measurements, radar is expected to be indispensable to the future of autonomous driving. Traditional radar signal processing techniques often cannot distinguish reflections from objects of interest from clutter and are generally limited to detecting peaks in the received signal. These peak detection methods effectively collapse the image-like radar signal into a sparse point cloud. In this paper, we demonstrate a deep-learning-based vehicle detection solution which operates on the image-like tensor instead of the point cloud resulted by peak detection.To the best of our knowledge, we are the first to implement such a system.

/pdf/vehicle-detection-with-automotive-radar-using-deep-learning-3c09d1v1y5.pdf

Vehicle Detection With Automotive Radar Using Deep Learning on Range-Azimuth-Doppler Tensors

This letter presents a novel radar based, single-frame, multi-class detection method for moving road users ( pedestrian, cyclist, car ), which utilizes low-level radar cube data. The method provides class information both on the radar target- and object-level. Radar targets are classified individually after extending the target features with a cropped block of the 3D radar cube around their positions, thereby capturing the motion of moving parts in the local velocity distribution. A Convolutional Neural Network (CNN) is proposed for this classification step. Afterwards, object proposals are generated with a clustering step, which not only considers the radar targets’ positions and velocities, but their calculated class scores as well. In experiments on a real-life dataset we demonstrate that our method outperforms the state-of-the-art methods both target- and object-wise by reaching an average of 0.70 (baseline: 0.68) target-wise and 0.56 (baseline: 0.48) object-wise F1 score. Furthermore, we examine the importance of the used features in an ablation study.

/pdf/cnn-based-road-user-detection-using-the-3d-radar-cube-3hz95a9vk1.pdf

CNN Based Road User Detection Using the 3D Radar Cube

Multi-object detection and multi-object-tracking in diverse driving situations is the main challenge in autonomous vehicles. Vehicle manufacturers and research organizations are addressing this problem, with multiple sensors such as camera, LiDAR, RADAR, ultrasonic-sensors, GPS, and Vehicle-to-Everything-technology. Deep Neural Networks (DNN) are playing a predominant role to solve this. Fusing the sensing modalities with DNN will be the leading solution to this challenge. This paper evaluates the state-of-the-art techniques that address this challenge, with three primary sensors camera, LiDAR, and RADAR with DNN, and fusion of sensor data with DNN. The analysis shows that there exists an excellent potential to design a more optimized solution to address this challenge. This work proposes a perception model for autonomous vehicles.

Multi-Object Detection and Tracking, Based on DNN, for Autonomous Vehicles: A Review

Various autonomous or assisted driving strategies have been facilitated through the accurate and reliable perception of the environment around a vehicle. Among the commonly used sensors, radar has usually been considered as a robust and cost-effective solution even in adverse driving scenarios, e.g., weak/strong lighting or bad weather. Instead of considering fusing the unreliable information from all available sensors, perception from pure radar data becomes a valuable alternative that is worth exploring. In this paper, we propose a deep radar object detection network, named RODNet, which is cross-supervised by a camera-radar fused algorithm without laborious annotation efforts, to effectively detect objects from the radio frequency (RF) images in real-time. First, the raw signals captured by millimeter-wave radars are transformed to RF images in range-azimuth coordinates. Second, our proposed RODNet takes a snippet of RF images as the input to predict the likelihood of objects in the radar field of view (FoV). Two customized modules are also added to handle multi-chirp information and object relative motion. The proposed RODNet is cross-supervised by a novel 3D localization of detected objects using a camera-radar fusion (CRF) strategy in the training stage. Due to no existing public dataset available for our task, we create a new dataset, named CRUW, 1 1 The dataset and code are available at https://www.cruwdataset.org/ . which contains synchronized RGB and RF image sequences in various driving scenarios. With intensive experiments, our proposed cross-supervised RODNet achieves 86% average precision and 88% average recall of object detection performance, which shows the robustness in various driving conditions.

RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization

Millimeter-wave (mmW) radars are being increasingly integrated into commercial vehicles to support new advanced driver-assistance systems (ADAS) by enabling robust and high-performance object detection, localization, as well as recognition - a key component of new environmental perception. In this paper, we propose a novel radar multiple-perspectives convolutional neural network (RAMP-CNN) that extracts the location and class of objects based on further processing of the range-velocity-angle (RVA) heatmap sequences. To bypass the complexity of 4D convolutional neural networks (NN), we propose to combine several lower-dimension NN models within our RAMP-CNN model that nonetheless approaches the performance upper-bound with lower complexity. The extensive experiments show that the proposed RAMP-CNN model achieves better average recall (AR) and average precision (AP) than prior works in all testing scenarios (see Table. III ). Besides, the RAMP-CNN model is validated to work robustly under the nighttime, which enables low-cost radars as a potential substitute for pure optical sensing under severe conditions.

RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition

In this work, the authors present results for classification of different classes of targets (car, single and multiple people, bicycle) using automotive radar data and different neural networks. A fast implementation of radar algorithms for detection, tracking, and micro-Doppler extraction is proposed in conjunction with the automotive radar transceiver TEF810X and microcontroller unit SR32R274 manufactured by NXP Semiconductors. Three different types of neural networks are considered, namely a classic convolutional network, a residual network, and a combination of convolutional and recurrent network, for different classification problems across the four classes of targets recorded. Considerable accuracy (close to 100% in some cases) and low latency of the radar pre-processing prior to classification (∼0.55 s to produce a 0.5 s long spectrogram) are demonstrated in this study, and possible shortcomings and outstanding issues are discussed.

https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/iet-rsn.2018.0103

Aleksandar Angelov

Papers

Practical classification of different moving targets using automotive radar and deep neural networks