Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.

/pdf/deep-learning-for-safe-autonomous-driving-current-challenges-3u5n6rw5eh.pdf

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

With artificial intelligence technology being advanced by leaps and bounds, intelligent driving has attracted a huge amount of attention recently in research and development. In intelligent driving, lane line detection is a fundamental but challenging task particularly under complex road conditions. In this paper, we propose a simple yet appealing network called Ripple Lane Line Detection Network (RiLLD-Net), to exploit quick connections and gradient maps for effective learning of lane line features. RiLLD-Net can handle most common scenes of lane line detection. Then, in order to address challenging scenarios such as occluded or complex lane lines, we propose a more powerful network called Ripple-GAN, by integrating RiLLD-Net, confrontation training of Wasserstein generative adversarial networks, and multi-target semantic segmentation. Experiments show that, especially for complex or obscured lane lines, Ripple-GAN can produce a superior detection performance to other state-of-the-art methods.

/pdf/ripple-gan-lane-line-detection-with-ripple-lane-line-o2bb9fdj1t.pdf

Ripple-GAN: Lane Line Detection With Ripple Lane Line Detection Network and Wasserstein GAN

Sorting gangue from raw coal is an essential concern in coal mining engineering. Prior to separation, the location and shape of the gangue should be extracted from the raw coal image. Several approaches regarding automatic detection of gangue have been proposed to date; however, none of them is satisfying. Therefore, this paper aims to conduct gangue segmentation using a U-shape fully convolutional neural network (U-Net). The proposed network is trained to segment gangue from raw coal images collected under complex environmental conditions. The probability map outputted by the network was used to obtain the location and shape information of gangue. The proposed solution was trained on a dataset consisting of 54 shortwave infrared (SWIR) raw coal images collected from Datong Coalfield. The performance of the network was tested with six never seen images, achieving an average area under the receiver operating characteristics (AUROC) value of 0.96. The resulting intersection over union (IoU) was on average equal to 0.86. The results show the potential of using deep learning methods to perform gangue segmentation under various conditions.

Automatic Coal and Gangue Segmentation Using U-Net Based Fully Convolutional Networks

Lane detection (LD) under different illumination conditions is a vital part of lane departure warning system and vehicle localization which are current trends in the future smart cities. Recently, vision-based methods are proposed to detect lane markers in different road situations including abnormal marker cases. However, an inclusive framework for driverless cars has not been introduced yet. In this work, a novel LD and tracking method is proposed for the autonomous vehicle in the IoT-based framework (IBF). The IBF consists of three modules which are vehicle board (VB), cloud module (CM), and the vehicle remote controller. The LD and tracking are carried out initially by the VB, and then, in case of any failure, the whole set of data is passed to CM to be processed and the results are sent to the VB to perform the appropriate action. If the CM detects a lane departure, then the autonomous vehicle is driven remotely and the VB would be restarted. In addition to the proposed framework, an illumination invariance method is presented to detect lane markers under different light conditions. The simulation results with real-life data demonstrate lane-keeping rates of 95.3% and 95.2% in tunnels and on highways, respectively. The approximate processing time of the proposed method is 31 ms/frame which fulfills the real-time requirements.

/pdf/lane-detection-under-artificial-colored-light-in-tunnels-and-1676o8xeug.pdf

Lane detection under artificial colored light in tunnels and on highways: an IoT-based framework for smart city infrastructure

The uneven reflective metal surface has great influence on the high-precision measurement of the structured light 3-D shape. By analyzing the laser scattering and reflection models, a set of light stripe center’s subpixel extraction methods which has strong robustness is proposed. The method first cuts off the influence of the uneven fixed background by using the difference image method to process the bright field and the dark field. Then, the regional growth statistics method is used to eliminate the influence of random laser speckle noise, and then the gray-gravity method is used to obtain the coarse center of the laser stripe. The Sobel operator is used to obtain the gradient vector of the stripe pixel point, and then the normal direction field of the light stripe is obtained. The normal direction field vector is taken as the direction to find the $5\times5$ neighborhood of the coarse light stripe’s center, and then the gray-gravity method is reused to determine the center position of the laser stripe in the normal line direction; the pixel coordinates of the subpixel level are obtained by using bilinear interpolation. The experimental results show that the method can effectively eliminate the interference of flash point noise and has strong robustness. The detection error of the stripe is less than 0.1 pixels, which ensures the system measurement accuracy. Compared with other methods, the system’s resolution can reach 0.02 mm.

Laser Stripe Center Detection Under the Condition of Uneven Scattering Metal Surface for Geometric Measurement

Curve is the traffic accident-prone area in the traffic system of the structural road. How to effectively detect the lane-line and timely give the traffic information ahead for drivers is a difficult point for the assisted safe driving. The traditional lane detection technology is not very applicable in the curved road conditions. Thus, a curve detection algorithm which is based on straight-curve model is proposed in this paper and this method has good applicability for most curve road conditions. First, the method divides the road image into the region of interest and the road background region by analyzing the basic characteristics of the road image. The region of interest is further divided into the straight region and the curve region. At the same time, the straight-curve mathematical model is established. The mathematical equation of the straight model is obtained by using the improved Hough transform. The polynomial curve model is established according to the continuity of the road lane-line and the tangent relationship between the straight model and the curve model. Then, the parameters of the curve model equation are solved by the curve fitting method. Finally, the detection and identification of the straight and the curve are realized respectively and the road lane-line is reconstructed. Experiments show that this method can accurately identify the curve lane-line, provide effective traffic information, make early warning, and it also has certain universality.

Lane Detection of Curving Road for Structural Highway With Straight-Curve Model on Vision

To achieve online testing of high-frequency electric resistance welding (HF-ERW) tube quality, forecasting models were established for welding defect conditions with collected high-speed images of the joint melting phenomenon, based on a radial basis function neural network (RBFNN). Firstly, the dimensions of the collected image samples were deduced by principal component analysis (PCA). Then, the reduced-dimension image samples were set as inputs of both BPNN (back-propagation neural network) and, for comparison, RBFNN, which were trained so that the model parameters were optimized. Finally, the testing sample set was identified by trained networks. The experimental results show that RBFNN had better generalization ability for HF-ERW images than BPNN, which meant that the recognition rate of low-heat input status reached 100%, while the recognition rate of overheating input status reached 97.67%. They also show that the welding quality detection system based on a neural network is very effective and has a strong guiding significance for welding quality control.

Detection of HF-ERW status by neural network on imaging

For the purpose of solving the high quality required in some image based measurement systems, a new calibration algorithm for a non-uniform illumination field image is proposed on the basis of some existing algorithm. In this algorithm, by analysing the characteristics of the potential function for target detection, a new potential function for target detection was designed. A measurement function for standardisation of variety was deduced thereafter. After that, the mapping function from the original image to the background calibration image was given. Numerical simulations showed that the proposed algorithm can restrain the non-uniform background illumination field and calibrate the strong fluctuant background of the illumination field image and thus improve the quality of the image. Meanwhile, the algorithm is simple to carry out and very suitable in practice.

Xiang-Mo Zhao

Papers

Lane Detection of Curving Road for Structural Highway With Straight-Curve Model on Vision

Detection of HF-ERW status by neural network on imaging

Novel background calibration algorithm for image in non-uniform illumination field