With the rapid development of deep learning techniques, deep convolutional neural networks (DCNNs) have become more important for object detection. Compared with traditional handcrafted feature-based methods, the deep learning-based object detection methods can learn both low-level and high-level image features. The image features learned through deep learning techniques are more representative than the handcrafted features. Therefore, this review paper focuses on the object detection algorithms based on deep convolutional neural networks, while the traditional object detection algorithms will be simply introduced as well. Through the review and analysis of deep learning-based object detection techniques in recent years, this work includes the following parts: backbone networks, loss functions and training strategies, classical object detection architectures, complex problems, datasets and evaluation metrics, applications and future development directions. We hope this review paper will be helpful for researchers in the field of object detection.

A review of object detection based on deep learning

Ship detection in synthetic aperture radar (SAR) images is becoming a research hotspot. In recent years, as the rise of artificial intelligence, deep learning has almost dominated SAR ship detection community for its higher accuracy, faster speed, less human intervention, etc. However, today, there is still a lack of a reliable deep learning SAR ship detection dataset that can meet the practical migration application of ship detection in large-scene space-borne SAR images. Thus, to solve this problem, this paper releases a Large-Scale SAR Ship Detection Dataset-v1.0 (LS-SSDD-v1.0) from Sentinel-1, for small ship detection under large-scale backgrounds. LS-SSDD-v1.0 contains 15 large-scale SAR images whose ground truths are correctly labeled by SAR experts by drawing support from the Automatic Identification System (AIS) and Google Earth. To facilitate network training, the large-scale images are directly cut into 9000 sub-images without bells and whistles, providing convenience for subsequent detection result presentation in large-scale SAR images. Notably, LS-SSDD-v1.0 has five advantages: (1) large-scale backgrounds, (2) small ship detection, (3) abundant pure backgrounds, (4) fully automatic detection flow, and (5) numerous and standardized research baselines. Last but not least, combined with the advantage of abundant pure backgrounds, we also propose a Pure Background Hybrid Training mechanism (PBHT-mechanism) to suppress false alarms of land in large-scale SAR images. Experimental results of ablation study can verify the effectiveness of the PBHT-mechanism. LS-SSDD-v1.0 can inspire related scholars to make extensive research into SAR ship detection methods with engineering application value, which is conducive to the progress of SAR intelligent interpretation technology.

LS-SSDD-v1.0: A Deep Learning Dataset Dedicated to Small Ship Detection from Large-Scale Sentinel-1 SAR Images

Synthetic aperture radar (SAR) automatic target recognition (ATR) plays an important role in SAR image interpretation. However, at least hundreds of training samples are usually required for each target type in the existing SAR ATR algorithms. In this article, a novel few-shot learning framework named hybrid inference network (HIN) is proposed to tackle the problem of SAR target recognition with only a few training samples. The recognition procedure of HIN consists of two main stages. In the first stage, an embedding network is utilized to map the SAR images into an embedding space. In the second stage, a hybrid inference strategy that combines the inductive inference and the transductive inference is adopted to classify the samples in the embedding space. In the inductive inference section, each sample is recognized independently according to a metric based on Euclidean distance. In the transductive inference section, all samples are recognized as a whole according to their manifold structures by label propagation. Finally, in the hybrid inference section, the classification result is obtained by combining the above two inference methods. To train the framework more effectively, a novel loss function named enhanced hybrid loss is proposed to constrain samples to gain better interclass separability in the embedding space. Experimental results on the moving and stationary target acquisition and recognition (MSTAR) benchmark data set illustrate that HIN performs well in few-shot SAR image classification.

Hybrid Inference Network for Few-Shot SAR Automatic Target Recognition

In recent studies, synthetic aperture radar (SAR) automatic target recognition (ATR) algorithms that are based on the convolutional neural network (CNN) have achieved high recognition rates in the moving and stationary target acquisition and recognition (MSTAR) dataset. However, in a SAR ATR task, the feature maps with little information automatically learned by CNN will disturb the classifier. We design a new enhanced squeeze and excitation (enhanced-SE) module to solve this problem, and then propose a new SAR ATR network, i.e., the enhanced squeeze and excitation network (ESENet). When compared to the available CNN structures that are designed for SAR ATR, the ESENet can extract more effective features from SAR images and obtain better generalization performance. In the MSTAR dataset containing pure targets, the proposed method achieves a recognition rate of 97.32% and it exceeds the available CNN-based SAR ATR algorithms. Additionally, it has shown robustness to large depression angle variation, configuration variants, and version variants.

/pdf/sar-atr-of-ground-vehicles-based-on-esenet-5gechgkme8.pdf

SAR ATR of Ground Vehicles Based on ESENet

Synthetic aperture radar (SAR) image classification using deep neural network has drawn great attention, which generally requires various layers of deep model for feature learning. However, a deeper neural network will result in overfitting with limited training samples. In this paper, a multi-scale deep feature learning network with bilateral filtering (MDFLN-BF) is proposed for SAR image classification, which aims to extract discriminative features and reduce the requirement of labeled samples. In the proposed framework, MDFLN is proposed to extract features from SAR image on multiple scales, where the SAR image is stratified into different scales and a full convolutional network is utilized to extract features from each scale sub-image. Then, features of multiple scales are classified by multiple softmax classifiers and combined by majority vote algorithm. Further, bilateral filtering is developed to optimize the classification map based on spatial relation, which aims to improve the spatial smoothness. Experiments are tested on three SAR images with different sensors, bands, resolutions, and polarizations in order to prove the generalization ability. It is demonstrated that the proposed MDFLN-BF is able to yield superior results than other related deep networks.

Multi-scale deep feature learning network with bilateral filtering for SAR image classification

Speckle noise is an inherent but annoying property in the synthetic aperture radar (SAR) imaging. In this paper we investigate the influence of speckle on the classical convolutional neural network (CNN) for SAR target classification. Then a dual stage coupled CNN architecture, named despeckling and classification coupled CNNs (DCC-CNNs), is proposed to distinguish multiple categories of ground targets in SAR images with strong and varying speckle. It first applies the despeckling sub-network for noise reduction. After that, residual speckle features as well as target information would be learned by the classification sub-network in order to solve the noise robustness problem of CNN. Besides, a new quantitative measure is developed for the quality assessment of SAR target images. It takes into account structural properties of the speckled SAR image of the target of interest and consistency with visual perception. Finally, a series of comparative experiments and discussions are carried out to validate the proposed assessment criterion and DCC-CNNs. Using synthetic SAR images based on the public MSTAR datasets, results show that the overall classification accuracy for ten ground target classes could be higher than 82% at a variety of speckle noise levels.

Ground Target Classification in Noisy SAR Images Using Convolutional Neural Networks

Deep learning techniques have attracted much attention in the radar automatic target recognition. In this paper, we investigate an acceleration method of the convolutional neural network (CNN) on the field-programmable gate array (FPGA) for the embedded application of the millimeter-wave (mmW) radar-based human activity classification. Considering the micro-Doppler effect caused by a person's body movements, the spectrogram of mmW radar echoes is adopted as the CNN input. After that, according to the CNN architecture and the properties of the FPGA implementation, several parallel processing strategies are designed as well as data quantization and optimization of classification decision to accelerate the CNN execution. Finally, comparative experiments and discussions are carried out based on a measured dataset of nine individuals with four different actions by using a 77-GHz mmW radar. The results show that the proposed method not only maintains the high classification accuracy but also improves its execution speed, memory requirement, and power consumption. Specifically, compared with the implementation of the same network model on a graphics processing unit, it could achieve the speedup of about 30.42% at the cost of the classification accuracy loss of only 0.27%.

/pdf/acceleration-of-fpga-based-convolutional-neural-network-for-sqg3uavwij.pdf

Acceleration of FPGA Based Convolutional Neural Network for Human Activity Classification Using Millimeter-Wave Radar

The ghost phenomenon in synthetic aperture radar (SAR) imaging is primarily caused by azimuth or range ambiguities, which cause difficulties in SAR target detection application. To mitigate this influence, we propose a ship target detection method in spaceborne SAR imagery, using a hierarchical convolutional neural network (H-CNN). Based on the nature of ghost replicas and typical target classes, a two-stage CNN model is built to detect ship targets against sea clutter and the ghost. First, regions of interest (ROIs) were extracted from a large imaged scene during the coarse-detection stage. Unwanted ghost replicas represented major residual interference sources in ROIs, therefore, the other CNN process was executed during the fine-detection stage. Finally, comparative experiments and analyses, using Sentinel-1 SAR data and various assessment criteria, were conducted to validate H-CNN. Our results showed that the proposed method can outperform the conventional constant false-alarm rate technique and CNN-based models.

https://www.mdpi.com/2072-4292/11/6/620/pdf

A Hierarchical Convolution Neural Network (CNN)-Based Ship Target Detection Method in Spaceborne SAR Imagery

Deep learning (DL) techniques recently have attracted much attention in the synthetic aperture radar (SAR) automatic target recognition (ATR) Due to the coherent imaging pattern, SAR images inherently suffer from the speckle noise To mitigate its influence, this letter proposes a joint convolutional neural network (J-CNN) for simultaneous despeckling and classification of SAR targets It integrates a two-step process in the CNN framework but without the pooling operation during the despeckling phase Then, a new loss function is introduced, and its partial derivatives with respect to weights are given for the training of J-CNN Finally, comparative experiments with some classical network models are carried out based on synthetic SAR target images The results demonstrate that the proposed method not only significantly outperforms other models under strong speckle noise condition but also has an efficient architecture with fewer weight parameters

A Joint Convolutional Neural Network for Simultaneous Despeckling and Classification of SAR Targets

In view of synthetic aperture radar (SAR) target detection, traditional methods are based on hand-crafted feature extraction and classifier. Besides, deep learning (DL) based methods are research hotspots in recent year. However, their shortcomings cannot be neglected, i.e. detection accuracy of traditional method needs to be improved and DL features are difficult to interpret. To overcome these problems, a target detection method with multi-features in SAR imagery is proposed in this paper. It consists of two parallel sub-channels. DL features and hand-crafted features are extracted in these channels, respectively. Here, convolutional neural network (CNN) model is applied to capture DL features of original SAR images. Deep neural network (NN) is used to further analyze hand-crafted features. Furthermore, two sub-channel features are concatenated together in the main channel. After several layers network processing, fused deep features are extracted. Finally, softmax classifier is applied to discriminate ship target. According to the experiments based on Sentinel-1 SAR data, we can find that the detection performance is improved by the proposed method.

Tong Zheng

Papers

Ground Target Classification in Noisy SAR Images Using Convolutional Neural Networks

Acceleration of FPGA Based Convolutional Neural Network for Human Activity Classification Using Millimeter-Wave Radar

A Hierarchical Convolution Neural Network (CNN)-Based Ship Target Detection Method in Spaceborne SAR Imagery

A Joint Convolutional Neural Network for Simultaneous Despeckling and Classification of SAR Targets

Deep learning based target detection method with multi-features in SAR imagery