With the rise of deep convolutional neural networks, object detection has achieved prominent advances in past years. However, such prosperity could not camouflage the unsatisfactory situation of Small Object Detection (SOD), one of the notoriously challenging tasks in computer vision, owing to the poor visual appearance and noisy representation caused by the intrinsic structure of small targets. In addition, large-scale dataset for benchmarking small object detection methods remains a bottleneck. In this paper, we first conduct a thorough review of small object detection. Then, to catalyze the development of SOD, we construct two large-scale Small Object Detection dAtasets (SODA), SODA-D and SODA-A, which focus on the Driving and Aerial scenarios respectively. SODA-D includes 24828 high-quality traffic images and 278433 instances of nine categories. For SODA-A, we harvest 2513 high resolution aerial images and annotate 872069 instances over nine classes. The proposed datasets, as we know, are the first-ever attempt to large-scale benchmarks with a vast collection of exhaustively annotated instances tailored for multi-category SOD. Finally, we evaluate the performance of mainstream methods on SODA. We expect the released benchmarks could facilitate the development of SOD and spawn more breakthroughs in this field. Datasets and codes are available at: https://shaunyuan22.github.io/SODA.

Towards Large-Scale Small Object Detection: Survey and Benchmarks

Owing to the outstanding feature extraction capability, convolutional neural networks (CNNs) have been widely applied in hyperspectral image (HSI) classification problems and have achieved an impressive performance. However, it is well known that 2D convolution suffers from the absent consideration of spectral information, while 3D convolution requires a huge amount of computational cost. In addition, the cost of labeling and the limitation of computing resources make it urgent to improve the generalization performance of the model with scarcely labeled samples. To relieve these issues, we design an end-to-end 3D octave and 2D vanilla mixed CNN, namely Oct-MCNN-HS, based on the typical 3D-2D mixed CNN (MCNN). It is worth mentioning that two feature fusion operations are deliberately constructed to climb the top of the discriminative features and practical performance. That is, 2D vanilla convolution merges the feature maps generated by 3D octave convolutions along the channel direction, and homology shifting aggregates the information of the pixels locating at the same spatial position. Extensive experiments are conducted on four publicly available HSI datasets to evaluate the effectiveness and robustness of our model, and the results verify the superiority of Oct-MCNN-HS both in efficacy and efficiency.

3D Octave and 2D Vanilla Mixed Convolutional Neural Network for Hyperspectral Image Classification with Limited Samples

ABSTRACT Remote-sensing images comprise massive amount of spatial and semantic data that can be employed for several applications. Presently, deep learning (DL) models for RS image processing become a familiar research area. Due to the advancements of recent satellite imaging sensors , the issue of huge amount of data processing becomes a challenging problem. To accomplish this, deep transfer learning (DTL) models are developed to resolve the semantic gap among various datasets This study develops a new DTL-based fusion model for environmental remote-sensing image classification, called DTLF-ERSIC technique. The proposed technique focuses on the design of fusion model to combine multiple feature vectors and thereby attains maximum classification performance. The DTLF-ERSIC technique incorporates the entropy-based fusion of three feature extraction techniques, namely, Discrete Local Binary Pattern (DLBP), Residual Network (ResNet50), and EfficientNet models. Besides, a rain optimization algorithm (ROA) with fuzzy rule-based classifier (FRC) is applied to predict the class labels of the test RS images and shows the novelty of the work. A comprehensive experimental analysis of the DTLF-ERSIC technique takes place on benchmark dataset and examined the results in terms of different performance measures. The simulation results reported the supremacy of the DTLF-ERSIC technique over the recent state-of-art techniques.

https://www.tandfonline.com/doi/pdf/10.1080/22797254.2021.2017799?needAccess=true

Deep Transfer Learning based Fusion Model for Environmental Remote Sensing Image Classification Model

In hyperspectral image (HSI) classification, convolutional neural networks (CNN) have been attracting increasing attention because of their ability to represent spectral-spatial features. Nevertheless, the conventional CNN models perform convolution operation on regular-grid image regions with a fixed kernel size and as a result, they neglect the inherent relation between HSI data. In recent years, graph convolutional networks (GCN) used for data representation in a non-Euclidean space, have been successfully applied to HSI classification. However, conventional GCN methods suffer from a huge computational cost since they construct the adjacency matrix between all HSI pixels, and they ignore the local spatial context information of hyperspectral images. To alleviate these shortcomings, we propose a novel method termed spectral-spatial offset graph convolutional networks (SSOGCN). Different from the usually used GCN models that compute the adjacency matrix between all pixels, we construct an adjacency matrix only using pixels within a patch, which contains rich local spatial context information, while reducing the computation cost and memory consumption of the adjacency matrix. Moreover, to emphasize important local spatial information, an offset graph convolution module is proposed to extract more robust features and improve the classification performance. Comprehensive experiments are carried out on three representative benchmark data sets, and the experimental results effectively certify that the proposed SSOGCN method has more advantages than the recent state-of-the-art (SOTA) methods.

Spectral-Spatial Offset Graph Convolutional Networks for Hyperspectral Image Classification

Remote sensing (RS) crossmodal text-image retrieval has become a research hotspot in recent years for its application in semantic localization. However, since multiple inferences on slices are demanded in semantic localization, designing a crossmodal retrieval model with less computation but well performance becomes an emergent and challenging task. In this article, considering the characteristics of multi-scale and target redundancy in RS, a concise but effective crossmodal retrieval model (LW-MCR) is designed. The proposed model incorporates multi-scale information and dynamically filters out redundant features when encoding RS image, while text features are obtained via lightweight group convolution. To improve the retrieval performance of LW-MCR, we come up with a novel hidden supervised optimization method based on knowledge distillation. This method enables the proposed model to acquire dark knowledge of the multi-level layers and representation layers in the teacher network, which significantly improves the accuracy of our lightweight model. Finally, on the basis of contrast learning, we present a method employing unlabeled data to boost the performance of RS retrieval model further. The experiment results on four RS image-text datasets demonstrate the efficiency of LW-MCR in RS crossmodal retrieval (RSCR) tasks. We have released some codes of the semantic localization and made it open to access at <uri>https://github.com/xiaoyuan1996/retrievalSystem</uri>. 

A Lightweight Multi-Scale Crossmodal Text-Image Retrieval Method in Remote Sensing

The nascent graph representation learning has shown superiority for resolving graph data. Compared to conventional convolutional neural networks, graph-based deep learning has the advantages of illustrating class boundaries and modeling feature relationships. Faced with hyperspectral image (HSI) classification, the priority problem might be how to convert hyperspectral data into irregular domains from regular grids. In this regard, we present a novel method that performs the localized graph convolutional filtering on HSIs based on spectral graph theory. First, we conducted principal component analysis (PCA) preprocessing to create localized hyperspectral data cubes with unsupervised feature reduction. These feature cubes combined with localized adjacent matrices were fed into the popular graph convolution network in a standard supervised learning paradigm. Finally, we succeeded in analyzing diversified land covers by considering local graph structure with graph convolutional filtering. Experiments on real hyperspectral datasets demonstrated that the presented method offers promising classification performance compared with other popular competitors.

Hyperspectral Image Classification with Localized Graph Convolutional Filtering

Object detection is a focal point in remote sensing applications. Remote sensing images typically contain a large number of small objects and a wide range of orientations across objects. This results in great challenges to small object detection approaches based on remote sensing images. Methods directly employ channel relations with equal weights to construct information features leads to inadequate feature representation in complex image small object detection tasks. Multiscale detection methods improve the speed and accuracy of detection, while small objects themselves contain limited information, and the features are easily lost following down-sampling. During the detection, the feature images are independent across scales, resulting in a discontinuity at the detection scale. In this article, we propose the multiscale context and enhanced channel attention (MSCCA) model. MSCCA employs PeleeNet as the backbone network. In particular, the feature image channel attention is enhanced and the multiscale context information is fused with multiscale detection methods to improve the characterization ability of the convolutional neural network. The proposed MSCCA method is evaluated on two real datasets. Results show that for 512 × 512 input images, MSCCA was able to achieve 80.4% and 94.4% mAP on the DOTA and NWPU VHR-10, respectively. Meanwhile, the model size of MSCCA is 21% smaller than that of its predecessor. MSCCA can be considered as a practical lightweight oriented object detection model in remote sensing images.

/pdf/lightweight-oriented-object-detection-using-multiscale-50qq1nznk5.pdf

Shengliang Pu

Papers

Hyperspectral Image Classification with Localized Graph Convolutional Filtering

Lightweight Oriented Object Detection Using Multiscale Context and Enhanced Channel Attention in Remote Sensing Images