Deep learning can fully exploit the potential information of data, which facilitates effective target-background separation for hyperspectral target detection (HTD). The deep learning model usually requires a large number of labeled samples yet the available prior target spectra in the hyperspectral image (HSI) are extremely limited. In addition, network training also requires reliable input samples to ensure that the trained model has stronger data discrimination, but current detection methods often suffer from the problem that the background samples are impure. To address the above problems, we propose a lightweight convolutional neural network based on joint correlation distance constraints and density peak clustering (LCNN-CD) for HTD. First, the correlation distance between the prior target and HSI is calculated, and the pixels with high correlation are retained for expanding the target sample, which effectively handles the problem of insufficient target samples. Second, the density peak clustering algorithm is used to extract the main pure background samples, which effectively solves the problem that the background samples are sufficiently impure. Finally, a lightweight convolutional neural network model is designed, which is fed by the training samples (both target and background samples) to obtain the final detection results with low computational efficiency. The proposed method is compared with the classical and recently popular HTD methods on three real HSI datasets. Numerous experiments show that the proposed LCNN-CD method has better target detection performance and effectiveness.

A Lightweight Convolutional Neural Network Based on Joint Correlation Distance Constraints and Density Peak Clustering for Hyperspectral Target Detection

Considerable endeavors have been expended toward enhancing the representation performance for hyperspectral anomaly detection (HAD) through physical model-based methods and recent deep learning-based approaches. Of these methods, the low-rank representation (LRR) model is widely adopted for its formidable separation capabilities for background and target features, however, its practical applications are limited due to the reliance on manual parameter selection and subpar generalization performance. To this end, this article presents a new HAD baseline network, referred to as LRR-Net, which synergizes the LRR model with deep learning techniques. LRR-Net leverages the alternating direction method of multipliers (ADMM) optimizer to solve the LRR model efficiently and incorporates the solution as prior knowledge into the deep network to guide the optimization of parameters. Moreover, LRR-Net transforms the regularized parameters into trainable parameters of the deep neural network, thus alleviating the need for manual parameter tuning. Additionally, this article proposes a sparse neural network embedding to demonstrate the scalability of the LRR-Net framework. Empirical evaluations on eight distinct datasets illustrate the efficacy and superiority of the proposed approach compared to the state-of-the-art methods.

LRR-Net: An Interpretable Deep Unfolding Network for Hyperspectral Anomaly Detection

SCADA securing system using deep learning to prevent cyber infiltration.

Forest fires seriously jeopardize forestry resources and endanger people and property. The efficient identification of forest fire smoke, generated from inadequate combustion during the early stage of forest fires, is important for the rapid detection of early forest fires. By combining the Convolutional Neural Network (CNN) and the Lightweight Vision Transformer (Lightweight ViT), this paper proposes a novel forest fire smoke detection model: the SR-Net model that recognizes forest fire smoke from inadequate combustion with satellite remote sensing images. We collect 4,000 satellite remote sensing images, 2,000 each for clouds and forest fire smoke, from Himawari-8 satellite imagery located in forest areas of China and Australia, and the image data are used for training, testing, and validation of the model at a ratio of 3:1:1. Compared with existing models, the proposed SR-Net dominates in recognition accuracy (96.9%), strongly supporting its superiority over benchmark models: MobileNet (92.0%), GoogLeNet (92.0%), ResNet50 (84.0%), and AlexNet (76.0%). Model comparison results confirm the accuracy, computational efficiency, and generality of the SR-Net model in detecting forest fire smoke with high temporal resolution remote sensing images.

A forest fire smoke detection model combining convolutional neural network and vision transformer

Deep learning (DL) has recently risen to prominence in hyperspectral target detection (HTD). Nevertheless, how to tackle the extreme training sample imbalance together with achieving target highlighting and background suppression is challenging. In addition, due to the spectral redundancy of hyperspectral imagery (HSI), it is a new course for HTD through band selection (BS) to retain crucial bands, thereupon improving the subsequent detection performance. Accordingly, we propose a DL-based BS-HTD (DLBSTD) algorithm, incorporating DL-based BS with DL-based HTD for the first time. Most significantly, a multidepth and multibranch network (MDBN) for HTD based on a novel BS method is proposed. First, the BS method, including an alternating local–global reconstruction network (ALGRN) and a correlation measurement strategy, provides representative bands containing key target information for MDBN. For the training sample imbalance of MDBN, we develop a BS-based method to select multifarious representative background training samples and propose a target band random substitution (TBRS) strategy to augment an ample target training set. Finally, the MDBN composed of a multidepth feature extraction (MDFE) module, three fusion strategies, and the parallel local convolution and gated recurrent unit (Conv-GRU) fully taps the spectral feature relationships to highlight targets and suppress backgrounds. Compared with nine competitive HTD algorithms, we carry out plentiful experiments on four classical datasets exhibiting that the proposed DLBSTD has strong generalization and salient detection performance of target highlighting and background suppression.

A Multidepth and Multibranch Network for Hyperspectral Target Detection Based on Band Selection

ABSTRACT Hyperspectral image (HSI) classification method based on convolutional neural network (CNN) has been widely used and has achieved remarkable results. However, the convolution operation is good at extracting local features, it is difficult to capture the global representation. Recently, Vision Transformer (ViT) shines with cascading self-attention modules that can aggregate global representation between patch embeddings. The self-attention module cascaded in ViT can capture long-distance feature dependencies, but it is prone to ignore the local feature details. In order to address those series of problems, this article proposes a dual-branch feature interaction (DBFI) network based on CNN and ViT, which allows global information and local feature to fully interact to enhance feature representation capabilities. To resolve the problem of the semantic gap between the two branches, a feature interaction unit (FIU) based on the attention mechanism is designed as an information interactive bridge to connect the two branches. The features are branched from ViT to CNN through the upsampling stream to enhance the global perception of the network. Similarly, the down-sampling stream is used as a coupling to connect the CNN branch to the Transformer branch to supplement the local detailed features of the network. In addition, the channel attention and spectral attention in FIU can further emphasize the distinguishability of HSI features and improve the performance of network. Experiments are conducted on three benchmark HSI datasets demonstrate that the DBFI network is better than other classification methods.

Classification of hyperspectral image based on dual-branch feature interaction network

Hyperspectral image change detection (HSI-CD) has recently become a research hotspot. Current methods rely heavily on a huge amount of training samples to perform the change detection tasks. While acquiring data from the same region of bitemporal HSIs is extraordinarily time-consuming and laborious. Therefore, this letter proposes an unsupervised method based on 3-D depthwise separable convolutional autoencoders (DSConvAEs). First, the dual-branch symmetrical 3-D (DBS3-D) DSConvAE is pretrained with limited samples to obtain the optimal weights, which facilitates extracting discriminative spatial and spectral features subsequently. Second, we adopt the temporal-specific feature concatenation strategy to acquire comprehensive characteristics from bi-temporal HSIs. Third, the general autoencoders are employed at the end of the model to further explore the high-level and abstract feature vectors. Finally, we compare the mean square loss calculated from the spatial–spectral branches and apply threshold judgment to generate the ultimate detection maps. Experimental results on three public HSI datasets demonstrate that the proposed framework outperforms other comparative methods by significant improvements.

Depthwise Separable Convolutional Autoencoders for Hyperspectral Image Change Detection

The land cover classification of single-modal remote sensing (RS) data has recently reached a bottleneck. The joint use of multimodal RS data to improve the classification performances has received much attention. Convolutional neural networks are powerful tools in feature extraction and contextual modeling. While they have attendant drawbacks to capture the sequence attributes of spectral signatures and struggle to acquire discriminative spectral–spatial features from a global perspective due to limitations inherent in their network backbones. The transformer backbone is a promising approach for addressing these challenges and generating novel insights in the multimodal RS image classification. In this article, we present a new model called multimodal transformer network (MTNet) that leverages transformer advantages to capture both the specific and shared characteristics of hyperspectral (HS) and light detection and ranging (LiDAR) data. HS images contain a wide range of bands with rich spectral information and LiDAR data provide accurate elevation information without affecting by environmental factors. The well-designed module HS spectral transformer can learn spectrally local sequence information from neighboring bands of HS images, yielding groupwise spectral embeddings comprising rich diagnostic information about land covers. Furthermore, the HS and LiDAR spatial transformers aim to mine the pixelwise feature embedding relationships in a global manner, capturing spatial and elevation information of HS and LiDAR, respectively. Finally, the feature embedding tokens of two modalities are integrated jointly and a new transformer encoder is redesigned to explore the shared spatial characteristics between the two modalities. We evaluate the classification performances of the proposed MTNet on three public HS-LiDAR datasets by conducting extensive experiments, exhibiting superiority over conventional classifiers and state-of-the-art networks.

Multimodal Transformer Network for Hyperspectral and LiDAR Classification

ABSTRACT Each pixel in the hyperspectral image contains tens to hundreds of narrow bands of detailed spectral information captured by a hyperspectral sensor. Hyperspectral image classification (HSIC) is widely used in remote sensing image analysis. Convolutional neural network (CNN) is one of the most commonly used deep learning methods based on visual data processing. In recent years, hyperspectral classification method based on the convolutional neural network has shown excellent performance. However, single-scale feature extraction may lose some important detail information and cannot guarantee that the best spatial features are captured. Multi-scale character extraction and increasingly deep network structures can improve classification accuracy, but also bring a large number of computational parameters, resulting in higher computational costs. In response to this series of problems, this paper proposes a lightweight spatial-spectral network based on 3D − 2D multi-group feature extraction module (MGFM) for HSCI. In the 3D-MGFM, the input feature maps are grouped and processed in parallel, and then the information of each channel is integrated by point-wise convolution to achieve spatial-spectral feature fusion. We use dilated convolutions with different dilation rate on the convolution kernel to improve the problem of feature information loss, so that we can better learn deep-level spatial-spectral features. In the 2D-MGFM, we selectively emphasize spectral feature information through SENet and extract multi-scale features based on Depthwise Separable Convolution (DSC) grouping. Experimental results on four hyperspectral datasets demonstrate that the proposed method achieves better classification performance than other state-of-the-art methods.

Shufang Xu

Papers

A Multidepth and Multibranch Network for Hyperspectral Target Detection Based on Band Selection

Classification of hyperspectral image based on dual-branch feature interaction network

Depthwise Separable Convolutional Autoencoders for Hyperspectral Image Change Detection

Multimodal Transformer Network for Hyperspectral and LiDAR Classification

Lightweight Spatial-Spectral Network Based on 3D-2D Multi-Group Feature Extraction Module for Hyperspectral Image Classification