IEEE Transactions on Pattern Analysis and Machine Intelligence

A hyperspectral imaging sensor collects detailed spectral responses from ground objects using hundreds of narrow bands; this technology is used in many real-world applications. Band selection aims to select a small subset of hyperspectral bands to remove spectral redundancy and reduce computational costs while preserving the significant spectral information of ground objects. In this article, we review current hyperspectral band selection methods, which can be classified into six main categories: ranking based, searching based, clustering based, sparsity based, embedding-learning based, and hybrid-scheme based. With two widely used hyperspectral data sets, we illustrate the classification performances of several popular band selection methods. The challenges and research directions of hyperspectral band selection are also discussed.

Hyperspectral Band Selection: A Review

Semantic segmentation of remote sensing imagery has been employed in many applications and is a key research topic for decades. With the success of deep learning methods in the field of computer vision, researchers have made a great effort to transfer their superior performance to the field of remote sensing image analysis. This paper starts with a summary of the fundamental deep neural network architectures and reviews the most recent developments of deep learning methods for semantic segmentation of remote sensing imagery including non-conventional data such as hyperspectral images and point clouds. In our review of the literature, we identified three major challenges faced by researchers and summarize the innovative development to address them. As tremendous efforts have been devoted to advancing pixel-level accuracy, the emerged deep learning methods demonstrated much-improved performance on several public data sets. As to handling the non-conventional, unstructured point cloud and rich spectral imagery, the performance of the state-of-the-art methods is, on average, inferior to that of the satellite imagery. Such a performance gap also exists in learning from small data sets. In particular, the limited non-conventional remote sensing data sets with labels is an obstacle to developing and evaluating new deep learning methods.

A review of deep learning methods for semantic segmentation of remote sensing imagery

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral–spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral–spatial feature learning. Third, a sequential spectral–spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).

Residual Spectral–Spatial Attention Network for Hyperspectral Image Classification

In this paper, we propose a remote-sensing scene-classification method based on vision transformers. These types of networks, which are now recognized as state-of-the-art models in natural language processing, do not rely on convolution layers as in standard convolutional neural networks (CNNs). Instead, they use multihead attention mechanisms as the main building block to derive long-range contextual relation between pixels in images. In a first step, the images under analysis are divided into patches, then converted to sequence by flattening and embedding. To keep information about the position, embedding position is added to these patches. Then, the resulting sequence is fed to several multihead attention layers for generating the final representation. At the classification stage, the first token sequence is fed to a softmax classification layer. To boost the classification performance, we explore several data augmentation strategies to generate additional data for training. Moreover, we show experimentally that we can compress the network by pruning half of the layers while keeping competing classification accuracies. Experimental results conducted on different remote-sensing image datasets demonstrate the promising capability of the model compared to state-of-the-art methods. Specifically, Vision Transformer obtains an average classification accuracy of 98.49%, 95.86%, 95.56% and 93.83% on Merced, AID, Optimal31 and NWPU datasets, respectively. While the compressed version obtained by removing half of the multihead attention layers yields 97.90%, 94.27%, 95.30% and 93.05%, respectively.

Vision Transformers for Remote Sensing Image Classification

Generative adversarial networks (GANs) are famous for generating samples by training a generator and a discriminator via an adversarial procedure. For hyperspectral image classification, the collection of samples is always difficult. However, directly applying GAN to hyperspectral image classification exists two problems. One is that the generated samples lack discriminative information. Meanwhile, the discriminator has no discriminative ability for multiclassification. Another is that spatial and spectral information requires to be considered in hyperspectral image classification simultaneously. To address these problems, a novel multiclass spatial–spectral GAN (MSGAN) method is proposed. In MSGAN, two generators are devised to generate the samples containing spatial and spectral information, respectively, and the discriminator is devised to extract joint spatial–spectral features and output multiclass probabilities. Moreover, novel adversarial objectives for multiclass are defined. The discriminator is devised to predict training samples belonging to true classes and generated samples belonging to all the classes with the same probability. The generators are devised to make the discriminator mistake. By adversarial learning between the discriminator and generators, the classification performance of the discriminator is promoted with the assistance of discriminative generated samples. Experimental results on hyperspectral images demonstrate that the proposed method achieves encouraging classification performance compared with several state-of-the-art methods, especially with the limited training samples.

Classification of Hyperspectral Images Based on Multiclass Spatial–Spectral Generative Adversarial Networks

Hyperspectral image classification faces various levels of difficulty due to the use of different types of hyperspectral image data. Recently, spectral–spatial approaches have been developed by jointly taking care of spectral and spatial information. This paper presents a completely different approach from a subpixel target detection view point. It implements four stage processes, a preprocessing stage, which uses band selection (BS) and nonlinear band expansion, referred to as BS-then-nonlinear expansion (BSNE), a detection stage, which implements constrained energy minimization (CEM) to produce subpixel target maps, and an iterative stage, which develops an iterative CEM (ICEM) by applying Gaussian filters to capture spatial information, and then feeding the Gaussian-filtered CEM-detection maps back to BSNE band images to reprocess CEM in an iterative manner. Finally, in the last stage Otsu’s method is applied to converting ICEM-detected real-valued maps to discrete values for classification. The entire process is called BSNE-ICEM. Experimental results demonstrate BSNE-ICEM, which has advantages over support vector machine-based approaches in many aspects, such as easy implementation, fewer parameters to be used, and better false classification and precision rates.

A Subpixel Target Detection Approach to Hyperspectral Image Classification

This paper presents a new approach, called band subset selection (BSS)-based hyperspectral anomaly detection (AD), which selects multiple bands simultaneously as a band subset rather than selecting multiple bands one at a time as the tradition band selection (BS) does, referred to as sequential multiple BS (SQMBS). Its idea is to first use virtual dimensionality (VD) to determine the number of multiple bands, $n_{\mathrm {BS}}$ needed to be selected as a band subset and then develop two iterative process, sequential BSS (SQ-BSS) algorithm and successive BSS (SC-BSS) algorithm to find an optimal band subset numerically among all possible $n_{\mathrm {BS}}$ combinations out of the full band set. In order to terminate the search process the averaged least-squares error (ALSE) and 3-D receiver operating characteristic (3D ROC) curves are used as stopping criteria to evaluate performance relative to AD using the full band set. Experimental results demonstrate that BSS generally performs better background suppression while maintaining target detection capability compared to target detection using full band information.

Band Subset Selection for Anomaly Detection in Hyperspectral Imagery

Anomaly detection (AD) generally finds targets that are spectrally distinct from their surrounding neighborhoods but cannot discriminate its detected targets one from another. It cannot even perform classification because there is no prior knowledge about the data. This paper presents a new approach to AD, to be called a posteriori AD for unlabeled anomaly classification where a posteriori indicates that information obtained directly from processing data is used as new information for subsequent data processing. In particular, a posteriori AD uses a Gaussian filter to capture spatial correlation of detected anomalies as a posteriori information which is included as new information for further AD. In doing so, a posteriori AD develops an iterative version of AD, referred to as iterative anomaly detection (IAD), which implements AD by feeding back Gaussian-filtered AD maps in an iterative manner. It then uses an unsupervised target detection algorithm to identify spectrally distinct anomalies that can be used to specify particular anomaly classes. To terminate IAD, an automatic stopping rule is also derived. Finally, it uses identified distinct anomalies as desired target signatures to implement constrained energy minimization (CEM) to classify all detected anomalies into unlabeled classes. The experimental results show that a posteriori AD is indeed very effective in unlabeled anomaly classification.

A Posteriori Hyperspectral Anomaly Detection for Unlabeled Classification

This paper develops a new approach to band selection for multiple-target detection, called constrained-target band selection (CTBS). Its idea is derived from the concept of constrained energy minimization (CEM) by constraining a target of interest, while minimizing the variance resulting from the background (BKG). By taking advantage of CEM, the variance produced by a target of interest can be further used as a measure of prioritizing bands as well as a means of selecting bands for this particular target. As a result, two CTBS-based band prioritization (BP) criteria, called minimal variance-based BP (MinV-BP) and maximal variance-based BP (MaxV-BP), and two CTBS-based BS methods, called sequential forward CTBS (SF-CTBS) and sequential backward CTBS (SB-CTBS), can be derived for multiple-target detection. Since the bands selected by CTBS vary with targets of interest used to constrain CEM, in order for CTBS to be applied to multiple targets, a new fusion technique, called band fusion selection (BFS), is further developed for CTBS to integrate bands selected by different targets so that CTBS can work for all targets. Unlike most BS methods for target detection which generally simultaneously select a fixed set of bands for all targets of interest, the ideas of constraining multiple-target detection and using BFS are novelty of this paper. Experimental results show that CTBS performs well for multiple-target detection.

Lin Wang

Papers

Classification of Hyperspectral Images Based on Multiclass Spatial–Spectral Generative Adversarial Networks

A Subpixel Target Detection Approach to Hyperspectral Image Classification

Band Subset Selection for Anomaly Detection in Hyperspectral Imagery

A Posteriori Hyperspectral Anomaly Detection for Unlabeled Classification

Constrained-Target Band Selection for Multiple-Target Detection