Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for hyperspectral image classification tasks. Training deep neural networks, such as a convolutional neural network for classification requires a large number of labeled samples. However, in remote sensing applications, we usually only have a small amount of labeled data for training because they are expensive to collect, although we still have abundant unlabeled data. In this paper, we propose semi-supervised deep learning for hyperspectral image classification—our approach uses limited labeled data and abundant unlabeled data to train a deep neural network. More specifically, we use deep convolutional recurrent neural networks (CRNN) for hyperspectral image classification by treating each hyperspectral pixel as a spectral sequence. In the proposed semi-supervised learning framework, the abundant unlabeled data are utilized with their pseudo labels (cluster labels). We propose to use all the training data together with their pseudo labels to pre-train a deep CRNN, and then fine-tune using the limited available labeled data. Further, to utilize spatial information in the hyperspectral images, we propose a constrained Dirichlet process mixture model (C-DPMM), a non-parametric Bayesian clustering algorithm, for semi-supervised clustering which includes pairwise must-link and cannot-link constraints—this produces high-quality pseudo-labels, resulting in improved initialization of the deep neural network. We also derived a variational inference model for the C-DPMM for efficient inference. Experimental results with real hyperspectral image data sets demonstrate that the proposed semi-supervised method outperforms state-of-the-art supervised and semi-supervised learning methods for hyperspectral classification.

Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral–spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral–spatial feature learning. Third, a sequential spectral–spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).

Residual Spectral–Spatial Attention Network for Hyperspectral Image Classification

In recent years, deep learning based methods have attracted broad attention in the field of hyperspectral image classification. However, due to the massive parameters and the complex network structure, deep learning methods may not perform well when only few training samples are available. In this paper, we propose a small-scale data based method, multi-grained network (MugNet), to explore the application of deep learning approaches in hyperspectral image classification. MugNet could be considered as a simplified deep learning model which mainly targets at limited samples based hyperspectral image classification. Three novel strategies are proposed to construct MugNet. First, the spectral relationship among different bands, as well as the spatial correlation within neighboring pixels, are both utilized via a multi-grained scanning approach. The proposed multi-grained scanning strategy could not only extract the joint spectral-spatial information, but also combine different grains’ spectral and spatial relationship. Second, because there are abundant unlabeled pixels available in hyperspectral images, we take full advantage of these samples, and adopt a semi-supervised manner in the process of generating convolution kernels. At last, the MugNet is built upon the basis of a very simple network which does not include many hyperparameters for tuning. The performance of MugNet is evaluated on a popular and two challenging data sets, and comparison experiments with several state-of-the-art hyperspectral image classification methods are revealed.

MugNet: Deep learning for hyperspectral image classification using limited samples

Collecting labeled samples is quite costly and time-consuming for hyperspectral image (HSI) classification task. Semisupervised learning framework, which combines the intrinsic information of labeled and unlabeled samples, can alleviate the deficient labeled samples and increase the accuracy of HSI classification. In this letter, we propose a novel semisupervised learning framework that is based on spectral–spatial graph convolutional networks ( $\text{S}^{2}$ GCNs). It explicitly utilizes the adjacency nodes in graph to approximate the convolution. In the process of approximate convolution on graph, the proposed method makes full use of the spatial information of the current pixel. The experimental results on three real-life HSI data sets, i.e., Botswana Hyperion, Kennedy Space Center, and Indian Pines, show that the proposed $\text{S}^{2}$ GCN can significantly improve the classification accuracy. For instance, the overall accuracy on Indian data is increased from 66.8% (GCN) to 91.6%.

Spectral–Spatial Graph Convolutional Networks for Semisupervised Hyperspectral Image Classification

With the rapid development of spectral imaging techniques, classification of hyperspectral images (HSIs) has attracted great attention in various applications such as land survey and resource monitoring in the field of remote sensing. A key challenge in HSI classification is how to explore effective approaches to fully use the spatial–spectral information provided by the data cube. Multiple kernel learning (MKL) has been successfully applied to HSI classification due to its capacity to handle heterogeneous fusion of both spectral and spatial features. This approach can generate an adaptive kernel as an optimally weighted sum of a few fixed kernels to model a nonlinear data structure. In this way, the difficulty of kernel selection and the limitation of a fixed kernel can be alleviated. Various MKL algorithms have been developed in recent years, such as the general MKL, the subspace MKL, the nonlinear MKL, the sparse MKL, and the ensemble MKL. The goal of this paper is to provide a systematic review of MKL methods, which have been applied to HSI classification. We also analyze and evaluate different MKL algorithms and their respective characteristics in different cases of HSI classification cases. Finally, we discuss the future direction and trends of research in this area.

Multiple Kernel Learning for Hyperspectral Image Classification: A Review

In this letter, we propose a new spatio-spectral Laplacian support vector machine (SS-LapSVM) for semi-supervised hyperspectral image classification. The clustering assumption on spectral vectors is used to formulate a manifold regularizer, and neighborhood spatial constraints of hyperspectral images are designed to construct a spatial regularizer. Moreover, a non-iterative optimization procedure is presented to solve this dual-regularized SVM, which makes rapid classification possible. By combining spatial and spectral information together, SS-LapSVM can avoid the speckle-like misclassification of hyperspectral images in the original Lap-SVM. The performance of SS-LapSVM is evaluated on AVIRIS image data taken over Indiana's Indian Pine, and the results show that it can achieve accurate and rapid classification with a small number of labeled data, and outperform state-of-the-art semi-supervised approaches.

Semi-Supervised Hyperspectral Image Classification Using Spatio-Spectral Laplacian Support Vector Machine

Tensor representation is the most natural and effective way to preserve the structural information of hyperspectral image (HSI), and thus is very beneficial to HSI processing. This paper represents the spectral features of each testing pixel and its spatial neighbors as a Spatial Neighbor Tensor (SNT), whose spectral vectors can be simultaneously sparse coded by the spectral vectors of a few common training pixels. The obtained sparse coding coefficients could be regarded as a Slice Sparse Coding Tensor (SSCT), which can be adaptively learnt and utilized to predict the labels of HSI pixels. Furthermore, to improve the efficiency, the Compressive Dimensionality Reduction (CDR) is introduced into tensor slice sparse coding for optimizing SSCT and thus an SSCT based Classifier with CDR (SSCTC-CDR) is proposed for hyperspectral image classification (HIC). The performance of SSCTC-CDR is evaluated on three real HSI data, and the results show that it can obtain high-accurate classification result with relatively low computation lost.

/pdf/hyperspectral-image-classification-via-slice-sparse-coding-1aygbu0732.pdf

Lixia Yang

Papers

Semi-Supervised Hyperspectral Image Classification Using Spatio-Spectral Laplacian Support Vector Machine

Hyperspectral Image Classification via Slice Sparse Coding Tensor Based Classifier With Compressive Dimensionality Reduction