Advances in computing technology have fostered the development of new and powerful deep learning (DL) techniques, which have demonstrated promising results in a wide range of applications. Particularly, DL methods have been successfully used to classify remotely sensed data collected by Earth Observation (EO) instruments. Hyperspectral imaging (HSI) is a hot topic in remote sensing data analysis due to the vast amount of information comprised by this kind of images, which allows for a better characterization and exploitation of the Earth surface by combining rich spectral and spatial information. However, HSI poses major challenges for supervised classification methods due to the high dimensionality of the data and the limited availability of training samples. These issues, together with the high intraclass variability (and interclass similarity) –often present in HSI data– may hamper the effectiveness of classifiers. In order to solve these limitations, several DL-based architectures have been recently developed, exhibiting great potential in HSI data interpretation. This paper provides a comprehensive review of the current-state-of-the-art in DL for HSI classification, analyzing the strengths and weaknesses of the most widely used classifiers in the literature. For each discussed method, we provide quantitative results using several well-known and widely used HSI scenes, thus providing an exhaustive comparison of the discussed techniques. The paper concludes with some remarks and hints about future challenges in the application of DL techniques to HSI classification. The source codes of the methods discussed in this paper are available from: https://github.com/mhaut/hyperspectral_deeplearning_review .

Deep learning classifiers for hyperspectral imaging: A review

In recent years, deep learning (DL), a rebranding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, and natural language processing. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV, e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should not only be aware of advancements such as DL, but also be leading researchers in this area. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools, and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as they relate to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) nontraditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community

Convolutional neural network (CNN) has demonstrated impressive ability to represent hyperspectral images and to achieve promising results in hyperspectral image classification. However, traditional CNN models can only operate convolution on regular square image regions with fixed size and weights, and thus, they cannot universally adapt to the distinct local regions with various object distributions and geometric appearances. Therefore, their classification performances are still to be improved, especially in class boundaries. To alleviate this shortcoming, we consider employing the recently proposed graph convolutional network (GCN) for hyperspectral image classification, as it can conduct the convolution on arbitrarily structured non-Euclidean data and is applicable to the irregular image regions represented by graph topological information. Different from the commonly used GCN models that work on a fixed graph, we enable the graph to be dynamically updated along with the graph convolution process so that these two steps can be benefited from each other to gradually produce the discriminative embedded features as well as a refined graph. Moreover, to comprehensively deploy the multiscale information inherited by hyperspectral images, we establish multiple input graphs with different neighborhood scales to extensively exploit the diversified spectral–spatial correlations at multiple scales. Therefore, our method is termed multiscale dynamic GCN (MDGCN). The experimental results on three typical benchmark data sets firmly demonstrate the superiority of the proposed MDGCN to other state-of-the-art methods in both qualitative and quantitative aspects.

Multiscale Dynamic Graph Convolutional Network for Hyperspectral Image Classification

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

Change detection based on remote sensing (RS) images has a wide range of applications in many fields. However, many existing approaches for detecting changes in RS images with complex land covers still have room for improvement. In this article, a high-resolution RS image change detection approach based on a deep feature difference convolutional neural network (CNN) is proposed. This approach uses a CNN to learn the deep features from RS images and then uses transfer learning to compose a two-channel network with shared weight to generate a multiscale and multidepth feature difference map for change detection. The network is trained by a change magnitude guided loss function proposed in this article and needs to train with only a few pixel-level samples to generate change magnitude maps, which can help to remove some of the pseudochanges. Finally, the binary change map can be obtained by a threshold. The approach is tested on several data sets from different sensors, including WorldView-3, QuickBird, and Ziyuan-3. The experimental results show that the proposed approach achieves better performance compared with other classic approaches and has fewer missed detections and false alarms, which proves that the proposed approach has strong robustness and generalization ability.

A Feature Difference Convolutional Neural Network-Based Change Detection Method

A novel deep learning classification method for hyperspectral data based on convolutional neural network is proposed in this paper. Deep learning means bringing multiple layers instead of one to the structure. Through convolution layers and pooling layers, the features in different layers are extracted from original spectral feature images. The key of this method is to restructure spectral feature images and choose convolution filters with a reasonable size, so that the spectral features of different land coverings in high dimensions can be extracted properly. In our experiments, proposed method was applied for hyperspectral data in several different situations, and preferable classification performance were obtained through relative parameters adjustment, which were given recommended scope during our comparative experiments.

Convolutional neural network based classification for hyperspectral data

In order to improve the classification accuracy of hyperspectral image, its internal relations and mutual relations should be paid attention to. This paper proposes a compound non-linear dimensionality reduction method with the help of non-negative matrix factorization (NMF) to show the relationships between samples within the same class and locality preserving projections (LPP) to show the relationships between classes. Combining with the advantages of the two algorithms, the proposed method makes data samples within a class closer and data samples between different classes farther after the projection by containing orthogonal constraints for datasets. Together with maximum-likelihood classifiers, all the methods mentioned above constitute the classification method for real hyperspectral data. Experimental results show its advantage in parameter adaption and classification accuracy.

Hyperspectral image classification method based on orthogonal NMF and LPP

Manifold learning based unsupervised classification methods will be unable to obtain satisfactory results because of the lack of training samples. The employment of training samples' information makes manifold learning based classification become supervised, and thus brings the improvement on classification accuracy. In order to make full use of this information, we emphatically consider the hyperspectral data distribute by clusters. A novel supervised manifold learning method termed class encoding is proposed for hyperspectral data classification. The experimental results show that this algorithm has better classification performance than the existing supervised manifold learning algorithm.

Manifold learning based supervised hyperspectral data classification method using class encoding

Carbon emissions caused by the massive consumption of energy have brought enormous pressure on the Chinese government. Accurately and rapidly characterizing the spatiotemporal characteristics of Chinese city-level carbon emissions is crucial for policy decision making. Based on multi-dimensional data, including nighttime light (NTL) data, land use (LU) data, land surface temperature (LST) data, and added-value secondary industry (AVSI) data, a deep neural network ensemble (DNNE) model was built to analyze the nonlinear relationship between multi-dimensional data and province-level carbon emission statistics (CES) data. The city-level carbon emissions data were estimated, and the spatiotemporal characteristics were analyzed. As compared to the energy statistics released by partial cities, the results showed that the DNNE model based on multi-dimensional data could well estimate city-level carbon emissions data. In addition, according to a linear trend analysis and standard deviational ellipse (SDE) analysis of China from 2001 to 2019, we concluded that the spatiotemporal changes in carbon emissions at the city level were in accordance with the development of China’s economy. Furthermore, the results can provide a useful reference for the scientific formulation, implementation, and evaluation of carbon emissions reduction policies.

/pdf/carbon-emissions-estimation-and-spatiotemporal-analysis-of-ih7014f5.pdf

Carbon Emissions Estimation and Spatiotemporal Analysis of China at City Level Based on Multi-Dimensional Data and Machine Learning

To address the multi-classification problems of hyperspectral dataset, a new method with weighted kernel function based on Chernoff distance is proposed. Chernoff distance utilizes the information between categories and strengthens the separability of original dataset. The adjustable parameter in Chernoff distance can fit the hyperspectral dataset well compared with other least upper bounds. Pairwise decision tree reduces the number of subclassifiers that the dataset requires and improves the classification accuracy. The guidance of the weighed subclassifiers is global separability metric computed by Chernoff distance. Weighted subclassifiers highlight bands with more useful information and reduce accumulative error. Comparative experiment shows the effectiveness of the proposed method.

Fei Shen

Papers

Convolutional neural network based classification for hyperspectral data

Hyperspectral image classification method based on orthogonal NMF and LPP

Manifold learning based supervised hyperspectral data classification method using class encoding

Carbon Emissions Estimation and Spatiotemporal Analysis of China at City Level Based on Multi-Dimensional Data and Machine Learning

Multiclassification method for hyperspectral data based on Chernoff distance and pairwise decision tree strategy