Graph Convolutional Networks for Hyperspectral Image Classification
TL;DR: A new minibatch GCN is developed that is capable of inferring out-of-sample data without retraining networks and improving classification performance, and three fusion strategies are explored: additive fusion, elementwise multiplicative fusion, and concatenation fusion to measure the obtained performance gain.
Abstract: Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification due to their ability to capture spatial–spectral feature representations. Nevertheless, their ability in modeling relations between the samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or nongrid) data representation and analysis. In this article, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new minibatch GCN (called miniGCN hereinafter), which allows to train large-scale GCNs in a minibatch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without retraining networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batchwise network training (enabling the combination of CNNs and GCNs), we explore three fusion strategies: additive fusion, elementwise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS data sets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regard to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.
Citations
More filters
TL;DR: This article presents an attention-based adaptive spectral-spatial kernel improved residual network (A²S²K-ResNet) with spectral attention to capture discriminative spectral- Spatial features for HSI classification in an end-to-end training fashion.
Abstract: Hyperspectral images (HSIs) provide rich spectral–spatial information with stacked hundreds of contiguous narrowbands. Due to the existence of noise and band correlation, the selection of informative spectral–spatial kernel features poses a challenge. This is often addressed by using convolutional neural networks (CNNs) with receptive field (RF) having fixed sizes. However, these solutions cannot enable neurons to effectively adjust RF sizes and cross-channel dependencies when forward and backward propagations are used to optimize the network. In this article, we present an attention-based adaptive spectral–spatial kernel improved residual network ( A2S2K-ResNet ) with spectral attention to capture discriminative spectral–spatial features for HSI classification in an end-to-end training fashion. In particular, the proposed network learns selective 3-D convolutional kernels to jointly extract spectral–spatial features using improved 3-D ResBlocks and adopts an efficient feature recalibration (EFR) mechanism to boost the classification performance. Extensive experiments are performed on three well-known hyperspectral data sets, i.e., IP, KSC, and UP, and the proposed A2S2K-ResNet can provide better classification results in terms of overall accuracy (OA), average accuracy (AA), and Kappa compared with the existing methods investigated. The source code will be made available at https://github.com/suvojit- $0\times 55$ aa/A2S2K-ResNet.
185 citations
TL;DR: In this paper, the authors proposed the concepts of connected vehicles that exploit vehicular ad hoc network (VANET) communication, embedded system integrated with sensors which acquire the static and dynamic parameter of the electrical vehicle, and cloud integration and dig data analytics tools.
Abstract: Testing and implementation of integrated and intelligent transport systems (IITS) of an electrical vehicle need many high-performance and high-precision subsystems. The existing systems confine themselves with limited features and have driving range anxiety, charging and discharging time issues, and inter- and intravehicle communication problems. The above issues are the critical barriers to the penetration of EVs with a smart grid. This paper proposes the concepts which consist of connected vehicles that exploit vehicular ad hoc network (VANET) communication, embedded system integrated with sensors which acquire the static and dynamic parameter of the electrical vehicle, and cloud integration and dig data analytics tools. Vehicle control information is generated based on machine learning-based control systems. This paper also focuses on improving the overall performance (discharge time and cycle life) of a lithium ion battery, increasing the range of the electric vehicle, enhancing the safety of the battery that acquires the static and dynamic parameter and driving pattern of the electrical vehicle, establishing vehicular ad hoc network (VANET) communication, and handling and analyzing the acquired data with the help of various artificial big data analytics techniques.
173 citations
TL;DR: Compared to convex models, nonconvex modeling, which is capable of characterizing more complex real scenes and providing model interpretability technically and theoretically, has proven to be a feasible solution that reduces the gap between challenging HS vision tasks and currently advanced intelligent data processing models.
Abstract: Hyperspectral (HS) imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these HS products, mainly by seasoned experts. However, with an ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges for reducing the burden of manual labor and improving efficiency. For this reason, it is urgent that more intelligent and automatic approaches for various HS RS applications be developed. Machine learning (ML) tools with convex optimization have successfully undertaken the tasks of numerous artificial intelligence (AI)-related applications; however, their ability to handle complex practical problems remains limited, particularly for HS data, due to the effects of various spectral variabilities in the process of HS imaging and the complexity and redundancy of higher-dimensional HS signals. Compared to convex models, nonconvex modeling, which is capable of characterizing more complex real scenes and providing model interpretability technically and theoretically, has proven to be a feasible solution that reduces the gap between challenging HS vision tasks and currently advanced intelligent data processing models.
168 citations
Cites methods from "Graph Convolutional Networks for Hy..."
...That is, the graph is built by using a radial basis function (RBF) to measure the similarities among samples belonging to the same class [81],...
[...]
TL;DR: Extensive experiments conducted demonstrate the superiority and advancement of the S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines.
Abstract: As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 – hyperspectral and multispectral data, Berlin – hyperspectral and synthetic aperture radar (SAR) data, Augsburg – hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL .
117 citations
TL;DR: In this article, the authors proposed an end-member-guided unmixing network (EGU-Net), which is a two-stream Siamese deep network that learns an additional network from the pure or nearly pure endmembers to correct the weights of another unmixer by sharing network parameters and adding spectrally meaningful constraints.
Abstract: Over the past decades, enormous efforts have been made to improve the performance of linear or nonlinear mixing models for hyperspectral unmixing (HU), yet their ability to simultaneously generalize various spectral variabilities (SVs) and extract physically meaningful endmembers still remains limited due to the poor ability in data fitting and reconstruction and the sensitivity to various SVs. Inspired by the powerful learning ability of deep learning (DL), we attempt to develop a general DL approach for HU, by fully considering the properties of endmembers extracted from the hyperspectral imagery, called endmember-guided unmixing network (EGU-Net). Beyond the alone autoencoder-like architecture, EGU-Net is a two-stream Siamese deep network, which learns an additional network from the pure or nearly pure endmembers to correct the weights of another unmixing network by sharing network parameters and adding spectrally meaningful constraints (e.g., nonnegativity and sum-to-one) toward a more accurate and interpretable unmixing solution. Furthermore, the resulting general framework is not only limited to pixelwise spectral unmixing but also applicable to spatial information modeling with convolutional operators for spatial-spectral unmixing. Experimental results conducted on three different datasets with the ground truth of abundance maps corresponding to each material demonstrate the effectiveness and superiority of the EGU-Net over state-of-the-art unmixing algorithms. The codes will be available from the website: https://github.com/danfenghong/IEEE_TNNLS_EGU-Net.
96 citations
References
More filters
Proceedings Article•
01 Jan 2015TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.
111,197 citations
"Graph Convolutional Networks for Hy..." refers methods in this paper
...Adam [43] is used to optimize the networks....
[...]
Posted Content•
TL;DR: A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin.
Abstract: We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.
15,696 citations
"Graph Convolutional Networks for Hy..." refers background or methods in this paper
...Comparatively, graph convolutional networks (GCNs) [32] are a hot topic and emerging network architecture, which is able to effectively handle graph structure data by modeling relations between samples (or vertexes)....
[...]
...By limiting K = 1 and assigning the largest eigenvalue λmax of L̃ to 2 [32], (15) can be further simplified to...
[...]
...7) For the GCN, similar to [32], a graph convolutional hidden layer with 128 units is implemented in the GCN before feeding the features into the softmax layer, where the adjacency matrix à can be computed using KNN-based graph (K = 10 in our case)....
[...]
01 Jan 1976
TL;DR: The framework of a national land use and land cover classification system is presented for use with remote sensor data and uses the features of existing widely used classification systems that are amenable to data derived from re-mote sensing sources.
Abstract: The framework of a national land use and land cover classification system is presented for use with remote sensor data. The classification system has been developed to meet the needs of Federal and State agencies for an up-to-date overview of land use and land cover throughout the country on a basis that is uniform in categorization at the more generalized first and second levels and that will be receptive to data from satellite and aircraft remote sensors. The pro-posed system uses the features of existing widely used classification systems that are amenable to data derived from re-mote sensing sources. It is intentionally left open-ended so that Federal, regional, State, and local agencies can have flexibility in developing more detailed land use classifications at the third and fourth levels in order to meet their particular needs and at the same time remain compatible with each other and the national system. Revision of the land use classification system as presented in US Geological Survey Circular 671 was undertaken in order to incorporate the results of extensive testing and review of the categorization and definitions.
4,154 citations
TL;DR: This paper presents an overview of un Mixing methods from the time of Keshava and Mustard's unmixing tutorial to the present, including Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixed algorithms.
Abstract: Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.
2,373 citations
"Graph Convolutional Networks for Hy..." refers background in this paper
...However, the high spectral mixing between materials [5] and spectral variability and complex noise effects [6] bring difficulties in extracting discriminative information from such data....
[...]
TL;DR: This paper proposes a 3-D CNN-based FE model with combined regularization to extract effective spectral-spatial features of hyperspectral imagery and reveals that the proposed models with sparse constraints provide competitive results to state-of-the-art methods.
Abstract: Due to the advantages of deep learning, in this paper, a regularized deep feature extraction (FE) method is presented for hyperspectral image (HSI) classification using a convolutional neural network (CNN). The proposed approach employs several convolutional and pooling layers to extract deep features from HSIs, which are nonlinear, discriminant, and invariant. These features are useful for image classification and target detection. Furthermore, in order to address the common issue of imbalance between high dimensionality and limited availability of training samples for the classification of HSI, a few strategies such as L2 regularization and dropout are investigated to avoid overfitting in class data modeling. More importantly, we propose a 3-D CNN-based FE model with combined regularization to extract effective spectral-spatial features of hyperspectral imagery. Finally, in order to further improve the performance, a virtual sample enhanced method is proposed. The proposed approaches are carried out on three widely used hyperspectral data sets: Indian Pines, University of Pavia, and Kennedy Space Center. The obtained results reveal that the proposed models with sparse constraints provide competitive results to state-of-the-art methods. In addition, the proposed deep FE opens a new window for further research.
2,059 citations
"Graph Convolutional Networks for Hy..." refers methods in this paper
...[27] adopted convolutional neural networks (CNNs) to extract spatial–spectral features more effectively from HS images, thereby yielding higher classification performance....
[...]