Hyperspectral Image Classification Based on Superpixel Pooling Convolutional Neural Network with Transfer Learning
TL;DR: Wang et al. as mentioned in this paper proposed an effective spectral-spatial HSI classification scheme based on superpixel pooling convolutional neural network with transfer learning (SP-CNN), which includes three stages.
Abstract: Deep learning-based hyperspectral image (HSI) classification has attracted more and more attention because of its excellent classification ability. Generally, the outstanding performance of these methods mainly depends on a large number of labeled samples. Therefore, it still remains an ongoing challenge how to integrate spatial structure information into these frameworks to classify the HSI with limited training samples. In this study, an effective spectral-spatial HSI classification scheme is proposed based on superpixel pooling convolutional neural network with transfer learning (SP-CNN). The suggested method includes three stages. The first part consists of convolution and pooling operation, which is a down-sampling process to extract the main spectral features of an HSI. The second part is composed of up-sampling and superpixel (homogeneous regions with adaptive shape and size) pooling to explore the spatial structure information of an HSI. Finally, the hyperspectral data with each superpixel as a basic input rather than a pixel are fed to fully connected neural network. In this method, the spectral and spatial information is effectively fused by using superpixel pooling technique. The use of popular transfer learning technology in the proposed classification framework significantly improves the training efficiency of SP-CNN. To evaluate the effectiveness of the SP-CNN, extensive experiments were conducted on three common real HSI datasets acquired from different sensors. With 30 labeled pixels per class, the overall classification accuracy provided by this method on three benchmarks all exceeded 93%, which was at least 4.55% higher than that of several state-of-the-art approaches. Experimental and comparative results prove that the proposed algorithm can effectively classify the HSI with limited training labels.
Citations
More filters
TL;DR: Its applications in agriculture are summarized, include ripeness and component prediction, different classification themes, and plant disease detection, and the prospects of future works are put forward.
Abstract: Hyperspectral imaging is a non-destructive, nonpolluting, and fast technology, which can capture up to several hundred images of different wavelengths and offer relevant spectral signatures. Hyperspectral imaging technology has achieved breakthroughs in the acquisition of agricultural information and the detection of external or internal quality attributes of the agricultural product. Deep learning techniques have boosted the performance of hyperspectral image analysis. Compared with traditional machine learning, deep learning architectures exploit both spatial and spectral information of hyperspectral image analysis. To scrutinize thoroughly the current efforts, provide insights, and identify potential research directions on deep learning for hyperspectral image analysis in agriculture, this paper presents a systematic and comprehensive review. Firstly, its applications in agriculture are summarized, include ripeness and component prediction, different classification themes, and plant disease detection. Then, the recent achievements are reviewed in hyperspectral image analysis from the aspects of the deep learning models and the feature networks. Finally, the existing challenges of hyperspectral image analysis based on deep learning are summarized and the prospects of future works are put forward.
76 citations
TL;DR: The squeeze and excitation (SE) network is combined with convolutional neural networks (SE-CNN) in this work to increase its performance in extracting features and classifying HSI.
Abstract: In the domain of remote sensing, the classification of hyperspectral image (HSI) has become a popular topic. In general, the complicated features of hyperspectral data cause the precise classification difficult for standard machine learning approaches. Deep learning-based HSI classification has lately received a lot of interest in the field of remote sensing and has shown promising results. As opposed to conventional hand-crafted feature-based classification approaches, deep learning can automatically learn complicated features of HSIs with a greater number of hierarchical layers. Because HSI's data structure is complicated, applying deep learning to it is difficult. The primary objective of this research is to propose a deep feature extraction model for HSI classification. Deep networks can extricate features of spatial and spectral from HSI data simultaneously, which is advantageous for increasing the performances of the proposed system. The squeeze and excitation (SE) network is combined with convolutional neural networks (SE-CNN) in this work to increase its performance in extracting features and classifying HSI. The squeeze and excitation block is designed to improve the representation quality of a CNN. Three benchmark datasets are utilized in the experiment to evaluate the proposed model: Pavia Centre, Pavia University, and Salinas. The proposed model's performance is validated by a performance comparison with current deep transfer learning approaches such as VGG-16, Inception-v3, and ResNet-50. In terms of accuracy on each class of datasets and overall accuracy, the proposed SE-CNN model outperforms the compared models. The proposed model achieved an overall accuracy of 96.05% for Pavia University, 98.94% for Pavia Centre dataset, and 96.33% for Salinas dataset.
64 citations
TL;DR: In this article, the authors proposed a deep learning based framework to enhance the diagnostic values of chest X-ray images for improved clinical outcomes, which is realized as a variant of the conventional SqueezeNet classifier with segmentation capabilities.
27 citations
TL;DR: Self-Matching class activation mapping (CAM) as discussed by the authors assigns a pixel-wise weight matrix to feature maps of different channels by matching them with the input SAR image, which can be well preserved in an accurate visual explanation heatmap of a CNN for SAR image interpretation.
Abstract: Synthetic aperture radar (SAR) image interpretation has long been an important but challenging task in SAR imaging processing. Generally, SAR image interpretation comprises complex procedures including filtering, feature extraction, image segmentation, and target recognition, which greatly reduce the efficiency of data processing. In an era of deep learning, numerous automatic target recognition methods have been proposed based on convolutional neural networks (CNNs) due to their strong capabilities for data abstraction and mining. In contrast to general methods, CNNs own an end-to-end structure where complex data preprocessing is not needed, thus the efficiency can be improved dramatically once a CNN is well trained. However, the recognition mechanism of a CNN is unclear, which hinders its application in many scenarios. In this paper, Self-Matching class activation mapping (CAM) is proposed to visualize what a CNN learns from SAR images to make a decision. Self-Matching CAM assigns a pixel-wise weight matrix to feature maps of different channels by matching them with the input SAR image. By using Self-Matching CAM, the detailed information of the target can be well preserved in an accurate visual explanation heatmap of a CNN for SAR image interpretation. Numerous experiments on a benchmark dataset (MSTAR) verify the validity of Self-Matching CAM.
19 citations
TL;DR: This study scrutinizes the efficacy of Artificial Neural Network, Maximum Likelihood Classifier (MLC) and Support Vector Machine (SVM) over hybrid datasets including optical, radar, DEMs and their derivatives and shows that SVM and MLC are much better than ANN.
16 citations
References
More filters
TL;DR: A new superpixel algorithm is introduced, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels and is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.
Abstract: Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.
7,849 citations
TL;DR: It can be shown that the order of accuracy of the cubic convolution method is between that of linear interpolation and that of cubic splines.
Abstract: Cubic convolution interpolation is a new technique for resampling discrete data. It has a number of desirable features which make it useful for image processing. The technique can be performed efficiently on a digital computer. The cubic convolution interpolation function converges uniformly to the function being interpolated as the sampling increment approaches zero. With the appropriate boundary conditions and constraints on the interpolation kernel, it can be shown that the order of accuracy of the cubic convolution method is between that of linear interpolation and that of cubic splines. A one-dimensional interpolation function is derived in this paper. A separable extension of this algorithm to two dimensions is applied to image data.
3,280 citations
TL;DR: Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than some traditional methods, such as support vector machines and the conventional deep learning-based methods.
Abstract: Recently, convolutional neural networks have demonstrated excellent performance on various visual tasks, including the classification of common two-dimensional images. In this paper, deep convolutional neural networks are employed to classify hyperspectral images directly in spectral domain. More specifically, the architecture of the proposed classifier contains five layers with weights which are the input layer, the convolutional layer, the max pooling layer, the full connection layer, and the output layer. These five layers are implemented on each spectral signature to discriminate against others. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than some traditional methods, such as support vector machines and the conventional deep learning-based methods.
1,316 citations
TL;DR: Experimental results based on several hyperspectral image data sets demonstrate that the proposed pixel-pair method can achieve better classification performance than the conventional deep learning-based method.
Abstract: The deep convolutional neural network (CNN) is of great interest recently. It can provide excellent performance in hyperspectral image classification when the number of training samples is sufficiently large. In this paper, a novel pixel-pair method is proposed to significantly increase such a number, ensuring that the advantage of CNN can be actually offered. For a testing pixel, pixel-pairs, constructed by combining the center pixel and each of the surrounding pixels, are classified by the trained CNN, and the final label is then determined by a voting strategy. The proposed method utilizing deep CNN to learn pixel-pair features is expected to have more discriminative power. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than the conventional deep learning-based method.
676 citations
TL;DR: This letter proposes to adaptively learn a suitable feature representation from unlabeled data by learning a feature mapping function based on stacked sparse autoencoder that embeds the learned spectral-spatial feature into a linear support vector machine for classification.
Abstract: In this letter, different from traditional methods using original spectral features or handcraft spectral–spatial features, we propose to adaptively learn a suitable feature representation from unlabeled data. This is achieved by learning a feature mapping function based on stacked sparse autoencoder. Considering that hyperspectral imagery (HSI) is intrinsically defined in both the spectral and spatial domains, we further establish two variants of feature learning procedures for sparse spectral feature learning and multiscale spatial feature learning. Finally, we embed the learned spectral–spatial feature into a linear support vector machine for classification. Experiments on two hyperspectral images indicate the following: 1) the learned spectral–spatial feature representation is more discriminative for HSI classification compared to previously hand-engineered spectral–spatial features, especially when the training data are limited and 2) the learned features appear not to be specific to a particular image but general in that they are applicable to multiple related images (e.g., images acquired by the same sensor but varying with location or time).
356 citations