Central to the looming paradigm shift toward data-intensive science, machine-learning techniques are becoming increasingly important. In particular, deep learning has proven to be both a major breakthrough and an extremely powerful tool in many fields. Shall we embrace deep learning as the key to everything? Or should we resist a black-box solution? These are controversial issues within the remote-sensing community. In this article, we analyze the challenges of using deep learning for remote-sensing data analysis, review recent advances, and provide resources we hope will make deep learning in remote sensing seem ridiculously simple. More importantly, we encourage remote-sensing scientists to bring their expertise into deep learning and use it as an implicit general model to tackle unprecedented, large-scale, influential challenges, such as climate change and urbanization.

/pdf/deep-learning-in-remote-sensing-a-comprehensive-review-and-3o9ov8r19z.pdf

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

In this paper, we designed an end-to-end spectral–spatial residual network (SSRN) that takes raw 3-D cubes as input data without feature engineering for hyperspectral image classification. In this network, the spectral and spatial residual blocks consecutively learn discriminative features from abundant spectral signatures and spatial contexts in hyperspectral imagery (HSI). The proposed SSRN is a supervised deep learning framework that alleviates the declining-accuracy phenomenon of other deep learning models. Specifically, the residual blocks connect every other 3-D convolutional layer through identity mapping, which facilitates the backpropagation of gradients. Furthermore, we impose batch normalization on every convolutional layer to regularize the learning process and improve the classification performance of trained models. Quantitative and qualitative results demonstrate that the SSRN achieved the state-of-the-art HSI classification accuracy in agricultural, rural–urban, and urban data sets: Indian Pines, Kennedy Space Center, and University of Pavia.

Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework which divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.

Deep Learning for Hyperspectral Image Classification: An Overview

Traditional imagery—provided, for example, by RGB and/or NIR sensors—has proven to be useful in many agroforestry applications. However, it lacks the spectral range and precision to profile materials and organisms that only hyperspectral sensors can provide. This kind of high-resolution spectroscopy was firstly used in satellites and later in manned aircraft, which are significantly expensive platforms and extremely restrictive due to availability limitations and/or complex logistics. More recently, UAS have emerged as a very popular and cost-effective remote sensing technology, composed of aerial platforms capable of carrying small-sized and lightweight sensors. Meanwhile, hyperspectral technology developments have been consistently resulting in smaller and lighter sensors that can currently be integrated in UAS for either scientific or commercial purposes. The hyperspectral sensors’ ability for measuring hundreds of bands raises complexity when considering the sheer quantity of acquired data, whose usefulness depends on both calibration and corrective tasks occurring in pre- and post-flight stages. Further steps regarding hyperspectral data processing must be performed towards the retrieval of relevant information, which provides the true benefits for assertive interventions in agricultural crops and forested areas. Considering the aforementioned topics and the goal of providing a global view focused on hyperspectral-based remote sensing supported by UAV platforms, a survey including hyperspectral sensors, inherent data processing and applications focusing both on agriculture and forestry—wherein the combination of UAV and hyperspectral sensors plays a center role—is presented in this paper. Firstly, the advantages of hyperspectral data over RGB imagery and multispectral data are highlighted. Then, hyperspectral acquisition devices are addressed, including sensor types, acquisition modes and UAV-compatible sensors that can be used for both research and commercial purposes. Pre-flight operations and post-flight pre-processing are pointed out as necessary to ensure the usefulness of hyperspectral data for further processing towards the retrieval of conclusive information. With the goal of simplifying hyperspectral data processing—by isolating the common user from the processes’ mathematical complexity—several available toolboxes that allow a direct access to level-one hyperspectral data are presented. Moreover, research works focusing the symbiosis between UAV-hyperspectral for agriculture and forestry applications are reviewed, just before the paper’s conclusions.

/pdf/hyperspectral-imaging-a-review-on-uav-based-sensors-data-h7zu88r0tz.pdf

Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.

/pdf/deep-learning-in-remote-sensing-a-review-3epn5qlsfr.pdf

Deep learning in remote sensing: a review

The deep convolutional neural network (CNN) is of great interest recently. It can provide excellent performance in hyperspectral image classification when the number of training samples is sufficiently large. In this paper, a novel pixel-pair method is proposed to significantly increase such a number, ensuring that the advantage of CNN can be actually offered. For a testing pixel, pixel-pairs, constructed by combining the center pixel and each of the surrounding pixels, are classified by the trained CNN, and the final label is then determined by a voting strategy. The proposed method utilizing deep CNN to learn pixel-pair features is expected to have more discriminative power. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than the conventional deep learning-based method.

Hyperspectral Image Classification Using Deep Pixel-Pair Features

In this letter, a novel anomaly detection framework with transferred deep convolutional neural network (CNN) is proposed. The framework is designed by considering the following facts: 1) a reference data with labeled samples are utilized, because no prior information is available about the image scene for anomaly detection and 2) pixel pairs are generated to enlarge the sample size, since the advantage of CNN can be realized only if the number of training samples is sufficient. A multilayer CNN is trained by using difference between pixel pairs generated from the reference image scene. Then, for each pixel in the image for anomaly detection, difference between pixel pairs, constructed by combining the center pixel and its surrounding pixels, is classified by the trained CNN with the result of similarity measurement. The detection output is simply generated by averaging these similarity scores. Experimental performance demonstrates that the proposed algorithm outperforms the classic Reed-Xiaoli and the state-of-the-art representation-based detectors, such as sparse representation-based detector (SRD) and collaborative representation-based detector.

Transferred Deep Learning for Anomaly Detection in Hyperspectral Imagery

An interesting target detection framework with transferred deep convolutional neural network (CNN) is proposed. For CNN, many labeled samples are needed to train the multi-layer network. However, for target detection tasks, only few target spectral signatures are available, or they are unknown in anomaly detection. In this work, we employ a reference data and further generate pixel-pairs to enlarge the sample size. A multi-layer CNN is trained by using difference between pixel-pairs generated from the reference image scene. During testing, there are two cases: (1) for anomaly detection, difference between pixel-pairs, constructed by combing the center pixel and its surrounding pixels, is classified by the trained CNN with result of similarity measurement; and (2) for supervised target detection, difference between pixel-pairs, constructed by combing the testing pixel and the known spectral signatures, is classified. The detection output is simply generated by averaging these similarity scores. Experimental performance demonstrates that the proposed strategy outperforms the classic detectors.

Transferred deep learning for hyperspectral target detection

Social recommendation systems leverage the abundant social information of users existing in the current Internet to mitigate the problem of data sparsity, ultimately enhancing recommendation performance. However, most existing recommendation systems that introduce social information ignore the negative messages passed by high-order neighbor nodes and aggregate messages without filtering, which results in a decline in the performance of the recommendation system. Considering this problem, we propose a novel social recommendation model based on graph neural networks (GNNs) called the preference-aware light graph convolutional network (PLGCN), which contains a subgraph construction module using unsupervised learning to classify users according to their embeddings and then assign users with similar preferences to a subgraph to filter useless or even negative messages from users with different preferences to attain even better recommendation performance. We also designed a feature aggregation module to better combine user embeddings with social and interaction information. In addition, we employ a lightweight GNN framework to aggregate messages from neighbors, removing nonlinear activation and feature transformation operations to alleviate the overfitting problem. Finally, we carried out comprehensive experiments using two publicly available datasets, and the results indicate that PLGCN outperforms the current state-of-the-art (SOTA) method, especially in dealing with the problem of cold start. The proposed model has the potential for practical applications in online recommendation systems, such as e-commerce, social media, and content recommendation.

/pdf/preference-aware-light-graph-convolution-network-for-social-1r1h5262.pdf

Guodong Wu

Papers

Hyperspectral Image Classification Using Deep Pixel-Pair Features

Transferred Deep Learning for Anomaly Detection in Hyperspectral Imagery

Transferred deep learning for hyperspectral target detection

Preference-Aware Light Graph Convolution Network for Social Recommendation

Pig Audio Recognition Based on Deep Neural Network (Dnn) and Hidden Markov Models (Hmm)