Each sample in the hyperspectral remote sensing image has high-dimensional features and contains rich spatial and spectral information, which greatly increases the difficulty of feature selection and mining. In view of these difficulties, we propose a novel Triple-attention Guided Residual Dense and BiLSTM networks(TARDB-Net) to reduce redundant features while increasing feature fusion capabilities, which ultimately improves the ability to classify hyperspectral images. First, a novel Triple-attention mechanism is proposed to assign different weights to each feature. Then, the residual network is used to perform the residual operation on the features, and the initial features of the multiple residual blocks and the generated deep residual features are intensively fused, retaining a host number of prior features. And use the bidirectional long short-term memory network to integrate the contextual semantics of deep fusion features. Finally, the classification task is completed by Softmax classifier. Experiments on three hyperspectral datasets—Indian Pines, University of Pavia, and Salinas—show that under 10% of the training samples, the overall accuracy of our method is 87%, 96% and 96%, which is superior to several well-known methods.

TARDB-Net: triple-attention guided residual dense and BiLSTM networks for hyperspectral image classification

Pansharpening is normally utilized to take full advantage of all the available spectral and spatial information that are derived from a low-spatial-resolution multispectral (MS) image and its associated high-spatial-resolution (HR) panchromatic (PAN) image, respectively, producing a fused MS image with high spectral and spatial resolutions. Many methods have been recently developed based on convolutional neural networks (CNNs) for the pansharpening task, but most of them still have some drawbacks: 1) The information cannot efficiently flow in their simple stacked convolutional architectures, thereby hindering the representation ability of the networks. 2) They are commonly trained using supervised learning, which does not only require an extra effort to produce the simulated training data, but can also lead to scale-related problems in the fusion results. In this article, we propose a novel unsupervised CNN-based pansharpening method to overcome these limitations. Specifically, we design an iterative network architecture, in which a PAN-guided strategy and a set of skip connections are adopted to continuously extract and fuse the features from the input, thus enhancing the information reuse and transmission. Besides, we propose a new loss function for unsupervised training in which the relationships between the input MS and PAN images and the fused MS image are used to design the spatial constrains and spectral consistency, respectively. The typical quality index with no-reference is also added to this function to further adjust the spectral and spatial qualities. The designed loss function allows the network to be learned only on input images, without any hand-crafted labels (reference HR MS image). We evaluated the effectiveness of our designed network architecture and the combined loss function, and the experiments testify that our unsupervised strategy can also obtain promising results with minor spectral and spatial distortions compared with other traditional and supervised methods.

/pdf/pansharpening-via-unsupervised-convolutional-neural-networks-1bidg8mr2z.pdf

Pansharpening via Unsupervised Convolutional Neural Networks

A comprehensive review on deep learning based remote sensing image super-resolution methods

Convolutional neural networks have been proven to be of great benefit for single-image super-resolution (SISR). However, previous works do not make full use of multi-scale features and ignore the inter-scale correlation between different upsampling factors, resulting in sub-optimal performance. Instead of blindly increasing the depth of the network, we are committed to mining image features and learning the inter-scale correlation between different upsampling factors. To achieve this, we propose a Multi-scale Dense Cross Network (MDCN), which achieves great performance with fewer parameters and less execution time. MDCN consists of multi-scale dense cross blocks (MDCBs), hierarchical feature distillation block (HFDB), and dynamic reconstruction block (DRB). Among them, MDCB aims to detect multi-scale features and maximize the use of image features flow at different scales, HFDB focuses on adaptively recalibrate channel-wise feature responses to achieve feature distillation, and DRB attempts to reconstruct SR images with different upsampling factors in a single model. It is worth noting that all these modules can run independently. It means that these modules can be selectively plugged into any CNN model to improve model performance. Extensive experiments show that MDCN achieves competitive results in SISR, especially in the reconstruction task with multiple upsampling factors. The code is provided at https://github.com/MIVRC/MDCN-PyTorch .

MDCN: Multi-Scale Dense Cross Network for Image Super-Resolution

Convolutional neural networks have been proven to be of great benefit for single-image super-resolution (SISR). However, previous works do not make full use of multi-scale features and ignore the inter-scale correlation between different upsampling factors, resulting in sub-optimal performance. Instead of blindly increasing the depth of the network, we are committed to mining image features and learning the inter-scale correlation between different upsampling factors. To achieve this, we propose a Multi-scale Dense Cross Network (MDCN), which achieves great performance with fewer parameters and less execution time. MDCN consists of multi-scale dense cross blocks (MDCBs), hierarchical feature distillation block (HFDB), and dynamic reconstruction block (DRB). Among them, MDCB aims to detect multi-scale features and maximize the use of image features flow at different scales, HFDB focuses on adaptively recalibrate channel-wise feature responses to achieve feature distillation, and DRB attempts to reconstruct SR images with different upsampling factors in a single model. It is worth noting that all these modules can run independently. It means that these modules can be selectively plugged into any CNN model to improve model performance. Extensive experiments show that MDCN achieves competitive results in SISR, especially in the reconstruction task with multiple upsampling factors. The code will be provided at this https URL.

MDCN: Multi-scale Dense Cross Network for Image Super-Resolution

Remote sensing image super-resolution has always been a major research focus, and many deep-learning-based algorithms have been proposed in recent years. However, since the structure of remote sensing images tends to be much more complex than that of natural images, several difficulties still remain for remote sensing images super-resolution. First, it is difficult to depict the nonlinear mapping between high-resolution (HR) and low-resolution (LR) images of different scenes with the same model. Second, the wide range of scales within the ground objects in remote sensing images makes it difficult for single-scale convolution to effectively extract features of various scales. To address the above-mentioned issues, we propose a multiscale attention network (MSAN) to extract the multilevel features of remote sensing images. The basic component of MSAN is the multiscale activation feature fusion block (MAFB). In addition, a scene-adaptive super-resolution strategy for remote sensing images is employed to more accurately describe the structural characteristics of different scenes. The experiments undertaken on several data sets confirm that the proposed algorithm outperforms the other state-of-the-art algorithms, in both evaluation indices and visual results.

Scene-Adaptive Remote Sensing Image Super-Resolution Using a Multiscale Attention Network

Compared with traditional remote sensing images, satellite remote sensing video contains more useful information and can capture continuous dynamic video. Recently, many deep-learning based methods have been proposed for video super resolution. However, these methods tend to ignore the structural information and characteristics for video satellite imagery such as small ground targets, a wide range of scales and weak textures. To this end, this paper proposes a single-and-multi-frame ensembled framework called SMFE for remote sensing videos super-resolution. The SMFE framework combines a non-local based single image super resolution (SISR) network and a state-of-the-arts multi-frame super resolution (MFSR) network EDVR. Experiments have been performed to demonstrate the effectiveness of the proposed method on Jilin-1.

Shu Zhang

Papers

Scene-Adaptive Remote Sensing Image Super-Resolution Using a Multiscale Attention Network

Video Satellite Imagery Super Resolution for ‘Jilin-1’ via a Single-and-Multi Frame Ensembled Framework

A super-resolution fusion video imaging spectrometer based on single-pixel camera