A pansharpening scheme using spectral graph wavelet transforms and convolutional neural networks
11 Jan 2021-International Journal of Remote Sensing (Taylor & Francis)-Vol. 42, Iss: 8, pp 2898-2919
TL;DR: This paper illustrates the pansharpening approach that is based on multistage multichannel spectral graph wavelet transform and convolutional neural network (SGWT-PNN) and demonstrates the effectiveness of the proposed scheme applied on datasets collected by different satellites.
Abstract: The objective of the multispectral pansharpening scheme is to obtain high spatial-spectral resolution multispectral (MS) images using high spectral resolution MS and high spatial resolution panchro...
Citations
More filters
TL;DR: Wang et al. as mentioned in this paper proposed a prediction model based on graph wavelet transform (GWT) and attention weighted random forest (RF) to predict the severity of Parkinson's disease.
Abstract: The progress prediction of Parkinson's disease (PD) is one of the most important issues in early diagnosis of PD. Many researches have been conducted in this field, however, most existing methods focus on the selection of baseline features and regressors to reduce prediction errors. Different from the previous studies, the main goal of this paper is to obtain more effective features by feature transformation of baseline features to improve the prediction performance. Therefore, this paper proposes a prediction model based on graph wavelet transform (GWT) and attention weighted random forest (RF). Firstly, a clustering algorithm is adopted to reduce the prediction error of the model. Next, a multi-scale analysis of the feature vectors by GWT is conducted to yield a frequency feature representation that is more relevant to the target value. Finally, the frequency features are input into the attention weighted RF to predict the severity of PD, allowing the results of decision trees with better predictive performance in the RF to be highlighted while reducing the risk of overfitting. The effectiveness of the method is evaluated on the Parkinson's telemonitoring dataset collected by the University of Oxford. The experimental results show that the mean absolute error and root mean squared error of the proposed method for predicting PD severity (motor- and total-UPDRS) are 1.53, 2.13 and 1.91, 2.70, respectively. Compared with the quoted optimal method, the errors are reduced by 7.27%, 4.05% and 5.45%, 1.10%, respectively. This indicates that the proposed method has better prediction performance.
8 citations
TL;DR: Wang et al. as mentioned in this paper proposed a novel multisource remote sensing image fusion algorithm, which integrates the contrast saliency map (CSM) and the sum-modified-Laplacian (SML) in the nonsubsampled shearlet transform (NSST) domain.
Abstract: The rapid development of remote sensing and space technology provides multisource remote sensing image data for earth observation in the same area. Information provided by these images, however, is often complementary and cooperative, and multisource image fusion is still challenging. This paper proposes a novel multisource remote sensing image fusion algorithm. It integrates the contrast saliency map (CSM) and the sum-modified-Laplacian (SML) in the nonsubsampled shearlet transform (NSST) domain. The NSST is utilized to decompose the source images into low-frequency sub-bands and high-frequency sub-bands. Low-frequency sub-bands reflect the contrast and brightness of the source images, while high-frequency sub-bands reflect the texture and details of the source images. Using this information, the contrast saliency map and SML fusion rules are introduced into the corresponding sub-bands. Finally, the inverse NSST reconstructs the fusion image. Experimental results demonstrate that the proposed multisource remote image fusion technique performs well in terms of contrast enhancement and detail preservation.
6 citations
TL;DR: In this paper, a new signal decomposition method is proposed for early fault diagnosis of rolling bearings with entropy participation, which is introduced into the framework of complete ensemble intrinsic time-scale decomposition with adaptive noise (CEITDAN).
Abstract: The early fault diagnosis of rolling bearings has always been a difficult problem due to the interference of strong noise. This paper proposes a new method of early fault diagnosis for rolling bearings with entropy participation. First, a new signal decomposition method is proposed in this paper: intrinsic time-scale decomposition based on time-varying filtering. It is introduced into the framework of complete ensemble intrinsic time-scale decomposition with adaptive noise (CEITDAN). Compared with traditional intrinsic time-scale decomposition, intrinsic time-scale decomposition based on time-varying filtering can improve frequency-separation performance. It has strong robustness in the presence of noise interference. However, decomposition parameters (the bandwidth threshold and B-spline order) have significant impacts on the decomposition results of this method, and they need to be artificially set. Aiming to address this problem, this paper proposes rolling-bearing fault diagnosis optimization based on an improved coyote optimization algorithm (COA). First, the minimal generalized refined composite multiscale sample entropy parameter was used as the objective function. Through the improved COA algorithm, optimal intrinsic time-scale decomposition parameters based on time-varying filtering that match the input signal are obtained. By analyzing generalized refined composite multiscale sample entropy (GRCMSE), whether the mode component is dominated by the fault signal is determined. The signal is reconstructed and decomposed again. Finally, the mode component with the highest energy in the central frequency band is selected for envelope spectrum variation for fault diagnosis. Lastly, simulated and experimental signals were used to verify the effectiveness of the proposed method.
5 citations
TL;DR: In this article , a CNN model is used to extract the PAN detail image that is suitable for the MRA-based pansharpening scheme which significantly reduces the spatial and spectral distortions.
Abstract: Pansharpening produces a high spatial-spectral resolution pansharpened image by combining multispectral (MS) and panchromatic (PAN) images. In the traditional multi-resolution analysis (MRA) method, detailed PAN images are extracted by transformation methods that are injected into MS images. This gives spatial and spectral distortions in the pansharpened image. These distortions can be reduced in the pansharpened image by the correct matching of the PAN detail image component. This correct matching is possible by the convolutional neural network (CNN)–based models. This paper obtains the detailed image component using the CNN models. This CNN model extracts the PAN detail image that is suitable for the MRA-based pansharpening scheme which significantly reduces the spatial and spectral distortions. It is demonstrated by qualitative and quantitative analysis applied on GeoEye-1 and IKONOS satellite images and shows the effectiveness of the proposed scheme.
1 citations
TL;DR: In this article , a dual-channel feature extraction module is constructed to obtain a SAR image feature map, and an attention-based feature fusion module is designed to achieve spectral fidelity of the fused images.
Abstract: In the field of image fusion, spatial detail blurring and color distortion appear in synthetic aperture radar (SAR) images and multispectral (MS) during the traditional fusion process due to the difference in sensor imaging mechanisms. To solve this problem, this paper proposes a fusion method for SAR images and MS images based on a convolutional neural network. In order to make use of the spatial information and different scale feature information of high-resolution SAR image, a dual-channel feature extraction module is constructed to obtain a SAR image feature map. In addition, different from the common direct addition strategy, an attention-based feature fusion module is designed to achieve spectral fidelity of the fused images. In order to obtain better spectral and spatial retention ability of the network, an unsupervised joint loss function is designed to train the network. In this paper, the Sentinel 1 SAR images and Landsat 8 MS images are used as datasets for experiments. The experimental results show that the proposed algorithm has better performance in quantitative and visual representation when compared with traditional fusion methods and deep learning algorithms.
References
More filters
TL;DR: Although the new index is mathematically defined and no human visual system model is explicitly employed, experiments on various image distortion types indicate that it performs significantly better than the widely used distortion metric mean squared error.
Abstract: We propose a new universal objective image quality index, which is easy to calculate and applicable to various image processing applications. Instead of using traditional error summation methods, the proposed index is designed by modeling any image distortion as a combination of three factors: loss of correlation, luminance distortion, and contrast distortion. Although the new index is mathematically defined and no human visual system model is explicitly employed, our experiments on various image distortion types indicate that it performs significantly better than the widely used distortion metric mean squared error. Demonstrative images and an efficient MATLAB implementation of the algorithm are available online at http://anchovy.ece.utexas.edu//spl sim/zwang/research/quality_index/demo.html.
5,285 citations
"A pansharpening scheme using spectr..." refers background in this paper
...…quality of the pansharpened images are used in the degraded-scale assessment are spectral angle mapper (SAM) (Yuhas, Goetz, and Boardman 1992), relative dimensionless global error (ERGAS) (2002), and Q-index (Q4) (Wang and Bovik 2002), spatial correlation coefficient (SCC) (Alparone et al. 2008)....
[...]
27 Jun 2016
TL;DR: In this article, a very deep convolutional network inspired by VGG-net was used for image superresolution, which achieved state-of-the-art performance in accuracy.
Abstract: We present a highly accurate single-image superresolution (SR) method. Our method uses a very deep convolutional network inspired by VGG-net used for ImageNet classification [19]. We find increasing our network depth shows a significant improvement in accuracy. Our final model uses 20 weight layers. By cascading small filters many times in a deep network structure, contextual information over large image regions is exploited in an efficient way. With very deep networks, however, convergence speed becomes a critical issue during training. We propose a simple yet effective training procedure. We learn residuals only and use extremely high learning rates (104 times higher than SRCNN [6]) enabled by adjustable gradient clipping. Our proposed method performs better than existing methods in accuracy and visual improvements in our results are easily noticeable.
4,136 citations
Posted Content•
TL;DR: This work presents a highly accurate single-image superresolution (SR) method using a very deep convolutional network inspired by VGG-net used for ImageNet classification and uses extremely high learning rates enabled by adjustable gradient clipping.
Abstract: We present a highly accurate single-image super-resolution (SR) method. Our method uses a very deep convolutional network inspired by VGG-net used for ImageNet classification \cite{simonyan2015very}. We find increasing our network depth shows a significant improvement in accuracy. Our final model uses 20 weight layers. By cascading small filters many times in a deep network structure, contextual information over large image regions is exploited in an efficient way. With very deep networks, however, convergence speed becomes a critical issue during training. We propose a simple yet effective training procedure. We learn residuals only and use extremely high learning rates ($10^4$ times higher than SRCNN \cite{dong2015image}) enabled by adjustable gradient clipping. Our proposed method performs better than existing methods in accuracy and visual improvements in our results are easily noticeable.
3,628 citations
TL;DR: A novel method for constructing wavelet transforms of functions defined on the vertices of an arbitrary finite weighted graph using the spectral decomposition of the discrete graph Laplacian L, based on defining scaling using the graph analogue of the Fourier domain.
1,681 citations
Journal Article•
TL;DR: In this paper, the results of three different methods used to merge the information contents of the Landsat Thermatic Mapper (TM) and Satellite Pour l'Observation de la Terre (SPOT) panchromatic data are compared.
Abstract: The merging of multisensor image data is becoming a widely used procedure because of the complementary nature of various data sets. Ideally, the method used to merge data sets with high-spatial and high-spectral resolution should not distort the spectral characteristics of the high-spectral resolution data. This paper compares the results of three different methods used to merge the information contents of the Landsat Thermatic Mapper (TM) and Satellite Pour l'Observation de la Terre (SPOT) panchromatic data. The comparison is based on spectral characteristics and is made using statistical, visual, and graphical analyses of the results
1,261 citations