MLR-DBPFN: A Multi-scale Low Rank Deep Back Projection Fusion Network for Anti-noise Hyperspectral and Multispectral Image Fusion

TLDR: Wang et al. as discussed by the authors proposed a multi-scale low-rank deep back projection fusion network (MLR-DBPFN) to fuse LR hyperspectral (HS) data and HR multispectral data.

Abstract: Fusing low spatial resolution (LR) hyperspectral (HS) data and high spatial resolution (HR) multispectral (MS) data aims to obtain HR HS data. However, due to bad weather and aging of sensor equipment, HS images usually contain a lot of noise, e.g., Gaussian noise, strip noise and mixed noise, which would make the fused image have low quality. To solve this problem, we propose the multi-scale low rank deep back projection fusion network (MLR-DBPFN). First, HS and MS are superimposed, and multi-scale spectral features of the stacked image are extracted through multi-scale low-rank decomposition and convolution operation, which effectively remove noisy spectral features. Second, the up-sampling and down-sampling network mechanisms are used to extract the multi-scale spatial features from each layer of spectral features. Finally, the multi-scale spectral features and multi-scale spatial features are combined for network training, and the weight of the noisy spectrum features is reduced through the network feedback mechanism, which suppresses the noisy spectrum and improves the noisy HS fusion performance. Experimental results on datasets of different noise demonstrate that MLR-DBPFN has superior spatial and spectral fidelity, comparative fusion quality, and robust anti-noise performance compared with state-of-the-art methods.