Generative adversarial network in medical imaging: A review.

Deep learning on image denoising: An overview.

Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

Image fusion meets deep learning: A survey and perspective

The fusion of high spatial resolution panchromatic (PAN) data with simultaneously acquired multispectral (MS) data with the lower spatial resolution is a hot topic, which is often called pansharpening. In this article, we exploit the combination of machine learning techniques and fusion schemes introduced to address the pansharpening problem. In particular, deep convolutional neural networks (DCNNs) are proposed to solve this issue. The latter is combined first with the traditional component substitution and multiresolution analysis fusion schemes in order to estimate the nonlinear injection models that rule the combination of the upsampled low-resolution MS image with the extracted details exploiting the two philosophies. Furthermore, inspired by these two approaches, we also developed another DCNN for pansharpening. This is fed by the direct difference between the PAN image and the upsampled low-resolution MS image. Extensive experiments conducted both at reduced and full resolutions demonstrate that this latter convolutional neural network outperforms both the other detail injection-based proposals and several state-of-the-art pansharpening methods.

Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening

Denoising of 3D magnetic resonance images using a residual encoder-decoder Wasserstein generative adversarial network.

Optical coherence tomography (OCT) has become a very promising diagnostic method in clinical practice, especially for ophthalmic diseases. However, speckle noise and low sampling rates have intensively reduced the quality of OCT images, which prevents the development of OCT-assisted diagnosis. Therefore, we propose a generative adversarial network-based approach (named SDSR-OCT) to simultaneously denoise and super-resolve OCT images. Moreover, we trained three different super-resolution models with different upscale factors (2× , 4× and 8×) to adapt to the corresponding downsampling rates. We also quantitatively and qualitatively compared our proposed method with some well-known algorithms. The experimental results show that our approach can effectively suppress speckle noise and can super-resolve OCT images at different scales.

Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network

Due to the limitation of the satellite sensor, it is difficult to acquire a high-resolution (HR) multispectral (HRMS) image directly. The aim of pansharpening (PNN) is to fuse the spatial in panchromatic (PAN) with the spectral information in multispectral (MS). Recently, deep learning has drawn much attention, and in the field of remote sensing, several pioneering attempts have been made related to PNN. However, the big size of remote sensing data will produce more training samples, which require a deeper neural network. Most current networks are relatively shallow and raise the possibility of detail loss. In this letter, we propose a residual encoder–decoder conditional generative adversarial network (RED-cGAN) for PNN to produce more details with sharpened images. The proposed method combines the idea of an autoencoder with generative adversarial network (GAN), which can effectively preserve the spatial and spectral information of the PAN and MS images simultaneously. First, the residual encoder–decoder module is adopted to extract the multiscale features from the last step to yield pansharpened images and relieve the training difficulty caused by deepening the network layers. Second, to further enhance the performance of the generator to preserve more spatial information, a conditional discriminator network with the input of PAN and MS images is proposed to encourage that the estimated MS images share the same distribution as that of the referenced HRMS images. The experiments conducted on the Worldview2 (WV2) and Worldview3 (WV3) images demonstrate that our proposed method provides better results than several state-of-the-art PNN methods.

Residual Encoder–Decoder Conditional Generative Adversarial Network for Pansharpening

Compressed sensing magnetic resonance imaging (CS-MRI) is a theoretical framework that can accurately reconstruct images from undersampled k-space data with a much lower sampling rate than the one set by the classical Nyquist-Shannon sampling theorem. Therefore, CS-MRI can efficiently accelerate acquisition time and relieve the psychological burden on patients while maintaining high imaging quality. The problems with traditional CS-MRI reconstruction are solved by iterative numerical solvers, which usually suffer from expensive computational cost and the lack of accurate handcrafted priori. In this paper, inspired by deep learning's (DL's) fast inference and excellent end-to-end performance, we propose a novel cascaded convolutional neural network called MD-Recon-Net to facilitate fast and accurate MRI reconstruction. Especially, different from existing DL-based methods, which operate on single domain data or both domains in a certain order, our proposed MD-Recon-Net contains two parallel and interactive branches that simultaneously perform on k-space and spatial-domain data, exploring the latent relationship between k-space and the spatial domain. The simulated experimental results show that the proposed method not only achieves competitive visual effects to several state-of-the-art methods, but also outperforms other DL-based methods in terms of model scale and computational cost.

MD-Recon-Net: A Parallel Dual-Domain Convolutional Neural Network for Compressed Sensing MRI

Compressed sensing magnetic resonance imaging (CS-MRI) is a theoretical framework that can accurately reconstruct images from undersampled ${k}$ -space data with a much lower sampling rate than the one set by the classical Nyquist–Shannon sampling theorem. Therefore, CS-MRI can efficiently accelerate acquisition time and relieve the psychological burden on patients while maintaining high imaging quality. The problems with traditional CS-MRI reconstruction are solved by iterative numerical solvers, which usually suffer from expensive computational cost and the lack of accurate handcrafted priori. In this article, inspired by deep learning’s (DL’s) fast inference and excellent end-to-end performance, we propose a novel cascaded convolutional neural network called MRI dual-domain reconstruction network (MD-Recon-Net) to facilitate fast and accurate magnetic resonance imaging reconstruction. Especially, different from existing DL-based methods, which operate on single domain data or both domains in a certain order, our proposed MD-Recon-Net contains two parallel and interactive branches that simultaneously perform on ${k}$ -space and spatial-domain data, exploring the latent relationship between ${k}$ -space and the spatial domain. The simulated experimental results show that the proposed method not only achieves competitive visual effects to several state-of-the-art methods but also outperforms other DL-based methods in terms of model scale and computational cost.

Maosong Ran

Papers

Denoising of 3D magnetic resonance images using a residual encoder-decoder Wasserstein generative adversarial network.

Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network

Residual Encoder–Decoder Conditional Generative Adversarial Network for Pansharpening

MD-Recon-Net: A Parallel Dual-Domain Convolutional Neural Network for Compressed Sensing MRI

MD-Recon-Net: A Parallel Dual-Domain Convolutional Neural Network for Compressed Sensing MRI