In this article, we review recent uses of convolutional neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, superresolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions.

/pdf/convolutional-neural-networks-for-inverse-problems-in-50y28exfec.pdf

Convolutional Neural Networks for Inverse Problems in Imaging: A Review

Deep learning for pixel-level image fusion: Recent advances and future prospects

In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, super-resolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions: Where does the training data come from? What is the architecture of the CNN? and How is the learning problem formulated and solved? We also bring together a few key theoretical papers that offer perspective on why CNNs are appropriate for inverse problems and point to some next steps in the field.

/pdf/a-review-of-convolutional-neural-networks-for-inverse-u3g8iywd5g.pdf

A Review of Convolutional Neural Networks for Inverse Problems in Imaging

Most previous fusion strategies either fail to fully utilize temporal information or cost too much time, and how to effectively fuse temporal information from consecutive frames plays an important role in video super-resolution (SR). In this study, we propose a novel progressive fusion network for video SR, which is designed to make better use of spatio-temporal information and is proved to be more efficient and effective than the existing direct fusion, slow fusion or 3D convolution strategies. Under this progressive fusion framework, we further introduce an improved non-local operation to avoid the complex motion estimation and motion compensation (ME&MC) procedures as in previous video SR approaches. Extensive experiments on public datasets demonstrate that our method surpasses state-of-the-art with 0.96 dB in average, and runs about 3 times faster, while requires only about half of the parameters.

/pdf/progressive-fusion-video-super-resolution-network-via-4z274qe5p8.pdf

Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations

Image reconstruction is a key problem for electrical resistance tomography (ERT). Because of the soft-field nature and the ill-posed problem in solving inverse problem, traditional image reconstruction methods cannot achieve high accuracy and the process is usually time consuming. Since deep learning is good at mapping complicated nonlinear function, a deep learning method based on convolutional neural network (CNN) is proposed for image reconstruction of ERT. To establish the database, 41122 samples were generated with numerical simulations. 10-fold cross validation was used to divide all samples into training set and validation set. The network structure was based on LeNet, and refined by applying dropout layer and moving average. After 346 training epochs, the image correlation coefficient (ICC) on validation set was 0.95. When white Gaussian noise with a signal-to-noise ratio of 30, 40, and 50 were added to validation set, the ICC was 0.79, 0.89, and 0.93, respectively, which proved the anti-noise capability of the network. The reconstruction results on samples which have more inclusions, different conductivity, and other shapes explained the network has good generalization ability. Furthermore, experimental data from a 16-electrode industrial ERT system was used to compare the accuracy of the proposed model with some typical reconstruction methods. Results show that the proposed CNN method has better reconstruction results than LBP, Tikhonov, and Landweber.

Image Reconstruction Based on Convolutional Neural Network for Electrical Resistance Tomography

Video superresolution (SR) techniques are of essential usages for high-resolution display devices due to the current lack of high-resolution videos. Although many algorithms have been proposed, video SR still remains a very challenging inverse problem under different conditions. In this paper, we propose a new method for video SR named motion compensation and residual net (MCResNet). We use optical flow algorithm for motion estimation and motion compensation as a preprocessing step. Then, we employ a novel deep residual convolutional neural network (CNN) to predict a high-resolution image using multiple motion compensated observations. The new residual CNN model preserves the low-frequency contents and facilitates the restoration of high-frequency details. Our method is able to handle large and complex motions adaptively. Extensive experimental results validate that our proposed method outperforms state-of-the-art single-image-based and multi-frame-based algorithms for video SR quantitatively and qualitatively.

Video Superresolution via Motion Compensation and Deep Residual Learning

Video super-resolution (SR) aims at restoring fine details and enhancing visual experience for low-resolution videos. In this paper, we propose a very deep non-simultaneous fully recurrent convolutional network for video SR. To make full use of temporal information, we employ motion compensation, very deep fully recurrent convolutional layers, and late fusion in our system. Residual connection is also employed in our recurrent structure for more accurate SR. Finally, a new model ensemble strategy is used to combine our method with a single-image SR method. Experimental results demonstrate that the proposed method is better than that of the state-of-the-art SR methods on quantitative visual quality assessment.

Video Super-Resolution Using Non-Simultaneous Fully Recurrent Convolutional Network

Video super-resolution (SR) aims at restoring finer details and enhancing visual experience. In this paper, we propose a novel method named residual recurrent convolutional network (RRCN) for video SR. In our method, motion compensation and bidirectional residual convolutional network are combined to model the spatial and temporal non-linear mappings. To leverage sufficient amount of temporal information, we employ motion compensation, bidirectional recurrent convolutional layers and late fusion in of our network. We also apply residual connections in our recurrent structure for more accurate SR. Experimental results demonstrate the superiority of the proposed method over state-of-the-art single-image and multi-frame based SR approaches in terms of both quantitative assessment and visual quality.

Video super-resolution using motion compensation and residual bidirectional recurrent convolutional network

Video super-resolution with inverse recurrent net and hybrid local fusion

OBJECTIVE
To explore the role of Marginal Zone B and B-1 Cell-Specific Protein (MZB1), a novel molecule associated with periodontitis, in migration of human periodontal ligament cells (hPDLCs) and alveolar bone orchestration.


BACKGROUND
MZB1 is an ER-localized protein and its upregulation has been found to be associated with a variety of human diseases. However, few studies have investigated the effect and mechanism of MZB1 on hPDLCs in periodontitis.


METHODS
Gene expression profiles in human gingival tissues were acquired from the Gene Expression Omnibus (GEO) database, and candidate molecules were then selected through bioinformatic analysis. Subsequently, we identified the localization and expression of MZB1 in human gingival tissues, mice, and hPDLCs by immunofluorescence, RT-qPCR, and Western blot. Dual-luciferase reporter assay was applied to assess the binding of miR-185-5p to MZB1. Furthermore, the effects of MZB1 on cell migration, proliferation, and apoptosis in vitro were investigated by wound-healing assay, transwell assay, CCK-8 assay, and flow cytometry analysis. Finally, Micro-CT analysis and H&E staining were performed to examine the effects of MZB1 on alveolar bone loss in vivo.


RESULTS
Bioinformatic analysis discovered that MZB1 was one of the most significantly increased genes in periodontitis patients. MZB1 was markedly increased in the gingival tissues of periodontitis patients, in the mouse models, and in the hPDLCs treated with lipopolysaccharide of Porphyromonas gingivalis (LPS-PG). Furthermore, in vitro experiments showed that MZB1, as a target gene of miR-185-5p, inhibited migration of hPDLCs. Overexpression of MZB1 specifically upregulated the phosphorylation of p65, while pretreatment of MZB1-overexpressed hPDLCs with PDTC (NF-κB inhibitor) notably reduced the p-p65 level and promoted cell migration. In addition, the mRNA expression levels of alkaline phosphatase (ALP) and Runt-related transcription factor 2 (Runx2) were inhibited in MZB1-overexpressed hPDLCs and miR-185-5p inhibitor treated hPDLCs, respectively. In vivo experiments showed that knockdown of MZB1 alleviated the loss of alveolar bone.


CONCLUSION
As a target gene of miR-185-5p, MZB1 plays a crucial role in inhibiting the migration of hPDLCs through NF-κB signaling pathway and deteriorating alveolar bone loss.

Dingyi Li

Papers

Video Superresolution via Motion Compensation and Deep Residual Learning

Video Super-Resolution Using Non-Simultaneous Fully Recurrent Convolutional Network

Video super-resolution using motion compensation and residual bidirectional recurrent convolutional network

Video super-resolution with inverse recurrent net and hybrid local fusion

MZB1 targeted by miR-185-5p inhibits the migration of human periodontal ligament cells through NF-κB signaling and promotes alveolar bone loss.