Showing papers on "Bicubic interpolation published in 2020"
TL;DR: This paper proposes a new solution (named as Multi-Receptive-Field Network - MRFN), which outperforms existing SISR solutions in three different aspects and can achieve more accurate recovering results than most state-of-the-art methods with significantly less complexity.
Abstract: Recently, convolutional neural network (CNN) based models have shown great potential in the task of single image super-resolution (SISR). However, many state-of-the-art SISR solutions are reproducing some tricks proven effective in other vision tasks, such as pursuing a deeper model. In this paper, we propose a new solution (named as Multi-Receptive-Field Network - MRFN), which outperforms existing SISR solutions in three different aspects. First, from receptive field: a novel multi-receptive-field (MRF) module is proposed to extract and fuse features in different receptive fields from local to global. Integrating these hierarchical features can generate better mappings on recovering high-fidelity details at different scales. Second, from network architectures: both dense skip connections and deep supervision are utilized to combine features from the current MRF module and preceding ones for training more representative features. Moreover, a deconvolution layer is embedded at the end of the network to avoid artificial priors induced by numerical data pre-processing (e.g., bicubic stretching), and speed up the restoration process. Finally, from error modeling: different from $L1$ and $L2$ loss functions, we proposed a novel two-parameter training loss called Weighted Huber loss function which can adaptively adjust the value of back-propagated derivative according to the residual value, thus fit the reconstruction error more effectively. Extensive qualitative and quantitative evaluation results on benchmark datasets demonstrate that our proposed MRFN can achieve more accurate recovering results than most state-of-the-art methods with significantly less complexity.
81 citations
TL;DR: Although training was performed only on short-axis cardiac MRI examinations, the proposed strategy appeared to improve quality in other imaging planes and recovered high-frequency spatial information.
Abstract: Background Cardiac MRI is limited by long acquisition times, yet faster acquisition of smaller-matrix images reduces spatial detail. Deep learning (DL) might enable both faster acquisition and higher spatial detail via super-resolution. Purpose To explore the feasibility of using DL to enhance spatial detail from small-matrix MRI acquisitions and evaluate its performance against that of conventional image upscaling methods. Materials and Methods Short-axis cine cardiac MRI examinations performed between January 2012 and December 2018 at one institution were retrospectively collected for algorithm development and testing. Convolutional neural networks (CNNs), a form of DL, were trained to perform super resolution in image space by using synthetically generated low-resolution data. There were 70%, 20%, and 10% of examinations allocated to training, validation, and test sets, respectively. CNNs were compared against bicubic interpolation and Fourier-based zero padding by calculating the structural similarity index (SSIM) between high-resolution ground truth and each upscaling method. Means and standard deviations of the SSIM were reported, and statistical significance was determined by using the Wilcoxon signed-rank test. For evaluation of clinical performance, left ventricular volumes were measured, and statistical significance was determined by using the paired Student t test. Results For CNN training and retrospective analysis, 400 MRI scans from 367 patients (mean age, 48 years ± 18; 214 men) were included. All CNNs outperformed zero padding and bicubic interpolation at upsampling factors from two to 64 (P .05), and super-resolved full-resolution images appeared to further enhance anatomic detail. Conclusion Deep learning outperformed conventional upscaling methods and recovered high-frequency spatial information. Although training was performed only on short-axis cardiac MRI examinations, the proposed strategy appeared to improve quality in other imaging planes. © RSNA, 2020 Online supplemental material is available for this article.
64 citations
14 Jun 2020
TL;DR: Inspired by the literature on generalized sampling, this work proposes a method for improving the performance of DNNs that have been trained with a fixed kernel on observations acquired by other kernels, and shows that this approach outperforms other super-resolution methods, which are designed for general downscaling kernels.
Abstract: The single image super-resolution task is one of the most examined inverse problems in the past decade. In the recent years, Deep Neural Networks (DNNs) have shown superior performance over alternative methods when the acquisition process uses a fixed known downscaling kernel---typically a bicubic kernel. However, several recent works have shown that in practical scenarios, where the test data mismatch the training data (e.g. when the downscaling kernel is not the bicubic kernel or is not available at training), the leading DNN methods suffer from a huge performance drop. Inspired by the literature on generalized sampling, in this work we propose a method for improving the performance of DNNs that have been trained with a fixed kernel on observations acquired by other kernels. For a known kernel, we design a closed-form correction filter that modifies the low-resolution image to match one which is obtained by another kernel (e.g. bicubic), and thus improves the results of existing pre-trained DNNs. For an unknown kernel, we extend this idea and propose an algorithm for blind estimation of the required correction filter. We show that our approach outperforms other super-resolution methods, which are designed for general downscaling kernels.
62 citations
01 Jun 2020
TL;DR: SSR-VFD is the first work that advocates a machine learning approach to generate high-resolution vector fields from low-resolution ones, and lies in the use of three separate neural nets that take the three components of a low- Resolution vector field as input and jointly output a synthesized high- resolution vector field.
Abstract: We present SSR-VFD, a novel deep learning framework that produces coherent spatial super-resolution (SSR) of three-dimensional vector field data (VFD). SSR-VFD is the first work that advocates a machine learning approach to generate high-resolution vector fields from low-resolution ones. The core of SSR-VFD lies in the use of three separate neural nets that take the three components of a low-resolution vector field as input and jointly output a synthesized high-resolution vector field. To capture spatial coherence, we take into account magnitude and angle losses in network optimization. Our method can work in the in situ scenario where VFD are down-sampled at simulation time for storage saving and these reduced VFD are upsampled back to their original resolution during postprocessing. To demonstrate the effectiveness of SSR-VFD, we show quantitative and qualitative results with several vector field data sets of different characteristics and compare our method against volume upscaling using bicubic interpolation, and two solutions based on CNN and GAN, respectively.
47 citations
TL;DR: Experimental results show better performance of S-K algorithm than the considered other methods, including bilinear and bicubic interpolation and quasi IIR (Infinite Impulse Response) approximation.
44 citations
TL;DR: Based on the prior knowledge of the model of Fresnel zone plate with circular diffraction gratings, the pixel density is increased by the bicubic interpolation (BIP) method inside the hologram to enhance the low-frequency terms of the object as mentioned in this paper.
35 citations
23 Aug 2020
TL;DR: The proposed depth guided degradation model learning-based image super-resolution (DGDML-SR) achieves visually pleasing results and can outperform the state-of-the-arts in perceptual metrics.
Abstract: In the past few years, we have witnessed the great progress of image super-resolution (SR) thanks to the power of deep learning. However, a major limitation of the current image SR approaches is that they assume a pre-determined degradation model or kernel, e.g. bicubic, controls the image degradation process. This makes them easily fail to generalize in a real-world or non-ideal environment since the degradation model of an unseen image may not obey the pre-determined kernel used when training the SR model. In this work, we introduce a simple yet effective zero-shot image super-resolution model. Our zero-shot SR model learns an image-specific super-resolution network (SRN) from a low-resolution input image alone, without relying on external training sets. To circumvent the difficulty caused by the unknown internal degradation model of an image, we propose to learn an image-specific degradation simulation network (DSN) together with our image-specific SRN. Specifically, we exploit the depth information, naturally indicating the scales of local image patches, of an image to extract the unpaired high/low-resolution patch collection to train our networks. According to the benchmark test on four datasets with depth labels or estimated depth maps, our proposed depth guided degradation model learning-based image super-resolution (DGDML-SR) achieves visually pleasing results and can outperform the state-of-the-arts in perceptual metrics.
34 citations
TL;DR: In this paper, the pseudo-inverse image formation model is used as part of the network architecture in conjunction with perceptual losses and a smoothness constraint that eliminates the artifacts originating from these perceptual losses.
18 citations
18 citations
TL;DR: The results show that the proposed method produces high quality images with clear edges and transmits most of the information of source images into all-in-focused image.
Abstract: Multi-focus image fusion methods combine two or more images which have blurred and defocused parts to create an all-in-focused image. All-in-focused image has more information, clearer parts and clearer edges than the source images. In this paper, a new approach for multi-focus image fusion is proposed. Firstly, the information of source images is enhanced using bicubic interpolation-based super-resolution method. Secondly, source images with high resolution are decomposed into four sub-bands which are LL (low-low), LH (low-high), HL (high-low) and HH (high-high) using Stationary Wavelet Transform with dmey (Discrete Meyer) filter. Then, a new fusion rule which depend on gradient-based method with sobel operator is implemented to create fused images with good visuality. The weight coefficients which show the importance rates of corresponding pixels in source images for fused image are calculated using designed formula based on gradient magnitudes. The each pixel of fused sub-bands is created using these weight coefficients and fused image is reconstructed using Inverse Stationary Wavelet Transform. Lastly, the performance evaluation of proposed method is measured using three different metrics which are objective, subjective and time criterion metrics. Besides these features, the new dataset which is different from the datasets in the literature is created and used firstly in this paper. The results show that the proposed method produces high quality images with clear edges and transmits most of the information of source images into all-in-focused image.
15 citations
TL;DR: It was observed that SRGAN was superior to the classical approaches not only in increasing the resolution but also in the noise cleaning area.
Abstract: Bilinear and Bicubic interpolation techniques are frequently used to increase image resolution. These techniques with data modeling approach are replaced by intelligent systems that can learn automatically from data. SRGAN is a modern Generative Adversarial Network developed as an alternative to classical interpolation techniques. His ability to produce images in super resolution has attracted the attention of many researchers. In this study, noise elimination performance of super resolution generative adversarial network (SRGAN) with image magnification was investigated. The results of the noise cleaning were compared with the classical approaches (mean, median, adaptive filters). SSIM, PSNR and FFT_MSE metrics were evaluated in experimental studies using images in the data set Camelyon17. When the results were evaluated, it was observed that SRGAN was superior to the classical approaches not only in increasing the resolution but also in the noise cleaning area.
Posted Content•
TL;DR: Two hardware-efficient interpolation methods are proposed for super-resolution imaging platforms, mainly for the mobile application and results indicate that the proposed approach is practically applicable to real-world applications.
Abstract: Super-resolution imaging (S.R.) is a series of techniques that enhance the resolution of an imaging system, especially in surveillance cameras where simplicity and low cost are of great importance. S.R. image reconstruction can be viewed as a three-stage process: image interpolation, image registration, and fusion. Image interpolation is one of the most critical steps in the S.R. algorithms and has a significant influence on the quality of the output image. In this paper, two hardware-efficient interpolation methods are proposed for these platforms, mainly for the mobile application. Experiments and results on the synthetic and real image sequences clearly validate the performance of the proposed scheme. They indicate that the proposed approach is practically applicable to real-world applications. The algorithms are implemented in a Field Programmable Gate Array (FPGA) device using a pipelined architecture. The implementation results show the advantages of the proposed methods regarding area, performance, and output quality.
TL;DR: This article presents a multi-degradation, unsupervised SR method based on deep learning that renders the state-of-the-art results compared with existing un supervised SR methods, and achieves competitive results in contrast with supervised SR methods.
Abstract: In remote sensing, it is desirable to improve image resolution by using the image super-resolution (SR) technique. However, there are two challenges: the first one is that high-resolution (HR) images are insufficient or unavailable; another one is that the single degradation model such as bicubic (BIC) cannot super-resolve favorable images in the real world. To address the above two problems, this article presents a multi-degradation, unsupervised SR method based on deep learning. This framework consists of a degrader D to fit the image degradation model and a generator G to generate SR image. By introducing D, calculating the loss function between SR image and HR image as supervised SR methods did can be converted into calculating loss between low resolution (LR) image and image degraded by SR image, thereby realizing unsupervised learning. Experiments on several degradation models show that our method renders the state-of-the-art results compared with existing unsupervised SR methods, and achieves competitive results in contrast with supervised SR methods. Moreover, for real remote sensing images obtained by the Jilin-1 satellite, our method obtained more plausible results visually, which demonstrate the potential in real-world applications.
TL;DR: Training the CNN by increasing the image size using the interpolation method is a useful method, and comparisons of the average classification accuracy of the chest X-ray images showed a stable and high-average classification accuracy using the extrapolation method.
Abstract: Background Deep learning using convolutional neural networks (CNN) has achieved significant results in various fields that use images. Deep learning can automatically extract features from data, and CNN extracts image features by convolution processing. We assumed that increasing the image size using interpolation methods would result in an effective feature extraction. To investigate how interpolation methods change as the number of data increases, we examined and compared the effectiveness of data augmentation by inversion or rotation with image augmentation by interpolation when the image data for training were small. Further, we clarified whether image augmentation by interpolation was useful for CNN training. To examine the usefulness of interpolation methods in medical images, we used a Gender01 data set, which is a sex classification data set, on chest radiographs. For comparison of image enlargement using an interpolation method with data augmentation by inversion and rotation, we examined the results of two- and four-fold enlargement using a Bilinear method. Results The average classification accuracy improved by expanding the image size using the interpolation method. The biggest improvement was noted when the number of training data was 100, and the average classification accuracy of the training model with the original data was 0.563. However, upon increasing the image size by four times using the interpolation method, the average classification accuracy significantly improved to 0.715. Compared with the data augmentation by inversion and rotation, the model trained using the Bilinear method showed an improvement in the average classification accuracy by 0.095 with 100 training data and 0.015 with 50,000 training data. Comparisons of the average classification accuracy of the chest X-ray images showed a stable and high-average classification accuracy using the interpolation method. Conclusion Training the CNN by increasing the image size using the interpolation method is a useful method. In the future, we aim to conduct additional verifications using various medical images to further clarify the reason why image size is important.
31 Jan 2020
TL;DR: In this paper, the authors apply commonly known methods of nonadaptive interpolation (nearest pixel, bilinear, B-spline, bicubic, Hermite spline) and sampling (point sampling, supersampling, mip-map pre-filtering, ripmap prefiltering and FAST) to the problem of projective image transformation and compare their computational complexity, describe their artifacts and than experimentally measure their quality and working time on mobile processor with ARM architecture.
Abstract: In this work we apply commonly known methods of non-adaptive interpolation (nearest pixel, bilinear, B-spline, bicubic, Hermite spline) and sampling (point sampling, supersampling, mip-map pre-filtering, rip-map pre-filtering and FAST) to the problem of projective image transformation. We compare their computational complexity, describe their artifacts and than experimentally measure their quality and working time on mobile processor with ARM architecture. Those methods were widely developed in the 90s and early 2000s, but were not in an area of active research in resent years due to a lower need in computationally efficient algorithms. However, real-time mobile recognition systems, which collect more and more attention, do not only require fast projective transform methods, but also demand high quality images without artifacts. As a result, in this work we choose methods appropriate for those systems, which allow to avoid artifacts, while preserving low computational complexity. Based on the experimental results for our setting they are bilinear interpolation combined with either mip-map pre-filtering or FAST sampling, but could be modified for specific use cases.
TL;DR: In this paper, the authors reconstruct a 3D image of the spine from a sequence of 2D MRI slices along any one axis with reasonable slice gap, and then they use a simple geometric method to slice out any possible location along any axis and get the information in that region.
Abstract: Magnetic resonance imaging (MRI) is a very effective method for identifying any abnormality in the structure and physiology of the spine. However, MRI is time consuming as well as costly. In this work, the authors propose an algorithm which can reduce the time of MRI and thus the cost, with minimal compromise on accuracy. They reconstruct a three-dimensional (3D) image of the spine from a sequence of 2D MRI slices along any one axis with reasonable slice gap. In order to preserve the image at the edges properly, they regenerate the 3D image by using a combination of bicubic and bilinear interpolation along the orthogonal axis. From the reconstructed 3D, they use a simple geometric method to slice out any possible location along any axis and get the information in that region. They have tested their algorithm on real data, and found that their algorithm reduces the time by 80%, with high internal data preservation accuracy of about 96%.
29 Oct 2020
TL;DR: In this article, two hardware-efficient interpolation methods are proposed for these platforms, mainly for the mobile application Experiments and results on the synthetic and real image sequences clearly validate the performance of the proposed scheme They indicate that the proposed approach is practically applicable to real-world applications.
Abstract: Super-resolution imaging (SR) is a series of techniques that enhance the resolution of an imaging system, especially in surveillance cameras where simplicity and low cost are of great importance SR image reconstruction can be viewed as a three-stage process: image interpolation, image registration, and fusion Image interpolation is one of the most critical steps in the SR algorithms and has a significant influence on the quality of the output image In this paper, two hardware-efficient interpolation methods are proposed for these platforms, mainly for the mobile application Experiments and results on the synthetic and real image sequences clearly validate the performance of the proposed scheme They indicate that the proposed approach is practically applicable to real-world applications The algorithms are implemented in a Field Programmable Gate Array (FPGA) device using a pipelined architecture The implementation results show the advantages of the proposed methods regarding area, performance, and output quality
TL;DR: A real-time image super-resolution method with good reconstruction performance that replaces the default upsampling method (bicubic interpolation) with a pixel shuffling layer and is not only fast but also accurate.
Abstract: The aim of single-image super-resolution is to recover a high-resolution image based on a low-resolution image. Deep convolutional neural networks have largely enhanced the reconstruction performance of image super-resolution. Since the input image is always bicubic-interpolated, the main weakness of deep convolutional neural networks is that they are time-consuming. Moreover, fast convolutional neural networks can perform real-time image super-resolution but are unable to achieve reliable performance. To address those drawbacks, we propose a real-time image super-resolution method with good reconstruction performance. We replace the default upsampling method (bicubic interpolation) with a pixel shuffling layer. Local and global residual connections are taken to guarantee better performance. As shown in Fig. 1, our proposed method is not only fast but also accurate.
Posted Content•
TL;DR: This paper designs a Frequency Decomposition module to separate different frequency components to comprehensively compensate the information lost for real LR image, and designs a Region-adaptive Frequency Aggregation module by leveraging dynamic convolution and spatial attention to adaptively restore frequency components for different regions.
Abstract: Traditional single image super-resolution (SISR) methods that focus on solving single and uniform degradation (i.e., bicubic down-sampling), typically suffer from poor performance when applied into real-world low-resolution (LR) images due to the complicated realistic degradations. The key to solving this more challenging real image super-resolution (RealSR) problem lies in learning feature representations that are both informative and content-aware. In this paper, we propose an Omni-frequency Region-adaptive Network (ORNet) to address both challenges, here we call features of all low, middle and high frequencies omni-frequency features. Specifically, we start from the frequency perspective and design a Frequency Decomposition (FD) module to separate different frequency components to comprehensively compensate the information lost for real LR image. Then, considering the different regions of real LR image have different frequency information lost, we further design a Region-adaptive Frequency Aggregation (RFA) module by leveraging dynamic convolution and spatial attention to adaptively restore frequency components for different regions. The extensive experiments endorse the effective, and scenario-agnostic nature of our OR-Net for RealSR.
01 Oct 2020
TL;DR: A novel learning-based image downscaling method, Hamiltonian Rescaling Network (HRNet), based on the discretization of Hamiltonian System, which formulate a mechanism of iterative correction of the error caused by information missing during image or feature down scaling.
Abstract: Image downscaling has become a classical problem in image processing and has recently connected to image super-resolution (SR), which restores high-quality images from low-resolution ones generated by predetermined downscaling kernels (e.g., bicubic). However, most existing image downscaling methods are deterministic and lose information during the downscaling process, while rarely designing specific downscaling methods for image SR. In this paper, we propose a novel learning-based image downscaling method, Hamiltonian Rescaling Network (HRNet). The design of HRNet is based on the discretization of Hamiltonian System, a pair of iterative updating equations, which formulate a mechanism of iterative correction of the error caused by information missing during image or feature downscaling. Extensive experiments demonstrate the effectiveness of our proposed method in terms of both quantitative and qualitative results.
TL;DR: Evaluation results demonstrate that the proposed deep model, entitled Deep Block Super Resolution (DBSR), outperforms state-of-the-art alternatives in both areas of medical and general super-resolution enhancement from a single input image.
Abstract: This study presents a method to reconstruct a high-resolution image using a deep convolution neural network. We propose a deep model, entitled Deep Block Super Resolution (DBSR), by fusing the output features of a deep convolutional network and a shallow convolutional network. In this way, our model benefits from high frequency and low frequency features extracted from deep and shallow networks simultaneously. We use the residual layers in our model to make repetitive layers, increase the depth of the model, and make an end-to-end model. Furthermore, we employed a deep network in up-sampling step instead of bicubic interpolation method used in most of the previous works. Since the image resolution plays an important role to obtain rich information from the medical images and helps for accurate and faster diagnosis of the ailment, we use the medical images for resolution enhancement. Our model is capable of reconstructing a high-resolution image from low-resolution one in both medical and general images. Evaluation results on TSA and TZDE datasets, including MRI images, and Set5, Set14, B100, and Urban100 datasets, including general images, demonstrate that our model outperforms state-of-the-art alternatives in both areas of medical and general super-resolution enhancement from a single input image.
TL;DR: In this study, an efficient self-learning method for image super-resolution (SR) is presented and experimental results confirm that the proposed method performance is promising.
Abstract: In this study, an efficient self-learning method for image super-resolution (SR) is presented. In the proposed algorithm, the input image is divided into equal size patches. Using these patches, a dictionary is learned based on K-SVD, referred to as high resolution (HR) dictionary. Then, by down-sampling, the columns of the dictionary, called atoms, a low resolution (LR) version of the dictionary is obtained. An initial estimate of the SR image is constructed using the bicubic interpolation on the input image. Then in an iterative algorithm, the difference between the down-sampled version of the estimated SR image and the input image is obtained. This difference image, which includes reconstructed details is enlarged using sparse representation and LR/HR dictionaries. The enlarged detail is added to the latest reconstructed SR image. This process gradually improves the quality of the initial SR image. After several iterations, the reconstructed image is an SR version of the input image. Experimental results confirm that the proposed method performance is promising.
TL;DR: Compared to bilinear, bicubic, iterative curvature-based interpolation (ICBI), novel edge orientation adaptive interpolation scheme for resolution enhancement of still images (NEDI), super-resolution using iterative Wiener filter based on nonlocal means (SR-NLM) and rational ball cubic B-spline (RBC), the proposed method can improve peak signal to noise ratio (PSNR) and structural similarity index (SSIM).
Abstract: Image interpolation is important in image zooming. To improve the quality of image zooming, in this work, we proposed a class of rational quadratic trigonometric Hermite functions with two shape parameters and two classes of C 1 -continuous Coons patches constructions over a triangular domain by improved side–side method and side–vertex method. Altering the values of shape parameters can adjust the interior shape of the triangular Coons patch without influencing the function values and partial derivatives of the boundaries. In order to deal with the problem of well-posedness in image zooming, we discussed symmetrical sufficient conditions for region control of shape parameters in the improved side–side method and side–vertex method. Some examples demonstrate the proposed methods are effective in surface design and digital image zooming. C 1 -continuous Coons patches constructed by the proposed methods can interpolate to scattered 3D data. By up-sampling to the constructed interpolation surface, high-resolution images can be obtained. Image zooming experiment and analysis show that compared to bilinear, bicubic, iterative curvature-based interpolation (ICBI), novel edge orientation adaptive interpolation scheme for resolution enhancement of still images (NEDI), super-resolution using iterative Wiener filter based on nonlocal means (SR-NLM) and rational ball cubic B-spline (RBC), the proposed method can improve peak signal to noise ratio (PSNR) and structural similarity index (SSIM). Edge detection using Prewitt operator shows that the proposed method can better preserve sharp edges and textures in image zooming. The proposed methods can also improve the visual effect of the image, therefore it is efficient in computation for image zooming.
05 Oct 2020
TL;DR: In this paper, a super resolution application was carried out on RGB thermal images of human faces by using adversarial generating networks and batch normalization layers were used in both the generative network and the discriminator network part in order to avoid the problem of gradient disappearance during the training of the Generative Adversarial Network (GAN).
Abstract: Thermal imaging is an imaging system based on invisible infrared energy (heat) and the general structure of the image is determined by colors and shapes formed according to infrared energy. Although it is generally used for security purposes, it is open to use in a wide variety of sectors. Especially in recent years, thermal imaging systems have found a wide range of use in the medical field. Thermal imaging is an imaging system that is difficult to design and expensive. Therefore, interest in super resolution applications in the field of thermal imaging to increase the resolution of thermal images has increased considerably in recent years. Here, developments in the field of deep learning have accelerated these studies and increased the success. In this study, super resolution application was carried out on RGB thermal images of human faces by using adversarial generating networks. In this study, the images obtained from the Variocam HD thermal camera were used as high resolution images, while the images obtained from the Flir One Pro thermal camera were used as low resolution images. For this project, 5 pairs of thermal images (high resolution-low resolution) belonging to 12 people were used. Of these images, 45 are separated as double image training data set, and 15 double images as test data set. Batch normalization layers were used in both the generative network and the discriminator network part in order to avoid the problem of gradient disappearance during the training of the generative adversarial network and to provide faster training of the network. Since residual blocks facilitate the training difficulties of very deep networks and increase the success performance, a skip connection has been applied in the generator network similar to ResNet. The success performance of the results obtained as a result of training the network was evaluated using the image quality metrics PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). As a result, an increase of approximately 1.5dB in PSNR values and an increase of approximately 6% in SSIM values were observed compared to bicubic interpolation. In future studies, an alternative solution to high cost thermal camera systems can be offered as a result of training the deep network by using more data and by simulating the color tones of images obtained from two different cameras.
TL;DR: The results of evaluations on a wide variety of images show that the proposed SICNN achieves advantages over the state-of-the-art methods in terms of both numerical results and visual quality.
Abstract: Deep convolutional neural networks have recently achieved dramatic success in super-resolution (SR) performance in the past few years. However, the parameters of the mapping functions of these networks require an external dataset for training. In this paper, we propose a convolutional network for image super-resolution reconstruction that can be trained using an internal dataset constructed using a single image. The proposed single image convolutional neural network (SICNN) is designed with two branches. First, a large scale-feature branch trains the feature mappings that are from the low resolution (LR) image patches to the high- resolution image (HR) patches. The LR image patches are the enlarged image patches via bicubic interpolation. Second, the small scale-feature branch trains the feature mappings that are from the down-sampling image patches to the enlarged image patches. In contrast to the existing SR networks, the SICNN enjoys two desirable properties: 1) it does not require external datasets to conduct training, and 2) it enlarges an SR image at an arbitrary scale while restoring the clear edges and textures. The results of evaluations on a wide variety of images show that the proposed SICNN achieves advantages over the state-of-the-art methods in terms of both numerical results and visual quality.
TL;DR: A new filtering algorithm for reducing phase noise based on the bicubic interpolation method (BIM) is presented and the unwrapped phase map accuracy is enhanced by the combination of BIM with a conventional unwrapping algorithm.
Abstract: This paper presents a new filtering algorithm for reducing phase noise based on the bicubic interpolation method (BIM). The unwrapped phase map accuracy is enhanced by the combination of BIM with a conventional unwrapping algorithm. The bicubic interpolation filtering (BIF) and bicubic interpolation smoothed filtering (BISF) methods are two powerful low-pass filters. The simulation shows that the BIF and BISF convert the initial noise distribution to Gaussian distribution. The Mirau interferometer is used to improve the performance of the proposed filtering algorithms. The root mean square error between two quality-guided path unwrapping algorithms and BISF method is estimated at approximately 31 nm.
TL;DR: A super-resolution (SR) technique for enhancement of infrared (IR) images that uses bicubic interpolation and minimum mean square error (MMSE) estimation in the prediction of the HR image with a scheme that can be interpreted as a feed-forward neural network.
Abstract: This paper presents a super-resolution (SR) technique for enhancement of infrared (IR) images. The suggested technique relies on the image acquisition model, which benefits from the sparse representations of low-resolution (LR) and high-resolution (HR) patches of the IR images. It uses bicubic interpolation and minimum mean square error (MMSE) estimation in the prediction of the HR image with a scheme that can be interpreted as a feed-forward neural network. The suggested algorithm to overcome the problem of having only LR images due to hardware limitations is represented with a big data processing model. The performance of the suggested technique is compared with that of the standard regularized image interpolation technique as well as an adaptive block-by-block least-squares (LS) interpolation technique from the peak signal-to-noise ratio (PSNR) perspective. Numerical results reveal the superiority of the proposed SR technique.
Posted Content•
TL;DR: A unified generative model for the image restoration, which elaborately configures the degradation process from the latent clean image to the observed corrupted one, and designed a variational inference algorithm to learn all parameters involved in the proposed model with explicit form of objective loss.
Abstract: Deep neural networks (DNNs) have achieved significant success in image restoration tasks by directly learning a powerful non-linear mapping from corrupted images to their latent clean ones. However, there still exist two major limitations for these deep learning (DL)-based methods. Firstly, the noises contained in real corrupted images are very complex, usually neglected and largely under-estimated in most current methods. Secondly, existing DL methods are mostly trained on one pre-assumed degradation process for all of the training image pairs, such as the widely used bicubic downsampling assumption in the image super-resolution task, inevitably leading to poor generalization performance when the true degradation does not match with such assumed one. To address these issues, we propose a unified generative model for the image restoration, which elaborately configures the degradation process from the latent clean image to the observed corrupted one. Specifically, different from most of current methods, the pixel-wisely non-i.i.d. Gaussian distribution, being with more flexibility, is adopted in our method to fit the complex real noises. Furthermore, the method is built on the general image degradation process, making it capable of adapting diverse degradations under one single model. Besides, we design a variational inference algorithm to learn all parameters involved in the proposed model with explicit form of objective loss. Specifically, beyond traditional variational methodology, two DNNs are employed to parameterize the posteriori distributions, one to infer the distribution of the latent clean image, and another to infer the distribution of the image noise. Extensive experiments demonstrate the superiority of the proposed method on three classical image restoration tasks, including image denoising, image super-resolution and JPEG image deblocking.
23 Aug 2020
TL;DR: A novel super resolution network with large receptive field called SCA-SR with self-calibrated convolutions to low-level vision task for the first time to significantly enlarge the receptive field of SR model and reduce the noise introduced by individual model.
Abstract: Single Image Super-Resolution in practical scenarios is quite challenging, because of more complex degradation than bicubic downsampling and diverse degradation differences among devices. To solve this problem, we develop a novel super resolution network with large receptive field called SCA-SR. The contributions mainly contain the following four points. First, we introduce self-calibrated convolutions to low-level vision task for the first time to significantly enlarge the receptive field of SR model. Second, Cutblur methods are used to improve the generalization of model. Third, long skip connection was used in model design to improve the convergence of deep model structure. Fourth, we use both self-ensemble and model-ensemble to improve the robustness of model and reduce the noise introduced by individual model. According to the preliminary results of AIM 2020 Real Image Super-Resolution Challenge, our solution ranks third in both \(\times \)2 and \(\times \)3 tracks.
TL;DR: The experiments prove that the proposed RPSRMD, which includes RPGen and PResNet as two core components, is superior to many state-of-the-art SR methods that were designed and trained to handle multi-degradation.
Abstract: Although SISR (Single Image Super Resolution) problem can be effectively solved by deep learning based methods, the training phase often considers single degradation type such as bicubic interpolation or Gaussian blur with fixed variance. These priori hypotheses often fail and lead to reconstruction error in real scenario. In this paper, we propose an end-to-end CNN model RPSRMD to handle SR problem in multiple Gaussian degradations by extracting and using as side information a shared image prior that is consistent in different Gaussian degradations. The shared image prior is generated by an AED network RPGen with a rationally designed loss function that contains two parts: consistency loss and validity loss. These losses supervise the training of AED to guarantee that the image priors of one image with different Gaussian blurs to be very similar. Afterwards we carefully designed a SR network, which is termed as PResNet (Prior based Residual Network) in this paper, to efficiently use the image priors and generate high quality and robust SR images when unknown Gaussian blur is presented. When we applied variant Gaussian blurs to the low resolution images, the experiments prove that our proposed RPSRMD, which includes RPGen and PResNet as two core components, is superior to many state-of-the-art SR methods that were designed and trained to handle multi-degradation.