Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
01 Oct 2019
TL;DR: Zhang et al. as mentioned in this paper investigated the robustness of deep learning-based super-resolution methods against adversarial attacks, which can significantly degrade the super-resolved images without noticeable distortion in the attacked low-resolution images.
Abstract: Single-image super-resolution aims to generate a high-resolution version of a low-resolution image, which serves as an essential component in many image processing applications. This paper investigates the robustness of deep learning-based super-resolution methods against adversarial attacks, which can significantly deteriorate the super-resolved images without noticeable distortion in the attacked low-resolution images. It is demonstrated that state-of-the-art deep super-resolution methods are highly vulnerable to adversarial attacks. Different levels of robustness of different methods are analyzed theoretically and experimentally. We also present analysis on transferability of attacks, and feasibility of targeted attacks and universal attacks.
38 citations
TL;DR: This study presents high-quality and high-resolution tomographic images of biological samples to demonstrate the experimental feasibility of super-resolution reconstruction and shows the increase in resolution at high sensitivity and with the ability to make quantitative measurements.
Abstract: The conventional form of computed tomography using X-ray attenuation without any contrast agents is of limited use for the characterization of soft tissue in many fields of medical and biological studies. Grating-based phase-contrast computed tomography (gbPC-CT) is a promising alternative imaging method solving the low soft tissue contrast without the need of any contrast agent. While highly sensitive measurements are possible using conventional X-ray sources the spatial resolution does often not fulfill the requirements for specific imaging tasks, such as visualization of pathologies. The focus of this study is the increase in spatial resolution without loss of sensitivity. To overcome this limitation a super-resolution reconstruction based on sub-pixel shifts involving a deconvolution of the image data during each iteration is applied. In our study we achieve an effective pixel size of 28 μm with a conventional rotating anode tube and a photon-counting detector. We also demonstrate that the method can upgrade existing setups to measure tomographies with higher resolution. The results show the increase in resolution at high sensitivity and with the ability to make quantitative measurements. The combination of sparse sampling and statistical iterative reconstruction may be used to reduce the total measurement time. In conclusion, we present high-quality and high-resolution tomographic images of biological samples to demonstrate the experimental feasibility of super-resolution reconstruction.
37 citations
TL;DR: The SRNPU is the first ASIC implementation of the CNN-based SR algorithm which supports real-time Full-HD up-scaling and achieves higher restoration performance and power efficiency than previous SR hardware implementations.
Abstract: In this article, we propose an energy-efficient convolutional neural network (CNN) based super-resolution (SR) processor, super-resolution neural processing unit (SRNPU), for mobile applications. Traditionally, it is hard to realize real-time CNN-based SR on resource-limited platforms like mobile devices due to its massive amount of computation workload and communication bandwidth with external memory. The SRNPU can support the tile-based selective super-resolution (TSSR) which dynamically selects the proper sized CNN in a tile-by-tile manner. The TSSR reduces the computational workload of CNN-based SR by 31.1 % while maintaining image restoration performance. Moreover, a proposed selective caching based convolutional layer fusion (SC2LF) can reduce 78.8 % of external memory bandwidth with 93.2 % smaller on-chip memory footprint compared with previous layer fusion methods, by only caching short reuse distance intermediate feature maps. Additionally, reconfigurable cyclic ring architecture in the SRNPU enables maintaining high PE utilization by amortizing the reloading process caused by SC2LF operation under various convolutional layer configurations. The SRNPU is fabricated in 65 nm CMOS technology and occupies $4 \times 4$ mm2 die area. The SRNPU has a peak power efficiency of 1.9 TOPS/W at 0.75 V, 50 MHz. The SRNPU achieves 31.8 fps $\times 2$ scale Full-HD generation and 88.3 fps $\times 4$ scale Full-HD generation with higher restoration performance and power efficiency than previous SR hardware implementations. To the best of our knowledge, the SRNPU is the first ASIC implementation of the CNN-based SR algorithm which supports real-time Full-HD up-scaling.
36 citations
Cites background from "Image super-resolution"
...For instance, to maintain an appropriate video frame rate under an unstable communication environment or to reduce memory footprint requirement on resource-limited monitoring systems [5], [6]....
[...]
22 Jul 2018
TL;DR: The experimental results demonstrate that the proposed SR method based on a transferred generative adversarial network (TGAN) is superior to SRCNN and SRGAN in terms of both the objective evaluation and the subjective perspective.
Abstract: Single image super-resolution (SR) has been widely studied in recent years as a crucial technique for remote sensing applications. This paper proposes a SR method for remote sensing images based on a transferred generative adversarial network (TGAN). Different from the previous GAN-based SR approaches, the novelty of our method mainly reflects from two aspects. First, the batch normalization layers are removed to reduce the memory consumption and the computational burden, as well as raising the accuracy. Second, our model is trained in a transfer-learning fashion to cope with the insufficiency of training data, which is the crux of applying deep learning methods to remote sensing applications. The model is firstly trained on an external dataset DIV2K and further fine-tuned with the remote sensing dataset. Our experimental results demonstrate that the proposed method is superior to SRCNN and SRGAN in terms of both the objective evaluation and the subjective perspective.
34 citations
Cites background from "Image super-resolution"
...More and more researchers prefer to reconstruct high resolution(HR) images from low resolution (LR) ones by an image processing technology called super-resolution (SR), rather than devoting to physical imaging technology [1]....
[...]
TL;DR: An aerial image super-resolution method by training convolutional neural networks with respect to wavelet analysis to enable image restoration subject to sophisticated culture variability and validate the effectiveness of the method for restoring complicated aerial images.
Abstract: We develop an aerial image super-resolution method by training convolutional neural networks (CNNs) with respect to wavelet analysis. To this end, we commence by performing wavelet decomposition to aerial images for multiscale representations. We then train multiple CNNs for approximating the wavelet multiscale representations, separately. The multiple CNNs thus trained characterize aerial images in multiple directions and multiscale frequency bands, and thus enable image restoration subject to sophisticated culture variability. For inference, the trained CNNs regress wavelet multiscale representations from a low-resolution aerial image, followed by wavelet synthesis that forms a restored high-resolution aerial image. Experimental results validate the effectiveness of our method for restoring complicated aerial images.
34 citations
Cites background from "Image super-resolution"
...It is observed that image representations at different frequencies reflect different features for super resolution, where high-frequency details play an important role [7]....
[...]
References
More filters
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Abstract: Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
40,609 citations
Book•
23 May 2011TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
17,433 citations
TL;DR: It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.
Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the
16,538 citations
"Image super-resolution" refers background in this paper
...As the l2 norm represents a smoothing prior and the l1 norm tends to preserve the edges, the lp ( ≤ ≤ p 1 2) norm achieves a balance between them, thereby avoiding the staircase effect [110]....
[...]
TL;DR: In this article, a constrained optimization type of numerical algorithm for removing noise from images is presented, where the total variation of the image is minimized subject to constraints involving the statistics of the noise.
15,225 citations
"Image super-resolution" refers background in this paper
...[93,103], based on the fact that an image is naturally “blocky” and discontinuous....
[...]
Book•
01 Jan 1977
8,009 citations
"Image super-resolution" refers background in this paper
...In the early years, the smoothness of natural images was mainly considered, which leads to the quadratic property of the regularizations [99,100]....
[...]