Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
TL;DR: This work designs a novel residual balanced attention network (RBAN) as a generator to estimate super-resolution results from the LR inputs and applies a UNet-shape discriminator for adversarial training to encourage RBAN to generate more realistic textures.
Abstract: Limited resolution is one of the most important factors hindering the application of remote sensing images (RSIs). Single-image super resolution (SISR) is a technique to improve the spatial resolution of digital images and has attracted the attention of many researchers. In recent years, with the advancement of deep learning (DL) frameworks, many DL-based SISR models have been proposed and achieved state-of-the-art performance; however, most SISR models for RSIs use the bicubic downsampler to construct low-resolution (LR) and high-resolution (HR) training pairs. Considering that the quality of the actual RSIs depends on a variety of factors, such as illumination, atmosphere, imaging sensor responses, and signal processing, training on “ideal” datasets results in a dramatic drop in model performance on real RSIs. To address this issue, we propose to build a more realistic training dataset by modeling the degradation with blur kernels and imaging noises. We also design a novel residual balanced attention network (RBAN) as a generator to estimate super-resolution results from the LR inputs. To encourage RBAN to generate more realistic textures, we apply a UNet-shape discriminator for adversarial training. Both referenced evaluations on synthetic data and non-referenced evaluations on actual images were carried out. Experimental results validate the effectiveness of the proposed framework, and our model exhibits state-of-the-art performance in quantitative evaluation and visual quality. We believe that the proposed framework can facilitate super-resolution techniques from research to practical applications in RSIs processing.
7 citations
TL;DR: A denoised patch dictionary based single image super resolution algorithm is proposed to enhance the robustness to noise performance, and is further modified by using a wavelet based fusion algorithm which combines the result of proposed method with direct super resolution image, and super resolved image after denoising to preserve the finer details of thesuper resolved image.
7 citations
TL;DR: This paper presents the gradient-guided image super-resolution reconstruction for terahertz imaging to improve the image quality, taking advantage of super- resolution reconstruction based on adaptive super-pixel gradient field transform, and introduces spatial entropy-based enhancement and a bilateral filter to ensure better performance.
Abstract: This paper presents the gradient-guided image super-resolution reconstruction for terahertz imaging to improve the image quality, taking advantage of super-resolution reconstruction based on adaptive super-pixel gradient field transform. Moreover, spatial entropy-based enhancement and a bilateral filter are introduced to ensure better performance of the reconstruction. Furthermore, we compare the performance of reconstruction operated on terahertz images with frequencies of 0.1 THz, 0.3 THz, 0.5 THz, and 0.7 THz. Experimental results demonstrate that this method successfully improves the image quality and reconstruct high-resolution images from low-resolution images with the peak signal-to-noise ratio and structural similarity index improved. In addition, the signal frequency and intensity are demonstrated to affect the performance of reconstruction.
7 citations
Posted Content•
TL;DR: A comprehensive introduction of the linear transform based tensor algebra is provided from the signal processing viewpoint and a detailed case study of the ways and means of the low rank modeling in many signal processing applications are presented.
Abstract: Modeling of multidimensional signal using tensor is more convincing than representing it as a collection of matrices. The tensor based approaches can explore the abundant spatial and temporal structures of the mutlidimensional signal. The backbone of this modeling is the mathematical foundations of tensor algebra. The linear transform based tensor algebra furnishes low complex and high performance algebraic structures suitable for the introspection of the multidimensional signal. A comprehensive introduction of the linear transform based tensor algebra is provided from the signal processing viewpoint. The rank of a multidimensional signal is a precious property which gives an insight into the structural aspects of it. All natural multidimensional signals can be approximated to a low rank signal without losing significant information. The low rank approximation is beneficial in many signal processing applications such as denoising, missing sample estimation, resolution enhancement, classification, background estimation, object detection, deweathering, clustering and much more applications. Detailed case study of the ways and means of the low rank modeling in the above said signal processing applications are also presented.
7 citations
Cites background from "Image super-resolution"
...Low rank models and total variation functions are commonly used as regularization function in such inverse problems to stabilize the inversion process [57] [8] [34] [51]....
[...]
...14 and it can be mathematically denoted as, [45, 57], Yk = HkX + Nk , for k = 1 · · ·K (17) where, X is the HR image to synthesize, Hk denotes the combined effect of blurring and down sampling on kth observation Yk , Nk is the noise effects on k th image and K is the number of LR observations....
[...]
01 Mar 2021
TL;DR: In this article, a method is proposed to estimate the motion of a planar object in a video frame by minimizing the squared distance between the transformed image and a reference, computed over a user defined region of interest, and uses the partial derivatives in order to significantly speed up the computation.
Abstract: One task often encountered in surveillance videos is the recognition of a target—e.g. the license plate of a vehicle. Often, the quality of a single video frame does not permit a reliable recognition. If multiple frames are available, it is possible to combine them in order to generate a single image with lower noise (frame averaging) and/or higher resolution (super-resolution). In order for these techniques to work, it is necessary to accurately estimate the motion of the object of interest in the recorded footage. In this paper, we introduce a method capable of accurately computing the perspective transformation that describes the motion of a planar object. The method minimizes the squared distance between the transformed image and a reference, computed over a user-defined region of interest, and uses the partial derivatives in order to significantly speed up the computation. This approach is inspired by the well known Kanade–Lucas–Tomasi feature tracker.
7 citations
References
More filters
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Abstract: Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
40,609 citations
Book•
23 May 2011TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
17,433 citations
TL;DR: It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.
Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the
16,538 citations
"Image super-resolution" refers background in this paper
...As the l2 norm represents a smoothing prior and the l1 norm tends to preserve the edges, the lp ( ≤ ≤ p 1 2) norm achieves a balance between them, thereby avoiding the staircase effect [110]....
[...]
TL;DR: In this article, a constrained optimization type of numerical algorithm for removing noise from images is presented, where the total variation of the image is minimized subject to constraints involving the statistics of the noise.
15,225 citations
"Image super-resolution" refers background in this paper
...[93,103], based on the fact that an image is naturally “blocky” and discontinuous....
[...]
Book•
01 Jan 1977
8,009 citations
"Image super-resolution" refers background in this paper
...In the early years, the smoothness of natural images was mainly considered, which leads to the quadratic property of the regularizations [99,100]....
[...]