Image super-resolution
TL;DR: This paper aims to provide a review of SR from the perspective of techniques and applications, and especially the main contributions in recent years, and discusses the current obstacles for future research.
About: This article is published in Signal Processing.The article was published on 2016-11-01. It has received 378 citations till now.
Citations
More filters
19 Nov 2018-ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
TL;DR: Visual comparison and accuracy assessment on the basis of confusion matrix and Pearson’s Kappa coefficient revealed that SRP super-resolved output classified using radial basis function (RBF) kernel based SVM is the best outcome thereby highlighting the superiority of SR over simple scaling up and resampling approaches.
Abstract: . Urban areas despite being heterogeneous in nature are characterized as mixed pixels in medium to coarse resolution imagery which renders their mapping as highly inaccurate. A detailed classification of urban areas therefore needs both high spatial and spectral resolution marking the essentiality of different satellite data. Hyperspectral sensors with more than 200 contiguous bands over a narrow bandwidth of 1–10 nm can distinguish identical land use classes. However, such sensors possess low spatial resolution. As the exchange of rich spectral and spatial information is difficult at hardware level resolution enhancement techniques like super resolution (SR) hold the key. SR preserves the spectral characteristics and enables feature visualization at a higher spatial scale. Two SR algorithms: Anchored Neighbourhood Regression (ANR) and Sparse Regression and Natural Prior (SRP) have been executed on an airborne hyperspectral scene of Advanced Visible/Near Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) for the mixed environment centred on Kankaria Lake in the city of Ahmedabad thereby bringing down the spatial resolution from 8.1 m to 4.05 m. The generated super resolved outputs have been then used to map ten urban material and land cover classes identified in the study area using supervised Spectral Angle Mapper (SAM) and Support Vector Machine (SVM) classification methods. Visual comparison and accuracy assessment on the basis of confusion matrix and Pearson’s Kappa coefficient revealed that SRP super-resolved output classified using radial basis function (RBF) kernel based SVM is the best outcome thereby highlighting the superiority of SR over simple scaling up and resampling approaches.
4 citations
TL;DR: This paper proposes a novel face hallucination algorithm to embed group patches for accurate prior representation and reconstruction and demonstrates the superiority of the proposed method when compared with state-of-the-art face hallucinated results both on subjective and objective qualities.
Abstract: Face hallucination refers an application-specific super-resolution (SR) which predicts high-resolution images from one or multiple low-resolution inputs. Learning-based SR algorithms infer latent HR images by the guidance of coexisted priors from training samples. Various regularization methods have been successfully applied in face hallucination to ameliorate its ill-posed nature. But most of them only consider the local manifold geometry of a single patch which results in an unstable solution for SR reconstruction. In this paper, we propose a novel face hallucination algorithm to embed group patches for accurate prior representation and reconstruction. First, we select multiple recurrent self-similar patches to form a group embedding matrix. Then, a graph regularization term and another multiple-manifold regularization term are used to exploit accurate representation for SR performance. Our resulting ADMM algorithm gives a stable solution in an iterative manner. Furthermore, we use a two-step searching strategy for accelerated patch matching. Experimental results on the LFW database, FEI database, and some real-world images demonstrate the superiority of the proposed method when compared with state-of-the-art face hallucination results both on subjective and objective qualities.
4 citations
Cites background from "Image super-resolution"
...Therefore, it is necessary to enhance the resolution and quality of the LR input facial images by super-resolution (SR) algorithms [1]....
[...]
TL;DR: This article proposes to automatically select a set of frames from the depth sequence of the camera because they provide a good view of the face in terms of pose and distance and designs a progressive refinement approach to reconstruct a higher-resolution model from the selected low-resolution frames.
Abstract: Face recognition from two-dimensional (2D) still images and videos is quite successful even with “in the wild” conditions. Instead, less consolidated results are available for the cases in which face data come from non-conventional cameras, such as infrared or depth. In this article, we investigate this latter scenario assuming that a low-resolution depth camera is used to perform face recognition in an uncooperative context. To this end, we propose, first, to automatically select a set of frames from the depth sequence of the camera because they provide a good view of the face in terms of pose and distance. Then, we design a progressive refinement approach to reconstruct a higher-resolution model from the selected low-resolution frames. This process accounts for the anisotropic error of the existing points in the current 3D model and the points in a newly acquired frame so that the refinement step can progressively adjust the point positions in the model using a Kalman-like estimation. The quality of the reconstructed model is evaluated by considering the error between the reconstructed models and their corresponding high-resolution scans used as ground truth. In addition, we performed face recognition using the reconstructed models as probes against a gallery of reconstructed models and a gallery with high-resolution scans. The obtained results confirm the possibility to effectively use the reconstructed models for the face recognition task.
4 citations
TL;DR: In this article , a new structure variance loss is introduced to build a bridge between deep encoder-decoders and variance minimization, and provide a new way to minimize the variance by forcing different intermediate decoding outputs (paths) to reach an agreement.
Abstract: Deep encoder–decoders are the model of choice for pixel-level estimation due to their redundant deep architectures. Yet they still suffer from the vanishing supervision information issue that affects convergence because of their overly deep architectures. In this work, we propose and theoretically derive an enhanced deep supervision (EDS) method which improves on conventional deep supervision (DS) by incorporating variance minimization into the optimization. A new structure variance loss is introduced to build a bridge between deep encoder–decoders and variance minimization, and provides a new way to minimize the variance by forcing different intermediate decoding outputs (paths) to reach an agreement. We also design a focal weighting strategy to effectively combine multiple losses in a scale-balanced way, so that the supervision information is sufficiently enforced throughout the encoder–decoders. To evaluate the proposed method on the pixel-level estimation task, a novel multipath residual encoder is proposed and extensive experiments are conducted on four challenging density estimation and crowd counting benchmarks. The experimental results demonstrate the superiority of our EDS over other paradigms, and improved estimation performance is reported using our deeply supervised encoder–decoder.
4 citations
TL;DR: This work proposes the partially supervised strategy to match the most corresponding mapping matrix, which considers the class information moderately to estimate the HR patches, and outperforms other competing methods in terms of both quantity and quality.
4 citations
References
More filters
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Abstract: Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
40,609 citations
Book•
23 May 2011TL;DR: It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
17,433 citations
TL;DR: It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.
Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the
16,538 citations
"Image super-resolution" refers background in this paper
...As the l2 norm represents a smoothing prior and the l1 norm tends to preserve the edges, the lp ( ≤ ≤ p 1 2) norm achieves a balance between them, thereby avoiding the staircase effect [110]....
[...]
TL;DR: In this article, a constrained optimization type of numerical algorithm for removing noise from images is presented, where the total variation of the image is minimized subject to constraints involving the statistics of the noise.
15,225 citations
"Image super-resolution" refers background in this paper
...[93,103], based on the fact that an image is naturally “blocky” and discontinuous....
[...]
Book•
01 Jan 1977
8,009 citations
"Image super-resolution" refers background in this paper
...In the early years, the smoothness of natural images was mainly considered, which leads to the quadratic property of the regularizations [99,100]....
[...]