Super-resolution, the process of obtaining one or more high-resolution images from one or more low-resolution observations, has been a very attractive research topic over the last two decades. It has found practical applications in many real-world problems in different fields, from satellite and aerial imaging to medical image processing, to facial image analysis, text image analysis, sign and number plates reading, and biometrics recognition, to name a few. This has resulted in many research papers, each developing a new super-resolution algorithm for a specific purpose. The current comprehensive survey provides an overview of most of these published works by grouping them in a broad taxonomy. For each of the groups in the taxonomy, the basic concepts of the algorithms are first explained and then the paths through which each of these groups have evolved are given in detail, by mentioning the contributions of different authors to the basic concepts of each group. Furthermore, common issues in super-resolution algorithms, such as imaging models and registration algorithms, optimization of the cost functions employed, dealing with color information, improvement factors, assessment of super-resolution algorithms, and the most commonly employed databases are discussed.

/pdf/super-resolution-a-comprehensive-survey-xjl42yt9nt.pdf

Super-resolution: a comprehensive survey

This paper addresses the very low resolution (VLR) problem in face recognition in which the resolution of the face image to be recognized is lower than 16 × 16. With the increasing demand of surveillance camera-based applications, the VLR problem happens in many face application systems. Existing face recognition algorithms are not able to give satisfactory performance on the VLR face image. While face super-resolution (SR) methods can be employed to enhance the resolution of the images, the existing learning-based face SR methods do not perform well on such a VLR face image. To overcome this problem, this paper proposes a novel approach to learn the relationship between the high-resolution image space and the VLR image space for face SR. Based on this new approach, two constraints, namely, new data and discriminative constraints, are designed for good visuality and face recognition applications under the VLR problem, respectively. Experimental results show that the proposed SR algorithm based on relationship learning outperforms the existing algorithms in public face databases.

Very Low Resolution Face Recognition Problem

The key objective of super-resolution (SR) imaging is to reconstruct a higher-resolution image based on a set of images, acquired from the same scene and denoted as ‘low-resolution’ images, to overcome the limitation and/or ill-posed conditions of the image acquisition process for facilitating better content visualization and scene recognition. In this paper, we provide a comprehensive review of SR image and video reconstruction methods developed in the literature and highlight the future research challenges. The SR image approaches reconstruct a single higher-resolution image from a set of given lower-resolution images, and the SR video approaches reconstruct an image sequence with a higher-resolution from a group of adjacent lower-resolution image frames. Furthermore, several SR applications are discussed to contribute some insightful comments on future SR research directions. Specifically, the SR computations for multi-view images and the SR video computation in the temporal domain are discussed.

A survey on super-resolution imaging

1. Introduction.- 1 Fundamentals of the Theory of Digital Signal Processing.- 2. Elements of Signal Theory.- 2.1 Signals as Mathematical Functions.- 2.2 Signal Space.- 2.3 The Most Common Systems of Basis Functions.- 2.3.1 Impulse Basis Functions.- 2.3.2 Harmonic Functions.- 2.3.3 Walsh Functions.- 2.3.4 Haar Functions.- 2.3.5 Sampling Functions.- 2.4 Continuous Representations of Signals.- 2.5 Description of Signal Transformations.- 2.5.1 Linear Transformations.- 2.5.2 Nonlinear Element-by-Element Transforms.- 2.6 Representation of Linear Transformations with Respect to Discrete Bases.- 2.6.1 Representation Using Vector Responses.- 2.6.2 Matrix Representations.- 2.6.3 Representation of Operators by Means of Their Eigenfunctions and Eigenvalues.- 2.7 Representing Operator with Respect to Continuous Bases.- 2.7.1 Operator Kernel.- 2.7.2 Description in Terms of Impulse Responses.- 2.7.3 Description Using Frequency Transfer Functions.- 2.7.4 Description with Input and Output Signals Referred to Different Bases.- 2.7.5 Description Using Eigenfunctions.- 2.8 Examples of Linear Operators.- 2.8.1 Shift-Invariant Filters.- 2.8.2 The Identity Operator.- 2.8.3 Shi ft Operator.- 2.8.4 Sampling Oparator.- 2.8.5 Gating Operator (Multiplier).- 3. Discretization and Quantization of Signals.- 3.1 Generalized Quantization.- 3.2 Concepts of Discretization and Element-by-Element Quantization.- 3.2.1 Discretization.- 3.2.2 Element-by-Element Quantization.- 3.3 The Sampling Theorem.- 3.4 Sampling Theory for Two-Dimensional Signals.- 3.5 Errors of Discretization and Restoration of Signals in Sampling Theory.- 3.6 Other Approaches to Discretization.- 3.7 Optimal Discrete Representation and Dimensionality of Signals.- 3.8 Element-by-Element Quantization.- 3.9 Examples of Optimum Quantization.- 3.9.1 Example: The Threshold Metric.- 3.9.2 Example: Power Criteria for the Absolute Value of the Quantization Error.- 3.9.3 Example: Power Criteria for the Relative Quantization Error.- 3.10 Quantization in the Presence of Noise. Quantization and Representation of Numbers in Digital Processors.- 3.11 Review of Picture-Coding Methods.- 4. Discrete Representations of Linear Transforms.- 4.1 Problem Formulation and General Approach.- 4.2 Discrete Representation of Shift-Invariant Filters for Band-Limited Signals.- 4.3 Digital Filters.- 4.4 Transfer Functions and Impulse Responses of Digital Filters.- 4.5 Boundary Effects in Digital Filtering.- 4.6 The Discrete Fourier Transform (DFT).- 4.7 Shifted, Odd and Even DFTs.- 4.8 Using Discrete Fourier Transforms.- 4.8.1 Calculating Convolutions.- 4.8.2 Signal Interpolation.- 4.9 Walsh and Similar Transforms.- 4.10 The Haar Tansform. Addition Elements of Matrix Calculus.- 4.11 Other Orthogonal Tansforms. General Representations. Review of Applications.- 5. Linear Transform Algorithms.- 5.1 Fast Algorithms of Discrete Orthogonal Transforms.- 5.2 Fast Haar Transform (FHT) Algorithms.- 5.3 Fast Walsh Transform (FWT) Algorithms.- 5.4 Fast Discrete Fourier Transform (FFT) Algorithms.- 5.5 Review of Other Fast Algorithms. Features of Two-Dimensional Transforms.- 5.5.1 Truncated FFT and FWT Algorithms.- 5.5.2 Transition Matrices Between Various Transforms.- 5.5.3 Calculation of Two-Dimensional Transforms.- 5.6 Combined DFT Algorithms.- 5.6.1 Combined DFT Algorithms of Real Sequences.- 5.6.2 Combined SDFT (1/2, 0) Algorithms of Even and Real Even Sequences.- 5.7 Recursive DFT Algorithms.- 5.8 Fast Algorithms for Calculating the DFT and Signal Convolution with Decreased Multiplication.- 6. Digital Statistical Methods.- 6.1 Principles of the Statistical Description of Pictures.- 6.2 Measuring the Grey-Level Distribution.- 6.2.1 Step Smoothing.- 6.2.2 Smoothing by Sliding Summation.- 6.2.3 Smoothing with Orthogonal Transforms.- 6.3 The Estimation of Correlation Functions and Spectra.- 6.3.1 Averaging Local Spectra.- 6.3.2 Masking (Windowing) the Process by Smooth Functions.- 6.3.3 Direct Smoothing of Spectra.- 6.4 Generating Pseudorandom Numbers.- 6.5 Measuring Picture Noise.- 6.5.1 The Prediction Method.- 6.5.2 The Voting Method.- 6.5.3 Measuring the Variance and the Auto-Correlation Function of Additive Wideband Noise.- 6.5.4 Evaluation of the Intensity and Frequency of the Harmonic Components of Periodic Interference and Other Types of Interference with Narrow Spectra.- 6.5.5 Evaluation of the Parameters of Pulse Noise, Quantization Noise and Strip-Like Noise.- 2 Picture Processing.- 7. Correcting Imaging Systems.- 7.1 Problem Formulation.- 7.2 Suppression of Additive Noise by Linear Filtering.- 7.3 Filtering of Pulse Interference.- 7.4 Correction of Linear Distortion.- 7.5 Correction of Amplitude Characteristics.- 8. Picture Enhancement and Preparation.- 8.1 Preparation Problems and Visual Analysis of Pictures.- 8.1.1 Feature Processing.- 8.1.2 Geometric Transformations.- 8.2 Adaptive Quantization of Modes.- 8.3 Preparation by Nonlinear Transformation of the Video Signal Scale.- 8.4 Linear Preparation Methods.- 8.5 Methods of Constructing Graphical Representation: Computer Graphics.- 8.6 Geometric Picture Transformation.- 8.6.1 Bilinear Interpolation.- 8.6.2 Interpolation Using DFT and SDFT.- 9. Measuring the Coordinates of Objects in Pictures.- 9.1 Problem Formulation.- 9.2 Localizing a Precisely Known Object in a Spatially Homogeneous Picture.- 9.3 Uncertainty in the Object and Picture Inhomogeneity. Localization in "Blurred" Pictures.- 9.3.1 "Exhaustive" Estimator.- 9.3.2 Estimator Seeking an Averaged Object.- 9.3.3 Adjustable Estimator with Fragment-by Fraament Optimal Fi1tering.- 9.3.4 Non-Adjustable Estimator.- 9.3.5 Localization in Blurred and Noisy Pictures.- 9.4 Optimal Localization and Picture Contours. Choice of Reference Objects.- 9.5 Algorithm for the Automatic Detection and Extraction of Bench-Marks in Aerial and Space Photographs.- 10-Conclusion.- References.

Digital picture processing : an introduction

This paper addresses the very low resolution (VLR) problem in face recognition in which the resolution of face image to be recognized is lower than 16×16. The VLR problem happens in many surveillance camera-based applications and existing face recognition algorithms are not able to give satisfactory performance on VLR face image. While face super-resolution (SR) methods can be employed to enhance the resolution of the images, the existing learning-based face SR methods do not perform well on such a very low resolution face image. To overcome this problem, this paper models the SR problem under VLR case as a regression problem with two constraints. First, a new data constraint is design to perform the error measurement on high resolution image space which provides more detailed and discriminative information. Second, discriminative constraint is proposed and incorporated in the training stage so that the reconstructed HR image has higher discriminability. CMU-PIE, FRGC and surveillant camera face (SCface) databases are selected for experiments. Experimental results show that the proposed method outperforms the existing methods, in terms of image quality and recognition accuracy.

/pdf/very-low-resolution-face-recognition-problem-3bkyykjcbm.pdf

Very low resolution face recognition problem

Imaging plays a key role in many diverse areas of application, such as astronomy, remote sensing, microscopy, and tomography. Owing to imperfections of measuring devices (e.g., optical degradations, limited size of sensors) and instability of the observed scene (e.g., object motion, media turbulence), acquired images can be indistinct, noisy, and may exhibit insufficient spatial and temporal resolution. In particular, several external effects blur images. Techniques for recovering the original image include blind deconvolution (to remove blur) and superresolution (SR). The stability of these methods depends on having more than one image of the same frame. Differences between images are necessary to provide new information, but they can be almost unperceivable. State-of-the-art SR techniques achieve remarkable results in resolution enhancement by estimating the subpixel shifts between images, but they lack any apparatus for calculating the blurs. In this paper, after introducing a review of current SR techniques we describe two recently developed SR methods by the authors. First, we introduce a variational method that minimizes a regularized energy function with respect to the high resolution image and blurs. In this way we establish a unifying way to simultaneously estimate the blurs and the high resolution image. By estimating blurs we automatically estimate shifts with subpixel accuracy, which is inherent for good SR performance. Second, an innovative learning-based algorithm using a neural architecture for SR is described. Comparative experiments on real data illustrate the robustness and utilization of both methods.

/pdf/superresolution-imaging-a-survey-of-current-techniques-2k1r15mrk5.pdf

Superresolution imaging: a survey of current techniques

This paper presents a new image denoising method based on truncating the original noisy coefficients of a Pseudo-
Wigner distribution (PWD) calculated through 1D directional windows. This method has been tested both for
additive and multiplicative noisy images. The coefficients are selected according to their local directionality to
take into account the image anisotropy. Next, the PWD is inverted and the set of different directional images
are averaged. When the ground truth image reference is available, the peak signal-to-noise ratio (PSNR) metric
is used to evaluate the resulting denoised images in comparison with other alternative methods. The described
method is based on the use of the Renyi entropy extracted from a joint spatial frequency representation such as
the Wigner distribution. A comparison with other competitive techniques is described and tested for real-world
images. In particular, some experimental results are presented in the area of synthetic aperture radar (SAR)
and retinal imaging, showing the effectiveness of the method in comparison with other alternative techniques
through the use of two different non-reference image quality metrics.

Image denoising and quality assessment through the Renyi entropy

This paper presents the results of measuring the image quality of a video compression system based in the H.264
standard using the Anisotropic Quality Index (AQI). These results have been compared with the quality measured by
means of the traditionally used Peak Signal to Noise Ratio (PSNR). The PSNR has demonstrated to be an unreliable way
to compute the perceptual quality of images. Although it is widely used because its simplicity and immediacy to be
computed, the PSNR and other methods based in the image differences measurement (as the Root Mean Squared Error
or RMSE) experience the problem of not properly reflecting the real perceptual image quality. Images with the same
amount of noise can present similar PSNRs values even with very different perceptual appearance. In the other side, the
AQI has proven to be a more reliable way to analytically measure the perceptual image quality. This new measure is
based on the use of a particular type of the high-order Renyi entropies. This method is based on measuring the anisotropy
of the image through the variance of the expected value of the pixel-wise directional image entropy. Moreover, the AQI
has the additional benefit of not needing a reference image. The reference image, compulsory in the PSNR computation,
is usually impossible to obtain in real situations, thus relegating the PSNR only to test-bench developments. The
possibility of computing the AQI opens the ability of self-regulated compression systems based on the adjustment of
parameters that exhibit greater influence on the final image quality. This work shows the results of compressing several
standard video sequences using the H.264 video compression standard. Compared with the PSNR, the AQI represents a
better indicator of the perceptual quality of images.

Elena Gil

Papers

Superresolution imaging: a survey of current techniques

Image denoising and quality assessment through the Renyi entropy

Anisotropic quality measurement applied to H.264 video compression