A new approach toward increasing spatial resolution is required to overcome the limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an high-resolution (HR) image (or sequence) from observed multiple low-resolution (LR) images. Such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement. In this article, we use the term "SR image reconstruction" to refer to a signal processing approach toward resolution enhancement because the term "super" in "super resolution" represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts. To this purpose, we present the technical review of various existing SR methodologies which are often employed. Before presenting the review of existing SR algorithms, we first model the LR image acquisition process.

Super-resolution image reconstruction: a technical overview

Super-resolution reconstruction produces one or a set of high-resolution images from a set of low-resolution images. In the last two decades, a variety of super-resolution methods have been proposed. These methods are usually very sensitive to their assumed model of data and noise, which limits their utility. This paper reviews some of these methods and addresses their shortcomings. We propose an alternate approach using L/sub 1/ norm minimization and robust regularization based on a bilateral prior to deal with different data and noise models. This computationally inexpensive method is robust to errors in motion and blur estimation and results in images with sharp edges. Simulation results confirm the effectiveness of our method and demonstrate its superiority to other super-resolution methods.

Fast and robust multiframe super resolution

Super-resolution reconstruction proposes a fusion of several low-quality images into one higher quality result with better optical resolution. Classic super-resolution techniques strongly rely on the availability of accurate motion estimation for this fusion task. When the motion is estimated inaccurately, as often happens for nonglobal motion fields, annoying artifacts appear in the super-resolved outcome. Encouraged by recent developments on the video denoising problem, where state-of-the-art algorithms are formed with no explicit motion estimation, we seek a super-resolution algorithm of similar nature that will allow processing sequences with general motion patterns. In this paper, we base our solution on the Nonlocal-Means (NLM) algorithm. We show how this denoising method is generalized to become a relatively simple super-resolution algorithm with no explicit motion estimation. Results on several test movies show that the proposed method is very successful in providing super-resolution on general sequences.

Generalizing the Nonlocal-Means to Super-Resolution Reconstruction

Super-Resolution reconstruction produces one or a set of high-resolution images from a sequence of low-resolution frames. This article reviews a variety of Super-Resolution methods proposed in the last 20 years, and provides some insight into, and a summary of, our recent contributions to the general Super-Resolution problem. In the process, a detailed study of several very important aspects of Super-Resolution, often ignored in the literature, is presented. Spe- cifically, we discuss robustness, treatment of color, and dynamic operation modes. Novel methods for addressing these issues are accompanied by experimental results on simulated and real data. Finally, some future challenges in Super-Resolution are outlined and discussed. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 14, 47-57, 2004; Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/ima.20007

/pdf/advances-and-challenges-in-super-resolution-k5ja87c9m5.pdf

Advances and Challenges in Super-Resolution

Super-resolution, the process of obtaining one or more high-resolution images from one or more low-resolution observations, has been a very attractive research topic over the last two decades. It has found practical applications in many real-world problems in different fields, from satellite and aerial imaging to medical image processing, to facial image analysis, text image analysis, sign and number plates reading, and biometrics recognition, to name a few. This has resulted in many research papers, each developing a new super-resolution algorithm for a specific purpose. The current comprehensive survey provides an overview of most of these published works by grouping them in a broad taxonomy. For each of the groups in the taxonomy, the basic concepts of the algorithms are first explained and then the paths through which each of these groups have evolved are given in detail, by mentioning the contributions of different authors to the basic concepts of each group. Furthermore, common issues in super-resolution algorithms, such as imaging models and registration algorithms, optimization of the cost functions employed, dealing with color information, improvement factors, assessment of super-resolution algorithms, and the most commonly employed databases are discussed.

/pdf/super-resolution-a-comprehensive-survey-xjl42yt9nt.pdf

Super-resolution: a comprehensive survey

Superresolution reconstruction produces a high-resolution image from a set of low-resolution images. Previous iterative methods for superresolution had not adequately addressed the computational and numerical issues for this ill-conditioned and typically underdetermined large scale problem. We propose efficient block circulant preconditioners for solving the Tikhonov-regularized superresolution problem by the conjugate gradient method. We also extend to underdetermined systems the derivation of the generalized cross-validation method for automatic calculation of regularization parameters. The effectiveness of our preconditioners and regularization techniques is demonstrated with superresolution results for a simulated sequence and a forward looking infrared (FLIR) camera image sequence.

A computationally efficient superresolution image reconstruction algorithm

In many image restoration/resolution enhancement applications, the blurring process, i.e., point spread function (PSF) of the imaging system, is not known or is known only to within a set of parameters. We estimate these PSF parameters for this ill-posed class of inverse problem from raw data, along with the regularization parameters required to stabilize the solution, using the generalized cross-validation method (GCV). We propose efficient approximation techniques based on the Lanczos algorithm and Gauss quadrature theory, reducing the computational complexity of the GCV. Data-driven PSF and regularization parameter estimation experiments with synthetic and real image sequences are presented to demonstrate the effectiveness and robustness of our method.

Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement

Superresolution produces high-quality, high-resolution images from a set of degraded, low-resolution images where relative frame-to-frame motions provide different looks at the scene. Superresolution translates data temporal bandwith into enhanced spatial resolution. If considered together on a reference grid, given low-resolution data are nonuniformly sampled. However, data from each frame are sampled regularly on a rectangular grid. This special type of nonuniform sampling is called interlaced sampling. We propose a new wavelet-based interpolation-restoration algorithm for superresolution. Our efficient wavelet interpolation technique takes advantage of the regularity and structure inherent in interlaced data, thereby significantly reducing the computational burden. We present one- and two-dimensional superresolution experiments to demonstrate the effectiveness of our algorithm.

A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution)

Numerical algorithms for image superresolution

Superresolution produces high-quality, high-resolution images from a set of degraded, low-resolution images where relative frame-to-frame motions provide differ- ent looks at the scene. Superresolution translates data temporal bandwidth into enhanced spatial resolution. If considered together on a reference grid, given low-resolution data are nonuniformly sampled. However, data from each frame are sampled regularly on a rectangular grid. This special type of nonuniform sampling is called interlaced sampling. We propose a new wavelet-based interpolation-restoration algorithm for superresolution. Our efficient wavelet interpolation technique takes advantage of the regularity and structure inherent in interlaced data, thereby significantly reducing the computational burden. We present one- and two-dimensional superresolution experiments to demonstrate the effec- tiveness of our algorithm.

Nhat Nguyen

Papers

A computationally efficient superresolution image reconstruction algorithm

Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement

A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution)

Numerical algorithms for image superresolution

Interpolation-restoration method for superresolution (wavelet superresolution)*