Super-resolution image reconstruction

TLDR: The SR image reconstruction method estimates an HR image with finer spectral details from multiple LR observations degraded by blur, noise, and aliasing, and the major advantage of this approach is that it may cost less and the existing LR imaging systems can still be utilized.

Abstract: The spatial resolution that represents the number of pixels per unit area in an image is the principal factor in determining the quality of an image. With the development of image processing applications, there is a big demand for high-resolution (HR) images since HR images not only give the viewer a pleasing picture but also offer additional detail that is important for the analysis in many applications. The current technology to obtain HR images mainly depends on sensor manufacturing technology that attempts to increase the number of pixels per unit area by reducing the pixel size. However, the cost for high-precision optics and sensors may be inappropriate for general purpose commercial applications, and there is a limitation to pixel size reduction due to shot noise encountered in the sensor itself. Therefore, a resolution enhancement approach using signal processing techniques has been a great concern in many areas, and it is called super-resolution (SR) (or HR) image reconstruction or simply resolution enhancement in the literature. In this issue, we use the term “SR image reconstruction” to refer to a signal processing approach toward resolution enhancement, because the term “super” very well represents the characteristics of the technique overcoming the inherent resolution limitation of low-resolution (LR) imaging systems. The term SR was originally used in optics, and it refers to the algorithms that mainly operate on a single image to extrapolate the spectrum of an object beyond the diffraction limit (SR restoration). These two SR concepts (SR reconstruction and SR restoration) have a common focus in the aspect of recovering high-frequency information that is lost or degraded during the image acquisition. However, the cause of the loss of high-frequency information differs between these two concepts. SR restoration in optics attempts to recover information beyond the diffraction cutoff frequency, while the SR reconstruction method in engineering tries to recover high-frequency components corrupted by aliasing. We hope that readers do not confuse the super resolution in this issue with the term super resolution used in optics. SR image reconstruction algorithms investigate the relative motion information between multiple LR images (or a video sequence) and increase the spatial resolution by fusing them into a single frame. In doing so, it also removes the effect of possible blurring and noise in the LR images. In summary, the SR image reconstruction method estimates an HR image with finer spectral details from multiple LR observations degraded by blur, noise, and aliasing. The major advantage of this approach is that it may cost less and the existing LR imaging systems can still be utilized. Considering the maturity of this field and its various prospective applications, it seems timely and appropriate to discuss and adjust the topic of SR in the special issue of the magazine, since we do not have enough materials for ready disposal. This special section contains five articles covering various aspects of SR techniques. The first article, “Super-Resolution Image Reconstruction: A Technical Overview” by Sungcheol Park, Minkyu Park, and Moon Gi Kang, provides an introduction to the concepts and definitions of the SR image reconstruction as well as an overview of various existing SR algorithms. Advanced issues that are currently under investigation in this area are also discussed. The second article, “High-Resolution Images from Low-Resolution Compressed Video,” by Andrew C. Segall, Rafael Molina, and Aggelos K. Katsaggelos, considers the SR techniques for compressed video. Since images are routinely compressed prior to transmission and storage in current acquisition systems, it is important to take into account the characteristics of compression systems in developing the SR techniques. In this article, they survey models for the compression system and develop SR techniques within the Bayesian framework. The third article, by Deepu Rajan, Subhasis Chaudhuri, and Manjunath V. Joshi, titled “Multi-Objective Super-Resolution Technique: Concept and Examples,”