In many imaging systems, the detector array is not sufficiently dense to adequately sample the scene with the desired field of view. This is particularly true for many infrared focal plane arrays. Thus, the resulting images may be severely aliased. This paper examines a technique for estimating a high-resolution image, with reduced aliasing, from a sequence of undersampled frames. Several approaches to this problem have been investigated previously. However, in this paper a maximum a posteriori (MAP) framework for jointly estimating image registration parameters and the high-resolution image is presented. Several previous approaches have relied on knowing the registration parameters a priori or have utilized registration techniques not specifically designed to treat severely aliased images. In the proposed method, the registration parameters are iteratively updated along with the high-resolution image in a cyclic coordinate-descent optimization procedure. Experimental results are provided to illustrate the performance of the proposed MAP algorithm using both visible and infrared images. Quantitative error analysis is provided and several images are shown for subjective evaluation.

/pdf/joint-map-registration-and-high-resolution-image-estimation-nxfuinxwsn.pdf

Joint MAP registration and high-resolution image estimation using a sequence of undersampled images

This paper addresses the problem of recovering a super-resolved image from a set of warped blurred and decimated versions thereof. Several algorithms have already been proposed for the solution of this general problem. In this paper, we concentrate on a special case where the warps are pure translations, the blur is space invariant and the same for all the images, and the noise is white. We exploit previous results to develop a new highly efficient super-resolution reconstruction algorithm for this case, which separates the treatment into de-blurring and measurements fusion. The fusion part is shown to be a very simple non-iterative algorithm, preserving the optimality of the entire reconstruction process, in the maximum-likelihood sense. Simulations demonstrate the capabilities of the proposed algorithm.

A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur

A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.

/pdf/a-fast-image-super-resolution-algorithm-using-an-adaptive-4a8kx8r4zt.pdf

A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter

The attainment of super resolution (SR) from a sequence of degraded undersampled images could be viewed as reconstruction of the high-resolution (HR) image from a finite set of its projections on a sampling lattice. This can then be formulated as an optimization problem whose solution is obtained by minimizing a cost function. The approaches adopted and their analysis to solve the formulated optimization problem are crucial, The image acquisition scheme is important in the modeling of the degradation process. The need for model accuracy is undeniable in the attainment of SR along with the design of the algorithm whose robust implementation will produce the desired quality in the presence of model parameter uncertainty. To keep the presentation focused and of reasonable size, data acquisition with multisensors instead of, say a video camera is considered.

/pdf/mathematical-analysis-of-super-resolution-methodology-4uhipd1xhw.pdf

Mathematical analysis of super-resolution methodology

Forward looking infrared (FLIR) detector arrays generally produce spatially undersampled images because the FLIR arrays cannot be made dense enough to yield a sufficiently high spatial sampling frequency. Multi-frame techniques, such as microscanning, are an effective means of reducing aliasing and increasing resolution in images produced by staring imaging systems. These techniques involve interlacing a set of image frames that have been shifted with respect to each other during acquisition. The FLIR system is mounted on a moving platform, such as an aircraft, and the vibrations associated with the platform are used to generate the shifts. Since a fixed number of image frames is required, and the shifts are random, the acquired frames will not fall on a uniformly spaced grid. Furthermore, some of the acquired frames may have almost similar shifts thus making them unusable for high-resolution image reconstruction. In this paper, we utilize a gradient-based registration algorithm to estimate the shifts between the acquired frames and then use a weighted nearest-neighbor approach for placing the frames onto a uniform grid to form a final high-resolution image. Blurring by the detector and optics of the imaging system limits the increase in image resolution when microscanning is attempted at sub-pixel movements of less than half the detector width. We resolve this difficulty by the application of the Wiener filter, designed using the modulation transfer function (MTF) of the imaging system, to the high-resolution image. Simulation and experimental results are presented to verify the effectiveness of the proposed technique. The techniques proposed herein are significantly faster than alternate techniques, and are found to be especially suitable for real-time applications.

Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames

We introduce and analyze techniques for the reduction of aliased signal energy in a staring infrared imaging system. A standard staring system uses a fixed two-dimensional detector array that corresponds to a fixed spatial sampling frequency determined by the detector pitch or spacing. Aliasing will occur when sampling a scene containing spatial frequencies exceeding half the sampling frequency. This aliasing can significantly degrade the image quality. The aliasing reduction schemes presented here, referred to as microscanning, exploit subpixel shifts between time frames of an image sequence. These multiple images are used to reconstruct a single frame with reduced aliasing. If the shifts are controlled, using a mirror or beam steerer for example, one can obtain a uniformly sampled microscanned image. The reconstruction in this case can be accomplished by a straightforward interlacing of the time frames. If the shifts are uncontrolled, the effective sampling may be nonuniform and reconstruction becomes more complex. A sampling
model is developed and the aliased signal energy is analyzed for the microscanning techniques. Finally, a number of experimental results are presented that illustrate the perlormance of the microscanning methods.

/pdf/aliasing-reduction-in-staring-infrared-imagers-utilizing-2bg2d1tdv6.pdf

Thomas M. Stadtmiller

Papers

Aliasing reduction in staring infrared imagers utilizing subpixel techniques