Preserving edge structures is a challenge to image interpolation algorithms that reconstruct a high-resolution image from a low-resolution counterpart. We propose a new edge-guided nonlinear interpolation technique through directional filtering and data fusion. For a pixel to be interpolated, two observation sets are defined in two orthogonal directions, and each set produces an estimate of the pixel value. These directional estimates, modeled as different noisy measurements of the missing pixel are fused by the linear minimum mean square-error estimation (LMMSE) technique into a more robust estimate, using the statistics of the two observation sets. We also present a simplified version of the LMMSE-based interpolation algorithm to reduce computational cost without sacrificing much the interpolation performance. Experiments show that the new interpolation techniques can preserve edge sharpness and reduce ringing artifacts

/pdf/an-edge-guided-image-interpolation-algorithm-via-directional-eofu18ql3a.pdf

An edge-guided image interpolation algorithm via directional filtering and data fusion

Designing effective image priors is of great interest to image super-resolution (SR), which is a severely under-determined problem. An edge smoothness prior is favored since it is able to suppress the jagged edge artifact effectively. However, for soft image edges with gradual intensity transitions, it is generally difficult to obtain analytical forms for evaluating their smoothness. This paper characterizes soft edge smoothness based on a novel SoftCuts metric by generalizing the Geocuts method . The proposed soft edge smoothness measure can approximate the average length of all level lines in an intensity image. Thus, the total length of all level lines can be minimized effectively by integrating this new form of prior. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image SR, by adaptively normalizing image edges according to their alpha-channel description. This leads to the adaptive SoftCuts algorithm, which represents a unified treatment of edges with different contrasts and scales. Experimental results are presented which demonstrate the effectiveness of the proposed method.

/pdf/softcuts-a-soft-edge-smoothness-prior-for-color-image-super-4iajxullvf.pdf

SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution

This paper presents a novel non-local iterative back-projection (NLIBP) algorithm for image enlargement. The iterative back-projection (IBP) technique iteratively reconstructs a high resolution (HR) image from its blurred and downsampled low resolution (LR) counterpart. However, the conventional IBP methods often produce many “jaggy” and “ringing” artifacts because the reconstruction errors are back projected into the reconstructed image isotropically and locally. In natural images, usually there exist many non-local redundancies which can be exploited to improve the image reconstruction quality. Therefore, we propose to incorporate adaptively the non-local information into the IBP process so that the reconstruction errors can be reduced. Experimental results demonstrated that the proposed NLBP can reconstruct faithfully the HR images with sharp edges and texture structures. It outperforms the state-of-the-art methods in both PSNR and visual perception.

/pdf/nonlocal-back-projection-for-adaptive-image-enlargement-1xu9ukmstk.pdf

Nonlocal back-projection for adaptive image enlargement

This paper proposes a two-stage framework for fast image interpolation via random forests (FIRF). The proposed FIRF method gives high accuracy, as well as requires low computation. The underlying idea of this proposed work is to apply random forests to classify the natural image patch space into numerous subspaces and learn a linear regression model for each subspace to map the low-resolution image patch to high-resolution image patch. The FIRF framework consists of two stages. Stage 1 of the framework removes most of the ringing and aliasing artifacts in the initial bicubic interpolated image, while Stage 2 further refines the Stage 1 interpolated image. By varying the number of decision trees in the random forests and the number of stages applied, the proposed FIRF method can realize computationally scalable image interpolation. Extensive experimental results show that the proposed FIRF(3, 2) method achieves more than 0.3 dB improvement in peak signal-to-noise ratio over the state-of-the-art nonlocal autoregressive modeling (NARM) method. Moreover, the proposed FIRF(1, 1) obtains similar or better results as NARM while only takes its 0.3% computational time.

Fast Image Interpolation via Random Forests

In this paper, we propose a new class of techniques to identify periodicities in data. We target the period estimation directly rather than inferring the period from the signal’s spectrum. By doing so, we obtain several advantages over the traditional spectrum estimation techniques such as DFT and MUSIC. Apart from estimating the unknown period of a signal, we search for finer periodic structure within the given signal. For instance, it might be possible that the given periodic signal was actually a sum of signals with much smaller periods. For example, adding signals with periods 3, 7, and 11 can give rise to a period 231 signal. We propose methods to identify these “hidden periods” 3, 7, and 11. We first propose a new family of square matrices called Nested Periodic Matrices (NPMs), having several useful properties in the context of periodicity. These include the DFT, Walsh–Hadamard, and Ramanujan periodicity transform matrices as examples. Based on these matrices, we develop high dimensional dictionary representations for periodic signals. Various optimization problems can be formulated to identify the periods of signals from such representations. We propose an approach based on finding the least $l_{2}$ norm solution to an under-determined linear system. Alternatively, the period identification problem can also be formulated as a sparse vector recovery problem and we show that by a slight modification to the usual $l_{1}$ norm minimization techniques, we can incorporate a number of new and computationally simple dictionaries.

Nested Periodic Matrices and Dictionaries: New Signal Representations for Period Estimation

Image interpolation is a very important topic in digital image processing, especially as consumer digital photography outgrows regular film photography. From enlarging consumer images to creating large artistic prints interpolation is at the heart of it all. This paper presents a fast and efficient interpolation algorithm that produces good visual results while maintaining the computational cost close to polynomial interpolation.

Fast edge directed polynomial interpolation

The detection and estimation of machine vibration multiperiodic signals of unknown periods in white Gaussian noise is investigated. New estimates for the subsignals (signals making up the received signal) and their periods are derived using an orthogonal subspace decomposition approach. The concept of exactly periodic signals is introduced. This in turn simplifies and enhances the understanding of periodic signals.

Orthogonal, exactly periodic subspace decomposition

An edge directed demosaicing algorithm for determining an edge direction from an input color filter array (CFA) sampled image is disclosed. Aspects of the present invention include calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error. Interpolation errors are also calculated for the current missing green pixel in a North-South (NS) direction at known neighboring green pixels, and the NS interpolation errors are averaged to obtain a NS error. An EW or NS direction indicated by a minimum of the EW error and the NS error is then selected as the edge direction.

Fast edge directed demosaicing

A Heart Beat System (HBS) enables both inertial navigation system (INS) technology and other sensors, e.g., laser radar (LIDAR) systems, cameras and the like to be precisely time-coupled. The result of this coupling enables microsecond control by the HBS to each sensor slave component. This synchronization enables the precise time of each sensor measurement to be known which in turn allows highly accurate world coordinates to be computed from each set of received date, e.g., laser pulses range data. Such precision results in 3-dimensional point cloud data with relative accuracies in the 5-10 cm range and absolute positioning accuracies in the sub 40 cm range. This precision further enables multiple sensors to be used simultaneously with outstanding registration which leads to fused 3D point cloud databases representing a more comprehensive urban scene. Finally, 3D data collected from subsequent missions can be used for precision change detection analysis.

Darian Muresan

Papers

Fast edge directed polynomial interpolation

Orthogonal, exactly periodic subspace decomposition

Fast edge directed demosaicing

Process and system for three-dimensional urban modeling

Calculating interpolation errors for interpolation edge detection