Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation.

Image Registration With Fourier-Based Image Correlation: A Comprehensive Review of Developments and Applications

Rapid mapping of near-shore and coastal regions has become an indispensable task for the local authority to serve the purpose of coastal management and post-disaster monitoring. Aerial photogrammetry and satellite remote sensing have been utilized to fulfill such a task in the last few decades. Airborne LiDAR can further compensate the drawbacks of these image capturing approaches as a result of the direct geo-referenced 3D point cloud. The recent introduction of multispectral airborne LiDAR, such as the Teledyne Optech Titan, can potentially enhance the capability of water mapping, minimize the involvement of manual intervention and reduce the use of supplementary information or ancillary data. This study demonstrates the use of multispectral airborne LiDAR data for automatic land-water classification under different coastal and inland river environments. Two automatic training data selection methods are proposed. The first method utilizes Gaussian mixture model (GMM) to split preliminarily the land and water region based on the elevation/intensity histogram, and the second method is developed based on the use of scan line intensity-elevation ratio (SLIER). Subsequently, various LiDAR-derived feature sets, particularly based on the multispectral LiDAR intensity, are constructed in order to serve as an input for the log-likelihood classification model. Two optional post-classification enhancements can be implemented to further adjust the misclassified data points. The proposed workflow was evaluated with four Optech Titan datasets collected for different near-shore and river environments that are located nearby Lake Ontario, Ontario, Canada. Our experimental work demonstrated that the multispectral LiDAR intensity data was capable of enhancing the classification capability, where an overall accuracy better than 96% was achieved in most of the cases.

Automatic land-water classification using multispectral airborne LiDAR data for near-shore and river environments

As the resolution of X-ray tomography improves, the limited long-term stability and accuracy of nanoimaging tools does not allow computing artifact-free three-dimensional (3D) reconstructions without an additional step of numerical alignment of the measured projections. However, the common iterative alignment methods are significantly more computationally demanding than a simple tomographic reconstruction of the acquired volume. Here, we address this issue and present an alignment toolkit, which exploits methods with deep-subpixel accuracy combined with a multi-resolution scheme. This leads to robust and accurate alignment with significantly reduced computational and memory requirements. The performance of the presented methods is demonstrated on simulated and measured datasets for tomography and also laminography acquisition geometries. A GPU accelerated implementation of our alignment framework is publicly available.

/pdf/alignment-methods-for-nanotomography-with-deep-subpixel-2c1oary9qc.pdf

Alignment methods for nanotomography with deep subpixel accuracy.

Dynamic process and artificial mechanism of coastline change in the Pearl River Estuary

In order to solve the problem of high computational complexity in block-based methods for copy-move forgery detection, we divide image into texture part and smooth part to deal with them separately. Keypoints are extracted and matched in texture regions. Instead of using all the overlapping blocks, we use nonoverlapping blocks as candidates in smooth regions. Clustering blocks with similar color into a group can be regarded as a preprocessing operation. To avoid mismatching due to misalignment, we update candidate blocks by registration before projecting them into hash space. In this way, we can reduce computational complexity and improve the accuracy of matching at the same time. Experimental results show that the proposed method achieves better performance via comparing with the state-of-the-art copy-move forgery detection algorithms and exhibits robustness against JPEG compression, rotation, and scaling.

/pdf/nonoverlapping-blocks-based-copy-move-forgery-detection-2n2brkw5cy.pdf

Nonoverlapping Blocks Based Copy-Move Forgery Detection

A new, fast and computationally efficient lateral subpixel shift registration algorithm is presented. It is limited to register images that differ by small subpixel shifts otherwise its performance degrades. This algorithm significantly improves the performance of the single-step discrete Fourier transform approach proposed by Guizar-Sicairos and can be applied efficiently on large dimension images. It reduces the dimension of Fourier transform of the cross correlation matrix and reduces the discrete Fourier transform (DFT) matrix multiplications to speed up the registration process. Simulations show that our algorithm reduces computation time and memory requirements without sacricing the accuracy associated with the usual FFT approach accuracy.

High-Speed Image Registration Algorithm with Subpixel Accuracy

There is an increased demand for understanding the accurate position of the shorelines. The automatic extraction of shorelines utilizing the digital elevation models (DEMs) obtained from light detection and ranging (LiDAR), aerial images, and multispectral images has become very promising. In this article, we develop two innovative algorithms that can effectively extract shorelines depending on the available data sources. The first is a multistep morphological technique that works on LiDAR DEM with respect to a tidal datum, whereas the second depends on the availability of training data to extract shorelines from LiDAR DEM fused with aerial images. Unlike similar techniques, the morphological approach detects and eliminates the outliers that result from waves, etc., by means of an anomaly test with neighborhood constraints. Additionally, it eliminates docks, bridges, and fishing piers along the extracted shorelines by means of Hough transform. The second approach extracts the shoreline by means of color space conversion of the aerial images and the support vector machines classifier to segment the fused data into water and land. We perform Monte-Carlo simulations to estimate the confidence interval for the error in shoreline position. Compared with other relevant techniques in literature, the proposed methods offer better accuracy in shoreline extraction.

Shoreline extraction from light detection and ranging digital elevation model data and aerial images

A scanning electron microscope (SEM) is a type of electron microscope that produces images of a sample by scanning it with a focused beam of electrons. The electrons interact with the sample atoms, producing various signals that are collected by detectors. The gathered signals contain information about the sample’s surface topography and composition. The electron beam is generally scanned in a raster scan pattern, and the beam’s position is combined with the detected signal to produce an image. The most common configuration for an SEM produces a single value per pixel, with the results usually rendered as grayscale images. The captured images may be produced with insufficient brightness, anomalous contrast, jagged edges, and poor quality due to low signal-to-noise ratio, grained topography and poor surface details. The segmentation of the SEM images is a tackling problems in the presence of the previously mentioned distortions. In this paper, we are stressing on the clustering of these type of images. In that sense, we evaluate the performance of the well-known unsupervised clustering and classification techniques such as connectivity based clustering (hierarchical clustering), centroid-based clustering, distribution-based clustering and density-based clustering. Furthermore, we propose a new spatial fuzzy clustering technique that works efficiently on this type of images and compare its results against these regular techniques in terms of clustering validation metrics.

A new clustering algorithm for scanning electron microscope images

In this paper we present a technique to detect pedestrian. Histogram of gradients (HOG) and Haar wavelets
with the aid of support vector machines (SVM) and AdaBoost classifiers show good identification performance
on different objects classification including pedestrians. We propose a new shape descriptor derived from the
intra-relationship between gradient orientations in a way similar to the HOG. The proposed descriptor is a two
2-D grid of orientation similarities measured at different offsets. The gradient magnitudes and phases derived
from a sliding window with different scales and sizes are used to construct two 2-D symmetric grids. The first grid
measures the co-occurence of the phases while the other one measures the corresponding percentage of gradient
magnitudes for the measured orientation similarity. Since the resultant matrices will be symmetric, the feature
vector is formed by concatenating the upper diagonal grid coefficients collected in a raster way. Classification is
done using SVM classifier with radial basis kernel. Experimental results show improved performance compared
to the current state-of-art techniques.

An improved algorithm for pedestrian detection

A neighborhood-dependent component feature learning method for regression analysis in single-image superresolution is presented. Given a low-resolution input, the method uses a directional Fourier phase feature component to adaptively learn the regression kernel based on local covariance to estimate the high-resolution image. The unique feature of the proposed method is that it uses image features to learn about the local covariance from geometric similarity between the low-resolution image and its high-resolution counterpart. For each patch in the neighborhood, we estimate four directional variances to adapt the interpolated pixels. This gives us edge information and Fourier phase gives features, which are combined to interpolate using kernel regression. In order to compare quantitatively with other state-of-the-art techniques, root-mean-square error and measure mean-square similarity are computed for the example images, and experimental results show that the proposed algorithm outperforms similar techniques available in the literature, especially at higher resolution scales.

Mohammad Karim

Papers

High-Speed Image Registration Algorithm with Subpixel Accuracy

Shoreline extraction from light detection and ranging digital elevation model data and aerial images

A new clustering algorithm for scanning electron microscope images

An improved algorithm for pedestrian detection

Hybrid edge and feature-based single-image superresolution