As a fundamental and critical task in various visual applications, image matching can identify then correspond the same or similar structure/content from two or more images. Over the past decades, growing amount and diversity of methods have been proposed for image matching, particularly with the development of deep learning techniques over the recent years. However, it may leave several open questions about which method would be a suitable choice for specific applications with respect to different scenarios and task requirements and how to design better image matching methods with superior performance in accuracy, robustness and efficiency. This encourages us to conduct a comprehensive and systematic review and analysis for those classical and latest techniques. Following the feature-based image matching pipeline, we first introduce feature detection, description, and matching techniques from handcrafted methods to trainable ones and provide an analysis of the development of these methods in theory and practice. Secondly, we briefly introduce several typical image matching-based applications for a comprehensive understanding of the significance of image matching. In addition, we also provide a comprehensive and objective comparison of these classical and latest techniques through extensive experiments on representative datasets. Finally, we conclude with the current status of image matching technologies and deliver insightful discussions and prospects for future works. This survey can serve as a reference for (but not limited to) researchers and engineers in image matching and related fields.

/pdf/image-matching-from-handcrafted-to-deep-features-a-survey-27kn6w026w.pdf

Image Matching from Handcrafted to Deep Features: A Survey

A major yet largely unsolved problem in the semantic classification of very high resolution remote sensing images is the design and selection of appropriate features. At a ground sampling distance below half a meter, fine-grained texture details of objects emerge and lead to a large intraclass variability while generally keeping the between-class variability at a low level. Usually, the user makes an educated guess on what features seem to appropriately capture characteristic object class patterns. Here, we propose to avoid manual feature selection and let a boosting classifier choose optimal features from a vast Randomized Quasi-Exhaustive (RQE) set of feature candidates directly during training. This RQE feature set consists of a multitude of very simple features that are computed efficiently via integral images inside a sliding window. This simple but comprehensive feature candidate set enables the boosting classifier to assemble the most discriminative textures at different scale levels to classify a small number of broad urban land-cover classes. We do an extensive evaluation on several data sets and compare performance against multiple feature extraction baselines in different color spaces. In addition, we verify experimentally if we gain any classification accuracy if moving from boosting stumps to trees. Cross-validation minimizes the possible bias caused by specific training/testing setups. It turns out that boosting in combination with the proposed RQE feature set outperforms all baseline features while still remaining computationally efficient. Particularly boosting trees (instead of stumps) captures class patterns so well that results suggest to completely leave feature selection to the classifier.

Features, Color Spaces, and Boosting: New Insights on Semantic Classification of Remote Sensing Images

Automatic image registration is still an actual challenge in several fields. Although several methods for automatic image registration have been proposed in the last few years, it is still far from a broad use in several applications, such as in remote sensing. In this paper, a method for automatic image registration through histogram-based image segmentation (HAIRIS) is proposed. This new approach mainly consists in combining several segmentations of the pair of images to be registered, according to a relaxation parameter on the histogram modes delineation (which itself is a new approach), followed by a consistent characterization of the extracted objects-through the objects area, ratio between the axis of the adjust ellipse, perimeter and fractal dimension-and a robust statistical based procedure for objects matching. The application of the proposed methodology is illustrated to simulated rotation and translation. The first dataset consists in a photograph and a rotated and shifted version of the same photograph, with different levels of added noise. It was also applied to a pair of satellite images with different spectral content and simulated translation, and to real remote sensing examples comprising different viewing angles, different acquisition dates and different sensors. An accuracy below 1° for rotation and at the subpixel level for translation were obtained, for the most part of the considered situations. HAIRIS allows for the registration of pairs of images (multitemporal and multisensor) with differences in rotation and translation, with small differences in the spectral content, leading to a subpixel accuracy.

/pdf/hairis-a-method-for-automatic-image-registration-through-xgq78vo41r.pdf

HAIRIS: A Method for Automatic Image Registration Through Histogram-Based Image Segmentation

Semantic scene interpretation as a collection of meaningful regions in images is a fundamental problem in both photogrammetry and computer vision. Images of man-made scenes exhibit strong contextual dependencies in the form of spatial and hierarchical structures. In this paper, we introduce a hierarchical conditional random field to deal with the problem of image classification by modeling spatial and hierarchical structures. The probability outputs of an efficient randomized decision forest classifier are used as unary potentials. The spatial and hierarchical structures of the regions are integrated into pairwise potentials. The model is built on multi-scale image analysis in order to aggregate evidence from local to global level. Experimental results are provided to demonstrate the performance of the proposed method using images from eTRIMS dataset, where our focus is the object classes building, car, door, pavement, road, sky, vegetation, and window.

A hierarchical conditional random field model for labeling and classifying images of man-made scenes

As a fundamental and critical task in feature-based remote sensing image registration, feature matching refers to establishing reliable point correspondences from two images of the same scene In this article, we propose a simple yet efficient method termed linear adaptive filtering (LAF) for both rigid and nonrigid feature matching of remote sensing images and apply it to the image registration task Our algorithm starts with establishing putative feature correspondences based on local descriptors and then focuses on removing outliers using geometrical consistency priori together with filtering and denoising theory Specifically, we first grid the correspondence space into several nonoverlapping cells and calculate a typical motion vector for each one Subsequently, we remove false matches by checking the consistency between each putative match and the typical motion vector in the corresponding cell, which is achieved by a Gaussian kernel convolution operation By refining the typical motion vector in an iterative manner, we further introduce a progressive strategy based on the coarse-to-fine theory to promote the matching accuracy gradually In addition, an adaptive parameter setting strategy and posterior probability estimation based on the expectation–maximization algorithm enhance the robustness of our method to different data Most importantly, our method is quite efficient where the gridding strategy enables it to achieve linear time complexity Consequently, some sparse point-based tasks may inspire from our method when they are achieved by deep learning techniques Extensive feature matching and image registration experiments on several remote sensing data sets demonstrate the superiority of our approach over the state of the art

Robust Feature Matching for Remote Sensing Image Registration via Linear Adaptive Filtering

This paper presents a new algorithm which can be used to register images of the same or different modalities, e.g., images with multiple channels, such as X-rays, temperature or elevation, or simply images of different spectral bands. In particular, a correlation-based scheme is used, but instead of gray values, it correlates numbers formulated by different combinations of the extracted local Walsh coefficients of the images. Each image patch is expanded in terms of Walsh basis functions. Each Walsh basis function can be thought of as measuring a different aspect of local structure, e.g., horizontal edge, corner, etc. The coefficients of the expansion, therefore, can be thought of as dense local features, estimating at each point the degree of presence of, for example, a horizontal edge, a corner with contrast of a certain type, etc. These coefficients are normalized and used as digits in a chosen number system which allows one to create a unique number for each type of local structure. The choice of the basis of the number system allows one to give different emphasis to different types of local feature (e.g., corners versus edges), and, thus, the method we present forms a unified framework in terms of which several feature matching methods may be interpreted. The algorithm is compared with wavelet and contour based approaches, using simulated and real images. The two images are assumed to differ from each other by a rotation and a translation only.

G. Lazaridis

Papers

Image registration using the Walsh transform