Our framework is the synthesis of multispectral images (MS) at higher spatial resolution, which should be as close as possible to those that would have been acquired by the corresponding sensors if they had this high resolution. This synthesis is performed with the help of a high spatial but low spectral resolution image: the panchromatic (Pan) image. The fusion of the Pan and MS images is classically referred as pan-sharpening. A fused product reaches good quality only if the characteristics and differences between input images are taken into account. Dissimilarities existing between these two data sets originate from two causes-different times and different spectral bands of acquisition. Remote sensing physics should be carefully considered while designing the fusion process. Because of the complexity of physics and the large number of unknowns, authors are led to make assumptions to drive their development. Weaknesses and strengths of each reported method are raised and confronted to these physical constraints. The conclusion of this critical survey of literature is that the choice in the assumptions for the development of a method is crucial, with the risk to drastically weaken fusion performance. It is also shown that the Amelioration de la Resolution Spatiale par Injection de Structures concept prevents from introducing spectral distortion into fused products and offers a reliable framework for further developments.

/pdf/synthesis-of-multispectral-images-to-high-spatial-resolution-48x8fvro0n.pdf

Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics

In this paper, we present a new similarity measure for automatic change detection in multitemporal synthetic aperture radar images. This measure is based on the evolution of the local statistics of the image between two dates. The local statistics are estimated by using a cumulant-based series expansion, which approximates probability density functions in the neighborhood of each pixel in the image. The degree of evolution of the local statistics is measured using the Kullback-Leibler divergence. An analytical expression for this detector is given, allowing a simple computation which depends on the four first statistical moments of the pixels inside the analysis window only. The proposed change indicator is compared to the classical mean ratio detector and also to other model-based approaches. Tests on the simulated and real data show that our detector outperforms all the others. The fast computation of the proposed detector allows a multiscale approach in the change detection for operational use. The so-called multiscale change profile (MCP) is introduced to yield change information on a wide range of scales and to better characterize the appropriate scale. Two simple yet useful examples of applications show that the MCP allows the design of change indicators, which provide better results than a monoscale analysis

https://hal.archives-ouvertes.fr/hal-00582539/document

A New Statistical Similarity Measure for Change Detection in Multitemporal SAR Images and Its Extension to Multiscale Change Analysis

Feature matching, which refers to establishing reliable correspondence between two sets of features (particularly point features), is a critical prerequisite in feature-based registration. In this paper, we propose a flexible and general algorithm, which is called locally linear transforming (LLT), for both rigid and nonrigid feature matching of remote sensing images. We start by creating a set of putative correspondences based on the feature similarity and then focus on removing outliers from the putative set and estimating the transformation as well. We formulate this as a maximum-likelihood estimation of a Bayesian model with hidden/latent variables indicating whether matches in the putative set are outliers or inliers. To ensure the well-posedness of the problem, we develop a local geometrical constraint that can preserve local structures among neighboring feature points, and it is also robust to a large number of outliers. The problem is solved by using the expectation–maximization algorithm (EM), and the closed-form solutions of both rigid and nonrigid transformations are derived in the maximization step. In the nonrigid case, we model the transformation between images in a reproducing kernel Hilbert space (RKHS), and a sparse approximation is applied to the transformation that reduces the method computation complexity to linearithmic. Extensive experiments on real remote sensing images demonstrate accurate results of LLT, which outperforms current state-of-the-art methods, particularly in the case of severe outliers (even up to 80%).

Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming

The launch of high-resolution remote sensing satellites like TerraSAR-X, WorldView, and Ikonos has benefited the combined application of synthetic aperture radar (SAR) and optical imageries tremendously. Specifically, in case of natural calamities or disasters, decision makers can now easily use an old archived optical with a newly acquired (postdisaster) SAR image. Although the latest satellites provide the end user already georeferenced and orthorectified data products, still, registration differences exist between different data sets. These differences need to be taken care of through quick automated registration techniques before using the images in different applications. Specifically, mutual information (MI) has been utilized for the intricate SAR-optical registration problem. The computation of this metric involves estimating the joint histogram directly from image intensity values, which might have been generated from different sensor geometries and/or modalities (e.g., SAR and optical). Satellites carrying high-resolution remote sensing sensors like TerraSAR-X and Ikonos generate enormous data volume along with fine Earth observation details that might lead to failure of MI to detect correct registration parameters. In this paper, a solely histogram-based method to achieve automatic registration within TerraSAR-X and Ikonos images acquired specifically over urban areas is analyzed. Taking future sensors into a perspective, techniques like compression and segmentation for handling the enormous data volume and incompatible radiometry generated due to different SAR-optical image acquisition characteristics have been rightfully analyzed. The findings indicate that the proposed method is successful in estimating large global shifts followed by a fine refinement of registration parameters for high-resolution images acquired over dense urban areas.

Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas

When the scale-invariant feature transform (SIFT) is adopted in the registration of remote sensing images, a lot of incorrect matches of keypoints will appear owing to the significant difference in the image intensity between remote sensing images compared to visible images. Scale-orientation joint restriction criteria are proposed to achieve robust feature matching for keypoints in remote sensing images. Moreover, the feature descriptor of each keypoint is also refined to overcome the difference in the gradient intensity and orientation between remote image pairs. Experimental results for multidate, multispectral, and multisensor remote images indicate that the proposed method improves the match performance compared to intensity- and SIFT-based methods in terms of correct-match rate and aligning accuracy.

Robust Scale-Invariant Feature Matching for Remote Sensing Image Registration

Subpixel accuracy image registration is needed for applications such as digital elevation model extraction, change detection, pan-sharpening, and data fusion. In order to achieve this accuracy, the deformation between the two images to be registered is usually modeled by a displacement vector field which can be estimated by measuring rigid local shifts for each pixel in the image. In order to measure subpixel shifts, one uses image resampling. Sampling theory says that, if a continuous signal has been sampled according to the Nyquist criterion, a perfect continuous reconstruction can be obtained from the sampled version. Therefore, a shifted version of a sampled signal can be obtained by interpolation and resampling with a shifted origin. Since only a sampled version of the shifted signal is needed, the reconstruction needs only to be performed for the new positions of the samples, so the whole procedure comes to computing the value of the signal for the new sample positions. In the case of image registration, the similarity between the reference image and the shifted versions of the image to be registered is measured, assuming that the maximum of similarity determines the most likely shift. The image interpolation step is thus performed a high number of times during the similarity optimization procedure. In order to reduce the computation cost, approximate interpolations are performed. Approximate interpolators will introduce errors in the resampled image which may induce errors in the similarity measure and therefore produce errors in the estimated shifts. In this paper, it is shown that the interpolation has a smoothing effect which depends of the applied shift. This means that, in the case of noisy images, the interpolation has a denoising effect, and therefore, it increases the quality of the similarity estimation. Since this blurring is not the same for every shift, the similarity may be low for a null shift (no blurring) and higher for shifts close to half a pixel (strong blurring). This paper presents an analysis of the behavior of the different interpolators and their effects on the similarity measures. This analysis will be done for the two similarity measures: the correlation coefficient and the mutual information. Finally, a strategy to attenuate the interpolation artifacts is proposed

https://hal.archives-ouvertes.fr/hal-00578144/document

V. Muron

Papers

Analysis of Artifacts in Subpixel Remote Sensing Image Registration