Because of sensor malfunction and poor atmospheric conditions, there is usually a great deal of missing information in optical remote sensing data, which reduces the usage rate and hinders the follow-up interpretation. In the past decades, missing information reconstruction of remote sensing data has become an active research field, and a large number of algorithms have been developed. However, to the best of our knowledge, there has not, to date, been a study that has been aimed at expatiating and summarizing the current situation. This is therefore our motivation in this review. This paper provides an introduction to the principles and theories of missing information reconstruction of remote sensing data. We classify the established and emerging algorithms into four main categories, followed by a comprehensive comparison of them from both experimental and theoretical perspectives. This paper also predicts the promising future research directions.

Missing Information Reconstruction of Remote Sensing Data: A Technical Review

A cloud removal approach based on information cloning is introduced. The approach removes cloud-contaminated portions of a satellite image and then reconstructs the information of missing data utilizing temporal correlation of multitemporal images. The basic idea is to clone information from cloud-free patches to their corresponding cloud-contaminated patches under the assumption that land covers change insignificantly over a short period of time. The patch-based information reconstruction is mathematically formulated as a Poisson equation and solved using a global optimization process. Thus, the proposed approach can potentially yield better results in terms of radiometric accuracy and consistency compared with related approaches. Some experimental analyses on sequences of images acquired by the Landsat-7 Enhanced Thematic Mapper Plus sensor are conducted. The experimental results show that the proposed approach can process large clouds in a heterogeneous landscape, which is difficult for cloud removal approaches. In addition, quantitative and qualitative analyses on simulated data with different cloud contamination conditions are conducted using quality index and visual inspection, respectively, to evaluate the performance of the proposed approach.

Cloud Removal From Multitemporal Satellite Images Using Information Cloning

In the imaging process of optical remote sensing platforms, clouds are an inevitable barrier to the effective observation of sensors. To recover the original information covered by the clouds and the accompanying shadows, a nonnegative matrix factorization (NMF) and error correction method (S-NMF-EC) is proposed in this paper. Firstly, a cloud-free fused reference image is obtained by a reference image and two or more low-resolution images using the spatial and temporal nonlocal filter-based data fusion model (STNLFFM). Secondly, the cloud-free fused reference image is used to remove the cloud cover of the cloud-contaminated image based on NMF. Finally, the cloud removal result is further improved by error correction. It is worth noting that cloud detection is not required by S-NMF-EC, and the cloud-free information of the cloud-contaminated image is maximally retained. Both simulated and real-data experiments were conducted to validate the proposed S-NMF-EC method. Compared with other cloud removal methods, the results demonstrate that S-NMF-EC is visually and quantitatively effective (correlation coefficients ≥ 0.99) for the removal of thick clouds, thin clouds, and shadows.

Cloud removal in remote sensing images using nonnegative matrix factorization and error correction

It is well known that removing cloud-contaminated portions of a remotely sensed image and then filling in the missing data represent an important photo editing cumbersome task. In this paper, an efficient inpainting technique for the reconstruction of areas obscured by clouds or cloud shadows in remotely sensed images is presented. This technique is based on the Bandelet transform and the multiscale geometrical grouping. It consists of two steps. In the first step, the curves of geometric flow of different zones of the image are determined by using the Bandelet transform with multiscale grouping. This step allows an efficient representation of the multiscale geometry of the image's structures. Having well represented this geometry, the information inside the cloud-contaminated zone is synthesized by propagating the geometrical flow curves inside that zone. This step is accomplished by minimizing a functional whose role is to reconstruct the missing or cloud contaminated zone independently of the size and topology of the inpainting domain. The proposed technique is illustrated with some examples on processing aerial images. The obtained results are compared with those obtained by other clouds removal techniques.

A Bandelet-Based Inpainting Technique for Clouds Removal From Remotely Sensed Images

Thick-cloud contamination is a common problem in Landsat images, which limits their utilities in various land surface studies. This letter presents a new method for removing thick clouds based on a modified neighborhood similar pixel interpolator (NSPI) approach that was originally developed for filling gaps due to the Landsat ETM+ Scan Line Corrector (SLC)-off problem. The performance of the proposed method was evaluated with both simulated and real cloudy images and compared with that of a contextual multiple linear prediction (CMLP) method. The results show that the modified NSPI approach can greatly reduce the edge effects by CMLP. The reflectance restored by the modified NSPI approach is more accurate than that by CMLP, especially when the cloud-free auxiliary and cloudy images are acquired from different seasons and have different spectral characteristics.

A Modified Neighborhood Similar Pixel Interpolator Approach for Removing Thick Clouds in Landsat Images

A general method has been proposed recently for the contextual reconstruction of cloud-contaminated areas in multitemporal multispectral images. It is based on the idea of making the prediction process learn from information available in the cloud-free neighborhood of contaminated areas. Though promising, this method does not fully exploit all available information, thus leaving room for further methodological enhancements. This letter presents a postreconstruction methodology for improving the contextual reconstruction process by opportunely capturing spatial and spectral correlations characterizing the considered image. In addition, we propose a solution to a problem that has not yet been addressed in the remote sensing literature, i.e., the generation of an error map beside the reconstructed images to provide end-users with helpful indications about reconstruction reliability. Thorough experiments conducted on a multitemporal sequence of Landsat-7 ETM+ images are reported and discussed.

Contextual Spatiospectral Postreconstruction of Cloud-Contaminated Images

Recently, a novel general method for the contextual reconstruction of cloud-contaminated areas in multitemporal multispectral images has been proposed in the literature. Though promising, this method does not fully exploit all available information, thus leaving room for further methodological enhancements. The aim of this work is to improve the contextual reconstruction process by opportunely capturing spatial and spectral correlations characterizing the considered image. In addition, we propose a novel procedure to generate an error map for providing the end-user with helpful indications about the reconstruction reliability. Thorough experiments conducted on a multitemporal sequence of Landsat-7 ETM+ images are reported and discussed.

S. Benabdelkader

Papers

Contextual Spatiospectral Postreconstruction of Cloud-Contaminated Images

Cloud-contaminated image reconstruction with contextual spatio-spectral information