Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Deep learning in environmental remote sensing: Achievements and challenges

The surface urban heat island (SUHI), which represents the difference of land surface temperature (LST) in urban relativity to neighboring non-urban surfaces, is usually measured using satellite LST data. Over the last few decades, advancements of remote sensing along with spatial science have considerably increased the number and quality of SUHI studies that form the major body of the urban heat island (UHI) literature. This paper provides a systematic review of satellite-based SUHI studies, from their origin in 1972 to the present. We find an exponentially increasing trend of SUHI research since 2005, with clear preferences for geographic areas, time of day, seasons, research foci, and platforms/sensors. The most frequently studied region and time period of research are China and summer daytime, respectively. Nearly two-thirds of the studies focus on the SUHI/LST variability at a local scale. The Landsat Thematic Mapper (TM)/Enhanced Thematic Mapper (ETM+)/Thermal Infrared Sensor (TIRS) and Terra/Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) are the two most commonly-used satellite sensors and account for about 78% of the total publications. We systematically reviewed the main satellite/sensors, methods, key findings, and challenges of the SUHI research. Previous studies confirm that the large spatial (local to global scales) and temporal (diurnal, seasonal, and inter-annual) variations of SUHI are contributed by a variety of factors such as impervious surface area, vegetation cover, landscape structure, albedo, and climate. However, applications of SUHI research are largely impeded by a series of data and methodological limitations. Lastly, we propose key potential directions and opportunities for future efforts. Besides improving the quality and quantity of LST data, more attention should be focused on understudied regions/cities, methods to examine SUHI intensity, inter-annual variability and long-term trends of SUHI, scaling issues of SUHI, the relationship between surface and subsurface UHIs, and the integration of remote sensing with field observations and numeric modeling.

https://www.mdpi.com/2072-4292/11/1/48/pdf

Satellite Remote Sensing of Surface Urban Heat Islands: Progress, Challenges, and Perspectives

Image super-resolution

Because of sensor malfunction and poor atmospheric conditions, there is usually a great deal of missing information in optical remote sensing data, which reduces the usage rate and hinders the follow-up interpretation. In the past decades, missing information reconstruction of remote sensing data has become an active research field, and a large number of algorithms have been developed. However, to the best of our knowledge, there has not, to date, been a study that has been aimed at expatiating and summarizing the current situation. This is therefore our motivation in this review. This paper provides an introduction to the principles and theories of missing information reconstruction of remote sensing data. We classify the established and emerging algorithms into four main categories, followed by a comprehensive comparison of them from both experimental and theoretical perspectives. This paper also predicts the promising future research directions.

Missing Information Reconstruction of Remote Sensing Data: A Technical Review

Cloud detection is an important preprocessing step for the precise application of optical satellite imagery. In this paper, we propose a deep learning based cloud detection method named multi-scale convolutional feature fusion (MSCFF) for remote sensing images of different sensors. In the network architecture of MSCFF, the symmetric encoder-decoder module, which provides both local and global context by densifying feature maps with trainable convolutional filter banks, is utilized to extract multi-scale and high-level spatial features. The feature maps of multiple scales are then up-sampled and concatenated, and a novel multi-scale feature fusion module is designed to fuse the features of different scales for the output. The two output feature maps of the network are cloud and cloud shadow maps, which are in turn fed to binary classifiers outside the model to obtain the final cloud and cloud shadow mask. The MSCFF method was validated on hundreds of globally distributed optical satellite images, with spatial resolutions ranging from 0.5 to 50 m, including Landsat-5/7/8, Gaofen-1/2/4, Sentinel-2, Ziyuan-3, CBERS-04, Huanjing-1, and collected high-resolution images exported from Google Earth. The experimental results show that MSCFF achieves a higher accuracy than the traditional rule-based cloud detection methods and the state-of-the-art deep learning models, especially in bright surface covered areas. The effectiveness of MSCFF means that it has great promise for the practical application of cloud detection for multiple types of medium and high-resolution remote sensing images. Our established global high-resolution cloud detection validation dataset has been made available online ( http://sendimage.whu.edu.cn/en/mscff/ ).

Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors

Cloud cover is generally present in remotely sensed images, which limits the potential of the images for ground information extraction. Therefore, removing the clouds and recovering the ground information for the cloud-contaminated images is often necessary in many applications. In this paper, an effective method based on similar pixel replacement is developed to solve this task. A missing pixel is filled using an appropriate similar pixel within the remaining region of the target image. A multitemporal image is used as the guidance to locate the similar pixels. A pixel-offset based spatio-temporal Markov random fields (MRF) global function is built to find the most suitable similar pixel. The proposed method was tested on MODIS and Landsat images and their land surface temperature products, and the experiments verify that the proposed method can achieve highly accurate results and is effective at dealing with the obvious atmospheric and seasonal differences between multitemporal images.

Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model

In the imaging process of optical remote sensing platforms, clouds are an inevitable barrier to the effective observation of sensors. To recover the original information covered by the clouds and the accompanying shadows, a nonnegative matrix factorization (NMF) and error correction method (S-NMF-EC) is proposed in this paper. Firstly, a cloud-free fused reference image is obtained by a reference image and two or more low-resolution images using the spatial and temporal nonlocal filter-based data fusion model (STNLFFM). Secondly, the cloud-free fused reference image is used to remove the cloud cover of the cloud-contaminated image based on NMF. Finally, the cloud removal result is further improved by error correction. It is worth noting that cloud detection is not required by S-NMF-EC, and the cloud-free information of the cloud-contaminated image is maximally retained. Both simulated and real-data experiments were conducted to validate the proposed S-NMF-EC method. Compared with other cloud removal methods, the results demonstrate that S-NMF-EC is visually and quantitatively effective (correlation coefficients ≥ 0.99) for the removal of thick clouds, thin clouds, and shadows.

Cloud removal in remote sensing images using nonnegative matrix factorization and error correction

Filling dead pixels or removing uninteresting objects is often desired in the applications of remotely sensed images. In this paper, an effective image inpainting technology is presented to solve this task, based on multichannel nonlocal total variation. The proposed approach takes advantage of a nonlocal method, which has a superior performance in dealing with textured images and reconstructing large-scale areas. Furthermore, it makes use of the multichannel data of remotely sensed images to achieve spectral coherence for the reconstruction result. To optimize the proposed variation model, a Bregmanized-operator-splitting algorithm is employed. The proposed inpainting algorithm was tested on simulated and real images. The experimental results verify the efficacy of this algorithm.

Qing Cheng

Papers

Missing Information Reconstruction of Remote Sensing Data: A Technical Review

Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors

Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model

Cloud removal in remote sensing images using nonnegative matrix factorization and error correction

Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model