Showing papers in "Remote Sensing in 2020"
TL;DR: This work proposes a novel Siamese-based spatial–temporal attention neural network, which improves the F1-score of the baseline model from 83.9 to 87.3 with acceptable computational overhead and introduces a CD dataset LEVIR-CD, which is two orders of magnitude larger than other public datasets of this field.
Abstract: Remote sensing image change detection (CD) is done to identify desired significant changes between bitemporal images. Given two co-registered images taken at different times, the illumination variations and misregistration errors overwhelm the real object changes. Exploring the relationships among different spatial–temporal pixels may improve the performances of CD methods. In our work, we propose a novel Siamese-based spatial–temporal attention neural network. In contrast to previous methods that separately encode the bitemporal images without referring to any useful spatial–temporal dependency, we design a CD self-attention mechanism to model the spatial–temporal relationships. We integrate a new CD self-attention module in the procedure of feature extraction. Our self-attention module calculates the attention weights between any two pixels at different times and positions and uses them to generate more discriminative features. Considering that the object may have different scales, we partition the image into multi-scale subregions and introduce the self-attention in each subregion. In this way, we could capture spatial–temporal dependencies at various scales, thereby generating better representations to accommodate objects of various sizes. We also introduce a CD dataset LEVIR-CD, which is two orders of magnitude larger than other public datasets of this field. LEVIR-CD consists of a large set of bitemporal Google Earth images, with 637 image pairs (1024 × 1024) and over 31 k independently labeled change instances. Our proposed attention module improves the F1-score of our baseline model from 83.9 to 87.3 with acceptable computational overhead. Experimental results on a public remote sensing image CD dataset show our method outperforms several other state-of-the-art methods.
552 citations
TL;DR: A novel approach and the results achieved by a multi-disciplinary network called MapBiomas to reconstruct annual land use and land cover information between 1985 and 2017 for Brazil, based on random forest applied to Landsat archive using Google Earth Engine are described.
Abstract: Brazil has a monitoring system to track annual forest conversion in the Amazon and most recently to monitor the Cerrado biome. However, there is still a gap of annual land use and land cover (LULC) information in all Brazilian biomes in the country. Existing countrywide efforts to map land use and land cover lack regularly updates and high spatial resolution time-series data to better understand historical land use and land cover dynamics, and the subsequent impacts in the country biomes. In this study, we described a novel approach and the results achieved by a multi-disciplinary network called MapBiomas to reconstruct annual land use and land cover information between 1985 and 2017 for Brazil, based on random forest applied to Landsat archive using Google Earth Engine. We mapped five major classes: forest, non-forest natural formation, farming, non-vegetated areas, and water. These classes were broken into two sub-classification levels leading to the most comprehensive and detailed mapping for the country at a 30 m pixel resolution. The average overall accuracy of the land use and land cover time-series, based on a stratified random sample of 75,000 pixel locations, was 89% ranging from 73 to 95% in the biomes. The 33 years of LULC change data series revealed that Brazil lost 71 Mha of natural vegetation, mostly to cattle ranching and agriculture activities. Pasture expanded by 46% from 1985 to 2017, and agriculture by 172%, mostly replacing old pasture fields. We also identified that 86 Mha of the converted native vegetation was undergoing some level of regrowth. Several applications of the MapBiomas dataset are underway, suggesting that reconstructing historical land use and land cover change maps is useful for advancing the science and to guide social, economic and environmental policy decision-making processes in Brazil.
473 citations
TL;DR: The RF algorithm is the best machine-learning LULC classifier, among the six examined algorithms although it is necessary to further test the RF algorithm in different morphoclimatic conditions in the future.
Abstract: Rapid and uncontrolled population growth along with economic and industrial development, especially in developing countries during the late twentieth and early twenty-first centuries, have increased the rate of land-use/land-cover (LULC) change many times. Since quantitative assessment of changes in LULC is one of the most efficient means to understand and manage the land transformation, there is a need to examine the accuracy of different algorithms for LULC mapping in order to identify the best classifier for further applications of earth observations. In this article, six machine-learning algorithms, namely random forest (RF), support vector machine (SVM), artificial neural network (ANN), fuzzy adaptive resonance theory-supervised predictive mapping (Fuzzy ARTMAP), spectral angle mapper (SAM) and Mahalanobis distance (MD) were examined. Accuracy assessment was performed by using Kappa coefficient, receiver operational curve (RoC), index-based validation and root mean square error (RMSE). Results of Kappa coefficient show that all the classifiers have a similar accuracy level with minor variation, but the RF algorithm has the highest accuracy of 0.89 and the MD algorithm (parametric classifier) has the least accuracy of 0.82. In addition, the index-based LULC and visual cross-validation show that the RF algorithm (correlations between RF and normalised differentiation water index, normalised differentiation vegetation index and normalised differentiation built-up index are 0.96, 0.99 and 1, respectively, at 0.05 level of significance) has the highest accuracy level in comparison to the other classifiers adopted. Findings from the literature also proved that ANN and RF algorithms are the best LULC classifiers, although a non-parametric classifier like SAM (Kappa coefficient 0.84; area under curve (AUC) 0.85) has a better and consistent accuracy level than the other machine-learning algorithms. Finally, this review concludes that the RF algorithm is the best machine-learning LULC classifier, among the six examined algorithms although it is necessary to further test the RF algorithm in different morphoclimatic conditions in the future.
383 citations
TL;DR: An overview of the major changes in the production of the land cover maps, that have led to this increased accuracy, including aligning with the Sentinel 2 satellite system in the grid and coordinate system, improving the metric extraction, adding better auxiliary data, and improving the biome delineations, as well as enhancing the expert rules.
Abstract: In May 2019, Collection 2 of the Copernicus Global Land Cover layers was released. Next to a global discrete land cover map at 100 m resolution, a set of cover fraction layers is provided depicting the percentual cover of the main land cover types in a pixel. This additional continuous classification scheme represents areas of heterogeneous land cover better than the standard discrete classification scheme. Overall, 20 layers are provided which allow customization of land cover maps to specific user needs or applications (e.g., forest monitoring, crop monitoring, biodiversity and conservation, climate modeling, etc.). However, Collection 2 was not just a global up-scaling, but also includes major improvements in the map quality, reaching around 80% or more overall accuracy. The processing system went into operational status allowing annual updates on a global scale with an additional implemented training and validation data collection system. In this paper, we provide an overview of the major changes in the production of the land cover maps, that have led to this increased accuracy, including aligning with the Sentinel 2 satellite system in the grid and coordinate system, improving the metric extraction, adding better auxiliary data, improving the biome delineations, as well as enhancing the expert rules. An independent validation exercise confirmed the improved classification results. In addition to the methodological improvements, this paper also provides an overview of where the different resources can be found, including access channels to the product layer as well as the detailed peer-review product documentation.
312 citations
TL;DR: An overview of remote sensing systems, techniques, and vegetation indices along with their recent (2015–2020) applications in Precision agriculture is provided.
Abstract: Agriculture provides for the most basic needs of humankind: food and fiber. The introduction of new farming techniques in the past century (e.g., during the Green Revolution) has helped agriculture keep pace with growing demands for food and other agricultural products. However, further increases in food demand, a growing population, and rising income levels are likely to put additional strain on natural resources. With growing recognition of the negative impacts of agriculture on the environment, new techniques and approaches should be able to meet future food demands while maintaining or reducing the environmental footprint of agriculture. Emerging technologies, such as geospatial technologies, Internet of Things (IoT), Big Data analysis, and artificial intelligence (AI), could be utilized to make informed management decisions aimed to increase crop production. Precision agriculture (PA) entails the application of a suite of such technologies to optimize agricultural inputs to increase agricultural production and reduce input losses. Use of remote sensing technologies for PA has increased rapidly during the past few decades. The unprecedented availability of high resolution (spatial, spectral and temporal) satellite images has promoted the use of remote sensing in many PA applications, including crop monitoring, irrigation management, nutrient application, disease and pest management, and yield prediction. In this paper, we provide an overview of remote sensing systems, techniques, and vegetation indices along with their recent (2015–2020) applications in PA. Remote-sensing-based PA technologies such as variable fertilizer rate application technology in Green Seeker and Crop Circle have already been incorporated in commercial agriculture. Use of unmanned aerial vehicles (UAVs) has increased tremendously during the last decade due to their cost-effectiveness and flexibility in obtaining the high-resolution (cm-scale) images needed for PA applications. At the same time, the availability of a large amount of satellite data has prompted researchers to explore advanced data storage and processing techniques such as cloud computing and machine learning. Given the complexity of image processing and the amount of technical knowledge and expertise needed, it is critical to explore and develop a simple yet reliable workflow for the real-time application of remote sensing in PA. Development of accurate yet easy to use, user-friendly systems is likely to result in broader adoption of remote sensing technologies in commercial and non-commercial PA applications.
291 citations
TL;DR: This review is intended to assist agricultural researchers and practitioners to better understand the strengths and limitations of hyperspectral imaging to agricultural applications and promote the adoption of this valuable technology.
Abstract: Remote sensing is a useful tool for monitoring spatio-temporal variations of crop morphological and physiological status and supporting practices in precision farming. In comparison with multispectral imaging, hyperspectral imaging is a more advanced technique that is capable of acquiring a detailed spectral response of target features. Due to limited accessibility outside of the scientific community, hyperspectral images have not been widely used in precision agriculture. In recent years, different mini-sized and low-cost airborne hyperspectral sensors (e.g., Headwall Micro-Hyperspec, Cubert UHD 185-Firefly) have been developed, and advanced spaceborne hyperspectral sensors have also been or will be launched (e.g., PRISMA, DESIS, EnMAP, HyspIRI). Hyperspectral imaging is becoming more widely available to agricultural applications. Meanwhile, the acquisition, processing, and analysis of hyperspectral imagery still remain a challenging research topic (e.g., large data volume, high data dimensionality, and complex information analysis). It is hence beneficial to conduct a thorough and in-depth review of the hyperspectral imaging technology (e.g., different platforms and sensors), methods available for processing and analyzing hyperspectral information, and recent advances of hyperspectral imaging in agricultural applications. Publications over the past 30 years in hyperspectral imaging technology and applications in agriculture were thus reviewed. The imaging platforms and sensors, together with analytic methods used in the literature, were discussed. Performances of hyperspectral imaging for different applications (e.g., crop biophysical and biochemical properties’ mapping, soil characteristics, and crop classification) were also evaluated. This review is intended to assist agricultural researchers and practitioners to better understand the strengths and limitations of hyperspectral imaging to agricultural applications and promote the adoption of this valuable technology. Recommendations for future hyperspectral imaging research for precision agriculture are also presented.
282 citations
TL;DR: This review focuses on the state-of-the-art methods, applications, and challenges of AI for change detection, and the commonly used networks in AI forchange detection are described.
Abstract: Change detection based on remote sensing (RS) data is an important method of detecting changes on the Earth’s surface and has a wide range of applications in urban planning, environmental monitoring, agriculture investigation, disaster assessment, and map revision. In recent years, integrated artificial intelligence (AI) technology has become a research focus in developing new change detection methods. Although some researchers claim that AI-based change detection approaches outperform traditional change detection approaches, it is not immediately obvious how and to what extent AI can improve the performance of change detection. This review focuses on the state-of-the-art methods, applications, and challenges of AI for change detection. Specifically, the implementation process of AI-based change detection is first introduced. Then, the data from different sensors used for change detection, including optical RS data, synthetic aperture radar (SAR) data, street view images, and combined heterogeneous data, are presented, and the available open datasets are also listed. The general frameworks of AI-based change detection methods are reviewed and analyzed systematically, and the unsupervised schemes used in AI-based change detection are further analyzed. Subsequently, the commonly used networks in AI for change detection are described. From a practical point of view, the application domains of AI-based change detection methods are classified based on their applicability. Finally, the major challenges and prospects of AI for change detection are discussed and delineated, including (a) heterogeneous big data processing, (b) unsupervised AI, and (c) the reliability of AI. This review will be beneficial for researchers in understanding this field.
264 citations
Helmholtz Centre for Environmental Research - UFZ1, Humboldt University of Berlin2, University of Zurich3, University of Jena4, Flinders University5, German Aerospace Center6, Martin Luther University of Halle-Wittenberg7, Bavarian Forest National Park8, University of Freiburg9, Forschungszentrum Jülich10, Dresden University of Technology11, University of Nottingham12, Environment Agency13, Leipzig University14
TL;DR: The requirements for future geomorphology monitoring are focused on the implementation and linking of in-situ, close-range, air- and spaceborne RS technologies, geomorphic traits, and data science approaches as crucial components for a better understanding of the geomorphic impacts on complex ecosystems.
Abstract: The status, changes, and disturbances in geomorphological regimes can be regarded as controlling and regulating factors for biodiversity. Therefore, monitoring geomorphology at local, regional, and global scales is not only necessary to conserve geodiversity, but also to preserve biodiversity, as well as to improve biodiversity conservation and ecosystem management. Numerous remote sensing (RS) approaches and platforms have been used in the past to enable a cost-effective, increasingly freely available, comprehensive, repetitive, standardized, and objective monitoring of geomorphological characteristics and their traits. This contribution provides a state-of-the-art review for the RS-based monitoring of these characteristics and traits, by presenting examples of aeolian, fluvial, and coastal landforms. Different examples for monitoring geomorphology as a crucial discipline of geodiversity using RS are provided, discussing the implementation of RS technologies such as LiDAR, RADAR, as well as multi-spectral and hyperspectral sensor technologies. Furthermore, data products and RS technologies that could be used in the future for monitoring geomorphology are introduced. The use of spectral traits (ST) and spectral trait variation (STV) approaches with RS enable the status, changes, and disturbances of geomorphic diversity to be monitored. We focus on the requirements for future geomorphology monitoring specifically aimed at overcoming some key limitations of ecological modeling, namely: the implementation and linking of in-situ, close-range, air- and spaceborne RS technologies, geomorphic traits, and data science approaches as crucial components for a better understanding of the geomorphic impacts on complex ecosystems. This paper aims to impart multidimensional geomorphic information obtained by RS for improved utilization in biodiversity monitoring.
254 citations
TL;DR: Sentinel-2 has a positive impact on land cover/use monitoring, specifically in monitoring of crop, forests, urban areas, and water resources and the literature shows that the use of Sentinel-2 data produces high accuracies with machine-learning classifiers such as support vector machine (SVM) and Random forest (RF).
Abstract: The advancement in satellite remote sensing technology has revolutionised the approaches to monitoring the Earth’s surface. The development of the Copernicus Programme by the European Space Agency (ESA) and the European Union (EU) has contributed to the effective monitoring of the Earth’s surface by producing the Sentinel-2 multispectral products. Sentinel-2 satellites are the second constellation of the ESA Sentinel missions and carry onboard multispectral scanners. The primary objective of the Sentinel-2 mission is to provide high resolution satellite data for land cover/use monitoring, climate change and disaster monitoring, as well as complementing the other satellite missions such as Landsat. Since the launch of Sentinel-2 multispectral instruments in 2015, there have been many studies on land cover/use classification which use Sentinel-2 images. However, no review studies have been dedicated to the application of ESA Sentinel-2 land cover/use monitoring. Therefore, this review focuses on two aspects: (1) assessing the contribution of ESA Sentinel-2 to land cover/use classification, and (2) exploring the performance of Sentinel-2 data in different applications (e.g., forest, urban area and natural hazard monitoring). The present review shows that Sentinel-2 has a positive impact on land cover/use monitoring, specifically in monitoring of crop, forests, urban areas, and water resources. The contemporary high adoption and application of Sentinel-2 can be attributed to the higher spatial resolution (10 m) than other medium spatial resolution images, the high temporal resolution of 5 days and the availability of the red-edge bands with multiple applications. The ability to integrate Sentinel-2 data with other remotely sensed data, as part of data analysis, improves the overall accuracy (OA) when working with Sentinel-2 images. The free access policy drives the increasing use of Sentinel-2 data, especially in developing countries where financial resources for the acquisition of remotely sensed data are limited. The literature also shows that the use of Sentinel-2 data produces high accuracies (>80%) with machine-learning classifiers such as support vector machine (SVM) and Random forest (RF). However, other classifiers such as maximum likelihood analysis are also common. Although Sentinel-2 offers many opportunities for land cover/use classification, there are challenges which include mismatching with Landsat OLI-8 data, a lack of thermal bands, and the differences in spatial resolution among the bands of Sentinel-2. Sentinel-2 data show promise and have the potential to contribute significantly towards land cover/use monitoring.
234 citations
TL;DR: This work provides a code repository that allows computing LSTs from Landsat 4, 5, 7, and 8 within GEE, an online platform created to allow remote sensing users to easily perform big data analyses without increasing the demand for local computing resources.
Abstract: Land Surface Temperature (LST) is increasingly important for various studies assessing land surface conditions, e.g., studies of urban climate, evapotranspiration, and vegetation stress. The Landsat series of satellites have the potential to provide LST estimates at a high spatial resolution, which is particularly appropriate for local or small-scale studies. Numerous studies have proposed LST retrieval algorithms for the Landsat series, and some datasets are available online. However, those datasets generally require the users to be able to handle large volumes of data. Google Earth Engine (GEE) is an online platform created to allow remote sensing users to easily perform big data analyses without increasing the demand for local computing resources. However, high spatial resolution LST datasets are currently not available in GEE. Here we provide a code repository that allows computing LSTs from Landsat 4, 5, 7, and 8 within GEE. The code may be used freely by users for computing Landsat LST as part of any analysis within GEE.
221 citations
TL;DR: All LST retrieval methods provided satisfying results, with MWA having a slightly better accuracy with a Root Mean Square Error (RMSE) equals to 2.39 K and a lower bias error, while RTE and SCA presented similar results regardless of the season, while MWA differed from RTEand SCA for all seasons, especially in summer.
Abstract: Land Surface Temperature (LST) is an important parameter for many scientific disciplines since it affects the interaction between the land and the atmosphere. Many LST retrieval algorithms based on remotely sensed images have been introduced so far, where the Land Surface Emissivity (LSE) is one of the main factors affecting the accuracy of the LST estimation. The aim of this study is to evaluate the performance of LST retrieval methods using different LSE models and data of old and current Landsat missions. Mono Window Algorithm (MWA), Radiative Transfer Equation (RTE) method, Single Channel Algorithm (SCA) and Split Window Algorithm (SWA) were assessed as LST retrieval methods processing data of Landsat missions (Landsat 5, 7 and 8) over rural pixels. Considering the LSE models introduced in the literature, different Normalized Difference Vegetation Index (NDVI)-based LSE models were investigated in this study. Specifically, three LSE models were considered for the LST estimation from Landsat 5 Thematic Mapper (TM) and seven Enhanced Thematic Mapper Plus (ETM+), and six for Landsat 8. For the accurate evaluation of the estimated LST, in-situ LST data were obtained from the Surface Radiation Budget Network (SURFRAD) stations. In total, forty-five daytime Landsat images; fifteen images for each Landsat mission, acquired in the Spring-Summer-Autumn period in the mid-latitude region in the Northern Hemisphere were acquired over five SURFRAD rural sites. After determining the best LSE model for the study case, firstly, the LST retrieval accuracy was evaluated considering the sensor type: when using Landsat 5 TM, 7 ETM+, and 8 Operational Land Imager (OLI), and Thermal Infrared Sensor (TIRS) data separately, RTE, MWA, and MWA presented the best results, respectively. Then, the performance was evaluated independently of the sensor types. In this case, all LST methods provided satisfying results, with MWA having a slightly better accuracy with a Root Mean Square Error (RMSE) equals to 2.39 K and a lower bias error. In addition, the spatio-temporal and seasonal analyses indicated that RTE and SCA presented similar results regardless of the season, while MWA differed from RTE and SCA for all seasons, especially in summer. To efficiently perform this work, an ArcGIS toolbox, including all the methods and models analyzed here, was implemented and provided as a user facility for the LST retrieval from Landsat data.
TL;DR: An overview of the evolution of DL with a focus on image segmentation and object detection in convolutional neural networks (CNN) starts in 2012, when a CNN set new standards in image recognition, and lasts until late 2019.
Abstract: Deep learning (DL) has great influence on large parts of science and increasingly established itself as an adaptive method for new challenges in the field of Earth observation (EO). Nevertheless, the entry barriers for EO researchers are high due to the dense and rapidly developing field mainly driven by advances in computer vision (CV). To lower the barriers for researchers in EO, this review gives an overview of the evolution of DL with a focus on image segmentation and object detection in convolutional neural networks (CNN). The survey starts in 2012, when a CNN set new standards in image recognition, and lasts until late 2019. Thereby, we highlight the connections between the most important CNN architectures and cornerstones coming from CV in order to alleviate the evaluation of modern DL models. Furthermore, we briefly outline the evolution of the most popular DL frameworks and provide a summary of datasets in EO. By discussing well performing DL architectures on these datasets as well as reflecting on advances made in CV and their impact on future research in EO, we narrow the gap between the reviewed, theoretical concepts from CV and practical application in EO.
TL;DR: The results show that the Bagging–Cubic–KNN ensemble model outperformed other ensemble models and should be more widely applied for the sustainable management of flood-prone areas.
Abstract: Mapping flood-prone areas is a key activity in flood disaster management. In this paper, we propose a new flood susceptibility mapping technique. We employ new ensemble models based on bagging as a meta-classifier and K-Nearest Neighbor (KNN) coarse, cosine, cubic, and weighted base classifiers to spatially forecast flooding in the Haraz watershed in northern Iran. We identified flood-prone areas using data from Sentinel-1 sensor. We then selected 10 conditioning factors to spatially predict floods and assess their predictive power using the Relief Attribute Evaluation (RFAE) method. Model validation was performed using two statistical error indices and the area under the curve (AUC). Our results show that the Bagging–Cubic–KNN ensemble model outperformed other ensemble models. It decreased the overfitting and variance problems in the training dataset and enhanced the prediction accuracy of the Cubic–KNN model (AUC=0.660). We therefore recommend that the Bagging–Cubic–KNN model be more widely applied for the sustainable management of flood-prone areas.
TL;DR: The results indicate that temporal aggregation (e.g., median) is a promising method, which not only significantly reduces data volume (resulting in an easier and faster analysis) but also produces an equally high accuracy as time series data.
Abstract: Land cover information plays a vital role in many aspects of life, from scientific and economic to political. Accurate information about land cover affects the accuracy of all subsequent applications, therefore accurate and timely land cover information is in high demand. In land cover classification studies over the past decade, higher accuracies were produced when using time series satellite images than when using single date images. Recently, the availability of the Google Earth Engine (GEE), a cloud-based computing platform, has gained the attention of remote sensing based applications where temporal aggregation methods derived from time series images are widely applied (i.e., the use the metrics such as mean or median), instead of time series images. In GEE, many studies simply select as many images as possible to fill gaps without concerning how different year/season images might affect the classification accuracy. This study aims to analyze the effect of different composition methods, as well as different input images, on the classification results. We use Landsat 8 surface reflectance (L8sr) data with eight different combination strategies to produce and evaluate land cover maps for a study area in Mongolia. We implemented the experiment on the GEE platform with a widely applied algorithm, the Random Forest (RF) classifier. Our results show that all the eight datasets produced moderately to highly accurate land cover maps, with overall accuracy over 84.31%. Among the eight datasets, two time series datasets of summer scenes (images from 1 June to 30 September) produced the highest accuracy (89.80% and 89.70%), followed by the median composite of the same input images (88.74%). The difference between these three classifications was not significant based on the McNemar test (p > 0.05). However, significant difference (p < 0.05) was observed for all other pairs involving one of these three datasets. The results indicate that temporal aggregation (e.g., median) is a promising method, which not only significantly reduces data volume (resulting in an easier and faster analysis) but also produces an equally high accuracy as time series data. The spatial consistency among the classification results was relatively low compared to the general high accuracy, showing that the selection of the dataset used in any classification on GEE is an important and crucial step, because the input images for the composition play an essential role in land cover classification, particularly with snowy, cloudy and expansive areas like Mongolia.
TL;DR: A survey of the recent state-of-the-art deep learning techniques that mainly focused on point cloud data, introducing the popular 3D point cloud benchmark datasets, and discussing the application of deep learning in popular3D vision tasks including classification, segmentation and detection.
Abstract: A point cloud is a set of points defined in a 3D metric space. Point clouds have become one of the most significant data formats for 3D representation and are gaining increased popularity as a result of the increased availability of acquisition devices, as well as seeing increased application in areas such as robotics, autonomous driving, and augmented and virtual reality. Deep learning is now the most powerful tool for data processing in computer vision and is becoming the most preferred technique for tasks such as classification, segmentation, and detection. While deep learning techniques are mainly applied to data with a structured grid, the point cloud, on the other hand, is unstructured. The unstructuredness of point clouds makes the use of deep learning for its direct processing very challenging. This paper contains a review of the recent state-of-the-art deep learning techniques, mainly focusing on raw point cloud data. The initial work on deep learning directly with raw point cloud data did not model local regions; therefore, subsequent approaches model local regions through sampling and grouping. More recently, several approaches have been proposed that not only model the local regions but also explore the correlation between points in the local regions. From the survey, we conclude that approaches that model local regions and take into account the correlation between points in the local regions perform better. Contrary to existing reviews, this paper provides a general structure for learning with raw point clouds, and various methods were compared based on the general structure. This work also introduces the popular 3D point cloud benchmark datasets and discusses the application of deep learning in popular 3D vision tasks, including classification, segmentation, and detection.
TL;DR: LiCSBAS is developed, an open-source SAR interferometry (InSAR) time series analysis package that integrates with the automated Sentinel-1 InSAR processor (LiCSAR) and facilitates greater exploitation of globally available and abundant SAR datasets and enhance their applications for scientific research and societal benefit.
Abstract: For the past five years, the 2-satellite Sentinel-1 constellation has provided abundant and useful Synthetic Aperture Radar (SAR) data, which have the potential to reveal global ground surface deformation at high spatial and temporal resolutions. However, for most users, fully exploiting the large amount of associated data is challenging, especially over wide areas. To help address this challenge, we have developed LiCSBAS, an open-source SAR interferometry (InSAR) time series analysis package that integrates with the automated Sentinel-1 InSAR processor (LiCSAR). LiCSBAS utilizes freely available LiCSAR products, and users can save processing time and disk space while obtaining the results of InSAR time series analysis. In the LiCSBAS processing scheme, interferograms with many unwrapping errors are automatically identified by loop closure and removed. Reliable time series and velocities are derived with the aid of masking using several noise indices. The easy implementation of atmospheric corrections to reduce noise is achieved with the Generic Atmospheric Correction Online Service for InSAR (GACOS). Using case studies in southern Tohoku and the Echigo Plain, Japan, we demonstrate that LiCSBAS applied to LiCSAR products can detect both large-scale (>100 km) and localized (~km) relative displacements with an accuracy of <1 cm/epoch and ~2 mm/yr. We detect displacements with different temporal characteristics, including linear, periodic, and episodic, in Niigata, Ojiya, and Sanjo City, respectively. LiCSBAS and LiCSAR products facilitate greater exploitation of globally available and abundant SAR datasets and enhance their applications for scientific research and societal benefit.
TL;DR: Results indicate that U-Nets trained on weak labels outperform baseline methods with as few as 100 labels, and Neural networks can combine superior classification performance with efficient label usage, and allow pixel-level labels to be obtained from image labels.
Abstract: Accurate automated segmentation of remote sensing data could benefit applications from land cover mapping and agricultural monitoring to urban development surveyal and disaster damage assessment. While convolutional neural networks (CNNs) achieve state-of-the-art accuracy when segmenting natural images with huge labeled datasets, their successful translation to remote sensing tasks has been limited by low quantities of ground truth labels, especially fully segmented ones, in the remote sensing domain. In this work, we perform cropland segmentation using two types of labels commonly found in remote sensing datasets that can be considered sources of “weak supervision”: (1) labels comprised of single geotagged points and (2) image-level labels. We demonstrate that (1) a U-Net trained on a single labeled pixel per image and (2) a U-Net image classifier transferred to segmentation can outperform pixel-level algorithms such as logistic regression, support vector machine, and random forest. While the high performance of neural networks is well-established for large datasets, our experiments indicate that U-Nets trained on weak labels outperform baseline methods with as few as 100 labels. Neural networks, therefore, can combine superior classification performance with efficient label usage, and allow pixel-level labels to be obtained from image labels.
TL;DR: Wang et al. as mentioned in this paper proposed a double-branch dual-attention mechanism network (DBDA) for hyperspectral image classification, where two branches are designed to capture spectral and spatial features contained in HSI.
Abstract: In recent years, researchers have paid increasing attention on hyperspectral image (HSI) classification using deep learning methods. To improve the accuracy and reduce the training samples, we propose a double-branch dual-attention mechanism network (DBDA) for HSI classification in this paper. Two branches are designed in DBDA to capture plenty of spectral and spatial features contained in HSI. Furthermore, a channel attention block and a spatial attention block are applied to these two branches respectively, which enables DBDA to refine and optimize the extracted feature maps. A series of experiments on four hyperspectral datasets show that the proposed framework has superior performance to the state-of-the-art algorithm, especially when the training samples are signally lacking.
TL;DR: This paper reviews the use of deep learning in land use and land cover classification based on multispectral and hyperspectral images and introduces the available data sources and datasets used by literature studies to provide the readers with a framework to interpret the-state-of-the-art ofdeep learning in this context.
Abstract: Lately, with deep learning outpacing the other machine learning techniques in classifying images, we have witnessed a growing interest of the remote sensing community in employing these techniques for the land use and land cover classification based on multispectral and hyperspectral images; the number of related publications almost doubling each year since 2015 is an attest to that. The advances in remote sensing technologies, hence the fast-growing volume of timely data available at the global scale, offer new opportunities for a variety of applications. Deep learning being significantly successful in dealing with Big Data, seems to be a great candidate for exploiting the potentials of such complex massive data. However, there are some challenges related to the ground-truth, resolution, and the nature of data that strongly impact the performance of classification. In this paper, we review the use of deep learning in land use and land cover classification based on multispectral and hyperspectral images and we introduce the available data sources and datasets used by literature studies; we provide the readers with a framework to interpret the-state-of-the-art of deep learning in this context and offer a platform to approach methodologies, data, and challenges of the field.
TL;DR: The proposed end-to-end network, called the pyramid feature-based attention-guided Siamese network (PGA-SiamNet), is trained to capture possible changes using a convolutional neural network in a pyramid and effectively improved the long-range dependencies of the features by utilizing various attention mechanisms.
Abstract: In recent years, building change detection has made remarkable progress through using deep learning. The core problems of this technique are the need for additional data (e.g., Lidar or semantic labels) and the difficulty in extracting sufficient features. In this paper, we propose an end-to-end network, called the pyramid feature-based attention-guided Siamese network (PGA-SiamNet), to solve these problems. The network is trained to capture possible changes using a convolutional neural network in a pyramid. It emphasizes the importance of correlation among the input feature pairs by introducing a global co-attention mechanism. Furthermore, we effectively improved the long-range dependencies of the features by utilizing various attention mechanisms and then aggregating the features of the low-level and co-attention level; this helps to obtain richer object information. Finally, we evaluated our method with a publicly available dataset (WHU) building dataset and a new dataset (EV-CD) building dataset. The experiments demonstrate that the proposed method is effective for building change detection and outperforms the existing state-of-the-art methods on high-resolution remote sensing orthoimages in various metrics.
TL;DR: A modified U-Net model is introduced for semantic segmentation of landslides at a regional scale from EO data using ResNet34 blocks for feature extraction and is compared with conventional pixel-based and object-based methods.
Abstract: Mapping landslides using automated methods is a challenging task, which is still largely done using human efforts. Today, the availability of high-resolution EO data products is increasing exponentially, and one of the targets is to exploit this data source for the rapid generation of landslide inventory. Conventional methods like pixel-based and object-based machine learning strategies have been studied extensively in the last decade. In addition, recent advances in CNN (convolutional neural network), a type of deep-learning method, has been widely successful in extracting information from images and have outperformed other conventional learning methods. In the last few years, there have been only a few attempts to adapt CNN for landslide mapping. In this study, we introduce a modified U-Net model for semantic segmentation of landslides at a regional scale from EO data using ResNet34 blocks for feature extraction. We also compare this with conventional pixel-based and object-based methods. The experiment was done in Douglas County, a study area selected in the south of Portland in Oregon, USA, and landslide inventory extracted from SLIDO (Statewide Landslide Information Database of Oregon) was considered as the ground truth. Landslide mapping is an imbalanced learning problem with very limited availability of training data. Our network was trained on a combination of focal Tversky loss and cross-entropy loss functions using augmented image tiles sampled from a selected training area. The deep-learning method was observed to have a better performance than the conventional methods with an MCC (Matthews correlation coefficient) score of 0.495 and a POD (probability of detection) rate of 0.72 .
TL;DR: A Large-Scale SAR Ship detection dataset from Sentinel-1 and a Pure Background Hybrid Training mechanism (PBHT-mechanism) to suppress false alarms of land in large-scale SAR images to inspire related scholars to make extensive research into SAR ship detection methods with engineering application value.
Abstract: Ship detection in synthetic aperture radar (SAR) images is becoming a research hotspot. In recent years, as the rise of artificial intelligence, deep learning has almost dominated SAR ship detection community for its higher accuracy, faster speed, less human intervention, etc. However, today, there is still a lack of a reliable deep learning SAR ship detection dataset that can meet the practical migration application of ship detection in large-scene space-borne SAR images. Thus, to solve this problem, this paper releases a Large-Scale SAR Ship Detection Dataset-v1.0 (LS-SSDD-v1.0) from Sentinel-1, for small ship detection under large-scale backgrounds. LS-SSDD-v1.0 contains 15 large-scale SAR images whose ground truths are correctly labeled by SAR experts by drawing support from the Automatic Identification System (AIS) and Google Earth. To facilitate network training, the large-scale images are directly cut into 9000 sub-images without bells and whistles, providing convenience for subsequent detection result presentation in large-scale SAR images. Notably, LS-SSDD-v1.0 has five advantages: (1) large-scale backgrounds, (2) small ship detection, (3) abundant pure backgrounds, (4) fully automatic detection flow, and (5) numerous and standardized research baselines. Last but not least, combined with the advantage of abundant pure backgrounds, we also propose a Pure Background Hybrid Training mechanism (PBHT-mechanism) to suppress false alarms of land in large-scale SAR images. Experimental results of ablation study can verify the effectiveness of the PBHT-mechanism. LS-SSDD-v1.0 can inspire related scholars to make extensive research into SAR ship detection methods with engineering application value, which is conducive to the progress of SAR intelligent interpretation technology.
TL;DR: It can be concluded that selecting recorded landslides as prior knowledge to train and test the LSP models is the key reason for the higher prediction accuracy of the SML models, while the lack of a priori knowledge and target guidance is an important reasons for the low LSP accuracy ofThe USML models.
Abstract: Landslide susceptibility prediction (LSP) has been widely and effectively implemented by machine learning (ML) models based on remote sensing (RS) images and Geographic Information System (GIS). However, comparisons of the applications of ML models for LSP from the perspectives of supervised machine learning (SML) and unsupervised machine learning (USML) have not been explored. Hence, this study aims to compare the LSP performance of these SML and USML models, thus further to explore the advantages and disadvantages of these ML models and to realize a more accurate and reliable LSP result. Two representative SML models (support vector machine (SVM) and CHi-squared Automatic Interaction Detection (CHAID)) and two representative USML models (K-means and Kohonen models) are respectively used to scientifically predict the landslide susceptibility indexes, and then these prediction results are discussed. Ningdu County with 446 recorded landslides obtained through field investigations is introduced as case study. A total of 12 conditioning factors are obtained through procession of Landsat TM 8 images and high-resolution aerial images, topographical and hydrological spatial analysis of Digital Elevation Modeling in GIS software, and government reports. The area value under the curve of receiver operating features (AUC) is applied for evaluating the prediction accuracy of SML models, and the frequency ratio (FR) accuracy is then introduced to compare the remarkable prediction performance differences between SML and USML models. Overall, the receiver operation curve (ROC) results show that the AUC of the SVM is 0.892 and is slightly greater than the AUC of the CHAID model (0.872). The FR accuracy results show that the SVM model has the highest accuracy for LSP (77.80%), followed by the CHAID model (74.50%), the Kohonen model (72.8%) and the K-means model (69.7%), which indicates that the SML models can reach considerably better prediction capability than the USML models. It can be concluded that selecting recorded landslides as prior knowledge to train and test the LSP models is the key reason for the higher prediction accuracy of the SML models, while the lack of a priori knowledge and target guidance is an important reason for the low LSP accuracy of the USML models. Nevertheless, the USML models can also be used to implement LSP due to their advantages of efficient modeling processes, dimensionality reduction and strong scalability.
TL;DR: A systematic review of deep learning techniques applied to common remote sensing benchmarks for road extraction, indicating that the largest reported performance record is related to the deconvolutional nets applied to remote sensing images, and the F1 score metric of the generative adversarial network model, DenseNet method, and FCN-32 applied to UAV and Google Earth images are high.
Abstract: One of the most challenging research subjects in remote sensing is feature extraction, such as road features, from remote sensing images. Such an extraction influences multiple scenes, including map updating, traffic management, emergency tasks, road monitoring, and others. Therefore, a systematic review of deep learning techniques applied to common remote sensing benchmarks for road extraction is conducted in this study. The research is conducted based on four main types of deep learning methods, namely, the GANs model, deconvolutional networks, FCNs, and patch-based CNNs models. We also compare these various deep learning models applied to remote sensing datasets to show which method performs well in extracting road parts from high-resolution remote sensing images. Moreover, we describe future research directions and research gaps. Results indicate that the largest reported performance record is related to the deconvolutional nets applied to remote sensing images, and the F1 score metric of the generative adversarial network model, DenseNet method, and FCN-32 applied to UAV and Google Earth images are high: 96.08%, 95.72%, and 94.59%, respectively.
TL;DR: Version 3 of the global DEM (GDEM) based on stereo correlation of 1.8 million ASTER scenes was released and a second unique dataset was produced and released, which identifies the presence of permanent water bodies, and marks them as ocean, lake, or river.
Abstract: The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) is a 14-channel imaging instrument operating on NASA’s Terra satellite since 1999. ASTER’s visible–near infrared (VNIR) instrument, with three bands and a 15 m Instantaneous field of view (IFOV), is accompanied by an additional band using a second, backward-looking telescope. Collecting along-track stereo pairs, the geometry produces a base-to-height ratio of 0.6. In August 2019, the ASTER Science Team released Version 3 of the global DEM (GDEM) based on stereo correlation of 1.8 million ASTER scenes. The DEM has 1 arc-second latitude and longitude postings (~30 m) and employed cloud masking to avoid cloud-contaminated pixels. Custom software was developed to reduce or eliminate artifacts found in earlier GDEM versions, and to fill holes due to the masking. Each 1×1 degree GDEM tile was manually inspected to verify the completeness of the anomaly removal, which was generally excellent except across some large ice sheets. The GDEM covers all of the Earth’s land surface from 83 degrees north to 83 degrees south latitude. This is a unique, global high spatial resolution digital elevation dataset available to all users at no cost. In addition, a second unique dataset was produced and released. The raster-based ASTER Global Water Body Dataset (ASTWBD) identifies the presence of permanent water bodies, and marks them as ocean, lake, or river. An accompanying DEM file indicates the elevation for each water pixel. To date, over 100 million 1×1 degree GDEM tiles have been distributed.
TL;DR: Through this synthesis study, the many interdependencies of each step in the collection and processing chain are identified, and approaches to formalize and ensure a successful workflow and product development are outlined.
Abstract: With the increasing role that unmanned aerial systems (UAS) are playing in data collection for environmental studies, two key challenges relate to harmonizing and providing standardized guidance for data collection, and also establishing protocols that are applicable across a broad range of environments and conditions. In this context, a network of scientists are cooperating within the framework of the Harmonious Project to develop and promote harmonized mapping strategies and disseminate operational guidance to ensure best practice for data collection and interpretation. The culmination of these efforts is summarized in the present manuscript. Through this synthesis study, we identify the many interdependencies of each step in the collection and processing chain, and outline approaches to formalize and ensure a successful workflow and product development. Given the number of environmental conditions, constraints, and variables that could possibly be explored from UAS platforms, it is impractical to provide protocols that can be applied universally under all scenarios. However, it is possible to collate and systematically order the fragmented knowledge on UAS collection and analysis to identify the best practices that can best ensure the streamlined and rigorous development of scientific products.
TL;DR: A modeling framework to integrate climate data, remote sensing data and soil data to predict winter wheat yield based on the Google Earth Engine (GEE) platform is developed and shows that the models can accurately predict yield 1~2 months before the harvesting dates at the county level in China with an R² > 0.75 and yield error less than 10%.
Abstract: Wheat is one of the main crops in China, and crop yield prediction is important for regional trade and national food security. There are increasing concerns with respect to how to integrate multi-source data and employ machine learning techniques to establish a simple, timely, and accurate crop yield prediction model at an administrative unit. Many previous studies were mainly focused on the whole crop growth period through expensive manual surveys, remote sensing, or climate data. However, the effect of selecting different time window on yield prediction was still unknown. Thus, we separated the whole growth period into four time windows and assessed their corresponding predictive ability by taking the major winter wheat production regions of China as an example in the study. Firstly we developed a modeling framework to integrate climate data, remote sensing data and soil data to predict winter wheat yield based on the Google Earth Engine (GEE) platform. The results show that the models can accurately predict yield 1~2 months before the harvesting dates at the county level in China with an R2 > 0.75 and yield error less than 10%. Support vector machine (SVM), Gaussian process regression (GPR), and random forest (RF) represent the top three best methods for predicting yields among the eight typical machine learning models tested in this study. In addition, we also found that different agricultural zones and temporal training settings affect prediction accuracy. The three models perform better as more winter wheat growing season information becomes available. Our findings highlight a potentially powerful tool to predict yield using multiple-source data and machine learning in other regions and for crops.
TL;DR: An overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine, Sentinel Hub, Open Data Cube, System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud is presented.
Abstract: In recent years, Earth observation (EO) satellites have generated big amounts of geospatial data that are freely available for society and researchers. This scenario brings challenges for traditional spatial data infrastructures (SDI) to properly store, process, disseminate and analyze these big data sets. To meet these demands, novel technologies have been proposed and developed, based on cloud computing and distributed systems, such as array database systems, MapReduce systems and web services to access and process big Earth observation data. Currently, these technologies have been integrated into cutting edge platforms in order to support a new generation of SDI for big Earth observation data. This paper presents an overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine (GEE), Sentinel Hub, Open Data Cube (ODC), System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud. We also provide a comparison of these platforms according to criteria that represent capabilities of the EO community interest.
TL;DR: A new deep learning network, termed Building Residual Refine Network (BRRNet), for accurate and complete building extraction, which outperforms other five state-of-the-art methods in terms of the integrity of buildings and the accuracy of complex building footprints.
Abstract: Building extraction from high-resolution remote sensing images is of great significance in urban planning, population statistics, and economic forecast. However, automatic building extraction from high-resolution remote sensing images remains challenging. On the one hand, the extraction results of buildings are partially missing and incomplete due to the variation of hue and texture within a building, especially when the building size is large. On the other hand, the building footprint extraction of buildings with complex shapes is often inaccurate. To this end, we propose a new deep learning network, termed Building Residual Refine Network (BRRNet), for accurate and complete building extraction. BRRNet consists of such two parts as the prediction module and the residual refinement module. The prediction module based on an encoder–decoder structure introduces atrous convolution of different dilation rates to extract more global features, by gradually increasing the receptive field during feature extraction. When the prediction module outputs the preliminary building extraction results of the input image, the residual refinement module takes the output of the prediction module as an input. It further refines the residual between the result of the prediction module and the real result, thus improving the accuracy of building extraction. In addition, we use Dice loss as the loss function during training, which effectively alleviates the problem of data imbalance and further improves the accuracy of building extraction. The experimental results on Massachusetts Building Dataset show that our method outperforms other five state-of-the-art methods in terms of the integrity of buildings and the accuracy of complex building footprints.
TL;DR: This technical note focuses on the production of the foundation SST and IC analyses by OSTIA and aims to provide a comprehensive description of the current system configuration.
Abstract: The Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) system generates global, daily, gap-filled foundation sea surface temperature (SST) fields from satellite data and in situ observations. The SSTs have uncertainty information provided with them and an ice concentration (IC) analysis is also produced. Additionally, a global, hourly diurnal skin SST product is output each day. The system is run in near real time to produce data for use in applications such as numerical weather prediction. Data production is monitored routinely and outputs are available from the Copernicus Marine Environment Monitoring Service (CMEMS; marine.copernicus.eu). As an operational product, the OSTIA system is continuously under development. For example, since the original descriptor paper was published, the underlying data assimilation scheme that is used to generate the foundation SST analyses has been updated. Various publications have described these changes but a full description is not available in a single place. This technical note focuses on the production of the foundation SST and IC analyses by OSTIA and aims to provide a comprehensive description of the current system configuration.