scispace - formally typeset
Search or ask a question

Showing papers in "Remote Sensing in 2019"


Journal ArticleDOI
TL;DR: The CNN method is still in its infancy as most researchers will either use predefined parameters in solutions like Google TensorFlow or will apply different settings in a trial-and-error manner, Nevertheless, deep-learning can improve landslide mapping in the future if the effects of the different designs are better understood, enough training samples exist, and the results of augmentation strategies to artificially increase the number of existing samples are better understanding.
Abstract: There is a growing demand for detailed and accurate landslide maps and inventories around the globe, but particularly in hazard-prone regions such as the Himalayas. Most standard mapping methods require expert knowledge, supervision and fieldwork. In this study, we use optical data from the Rapid Eye satellite and topographic factors to analyze the potential of machine learning methods, i.e., artificial neural network (ANN), support vector machines (SVM) and random forest (RF), and different deep-learning convolution neural networks (CNNs) for landslide detection. We use two training zones and one test zone to independently evaluate the performance of different methods in the highly landslide-prone Rasuwa district in Nepal. Twenty different maps are created using ANN, SVM and RF and different CNN instantiations and are compared against the results of extensive fieldwork through a mean intersection-over-union (mIOU) and other common metrics. This accuracy assessment yields the best result of 78.26% mIOU for a small window size CNN, which uses spectral information only. The additional information from a 5 m digital elevation model helps to discriminate between human settlements and landslides but does not improve the overall classification accuracy. CNNs do not automatically outperform ANN, SVM and RF, although this is sometimes claimed. Rather, the performance of CNNs strongly depends on their design, i.e., layer depth, input window sizes and training strategies. Here, we conclude that the CNN method is still in its infancy as most researchers will either use predefined parameters in solutions like Google TensorFlow or will apply different settings in a trial-and-error manner. Nevertheless, deep-learning can improve landslide mapping in the future if the effects of the different designs are better understood, enough training samples exist, and the effects of augmentation strategies to artificially increase the number of existing samples are better understood.

458 citations


Journal ArticleDOI
TL;DR: A novel end-to-end CD method based on an effective encoderdecoder architecture for semantic segmentation named UNet++, where change maps could be learned from scratch using available annotated datasets, which outperforms the other state-of-the-art CD methods.
Abstract: Change detection (CD) is essential to the accurate understanding of land surface changes using available Earth observation data. Due to the great advantages in deep feature representation and nonlinear problem modeling, deep learning is becoming increasingly popular to solve CD tasks in remote-sensing community. However, most existing deep learning-based CD methods are implemented by either generating difference images using deep features or learning change relations between pixel patches, which leads to error accumulation problems since many intermediate processing steps are needed to obtain final change maps. To address the above-mentioned issues, a novel end-to-end CD method is proposed based on an effective encoder-decoder architecture for semantic segmentation named UNet++, where change maps could be learned from scratch using available annotated datasets. Firstly, co-registered image pairs are concatenated as an input for the improved UNet++ network, where both global and fine-grained information can be utilized to generate feature maps with high spatial accuracy. Then, the fusion strategy of multiple side outputs is adopted to combine change maps from different semantic levels, thereby generating a final change map with high accuracy. The effectiveness and reliability of our proposed CD method are verified on very-high-resolution (VHR) satellite image datasets. Extensive experimental results have shown that our proposed approach outperforms the other state-of-the-art CD methods.

408 citations


Journal ArticleDOI
TL;DR: The experimental results show that TempCNNs are more accurate than the current state of the art for SITS classification, and some general guidelines on the network architecture, common regularization mechanisms, and hyper-parameter values such as batch size are provided.
Abstract: Latest remote sensing sensors are capable of acquiring high spatial and spectral Satellite Image Time Series (SITS) of the world. These image series are a key component of classification systems that aim at obtaining up-to-date and accurate land cover maps of the Earth’s surfaces. More specifically, current SITS combine high temporal, spectral and spatial resolutions, which makes it possible to closely monitor vegetation dynamics. Although traditional classification algorithms, such as Random Forest (RF), have been successfully applied to create land cover maps from SITS, these algorithms do not make the most of the temporal domain. This paper proposes a comprehensive study of Temporal Convolutional Neural Networks (TempCNNs), a deep learning approach which applies convolutions in the temporal dimension in order to automatically learn temporal (and spectral) features. The goal of this paper is to quantitatively and qualitatively evaluate the contribution of TempCNNs for SITS classification, as compared to RF and Recurrent Neural Networks (RNNs) —a standard deep learning approach that is particularly suited to temporal data. We carry out experiments on Formosat-2 scene with 46 images and one million labelled time series. The experimental results show that TempCNNs are more accurate than the current state of the art for SITS classification. We provide some general guidelines on the network architecture, common regularization mechanisms, and hyper-parameter values such as batch size; we also draw out some differences with standard results in computer vision (e.g., about pooling layers). Finally, we assess the visual quality of the land cover maps produced by TempCNNs.

310 citations


Journal ArticleDOI
TL;DR: This paper performs a critical review on RS tasks that involve UAV data and their derived products as their main sources including raw perspective images, digital surface models, and orthophotos and focuses on solutions that address the “new” aspects of the U drone data including ultra-high resolution; availability of coherent geometric and spectral data; and capability of simultaneously using multi-sensor data for fusion.
Abstract: The unmanned aerial vehicle (UAV) sensors and platforms nowadays are being used in almost every application (e.g., agriculture, forestry, and mining) that needs observed information from the top or oblique views. While they intend to be a general remote sensing (RS) tool, the relevant RS data processing and analysis methods are still largely ad-hoc to applications. Although the obvious advantages of UAV data are their high spatial resolution and flexibility in acquisition and sensor integration, there is in general a lack of systematic analysis on how these characteristics alter solutions for typical RS tasks such as land-cover classification, change detection, and thematic mapping. For instance, the ultra-high-resolution data (less than 10 cm of Ground Sampling Distance (GSD)) bring more unwanted classes of objects (e.g., pedestrian and cars) in land-cover classification; the often available 3D data generated from photogrammetric images call for more advanced techniques for geometric and spectral analysis. In this paper, we perform a critical review on RS tasks that involve UAV data and their derived products as their main sources including raw perspective images, digital surface models, and orthophotos. In particular, we focus on solutions that address the “new” aspects of the UAV data including (1) ultra-high resolution; (2) availability of coherent geometric and spectral data; and (3) capability of simultaneously using multi-sensor data for fusion. Based on these solutions, we provide a brief summary of existing examples of UAV-based RS in agricultural, environmental, urban, and hazards assessment applications, etc., and by discussing their practical potentials, we share our views in their future research directions and draw conclusive remarks.

301 citations


Journal ArticleDOI
TL;DR: Experimental results reveal that object detectors achieve higher mean average precision (mAP) on the test dataset and have high generalization performance on new SAR imagery without land-ocean segmentation, demonstrating the benefits of the dataset the authors constructed.
Abstract: With the launch of space-borne satellites, more synthetic aperture radar (SAR) images are available than ever before, thus making dynamic ship monitoring possible. Object detectors in deep learning achieve top performance, benefitting from a free public dataset. Unfortunately, due to the lack of a large volume of labeled datasets, object detectors for SAR ship detection have developed slowly. To boost the development of object detectors in SAR images, a SAR dataset is constructed. This dataset labeled by SAR experts was created using 102 Chinese Gaofen-3 images and 108 Sentinel-1 images. It consists of 43,819 ship chips of 256 pixels in both range and azimuth. These ships mainly have distinct scales and backgrounds. Moreover, modified state-of-the-art object detectors from natural images are trained and can be used as baselines. Experimental results reveal that object detectors achieve higher mean average precision (mAP) on the test dataset and have high generalization performance on new SAR imagery without land-ocean segmentation, demonstrating the benefits of the dataset we constructed.

272 citations


Journal ArticleDOI
TL;DR: An effective remote sensing image scene classification architecture named CNN-CapsNet is proposed to make full use of the merits of these two models: CNN and CapsNet to lead to a competitive classification performance compared with the state-of-the-art methods.
Abstract: Remote sensing image scene classification is one of the most challenging problems in understanding high-resolution remote sensing images. Deep learning techniques, especially the convolutional neural network (CNN), have improved the performance of remote sensing image scene classification due to the powerful perspective of feature learning and reasoning. However, several fully connected layers are always added to the end of CNN models, which is not efficient in capturing the hierarchical structure of the entities in the images and does not fully consider the spatial information that is important to classification. Fortunately, capsule network (CapsNet), which is a novel network architecture that uses a group of neurons as a capsule or vector to replace the neuron in the traditional neural network and can encode the properties and spatial information of features in an image to achieve equivariance, has become an active area in the classification field in the past two years. Motivated by this idea, this paper proposes an effective remote sensing image scene classification architecture named CNN-CapsNet to make full use of the merits of these two models: CNN and CapsNet. First, a CNN without fully connected layers is used as an initial feature maps extractor. In detail, a pretrained deep CNN model that was fully trained on the ImageNet dataset is selected as a feature extractor in this paper. Then, the initial feature maps are fed into a newly designed CapsNet to obtain the final classification result. The proposed architecture is extensively evaluated on three public challenging benchmark remote sensing image datasets: the UC Merced Land-Use dataset with 21 scene categories, AID dataset with 30 scene categories, and the NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that the proposed method can lead to a competitive classification performance compared with the state-of-the-art methods.

254 citations


Journal ArticleDOI
TL;DR: A robust infrared patch-tensor model for detecting an infrared small target and the decomposition of the target and background is transformed into a tensor robust principle component analysis problem (TRPCA), which can be efficiently solved by alternating direction method of multipliers (ADMM).
Abstract: Excellent performance, real time and strong robustness are three vital requirements for infrared small target detection. Unfortunately, many current state-of-the-art methods merely achieve one of the expectations when coping with highly complex scenes. In fact, a common problem is that real-time processing and great detection ability are difficult to coordinate. Therefore, to address this issue, a robust infrared patch-tensor model for detecting an infrared small target is proposed in this paper. On the basis of infrared patch-tensor (IPT) model, a novel nonconvex low-rank constraint named partial sum of tensor nuclear norm (PSTNN) joint weighted l1 norm was employed to efficiently suppress the background and preserve the target. Due to the deficiency of RIPT which would over-shrink the target with the possibility of disappearing, an improved local prior map simultaneously encoded with target-related and background-related information was introduced into the model. With the help of a reweighted scheme for enhancing the sparsity and high-efficiency version of tensor singular value decomposition (t-SVD), the total algorithm complexity and computation time can be reduced dramatically. Then, the decomposition of the target and background is transformed into a tensor robust principle component analysis problem (TRPCA), which can be efficiently solved by alternating direction method of multipliers (ADMM). A series of experiments substantiate the superiority of the proposed method beyond state-of-the-art baselines.

241 citations


Journal ArticleDOI
TL;DR: The Google Earth Engine (GEE) is a cloud computing platform designed to store and process huge data sets (at petabyte-scale) for analysis and ultimate decision making.
Abstract: The Google Earth Engine (GEE) is a cloud computing platform designed to store and process huge data sets (at petabyte-scale) for analysis and ultimate decision making [...]

233 citations


Journal ArticleDOI
TL;DR: This study presents a novel method which classifies three tree species (Norway spruce, European beech, Silver fir), and dead spruce trees with crowns using full waveform ALS data acquired from three different sensors (wavelengths 532 nm, 1064 nm, 1550 nm).
Abstract: Knowledge about forest structures, particularly of deadwood, is fundamental for understanding, protecting, and conserving forest biodiversity. While individual tree-based approaches using single wavelength airborne laserscanning (ALS) can successfully distinguish broadleaf and coniferous trees, they still perform multiple tree species classifications with limited accuracy. Moreover, the mapping of standing dead trees is becoming increasingly important for damage calculation after pest infestation or biodiversity assessment. Recent advances in sensor technology have led to the development of new ALS systems that provide up to three different wavelengths. In this study, we present a novel method which classifies three tree species (Norway spruce, European beech, Silver fir), and dead spruce trees with crowns using full waveform ALS data acquired from three different sensors (wavelengths 532 nm, 1064 nm, 1550 nm). The ALS data were acquired in the Bavarian Forest National Park (Germany) under leaf-on conditions with a maximum point density of 200 points/m 2 . To avoid overfitting of the classifier and to find the most prominent features, we embed a forward feature selection method. We tested our classification procedure using 20 sample plots with 586 measured reference trees. Using single wavelength datasets, the highest accuracy achieved was 74% (wavelength = 1064 nm), followed by 69% (wavelength = 1550 nm) and 65% (wavelength = 532 nm). An improvement of 8–17% over single wavelength datasets was achieved when the multi wavelength data were used. Overall, the contribution of the waveform-based features to the classification accuracy was higher than that of the geometric features by approximately 10%. Our results show that the features derived from a multi wavelength ALS point cloud significantly improve the detailed mapping of tree species and standing dead trees.

220 citations


Journal ArticleDOI
TL;DR: An enhanced GPU based deep learning method to detect ship from the SAR images using the YOLOv2 architecture with less number of layers is proposed and the experimental results shows that the deep learning can make a big leap forward in improving the performance of SAR image ship detection.
Abstract: Synthetic aperture radar (SAR) imagery has been used as a promising data source for monitoring maritime activities, and its application for oil and ship detection has been the focus of many previous research studies. Many object detection methods ranging from traditional to deep learning approaches have been proposed. However, majority of them are computationally intensive and have accuracy problems. The huge volume of the remote sensing data also brings a challenge for real time object detection. To mitigate this problem a high performance computing (HPC) method has been proposed to accelerate SAR imagery analysis, utilizing the GPU based computing methods. In this paper, we propose an enhanced GPU based deep learning method to detect ship from the SAR images. The You Only Look Once version 2 (YOLOv2) deep learning framework is proposed to model the architecture and training the model. YOLOv2 is a state-of-the-art real-time object detection system, which outperforms Faster Region-Based Convolutional Network (Faster R-CNN) and Single Shot Multibox Detector (SSD) methods. Additionally, in order to reduce computational time with relatively competitive detection accuracy, we develop a new architecture with less number of layers called YOLOv2-reduced. In the experiment, we use two types of datasets: A SAR ship detection dataset (SSDD) dataset and a Diversified SAR Ship Detection Dataset (DSSDD). These two datasets were used for training and testing purposes. YOLOv2 test results showed an increase in accuracy of ship detection as well as a noticeable reduction in computational time compared to Faster R-CNN. From the experimental results, the proposed YOLOv2 architecture achieves an accuracy of 90.05% and 89.13% on the SSDD and DSSDD datasets respectively. The proposed YOLOv2-reduced architecture has a similarly competent detection performance as YOLOv2, but with less computational time on a NVIDIA TITAN X GPU. The experimental results shows that the deep learning can make a big leap forward in improving the performance of SAR image ship detection.

217 citations


Journal ArticleDOI
TL;DR: A new global ‘OCO-2’ SIF data set (GOSIF) with high spatial and temporal resolutions (i.e., 0.05°, 8-day) over the period 2000–2017 based on a data-driven approach, which is valuable for assessing terrestrial photosynthesis and ecosystem function, and benchmarking terrestrial biosphere and Earth system models.
Abstract: Solar-induced chlorophyll fluorescence (SIF) brings major advancements in measuring terrestrial photosynthesis. Several recent studies have evaluated the potential of SIF retrievals from the Orbiting Carbon Observatory-2 (OCO-2) in estimating gross primary productivity (GPP) based on GPP data from eddy covariance (EC) flux towers. However, the spatially and temporally sparse nature of OCO-2 data makes it challenging to use these data for many applications from the ecosystem to the global scale. Here, we developed a new global ‘OCO-2’ SIF data set (GOSIF) with high spatial and temporal resolutions (i.e., 0.05°, 8-day) over the period 2000–2017 based on a data-driven approach. The predictive SIF model was developed based on discrete OCO-2 SIF soundings, remote sensing data from the Moderate Resolution Imaging Spectroradiometer (MODIS), and meteorological reanalysis data. Our model performed well in estimating SIF (R2 = 0.79, root mean squared error (RMSE) = 0.07 W m−2 μm−1 sr−1). The model was then used to estimate SIF for each 0.05° × 0.05° grid cell and each 8-day interval for the study period. The resulting GOSIF product has reasonable seasonal cycles, and captures the similar seasonality as both the coarse-resolution OCO-2 SIF (1°), directly aggregated from the discrete OCO-2 soundings, and tower-based GPP. Our SIF estimates are highly correlated with GPP from 91 EC flux sites (R2 = 0.73, p < 0.001). They capture the expected spatial and temporal patterns and also have remarkable ability to highlight the crop areas with the highest daily productivity across the globe. Our product also allows us to examine the long-term trends in SIF globally. Compared with the coarse-resolution SIF that was directly aggregated from OCO-2 soundings, GOSIF has finer spatial resolution, globally continuous coverage, and a much longer record. Our GOSIF product is valuable for assessing terrestrial photosynthesis and ecosystem function, and benchmarking terrestrial biosphere and Earth system models.

Journal ArticleDOI
TL;DR: A new land cover strategy is designed and comprehensive methods, models, and procedures for NLCD 2016 implementation are developed to ensure the spatial and temporal coherence of land cover and land cover changes over 15 years.
Abstract: The National Land Cover Database (NLCD) 2016 provides a suite of data products, including land cover and land cover change of the conterminous United States from 2001 to 2016, at two- to three-year intervals. The development of this product is part of an effort to meet the growing demand for longer temporal duration and more frequent, accurate, and consistent land cover and change information. To accomplish this, we designed a new land cover strategy and developed comprehensive methods, models, and procedures for NLCD 2016 implementation. Major steps in the new procedures consist of data preparation, land cover change detection and classification, theme-based postprocessing, and final integration. Data preparation includes Landsat imagery selection, cloud detection, and cloud filling, as well as compilation and creation of more than 30 national-scale ancillary datasets. Land cover change detection includes single-date water and snow/ice detection algorithms and models, two-date multi-index integrated change detection models, and long-term multi-date change algorithms and models. The land cover classification includes seven-date training data creation and 14-run classifications. Pools of training data for change and no-change areas were created before classification based on integrated information from ancillary data, change-detection results, Landsat spectral and temporal information, and knowledge-based trajectory analysis. In postprocessing, comprehensive models for each land cover theme were developed in a hierarchical order to ensure the spatial and temporal coherence of land cover and land cover changes over 15 years. An initial accuracy assessment on four selected Landsat path/rows classified with this method indicates an overall accuracy of 82.0% at an Anderson Level II classification and 86.6% at the Anderson Level I classification after combining the primary and alternate reference labels. This methodology was used for the operational production of NLCD 2016 for the Conterminous United States, with final produced products available for free download.

Journal ArticleDOI
TL;DR: It is shown that the XGBoost feature selection approach effectively addresses the issue of high landscape heterogeneity and spectral complexities in the image data, successfully augmenting the RF model performance (providing a mean user’s accuracy > 0.82).
Abstract: Although a detailed analysis of land use and land cover (LULC) change is essential in providing a greater understanding of increased human-environment interactions across the coastal region of Bangladesh, substantial challenges still exist for accurately classifying coastal LULC. This is due to the existence of high-level landscape heterogeneity and unavailability of good quality remotely sensed data. This study, the first of a kind, implemented a unique methodological approach to this challenge. Using freely available Landsat imagery, eXtreme Gradient Boosting (XGBoost)-based informative feature selection and Random Forest classification is used to elucidate spatio-temporal patterns of LULC across coastal areas over a 28-year period (1990–2017). We show that the XGBoost feature selection approach effectively addresses the issue of high landscape heterogeneity and spectral complexities in the image data, successfully augmenting the RF model performance (providing a mean user’s accuracy > 0.82). Multi-temporal LULC maps reveal that Bangladesh’s coastal areas experienced a net increase in agricultural land (5.44%), built-up (4.91%) and river (4.52%) areas over the past 28 years. While vegetation cover experienced a net decrease (8.26%), an increasing vegetation trend was observed in the years since 2000, primarily due to the Bangladesh government’s afforestation initiatives across the southern coastal belts. These findings provide a comprehensive picture of coastal LULC patterns, which will be useful for policy makers and resource managers to incorporate into coastal land use and environmental management practices. This work also provides useful methodological insights for future research to effectively address the spatial and spectral complexities of remotely sensed data used in classifying the LULC of a heterogeneous landscape.

Journal ArticleDOI
TL;DR: This paper elaborate deep learning approaches designed for activity recognition in radar according to the dimension of radar returns, including 1D, 2D and 3D echoes and addresses some current research considerations and future opportunities.
Abstract: Radar, as one of the sensors for human activity recognition (HAR), has unique characteristics such as privacy protection and contactless sensing. Radar-based HAR has been applied in many fields such as human–computer interaction, smart surveillance and health assessment. Conventional machine learning approaches rely on heuristic hand-crafted feature extraction, and their generalization capability is limited. Additionally, extracting features manually is time–consuming and inefficient. Deep learning acts as a hierarchical approach to learn high-level features automatically and has achieved superior performance for HAR. This paper surveys deep learning based HAR in radar from three aspects: deep learning techniques, radar systems, and deep learning for radar-based HAR. Especially, we elaborate deep learning approaches designed for activity recognition in radar according to the dimension of radar returns (i.e., 1D, 2D and 3D echoes). Due to the difference of echo forms, corresponding deep learning approaches are different to fully exploit motion information. Experimental results have demonstrated the feasibility of applying deep learning for radar-based HAR in 1D, 2D and 3D echoes. Finally, we address some current research considerations and future opportunities.

Journal ArticleDOI
TL;DR: The experimental results reveal that retinaNet not only can efficiently detect multi-scale ships but also has a high detection accuracy and compared with other object detectors, RetinaNet achieves more than a 96% mean average precision (mAP).
Abstract: Independent of daylight and weather conditions, synthetic aperture radar (SAR) imagery is widely applied to detect ships in marine surveillance. The shapes of ships are multi-scale in SAR imagery due to multi-resolution imaging modes and their various shapes. Conventional ship detection methods are highly dependent on the statistical models of sea clutter or the extracted features, and their robustness need to be strengthened. Being an automatic learning representation, the RetinaNet object detector, one kind of deep learning model, is proposed to crack this obstacle. Firstly, feature pyramid networks (FPN) are used to extract multi-scale features for both ship classification and location. Then, focal loss is used to address the class imbalance and to increase the importance of the hard examples during training. There are 86 scenes of Chinese Gaofen-3 Imagery at four resolutions, i.e., 3 m, 5 m, 8 m, and 10 m, used to evaluate our approach. Two Gaofen-3 images and one Constellation of Small Satellite for Mediterranean basin Observation (Cosmo-SkyMed) image are used to evaluate the robustness. The experimental results reveal that (1) RetinaNet not only can efficiently detect multi-scale ships but also has a high detection accuracy; (2) compared with other object detectors, RetinaNet achieves more than a 96% mean average precision (mAP). These results demonstrate the effectiveness of our proposed method.

Journal ArticleDOI
TL;DR: Experimental results demonstrate that the proposed spectral-spatial attention network for hyperspectral image classification can fully utilize the spectral and spatial information to obtain competitive performance.
Abstract: Many deep learning models, such as convolutional neural network (CNN) and recurrent neural network (RNN), have been successfully applied to extracting deep features for hyperspectral tasks. Hyperspectral image classification allows distinguishing the characterization of land covers by utilizing their abundant information. Motivated by the attention mechanism of the human visual system, in this study, we propose a spectral-spatial attention network for hyperspectral image classification. In our method, RNN with attention can learn inner spectral correlations within a continuous spectrum, while CNN with attention is designed to focus on saliency features and spatial relevance between neighboring pixels in the spatial dimension. Experimental results demonstrate that our method can fully utilize the spectral and spatial information to obtain competitive performance.

Journal ArticleDOI
TL;DR: This semi-supervised deep learning approach demonstrates that remote sensing can overcome a lack of labeled training data by generating noisy data for initial training using unsupervised methods and retraining the resulting models with high quality labeled data.
Abstract: Remote sensing can transform the speed, scale, and cost of biodiversity and forestry surveys. Data acquisition currently outpaces the ability to identify individual organisms in high resolution imagery. We outline an approach for identifying tree-crowns in RGB imagery while using a semi-supervised deep learning detection network. Individual crown delineation has been a long-standing challenge in remote sensing and available algorithms produce mixed results. We show that deep learning models can leverage existing Light Detection and Ranging (LIDAR)-based unsupervised delineation to generate trees that are used for training an initial RGB crown detection model. Despite limitations in the original unsupervised detection approach, this noisy training data may contain information from which the neural network can learn initial tree features. We then refine the initial model using a small number of higher-quality hand-annotated RGB images. We validate our proposed approach while using an open-canopy site in the National Ecological Observation Network. Our results show that a model using 434,551 self-generated trees with the addition of 2848 hand-annotated trees yields accurate predictions in natural landscapes. Using an intersection-over-union threshold of 0.5, the full model had an average tree crown recall of 0.69, with a precision of 0.61 for the visually-annotated data. The model had an average tree detection rate of 0.82 for the field collected stems. The addition of a small number of hand-annotated trees improved the performance over the initial self-supervised model. This semi-supervised deep learning approach demonstrates that remote sensing can overcome a lack of labeled training data by generating noisy data for initial training using unsupervised methods and retraining the resulting models with high quality labeled data.

Journal ArticleDOI
TL;DR: UAV-based thermal imaging is a viable tool in precision agriculture and the three examined cameras are comparable in terms of their efficacy for plant phenotyping, with ICI 8640 P-series presenting the best results among the three systems.
Abstract: The growing popularity of Unmanned Aerial Vehicles (UAVs) in recent years, along with decreased cost and greater accessibility of both UAVs and thermal imaging sensors, has led to the widespread use of this technology, especially for precision agriculture and plant phenotyping. There are several thermal camera systems in the market that are available at a low cost. However, their efficacy and accuracy in various applications has not been tested. In this study, three commercially available UAV thermal cameras, including ICI 8640 P-series (Infrared Cameras Inc., USA), FLIR Vue Pro R 640 (FLIR Systems, USA), and thermoMap (senseFly, Switzerland) have been tested and evaluated for their potential for forest monitoring, vegetation stress detection, and plant phenotyping. Mounted on multi-rotor or fixed wing systems, these cameras were simultaneously flown over different experimental sites located in St. Louis, Missouri (forest environment), Columbia, Missouri (plant stress detection and phenotyping), and Maricopa, Arizona (high throughput phenotyping). Thermal imagery was calibrated using procedures that utilize a blackbody, handheld thermal spot imager, ground thermal targets, emissivity and atmospheric correction. A suite of statistical analyses, including analysis of variance (ANOVA), correlation analysis between camera temperature and plant biophysical and biochemical traits, and heritability were utilized in order to examine the sensitivity and utility of the cameras against selected plant phenotypic traits and in the detection of plant water stress. In addition, in reference to quantitative assessment of image quality from different thermal cameras, a non-reference image quality evaluator, which primarily measures image focus that is based on the spatial relationship of pixels in different scales, was developed. Our results show that (1) UAV-based thermal imaging is a viable tool in precision agriculture and (2) the three examined cameras are comparable in terms of their efficacy for plant phenotyping. Overall, accuracy, when compared against field measured ground temperature and estimating power of plant biophysical and biochemical traits, the ICI 8640 P-series performed better than the other two cameras, followed by FLIR Vue Pro R 640 and thermoMap cameras. Our results demonstrated that all three UAV thermal cameras provide useful temperature data for precision agriculture and plant phenotying, with ICI 8640 P-series presenting the best results among the three systems. Cost wise, FLIR Vue Pro R 640 is more affordable than the other two cameras, providing a less expensive option for a wide range of applications.

Journal ArticleDOI
TL;DR: A Double-Branch Multi-Attention mechanism network (DBMA) is proposed for HSI classification which has two branches to extract spectral and spatial feature respectively which can reduce the interference between the two types of feature.
Abstract: Recently, Hyperspectral Image (HSI) classification has gradually been getting attention from more and more researchers. HSI has abundant spectral and spatial information; thus, how to fuse these two types of information is still a problem worth studying. In this paper, to extract spectral and spatial feature, we propose a Double-Branch Multi-Attention mechanism network (DBMA) for HSI classification. This network has two branches to extract spectral and spatial feature respectively which can reduce the interference between the two types of feature. Furthermore, with respect to the different characteristics of these two branches, two types of attention mechanism are applied in the two branches respectively, which ensures to extract more discriminative spectral and spatial feature. The extracted features are then fused for classification. A lot of experiment results on three hyperspectral datasets shows that the proposed method performs better than the state-of-the-art method.

Journal ArticleDOI
TL;DR: This study has improved a newly-proposed remote sensing based ecological index with a sharpened land surface temperature image and then used the improved index to produce the time series of ecological-status images, showing that the RSEI-CVA approach would serve as a prototype method to quantify and detect ecological changes and hence promote ecological change detection at various scales.
Abstract: Increasing human activities have caused significant global ecosystem disturbances at various scales. There is an increasing need for effective techniques to quantify and detect ecological changes. Remote sensing can serve as a measurement surrogate of spatial changes in ecological conditions. This study has improved a newly-proposed remote sensing based ecological index (RSEI) with a sharpened land surface temperature image and then used the improved index to produce the time series of ecological-status images. The Mann–Kendall test and Theil–Sen estimator were employed to evaluate the significance of the trend of the RSEI time series and the direction of change. The change vector analysis (CVA) was employed to detect ecological changes based on the image series. This RSEI-CVA approach was applied to Fujian province, China to quantify and detect the ecological changes of the province in a period from 2002 to 2017 using Moderate Resolution Imaging Spectroradiometer (MODIS) data. The result shows that the RSEI-CVA method can effectively quantify and detect spatiotemporal changes in ecological conditions of the province, which reveals an ecological improvement in the province during the study period. This is indicated by the rise of mean RSEI scores from 0.794 to 0.852 due to an increase in forest area by 7078 km2. Nevertheless, CVA-based change detection has detected ecological declines in the eastern coastal areas of the province. This study shows that the RSEI-CVA approach would serve as a prototype method to quantify and detect ecological changes and hence promote ecological change detection at various scales.

Journal ArticleDOI
TL;DR: The first early on-orbit validation of ICESat-2 bathymetry and quantification of the bathymetric mapping performance of ATLAS using data acquired over St. Thomas, U.S. Virgin Islands is presented.
Abstract: NASA’s Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) was launched in September, 2018. The satellite carries a single instrument, ATLAS (Advanced Topographic Laser Altimeter System), a green wavelength, photon-counting lidar, enabling global measurement and monitoring of elevation with a primary focus on the cryosphere. Although bathymetric mapping was not one of the design goals for ATLAS, pre-launch work by our research team showed the potential to map bathymetry with ICESat-2, using data from MABEL (Multiple Altimeter Beam Experimental Lidar), NASA’s high-altitude airborne ATLAS emulator, and adapting the laser-radar equation for ATLAS specific parameters. However, many of the sensor variables were only approximations, which limited a full assessment of the bathymetric mapping capabilities of ICESat-2 during pre-launch studies. Following the successful launch, preliminary analyses of the geolocated photon returns have been conducted for a number of coastal sites, revealing several salient examples of seafloor detection in water depths of up to ~40 m. The geolocated seafloor photon returns cannot be taken as bathymetric measurements, however, since the algorithm used to generate them is not designed to account for the refraction that occurs at the air–water interface or the corresponding change in the speed of light in the water column. This paper presents the first early on-orbit validation of ICESat-2 bathymetry and quantification of the bathymetric mapping performance of ATLAS using data acquired over St. Thomas, U.S. Virgin Islands. A refraction correction, developed and tested in this work, is applied, after which the ICESat-2 bathymetry is compared against high-accuracy airborne topo-bathymetric lidar reference data collected by the U.S. Geological Survey (USGS) and the National Oceanic and Atmospheric Administration (NOAA). The results show agreement to within 0.43—0.60 m root mean square error (RMSE) over 1 m grid resolution for these early on-orbit data. Refraction-corrected bottom return photons are then inspected for four coastal locations around the globe in relation to Visible Infrared Imaging Radiometer Suite (VIIRS) Kd(490) data to empirically determine the maximum depth mapping capability of ATLAS as a function of water clarity. It is demonstrated that ATLAS has a maximum depth mapping capability of nearly 1 Secchi in depth for water depths up to 38 m and Kd(490) in the range of 0.05–0.12 m−1. Collectively, these results indicate the great potential for bathymetric mapping with ICESat-2, offering a promising new tool to assist in filling the global void in nearshore bathymetry.

Journal ArticleDOI
TL;DR: A systematic review of transfer learning application for scene classification using different datasets and different deep-learning models shows that transfer learning provides a powerful tool for remote-sensing scene classification.
Abstract: Remote-sensing image scene classification can provide significant value, ranging from forest fire monitoring to land-use and land-cover classification. Beginning with the first aerial photographs of the early 20th century to the satellite imagery of today, the amount of remote-sensing data has increased geometrically with a higher resolution. The need to analyze these modern digital data motivated research to accelerate remote-sensing image classification. Fortunately, great advances have been made by the computer vision community to classify natural images or photographs taken with an ordinary camera. Natural image datasets can range up to millions of samples and are, therefore, amenable to deep-learning techniques. Many fields of science, remote sensing included, were able to exploit the success of natural image classification by convolutional neural network models using a technique commonly called transfer learning. We provide a systematic review of transfer learning application for scene classification using different datasets and different deep-learning models. We evaluate how the specialization of convolutional neural network models affects the transfer learning process by splitting original models in different points. As expected, we find the choice of hyperparameters used to train the model has a significant influence on the final performance of the models. Curiously, we find transfer learning from models trained on larger, more generic natural images datasets outperformed transfer learning from models trained directly on smaller remotely sensed datasets. Nonetheless, results show that transfer learning provides a powerful tool for remote-sensing scene classification.

Journal ArticleDOI
TL;DR: The challenges in current nighttime light remote sensing research are summarized and four strategic directions are proposed, including improving nighttime light data; developing a long time series of consistent nighttime lightData; integrating nighttime light observations with other data and knowledge; and promoting multidisciplinary and interdisciplinary analyses of nighttime light observation.
Abstract: Nighttime light observations from remote sensing provide us with a timely and spatially explicit measure of human activities, and therefore enable a host of applications such as tracking urbanization and socioeconomic dynamics, evaluating armed conflicts and disasters, investigating fisheries, assessing greenhouse gas emissions and energy use, and analyzing light pollution and health effects. The new and improved sensors, algorithms, and products for nighttime lights, in association with other Earth observations and ancillary data (e.g., geo-located big data), together offer great potential for a deep understanding of human activities and related environmental consequences in a changing world. This paper reviews the advances of nighttime light sensors and products and examines the contributions of nighttime light remote sensing to perceiving the changing world from two aspects (i.e., human activities and environmental changes). Based on the historical review of the advances in nighttime light remote sensing, we summarize the challenges in current nighttime light remote sensing research and propose four strategic directions, including: Improving nighttime light data; developing a long time series of consistent nighttime light data; integrating nighttime light observations with other data and knowledge; and promoting multidisciplinary and interdisciplinary analyses of nighttime light observations.

Journal ArticleDOI
TL;DR: The study determined that cropland damaged by floods was 1.51% in April, 3.46% in June, 5.30% in August, located mostly in the Sylhet and Rangpur divisions.
Abstract: Bangladesh is one of the most flood-affected countries in the world. In the last few decades, flood frequency, intensity, duration, and devastation have increased in Bangladesh. Identifying flood-damaged areas is highly essential for an effective flood response. This study aimed at developing an operational methodology for rapid flood inundation and potential flood damaged area mapping to support a quick and effective event response. Sentinel-1 images from March, April, June, and August 2017 were used to generate inundation extents of the corresponding months. The 2017 pre-flood land cover maps were prepared using Landsat-8 images to identify major land cover on the ground before flooding. The overall accuracy of flood inundation mapping was 96.44% and the accuracy of the land cover map was 87.51%. The total flood inundated area corresponded to 2.01%, 4.53%, and 7.01% for the months April, June, and August 2017, respectively. Based on the Landsat-8 derived land cover information, the study determined that cropland damaged by floods was 1.51% in April, 3.46% in June, 5.30% in August, located mostly in the Sylhet and Rangpur divisions. Finally, flood inundation maps were distributed to the broader user community to aid in hazard response. The data and methodology of the study can be replicated for every year to map flooding in Bangladesh.

Journal ArticleDOI
TL;DR: It was observed that prediction accuracy reduces from Unmanned Aerial Systems (UASs) to satellite platforms, though advances in machine learning techniques could further assist in the generation of better calibration models.
Abstract: Towards the need for sustainable development, remote sensing (RS) techniques in the Visible-Near Infrared–Shortwave Infrared (VNIR–SWIR, 400–2500 nm) region could assist in a more direct, cost-effective and rapid manner to estimate important indicators for soil monitoring purposes. Soil reflectance spectroscopy has been applied in various domains apart from laboratory conditions, e.g., sensors mounted on satellites, aircrafts and Unmanned Aerial Systems. The aim of this review is to illustrate the research made for soil organic carbon estimation, with the use of RS techniques, reporting the methodology and results of each study. It also aims to provide a comprehensive introduction in soil spectroscopy for those who are less conversant with the subject. In total, 28 journal articles were selected and further analysed. It was observed that prediction accuracy reduces from Unmanned Aerial Systems (UASs) to satellite platforms, though advances in machine learning techniques could further assist in the generation of better calibration models. There are some challenges concerning atmospheric, radiometric and geometric corrections, vegetation cover, soil moisture and roughness that still need to be addressed. The advantages and disadvantages of each approach are highlighted and future considerations are also discussed at the end.

Journal ArticleDOI
TL;DR: A technique for data acquisition and image processing was developed utilizing small unmanned aerial vehicles, multispectral imaging, and deep learning convolutional neural networks to evaluate phenotypic characteristics on citrus crops.
Abstract: Traditional plant breeding evaluation methods are time-consuming, labor-intensive, and costly. Accurate and rapid phenotypic trait data acquisition and analysis can improve genomic selection and accelerate cultivar development. In this work, a technique for data acquisition and image processing was developed utilizing small unmanned aerial vehicles (UAVs), multispectral imaging, and deep learning convolutional neural networks to evaluate phenotypic characteristics on citrus crops. This low-cost and automated high-throughput phenotyping technique utilizes artificial intelligence (AI) and machine learning (ML) to: (i) detect, count, and geolocate trees and tree gaps; (ii) categorize trees based on their canopy size; (iii) develop individual tree health indices; and (iv) evaluate citrus varieties and rootstocks. The proposed remote sensing technique was able to detect and count citrus trees in a grove of 4,931 trees, with precision and recall of 99.9% and 99.7%, respectively, estimate their canopy size with overall accuracy of 85.5%, and detect, count, and geolocate tree gaps with a precision and recall of 100% and 94.6%, respectively. This UAV-based technique provides a consistent, more direct, cost-effective, and rapid method to evaluate phenotypic characteristics of citrus varieties and rootstocks.

Journal ArticleDOI
TL;DR: This research proposes a U-Net-based semantic segmentation method for the extraction of building footprints from high-resolution multispectral satellite images using the SpaceNet building dataset provided in the DeepGlobe Satellite Challenge of IEEE Conference on Computer Vision and Pattern Recognition 2018 (CVPR 2018).
Abstract: Automatic extraction of building footprints from high-resolution satellite imagery has become an important and challenging research issue receiving greater attention. Many recent studies have explored different deep learning-based semantic segmentation methods for improving the accuracy of building extraction. Although they record substantial land cover and land use information (e.g., buildings, roads, water, etc.), public geographic information system (GIS) map datasets have rarely been utilized to improve building extraction results in existing studies. In this research, we propose a U-Net-based semantic segmentation method for the extraction of building footprints from high-resolution multispectral satellite images using the SpaceNet building dataset provided in the DeepGlobe Satellite Challenge of IEEE Conference on Computer Vision and Pattern Recognition 2018 (CVPR 2018). We explore the potential of multiple public GIS map datasets (OpenStreetMap, Google Maps, and MapWorld) through integration with the WorldView-3 satellite datasets in four cities (Las Vegas, Paris, Shanghai, and Khartoum). Several strategies are designed and combined with the U-Net–based semantic segmentation model, including data augmentation, post-processing, and integration of the GIS map data and satellite images. The proposed method achieves a total F1-score of 0.704, which is an improvement of 1.1% to 12.5% compared with the top three solutions in the SpaceNet Building Detection Competition and 3.0% to 9.2% compared with the standard U-Net–based method. Moreover, the effect of each proposed strategy and the possible reasons for the building footprint extraction results are analyzed substantially considering the actual situation of the four cities.

Journal ArticleDOI
TL;DR: This review provides an overview of the techniques that are currently being used to map various attributes of mangrove, summarizes the studies that have been undertaken since 2010 on a variety of remote sensing applications for monitoring mangroves, and addresses the limitations of these studies.
Abstract: The mangrove ecosystem plays a vital role in the global carbon cycle, by reducing greenhouse gas emissions and mitigating the impacts of climate change. However, mangroves have been lost worldwide, resulting in substantial carbon stock losses. Additionally, some aspects of the mangrove ecosystem remain poorly characterized compared to other forest ecosystems due to practical difficulties in measuring and monitoring mangrove biomass and their carbon stocks. Without a quantitative method for effectively monitoring biophysical parameters and carbon stocks in mangroves, robust policies and actions for sustainably conserving mangroves in the context of climate change mitigation and adaptation are more difficult. In this context, remote sensing provides an important tool for monitoring mangroves and identifying attributes such as species, biomass, and carbon stocks. A wide range of studies is based on optical imagery (aerial photography, multispectral, and hyperspectral) and synthetic aperture radar (SAR) data. Remote sensing approaches have been proven effective for mapping mangrove species, estimating their biomass, and assessing changes in their extent. This review provides an overview of the techniques that are currently being used to map various attributes of mangroves, summarizes the studies that have been undertaken since 2010 on a variety of remote sensing applications for monitoring mangroves, and addresses the limitations of these studies. We see several key future directions for the potential use of remote sensing techniques combined with machine learning techniques for mapping mangrove areas and species, and evaluating their biomass and carbon stocks.

Journal ArticleDOI
TL;DR: To validate the Sentinel-2 operational Level 2A cloud mask, a software program named Active Learning Cloud Detection (ALCD) was developed, to produce reference cloud masks, designed to minimize human operator time thanks to a manually-supervised active learning method.
Abstract: The Sentinel-2 satellite mission, developed by the European Space Agency (ESA) for the Copernicus program of the European Union, provides repetitive multi-spectral observations of all Earth land surfaces at a high resolution. The Level 2A product is a basic product requested by many Sentinel-2 users: it provides surface reflectance after atmospheric correction, with a cloud and cloud shadow mask. The cloud/shadow mask is a key element to enable an automatic processing of Sentinel-2 data, and therefore, its performances must be accurately validated. To validate the Sentinel-2 operational Level 2A cloud mask, a software program named Active Learning Cloud Detection (ALCD) was developed, to produce reference cloud masks. Active learning methods allow reducing the number of necessary training samples by iteratively selecting them where the confidence of the classifier is low in the previous iterations. The ALCD method was designed to minimize human operator time thanks to a manually-supervised active learning method. The trained classifier uses a combination of spectral and multi-temporal information as input features and produces fully-classified images. The ALCD method was validated using visual criteria, consistency checks, and compared to another manually-generated cloud masks, with an overall accuracy above 98%. ALCD was used to create 32 reference cloud masks, on 10 different sites, with different seasons and cloud cover types. These masks were used to validate the cloud and shadow masks produced by three Sentinel-2 Level 2A processors: MAJA, used by the French Space Agency (CNES) to deliver Level 2A products, Sen2Cor, used by the European Space Agency (ESA), and FMask, used by the United States Geological Survey (USGS). The results show that MAJA and FMask perform similarly, with an overall accuracy around 90% (91% for MAJA, 90% for FMask), while Sen2Cor’s overall accuracy is 84%. The reference cloud masks, as well as the ALCD software used to generate them are made available to the Sentinel-2 user community.

Journal ArticleDOI
TL;DR: The performance of the models is evaluated using statistical metrics (i.e., sensitivity, specificity and accuracy) while the validation of the results is done by constructing the Receiver Operating Characteristics (ROC) Curve and Area Under Curve (AUC) values and by calculating the density of torrential pixels within FFPI classes.
Abstract: Concerning the significant increase in the negative effects of flash-floods worldwide, the main goal of this research is to evaluate the power of the Analytical Hierarchy Process (AHP), fi (kNN), K-Star (KS) algorithms and their ensembles in flash-flood susceptibility mapping. To train the two stand-alone models and their ensembles, for the first stage, the areas affected in the past by torrential phenomena are identified using remote sensing techniques. Approximately 70% of these areas are used as a training data set along with 10 flash-flood predictors. It should be remarked that the remote sensing techniques play a crucial role in obtaining eight out of 10 flash-flood conditioning factors. The predictive capability of predictors is evaluated through the Information Gain Ratio (IGR) method. As expected, the slope angle results in the factor with the highest predictive capability. The application of the AHP model implies the construction of ten pair-wise comparison matrices for calculating the normalized weights of each flash-flood predictor. The computed weights are used as input data in kNN–AHP and KS–AHP ensemble models for calculating the Flash-Flood Potential Index (FFPI). The FFPI also is determined through kNN and KS stand-alone models. The performance of the models is evaluated using statistical metrics (i.e., sensitivity, specificity and accuracy) while the validation of the results is done by constructing the Receiver Operating Characteristics (ROC) Curve and Area Under Curve (AUC) values and by calculating the density of torrential pixels within FFPI classes. Overall, the best performance is obtained by the kNN–AHP ensemble model.