In this article, we propose a deep learning-based algorithm for the classification of crop types from Sentinel-1 and Sentinel-2 time series data which is based on the celebrated transformer architecture. Crucially, we enable our algorithm to do early classification, i.e., predict crop types at arbitrary time points early in the year with a single trained model (progressive intra-season classification). Such early season predictions are of practical relevance for instance for yield forecasts or the modeling of agricultural water balances, therefore being important for the public as well as the private sector. Furthermore, we improve the mechanism of combining different data sources for the prediction task, allowing for both optical and radar data as inputs (multi-modal data fusion) without the need for temporal interpolation. We can demonstrate the effectiveness of our approach on an extensive data set from three federal states of Germany reaching an average F1 score of 0.92 using data of a complete growing season to predict the eight most important crop types and an F1 score above 0.8 when doing early classification at least one month before harvest time. In carefully chosen experiments, we can show that our model generalizes well in time and space.

/pdf/early-crop-classification-via-multi-modal-satellite-data-hqbpepmm.pdf

Early Crop Classification via Multi-Modal Satellite Data Fusion and Temporal Attention

Accurate plant-type (PT) detection forms an important basis for sustainable land management maintaining biodiversity and ecosystem services. In this sense, Sentinel-2 satellite images of the Copernicus program offer spatial, spectral, temporal, and radiometric characteristics with great potential for mapping and monitoring PTs. In addition, the selection of a best-performing algorithm needs to be considered for obtaining PT classification as accurate as possible. To date, no freely downloadable toolbox exists that brings the diversity of the latest supervised machine-learning classification algorithms (MLCAs) together into a single intuitive user-friendly graphical user interface (GUI). To fill this gap and to facilitate and automate the usage of MLCAs, here we present a novel GUI software package that allows systematically training, validating, and applying pixel-based MLCA models to remote sensing imagery. The so-called MLCA toolbox has been integrated within ARTMO’s software framework developed in Matlab which implements most of the state-of-the-art methods in the machine learning community. To demonstrate its utility, we chose a heterogeneous case study scene, a landscape in Southwest Iran to map PTs. In this area, four main PTs were identified, consisting of shrub land, grass land, semi-shrub land, and shrub land–grass land vegetation. Having developed 21 MLCAs using the same training and validation, datasets led to varying accuracy results. Gaussian process classifier (GPC) was validated as the top-performing classifier, with an overall accuracy (OA) of 90%. GPC follows a Laplace approximation to the Gaussian likelihood under the supervised classification framework, emerging as a very competitive alternative to common MLCAs. Random forests resulted in the second-best performance with an OA of 86%. Two other types of ensemble-learning algorithms, i.e., tree-ensemble learning (bagging) and decision tree (with error-correcting output codes), yielded an OA of 83% and 82%, respectively. Following, thirteen classifiers reported OA between 70% and 80%, and the remaining four classifiers reported an OA below 70%. We conclude that GPC substantially outperformed all classifiers, and thus, provides enormous potential for the classification of a diversity of land-cover types. In addition, its probabilistic formulation provides valuable band ranking information, as well as associated predictive variance at a pixel level. Nevertheless, as these are supervised (data-driven) classifiers, performances depend on the entered training data, meaning that an assessment of all MLCAs is crucial for any application. Our analysis demonstrated the efficacy of ARTMO’s MLCA toolbox for an automated evaluation of the classifiers and subsequent thematic mapping.

https://www.mdpi.com/2072-4292/14/18/4452/pdf?version=1662540848

Introducing ARTMO’s Machine-Learning Classification Algorithms Toolbox: Application to Plant-Type Detection in a Semi-Steppe Iranian Landscape

Machine learning (ML) methods and neural networks (NN) are widely implemented for crop types recognition and classification based on satellite images. However, most of these studies use several multi-temporal images which could be inapplicable for cloudy regions. We present a comparison between the classical ML approaches and U-Net NN for classifying crops with a single satellite image. The results show the advantages of using field-wise classification over pixel-wise approach. We first used a Bayesian aggregation for field-wise classification and improved on 1.5% results between majority voting aggregation. The best result for single satellite image crop classification is achieved for gradient boosting with an overall accuracy of 77.4% and macro F1-score 0.66.

Bayesian aggregation improves traditional single image crop classification approaches

Currently, remote sensing crop identification is mostly based on all available images acquired throughout crop growth. However, the available image and data resources in the early growth stage are limited, which makes early crop identification challenging. Different crop types have different phenological characteristics and seasonal rhythm characteristics, and their growth rates are different at different times. Therefore, making full use of crop growth characteristics to augment crop growth difference information at different times is key to early crop identification. In this study, we first calculated the differential features between different periods as new features based on images acquired during the early growth stage. Secondly, multi-temporal difference features of each period were constructed by combination, then a feature optimization method was used to obtain the optimal feature set of all possible combinations in different periods and the early key identification characteristics of different crops, as well as their stage change characteristics, were explored. Finally, the performance of classification and regression tree (Cart), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), and Support Vector Machine (SVM) classifiers in recognizing crops in different periods were analyzed. The results show that: (1) There were key differences between different crops, with rice changing significantly in period F, corn changing significantly in periods E, M, L, and H, and soybean changing significantly in periods E, M, N, and H. (2) For the early identification of rice, the land surface water index (LSWI), simple ratio index (SR), B11, and normalized difference tillage index (NDTI) contributed most, while B11, normalized difference red-edge3 (NDRE3), LSWI, the green vegetation index (VIgreen), red-edge spectral index (RESI), and normalized difference red-edge2 (NDRE2) contributed greatly to corn and soybean identification. (3) Rice could be identified as early as 13 May, with PA and UA as high as 95%. Corn and soybeans were identified as early as 7 July, with PA and UA as high as 97% and 94%, respectively. (4) With the addition of more temporal features, recognition accuracy increased. The GBDT and RF performed best in identifying the three crops in the early stage. This study demonstrates the feasibility of using crop growth difference information for early crop recognition, which can provide a new idea for early crop recognition.

/pdf/investigating-the-potential-of-crop-discrimination-in-early-s3lrjn2t.pdf

Investigating the Potential of Crop Discrimination in Early Growing Stage of Change Analysis in Remote Sensing Crop Profiles

Appropriate crop type mapping to monitor and control land management is very important in developing countries. It can be very useful where digital cadaster maps are not available or usage of Remote Sensing (RS) data is not utilized in the process of monitoring and inventory. The main goal of the present research is to compare and assess the importance of optical RS data in crop type classification using medium and high spatial resolution RS imagery in 2018. With this goal, Landsat 8 (L8) and Sentinel-2 (S2) data were acquired over the Tashkent Province between the crop growth period of May and October. In addition, this period is the only possible time for having cloud-free satellite images. The following four indices “Normalized Difference Vegetation Index” (NDVI), “Enhanced Vegetation Index” (EVI), and “Normalized Difference Water Index” (NDWI1 and NDWI2) were calculated using blue, red, near-infrared, shortwave infrared 1, and shortwave infrared 2 bands. Support-Vector-Machine (SVM) and Random Forest (RF) classification methods were used to generate the main crop type maps. As a result, the Overall Accuracy (OA) of all indices was above 84% and the highest OA of 92% was achieved together with EVI-NDVI and the RF method of L8 sensor data. The highest Kappa Accuracy (KA) was found with the RF method of L8 data when EVI (KA of 88%) and EVI-NDVI (KA of 87%) indices were used. A comparison of the classified crop type area with Official State Statistics (OSS) data about sown crops area demonstrated that the smallest absolute weighted average (WA) value difference (0.2 thousand ha) was obtained using EVI-NDVI with RF method and NDVI with SVM method of L8 sensor data. For S2-sensor data, the smallest absolute value difference result (0.1 thousand ha) was obtained using EVI with RF method and 0.4 thousand ha using NDVI with SVM method. Therefore, it can be concluded that the results demonstrate new opportunities in the joint use of Landsat and Sentinel data in the future to capture high temporal resolution during the vegetation growth period for crop type mapping. We believe that the joint use of S2 and L8 data enables the separation of crop types and increases the classification accuracy.

/pdf/irrigated-crop-types-mapping-in-tashkent-province-of-2kfdebo5.pdf

Irrigated Crop Types Mapping in Tashkent Province of Uzbekistan with Remote Sensing-Based Classification Methods

Nationwide and consistent information on agricultural land use forms an important basis for sustainable land management maintaining food security, (agro)biodiversity, and soil fertility, especially as German agriculture has shown high vulnerability to climate change. Sentinel-1 and Sentinel-2 satellite data of the Copernicus program offer time series with temporal, spatial, radiometric, and spectral characteristics that have great potential for mapping and monitoring agricultural crops. This paper presents an approach which synergistically uses these multispectral and Synthetic Aperture Radar (SAR) time series for the classification of 17 crop classes at 10 m spatial resolution for Germany in the year 2018. Input data for the Random Forest (RF) classification are monthly statistics of Sentinel-1 and Sentinel-2 time series. This approach reduces the amount of input data and pre-processing steps while retaining phenological information, which is crucial for crop type discrimination. For training and validation, Land Parcel Identification System (LPIS) data were available covering 15 of the 16 German Federal States. An overall map accuracy of 75.5% was achieved, with class-specific F1-scores above 80% for winter wheat, maize, sugar beet, and rapeseed. By combining optical and SAR data, overall accuracies could be increased by 6% and 9%, respectively, compared to single sensor approaches. While no increase in overall accuracy could be achieved by stratifying the classification in natural landscape regions, the class-wise accuracies for all but the cereal classes could be improved, on average, by 7%. In comparison to census data, the crop areas could be approximated well with, on average, only 1% of deviation in class-specific acreages. Using this streamlined approach, similar accuracies for the most widespread crop types as well as for smaller permanent crop classes were reached as in other Germany-wide crop type studies, indicating its potential for repeated nationwide crop type mapping.

/pdf/mapping-crop-types-of-germany-by-combining-temporal-111h7475.pdf

Martina Wenzl

Papers

Mapping Crop Types of Germany by Combining Temporal Statistical Metrics of Sentinel-1 and Sentinel-2 Time Series with LPIS Data