Accurate and automatic identification of tree species information at the individual tree scale is of great significance for fine-scale investigation and management of forest resources and scientific assessment of forest ecosystems. Despite the fact that numerous studies have been conducted on the delineation of individual tree crown and species classification using drone high-resolution red, green and blue (RGB) images, and Light Detection and Ranging (LiDAR) data, performing the above tasks simultaneously has rarely been explored, especially in complex forest environments. In this study, we improve upon the state of the Mask region-based convolution neural network (Mask R-CNN) with our proposed attention complementary network (ACNet) and edge detection R-CNN (ACE R-CNN) for individual tree species identification in high-density and complex forest environments. First, we propose ACNet as the feature extraction backbone network to fuse the weighted features extracted from RGB images and canopy height model (CHM) data through an attention complementary module, which is able to selectively fuse weighted features extracted from RGB and CHM data at different scales, and enables the network to focus on more effective information. Second, edge loss is added to the loss function to improve the edge accuracy of the segmentation, which is calculated through the edge detection filter introduced in the Mask branch of Mask R-CNN. We demonstrate the performance of ACE R-CNN for individual tree species identification in three experimental areas of different tree species in southern China with precision (P), recall (R), F1-score, and average precision (AP) above 0.9. Our proposed ACNet–the backbone network for feature extraction–has better performance in individual tree species identification compared with the ResNet50-FPN (feature pyramid network). The addition of the edge loss obtained by the Sobel filter further improves the identification accuracy of individual tree species and accelerates the convergence speed of the model training. This work demonstrates the improved performance of ACE R-CNN for individual tree species identification and provides a new solution for tree-level species identification in complex forest environments, which can support carbon stock estimation and biodiversity assessment.

/pdf/ace-r-cnn-an-attention-complementary-and-edge-detection-24g1del8.pdf

ACE R-CNN: An Attention Complementary and Edge Detection-Based Instance Segmentation Algorithm for Individual Tree Species Identification Using UAV RGB Images and LiDAR Data

A novel quantum model of mass function for uncertain information fusion

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Most research on forest tree species classification based on optical image data uses information such as spectral reflectance, vegetation index, texture, and phenology data. However, owing to the limited spectral resolution of multispectral images and the high cost of hyperspectral data, there is room for improvement in the classification of tree species in large areas based on optical images. The combined application of multispectral images and other auxiliary data can provide a new method for improving tree species classification accuracy. Hence, Sentinel-2 images were used to extract spectral reflectance, spectral index, texture, and phenological information. Data for topography, precipitation, air temperature, ultraviolet aerosol index, NO2 concentration, and other variables were included as auxiliary data. Models for forest tree species classification were constructed through feature combination and feature optimization using the random forest (RF), gradient tree boost (GTB), support vector machine (SVM), and classification and regression tree (CART) algorithms. The classification results of 16 feature combinations with the 4 classification methods were compared, and the contributions of different features to the classification models of forest tree species were evaluated. Finally, the optimal classification model was selected to identify the spatial distribution of forest tree species in the study area. The model based on feature optimization gave the best results among the 16 feature combination models. The overall accuracy and kappa coefficient were increased by 18% and 0.21, respectively, compared with the spectral classification model, and by 17% and 0.20, respectively, compared with the spectral and spectral index classification model. By analyzing the feature optimization model, it was found that terrain, ultraviolet aerosol index, and phenological information ranked as the top three features in terms of importance. Although the importance of spectral reflectance and spectral index features was lower, the number of feature variables accounted for a large proportion of the total. The importance of commonly used texture features was limited, and these features were not present in the feature optimization model. The RF algorithm had the highest classification accuracy, with an overall accuracy of 82.69% and a kappa coefficient of 0.80, among the four classification algorithms. The results of GTB were close to those of RF, and the difference in overall classification accuracy was only 0.14%. However, the results of the SVM and CART algorithms were relatively weaker, with overall classification accuracies of about 70%. It can be concluded that the combined application of Sentinel-2 images and auxiliary data can improve forest tree species classification accuracy. The model based on feature optimization achieved the highest classification accuracy among the 16 feature combination models. The spectral reflectance and spectral index data extracted from optical images are useful for tree species classification, but the effect of texture features was very limited. Auxiliary data, such as topographic features, ultraviolet aerosol index, phenological features, NO2 concentration features, topographic diversity features, precipitation features, temperature features, and multi-scale topographic location index data, can effectively improve forest tree species classification accuracy. The RF algorithm had the highest accuracy, and it can be used for tree species classification space distribution identification. The combined application of Sentinel-2 images and auxiliary data can improve classification accuracy, but the highest accuracy of the model was only 82.69%, which leaves room for improvement. Thus, more effective auxiliary data and the vertical structural parameters extracted from satellite LiDAR can be combined with multispectral images to improve forest tree species classification accuracy in future research.

/pdf/forest-tree-species-classification-based-on-sentinel-2-1cy0hekv.pdf

Forest Tree Species Classification Based on Sentinel-2 Images and Auxiliary Data

The goal of this research was to improve wetland classification by fully exploiting multi-source remotely sensed data. Three distinct classifiers were designed to distinguish individual or compound wetland categories using random forest (RF) classification. They were determined, in part, to best use the available remotely sensed features in order to maximize that information and to maximize classification accuracy. The results from these classifiers were integrated according to Dempster–Shafer theory (D–S theory). The developed method was tested on data collected from a study area in Northern Alberta, Canada. The data utilized were Landsat-8 and Sentinel-2 (multi-spectral), Sentinel-1 (synthetic aperture radar—SAR), and digital elevation model (DEM). Classification of fen, bog, marsh, swamps, and upland resulted in an overall accuracy of 0.93 using the proposed methodology, an improvement of 5% when compared to a traditional classification method based on the aggregated features from these data sources. It was noted that, with the traditional method, some pixels were misclassified with a high level of confidence (>85%). Such misclassification was significantly reduced (by ~10%) by the proposed method. Results also showed that some features important in separating compound wetland classes were not considered important using the traditional method based on the RF feature selection mechanism. When used in the proposed method, these features increased the classification accuracy, which demonstrated that the proposed method provided an effective means to fully employ available data to improve wetland classification.

/pdf/an-advanced-data-fusion-method-to-improve-wetland-2wag41du.pdf

An Advanced Data Fusion Method to Improve Wetland Classification Using Multi-Source Remotely Sensed Data

In this study, an object-oriented, decision-level fusion method is proposed for tree species classification based on spectral, textural, and structural features derived from multi-spectral and panchromatic imagery and Light Detection And Ranging (LiDAR) data. Murphy's average method based on the Dempster Shafer theory (DST) was used to calculate the combined mass function for decision making purposes. For individual feature groups, the mass functions were calculated using the support vector machine (SVM) classification method. The species examined included Norway maple, honey locust, Austrian pine, blue spruce, and white spruce. In addition to these species, a two- or three-species compound class was included in the decision process based on the normalized entropy in the presence of conflict that was itself determined according to whether individual groups of features were consistent. The developed method provided a mechanism to identify tree crowns, which could not be classified to one single species with a high confidence due to the conflict among feature groups. Data used in this study were obtained for the Keele Campus of York University, Toronto, Ontario. Among the 223 test crowns, 204 crowns were assigned to one single species, and the overall classification accuracy was 0.89. A decision could not be made for 19 crowns with confidence, and as a result, a two- or three-species compound class was assigned. The classification accuracy was higher than that obtained using SVM classification based on individual and combined spectral, structural, and textural features.

Qian Li

Papers

A decision-level fusion approach to tree species classification from multi-source remotely sensed data