Deriving individual tree crown (ITC) information from light detection and ranging (LiDAR) data is of great significance to forest resource assessment and smart management. After proof-of-concept studies, advanced deep learning methods have been shown to have high efficiency and accuracy in remote sensing data analysis and geoscience problem solving. This study proposes a novel concept for synergetic use of the YOLO-v4 deep learning network based on heightmaps directly generated from airborne LiDAR data for ITC segmentation and a computer graphics algorithm for refinement of the segmentation results involving overlapping tree crowns. This concept overcomes the limitations experienced by existing ITC segmentation methods that use aerial photographs to obtain texture and crown appearance information and commonly encounter interference due to heterogeneous solar illumination intensities or interlacing branches and leaves. Three generative adversarial networks (WGAN, CycleGAN, and SinGAN) were employed to generate synthetic images. These images were coupled with manually labeled training samples to train the network. Three forest plots, namely, a tree nursery, forest landscape and mixed tree plantation, were used to verify the effectiveness of our approach. The results showed that the overall recall of our method for detecting ITCs in the three forest plot types reached 83.6%, with an overall precision of 81.4%. Compared with reference field measurement data, the coefficient of determination R2 was ≥ 79.93% for tree crown width estimation, and the accuracy of our deep learning method was not influenced by the values of key parameters, yielding 3.9% greater accuracy than the traditional watershed method. The results demonstrate an enhancement of tree crown segmentation in the form of a heightmap for different forest plot types using the concept of deep learning, and our method bypasses the visual complications arising from aerial images featuring diverse textures and unordered scanned points with irregular geometrical properties.

Individual Tree Crown Segmentation and Crown Width Extraction From a Heightmap Derived From Aerial Laser Scanning Data Using a Deep Learning Framework

Accurate and automatic identification of tree species information at the individual tree scale is of great significance for fine-scale investigation and management of forest resources and scientific assessment of forest ecosystems. Despite the fact that numerous studies have been conducted on the delineation of individual tree crown and species classification using drone high-resolution red, green and blue (RGB) images, and Light Detection and Ranging (LiDAR) data, performing the above tasks simultaneously has rarely been explored, especially in complex forest environments. In this study, we improve upon the state of the Mask region-based convolution neural network (Mask R-CNN) with our proposed attention complementary network (ACNet) and edge detection R-CNN (ACE R-CNN) for individual tree species identification in high-density and complex forest environments. First, we propose ACNet as the feature extraction backbone network to fuse the weighted features extracted from RGB images and canopy height model (CHM) data through an attention complementary module, which is able to selectively fuse weighted features extracted from RGB and CHM data at different scales, and enables the network to focus on more effective information. Second, edge loss is added to the loss function to improve the edge accuracy of the segmentation, which is calculated through the edge detection filter introduced in the Mask branch of Mask R-CNN. We demonstrate the performance of ACE R-CNN for individual tree species identification in three experimental areas of different tree species in southern China with precision (P), recall (R), F1-score, and average precision (AP) above 0.9. Our proposed ACNet–the backbone network for feature extraction–has better performance in individual tree species identification compared with the ResNet50-FPN (feature pyramid network). The addition of the edge loss obtained by the Sobel filter further improves the identification accuracy of individual tree species and accelerates the convergence speed of the model training. This work demonstrates the improved performance of ACE R-CNN for individual tree species identification and provides a new solution for tree-level species identification in complex forest environments, which can support carbon stock estimation and biodiversity assessment.

/pdf/ace-r-cnn-an-attention-complementary-and-edge-detection-24g1del8.pdf

ACE R-CNN: An Attention Complementary and Edge Detection-Based Instance Segmentation Algorithm for Individual Tree Species Identification Using UAV RGB Images and LiDAR Data

Accurate and timely information on the number of densely-planted Chinese fir seedlings is essential for their scientific cultivation and intelligent management. However, in the later stage of cultivation, the overlapping of lateral branches among individuals is too severe to identify the entire individual in the UAV image. At the same time, in the high-density planting nursery, the terminal bud of each seedling has a distinctive characteristic of growing upward, which can be used as an identification feature. Still, due to the small size and dense distribution of the terminal buds, the existing recognition algorithm will have a significant error. Therefore, in this study, we proposed a model based on the improved network structure of the latest YOLOv5 algorithm for identifying the terminal bud of Chinese fir seedlings. Firstly, the micro-scale prediction head was added to the original prediction head to enhance the model’s ability to perceive small-sized terminal buds. Secondly, a multi-attention mechanism module composed of Convolutional Block Attention Module (CBAM) and Efficient Channel Attention (ECA) was integrated into the neck of the network to enhance further the model’s ability to focus on key target objects in complex backgrounds. Finally, the methods including data augmentation, Test Time Augmentation (TTA) and Weighted Boxes Fusion (WBF) were used to improve the robustness and generalization of the model for the identification of terminal buds in different growth states. The results showed that, compared with the standard version of YOLOv5, the recognition accuracy of the improved YOLOv5 was significantly increased, with a precision of 95.55%, a recall of 95.84%, an F1-Score of 96.54%, and an mAP of 94.63%. Under the same experimental conditions, compared with other current mainstream algorithms (YOLOv3, Faster R-CNN, and PP-YOLO), the average precision and F1-Score of the improved YOLOv5 also increased by 9.51-28.19 percentage points and 15.92-32.94 percentage points, respectively. Overall, The improved YOLOv5 algorithm integrated with the attention network can accurately identify the terminal buds of densely-planted Chinese fir seedlings in UAV images and provide technical support for large-scale and automated counting and precision cultivation of Chinese fir seedlings.

/pdf/recognition-of-terminal-buds-of-densely-planted-chinese-fir-1th2t5kv.pdf

Recognition of terminal buds of densely-planted Chinese fir seedlings using improved YOLOv5 by integrating attention mechanism

A novel algorithm of individual tree crowns segmentation considering three-dimensional canopy attributes using UAV oblique photos

Oblique photography technology based on UAV (unmanned aerial vehicle) provides an effective means for the rapid, real-scene 3D reconstruction of geographical objects on a watershed scale. However, existing research cannot achieve the automatic and high-precision reconstruction of water regions due to the sensitivity of water surface patterns to wind and waves, reflections of objects on the shore, etc. To solve this problem, a novel rapid reconstruction scheme for water regions in 3D models of oblique photography is proposed in this paper. It extracts the boundaries of water regions firstly using a designed eight-neighborhood traversal algorithm, and then reconstructs the triangulated irregular network (TIN) of water regions. Afterwards, the corresponding texture images of water regions are intelligently selected and processed using a designed method based on coordinate matching, image stitching and clipping. Finally, the processed texture images are mapped to the obtained TIN, and the real information about water regions can be reconstructed, visualized and integrated into the original real-scene 3D environment. Experimental results have shown that the proposed scheme can rapidly and accurately reconstruct water regions in 3D models of oblique photography. The outcome of this work can refine the current technical system of 3D modeling by UAV oblique photography and expand its application in the construction of twin watershed, twin city, etc.

/pdf/a-rapid-water-region-reconstruction-scheme-in-3d-watershed-1vk8utyp.pdf

A Rapid Water Region Reconstruction Scheme in 3D Watershed Scene Generated by UAV Oblique Photography

Chinese fir ( Cunninghamia lanceolate (Lamb.) Hook) individual tree parameters extraction is important for scientific forest management. However, high-precision parameters extraction by field investigation or spaceborne optical remote sensing is difficult when the forest is dense and the terrain is complex. This article proposes a framework for extracting individual tree parameters by combining low-cost and high-efficiency unmanned aerial vehicle-based oblique photogrammetry with manned airborne light detection and ranging data and explores the influence of spatial resolution on the accuracy of parameter extraction. The variable window filtering (VWF) was used to detect an individual tree. The marker-controlled watershed segmentation (MCWS) and seed region growing algorithms were used to delineate the crown. The individual tree detection using VWF based on 1 m resolution achieves precision of 80%. For the crown delineation, it is more accurate based on the 0.25 m resolution using MCWS algorithm with the detection accuracy of 65%. The results show that the proposed framework can effectively detect the tree and delineate the crown under complex terrain conditions and the optimal resolution for different parameter extraction is determined, which has important guiding significance to determine the flight parameters and reduce unnecessary data processing.

/pdf/individual-tree-parameters-estimation-for-chinese-fir-2i0z1mv7x6.pdf

Individual Tree Parameters Estimation for Chinese Fir ( Cunninghamia lanceolate (Lamb.) Hook) Plantations of South China Using UAV Oblique Photography: Possibilities and Challenges

Chinese fir (Cunninghamia lanceolata (Lamb.) Hook) is one of the important tree species in plantation in southern China. Rapid and accurate acquisition of individual tree above-ground biomass (IT-AGB) information is of vital importance for precise monitoring and scientific management of Chinese fir forest resources. Unmanned Aerial Vehicle (UAV) oblique photogrammetry technology can simultaneously obtain high-density point cloud data and high spatial resolution spectral information, which has been a main remote sensing source for obtaining forest fine three-dimensional structure information and provided possibility for estimating IT-AGB. In this study, we proposed a novel approach to estimate IT-AGB by introducing the color space intensity information into a regression-based model that incorporates three-dimensional point cloud and two-dimensional spectrum feature variables, and the accuracy was evaluated using a leave-one-out cross-validation approach. The results demonstrated that the intensity variables derived from the color space were strongly correlated with the IT-AGB and obviously improved the estimation accuracy. The model constructed by the combination of point cloud variables, vegetation index and RGB spatial intensity variables had high accuracy (R2 = 0.79; RMSECV = 44.77 kg; and rRMSECV = 0.25). Comparing the performance of estimating IT-AGB models with different spatial resolution images (0.05, 0.1, 0.2, 0.5 and 1 m), the model was the best at the spatial resolution of 0.2 m, which was significantly better than that of the other four. Moreover, we also divided the individual tree canopy into four directions (East, West, South and North) to develop estimation models respectively. The result showed that the IT-AGB estimation capacity varied significantly in different directions, and the West-model had better performance, with the estimation accuracy of 67%. This study indicates the potential of using oblique photogrammetry technology to estimate AGB at an individual tree scale, which can support carbon stock estimation as well as precision forestry application.

/pdf/estimating-individual-tree-above-ground-biomass-of-chinese-1sikl5gr.pdf

Estimating Individual Tree Above-Ground Biomass of Chinese Fir Plantation: Exploring the Combination of Multi-Dimensional Features from UAV Oblique Photos

The invention discloses a technology for carrying out single-tree biomass mapping in a high-canopy-density artificial forest region based on an inclined picture, is suitable for single-tree parameter extraction of the high-canopy-density artificial forest, and belongs to the technical field of application of inclined photography in forestry, and the key technical points are as follows: 1, adopting a three-dimensional reconstruction point cloud technology based on the picture and a control point; 2, adopting a tree height and crown area extraction technology under a high canopy density artificial forest condition; and 3, establishing a model inversion single-tree biomass technology by using small samples. The key problem to be solved comprises: 1, only small sample investigation is needed, the on-site measurement time is shortened, and the investigation efficiency is improved; 2, the oblique photography camera is carried by the unmanned aerial vehicle, the cost is low and the efficiency is high; and 3, the precision of single tree parameter extraction in the high-canopy-density artificial forest region reaches 80%. The technology is suitable for the artificial forest with higher canopy density, extracts single-tree parameters in the high canopy density artificial forest region based on the inclined picture for the first time, carries out a single-tree biomass mapping technology, and can provide a basic data basis for rapid and accurate investigation work of artificial forest resources.

Tian Yin

Papers

Individual Tree Parameters Estimation for Chinese Fir ( Cunninghamia lanceolate (Lamb.) Hook) Plantations of South China Using UAV Oblique Photography: Possibilities and Challenges

Estimating Individual Tree Above-Ground Biomass of Chinese Fir Plantation: Exploring the Combination of Multi-Dimensional Features from UAV Oblique Photos

A novel algorithm of individual tree crowns segmentation considering three-dimensional canopy attributes using UAV oblique photos

Technology for carrying out single-tree biomass mapping in high-canopy-density artificial forest region based on inclined pictures