Our research concerned the development of an autonomous robotic navigation system for orchard harvesting with a dual master-slave mode, the autonomous navigation tractor orchard transport robot being the master followed by a navigation orchard picking robot as the slave. This addresses the problem that in single master-slave navigation mode, agricultural combined harvesting equipment cannot stop repeatedly between rows of apple trees and drive continuously when turning. According to distances obtained from a global positioning system (GNSS), ground points were used to switch the navigation mode of the transport and picking robot. A cloth simulation filter (CSF) and random sample consensus (RANSAC) algorithm was used to obtain inter-row waypoints. The GNSS point was manually selected as the turn waypoint of the master and a kinematic model was used to compute the turn waypoints of the slave. Finally, we used a pure pursuit algorithm to track these waypoints sequentially to achieve master-slave navigation and ground head master-slave command navigation. The experimental results show that the data packet loss rate was less than 1.2% when the robot communicated in the orchard row within 50 m which meets the robot orchard communication requirements. The master-slave robot can achieve repeated stops in the row using follow navigation, which meets the demands of joint orchard harvesting. The maximum, minimum, mean and standard deviation of position deviation of the master robot were 5.3 cm, 0.8 cm, 2.4 cm, and 0.9 cm, respectively. The position deviations of the slave robot were larger than those of the master robot, with maximum, minimum, mean and standard deviation of 39.7 cm, 1.1 cm, 4.1 cm, and 5.6 cm, respectively. The maximum, minimum, mean and standard deviation of the following error between the master-slave robot were 4.4 cm, 0 cm, 1.3 cm, and 1 cm respectively. Concerning the ground head turn, the command navigation method allowed continuous turning, but the lateral deviation between robots was more than 0.3 m and less than 1 m, and the heading deviation was more than 10° and less than 90°. 

/pdf/development-of-a-combined-orchard-harvesting-robot-2y6u9ytq.pdf

Development of a Combined Orchard Harvesting Robot Navigation System

Our research concerned the development of an autonomous robotic navigation system for orchard harvesting with a dual master-slave mode, the autonomous navigation tractor orchard transport robot being the master followed by a navigation orchard picking robot as the slave. This addresses the problem that in single master-slave navigation mode, agricultural combined harvesting equipment cannot stop repeatedly between rows of apple trees and drive continuously when turning. According to distances obtained from a global positioning system (GNSS), ground points were used to switch the navigation mode of the transport and picking robot. A cloth simulation filter (CSF) and random sample consensus (RANSAC) algorithm was used to obtain inter-row waypoints. The GNSS point was manually selected as the turn waypoint of the master and a kinematic model was used to compute the turn waypoints of the slave. Finally, we used a pure pursuit algorithm to track these waypoints sequentially to achieve master-slave navigation and ground head master-slave command navigation. The experimental results show that the data packet loss rate was less than 1.2% when the robot communicated in the orchard row within 50 m which meets the robot orchard communication requirements. The master-slave robot can achieve repeated stops in the row using follow navigation, which meets the demands of joint orchard harvesting. The maximum, minimum, mean and standard deviation of position deviation of the master robot were 5.3 cm, 0.8 cm, 2.4 cm, and 0.9 cm, respectively. The position deviations of the slave robot were larger than those of the master robot, with maximum, minimum, mean and standard deviation of 39.7 cm, 1.1 cm, 4.1 cm, and 5.6 cm, respectively. The maximum, minimum, mean and standard deviation of the following error between the master-slave robot were 4.4 cm, 0 cm, 1.3 cm, and 1 cm respectively. Concerning the ground head turn, the command navigation method allowed continuous turning, but the lateral deviation between robots was more than 0.3 m and less than 1 m, and the heading deviation was more than 10° and less than 90°.

/pdf/development-of-a-combined-orchard-harvesting-robot-29od4m9e.pdf

Abstract The cultivation of meadow orchards provides an ecological benefit for biodiversity, which is significantly higher than in intensively cultivated orchards. However, the maintenance of meadow orchards is not economically profitable. The use of automation for pruning would reduce labour costs and avoid accidents. The goal of this research was, using photogrammetric point clouds, to automatically calculate tree models, without additional human input, as basis to estimate pruning points for meadow orchard trees. Pruning estimates require a knowledge of the major tree structure, containing the branch position, the growth direction and their topological connection. Therefore, nine apple trees were captured photogrammetrically as 3D point clouds using an RGB camera. To extract the tree models, the point clouds got filtered with a random forest algorithm, the trunk was extracted and the resulting point clouds were divided into numerous K-means clusters. The cluster centres were used to create skeleton models using methods of graph theory. For evaluation, the nodes and edges of the calculated and the manually created reference tree models were compared. The calculated models achieved a producer’s accuracy of 73.67% and a user's accuracy of 74.30% of the compared edges. These models now contain the geometric and topological structure of the trees and an assignment of their point clouds, from which further information, such as branch thickness, can be derived on a branch-specific basis. This is necessary information for the calculation of pruning areas and for the actual pruning planning, needed for the automation of tree pruning. 

/pdf/approach-for-graph-based-individual-branch-modelling-of-1jhlrv24.pdf

Approach for graph-based individual branch modelling of meadow orchard trees with 3D point clouds

Preventive maintenance of power lines, including cutting and pruning of tree branches, is essential to avoid interruptions in the energy supply. Automatic methods can support this risky task and also reduce time-consuming. Here, we propose a method in which the orientation and the grasping positions of tree branches are estimated. The proposed method firstly predicts the straight line (representing the tree branch extension) based on a convolutional neural network (CNN). Secondly, a Hough transform is applied to estimate the direction and position of the line. Finally, we estimate the grip point as the pixel point with the highest probability of belonging to the line. We generated a dataset based on internet searches and annotated 1868 images considering challenging scenarios with different tree branch shapes, capture devices, and environmental conditions. Ten-fold cross-validation was adopted, considering 90% for training and 10% for testing. We also assessed the method under corruptions (gaussian and shot) with different severity levels. The experimental analysis showed the effectiveness of the proposed method reporting F1-score of 96.78%. Our method outperformed state-of-the-art Deep Hough Transform (DHT) and Fully Convolutional Line Parsing (F-Clip). 

Line-based deep learning method for tree branch detection from digital images

Computer vision technology is moving more and more towards a three-dimensional approach, and plant phenotyping is following this trend. However, despite its potential, the complexity of the analysis of 3D representations has been the main bottleneck hindering the wider deployment of 3D plant phenotyping. In this review we provide an overview of typical steps for the processing and analysis of 3D representations of plants, to offer potential users of 3D phenotyping a first gateway into its application, and to stimulate its further development. We focus on plant phenotyping applications where the goal is to measure characteristics of single plants or crop canopies on a small scale in research settings, as opposed to large scale crop monitoring in the field. 

/pdf/how-to-make-sense-of-3d-representations-for-plant-1x55vxeu.pdf

How to make sense of 3D representations for plant phenotyping: a compendium of processing and analysis techniques

88. Approach for modeling single branches of meadow orchard trees with 3D point clouds

The cultivation of orchard meadows provides an ecological benefit for biodiversity, which is significantly higher than in intensively cultivated orchards. The goal of this research is to create a tree model to automatically determine possible pruning points for stand-alone trees within meadows. The algorithm which is presented here is capable of building a skeleton model based on a pre-segmented photogrammetric 3D point cloud. Good results were achieved in assigning the points to their leading branches and building a virtual tree model, reaching an overall accuracy of 95.19 %. This model provided the necessary information about the geometry of the tree for automated pruning.

Jonas Straub

Papers

88. Approach for modeling single branches of meadow orchard trees with 3D point clouds

Approach for modeling single branches of meadow orchard trees with 3D point clouds