「Unreal Engine 4」爆発エフェクトの作り方

This study proposes an intelligent spraying system with semantic segmentation of fruit trees in a pear orchard. A fruit tree detection system was developed using the SegNet model, a semantic segmentation structure. The system is trained with images categorized into five distinct classes. The learned deep learning model performed with an accuracy of 83.79%. Further, we fusion depth data from an RGB-D camera to prevent the tree in the background from being detected. To operate the nozzles, each image captured from the camera is separated lengthwise into quarters and mapped to the nozzles. Then, the nozzle was opened when the area of fruit trees in each zone exceeded 20%. Two types of field experiments were performed in a pear orchard to verify the effectiveness of our system. From the results obtained, we can confirm the satisfactory performance of our deep learning-based intelligent spraying system. It is expected that the introduction of this system to actual farms will signicantly reduce the amount of pesticide used and will make the work environment safer for farmers.

An Intelligent Spraying System with Deep Learning-based Semantic Segmentation of Fruit Trees in Orchards

Towards real-time forest inventory using handheld LiDAR

Forests can be efficiently monitored by automatic semantic segmentation of trees using satellite and/or aerial images. Still, several challenges can make the problem difficult, including the varying spectral signature of different trees, lack of sufficient labelled data, and geometrical occlusions. In this paper, we address the tree segmentation problem using multispectral imagery. While we carry out large-scale experiments on several deep learning architectures using various spectral input combinations, we also attempt to explore whether hand-crafted spectral vegetation indices can improve the performance of deep learning models in the segmentation of trees. Our experiments include benchmarking a variety of multispectral remote sensing image sets, deep semantic segmentation architectures, and various spectral bands as inputs, including a number of hand-crafted spectral vegetation indices. From our large-scale experiments, we draw several useful conclusions. One particularly important conclusion is that, with no additional computation burden, combining different categories of multispectral vegetation indices, such as NVDI, ARVI, and SAVI, within a single three-channel input, and using the state-of-the-art semantic segmentation architectures, tree segmentation accuracy can be improved under certain conditions, compared to using high-resolution visible and/or nearinfrared input. 

/pdf/deep-semantic-segmentation-of-trees-using-multispectral-1835bqra.pdf

Deep Semantic Segmentation of Trees Using Multispectral Images

Currently, there are many simultaneous localization and mapping (SLAM) algorithms for 3-D light detection and ranging (lidar) relying on features such as planes and edges. However, if a robot needs to work in a rubber forest, these simple features can be unstable due to the complexity of the environment. To address this problem, in this article, we propose the first SLAM system to make a deep adaptive adjustment for the particularity of the rubber-tapping robot and the rubber forest environment. In our article, tree trunks are used to construct a set of sparse maps called trunk atlas, which contain environmental information for localization and trunk position for rubber tapping. To ensure the quality of feature extraction, density field correction and multicriteria trunk detection are proposed. Due to the sparsity and stability of the trunk atlas, our SLAM system can ensure real-time performance. Experiments show that our lidar-only SLAM system can achieve nearly the same performance as state-of-the-art inertial-measurement-unit-aided algorithms in terms of runtime and localization accuracy. In addition, our trunk atlas also takes up less memory for storage than other algorithms. More specifically, it takes less than 50 kB to store a map of an area of approximately 5000 m <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> . 

A Forest 3-D Lidar SLAM System for Rubber-Tapping Robot Based on Trunk Center Atlas

This paper presents a pipeline for semantic segmentation of trees into their components. Given a single RGB-D image of a tree, we employ a deep network to predict labels to classify each pixel of the tree into trunk, branches, twigs and leaves. Multiple convolutional neural network architectures to combine the complementary modalities of depth and colour data are investigated. An asynchronous training approach where two networks trained separately on RGB and depth encoded as a 3-channel HHA image are combined using a late fusion architecture with different learning rates performs the best. Training and evaluation are performed on a synthetic dataset of 6 species of broadleaf trees. We further demonstrate the network's generalization capabilities, across various tree species on the synthetic dataset, achieving an accuracy of upto 92.5%. Furthermore, we present a qualitative evaluation of our approach on real-world data.

An Approach for Semantic Segmentation of Tree-like Vegetation

While the variety of applications for Aerial Manipulators (AMs) has increased over the last years, they are mostly limited to push-and-slide tasks. More complex manipulations of dynamic environments are poorly addressed and still require handcrafted designs of hardware, control, and trajectory planning. In this letter we focus on the active manipulation of articulated objects with AMs. We present a novel planning and control approach that allows the AM to execute complex interaction maneuvers with as little as possible priors given by the operator. Our framework combines sampling-based predictive control to generate pose trajectories with an impedance controller for compliant behaviours, applied to a fully-actuated flying platform. The framework leverages a physics engine to simulate the dynamics of the platform and the environment in order to find optimal motions to execute manipulation tasks. Experiments on two selected examples of pulling open a door and of turning a valve show the feasibility of the proposed approach.

/pdf/a-planning-and-control-framework-for-aerial-manipulation-of-lfqnibyx.pdf

A Planning-and-Control Framework for Aerial Manipulation of Articulated Objects

Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry. Additionally, closed-loop affordance inference is found to allow the agent to divide a task into multiple non-continuous motions and recover from failure and unexpected states. Finally, the pipeline is able to perform long-horizon mobile manipulation tasks, i.e. opening and closing an oven, in the real world with high success rates (opening: 71%, closing: 72%).

Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects

In this article, we investigate and deploy sampling-based control techniques for the challenging task of the mobile manipulation of articulated objects. By their nature, manipulation tasks necessitate environment interactions, which require the handling of nondifferentiable switching contact dynamics. These dynamics represent a strong limitation for traditional gradient-based optimization methods, such as model-predictive control and differential dynamic programming, which often rely on heuristics for trajectory generation. Sampling-based techniques alleviate these constraints but do not ensure robots' stability and input/state constraints either. On the other hand, real-world applications in human environments require safety and robustness to unexpected events. For this reason, we propose a novel framework for safe robotic manipulation of movable articulated objects. The framework combines sampling-based control together with control barrier functions and passivity theory that, thanks to formal stability guarantees, enhance the safety and robustness of the method. We also provide the practical insights that enable robust deployment of stochastic control using a conventional central processing unit. We deploy the algorithm on a ten-degree-of-freedom mobile manipulator robot. Finally, we open source our generic and multithreaded implementation.

Giuseppe Maria Rizzi

Papers

An Approach for Semantic Segmentation of Tree-like Vegetation

A Planning-and-Control Framework for Aerial Manipulation of Articulated Objects

Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects

Robust Sampling-Based Control of Mobile Manipulators for Interaction With Articulated Objects