Earlier work demonstrates the promise of deep-learning-based approaches for point cloud segmentation; however, these approaches need to be improved to be practically useful. To this end, we introduce a new model SqueezeSegV2. With an improved model structure, SqueezeSetV2 is more robust against dropout noises in LiDAR point cloud and therefore achieves significant accuracy improvement. Training models for point cloud segmentation requires large amounts of labeled data, which is expensive to obtain. To sidestep the cost of data collection and annotation, simulators such as GTA-V can be used to create unlimited amounts of labeled, synthetic data. However, due to domain shift, models trained on synthetic data often do not generalize well to the real world. Existing domain-adaptation methods mainly focus on images and most of them cannot be directly applied to point clouds. We address this problem with a domain-adaptation training pipeline consisting of three major components: 1) learned intensity rendering, 2) geodesic correlation alignment, and 3) progressive domain calibration. When trained on real data, our new model exhibits segmentation accuracy improvements of 6.0-8.6% over the original SqueezeSeg. When training our new model on synthetic data using the proposed domain adaptation pipeline, we nearly double test accuracy on real-world data, from 29.0% to 57.4%. Our source code and synthetic dataset are open sourced11https://github.com/xuanyuzhou98/SqueezeSegV2

SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud

The use of remote imagery captured by unmanned aerial vehicles (UAV) has tremendous potential for designing detailed site-specific weed control treatments in early post-emergence, which have not possible previously with conventional airborne or satellite images. A robust and entirely automatic object-based image analysis (OBIA) procedure was developed on a series of UAV images using a six-band multispectral camera (visible and near-infrared range) with the ultimate objective of generating a weed map in an experimental maize field in Spain. The OBIA procedure combines several contextual, hierarchical and object-based features and consists of three consecutive phases: 1) classification of crop rows by application of a dynamic and auto-adaptive classification approach, 2) discrimination of crops and weeds on the basis of their relative positions with reference to the crop rows, and 3) generation of a weed infestation map in a grid structure. The estimation of weed coverage from the image analysis yielded satisfactory results. The relationship of estimated versus observed weed densities had a coefficient of determination of r2=0.89 and a root mean square error of 0.02. A map of three categories of weed coverage was produced with 86% of overall accuracy. In the experimental field, the area free of weeds was 23%, and the area with low weed coverage (<5% weeds) was 47%, which indicated a high potential for reducing herbicide application or other weed operations. The OBIA procedure computes multiple data and statistics derived from the classification outputs, which permits calculation of herbicide requirements and estimation of the overall cost of weed management operations in advance.

Weed Mapping in Early-Season Maize Fields Using Object-Based Analysis of Unmanned Aerial Vehicle (UAV) Images - eScholarship

Earlier work demonstrates the promise of deep-learning-based approaches for point cloud segmentation; however, these approaches need to be improved to be practically useful To this end, we introduce a new model SqueezeSegV2 that is more robust to dropout noise in LiDAR point clouds With improved model structure, training loss, batch normalization and additional input channel, SqueezeSegV2 achieves significant accuracy improvement when trained on real data Training models for point cloud segmentation requires large amounts of labeled point-cloud data, which is expensive to obtain To sidestep the cost of collection and annotation, simulators such as GTA-V can be used to create unlimited amounts of labeled, synthetic data However, due to domain shift, models trained on synthetic data often do not generalize well to the real world We address this problem with a domain-adaptation training pipeline consisting of three major components: 1) learned intensity rendering, 2) geodesic correlation alignment, and 3) progressive domain calibration When trained on real data, our new model exhibits segmentation accuracy improvements of 60-86% over the original SqueezeSeg When training our new model on synthetic data using the proposed domain adaptation pipeline, we nearly double test accuracy on real-world data, from 290% to 574% Our source code and synthetic dataset will be open-sourced

Autonomous vehicles rely on their perception systems to acquire information about their immediate surroundings. It is necessary to detect the presence of other vehicles, pedestrians, and other relevant entities. Safety concerns and the need for accurate estimations have led to the introduction of lidar systems to complement camera- or radar-based perception systems. This article presents a review of state-of-the-art automotive lidar technologies and the perception algorithms used with those technologies. Lidar systems are introduced first by analyzing such a system?s main components, from laser transmitter to beamscanning mechanism. The advantages/disadvantages and the current status of various solutions are introduced and compared. Then, the specific perception pipeline for lidar data processing is detailed from an autonomous vehicle perspective. The model-driven approaches and emerging deep learning (DL) solutions are reviewed. Finally, we provide an overview of the limitations, challenges, and trends for automotive lidars and perception systems.

Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems

Remote sensing from unmanned aircraft systems (UAS) was expected to be an important new technology to assist farmers with precision agriculture, especially crop nutrient management. There a...

https://www.tandfonline.com/doi/pdf/10.1080/01431161.2017.1410300

What good are unmanned aircraft systems for agricultural remote sensing and precision agriculture

The recent activity in the area of autonomous vehicle navigation has initiated a series of reactions that stirred the automobile industry, pushing for the fast commercialization of this technology which, until recently, seemed futuristic. The LiDAR sensor is able to provide a detailed understanding of the environment surrounding the vehicle making it useful in a plethora of autonomous driving scenarios. Segmenting the 3D point cloud that is provided by modern LiDAR sensors, is the first important step towards the situational assessment pipeline that aims for the safety of the passengers. This step needs to provide accurate segmentation of the ground surface and the obstacles in the vehicle's path, and to process each point cloud in real time. The proposed pipeline aims to solve the problem of 3D point cloud segmentation for data received from a LiDAR in a fast and low complexity manner that targets real world applications. The two-step algorithm first extracts the ground surface in an iterative fashion using deterministically assigned seed points, and then clusters the remaining non-ground points taking advantage of the structure of the LiDAR point cloud. Our proposed algorithms outperform similar approaches in running time, while producing similar results and support the validity of this pipeline as a segmentation tool for real world applications.

Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications

3D model processing for high throughput phenotype extraction – the case of corn

Covariance based point cloud descriptors for object detection and recognition

The continuously growing need for increasing the production of food and reducing the degradation of water supplies, has led to the development of several precision agriculture systems over the past decade so as to meet the needs of modern societies. The present study describes a methodology for the detection and characterization of Nitrogen (N) deficiencies in corn fields. Current methods of field surveillance are either completed manually or with the assistance of satellite imaging, which offer infrequent and costly information to the farmers about the state of their fields. The proposed methodology promotes the use of small-scale Unmanned Aerial Vehicles (UAVs) and Computer Vision algorithms that operate with information in the visual (RGB) spectrum. Through this implementation, a lower cost solution for identifying N deficiencies is promoted. We provide extensive results on the use of commercial RGB sensors for delivering the essential information to farmers regarding the condition of their field, targeting the reduction of N fertilizers and the increase of the crop performance. Data is first collected by a UAV that hovers over a stressed area and collects high resolution RGB images at a low altitude. A recommendation algorithm identifies potential segments of the images that are candidates exhibiting N deficiency. Based on the feedback from experts in the area a training set is constructed utilizing the initial suggestions of the recommendation algorithm. Supervised learning methods are then used to characterize crop leaves that exhibit signs of N deficiency. The performance of 84.2% strongly supports the potential of this scheme to identify N-deficient leaves even in the case of images where the unhealthy leaves are heavily occluded by other healthy or stressed leaves.

Automation solutions for the evaluation of plant health in corn fields

In this paper, we introduce a new covariance based feature descriptor to be used on “colored” point clouds gathered by a mobile robot equipped with an RGB-D camera. Although many recent descriptors provide adequate results, there is not yet a clear consensus on how to best tackle “colored” point clouds. We present the notion of a covariance on RGB-D data. Covariances have not only been proven to be successful in image processing, but in other domains as well. Their main advantage is that they provide a compact and flexible description of point clouds. Our work is a first step towards demonstrating the usability of the concept of covariances in conjunction with RGB-D data. Experiments performed on an RGB-D database and compared to previous results show the increased performance of our method.

Dimitris Zermas

Papers

Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications

3D model processing for high throughput phenotype extraction – the case of corn

Covariance based point cloud descriptors for object detection and recognition

Automation solutions for the evaluation of plant health in corn fields

RGB-D object classification using covariance descriptors