Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

This chapter introduces the finite element method (FEM) as a tool for solution of classical electromagnetic problems. Although we discuss the main points in the application of the finite element method to electromagnetic design, including formulation and implementation, those who seek deeper understanding of the finite element method should consult some of the works listed in the bibliography section.

Introduction to the Finite Element Method

This paper deals with local 3D descriptors for surface matching. First, we categorize existing methods into two classes: Signatures and Histograms. Then, by discussion and experiments alike, we point out the key issues of uniqueness and repeatability of the local reference frame. Based on these observations, we formulate a novel comprehensive proposal for surface representation, which encompasses a new unique and repeatable local reference frame as well as a new 3D descriptor. The latter lays at the intersection between Signatures and Histograms, so as to possibly achieve a better balance between descriptiveness and robustness. Experiments on publicly available datasets as well as on range scans obtained with Spacetime Stereo provide a thorough validation of our proposal.

/pdf/unique-signatures-of-histograms-for-local-surface-dbxl9xe6lq.pdf

Unique signatures of histograms for local surface description

We propose an algorithm for semantic segmentation based on 3D point clouds derived from ego-motion. We motivate five simple cues designed to model specific patterns of motion and 3D world structure that vary with object category. We introduce features that project the 3D cues back to the 2D image plane while modeling spatial layout and context. A randomized decision forest combines many such features to achieve a coherent 2D segmentation and recognize the object categories present. Our main contribution is to show how semantic segmentation is possible based solely on motion-derived 3D world structure. Our method works well on sparse, noisy point clouds, and unlike existing approaches, does not need appearance-based descriptors.

Experiments were performed on a challenging new video database containing sequences filmed from a moving car in daylight and at dusk. The results confirm that indeed, accurate segmentation and recognition are possible using only motion and 3D world structure. Further, we show that the motion-derived information complements an existing state-of-the-art appearance-based method, improving both qualitative and quantitative performance.

/pdf/segmentation-and-recognition-using-structure-from-motion-ptapt751hs.pdf

Segmentation and Recognition Using Structure from Motion Point Clouds

Environment models serve as important resources for an autonomous robot by providing it with the necessary task-relevant information about its habitat. Their use enables robots to perform their tasks more reliably, flexibly, and efficiently. As autonomous robotic platforms get more sophisticated manipulation capabilities, they also need more expressive and comprehensive environment models: for manipulation purposes their models have to include the objects present in the world, together with their position, form, and other aspects, as well as an interpretation of these objects with respect to the robot tasks. The dissertation presented in this article (Rusu, PhD thesis, 2009) proposes Semantic 3D Object Models as a novel representation of the robot’s operating environment that satisfies these requirements and shows how these models can be automatically acquired from dense 3D range data.

/pdf/semantic-3d-object-maps-for-everyday-manipulation-in-human-van9xaq8u7.pdf

Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments

In this paper we describe and analyze a method based on local least square fitting for estimating the normals at all sample points of a point cloud data (PCD) set, in the presence of noise. We study the effects of neighborhood size, curvature, sampling density, and noise on the normal estimation when the PCD is sampled from a smooth curve in R2 or a smooth surface in R3 and noise is added. The analysis allows us to find the optimal neighborhood size using other local information from the PCD. Experimental results are also provided.

/pdf/estimating-surface-normals-in-noisy-point-cloud-data-1da45terxt.pdf

Estimating surface normals in noisy point cloud data

For a set S of points in Rd, an s-spanner is a subgraph of the complete graph with node set S such that any pair of points is connected via some path in the spanner whose total length is at most s times the Euclidean distance between the points. In this paper we propose a new sparse (1 + e)-spanner with O(n/ed) edges, where e is a specified parameter. The key property of this spanner is that it can be efficiently maintained under dynamic insertion or deletion of points, as well as under continuous motion of the points in both the kinetic data structures setting and in the more realistic blackbox displacement model we introduce. Our deformable spanner succinctly encodes all proximity information in a deforming point cloud, giving us efficient kinetic algorithms for problems such as the closest pair, the near neighbors of all points, approximate nearest neighbor search (aka approximate Voronoi diagram), well-separated pair decompositions, and approximate k-centers.

Deformable spanners and applications

Zonotopes are centrally symmetric polytopes with a very special structure: they are Minkowski sums of line segments. In this paper we propose to use zonotopes as bounding volumes for geometry in collision detection and other applications where the spatial relationship between two pieces of geometry is important. We show how to construct optimal, or approximately optimal zonotopes enclosing given set of points or other geometry. We also show how zonotopes can be used for efficient collision testing, based on their representation via their defining line segments --- without ever building their explicit description as polytopes. This implicit representation adds flexibility, power, and economy to the use of zonotopes as bounding volumes.

Zonotopes as bounding volumes

For a set S of points in R3, an s-spanner is a graph on S such that any pair of points is connected via some path in the spanner whose total length is at most s times the Euclidean distance between the points. In this paper we propose a new sparse (1+e)-spanner with O(n/ed) edges, where e is a specified parameter. The key property of this spanner is that it can be efficiently maintained under dynamic insertion or deletion ofpoints, as well as under continuous motion of the points in both the kinetic data structures setting and in the more realistic blackbox displacement model we introduce. Our deformable spanner succinctly encodes all proximity information in a deforming point cloud, giving us efficient kinetic algorithms for problems such as the closest pair, the near neighbors of all points, approximate nearest neighbor search (aka approximate Voronoi diagram), well-separated pair decomposition, and approximate k-centers.

/pdf/deformable-spanners-and-applications-4tkdyswqcz.pdf

In this paper, we propose to study deformable necklaces --- flexible chains of balls, called beads, in which only adjacent balls may intersect. Such objects can be used to model macro-molecules, muscles, rope, and other 'linear' objects in the physical world. In this paper, we exploit this linearity to develop geometric structures associated with necklaces that are useful in physical simulations. We show how these structures can be implemented efficiently and maintained under necklace deformation. In particular, we study a bounding volume hierarchy based on spheres built on a necklace. Such a hierarchy is easy to compute and is suitable for maintenance when the necklace deforms, as our theoretical and experimental results show. This hierarchy can be used for collision and self-collision detection. In particular, we achieve an upper bound of O(nlog n) in two dimensions and O(n 2-2/d) in d-dimensions, d 3, for collision checking. To our knowledge, this is the first sub-quadratic bound proved for a collision detection algorithm using predefined hierarchies. In addition, we show that the power diagram, with the help of some additional mechanisms, can be also used to detect self-collisions of a necklace in certain ways complementary to the sphere hierarchy.

/pdf/collision-detection-for-deforming-necklaces-1bxa1pnbh0.pdf

An Nguyen

Papers

Estimating surface normals in noisy point cloud data

Deformable spanners and applications

Zonotopes as bounding volumes

Deformable spanners and applications

Collision detection for deforming necklaces