Alvar Vinacua

Bio: Alvar Vinacua is an academic researcher from Polytechnic University of Catalonia. The author has contributed to research in topics: Rendering (computer graphics) & Visualization. The author has an hindex of 13, co-authored 48 publications receiving 491 citations.

Monte Carlo convolution for learning on non-uniformly sampled point clouds

Abstract: Deep learning systems extensively use convolution operations to process input data. Though convolution is clearly defined for structured data such as 2D images or 3D volumes, this is not true for other data types such as sparse point clouds. Previous techniques have developed approximations to convolutions for restricted conditions. Unfortunately, their applicability is limited and cannot be used for general point clouds. We propose an efficient and effective method to learn convolutions for non-uniformly sampled point clouds, as they are obtained with modern acquisition techniques. Learning is enabled by four key novelties: first, representing the convolution kernel itself as a multilayer perceptron; second, phrasing convolution as a Monte Carlo integration problem, third, using this notion to combine information from multiple samplings at different levels; and fourth using Poisson disk sampling as a scalable means of hierarchical point cloud learning. The key idea across all these contributions is to guarantee adequate consideration of the underlying non-uniform sample distribution function from a Monte Carlo perspective. To make the proposed concepts applicable to real-world tasks, we furthermore propose an efficient implementation which significantly reduces the GPU memory required during the training process. By employing our method in hierarchical network architectures we can outperform most of the state-of-the-art networks on established point cloud segmentation, classification and normal estimation benchmarks. Furthermore, in contrast to most existing approaches, we also demonstrate the robustness of our method with respect to sampling variations, even when training with uniformly sampled data only. To support the direct application of these concepts, we provide a ready-to-use TensorFlow implementation of these layers at https://github.com/viscom-ulm/MCCNN.

Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds.

Abstract: Deep learning systems extensively use convolution operations to process input data. Though convolution is clearly defined for structured data such as 2D images or 3D volumes, this is not true for other data types such as sparse point clouds. Previous techniques have developed approximations to convolutions for restricted conditions. Unfortunately, their applicability is limited and cannot be used for general point clouds. We propose an efficient and effective method to learn convolutions for non-uniformly sampled point clouds, as they are obtained with modern acquisition techniques. Learning is enabled by four key novelties: first, representing the convolution kernel itself as a multilayer perceptron; second, phrasing convolution as a Monte Carlo integration problem, third, using this notion to combine information from multiple samplings at different levels; and fourth using Poisson disk sampling as a scalable means of hierarchical point cloud learning. The key idea across all these contributions is to guarantee adequate consideration of the underlying non-uniform sample distribution function from a Monte Carlo perspective. To make the proposed concepts applicable to real-world tasks, we furthermore propose an efficient implementation which significantly reduces the GPU memory required during the training process. By employing our method in hierarchical network architectures we can outperform most of the state-of-the-art networks on established point cloud segmentation, classification and normal estimation benchmarks. Furthermore, in contrast to most existing approaches, we also demonstrate the robustness of our method with respect to sampling variations, even when training with uniformly sampled data only. To support the direct application of these concepts, we provide a ready-to-use TensorFlow implementation of these layers at this https URL

Omni‐directional Relief Impostors

Abstract: Relief impostors have been proposed as a compact and high-quality representation for high-frequency detail in 3D models. In this paper we propose an algorithm to represent a complex object through the combination of a reduced set of relief maps. These relief maps can be rendered with very few artifacts and no apparent deformation from any view direction. We present an efficient algorithm to optimize the set of viewing planes supporting the relief maps, and an image-space metric to select a sufficient subset of relief maps for each view direction. Selected maps (typically three) are rendered based on the well-known ray-height-field intersection algorithm implemented on the GPU. We discuss several strategies to merge overlapping relief maps while minimizing sampling artifacts and to reduce extra texture requirements. We show that our representation can maintain the geometry and the silhouette of a large class of complex shapes with no limit in the viewing direction. Since the rendering cost is output sensitive, our representation can be used to build a hierarchical model of a 3D scene.

Steady affine motions and morphs

Abstract: We propose to measure the quality of an affine motion by its steadiness, which we formulate as the inverse of its Average Relative Acceleration (ARA). Steady affine motions, for which ARA=0, include translations, rotations, screws, and the golden spiral. To facilitate the design of pleasing in-betweening motions that interpolate between an initial and a final pose (affine transformation), B and C, we propose the Steady Affine Morph (SAM), defined as At∘ B with A = C ∘ B-1. A SAM is affine-invariant and reversible. It preserves isometries (i.e., rigidity), similarities, and volume. Its velocity field is stationary both in the global and the local (moving) frames. Given a copy count, n, the series of uniformly sampled poses, Ai/n∘ B, of a SAM form a regular pattern which may be easily controlled by changing B, C, or n, and where consecutive poses are related by the same affinity A1/n. Although a real matrix At does not always exist, we show that it does for a convex and large subset of orientation-preserving affinities A. Our fast and accurate Extraction of Affinity Roots (EAR) algorithm computes At, when it exists, using closed-form expressions in two or in three dimensions. We discuss SAM applications to pattern design and animation and to key-frame interpolation.

Technical Section: User-guided inverse reflector design

Abstract: This paper proposes a technique for the design of luminaire reflector shapes from prescribed optical properties (far-field radiance distribution), geometrical constraints and user's knowledge. This is an important problem in the field of Lighting Engineering, more specifically for Luminaire Design. The reflector's shape to be found is just a part of a set of pieces called in Lighting Engineering an optical set. This is composed of a light bulb (the source), the reflector and usually a glass that acts as a diffusor for the light, and protects the system from dust and other environmental phenomena. Thus, we aim at the design and development of a system capable of generating automatically a reflector shape in a way such that the optical set emits a given, user-defined, far-field radiance distribution for a known bulb. In order to do so, light propagation inside and outside the optical set must be simulated and the resulting radiance distribution compared to the desired one. Constraints on the shape imposed by industry needs and expert's knowledge must be taken into account, bounding the set of possible shapes. The general approach taken is based on a minimization procedure on the space of possible reflector shapes, starting from a user-provided starting shape. The algorithm moves towards minimizing the distance, in the l^2 metric, between the resulting illumination from the reflector and the prescribed, ideal optical radiance distribution specified by the user. The initial shape and a provided confidence value are used during the whole process as a boundary for the space of spanned reflectors used during the simulation.

KPConv: Flexible and Deformable Convolution for Point Clouds

Abstract: We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.

PointConv: Deep Convolutional Networks on 3D Point Clouds

Abstract: Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and the density functions through kernel density estimation. A novel reformulation is proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure.

PCT: Point cloud transformer

Abstract: The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation and normal estimation tasks.

PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding

Abstract: We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. Our dataset consists of 573,585 part instances over 26,671 3D models covering 24 object categories. This dataset enables and serves as a catalyst for many tasks such as shape analysis, dynamic 3D scene modeling and simulation, affordance analysis, and others. Using our dataset, we establish three benchmarking tasks for evaluating 3D part recognition: fine-grained semantic segmentation, hierarchical semantic segmentation, and instance segmentation. We benchmark four state-of-the-art 3D deep learning algorithms for fine-grained semantic segmentation and three baseline methods for hierarchical semantic segmentation. We also propose a baseline method for part instance segmentation and demonstrate its superior performance over existing methods.

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data

Abstract: Deep learning techniques for point cloud data have demonstrated great potentials in solving classical problems in 3D computer vision such as 3D object classification and segmentation. Several recent 3D object classification methods have reported state-of-the-art performance on CAD model datasets such as ModelNet40 with high accuracy (~92\%). Despite such impressive results, in this paper, we argue that object classification is still a challenging task when objects are framed with real-world settings. To prove this, we introduce ScanObjectNN, a new real-world point cloud object dataset based on scanned indoor scene data. From our comprehensive benchmark, we show that our dataset poses great challenges to existing point cloud classification techniques as objects from real-world scans are often cluttered with background and/or are partial due to occlusions. We identify three key open problems for point cloud object classification, and propose new point cloud classification neural networks that achieve state-of-the-art performance on classifying objects with cluttered background. Our dataset and code are publicly available in our project page https://hkust-vgd.github.io/scanobjectnn/.