Posted Content•
ShapeNet: An Information-Rich 3D Model Repository
TL;DR: ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy, a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations.
Abstract: We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy. It is a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations. Annotations are made available through a public web-based interface to enable data visualization of object attributes, promote data-driven geometric analysis, and provide a large-scale quantitative benchmark for research in computer graphics and vision. At the time of this technical report, ShapeNet has indexed more than 3,000,000 models, 220,000 models out of which are classified into 3,135 categories (WordNet synsets). In this report we describe the ShapeNet effort as a whole, provide details for all currently available datasets, and summarize future plans.
Citations
More filters
Posted Content•
TL;DR: A hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set and proposes novel set learning layers to adaptively combine features from multiple scales to learn deep point set features efficiently and robustly.
Abstract: Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.
4,802 citations
TL;DR: This work proposes a new neural network module suitable for CNN-based high-level tasks on point clouds, including classification and segmentation called EdgeConv, which acts on graphs dynamically computed in each layer of the network.
Abstract: Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS.
3,727 citations
Posted Content•
TL;DR: This work describes how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrates results that outperform prior work on neural rendering and view synthesis.
Abstract: We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location $(x,y,z)$ and viewing direction $(\theta, \phi)$) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis. View synthesis results are best viewed as videos, so we urge readers to view our supplementary video for convincing comparisons.
2,435 citations
Cites background from "ShapeNet: An Information-Rich 3D Mo..."
...s [15,32] or occupancy elds [11,27]. However, these models are limited by their requirement of access to ground truth 3D geometry, typically obtained from synthetic 3D shape datasets such as ShapeNet [3]. Subsequent work has relaxed this requirement of ground truth 3D shapes by formulating dierentiable rendering functions that allow neural implicit shape representations to be optimized using only 2D...
[...]
Posted Content•
TL;DR: PyTorch Geometric is introduced, a library for deep learning on irregularly structured input data such as graphs, point clouds and manifolds, built upon PyTorch, and a comprehensive comparative study of the implemented methods in homogeneous evaluation scenarios is performed.
Abstract: We introduce PyTorch Geometric, a library for deep learning on irregularly structured input data such as graphs, point clouds and manifolds, built upon PyTorch. In addition to general graph data structures and processing methods, it contains a variety of recently published methods from the domains of relational learning and 3D data processing. PyTorch Geometric achieves high data throughput by leveraging sparse GPU acceleration, by providing dedicated CUDA kernels and by introducing efficient mini-batch handling for input examples of different size. In this work, we present the library in detail and perform a comprehensive comparative study of the implemented methods in homogeneous evaluation scenarios.
2,308 citations
Cites methods from "ShapeNet: An Information-Rich 3D Mo..."
...In addition, we provide embedded datasets like MNIST superpixels (Monti et al., 2017), FAUST (Bogo et al., 2014), ModelNet10/40 (Wu et al., 2015), ShapeNet (Chang et al., 2015), COMA (Ranjan et al., 2018), and the PCPNet dataset from Guerrero et al. (2018)....
[...]
..., 2015), ShapeNet (Chang et al., 2015), COMA (Ranjan et al....
[...]
27 Jun 2016
TL;DR: In this paper, two distinct network architectures of volumetric CNNs and multi-view CNNs are introduced, where they introduce multiresolution filtering in 3D. And they provide extensive experiments designed to evaluate underlying design choices.
Abstract: 3D shape models are becoming widely available and easier to capture, making available 3D information crucial for progress in object classification. Current state-of-theart methods rely on CNNs to address this problem. Recently, we witness two types of CNNs being developed: CNNs based upon volumetric representations versus CNNs based upon multi-view representations. Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations. In this paper, we aim to improve both volumetric CNNs and multi-view CNNs according to extensive analysis of existing approaches. To this end, we introduce two distinct network architectures of volumetric CNNs. In addition, we examine multi-view CNNs, where we introduce multiresolution filtering in 3D. Overall, we are able to outperform current state-of-the-art methods for both volumetric CNNs and multi-view CNNs. We provide extensive experiments designed to evaluate underlying design choices, thus providing a better understanding of the space of methods available for object classification on 3D data.
1,488 citations
References
More filters
Proceedings Article•
01 Dec 2012TL;DR: The Toyohashi Shape Benchmark (TSB), a publicly available new database of polygonal models collected from the World Wide Web, consisting of 10,000 models, is described as the largest 3D shape models to the authors' knowledge used for benchmark testing.
Abstract: In this paper, we describe the Toyohashi Shape Benchmark (TSB), a publicly available new database of polygonal models collected from the World Wide Web, consisting of 10,000 models, as the largest 3D shape models to our knowledge used for benchmark testing. TSB includes 352 categories with labels. It can be used for both 3D shape retrieval and 3D shape classification.
53 citations
"ShapeNet: An Information-Rich 3D Mo..." refers methods in this paper
...Source datasets from SHREC 2014: Princeton Shape Benchmark (PSB) [27], SHREC 2012 generic Shape Benchmark (SHREC12GTB) [16], Toyohashi Shape Benchmark (TSB) [29], Konstanz 3D Model Benchmark (CCCC) [32], Watertight Model Benchmark (WMB) [31], McGill 3D Shape Benchmark (MSB) [37], Bonn Architecture Benchmark (BAB) [33], Purdue Engineering Shape Benchmark (ESB) [9]....
[...]
Proceedings Article•
12 Jun 2014
TL;DR: This track is based on a new comprehensive 3D shape database that contains different types of models, such as generic, articulated, CAD and architecture models, which contains 8,987 triangle meshes that are classified into 171 categories.
Abstract: The objective of this track is to evaluate the performance of 3D shape retriev al approaches on a large-sale comprehensive 3D shape database that contains different types of models,such as generic, articulated, CAD and architecture models. The track is based on a new comprehensive 3D sha pe benchmark, which contains 8,987 triangle meshes that are classified into 171 categories. The benchmark w as compiled as a superset of existing benchmarks and presents a new challenge to retrieval methods as it com prises generic models as well as domainspecific model types. In this track, 14 runs have been submitted by 5 gro ups and their retrieval accuracies were evaluated using 7 commonly used performance metrics.
29 citations
"ShapeNet: An Information-Rich 3D Mo..." refers background in this paper
...However, those datasets are very small — the most recent SHREC iteration in 2014 [17] contains a “large” dataset with around 9,000 models consisting of models from a variety of sources organized into 171 categories (Table 1)....
[...]
29 Mar 2009
TL;DR: This work presents a freely downloadable shape benchmark especially designed for architectural 3D models, which currently contains 2257 objects from various content providers, including companies specialized on 3D CAD applications.
Abstract: When drafting new buildings, architects make intensive use of existing 3D models including building elements, furnishing, and environment elements. These models are either directly included into the draft or serve as a source for inspiration. To allow efficient reuse of existing 3D models, shape retrieval methods considering the specific requirements of architects must be developed. Unfortunately, common 3D shape benchmarks which are used to evaluate the performance of retrieval algorithms are not well suited for architectural data. First, they incorporate models which are not related to this domain, and second and even more important, the provided classification schemes usually do not match an architect's intuition regarding their notion of design and function. To overcome these drawbacks, we present a freely downloadable shape benchmark especially designed for architectural 3D models. It currently contains 2257 objects from various content providers, including companies specialized on 3D CAD applications. All models are classified according to a scheme developed in close cooperation with architects taking into account their specific requirements regarding design and function. Additionally, we show retrieval results for this benchmark using unsupervised and supervised shape retrieval methods and discuss the specific problems regarding retrieval of architectural 3D models.
28 citations
"ShapeNet: An Information-Rich 3D Mo..." refers methods in this paper
...Source datasets from SHREC 2014: Princeton Shape Benchmark (PSB) [27], SHREC 2012 generic Shape Benchmark (SHREC12GTB) [16], Toyohashi Shape Benchmark (TSB) [29], Konstanz 3D Model Benchmark (CCCC) [32], Watertight Model Benchmark (WMB) [31], McGill 3D Shape Benchmark (MSB) [37], Bonn Architecture Benchmark (BAB) [33], Purdue Engineering Shape Benchmark (ESB) [9]....
[...]
24 Nov 2014
TL;DR: This work uses two datasets from online 3D model repositories to evaluate against both human judgments of size and ground truth physical sizes of 3D models, and finds that an algorithmic approach can predict sizes more accurately than people.
Abstract: We address the problem of recovering reliable sizes for a collection of models defined using scales with unknown correspondence to physical units. Our algorithmic approach provides absolute size estimates for 3D models by combining category-based size priors and size observations from 3D scenes. Our approach handles un-observed 3D models without any user intervention. It also scales to large public 3D model databases and is appropriate for handling the open-world problem of rapidly expanding collections of 3D models. We use two datasets from online 3D model repositories to evaluate against both human judgments of size and ground truth physical sizes of 3D models, and find that an algorithmic approach can predict sizes more accurately than people.
26 citations
"ShapeNet: An Information-Rich 3D Mo..." refers methods in this paper
...We estimate the absolute dimensions of models using prior work in size estimation [25], followed by manual verification....
[...]