Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
Towaki Takikawa,Joey Litalien,Kangxue Yin,Karsten Kreis,Charles Loop,Derek Nowrouzezahrai,Alec Jacobson,Morgan McGuire,Sanja Fidler +8 more
- pp 11358-11367
TLDR
In this paper, an octree-based feature volume is used to adaptively fit shapes with multiple discrete levels of detail (LODs), and enables continuous LOD with SDF interpolation.Abstract:
Neural signed distance functions (SDFs) are emerging as an effective representation for 3D shapes. State-of-the-art methods typically encode the SDF with a large, fixed-size neural network to approximate complex shapes with implicit surfaces. Rendering with these large networks is, however, computationally expensive since it requires many forward passes through the network for every pixel, making these representations impractical for real-time graphics. We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality. We represent implicit surfaces using an octree-based feature volume which adaptively fits shapes with multiple discrete levels of detail (LODs), and enables continuous LOD with SDF interpolation. We further develop an efficient algorithm to directly render our novel neural SDF representation in real-time by querying only the necessary LODs with sparse octree traversal. We show that our representation is 2–3 orders of magnitude more efficient in terms of rendering speed compared to previous works. Furthermore, it produces state-of-the-art reconstruction quality for complex shapes under both 3D geometric and 2D image-space metrics.read more
Citations
More filters
Journal ArticleDOI
Instant neural graphics primitives with a multiresolution hash encoding
TL;DR: A versatile new input encoding that permits the use of a smaller network without sacrificing quality, thus significantly reducing the number of floating point and memory access operations is introduced, enabling training of high-quality neural graphics primitives in a matter of seconds, and rendering in tens of milliseconds at a resolution of 1920×1080.
Proceedings ArticleDOI
Block-NeRF: Scalable Large Scene Neural View Synthesis
Matthew Tancik,Vincent Casser,Xinchen Yan,Sabeek Pradhan,Ben Mildenhall,Pratul P. Srinivasan,Jonathan T. Barron,Henrik Kretzschmar +7 more
TL;DR: It is demonstrated that when scaling NeRF to render city-scale scenes spanning multiple blocks, it is vital to de-compose the scene into individually trained NeRFs, which decouples rendering time from scene size, enables rendering to scale to arbitrarily large environments, and allows per-block updates of the environment.
Posted Content
PlenOctrees for Real-time Rendering of Neural Radiance Fields
TL;DR: In this article, an octree-based 3D representation is proposed for real-time rendering of neural radiance fields (NeRFs), which can render 800x800 images at more than 150 FPS.
Journal ArticleDOI
Magic3D: High-Resolution Text-to-3D Content Creation
Chen-Hsuan Lin,Jun Gao,Luming Tang,Towaki Takikawa,Xiaohui Zeng,Xun Huang,Karsten Kreis,Sanja Fidler,Mingyao Liu,Tsung-Yi Lin +9 more
TL;DR: Zhang et al. as mentioned in this paper proposed a two-stage optimization framework to obtain a coarse model using a low-resolution diffusion prior and accelerate with a sparse 3D hash grid structure.
Proceedings ArticleDOI
Decomposing NeRF for Editing via Feature Field Distillation
TL;DR: The authors propose to distill the knowledge of off-the-shelf, self-supervised 2D image feature extractors such as CLIP-LSeg or DINO into a 3D feature field optimized in parallel to the radiance field.
References
More filters
Proceedings ArticleDOI
Marching cubes: A high resolution 3D surface construction algorithm
TL;DR: In this paper, a divide-and-conquer approach is used to generate inter-slice connectivity, and then a case table is created to define triangle topology using linear interpolation.
Proceedings Article
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke,Sam Gross,Francisco Massa,Adam Lerer,James Bradbury,Gregory Chanan,Trevor Killeen,Zeming Lin,Natalia Gimelshein,Luca Antiga,Alban Desmaison,Andreas Kopf,Edward Z. Yang,Zachary DeVito,Martin Raison,Alykhan Tejani,Sasank Chilamkurthy,Benoit Steiner,Lu Fang,Junjie Bai,Soumith Chintala +20 more
TL;DR: This paper details the principles that drove the implementation of PyTorch and how they are reflected in its architecture, and explains how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.
Posted Content
ShapeNet: An Information-Rich 3D Model Repository
Angel X. Chang,Thomas Funkhouser,Leonidas J. Guibas,Pat Hanrahan,Qixing Huang,Zimo Li,Silvio Savarese,Manolis Savva,Shuran Song,Hao Su,Jianxiong Xiao,Li Yi,Fisher Yu +12 more
TL;DR: ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy, a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations.
Proceedings ArticleDOI
Surface simplification using quadric error metrics
Michael Garland,Paul S. Heckbert +1 more
TL;DR: This work has developed a surface simplification algorithm which can rapidly produce high quality approximations of polygonal models, and which also supports non-manifold surface models.
Proceedings ArticleDOI
Progressive meshes
TL;DR: The progressive mesh (PM) representation is introduced, a new scheme for storing and transmitting arbitrary triangle meshes that addresses several practical problems in graphics: smooth geomorphing of level-of-detail approximations, progressive transmission, mesh compression, and selective refinement.