Deep Mesh Reconstruction From Single RGB Images via Topology Modification Networks
Junyi Pan,Xiaoguang Han,Weikai Chen,Jiapeng Tang,Kui Jia +4 more
- pp 9964-9973
TLDR
This paper presents an end-to-end single-view mesh reconstruction framework that is able to generate high-quality meshes with complex topologies from a single genus-0 template mesh and outperforms the current state-of-the-art methods both qualitatively and quantitatively.Abstract:
Reconstructing the 3D mesh of a general object from a single image is now possible thanks to the latest advances of deep learning technologies. However, due to the nontrivial difficulty of generating a feasible mesh structure, the state-of-the-art approaches often simplify the problem by learning the displacements of a template mesh that deforms it to the target surface. Though reconstructing a 3D shape with complex topology can be achieved by deforming multiple mesh patches, it remains difficult to stitch the results to ensure a high meshing quality. In this paper, we present an end-to-end single-view mesh reconstruction framework that is able to generate high-quality meshes with complex topologies from a single genus-0 template mesh. The key to our approach is a novel progressive shaping framework that alternates between mesh deformation and topology modification. While a deformation network predicts the per-vertex translations that reduce the gap between the reconstructed mesh and the ground truth, a novel topology modification network is employed to prune the error-prone faces, enabling the evolution of topology. By iterating over the two procedures, one can progressively modify the mesh topology while achieving higher reconstruction accuracy. Moreover, a boundary refinement network is designed to refine the boundary conditions to further improve the visual quality of the reconstructed mesh. Extensive experiments demonstrate that our approach outperforms the current state-of-the-art methods both qualitatively and quantitatively, especially for the shapes with complex topologies.read more
Citations
More filters
Proceedings ArticleDOI
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
TL;DR: This work proposes a differentiable rendering formulation for implicit shape and texture representations, showing that depth gradients can be derived analytically using the concept of implicit differentiation, and finds that this method can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.
Posted Content
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
TL;DR: This paper proposes a generative model for radiance fields which have recently proven successful for novel view synthesis of a single scene, and introduces a multi-scale patch-based discriminator to demonstrate synthesis of high-resolution images while training the model from unposed 2D images alone.
Proceedings Article
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
TL;DR: In this paper, a multi-scale patch-based discriminator is proposed to disentangle camera and scene properties. But the model is limited to a coarse discretization of the 3D space.
Proceedings ArticleDOI
Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes From a Single Image
TL;DR: Zhang et al. as mentioned in this paper propose a coarse-to-fine hierarchy with three components: room layout with camera pose, 3D object bounding boxes, and 3D mesh meshes.
Posted Content
Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision
TL;DR: In this article, a differentiable rendering formulation for implicit shape and texture representations is proposed, which can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Journal ArticleDOI
A method for registration of 3-D shapes
Paul J. Besl,H.D. McKay +1 more
TL;DR: In this paper, the authors describe a general-purpose representation-independent method for the accurate and computationally efficient registration of 3D shapes including free-form curves and surfaces, based on the iterative closest point (ICP) algorithm, which requires only a procedure to find the closest point on a geometric entity to a given point.
Book
Multiple view geometry in computer vision
Richard Hartley,Andrew Zisserman +1 more
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Multiple View Geometry in Computer Vision.
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
Proceedings ArticleDOI
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
TL;DR: This paper designs a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input and provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing.