HAL Id: hal-02541349
https://hal.inria.fr/hal-02541349
Submitted on 13 Apr 2020
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Connect-and-Slice: an hybrid approach for
reconstructing 3D objects
Hao Fang, Florent Lafarge
To cite this version:
Hao Fang, Florent Lafarge. Connect-and-Slice: an hybrid approach for reconstructing 3D objects.
CVPR 2020 - IEEE Conference on Computer Vision and Pattern Recognition, Jun 2020, Seattle /
Virtual, United States. �hal-02541349�
Connect-and-Slice: an hybrid approach for reconstructing 3D objects
Hao Fang Florent Lafarge
Universit
´
e C
ˆ
ote d’Azur, Inria
Firstname.Lastname@inria.fr
Abstract
Converting point clouds generated by Laser scanning,
multiview stereo imagery or depth cameras into compact
polygon meshes is a challenging problem in vision. Exist-
ing methods are either robust to imperfect data or scalable,
but rarely both. In this paper, we address this issue with an
hybrid method that successively connects and slices planes
detected from 3D data. The core idea consists in construct-
ing an efficient and compact partitioning data structure.
The later is i) spatially-adaptive in the sense that a plane
slices a restricted number of relevant planes only, and ii)
composed of components with different structural meaning
resulting from a preliminary analysis of the plane connec-
tivity. Our experiments on a variety of objects and sensors
show the versatility of our approach as well as its competi-
tiveness with respect to existing methods.
1. Introduction
Reconstructing surfaces from 3D data is still one of
the biggest challenges in computer vision. Most of exist-
ing methods, as the popular Poisson algorithm [15], are
designed to approximate free-form shapes by dense trian-
gular meshes. When the observed scenes contained geo-
metric regularities, typically man-made objects and urban
environments, these methods do not provide optimal re-
sults in terms of storage, rendering or editing capacity [2].
One prefers representing such scenes by more compact and
structure-aware Computer-Aided Design style models, i.e.
with concise polygon meshes in which each facet corre-
sponds to a large polygon [4].
Reconstructing concise polygon meshes is usually oper-
ated in two steps. First, planar primitives are detected from
input 3D data. A planar primitive is defined as the asso-
ciation of a plane and a subset of input data called inliers
from which the plane has been fitted to. Primitives are dis-
connected from each others and constitute an intermediate
representation between input 3D data and the output mesh.
The second step then consists in assembling primitives into
a surface mesh. This step constitutes the difficult part of
the problem. One strategy is to connect the primitives us-
ing proximity and structural considerations [1, 8, 17, 29].
Despite being fast and scalable, this solution is not robust
to defect-laden data, in particular when primitives are over-
or under-detected or when erroneous connections between
primitives exist. A more robust strategy consists in slicing
a 3D domain by extending the primitives. This leads to the
creation of a partition of polyhedral cells or polygonal facets
[5, 7, 24, 34]. The surface is then extracted by labeling the
cells as inside or outside the surface, or equivalently, by se-
lecting facets to be part of the surface. Because each prim-
itive exhaustively slices all the others, this solution is more
robust to defect-laden data than the first strategy. However,
its main shortcoming is the computational burden for slicing
the primitives into a partition of atomic surface and volume
elements, with typically unreasonable timing and memory
issues when more than one hundred primitives are handled.
None of these two strategies are both robust and scalable.
We address this problem with an hybrid method that suc-
cessively connects and slices primitives in a robust and scal-
able manner. The method is built on three important techni-
cal ingredients.
First, our algorithm has a preliminary step that ana-
lyzes the connectivity of primitives in order to search for
structurally-valid surface components. This allows us to
quickly process a part of the input primitives and solve ob-
vious primitive assembling situations.
Second, we construct a more flexible and lighter partition
data structure than the slicing methods. Our data structure
is spatially-adaptive in the sense that a primitive slices a re-
stricted number of relevant primitives based on spatial prox-
imity considerations. Moreover, its atomic elements have
different structural meanings that will guide the extraction
of the output surface.
Third, we propose a surface extraction mechanism that
operates from such an irregular data-structure in which cells
are not necessarily convex and free of intersection with
other cells. In particular, we do not measure data fidelity
to input 3D data directly, but instead to primitives: it allows
output models not to suffer from artifacts frequently found
with existing methods.
1
We demonstrate the potential of our algorithm in terms
of flexibility, robustness and scalability on different types of
objects, going from buildings to mechanical pieces through
even free-form shapes. In particular, we show our algorithm
is faster and more scalable than state-of-the-art methods by
a significant margin.
2. Related works
Primitive detection. Detecting geometric primitives
from 3D data is an instance of the general problem of fitting
mathematical models to data. Region growing [21, 28]
and Ransac [31] mechanisms constitute the most popular
methods for detecting planes from 3D data. Some recent
works detect and regularize primitives according to geo-
metric relationships such as parallelism or orthogonality,
either sequentially [20, 25] or simultaneously [22, 26].
Such regularities typically allow the complexity of the
subsequent assembling step to be reduced. Other recent
methods also allow the extraction of geometric primitives
at key abstraction levels of the observed objects [12].
Although these methods work well in practice, primitive
detection remains an ill-posed problem with no guarantee
that the output configurations adequately describe the
observed object [9].
Connectivity-based methods. Analyzing a connec-
tivity graph to detect and link points intersecting plane
triples [8, 29, 33] usually works well when the correct
connectivity between primitives can be recovered. To be
robust to challenging data, one interactive solution is to
automatically snap primitives when the connectivity is
obvious, and let the user complete the output surface for the
conflicting situations [1]. Another solution consists in mix-
ing polyhedral surface components with flexible free-form
patches [17, 16]. Such a representation however does not
offer the level of compactness and simplicity of CAD style
models. Despite being fast, connectivity-based methods
suffer from a lack of robustness to defect-laden data, in
particular to over- and under-detection of primitives and
erroneous connections between primitives. Our approach
exploits some principles of these methods as a preliminary
step to quickly solve obvious plane assembling situations
and lighten the time-consuming slicing operations.
Slicing-based methods. The core of these methods con-
sists in partitioning a 3D domain by extending primitives.
The partitioning data-structure is typically a 3D tesselation
of polyhedral cells, which are themselves composed of
polygonal facets. The output surface is then extracted
by selecting a subset of facets from the tesselation. Be-
cause each primitive naively slices all the others, such a
data-structure, also called a plane arrangement [23], is
particularly dense and time-consuming to compute. Some
methods decompose the slicing operations into spatial
blocks [7, 5]. Such piecewise partitions increase scalability
by a good margin. However it creates geometric artifacts
as blocks often do not align well with data. These methods
also add artificial primitives along vertical and horizontal
axes in the partition to be more robust to missing primitives,
assuming the observed object aligns with these arbitrary
directions. A discrete partitioning [30, 34] that avoids
computing the exact geometry of the whole partition is a
less costly option, but also engenders geometric artifacts
when the discretization is not fine enough. Another
possible solution consists in filtering and simplifying the
input set of primitives to remove redundant planes and
reduce the computational burden of the slicing operations
[24]. Although these methods offer a good robustness to
imperfect configurations of primitives, they do not scale
well. Our approach proposes two key ingredients to solve
this issue: a new light and spatially-adaptive partitioning
data-structure and a preliminary connectivity analysis that
reduces the number of primitives to be processed during
slicing operations.
Methods with geometric assumptions. Some works
also exploit strong geometric assumptions. The Manhattan-
World assumption [10] enforces planes to follow only three
orthogonal directions. This assumption reduces both the ge-
ometry of output 3D models and the solution space to ex-
plore. Such an assumption is interesting for modeling build-
ings [19] and approximating shapes very coarsely [14]. An-
other frequent geometric assumption is to restrict the output
surface to a disk-topology with a 2.5D view-dependent rep-
resentation. This is well adapted to reconstruct buildings
from airborne data [35, 36, 27, 18], facades from street-side
data [3], and indoor scenes from images [6]. Although these
assumptions efficiently reduce the solution space in general,
their use is restricted to specific applications. To the con-
trary, our approach remains generic.
3. Overview
Our algorithm takes as input a point cloud. It can also
start from a dense triangle mesh. The algorithm returns as
output a polygonal mesh which is 2d-manifold, watertight
and intersection-free. Optionally, the user can relax these
geometric guarantees. We first extract from the input 3D
data a set of primitives by standard methods [28, 31]. For
each detected primitive, we compute (i) a rough approx-
imation of its boundaries using α-shape [11], and (ii) an
oriented 2D bounding box, i.e. the smallest rectangle lying
on the detected plane that contains all its projected inliers.
We call a −bounding box, the oriented 2D bounding box
scaled up by an offset .
The algorithm operates in three steps illustrated in Fig-
ure 1. First, the connectivity relations between primitives
(a) input (b) connectivity analysis (c) space partitioning (d) surface extraction (e) output
Figure 1. Overview. Our algorithm starts from a point cloud and a set of primitives whose α-shapes are represented by colored polygons
(a). By analyzing the connectivity graph of primitives (see red edges), we extract some structurally-valid facets represented by colored
polygons with black edges (b). This quick connectivity analysis allows us to treat 35 of the 60 primitives of this model. We then build the
partitioning data-structure (see the pink wireframe) by slicing the spatially-close unprocessed primitives while embedding the structurally-
valid facets found in the previous step (c). The last step selects a subset of polygonal facets from the partition data-structure (d). The output
is a 2d-manifold polygon mesh in which each facet is a polygon supported by one of the primitives (e).
are analyzed in order to search for structurally-valid sur-
face components. This step, presented in Section 4, al-
lows us to quickly process a part of the input primitives
and solve obvious assembling situations before slicing op-
erations. We then build the partitioning data-structure in
Section 5 by slicing the spatially-close unprocessed prim-
itives while embedding the structurally-valid components
found in the previous step. Finally, the output surface is
recovered by selecting a subset of polygonal facets from the
partition data-structure using an energy minimization for-
mulation presented in Section 6.
4. Connectivity analysis
The objective of the first step is to quickly solve obvi-
ous local assemblings of some primitives by analyzing the
connectivity relations between them.
We define the notion of strong connectivity for char-
acterizing primitives that are spatially very close. When
detected from point clouds, two primitives are said
strongly-connected if at least two inlier points fitted each
to one of the two primitives are mutual neighbors in the
k-nearest neighbor graph of the input points. In case of
input meshes, two primitives are strongly-connected if at
least one inlier facet from the first primitive share an edge
with an inlier facet of the second primitive. We operate
our analysis on the connectivity graph where each node is
associated with a primitive, and each edge with a pair of
strongly-connected primitives. From real-world data, such
a graph usually contains errors with missing and invalid
connections. Our strategy is to search for structurally-valid
facets in this graph.
Extracting corners, creases and border polygons. We
first detect all the 3-cycles in the connectivity graph, i.e.
triples of primitives that are mutually connected. The point
located at the intersection of the three corresponding planes
is called a corner if it is close from the α-shapes of the three
primitives. In practice, we impose a maximal distance of
5% of the 3D bounding box diagonal. This condition allows
us to ignore a corner positioned far away from its primi-
tives, which typically occurs when primitives are nearly
parallel. We then detect creases, i.e. the line-segments link-
ing pairs of corners which have exactly two primitives in
common. Finally, we extract border polygons of each prim-
itive, i.e. the simple cycles of creases lying on the primitive.
Extracting structural facets. A primitive with border
polygons hosts a facet which is potentially a good candi-
date to be part of the output surface. In presence of one
border polygon, this facet is simply defined as its inside
surface. When two border polygons are nested, ie one of
these polygons is contained in the second one, we define
the facet as the surface in between the two polygons. When
border polygons intersect, we do not create facet to avoid
non-manifold degeneracies. Such a facet is called a struc-
tural facet if two conditions are respected:
• Data consistency: the facet must strongly overlap with the
α-shape of the primitive,
• Structural validity: all the creases lying on a primitive
must belong to the border polygons of that primitive.
The first condition checks whether the facet is well re-
covered by the α-shape of the primitive. In practice, we
impose an overlapping ratio between the facet and the α-
shape of the primitive higher than a threshold τ set to 0.9
in our experiments. The second condition guarantees that
the facet is unique and connect in a 2d-manifold way with
facets induced by the other primitives.
Structural facets connect between each others to form
2d-manifold polyhedral surface components that partially
describe the observed object. The border edges of these
components necessarily lie on the remaining primitives: we
call them anchor edges. We impose the structural facets
to be part of the final output mesh and discard their corre-
sponding primitives in the following steps. We denote by
P, the set of remaining primitives.
= 1 = 0.1 = 0.01
Input mesh & primitives
Figure 2. Soft-connectivity. When all the remaining primitives intersect with each others ( = 1), the 2D partition of the front facade of
the building is over-fragmented (see colored polygons on the top right frame with the anchor edges in red and the intersection lines in blue;
polygons with a black dot indicate they belong to the output surface on the left). Decreasing reduces the complexity of 2D partitions. In
presence of holes in the input mesh, primitive intersections can be missed when is too low (see the missing intersection between the front
and left facade in the case where = 0.01). is expressed as a ratio of the bounding box diagonal of the scene.
5. Space partitioning
Primitive slicing is usually performed in a greedy man-
ner in the literature. Typically, one first computes the slic-
ing domain of each primitive, i.e. the polygon lying on
the primitive plane and bounded by the 3D bounding box
of the observed object. Then, the 3D bounding box is di-
vided into polyhedra by inserting one per one each slic-
ing domain in an arbitrary order: the first slicing domain
splits the 3D bounding box into two polyhedra, the sec-
ond slicing domain typically splits the two polyhedra into
four polyhedra, etc. Because such a slicing strategy con-
siders the intersection of all pairs of slicing domains, the
number of polyhedra increases exponentially with respect
to the number of primitives. In practice, only a small
portion of these intersections is rel-
evant. To reduce the computational
burden of this operation, we re-
strict the pairs of primitives to be
sliced. We define the notion of soft-
connectivity to avoid intersecting slicing domains whose
primitives are not close enough. Two primitives are said
softly-connected if their -bounding boxes intersect inside
the 3D bounding box of the observed object (see inset). This
connectivity relationship is fast to compute and less restric-
tive than strong-connectivity.
In practice, we first intersect the slicing domains of
softly-connected primitives to form a 2D partitions of
polygonal facets. If anchor edges lie on primitives, they
are inserted into the corresponding 2D partitions. The an-
chor edges whose extremities are not connected to other an-
chor edges are extended until meeting an intersection line
or the border of the slicing domain. We finally split edges
that cross anchor edges. Figure 3 illustrates these different
slicing operations. Note that such a strategy generates a set
of polygonal facets which can eventually intersect between
each others without necessarily sharing an edge.
Figure 3. Slicing operations. Left: we first compute the slicing
domain (back lines) of primitive i and insert the anchor edges as-
sociated with this primitive (red segments). Middle: we then insert
line-segments defined as the intersection with the slicing domains
of softly connected primitives (blue lines) and extend anchor edges
whose extremities are not connected to other anchor edges (dashed
red lines). Right: the intersections of these different lines and
edges give the 2D partition of facets associated with primitive i.
The value of controls the complexity of the partitioning
data-structure, as illustrated in Figure 2. Choosing a low
value gives a set of light 2D partitions, low running time and
low memory consumption, but is less likely to be robust to
missing data.
We denote by F, the set of polygonal facets contained
in all the 2D partitions, and E, the set of edges. Note that
edges are typically adjacent to four facets, except in case of
anchor edges and rare situations where at least three primi-
tives intersect along the same line.
