3D Mapping with Semantic Knowledge
Andreas N¨uchter
1
, Oliver Wulf
2
, Kai Lingemann
1
, Joachim Hertzberg
1
,
Bernado Wagner
2
, and Hartmut Surmann
3
1
University of Osnabr¨uck, Institute for Computer Science,
Knowledge-Based Systems Research Group, Albrechtstraße 28,
D-49069 Osnabr¨uck, Germany
nuechter@informatik.uni-osnabrueck.de
WWW home page: http://www.informatik.uni-osnabrueck.de/nuechter/
2
University of Hannover, Institute for Systems Engineering (ISE/RTS),
Appelstraße 9A, D-30167 Hannover, Germany
3
Fraunhofer Institute for Autonomous Intelligent Systems (AIS),
Schloss Birlinghoven, D-53754 Sankt Augustin, Germany
Abstract. A basic task of rescue robot systems is mapping of the envi-
ronment. Localizing injured persons, guiding rescue workers and excava-
tion equipment requires a precise 3D map of the environment. This paper
presents a new 3D laser range finder and novel scan matching method for
the robot Kurt3D [9]. Compared to previous machinery [12], the apex an-
gle is enlarged to 360
◦
. The matching is based on semantic information.
Surface attributes are extracted and incorporated in a forest of search
trees in order to associate the data, i.e., to establish correspondences.
The new approach results in advances in speed and reliability.
1 Introduction
Rescue robotic systems are designed to assist rescue workers in earthquake, fire,
flooded, explosive and chemical disaster areas. Currently, many robots are ma-
nufactured, but most of them lack a reliable mapping method. Nevertheless, a
fundamental task of rescue is to localize injured persons and to map the envi-
ronment. To solve these tasks satisfactorily, the generated map of the disaster
environment has to be three-dimensional. Solving the problem of simultaneous
localization and mapping (SLAM) for 3D maps turns the localization into a
problem with six degrees of freedom. The x, y and z positions and the roll, yaw
and pitch orientations of the robot have to be considered. We are calling the
resulting SLAM variant 6D SLAM [10].
This paper addresses the problem of creating a consistent 3D scene in a
common coordinate system from multiple views. The proposed algorithms allow
to digitize large environments fast and reliably without any intervention and
solve the 6D SLAM problem. A 360
◦
3D laser scanner acquires data of the
environment and interprets the 3D points online. A fast variant of the iterative
closest points (ICP) algorithm [3] registers the 3D scans in a common coordinate
system and relocalizes the robot. The registration uses a forest of approximate
kd-trees. The resulting approach is highly reliable and fast, such that it can be
applied online to exploration and mapping in RoboCup Rescue.
The paper is organized as follows: The remainder of this section describes the
state of the art in automatic 3D mapping and presents the autonomous mobile
robot and the used 3D scanner. Section 2 describes briefly the online extraction
of semantic knowledge of the environment, followed by a discussion of the scan
matching using forests of trees (section 3). Section 4 presents experiments and
results and concludes.
1.1 3D Mapping – State of the Art
A few groups use 3D laser scanners [1,5,11,14,15]. The RESOLV project aimed
to model interiors for virtual reality and tele presence [11]. They used a RIEGL
laser range finder on robots and the ICP algorithm for scan matching [3]. The
AVENUE project develops a robot for modeling urban environments [1], using
an expensive CYRAX laser scanner and a feature-based scan matching approach
for registration of the 3D scans in a common coordinate system. Nevertheless, in
their recent work they do not use data of the laser scanner in the robot control
architecture for localization [5]. Triebel et al uses a SICK scanner on a 4 DOF
robotic arm mounted on a B21r platform to explore the environment [14].
Instead of using 3D scanners, which yield consistent 3D scans in the first
place, some groups have attempted to build 3D volumetric representations of
environments with 2D laser range finders [7, 8, 13, 15]. Thrun et al. [7, 13] use
two 2D laser range finder for acquiring 3D data. One laser scanner is mounted
horizontally, the other vertically. The latter one grabs a vertical scan line which
is transformed into 3D points based on the current robot pose. The horizontal
scanner is used to compute the robot pose. The precision of 3D data points
depends on that pose and on the precision of the scanner. Howard et al. uses
the restriction of flat ground and structured environments [8]. Wulf et al. let
the scanner rotate around the vertical axis. They acquire 3D data while moving,
thus the quality of the resulting map crucial depends on the pose estimate that
is given by inertial sensors, i.e., gyros [15]. In this paper we let rotate the scanner
continuously around its vertical axis, but accomplish the 3D mapping in a stop-
scan-go fashion, therefore acquiring consistent 3D scans as well.
Other approaches use information of CCD-cameras that provide a view of the
robot’s environment. Some groups try to solve 3D modeling by using a planar
SLAM methods and cameras, e.g., in [4].
1.2 Automatic 3D Sensing
The Robot Platform Kurt3D. Kurt3D (Fig. 1) is a mobile robot platform
with a size of 45 cm (length) × 33 cm (width) × 26 cm (height) and a weight of
15.6 kg, both indoor as well as outdoor models exist. Two 90 W motors (short-
term 200 W) are used to power the 6 wheels. Compared to the original Kurt3D
robot platform, the outdoor version has larger wheels, where the middle ones
are shifted outwards. Front and rear wheels have no tread pattern to enhance
rotating. Kurt3D operates for about 4 hours with one battery charge (28 NiMH
cells, capacity: 4500 mAh) charge. The core of the robot is a laptop computer
running a Linux operating system. An embedded 16-Bit CMOS microcontroller
is used to process commands to the motor. A CAN interface connects the laptop
with the microcontroller.
Fig. 1. The mobile robot platform Kurt3D offroad (left) and the 3D laser scanner
(right) The scanner rotates around the vertical axis. It’s technical basis is a SICK 2D
laser range finder (LMS-200).
The 3D Laser Scanner. As there is no commercial 3D laser range finder
available that could be used for mobile robots, it is common practice to assemble
3D sensors out of a standard 2D scanner and an additional servo drive [6, 12].
The scanner that is used for this experiment is based on a SICK LMS 291 in
combination with the RTS/ScanDrive developed at the University of Hannover.
Different orientations of the 2D scanner in combination with different turning
axes result in a number of possible scanning patterns. The scanning pattern that
is most suitable for this rescue application is the yawing scan with a vertical 2D
raw scan and rotation around the upright axis (see Fig. 1). The yawing scan
pattern results in the maximal possible field of view (360
◦
horizontal and 180
◦
vertical) and an uniform distribution of scan points.
As 3D laser scanner for autonomous search and rescue applications needs
fast and accurate data acquisition in combination with low power consumption,
the RTS/ScanDrive incorporates a number of improvements. One mechanical
improvement is the ability to turn continuously, which is implemented by using
slip rings for power and data connection to the 2D scanner. This leads to a
homogeneous distribution of scan points and saves the energy and time that is
needed for acceleration and deceleration of panning scanners. Another improve-
ment that becomes more important with short scanning times of a few seconds
is the compensation of systematic measurement errors. In this case the compen-
sation is done by sensor analysis and hard real-time synchronization, using a
Linux/RTAI operation system. These optimizations lead to scan times as short
as 3.2s for a yawing scan with 1.5
◦
horizontal and 1
◦
vertical resolution (240x181
points). For details on the RTS/ScanDrive see [17].
2 Extracting Semantic Information
The basic idea of labelling 3D points with semantic information is to use the
gradient between neighbouring points to differ between three categories, i.e.,
floor-, object- and ceiling-points. A 3D point cloud that is scanned in a yawing
scan configuration, can be described as a set of points p
i,j
= (φ
i
, r
i,j
, z
i,j
)
T
given
in a cylindrical coordinate system, with i the index of a vertical raw scan and j
the point index within one vertical raw scan counting bottom up. The gradient
α
i,j
is calculated by the following equation:
tan α
i,j
=
z
i,j
− z
i,j−1
r
i,j
− r
i,j−1
with −
1
2
π ≤ α
i,j
<
3
2
π.
The classification of point p
i,j
is directly derived from the gradient α
i,j
:
1. floor-points: α
i,j
< τ
2. object-points: τ ≤ α
i,j
≤ π − τ
3. ceiling-points: π − τ < α
i,j
with a constant τ that depends on the maximal ascent being accessible by the
robot (here: τ = 20
◦
).
Applied to real data, this simple definition causes two problems. As can be
seen in Fig. 2(a) noisy range data can lead to wrong classifications of floor- and
ceiling-points. Changing the differential quotient as follows solves this problem:
tan α
i,j
=
z
i,j
− z
i,j−k
r
i,j
− r
i,j−k
with k ∈
, k ≥ 1 the smallest number so that
q
(r
i,j
− r
i,j−k
)
2
+ (z
i,j
− z
i,j−k
)
2
> d
min
for a constant d
min
depending on the scanner’s depth accuracy σ (here: σ =
30 mm, d
min
= 2σ).
The second difficulty is the correct computation of the gradient across jump-
ing edges (see Fig. 2(b)). This problem is solved with a prior segmentation [16],
as the gradient α
i,j
is only calculated correctly if both points p
i,j
and p
i,j−k
belong to the same segment. The correct classification result can be seen in Fig.
2(c). Fig. 3 shows a 3D scan with the semantic labels.
(a)
(b)
(c)
Fig. 2. Extracting semantic information using a slice of a 3D scan. (a) Problems with
simple gradiant definition, marked with circles. (b) Problems with jump edges. (c)
Correct semantic classification.
Fig. 3. Semantically labeled 3D point cloud from a single 360
◦
3D scan. Red points
mark the ceiling, yellow points objects, blue points the floor and green points corre-
spond to artefacts from scanning the RTS/ScanDrive and the robot.
3 Scan Registration and Robot Relocalization
Multiple 3D scans are necessary to digitalize environments without occlusions.
To create a correct and consistent model, the scans have to be merged into
one coordinate system. This process is called registration. If the localization
of the robot with the 3D scanner were precise, the registration could be done
directly based on the robot pose. However, due to the unprecise robot sensors,
self localization is erroneous, so the geometric structure of overlapping 3D scans
has to be considered for registration. Furthermore, Robot motion on natural
surfaces has to cope with yaw, pitch and roll angles, turning pose estimation into
a problem in six mathematical dimensions. A fast variant of the ICP algorithm
registers the 3D scans in a common coordinate system and relocalizes the robot.
The basic algorithm was invented in 1992 and can be found, e.g., in [3].
