scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Evaluating the Capability of OpenStreetMap for Estimating Vehicle Localization Error

01 Oct 2019-pp 142-149
TL;DR: This paper explores the potential of using OpenStreetMap (OSM) as a proxy to estimate vehicle localization error from map matching using LiDAR scans and ND maps and uses random forest regression to estimate mean 3D localization error.
Abstract: Accurate localization is an important part of successful autonomous driving. Recent studies suggest that when using map-based localization methods, the representation and layout of real-world phenomena within the prebuilt map is a source of error. To date, the investigations have been limited to 3D point clouds and normal distribution (ND) maps. This paper explores the potential of using OpenStreetMap (OSM) as a proxy to estimate vehicle localization error. Specifically, the experiment uses random forest regression to estimate mean 3D localization error from map matching using LiDAR scans and ND maps. Six map evaluation factors were defined for 2D geographic information in a vector format. Initial results for a 1.2 km path in Shinjuku, Tokyo, show that vehicle localization error can be estimated with 56.3% model prediction accuracy with two existing OSM data layers only. When OSM data quality issues (inconsistency and completeness) were addressed, the model prediction accuracy was improved to 73.1%.

Summary (5 min read)

Introduction

  • Recent studies suggest that when using map-based localization methods, the representation and layout of real-world phenomena within the prebuilt map is a source of error.
  • To date, the investigations have been limited to 3D point clouds and normal distribution (ND) maps.
  • This paper explores the potential of using OpenStreetMap (OSM) as a proxy to estimate vehicle localization error.
  • Specifically, the experiment uses random forest regression to estimate mean 3D localization error from map matching using LiDAR scans and.

ND maps. Six map evaluation factors were defined for 2D geographic information in a vector format. Initial results for a

  • 2 km path in Shinjuku, Tokyo, show that vehicle localization error can be estimated with 56.3% model prediction accuracy with two existing OSM data layers only.
  • Within a tunnel or urban canyon environment, even with a locally and globally accurate map, the lack of longitudinal features in the map can cause localization error in the moving direction.
  • Secondly, the large size of the point cloud data can make it challenging to manage.
  • The resulting work can enable the estimation of localization error without requiring the collection of data to create a prebuilt map.

II. BACKGROUND

  • While Global Navigation Satellite Systems can provide a position with meter-level accuracy, this is insufficient for the application of autonomous driving.
  • Subsequently, the LiDAR scan from an autonomous vehicle can then be used to match against the prebuilt map to obtain a position.
  • There is yet to be an international standard for required accuracy.
  • As guidance, the Japanese government’s Cross-ministerial Strategic Innovation Promotion (SIP) Program recommends an accuracy of less than 25 cm.

B. Sources of localization error for LiDAR map matching

  • Sources of localization error for map matching using LiDAR can be divided broadly into four categories: 1) Input scan; 2) Matching algorithm; 3) Dynamic phenomena, and; 4) Prebuilt map.
  • The lower-end Velodyne VLP-16 has 16 laser transmitting and receiving sensors with a range of up to 100m.
  • Errors can also be introduced in the post-processing where the input scan is required to be downsampled for the matching algorithm.
  • During the localization phase, some phenomena or features may have moved or shifted.
  • In these cases, it is the physical attributes of the phenomena and its representation on the map which is the source of localization error.

C. Quantifying the sources of map-derived errors

  • To quantify the sources of map-derived errors, Javanmardi et al. [4] defined four criteria to evaluate a map’s capability for vehicle localization: 1) feature sufficiency; 2) layout; 3) local similarity, and; 4) representation quality.
  • An insufficient number of features in the vicinity (such as found in open rural areas) may result in lower localization accuracy.
  • Even if there are lots of high-quality features nearby, if they are all concentrated in one direction, the quality of the matching degrades.
  • The closer the map is to reality, the more accurate the prediction.
  • A flat wall can be highly abstracted as a single line but still have high representation quality.

D. OpenStreetMap

  • First introduced in 2004, OpenStreetMap is an open source collaborative project providing user-generated street maps of the world.
  • In the past, commercial and governmental mapping organizations have also donated data towards the project.
  • There is a set of commonly used tags for primary features which operate as an ‘informal standard’ [9].
  • While fire departments, police stations and post offices are well represented, temples and shrines are underrepresented.
  • In fact, in certain areas, OSM is more complete and more accurate (for both location and semantic information) than corresponding proprietary datasets [12], [13].

A. Study area

  • The study area is Shinjuku, Tokyo, Japan.
  • The architecture is relatively heterogeneous, with buildings ranging from lowrise shops to multi-story structures.
  • Similarly, roads vary from single narrow lanes to wide multi-lane carriageways.

B. Mapping data

  • Mapping data from OSM was extracted from the geofabrik.de server.
  • Note that each tag can be represented by a maximum of three layers, one for each geometry primitive (point, line, polygon).
  • On wide multi-lane roads or within the rural environment, buildings may not be present or visible by the LiDAR scanner.
  • Linear features, such as guard rails and traffic barriers, are classed as ‘barrier_line’.
  • By creating these ‘completed’ layers, it allowed the assessment of the capability of OSM, in a scenario where users had correctly and fully mapped all required features.

C. Localization error data

  • For localization error, data from a previous experiment was used.
  • From this point cloud map, an ND map was generated with a 2.0 m grid size.
  • Next, during the localization phase, a second scan was obtained with the laser scan range limited to 20 m.
  • Thirdly, to evaluate the map factors, sample points along the experiment path at 1.0m intervals were selected.
  • For each sample point, mean 3D localization error was calculated by averaging the localization error from 441 initial guesses at different positions evenly distributed around the sample point, at 0.2 m intervals within a range of 2 m.

D. Formulating map evaluation factors

  • Javanmardi et al. [4] proposed ten map evaluation factors to quantify the capability of ND map for vehicle selflocalization.
  • Unlike the four map evaluation criteria, the map factors were designed specifically to evaluate 3D ND maps and cannot be easily transferred to another mapping format without modification.
  • Local similarity and representation quality were not considered in this study, due to the high level of abstraction of OSM and the lack of a higher quality dataset available for comparison.
  • To model the behavior of the laser scanner and localization process, two auxiliary layers were produced: view lines and view polygons.

E. Factors for feature sufficiency criterion

  • Unlike 3D ND map formats, there are three geometry primitives for 2D vector mapping: points, lines, and polygons.
  • To account for the differences between the geometric representations, feature count was calculated based on the intersection points between the layer and the view lines or view polygons.
  • At these end points of the view lines, it is assumed that the sensor cannot ‘see’ any further past the opaque building walls.
  • To remedy this, points were generated along the intersection with view lines at 10 cm intervals i.e. the portion of the view line which overlaps with any polygon barrier were converted into a series of points.
  • This, however, does not affect the factors, as each layer is considered separately during the calculation.

1) Angular dispersion of features

  • Angular dispersion is a measure of the arrangement of features around the sample point.
  • For this factor, the dispersion of the intersection points between the view lines or view polygons and the respective layer was calculated.

2) View line mean length

  • For each sample point, the mean length for every view line which intersected with the building layer was calculated.
  • Nonintersecting view lines were not considered as they would skew the metric.

3) Area of view polygon

  • It represents the theoretical area that the laser scanner can ‘see’.
  • A small area suggests that there are lots of building features in the vicinity to localize against.

4) Compactness of view polygon

  • A compact field of view could suggest that there are a lot of building features nearby to localize against.
  • To measure compactness, the ratio between the area of the view polygon and the area of a circle with the same radius was calculated.

5) The variance of building face direction

  • Javanmardi et al.’s [4] suggest that if the features in the local vicinity face a greater variety of directions, then the positioning uncertainty decreases.
  • To measure this, at all points of intersection between the view lines and the building layer, the normal of the face was calculated.
  • Subsequently, for each sample point, the variance of all the normal vectors was calculated.

G. Random forest regression

  • To obtain a prediction of localization error, random forest regression was used.
  • It uses multiple decision trees and bootstrap aggregation to form a prediction.
  • Random forest regression aims to reduce variance in predictions while maintaining a low bias, thus controlling overfitting.
  • For the implementation, scikit-learn (a machine learning Python library) was used.

A. General results

  • Six models were evaluated, using different combinations of data layers from OSM.
  • Model 1 used just the buildings layer directly from OSM.
  • Model 3 included the completed polygon barriers.
  • To evaluate the goodness of fit, the R-squared (R2), root mean square error (RMSE), mean absolute error (MAE), and accuracy are presented.
  • The models and their prediction errors are presented in TABLE IV.

B. Factor importance

  • One of the advantages of random forest regression is the ability to evaluate the importance of different variables used for prediction.
  • TABLE V. shows an overview of the importance of each factor and data layer, with the two most important variables in bold.
  • Note that for models 2 to 5, the feature count of polygon barriers is the first or second most important factor for predicting localization error (ranging from 0.18 to 0.38).
  • With the inclusion of point barriers data in models 5 and 6, the angular dispersion factor has high importance (0.20 and 0.22).
  • Aside from model 1, the importance of factors based on view line and view polygon geometry is below 0.1.

A. The capability of OpenStreetMap for estimating vehicle localization error

  • Show that using both the buildings and polygon barriers layer from OSM, the model can achieve 56.5% prediction accuracy.
  • By improving the completeness of the polygon barriers, the model prediction accuracy is then increased to 62.3%.
  • The work described in this paper show that it is possible to estimate vehicle localization error with reasonable success.
  • Further work is required to devise a more appropriate accuracy metric which better describes a model’s ability to detect these peaks in localization error, and thus its suitability for the autonomous driving application.

B. Comparison with other studies

  • It is important to bear in mind the difference in methodology – while [5] uses principal component regression, this study uses random forest regression.
  • Further work is therefore required to confirm if OpenStreetMap can truly achieve comparable results with existing approaches.

C. The importance of different factors and data layers

  • Of the factors, feature count is the most important for all six models.
  • There are several possible explanations for this result.
  • Firstly, the middle half of the experiment route is populated by many polygon barriers .
  • In addition, in this central section of the experiment path, the combination of wide multi-lane roads, restricted 20m scanner range, and building setback means that buildings cannot be necessarily ‘seen’ by the LiDAR sensor.
  • In other areas where there are no point or polygon barriers, other data layers such as buildings and line barriers could become more important.

D. The impact of data quality of OSM

  • As discussed in Section II.D, OpenStreetMap suffers from data quality issues such as inconsistency and incompleteness.
  • This is encouraging and could suggest that as the quality of OSM improves over time, its capability to estimate localization error also increases.
  • From a data handling and computation perspective, it is much ‘lighter’.
  • This means feature scale phenomena such as local similarity cannot be easily evaluated.
  • Beyond OpenStreetMap, the same factors could be used with better quality 2D GI.

F. Generalizability of the model

  • The model is currently only trained on data from a relatively short path, resulting in relatively low generalizability.
  • The addition of more data from multiple paths should reduce any random patterns that may previously have appeared predictive.
  • This would, in turn, enable the use of OSM to predict localization error beyond the modeled area.
  • Coupled with the wide coverage of OSM, it could theoretically be possible to predict localization error for large areas, anywhere in the world, without collecting data to create a prebuilt map as the mapping data is readily available.

G. Alternatives to random forest regression

  • Random forest regression is only one machine learning approach for prediction.
  • In some cases, alternative approaches such as support vector machine or gradient boosting machines could outperform random forest regression and provide a more generalizable model.
  • These approaches, however, also require additional manual hyperparameter tuning.

H. Source of localization error

  • As described in Section II.B, there are many sources of localization error for LiDAR map matching.
  • The precise sources of localization error, however, remain difficult to ascertain.
  • Within the experiment, multiple measures were taken to ensure the reduction of non-map errors, so that any remaining localization error evaluated was directly related to the map as much as possible.
  • Alternatively, artificial objects could be installed in the environment to improve map matching performance.
  • Furthermore, knowing the specific location of areas of high localization error could inform semi-autonomous vehicle systems of where to disengage autopilot and to prepare for human driver takeover in a timely manner.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Abstract Accurate localization is an important part of
successful autonomous driving. Recent studies suggest that when
using map-based localization methods, the representation and
layout of real-world phenomena within the prebuilt map is a
source of error. To date, the investigations have been limited to
3D point clouds and normal distribution (ND) maps. This paper
explores the potential of using OpenStreetMap (OSM) as a proxy
to estimate vehicle localization error. Specifically, the
experiment uses random forest regression to estimate mean 3D
localization error from map matching using LiDAR scans and
ND maps. Six map evaluation factors were defined for 2D
geographic information in a vector format. Initial results for a
1.2 km path in Shinjuku, Tokyo, show that vehicle localization
error can be estimated with 56.3% model prediction accuracy
with two existing OSM data layers only. When OSM data
quality issues (inconsistency and completeness) were addressed,
the model prediction accuracy was improved to 73.1%.
I. I
NTRODUCTION
Accurate localization is a crucial requirement of successful
autonomous driving, allowing self-driving vehicles to navigate
safely. Major sources of positioning information are Global
Navigation Satellite Systems (GNSS) such as GPS,
GLONASS, Galileo or BeiDou. The positional accuracy using
just GNSS, however, can exhibit errors of tens of meters
within dense urban environments due to buildings blocking
and reflecting the satellite signal [1]. To address this issue,
self-driving vehicles use vision and range-based sensors to
determine where they are located. One such method utilizes
light detection and ranging (LiDAR) sensors for map
matching, whereby the input scan is registered against a
prebuilt map to obtain a position. Despite the advancement in
LiDAR sensor technology, map-based localization can still
suffer from error, arising from sources such as the input scan
or the matching algorithm.
To improve map-based localization accuracy, research so
far has focused on refining the map matching algorithms [2] as
well as producing increasingly accurate High Definition maps
[3]. In contrast, there is a lack of research on the impact of the
map itself, and the representation of the environment, as a
source of localization error. For example, within a tunnel or
urban canyon environment, even with a locally and globally
accurate map, the lack of longitudinal features in the map can
cause localization error in the moving direction. So far, the
author is aware of only one set of studies which have been
conducted to evaluate the relation between the representation
within the map and localization accuracy [4], [5].
To date, the evaluation of a map’s capability for vehicle
localization has been limited to the 3D point cloud format.
Kelvin Wong, Ehsan Javanmardi, Mahdi Javanmardi, Yanlei Gu and
Shunsuke Kamijo are with The Institute of Industrial Science (IIS), The
There are several challenges with this approach. Firstly,
localization error can only be estimated for an area if a prebuilt
point cloud map is available. If not, data must be collected,
which can be costly and time-consuming. These 3D point
cloud maps also require substantial effort to maintain and keep
up to date. Secondly, the large size of the point cloud data can
make it challenging to manage. For example, a 300×300 m
area may consist of over 250 million points. Coupled with the
unstructured nature of the data, this leads to a large amount of
pre-processing, processing, and subsequent management.
Considering these challenges, there is a need to explore
whether other mapping formats and sources of data, such as
geographic information, can offer an alternative approach for
estimating vehicle localization error.
This paper aims to investigate whether it is possible to
estimate vehicle localization error using OpenStreetMap
(OSM) as a proxy. OpenStreetMap is a freely editable 2D map
of the world. The main objective is to leverage OSM’s public
availability, wide coverage, and comparatively ‘light’ data to
enable a more efficient alternative for estimating localization
error. This work builds upon Javanmardi et al.’s [4] format-
agnostic map evaluation criteria, by defining six new factors
specific for 2D geographic information (as used by
OpenStreetMap). The resulting work can enable the estimation
of localization error without requiring the collection of data to
create a prebuilt map. It can help identify and highlight areas
with high localization error either as the result of the map
itself, or the real-world environment. Further, the estimated
error model can be used in conjunction with other sensor
information to improve the accuracy of localization [6].
The remainder of this paper is structured as follows: in
Section II, a short overview of vehicle localization and its
sources of error is presented, followed by an overview of
geographic information and OpenStreetMap. Section III
describes the datasets used in this study, as well as the
formulation of the six map evaluation factors. The results of
the experiment are presented in Section IV. Following a
discussion on the capability of OSM, and the advantages and
challenges of the approach, the final Section presents the
conclusions and future work.
II. B
ACKGROUND
A. Vehicle localization
Localization is the estimation of where a vehicle is situated
in the world. While Global Navigation Satellite Systems can
provide a position with meter-level accuracy, this is
insufficient for the application of autonomous driving. One
University of Tokyo, Tokyo, 153-8505 Japan. (Tel: +81-(0)3-5452-6273; e-
mail: {kelvinwong, ehsan, mahdi, guyanlei, kamijo}@kmj.iis.u-tokyo.ac.jp.
Evaluating the Capability of OpenStreetMap for Estimating Vehicle
Localization Error
Kelvin Wong, Ehsan Javanmardi, Mahdi Javanmardi, Yanlei Gu and Shunsuke Kamijo

meter can be the difference between driving safely on the road
in the correct lane and driving along the pavement.
Centimeter-level positional accuracy is therefore required for
successful autonomous driving. To achieve this, autonomous
vehicles are equipped with a myriad of sensors, including
LiDAR, radar, and onboard cameras.
One approach for localization is LiDAR map matching.
For this, a map is prebuilt using a high-end mobile mapping
system. Subsequently, the LiDAR scan from an autonomous
vehicle can then be used to match against the prebuilt map to
obtain a position. Despite the importance of accurate vehicle
self-localization, there is yet to be an international standard for
required accuracy. As guidance, the Japanese government’s
Cross-ministerial Strategic Innovation Promotion (SIP)
Program recommends an accuracy of less than 25 cm. This
value is based on satellite image resolution and the physical
width of a car tire.
B. Sources of localization error for LiDAR map matching
Sources of localization error for map matching using
LiDAR can be divided broadly into four categories: 1) Input
scan; 2) Matching algorithm; 3) Dynamic phenomena, and; 4)
Prebuilt map.
Firstly, errors from the input scan may be dictated by
choice of the laser scanner. For example, the lower-end
Velodyne VLP-16 has 16 laser transmitting and receiving
sensors with a range of up to 100m. In contrast, the latest
model (VLS-128) has 128 sensors and up to 300 m range,
allowing the capture of more detail in a wider area, thus
enabling more accurate map matching. In addition, distortion
can be introduced during the capture phase when in motion –
a vehicle traveling at 20 m/s, with a scanner at 10Hz would
result in a 2 m distortion as the scanner rotates 360 degrees.
Errors can also be introduced in the post-processing where the
input scan is required to be downsampled for the matching
algorithm. While this reduces noise within the data, it can also
remove features and details useful for map matching.
Secondly, the choice of the matching algorithm can be a
source of error. As matching is an optimization problem, some
algorithms are designed to be more robust to local optimum,
while others are more immune to initialization error [7]. In
general, Normal Distribution Transform (NDT) is more
resilient to sensor noise and sampling resolution (than other
algorithms such as Iterative Closest Point) but suffers from
slower convergence speed [8].
Thirdly, dynamic phenomena in the environment can
contribute error. During the mapping phase, if a large vehicle
such as a lorry or a bus blocks the laser scanner, then a large
portion of the map will be missing. During the localization
phase, some phenomena or features may have moved or
shifted. For example, parked cars found in the prebuilt map
may have since moved or buildings may have been built or
demolished. Furthermore, seasonal changes such as the
presence and absence of leaves on trees between the mapping
and localization phases can introduce error.
Lastly, the prebuilt map itself is a source of error. As
mentioned earlier, having a highly accurate map does not
always result in a low localization error. There are instances
within the environment, e.g. tunnels, urban canyons, open
spaces whereby map matching is not a suitable localization
method. In these cases, it is the physical attributes of the
phenomena and its representation on the map which is the
source of localization error. This is discussed further in the
next section.
C. Quantifying the sources of map-derived errors
To quantify the sources of map-derived errors, Javanmardi
et al. [4] defined four criteria to evaluate a map’s capability for
vehicle localization: 1) feature sufficiency; 2) layout; 3) local
similarity, and; 4) representation quality.
Feature sufficiency describes the number of mapped
phenomena in the vicinity which can be used for localization.
For example, within urban areas, buildings and other built
structures provide lots of features to match against, resulting
in more accurate localization. An insufficient number of
features in the vicinity (such as found in open rural areas) may
result in lower localization accuracy.
The layout is also an important consideration, as it is not
simply the number of features in the vicinity, but also how they
are arranged. Even if there are lots of high-quality features
nearby, if they are all concentrated in one direction, the quality
of the matching degrades.
To further compound the issue, in certain scenarios, even
with sufficient and well-distributed features, accurate
localization remains difficult due to local similarity. In these
cases, there may be geometrically similar features either side
of the vehicle, making it difficult during the optimization
process to determine the position in the traveling direction.
Lastly, representation quality describes how well a map
feature represents its real-world counterpart. In theory, the
closer the map is to reality, the more accurate the prediction. It
is important to note, however, that this criterion is slightly
different from the level of abstraction or generalization of the
map. For example, a flat wall can be highly abstracted as a
single line but still have high representation quality.
D. OpenStreetMap
First introduced in 2004, OpenStreetMap is an open source
collaborative project providing user-generated street maps of
the world. As of March 2019, the project has over 5.2 million
contributors to the project who create and edit map data using
GPS traces, local knowledge, and aerial imagery. In the past,
commercial and governmental mapping organizations have
also donated data towards the project.
Features added to OSM are modeled and stored as tagged
geometric primitives (points, lines, and polygons). For
example, a road may be represented as a line vector geometry
with tags such as highway = primary; name:en = Ome Kaido;
oneway = yes’. Despite OSM operating on a free-tagging
system (and thus allowing a theoretically unlimited number of
attributes and tag combinations), there is a set of commonly
used tags for primary features which operate as an ‘informal
standard’ [9].
As with any user-generated content, OSM attracts
concerns about data quality and credibility. Extensive research
has been conducted on assessing the quality of OSM datasets,
including evaluating the completeness of the data. For

example, Barrington-Leigh and Millard-Ball [10] estimate the
global OSM street layer to be 83% complete, with more than
40% of countries having fully mapped networks. When
comparing the number of OSM tags in Japan to official
government counts (TABLE I. ), ‘completeness’ varies with
different features [11]. While fire departments, police stations
and post offices are well represented, temples and shrines are
underrepresented. Despite this variance in completeness, OSM
still represents a viable and useful source of geospatial data. In
fact, in certain areas, OSM is more complete and more
accurate (for both location and semantic information) than
corresponding proprietary datasets [12], [13]. As mapping data
are abstract representations of the world, it can also be argued
that a map can never truly be complete. Instead, mapping data
should be collected as fit-for-purpose, and specific for the
required application. OSM is successfully used in a wide
variety of practical and scientific applications from different
domains, demonstrating the usefulness of crowdsourced
mapping data.
TABLE I. C
OVERAGE OF
OSM
TAGS IN
J
APAN
2017
[11]
Tag # Actual # in OSM Coverage
School 36,024 45,568 126.5%
a
Fire department 5,604 5,028 89.7%
Police station 15,034 13,152 87.5%
Post office 24,052 20,795 86.5%
Traffic lights 191,770 108,498 56.6%
Convenience store 55,176 30,710 55.7%
Bank 13,595 7,077 52.1%
Gas station 32,333 8,944 27.7%
Pharmacy 58,326 7,842 13.4%
Shrine 88,281 10,292 11.7%
Temple 85,045 9,610 11.3%
Postal post 181,523 7,522 4.1%
Vending machine 3,648,600 10,311 0.3%
a. Note that the coverage of schools is higher than 100%. This is the result of schools comprising of
multiple buildings and therefore having more than one polygon or point represented within OSM.
E. Related work
There are a several studies on the estimation of vehicle
localization error in literature. Akai et al. [6] estimate the error
of 3D NDT scan matching in a pre-experiment, which was
then subsequently used in the localization phase for pose
correction. Javanmardi et al. [4] proposed ten map evaluation
factors to quantify the capability of normal distribution (ND)
maps for vehicle self-localization, allowing error to be
modelled with an RMSE and R
2
of 0.52 m and 0.788
respectively. The subsequent error model was then used to
dynamically determine the NDT map resolution, resulting in a
reduction of map size by 32.4% while keeping mean error
within 0.141 m. For both studies [4], [6], the estimation of
localization error used 3D point cloud and ND maps. As far as
the authors are aware, no previous study has investigated the
use of OpenStreetMap as a proxy for the estimation of
autonomous vehicle localization error.
III.
M
ETHOD
A. Study area
The study area is Shinjuku, Tokyo, Japan. The architecture
is relatively heterogeneous, with buildings ranging from low-
rise shops to multi-story structures. Similarly, roads vary from
single narrow lanes to wide multi-lane carriageways. The
experiment path is 1.2 km and is shown in Figure 1.
Figure 1. Experiment path in Shinjuku, Tokyo, Japan.
B. Mapping data
Mapping data from OSM was extracted from the
geofabrik.de server. For the study area, 49 data layers
(referencing 23 tags) were available. Note that each tag can be
represented by a maximum of three layers, one for each
geometry primitive (point, line, polygon). TABLE II. shows a
summary of all the tags found in the study area and their
feature counts.
TABLE II. OSM
TAGS AND FEATURE COUNT IN THE STUDY AREA
OSM tags Feature count
OSM tags Feature count
aeroway 12 natural 595
amenity 2,109 office 35
barrier 675 place 4
boundary 16 power 20
building 10,723
public
transport
114
craft 2 railway 485
emergency 8 route 100
highway 4,021 shop 580
historic 7 tourism 168
landuse 202 unknown 598
leisure 87 waterway 2
man made 12
Note that OSM tags can represent geometric or semantic
features. For example, features labeled as ‘building’ may
represent the physical walls of the structure, whereas a
‘boundary’ feature may represent an administrative region,
e.g., ward boundary. For this experiment, the aim is to model
what the LiDAR scanner ‘sees’ during the localization phase.
As such, only the geometric features from OSM were useful.

To model the surrounding environment, four OSM data
layers were used: 1) building_polygon; 2) barrier_polygon; 3)
barrier_line, and; 4) barrier_point. The buildings layer was
selected as they represent the most stable feature within the
urban landscape. Buildings, in general, are least affected by
seasonality or time of day (unlike other features such as trees).
For vehicle localization, however, the use of buildings alone
for map matching may be insufficient. For example, on wide
multi-lane roads or within the rural environment, buildings
may not be present or visible by the LiDAR scanner. In these
scenarios, roadside features (such as central reservations,
guard rails, and pole-like features) become increasingly
important. Within OSM, these are classified under the
‘barriers’ tag. Areal features, such as hedges, are classed under
‘barrier_polygon’. Linear features, such as guard rails and
traffic barriers, are classed as ‘barrier_line’. Lastly, pole-like
features (e.g., bollards) are classed as ‘barrier_point’.
The completeness of OSM varies within the study area. For
buildings, the data is 74.3% complete when compared to GSI
Japan building footprints, using area as a crude measure. For
barriers, an authoritative equivalent dataset was not available
for reference. In a visual comparison with aerial imagery, over
half of the hedges and central reservations (polygon barriers)
were missing along the experiment path. There were no line or
point barriers mapped within 20 m of the experiment path
(although they were present in the wider study area).
To mitigate any issues of inconsistency and
incompleteness (as highlighted in Section II.D), an additional
set of ‘completed’ OSM layers were created. Supplementary
data (aerial imagery and base map) from ATEC and the
Geospatial Information Authority of Japan were used to
support the manual digitization. By creating these ‘completed’
layers, it allowed the assessment of the capability of OSM, in
a scenario where users had correctly and fully mapped all
required features. This allowed for a more robust evaluation,
without being hindered by data quality issues. In total, 63
polygon-barriers, 31 line-barriers, and 204 point-barriers were
manually added along the experiment path. The ‘completed’
OSM layers are presented in Figure 2. TABLE III. presents a
summary of all the mapping data used in the study.
TABLE III. S
UMMARY OF MAPPING DATA USED
Layer
name
Type Description Count
Original OpenStreetMap layers
Building Polygon
Individual buildings or groups of
connected buildings, e.g.,
apartments, office, cathedral
10,723
Polygon
barriers
Polygon
Areal barriers and obstacles, e.g.
hedges, central reservation
410
Completed OpenStreetMap layers
Polygon
barriers
(completed)
Polygon
Areal barriers and obstacles,
manually completed using aerial
imagery and external reference
maps.
473
Line
barriers
(completed)
Line
Linear barriers and obstacles, e.g.
guard rails. Manually digitized
using aerial imagery and external
229
Point
barriers
(completed)
Point
Point barriers and obstacles, e.g.
pole-like objects, lamp posts,
traffic lights. Manually digitized
using aerial imagery and external
reference maps
271
Figure 2. Map showing the ‘completed’ OSM layers and experiment path
C. Localization error data
For localization error, data from a previous experiment was
used. Specifically, the mean 3D localization error from map
matching using a LiDAR input scan and a prebuilt normal
distribution (ND) map with a 2.0 m grid size. The process of
obtaining the mean 3D localization error is described below.
Firstly, a point cloud map of the area was captured using a
Velodyne VLP-16 mounted on the roof of a vehicle at the
height of 2.45 m. The vehicle speed was limited to 2 m/s, with
a scanner frequency of 20 Hz to mitigate motion distortion.
From this point cloud map, an ND map was generated with a
2.0 m grid size. Next, during the localization phase, a second
scan was obtained with the laser scan range limited to 20 m.
This second scan was then registered to the ND map (NDT
map matching) to obtain a location. Thirdly, to evaluate the
map factors, sample points along the experiment path at 1.0m
intervals were selected. For each sample point, mean 3D
localization error was calculated by averaging the localization
error from 441 initial guesses at different positions evenly
distributed around the sample point, at 0.2 m intervals within
a range of 2 m. Further details on the experimental set-up and
the creation of the data can be found in [4].
D. Formulating map evaluation factors
Javanmardi et al. [4] proposed ten map evaluation factors
to quantify the capability of ND map for vehicle self-
localization. Unlike the four map evaluation criteria, the map
factors were designed specifically to evaluate 3D ND maps
!
(
!(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(!(
!
(
!(
!(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
( !
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!(
!
(
!
(
!(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(!(
!
(
!
(
!(
!(
!(
!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!(!
(
!
(
!
(
!
(
!
(
!
(
!(!(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!(
!(
!
(
!(
!
(
!
(
!
(
!
(
!
(
!(
!
(
!
(
!(
!
(
!
(
!
(
!
(
!(
Key
!
(
Point barriers
Line barriers
Polygon barriers
Buildings
Experiment path
±
0 150
Meters

and cannot be easily transferred to another mapping format
without modification. Therefore, six new factors for 2D vector
mapping format were defined, based on Javanmardi et al.’s [4]
map evaluation criteria. Specifically, the focus was on feature
sufficiency of the map and layout of the map. Local similarity
and representation quality were not considered in this study,
due to the high level of abstraction of OSM and the lack of a
higher quality dataset available for comparison.
To model the behavior of the laser scanner and localization
process, two auxiliary layers were produced: view lines and
view polygons. Firstly, for each sample point, 360 view lines,
each 20 m in length, radiating outwards from the center at
interval, were generated (Figure 3a). These lines represent the
laser scanner’s 20 m range. Within the urban environment, the
walls of buildings represent a ‘solid’ and opaque barrier for the
laser scanner. As OSM is 2D, it is assumed that buildings are
infinitely tall, and view lines were clipped where they intersect
with the building layer (Figure 3b). This assumption was based
on the 30° vertical field of view (+/- 15° up and down) of the
Velodyne VLP-16. Mounted at 2.45 m, and with a laser range
of 20 m, the effective vertical view is between 0 to 7.8 m. This
is equivalent to an average two-story building. Considering the
buildings in the experiment area were, at a minimum, three
stories high, this assumption was acceptable. Secondly, from
these clipped view lines, a minimum bounding convex hull
was generated as an areal representation of what the laser
scanner ‘sees’ and is referred to as a view polygon (Figure 3c).
Figure 3. View lines and view polygons.
E. Factors for feature sufficiency criterion
1) Feature count
The first factor is the feature count. It is an adaption of
Javanmardi et al.’s [4] factor, providing a simple count of all
nearby features. However, unlike 3D ND map formats, there
are three geometry primitives for 2D vector mapping: points,
lines, and polygons. Further, the 3D nature of the scanner must
be abstracted in 2D.
To account for the differences between the geometric
representations, feature count was calculated based on the
intersection points between the layer and the view lines or
view polygons.
For buildings, feature count was the total number of view
lines which intersected with the buildings layer. In this
scenario, the intersection points were also the endpoints of the
view lines as they were previously clipped (as described in
Section III.D). At these end points of the view lines, it is
assumed that the sensor cannot ‘see’ any further past the
opaque building walls.
For point barriers, this was a simple count of features
which intersected with (and were therefore within) the view
polygon.
For line barriers, all intersection points between all view
lines and line barriers were counted.
Polygon barriers, unlike buildings, were not considered to
be infinitely tall and opaque as the LiDAR sensor can often
‘see’ over them. Modelling this was not as straight forward, as
the intersection between a view line and a polygon barrier is a
line. To remedy this, points were generated along the
intersection with view lines at 10 cm intervals i.e. the portion
of the view line which overlaps with any polygon barrier were
converted into a series of points. The value of 10 cm was
arbitrary chosen as an optimal balance between resolution and
computation time, after trialing multiple other values (1 cm, 5
cm, 25 cm). These generated points were then counted for the
feature count. One point to note is that this method results in
the generation of multiple points per view line, unlike other
layers which may only have single intersection points per
feature. This, however, does not affect the factors, as each
layer is considered separately during the calculation.
F. Factors for the layout of the map
1) Angular dispersion of features
Angular dispersion is a measure of the arrangement of
features around the sample point. Akin to Javanmardi et al.’s
[4] Feature DOP measure (which in turn was inspired by
GDOP in global positioning systems), this factor describes the
uniformity of dispersion and is calculated as follows:
 =
sin
+
cos
where α is the azimuth of the features, in radians. The factor
returns a value between 0 and 1, with a value of 0 indicating a
uniform dispersion and a value of 1 meaning a complete
concentration in one direction. For this factor, the dispersion
of the intersection points between the view lines or view
polygons and the respective layer was calculated.
2) View line mean length
For each sample point, the mean length for every view line
which intersected with the building layer was calculated. Non-
intersecting view lines were not considered as they would
skew the metric.
3) Area of view polygon
This is the area of the view polygon. It represents the
theoretical area that the laser scanner can ‘see’. A small area
suggests that there are lots of building features in the vicinity
to localize against.
Key
!
Sample point
OSM Buildings
Generated view lines (20 m)
Clipped view lines
View polygon
! !
!
±
(a) (b)
(c)

Citations
More filters
Proceedings ArticleDOI
19 Sep 2021
TL;DR: The CommonRoad toolbox as mentioned in this paper provides converters for OpenStreetMap, Lanelet/Lanelet2, OpenDRIVE, and SUMO, as well as a graphical user interface that allows one to efficiently create and manipulate CommonRoad maps and scenarios.
Abstract: Maps are essential for testing autonomous driving functions. Several map and scenario formats are available. However, they are usually not compatible with each other, limiting their usability. In this paper, we address this problem using our open-source toolbox that provides map converters from different formats to the well-known CommonRoad format. Our toolbox provides converters for OpenStreetMap, Lanelet/Lanelet2, OpenDRIVE, and SUMO. Additionally, a graphical user interface is included, which allows one to efficiently create and manipulate CommonRoad maps and scenarios. We demonstrate the functionality of the toolbox by creating CommonRoad maps and scenarios based on other map formats and manually-created map data.

4 citations

References
More filters
Proceedings ArticleDOI
03 May 2010
TL;DR: This work proposes an extension to this approach to vehicle localization that yields substantial improvements over previous work in vehicle localization, including higher precision, the ability to learn and improve maps over time, and increased robustness to environment changes and dynamic obstacles.
Abstract: Autonomous vehicle navigation in dynamic urban environments requires localization accuracy exceeding that available from GPS-based inertial guidance systems. We have shown previously that GPS, IMU, and LIDAR data can be used to generate a high-resolution infrared remittance ground map that can be subsequently used for localization [4]. We now propose an extension to this approach that yields substantial improvements over previous work in vehicle localization, including higher precision, the ability to learn and improve maps over time, and increased robustness to environment changes and dynamic obstacles. Specifically, we model the environment, instead of as a spatial grid of fixed infrared remittance values, as a probabilistic grid whereby every cell is represented as its own gaussian distribution over remittance values. Subsequently, Bayesian inference is able to preferentially weight parts of the map most likely to be stationary and of consistent angular reflectivity, thereby reducing uncertainty and catastrophic errors. Furthermore, by using offline SLAM to align multiple passes of the same environment, possibly separated in time by days or even months, it is possible to build an increasingly robust understanding of the world that can be then exploited for localization. We validate the effectiveness of our approach by using these algorithms to localize our vehicle against probabilistic maps in various dynamic environments, achieving RMS accuracy in the 10cm-range and thus outperforming previous work. Importantly, this approach has enabled us to autonomously drive our vehicle for hundreds of miles in dense traffic on narrow urban roads which were formerly unnavigable with previous localization methods.

615 citations


"Evaluating the Capability of OpenSt..." refers methods in this paper

  • ...To improve map-based localization accuracy, research so far has focused on refining the map matching algorithms [2] as well as producing increasingly accurate High Definition maps [3]....

    [...]

01 Jan 2009
TL;DR: This dissertation extends the original two-dimensional NDT registration algorithm of Biber and Straser to 3D and introduces a number of improvements and proposes to use a combination of local visual features and Colour-NDT for robust registration of coloured 3D scans.
Abstract: This dissertation is concerned with three-dimensional (3D) sensing and 3D scan representation. Three-dimensional records are important tools in several disciplines; such as medical imaging, archaeology, and mobile robotics. This dissertation proposes the normal-distributions transform, NDT, as a general 3D surface representation with applications in scan registration, localisation, loop detection, and surface-structure analysis. After applying NDT, the surface is represented by a smooth function with analytic derivatives. This representation has several attractive properties. The smooth function representation makes it possible to use standard numerical optimisation methods, such as Newton’s method, for 3D registration. This dissertation extends the original two-dimensional NDT registration algorithm of Biber and Straser to 3D and introduces a number of improvements. The 3D-NDT scan-registration algorithm is compared to current de facto standard registration algorithms. 3D-NDT scan registration with the proposed extensions is shown to be more robust, more accurate, and faster than the popular ICP algorithm. An additional benefit is that 3D-NDT registration provides a confidence measure of the result with little additional effort. Furthermore, a kernel-based extension to 3D-NDT for registering coloured data is proposed. Approaches based on local visual features typically use only a small fraction of the available 3D points for registration. In contrast, Colour-NDT uses all of the available 3D data. The dissertation proposes to use a combination of local visual features and Colour-NDT for robust registration of coloured 3D scans. Also building on NDT, a novel approach using 3D laser scans to perform appearance-based loop detection for mobile robots is proposed. Loop detection is an importantproblem in the SLAM (simultaneous localisation and mapping) domain. The proposed approach uses only the appearance of 3D point clouds to detect loops and requires nopose information. It exploits the NDT surface representation to create histograms based on local surface orientation and smoothness. The surface-shape histograms compress the input data by two to three orders of magnitude. Because of the high compression rate, the histograms can be matched efficiently to compare the appearance of two scans. Rotation invariance is achieved by aligning scans with respect to dominant surface orientations. In order to automatically determine the threshold that separates scans at loop closures from nonoverlapping ones, the proposed approach uses expectation maximisation to fit a Gamma mixture model to the output similarity measures. In order to enable more high-level tasks, it is desirable to extract semantic information from 3D models. One important task where such 3D surface analysis is useful is boulder detection for mining vehicles. This dissertation presents a method, also inspired by NDT, that provides clues as to where the pile is, where the bucket should be placed for loading, and where there are obstacles. The points of 3D point clouds are classified based on the surrounding surface roughness and orientation. Other potential applications include extraction of drivable paths over uneven surfaces.

313 citations


"Evaluating the Capability of OpenSt..." refers background in this paper

  • ...In general, Normal Distribution Transform (NDT) is more resilient to sensor noise and sampling resolution (than other algorithms such as Iterative Closest Point) but suffers from slower convergence speed [8]....

    [...]

01 Jan 2010
TL;DR: The strong demand for freely available spatial data, though, has boosted the number of VGI available on the Internet and one of the most complex and promising projects in recent years is OpenStreetMap (OSM).
Abstract: In connection with the Web 2.0 movement of the Internet (O’Reilly, 2005) and the progressive development of tools and applications for the collection and provision of spatial information (Turner, 2006), the quality and quantity of so-called Volunteered Geographic Information (VGI) (Goodchild, 2007) underwent a fast-paced worldwide development. Some even speak of a “Wikification of GIS” (Sui, 2008). This spatial data, mostly collected by volunteers, is freely available for the Internet user and can (under certain licensing conditions) be applied to numerous GIS projects and applications. Through advanced data donations, but also by a variety of other non-proprietary data sources, some of these free data providers are able to offer a vast variety of different information. This development in recent years stands in strong contrast to the very expensive commercial spatial data provided by a few companies. Much of this proprietary data is widely used today, for example, in car navigation devices or cell phones. The strong demand for freely available spatial data, though, has boosted the number of VGI available on the Internet. They can be found in very simple forms such as in Wikipedia entries that provide some spatial information like lat-long coordinates (geotag), or in so-called mashups in Google Earth or Google Maps, which combine different information sources. One of the most complex and promising projects in recent years, however, is OpenStreetMap (OSM).

313 citations


"Evaluating the Capability of OpenSt..." refers background in this paper

  • ...In fact, in certain areas, OSM is more complete and more accurate (for both location and semantic information) than corresponding proprietary datasets [12], [13]....

    [...]

Journal ArticleDOI
10 Aug 2017-PLOS ONE
TL;DR: Two complementary, independent methods are used to assess the completeness of OSM road data in each country in the world and find that globally, OSM is ∼83% complete, and more than 40% of countries—including several in the developing world—have a fully mapped street network.
Abstract: OpenStreetMap, a crowdsourced geographic database, provides the only global-level, openly licensed source of geospatial road data, and the only national-level source in many countries. However, researchers, policy makers, and citizens who want to make use of OpenStreetMap (OSM) have little information about whether it can be relied upon in a particular geographic setting. In this paper, we use two complementary, independent methods to assess the completeness of OSM road data in each country in the world. First, we undertake a visual assessment of OSM data against satellite imagery, which provides the input for estimates based on a multilevel regression and poststratification model. Second, we fit sigmoid curves to the cumulative length of contributions, and use them to estimate the saturation level for each country. Both techniques may have more general use for assessing the development and saturation of crowd-sourced data. Our results show that in many places, researchers and policymakers can rely on the completeness of OSM, or will soon be able to do so. We find (i) that globally, OSM is ∼83% complete, and more than 40% of countries-including several in the developing world-have a fully mapped street network; (ii) that well-governed countries with good Internet access tend to be more complete, and that completeness has a U-shaped relationship with population density-both sparsely populated areas and dense cities are the best mapped; and (iii) that existing global datasets used by the World Bank undercount roads by more than 30%.

259 citations


"Evaluating the Capability of OpenSt..." refers background in this paper

  • ...example, Barrington-Leigh and Millard-Ball [10] estimate the global OSM street layer to be ∼83% complete, with more than 40% of countries having fully mapped networks....

    [...]

  • ...For example, Barrington-Leigh and Millard-Ball [10] estimate the global OSM street layer to be ∼83% complete, with more than 40% of countries having fully mapped networks....

    [...]

Proceedings ArticleDOI
01 Jun 2017
TL;DR: A localization approach that is based on a point-cloud matching method (normal distribution transform “NDT”) and road-marker matching based on the light detection and ranging intensity and a particle-filtering algorithm is presented.
Abstract: In this paper, we present a localization approach that is based on a point-cloud matching method (normal distribution transform “NDT”) and road-marker matching based on the light detection and ranging intensity. Point-cloud map-based localization methods enable autonomous vehicles to accurately estimate their own positions. However, accurate localization and “matching error” estimations cannot be performed when the appearance of the environment changes, and this is common in rural environments. To cope with these inaccuracies, in this work, we propose to estimate the error of NDT scan matching beforehand (off-line). Then, as the vehicle navigates in the environment, the appropriate uncertainty is assigned to the scan matching. 3D NDT scan matching utilizes the uncertainty information that is estimated off-line, and is combined with a road-marker matching approach using a particle-filtering algorithm. As a result, accurate localization can be performed in areas in which 3D NDT failed. In addition, the uncertainty of the localization is reduced. Experimental results show the performance of the proposed method.

84 citations


"Evaluating the Capability of OpenSt..." refers methods in this paper

  • ...For both studies [4], [6], the estimation of localization error used 3D point cloud and ND maps....

    [...]

  • ...Further, the estimated error model can be used in conjunction with other sensor information to improve the accuracy of localization [6]....

    [...]

  • ...[6] estimate the error of 3D NDT scan matching in a pre-experiment, which was then subsequently used in the localization phase for pose correction....

    [...]

Frequently Asked Questions (2)
Q1. What have the authors contributed in "Evaluating the capability of openstreetmap for estimating vehicle localization error" ?

This paper explores the potential of using OpenStreetMap ( OSM ) as a proxy to estimate vehicle localization error. 

Future work is required in adding more data layers from OSM, as well as other layers which may not currently be available, such as curb information, road markings, and landmarks.