Mapping Peering Interconnections to a Facility
Vasileios Giotsas
CAIDA / UC San Diego
vgiotsas@caida.org
Georgios Smaragdakis
MIT / TU Berlin
gsmaragd@csail.mit.edu
Bradley Huffaker
CAIDA / UC San Diego
bhuffake@caida.org
Matthew Luckie
University of Waikato
mjl@wand.net.nz
kc claffy
CAIDA / UC San Diego
kc@caida.org
ABSTRACT
Annotating Internet interconnections with robust phys-
ical coordinates at the level of a building facilitates net-
work management including interdomain troubleshoot-
ing, but also has practical value for helping to locate
points of attacks, congestion, or instability on the In-
ternet. But, like most other aspects of Internet inter-
connection, its geophysical locus is generally not pub-
lic; the facility used for a given link must be inferred to
construct a macroscopic map of peering. We develop a
methodology, called constrained facility search, to infer
the physical interconnection facility where an intercon-
nection occurs among all possible candidates. We rely
on publicly available data about the presence of net-
works at different facilities, and execute traceroute mea-
surements from more than 8,500 available measurement
servers scattered around the world to identify the tech-
nical approach used to establish an interconnection. A
key insight of our method is that inference of the tech-
nical approach for an interconnection sufficiently con-
strains the number of candidate facilities such that it
is often possible to identify the specific facility where
a given interconnection occurs. Validation via private
communication with operators confirms the accuracy
of our method, which outperforms heuristics based on
naming schemes and IP geolocation. Our study also re-
veals the multiple roles that routers play at interconnec-
tion facilities; in many cases the same router implements
both private interconnections and public peerings, in
some cases via multiple Internet exchange points. Our
study also sheds light on peering engineering strategies
used by different types of networks around the globe.
Permission to make digital or hard copies of all or part of this work for personal
or classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice
and the full citation on the first page. Copyrights for components of this work
owned by others than ACM must be honored. Abstracting with credit is per-
mitted. To copy otherwise, or republish, to post on servers or to redistribute to
lists, requires prior specific permission and/or a fee. Request permissions from
permissions@acm.org.
CoNEXT ’15 December 01-04, 2015, Heidelberg, Germany
c
2015 ACM. ISBN 978-1-4503-3412-9/15/12. . . $15.00
DOI: 10.1145/2716281.2836122
CCS Concepts
•Networks → Network measurement; Physical topolo-
gies;
Keywords
Interconnections; peering facilities; Internet mapping
1. INTRODUCTION
Measuring and modeling the Internet topology at the
logical layer of network interconnection, i. e., autonomous
systems (AS) peering, has been an active area for nearly
two decades. While AS-level mapping has been an im-
portant step to understanding the uncoordinated forma-
tion and resulting structure of the Internet, it abstracts
a much richer Internet connectivity map. For example,
two networks may interconnect at multiple physical lo-
cations around the globe, or even at multiple locations
in the same city [55, 49].
There is currently no comprehensive mapping of in-
terconnections to physical locations where they occur [61].
There are good reasons for the dearth of this informa-
tion: evolving complexity and scale of networking in-
frastructure, information hiding properties of the rout-
ing system (BGP), security and commercial sensitivi-
ties, and lack of incentives to gather or share data. But
this opacity of the Internet infrastructure hinders re-
search and development efforts as well as network man-
agement. For example, annotating peering interconnec-
tions with robust physical coordinates at the level of a
building facilitates network troubleshooting and diag-
nosing attacks [43] and congestion [46]. Knowledge of
geophysical locations of interconnections also enables
assessment of the resilience of interconnections in the
event of natural disasters [53, 20], facility or router out-
ages [6], peering disputes [46], and denial of service at-
tacks [22, 60]. This information can also elucidate the
role of emerging entities, e. g., colocation facilities, car-
rier hotels, and Internet exchange points (IXP), that
enable today’s richly interconnected ecosystem [44, 7,
18, 19, 15]. It also increases traffic flow transparency,
e. g., to identify unwanted transit paths through specific
countries, and inform peering decisions in a competitive
interconnection market.
In this paper we describe a measurement and infer-
ence methodology to map a given interconnection to
a physical facility. We first create and update a de-
tailed map of interconnection facilities and the networks
present at them. This map requires manual assembly,
but fortunately the information is increasingly publicly
available in recent years, partly due to the fact that
many networks require it be available in order to es-
tablish peering [4], and many IXPs publish information
about which networks are present at which of their fa-
cilities in order to attract participating networks [18].
Interconnection facilities also increasingly make the list
of participant members available in their website or in
PeeringDB [45]. While it is a substantial investment of
time to keep such a list current, we find it is feasible.
However, a well-maintained mapping of networks to
facilities does not guarantee the ability to accurately
map all interconnections involving two ASes to specific
physical facilities, since many networks peer at mul-
tiple locations even within a city. Mapping a single
interconnection to a facility is a search problem with
a potentially large solution space; however, additional
constraints can narrow the search. The contributions of
this work are as follows:
• We introduce and apply a measurement method-
ology, called constrained facility search, which in-
fers the physical facilities where two ASes intercon-
nect from among all (sometimes dozens of) possi-
ble candidates, and also infers the interconnection
method, e.g. public peering at an IXP, private
peering via cross-connect, point-to-point connec-
tion tunnelled over an IXP, or remote peering.
• We validate the accuracy of our methodology using
direct feedback, BGP communities, DNS records,
and IXP websites and find our algorithm achieves
at least 90% accuracy for each type of interconnec-
tion and outperforms heuristics based on naming
schemes and IP geolocation.
• We demonstrate our methodology using case stud-
ies of a diverse set of interconnections involving
content providers (Google, Akamai, Yahoo, Lime-
light, and Cloudflare) as well as transit providers
(Level3, Cogent, Deutsche Telekom, Telia, and NTT).
Our study reveals the multiple roles that routers
play at interconnection facilities; frequently the
same router implements both public and private
peering in some cases via multiple facilities.
2. BACKGROUND AND TERMINOLOGY
Interconnection is a collection of business practices
and technical mechanisms that allows individually man-
aged networks (ASes) to exchange traffic [11]. The
two primary forms of interconnection are transit, when
one AS sells another ISP access to the global Internet,
and peering, when two ISPs interconnect to exchange
customer traffic, although complicated relationships ex-
ist [28, 31]. Whether and how to interconnect requires
careful consideration, and depends on traffic volume ex-
changed between the networks, their customer demo-
graphics, peering and security policies, and the cost to
maintain the interconnection [50].
Interconnection Facility. An interconnection fa-
cility is a physical location (a building or part of one)
that supports interconnection of networks. These facil-
ities lease customers secure space to locate and operate
network equipment. They also provide power, cooling,
fire protection, dedicated cabling to support different
types of network connection, and in many cases admin-
istrative support. Large companies such as Equinix [27],
Telehouse [59], and Interxion [36] operate such facilities
around the globe. Smaller companies operate intercon-
nection facilities in a geographic region or a city. Most
interconnection facilities are carrier-neutral, although
some are operated by carriers, e. g., Level3. In large
communication hubs, such as in large cities, an intercon-
nection facility operator may operate multiple facilities
in the same city, and connect them, so that networks
participating at one facility can access networks at an-
other facility in the same city.
Internet Exchange Point. An Internet Exchange
Point (IXP) is a physical infrastructure composed of
layer-2 Ethernet switches where participating networks
can interconnect their routers using the switch fabric.
At every IXP there is one or more (for redundancy)
high-end switches called core switches (the center switch
in Figure 1). IXPs partner with interconnection facili-
ties in the city they operate and install access switches
there (switches at facilities 1 and 2 in Figure 1). These
switches connect via high bandwidth connections to the
core switches. In order to scale, some IXPs connect mul-
tiple access switches to back-haul switches. The back-
haul switch then connects to the core switch. All IXP
members connected to the same access switch or back-
haul switch exchange traffic locally if they peer; the
rest exchange traffic via the core switch. Thus, routers
owned by members of IXPs may be located at different
facilities associated with the same IXP [18].
Popular peering engineering options today are:
Private Peering with Cross-connect. A cross-
connect is a piece of circuit-switched network equipment
that physically connects the interfaces of two networks
at the interconnection facility. It can be either copper
or fiber with data speeds up to tens of Gbps. Cross-
connects can be established between members that host
their network equipment in different facilities of the
same interconnection facility operator, if these facilities
are interconnected. The downside of a cross-connect is
operational overhead: it is largely a manual process to
establish, update, or replace one.
Some large facilities have thousands of cross-connects,
e. g., Equinix reported 161.7K cross-connects across all
its colocation facilities, with more than half in the Amer-
AS A
AS B
AS C
AS F
AS EAS D
FACILITY 1 FACILITY 2
FACILITY 3
IXPIXP
private peering
“tethering”
2.4.1 Private Peering
Cross Connect
private peering
cross connect
public peering
public peering
accessaccessaccessaccess
core
core
2.4.2Remote Peeringremote peering
tunneled
direct
Figure 1: Interconnection facilities host routers of many different networks and partner with IXPs to support different
types of interconnection, including cross-connects (private peering with dedicated medium), public peering (peering
established over shared switching fabric), tethering (private peering using VLAN on shared switching fabric), and
remote peering (transport to IXP provided by reseller).
icas (Q2 2015) [3]. Cross-connects are installed by the
interconnection facilities but members of IXPs can or-
der cross-connects via the IXP for partnered intercon-
nection facilities, in some cases with a discount. For ex-
ample DE-CIX in Frankfurt has facilitated more than
900 cross-connects as of February 2015 [2].
Public Peering. Public peering, also referred to
as public interconnect, is the establishment of peering
connections between two members of an IXP via the
IXP’s switch fabric. IXPs are allocated IP prefix(es)
and often an AS number by a Regional Internet Reg-
istry. The IXP assigns an IP from this range to the
IXP-facing router interfaces of its IXP members to en-
able peering over its switch fabric [10]. One way to es-
tablish connectivity between two ASes is to establish a
direct BGP session between two of their respective bor-
der routers. Thus, if two IXP member ASes wanted to
exchange traffic via the IXP’s switching fabric, they es-
tablish a bi-lateral BGP peering session at the IXP. An
increasing number of IXPs offer their members the use
of route server to establish multi-lateral peering to sim-
plify public peering [32, 54]. With multi-lateral peering
an IXP member establishes a single BGP session to the
IXP’s route server and receives routes from other partic-
ipants using the route server. The advantage of public
peering is that by leasing one IXP port it is possible to
exchange traffic with potentially a large fraction of the
IXP members [57].
Private Interconnects over IXP. An increasing
number of IXPs offer private interconnects over their
public switch fabric. This type of private peering is also
called tethering or IXP metro VLAN. With tethering, a
point-to-point virtual private line is established via the
already leased port to reach other members of the IXP
via a virtual local area network (VLAN), e. g., IEEE
802.1Q. Typically there is a setup cost. In some cases
this type of private interconnect enables members of an
IXP to privately reach networks located in other facili-
ties where those members are not present, e. g., transit
providers or customers, or to privately connect their in-
frastructure across many facilities.
Remote Peering. Primarily larger IXPs, but also
some smaller ones, have agreements with partners, e. g.,
transport networks, to allow remote peering [14]. In
this case, the router of the remote peer can be located
anywhere in the world and connects to the IXP via an
Ethernet-over-MPLS connection. An advantage of re-
mote peering is that it does not require maintaining
network equipment at the remote interconnection fa-
cilities. Approximately 20% (and growing) of AMS-IX
participants were connected this way [18] in 2013. Re-
mote peering is also possible between a remote router
at the PoP of an ISP and a router present at an inter-
connection facility.
3. DATASETS AND MEASUREMENTS
To infer details of a given interconnection, we need
information about the prefixes of the two networks and
physical facilities where they are present. This section
describes the publicly available data that we collected
and analyzed for this study, and the publicly available
measurement servers (vantage points) we utilized.
3.1 Data Sources
3.1.1 Facility Information
For a given network we developed (and continue to
maintain to keep current) a list of the interconnection
facilities where it is present. Despite the fact that facili-
ties for commercial usage must be known to the network
operators to facilitate the establishment of new peering
links and to attract new customers, and in some cases it
is required to be public (e. g., for facilities that partner
with IXPs in Europe), the information is not available
in one form.
We started by compiling an AS-to-facilities mapping
using the list of interconnection facilities and associ-
ated networks (ASNs) available in PeeringDB [45]. Al-
0 20 40 60 80 100 120 140
AS
0.0
0.2
0.4
0.6
0.8
1.0
Fraction of facilities in PDB
10
0
10
1
10
2
10
3
Number of facilities
Total numbe r of facilitie s
Fr action in Pe e r ingDB
Figure 2: Number of interconnection facilities for 152
ASes extracted from their official website, and the as-
sociated fraction of facilities that appear in PeeringDB.
though this list is maintained on a volunteer basis (op-
erators contribute information for their own networks),
and may not be regularly updated for some networks,
it is the most widely used source of peering information
among operators, and it allows us to bootstrap our al-
gorithms. Due to its manual compilation process, there
are cases where different naming schemes are used for
the same city or country. To remove such discrepancies,
we convert country and city names to standard ISO and
UN names. If the distance between two cities is less
than 5 miles, we map them to the same metropolitan
area. We calculate the distance by translating the post-
codes of the facilities to geographical coordinates. For
example, we group Jersey City and New York City into
the NYC metropolitan area.
To augment the list of collected facilities, we extracted
colocation information from web pages of Network Op-
erating Centers (NOCs), where AS operators often doc-
ument their peering interconnection facilities. Extract-
ing information from these individual websites is a te-
dious process, so we did so only for the subset of net-
works that we encountered in our traceroutes and for a
network’s PeeringDB data did not seem to reflect the
geographic scope reported on the network’s own web
site. For example, we investigated cases where a NOC
web site identified a given AS as global but PeeringDB
listed facilities for that AS only in a single country.
Figure 2 summarizes the additional information ob-
tained from NOC websites. The gray bars show the
fraction of facilities found in PeeringDB. We checked
152 ASes with PeeringDB records, and found that Peer-
ingDB misses 1,424 AS-to-facility links for 61 ASes; for
4 of these ASes PeeringDB did not list any facility.
Interestingly, the ASes with missing PeeringDB infor-
mation provided detailed data on their NOC websites,
meaning that they were not intending to hide their pres-
ence.
3.1.2 IXP Information
London
New York
Paris
Frankfurt
Amsterdam
San Jose
Moscow
Los Angeles
Stockholm
Manchester
Miami
Berlin
Tokyo
Kiev
Sao Paulo
Vienna
Singapore
Auckland
Hong Kong
Melbourne
Montreal
Zurich
Prague
Seattle
Chicago
Dallas
Hamburg
Atlanta
Bucharest
Madrid
Milan
Duesseldorf
Sofia
St.Petersburg
0
5
10
15
20
25
30
35
40
45
Number of facilities
Figure 3: Metropolitan areas with at least 10 intercon-
nection facilities.
We use various publicly available sources to get an
up-to-date list of IXPs, their prefixes, and associated
interconnection facilities. This information is largely
available from IXP websites. We also use lists from
PeeringDB and Packet Clearing House (PCH). Useful
lists are provided by regional consortia of IXPs such
as Euro-IX (also lists IXPs in North America), Af-IX,
LAC-IX, and APIX that maintain databases for the af-
filiated IXPs and their members. Some IXPs may be
inactive; PCH regularly updates their list and annotates
inactive IXPs. To further filter out inactive IXPs, for
our study, we consider only IXPs that (i) we were able
to confirm the IXP IP address blocks from at least three
of these data sources, and (ii) we could associate at least
one active member from at least two of the above data
sources. We ended up with 368 IXPs in 263 cities in
87 countries. IXPs belonging to the same operators in
different cities may be different entries, e. g., DE-CIX
Frankfurt and DE-CIX Munich.
We then create a list of IXPs where a network is
a member, and annotate which facilities partner with
these exchange points. For private facilities, we use
PeeringDB data augmented with information available
at the IXP websites and databases of the IXP consortia.
We again encountered cases where missing IXP informa-
tion from PeeringDB was found on IXP websites. For
example, the PeeringDB record of the JPNAP Tokyo
I exchange does not list any partner colocation facil-
ities, while the JPNAP website lists two facilities [5].
Overall, we extracted additional data from IXP web-
sites for 20 IXPs that we encountered in traces but for
which PeeringDB did not list any interconnection fa-
cilities. PeeringDB was not missing the records of the
facilities, only their association with the IXPs.
By combining all the information we collected for fa-
cilities, we compiled a list of 1,694 facilities in 95 coun-
tries and 684 cities for April 2015. The regional dis-
tribution of the facilities is as follows: 503 in North
America, 860 in Europe, 143 in Asia, 84 in Oceania, 73
in South America, and 31 in Africa. Notice that these
facilities can be operated by colocation operators or by
carriers. Figure 3 shows the cities with at least 10 colo-
cation facilities. It is evident that for large metropoli-
tan areas the problem of pinpointing a router’s PoP at
the granularity of interconnection facility is consider-
ably more challenging than determining PoP locations
at a city-level granularity.
On average a metropolitan area has about 3 times
more interconnection facilities than IXPs. This hap-
pens because the infrastructure of IXPs is often dis-
tributed among several facilities in a city, or even across
neighboring cities, for purposes of redundancy and ex-
panded geographical coverage. For example, the topol-
ogy of DE-CIX in Frankfurt spans 18 interconnection
facilities. ASes tend to connect to more interconnec-
tion facilities than IXPs, with 54% of the ASes in our
dataset connected to more than one IXPs and 66% of
the ASes connected at more than one interconnection
facilities. This may be intuitive since connectivity to an
IXP requires presence at least one interconnection facil-
ity that partners with the IXP. However, we observe the
opposite behavior for a relatively small number of ASes
that use fewer than 10 interconnection facilities. This
behavior is consistent with two aspects of the peering
ecosystem: (i) an interconnection facility may partner
with multiple IXPs, so presence at one facility could al-
low connectivity to multiple IXPs, and (ii) remote peer-
ing allows connectivity to an IXP through an IXP port
reseller, in which case presence at an IXP does not nec-
essarily require physical presence at one of its partner
facilities. For instance, about 20% of all AMS-IX par-
ticipants connect remotely [18].
3.2 Vantage Points and Measurements
To perform targeted traceroute campaigns we used
publicly available traceroute servers, RIPE Atlas, and
looking glasses. We augmented our study with existing
daily measurements, from iPlane and CAIDA’s Archi-
pelago infrastructures, that in some cases had already
traversed interconnections we considered. Table 1 sum-
marizes characteristics of our vantage points.
RIPE Atlas. RIPE Atlas is an open distributed
Internet measurement platform that relies on measure-
ment devices connected directly to home routers, and
a smaller set of powerful measurement collectors (an-
chors) used for heavy measurements and synchroniza-
tion of the distributed measurement infrastructure. The
end-host devices can be scheduled to perform tracer-
oute, ping, and DNS resolution on the host. We em-
ployed ICMP Paris (supported by RIPE Atlas) tracer-
oute to mitigate traceroute artifacts caused by load bal-
ancing [9]. We also used existing public measurements
gathered previously by RIPE Atlas nodes (e. g., periodic
traceroute queries to Google from all Atlas nodes).
Looking Glasses. A looking glass provides a web-
based or telnet interface to a router and allows the
execution of non-privileged debugging commands. In
RIPE LGs iPlane Ark Total
Atlas unique
Vantage Pts. 6385 1877 147 107 8517
ASNs 2410 438 117 71 2638
Countries 160 79 35 41 170
Table 1: Characteristics of the four traceroute measure-
ment platforms we utilized.
many cases a looking glass provides access to routers in
different cities, as well multiple sites at the same city.
Many looking glasses are also colocated with IXPs. Of-
ten looking glass operators enforce probing limitations
through mandatory timeouts or by blocking users who
exceed the operator-supported probing rate. Therefore,
looking glasses are appropriate only for targeted queries
and not for scanning a large range of addresses. To con-
form to the probing rate limits, we used a timeout of 60
seconds between each query to the same looking glass.
We extracted publicly available and traceroute-capable
looking glasses from PeeringDB, traceroute.org [39], and
previous studies [41]. After filtering out inactive or oth-
erwise unavailable looking glasses, we ended up with
1877 looking glasses in 438 ASes and 472 cities includ-
ing many in members of IXPs and 21 offered by IXPs.
An increasing number of networks run public looking
glass servers capable of issuing BGP queries [32], e. g.,
“show ip bgp summary”, “prefix info”, “neighbor info”.
We identified 168 that support such queries and we used
them to augment our measurements. These types of
looking glasses allow us to list the BGP sessions estab-
lished with the router running the looking glass, and
indicate the ASN and IP address of the peering router,
as well as showing metainformation about the intercon-
nection, e. g., via BGP communities [31].
iPlane. The iPlane project [47] performs daily IPv4
traceroute campaigns from around 300 PlanetLab nodes.
iPlane employs Paris traceroute to target other Plan-
etLab nodes and a random fraction of the advertised
address space. We used two daily archives of traceroute
measurements, collected a week apart, from all the ac-
tive nodes at the time of our measurements.
CAIDA Archipelago (Ark). CAIDA maintains
Ark, a globally distributed measurement platform with
107 nodes deployed in 92 cities (as of May 2015 when
we gathered the data). These monitors are divided into
three teams, each of which performs Paris traceroutes
to a randomly selected IP address in every announced
/24 network in the advertised address space in about
2-3 days. We analyzed one dataset collected when we
performed the traceroute campaigns with RIPE Atlas
and the looking glasses.
Targeted traceroutes. It takes about 5 minutes
for a full traceroute campaign using more than 95% of
all active RIPE Atlas nodes for one target. The time
required by each looking glass to complete a traceroute
measurement to a single target depends on the number
of locations provided by each looking glass. The largest
![Figure 4: Overview of our interconnection-to-facility mapping process. Using a large set of traceroutes toward the target ASes, we first resolve IP addresses to routers [40], and map IP addresses to ASes [21]. We feed this data, along with a constructed list of interconnection facilities for each AS and IXP, to our Constrained Facility Search (CFS) algorithm. The CFS algorithm iteratively constrains the possible interconnection facilities for inferred peerings, using targeted traceroutes as necessary to narrow the possible facilities for a router until it converges.](/figures/figure-4-overview-of-our-interconnection-to-facility-mapping-2mcfivpd.webp)
