An Overview of Routing Optimization for
Internet Traffic Engineering
Ning Wang, Kin Hon Ho, George Pavlou and Michael Howarth
Abstract
Traffic Engineering (TE) is an important mechanism for Internet Network Providers (INPs)
seeking to optimize network performance and traffic delivery. Routing optimization plays a
key role in traffic engineering, finding efficient routes so as to achieve the desired network
performance. In this article, we review Internet traffic engineering from the perspective of
routing optimization. We provide a taxonomy of routing algorithms in the literature, dating
from the advent of the TE concept in the late 1990s. We classify the algorithms into multiple
dimensions, namely unicast/multicast, intra-/inter-domain, IP-/MPLS-oriented and
offline/online TE schemes. In addition, we investigate some important traffic engineering
issues, including robustness, TE interactions and interoperability with overlay selfish routing.
While revisiting the existing solutions, we also point out some important issues that are
worthy of investigation in future research activities.
1. Introduction
The Internet is currently experiencing a transition from point-to-point Best Effort (BE)
communications towards a multi-service network that supports many types of multimedia
applications, with potentially high bandwidth demand. Thanks to the rapid development of
communication network hardware, adding physical resources (e.g., fast-speed switching and
routing elements, high capacity network links etc.) to the existing Internet has become
relatively cheap in recent years. Typically, the advent of increasingly high-speed links has
offered opportunities for IP Network Providers (INPs) to adopt a strategy of bandwidth over-
provisioning in their networks. Nevertheless, this approach is currently only applicable to the
core network, and the demand from sharply growing customer traffic over the global Internet
still cannot be satisfied. The measurement results presented in [1] indicate that bottlenecks of
the Internet backbone are not only located at inter-domain links between Autonomous
Systems (ASes), but also within individual domains. Given this information, it is essential for
INPs to perform efficient resource optimization both intra- and inter-domain so as to
eliminate these bottlenecks. Internet Traffic Engineering (TE) is the process of performing
this task. In [2], TE is defined as large-scale network engineering for dealing with IP network
performance evaluation and optimization. A more straightforward explanation of TE is also
given in [3]: “to put the traffic where the network bandwidth is available”. From this
statement, we note that the fundamental task of traffic engineering is to perform appropriate
route selection such that the given bandwidth capacity is able to support maximum customer
traffic without causing network congestion. From this perspective, the nature of traffic
engineering is effectively a routing optimization for enhancing network service capability.
Figure 1 illustrates this with a simple TE example. We assume that the bandwidth capacity of
each link is 10Mbps, and there are three individual customer flows injected at node A,
heading towards node C. If conventional shortest path routing is applied, all the customer
flows are routed on the direct link A-C, thus causing the link utilization to be as high as 180%
(
10/36
×
). On the other hand, if the three flows are routed through different paths, as shown
in Figure 1(b), the total traffic within the network is evenly distributed without causing link
congestion. As this example illustrates, routing optimization that uses alternative multiple
paths other than conventional shortest path based approaches can be an effective means to
improving the network service capability.
Two major issues that have recently received attention in TE approaches are Quality of
Service (QoS) and resilience. First, many of the new multimedia applications not only have
bandwidth requirements, but also require other QoS guarantees, such as end-to-end delay,
jitter or packet loss probability. These QoS requirements impose new challenges on INPs’
traffic engineering in that the end-to-end QoS demands need to be satisfied through TE
mechanisms. Second, given the fact that network node and link failure are still frequent
events on the Internet, TE solutions have to consider how to minimize the impact of failures
on the network performance and resource utilization. There exists a large amount of work in
the literature on QoS routing and path protection/restoration respectively. In order to restrict
the scope of our survey, it is worth clarifying the relationship and difference between TE and
QoS routing / resilience schemes. According to [4], TE objectives can be classified into
traffic-oriented and resource-oriented. Most QoS-aware and resilience-aware TE schemes
belong to the traffic-oriented category, which puts more emphasis on improving the
performance perceived by the customer sending traffic. According to this criterion, if a QoS
routing scheme is implemented exclusively from a customer’s viewpoint without considering
global network optimization, then it is known as selfish routing; we do not consider this in
this article, although we note that a comprehensive survey on QoS aware selfish routing can
be found in [5]. As far as resilience is concerned, the objective is to avoid sub-optimal
resource utilization (resource oriented) and negative impacts on traffic delivery (traffic
oriented) in case of link/node failure. We will discuss detailed robustness-aware TE solutions
in section 6.
Figure 1. A simple TE example
Many papers have been published in the area of routing optimization. As a result, it is by no
means an easy task to classify various TE solutions, and present a comprehensive and clear
survey. In this paper, we classify these TE routing approaches according to four orthogonal
criteria: (1) traffic optimization scope, (2) routing enforcement mechanism, (3) time/state
dependence or availability of traffic demand and (4) traffic type. First of all, TE can be
classified into two categories according to the scope: intra-domain and inter-domain. In intra-
domain TE, optimization focuses on how to control traffic routing within a single AS. In
contrast, inter-domain TE considers how to optimize traffic that travels across multiple ASes.
Inter-domain TE paradigms can be generally classified into two categories. The first, which
has been extensively addressed is how to control inter-domain traffic within the local AS, e.g.,
to find optimal ingress/egress points for inter-domain traffic that is injected into or delivered
out of the local domain. The second category, which has not yet been well studied, considers
“end-to-end” TE optimization across multiple ASes. In this scenario, individual ASes may
need cooperation with each other in order to deliver the traffic over the desired inter-domain
routes. Second, from the perspective of routing enforcement, there exist two distinct TE
mechanisms, IP-based and MPLS-based. For IP-based TE, routing is optimized by adjusting
the routing parameters of the underlying IP routing protocols such as OSPF/ISIS and BGP.
On the other hand, MPLS-based TE adopts packet encapsulation and explicit routing with
dedicated Label Switching Paths (LSPs). Third, traffic engineering can be categorized into
offline and online. In offline TE, all traffic demands from customers are assumed to be known
a priori to some greater or lesser extent, and the TE task is then tp efficiently map the
predicted traffic demand onto the physical network. In contrast, for the online TE case, the
INP needs to perform lightweight and efficient path selection one by one for each incoming
flow, without knowing any traffic demand in advance. Finally, we should mention that
Internet traffic consists of different types of flows, such as IP unicast, multicast and various
types of overlay traffic such as Virtual Private Network (VPN) and Content Distribution
Network (CDN) flows. Routing optimization of these different traffic types may require
different solutions. In this paper we will survey not only the common unicast TE, but also
multicast TE which is emerging as a popular approach given recent progress in IP multicast.
To summarize, an overall taxonomy of Internet traffic engineering is presented in Figure 2,
and this article is organized following the structure of this diagram. The objective of this
article is thus to provide a comprehensive survey on routing optimization for all the
components in the TE hierarchy. The rest of the article is organized as follows. We specify in
Section 2 the detailed characteristics of different types of TE according to Figure 2. In Section
3 we introduce intra-domain traffic engineering, which includes both MPLS and IP-based
routing optimization algorithms. In Section 4 we move on to inter-domain traffic engineering,
which we further divide into inbound and outbound TE. In Section 5, multicast traffic
engineering is presented. We then discuss in Section 6 some important interactions between
current traffic engineering approaches. Finally we provide a summary in Section 7. It is worth
mentioning that this survey does not claim to be exhaustive, although we attempt not to omit
any important works in the area.
Traffic Engineering
Unicast TE Multicast TE
Intra-domain
Inter-domain
MPLS oriented
IP oriented
Offline
Online
Figure 2. Hierarchical classification of Internet traffic engineering
2. Traffic Engineering Classifications
2.1 Intra-domain TE vs. Inter-domain TE
The task of intra-domain traffic engineering is to optimize customer traffic routing between
AS border routers (ASBRs) within a single domain. In comparison, inter-domain traffic
engineering deals with the problem of optimizing inter-domain traffic traveling across
multiple ASes. As mentioned above, most of the existing literature focuses on how to select
ASBRs optimally as the ingress/egress points for inter-domain traffic that travels across the
local AS. That is to say, if the traffic has multiple potential ASBRs from which it can enter or
leave the local domain, then the problem of inter-domain TE for an INP is: “which ASBR(s)
should be used as the ingress/egress point(s) for routing the traffic through the local network,
so that the network resource utilization is optimized?” According to the control over how
traffic enters/leaves the domain, inter-domain traffic engineering can be further classified into
inbound TE and outbound TE. Figure 3 presents a simple example to illustrate the difference
between intra- and inter-domain traffic engineering semantics, specifically using outbound
traffic engineering as an example for inter-domain TE. We assume that traffic destined to the
remote prefix 20.20.20.0/24 (AS200) is injected into the local AS (AS100, 10.10.10.0/24) via
ASBR 10.10.10.3, and both the internal peers 10.10.10.1 and 10.10.10.2 can provide a route
to AS200 (i.e., both routers receive reachability information towards 20.20.20.0/24 through
external BGP advertisements). In this scenario, the decision to use ASBR 10.10.10.1 or
10.10.10.2 (or both for load balancing with inter-domain multiple paths) as the egress point is
the task of inter-domain/outbound TE. Once the egress point has been selected, say ASBR
10.10.10.1, intra-domain traffic engineering is then responsible for selecting the best intra-
domain path between each pair of ASBRs in the network. In this simple example, intra-
domain TE attempts to find an optimal internal path (or multiple paths if intra-domain multi-
paths are allowed) from ASBR 10.10.10.3 to ASBR 10.10.10.1 as well as an optimal path C
from 10.10.10.3 to ASBR 10.10.10.2.
Despite their clear difference in definition, intra- and inter-domain traffic engineering should
not be considered independently of each other in practice, since the network configuration of
one could potentially impact the other. Research has emerged recently on the interaction
between the two types of TE, and some results are presented in [6][7]. We will provide more
details on the interaction between intra- and inter-domain traffic engineering in Section 6.2.
Figure 3. Intra- and Inter-domain traffic engineering
2.2 MPLS based TE vs. IP based TE
The concept of traffic engineering was first introduced in Multi-Protocol Label Switching
(MPLS) based environments [4][8]. By intelligently setting up dedicated Label Switched
Paths (LSPs) for delivering encapsulated IP packets, MPLS oriented traffic engineering can
provide an efficient paradigm for traffic optimization. The most distinct advantage of MPLS
oriented TE is its capability of explicit routing and arbitrary splitting of traffic, which is
highly flexible for both routing and forwarding optimization purposes. However, since traffic
trunks are delivered through dedicated LSPs, scalability and robustness become issues in
MPLS oriented TE. First, the total number of LSPs (assuming full mesh or equivalent) within
a domain is O(N
2
) where N is the number of ASBRs. This means that the overhead of setting
up LSPs can be very high in large-size networks. In addition, path protection mechanisms
(e.g., using backup paths) are necessary in MPLS oriented TE, as otherwise traffic cannot be
automatically delivered through alternative paths in case of any link failure in active LSPs.
The first IP-based traffic engineering solution was proposed by Fortz et al [9]-[11]. The basic
idea of their approach is to set the link weights of Interior Gateway Protocols (IGP) according
to the given network topology and traffic demand, so as to control intra-domain traffic and
meet TE objectives. More recently, schemes that manipulate BGP routing attributes, known
as BGP tweaking [12], have also been proposed for inter-domain traffic engineering. In
comparison to the MPLS-based approach, these IP-based TE solutions lack flexibility in path
selection, since explicit routing and uneven traffic splitting are not supported. However, the
IP-based approach has better scalability and failure resilience than MPLS-based TE, because
no overhead for dedicated LSPs is required, and also because traffic can be automatically
delivered via alternative shortest paths in case of link failure, without explicitly provisioning
backup paths. However, given this type of auto-rerouting in the IP based environment, recent
research work [13] has suggested that a single link failure can introduce dramatic changes to
traffic distribution even across multiple domains, as a significant proportion of traffic will
switch to new shortest paths once the network topology has changed. This low TE robustness
is in comparison to the MPLS TE schemes, where a single link failure does not impact other
primary LSPs unless they are using the faulty link. Table 1 summarizes the key differences
between MPLS-based TE and IP-based TE.
MPLS oriented TE IP oriented TE
Routing mechanism
Explicit routing with packet
encapsulation
Plain IGP/BGP based routing
Routing optimization
Constraint based routing (CBR) IGP link weight adjustment
BGP route attribute adjustment
Multi-path forwarding
Arbitrary traffic splitting Even traffic splitting only
Hardware requirement
MPLS capable routers required Conventional IP routers
Route Selection flexibility
More flexible - arbitrary path Less flexible – shortest path only
Scalability (overhead in
maintaining network state)
Less scalable More scalable, with scalability of
underlying routing protocol
Failure impact on traffic
delivery
High (normally need backup
paths in case of failures)
Low
Failure impact on TE
performance
Low High
Table 1. MPLS/IP TE comparison
2.3 Offline TE vs. Online TE
The third part of our taxonomy is to classify traffic engineering into offline and online. As
previously mentioned, the principal difference between offline and online traffic engineering
is the availability of a traffic matrix (TM). The concept of traffic matrix was originally
associated with intra-domain TE, where ingress/egress points of traffic are fixed. In this case,
the overall traffic demand on the network can be represented by a matrix TM, e.g., with each
element t(i,j) of the TM being the total bandwidth demand of all individual traffic flows
(known as traffic trunk) from ingress node i to egress node j. When inter-domain traffic
engineering is concerned, ingress/egress nodes for a traffic trunk might not be specified;
instead the traffic is from some source (e.g an AS) to some destination (e.g. represented by a
destination address or by a next-hop AS or a destination AS).
In some scenarios it is possible for an INP to forecast the traffic matrix before routing
optimization is performed. Currently, there exist two principal inputs from which traffic
matrix can be forecast: a Service Level Specification (SLS) and monitoring/measurement.
