Coping with Episodic Connectivity in Heterogeneous
Networks
Rao Naveed Bin Rais
INRIA
Sophia Antipolis, France
nbrais@sophia.inria.fr
Thierry Turletti
INRIA
Sophia Antipolis, France
turletti@sophia.inria.fr
Katia Obraczka
UCSC
Santa Cruz, CA, USA
katia@cse.ucsc.edu
ABSTRACT
In this paper, we present an efficient message delivery mechanism
that enables distribution/dissemination of messages in an internet
connecting heterogeneous networks and prone to disruptions in
connectivity. We call our protocol MeDeHa (pronounced
“medea”) for Message Delivery in Heterogeneous, Disruption-
prone Networks. MeDeHa is complementary to the IRTF’s
Bundle Architecture: while the Bundle Architecture provides
storage above the transport layer in order to enable
interoperability among networks that support different types of
transport layers, MeDeHa stores data at the link layer addressing
heterogeneity at lower layers (e.g., when intermediate nodes do
not support higher-layer protocols). MeDeHa also takes
advantage of network heterogeneity (e.g., nodes supporting more
than one network) to improve message delivery. For example, in
the case of IEEE 802.11 networks, participating nodes may use
both infrastructure- and ad hoc modes to deliver data to otherwise
unavailable destinations. Another important feature of MeDeHa is
that there is no need to deploy special-purpose nodes such as
message ferries, data mules, or throwboxes in order to relay data
to intended destinations, or to connect to the backbone network
wherever infrastructure is available. The network is able to store
data destined to temporarily unavailable nodes for some time
depending upon existing storage as well as quality-of-service
issues such as delivery delay bounds imposed by the application.
We evaluate MeDeHa via simulations using indoor scenarios (e.g.
convention centers, exposition halls, museums etc.) and show
significant improvement in delivery ratio in the face of episodic
connectivity. We also showcase MeDeHa’s support for different
levels of quality-of-service through traffic differentiation and
message prioritization.
Categories and Subject Descriptors
C.2.1 [Network Architecture and Design]: Network
communications, C.2.2 [Network Protocols]: Routing protocols
General Terms
Algorithms, Design
Keywords
Disruption tolerance, Episodic connectivity, Heterogeneous
networks, Node relaying, Store-carry-and-forward
1. INTRODUCTION
It is envisioned that the Internet of the future will be highly
heterogeneous not only due to the wide variety of end devices (in
terms of their capabilities, e.g., storage, processing time, battery
lifetime, mobility, and traffic characteristics) it interconnects, but
also in terms of the underlying networks it comprises. As
illustrated in Figure 1, such networks range from wired- and
wireless backbones (e.g. community wireless mesh networks) to
wireless infrastructure-based and ad-hoc networks (MANETs).
Furthermore, current and emerging applications, such as
emergency response, environmental monitoring, smart
environments (e.g., smart offices, homes, museums, etc.), and
vehicular networks, among others imply frequent and arbitrarily
long-lived disruptions in connectivity. The resulting disruption- or
delay-tolerant networks (DTNs) will likely become an integral
component of future internetworks.
Figure 1 An example of a heterogeneous internetwork with a
wired backbone, wireless infrastructure-based, and ad-hoc
networks prone to episodic connectivity
As will become clear in Section 6, which describes related work,
to-date, there are no comprehensive solutions targeting message
delivery in heterogeneous networked environments prone to
connectivity disruptions. Existing proposals either: (1) extend
MANETs to handle episodic connectivity [1,2,3,4], (2) augment
the coverage of access points in infrastructure-based wireless
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
MSWiM’08, October 27–31, 2008, Vancouver, BC, Canada.
Copyright 2008 ACM 978-1-60558-235-1/08/10...$5.00.
211
networks by, for example, making use of multi-channel radios or
switching from infrastructure mode in 802.11 [5,6,7,8], (3)
provide MANETs with Internet connectivity by using special-
purpose gateway nodes and a mechanism to discover them as part
of route discovery in on-demand MANET routing [9], or (4)
handle heterogeneity only at higher layers of the protocol stack
(e.g., Bundle Architecture [10,11]).
In this paper we propose MeDeHa (Message Delivery in
Heterogeneous, Disruption-prone Networks, pronounced
“medea”) – a general, yet efficient framework for data delivery in
heterogeneous internets prone to disruptions in connectivity. To
cope with arbitrarily long-lived connectivity disruptions, we use
available storage within the network to save messages for
destinations that are currently unreachable; once these
destinations re-connect, messages destined to them are delivered.
With respect to using in-network storage, MeDeHa is
complementary to the Bundle Architecture proposed by the
IRTF’s Delay-Tolerant Networking Research Group (DTNRG)
[10,11]. While the Bundle Architecture provides storage above the
transport layer (in order to enable interoperability among
networks that support different types of transport layers),
MeDeHa, stores data at the link layer addressing heterogeneity at
a lower layer (e.g., when intermediate nodes do not support
higher-layer protocols). MeDeHa is also able to provide different
levels of quality-of-service through traffic differentiation and
message prioritization by controlling when messages are
forwarded and for how long they are stored.
Besides, unlike existing proposals such as message ferries [12],
data mules [13], or throwboxes [14], MeDeHa does not require
any special-purpose nodes. Note that there is a difference between
introducing special-purpose nodes in the network to perform the
task of relaying (like message ferries [12], data mules [13], and
throwboxes [14]) and the nodes with special capabilities that are
integral part of underlying network (like APs in case of IEEE
802.11). Of course, whenever available, MeDeHa utilizes nodes
with more resources and capabilities like APs, but they are part of
the underlying network. Furthermore, we take advantage of the
underlying heterogeneity (e.g., in the context of IEEE 802.11
networks, the nodes’ ability to operate in infrastructure or ad-hoc
modes) to enable message delivery across different networks.
Our current implementation of MeDeHa performs message
delivery in an internet comprised of an infrastructure-based
wireless network where mobile nodes roam freely between access
point regions of connectivity, and become temporarily
disconnected from the network. Simulation results obtained with a
variety of mobility, traffic and connectivity conditions show that
MeDeHa is able to improve message delivery ratio significantly.
We performed simulations to analyze the behavior of MeDeHa in
terms of delivery ratio as a function of data rates, buffer sizes and
disconnection times and observed class-wise behavior of traffic
according to some quality-of-service.
The remainder of this paper is organized as follows: Section 2
provides a detailed description of MeDeHa while MeDeHa’s
current implementation is presented in Section 3. The
experimental methodology is briefly described in Section 4. In
Section 5, we present simulation results reporting the performance
of MeDeHa. Related work is reviewed in Section 6 and finally,
concluding remarks and some future directions are given in
Section 7.
2. MeDeHa OVERVIEW
MeDeHa’s main functional components are:
Message relaying: Unlike several DTN solutions, which employ
specialized nodes to aid with message delivery [12,13,14], in
MeDeHa any node in the network can relay messages under the
store-carry-and-forward paradigm [11]. We thus avoid using any
explicit discovery mechanism for finding specialized nodes (e.g.,
gateway to the backbone). Nodes may also take advantage of
network heterogeneity to improve message delivery. For example,
802.11-capable nodes may periodically switch between
infrastructure- and ad-hoc modes to get messages delivered across
both networks.
Buffering: In an environment with intermittent connectivity, it is
necessary to use network nodes to store messages if a route to the
intended destination(s) is not available. An important question is
where to buffer these messages. In MeDeHa any node can relay
messages and therefore needs to store messages whose
destination(s) is(are) not available. However, we again try to take
advantage of network heterogeneity. For example, Access Points
(APs) in infrastructure-based wireless networks or mesh routers in
the case of wireless meshes are perfect candidates to serve as
temporary storage for undelivered messages as they usually
exhibit higher resource availability. It is true that most current
off-the-shelf APs do not typically come equipped with mass
storage. We argue that adding this capability to next-generation
APs is viable and will not considerably impact cost, especially if
there is market demand. Furthermore, co-locating a general-
purpose computing device with APs is another alternate given
current AP technology.
Note that in the Bundle Architecture [10], buffering is performed
above the transport layer; which in itself restricts the types of
nodes that can perform this functionality. For instance, it rules out
APs as buffering nodes, as APs usually run only the two lower
protocol layers. In MeDeHa, buffering is done at layer 2 which
enables almost any network-enabled device to relay and buffer
messages. Moreover, in MeDeHa, quality-of-service is supported
by enforcing application-specific requirements at the message
forwarding and storage level. For instance, data belonging to real-
time flows would be discarded after a pre-defined time interval
specified by the application.
Topology and content information exchange: Nodes periodically
exchange information that is used in building their routing tables.
This information includes a node’s knowledge about the topology
(e.g., its own neighborhood as well as what it knows about other
nodes). Nodes also exchange a summary of their message buffer,
their current state in terms of resources (e.g., how much storage
left, remaining battery lifetime, etc.). This information is used by
relay selection [15,16]. This signaling contributes to the overhead
of MeDeHa. Therefore, there is a tradeoff between the overhead
incurred by the protocol, how fresh paths are, and how well relay
selection performs. Note that if neighborhood information is
already made available by the underlying layer-2 protocol (e.g.,
beaconing, AP association/disassociation), MeDeHa simply
makes use of it.
Traffic differentiation: In order to satisfy application-specific
requirements, MeDeHa uses message tags to carry information
such as message priority, time-to-live (or TTL, which is the
maximum amount of time the message should remain in the
network), scope (e.g., maximum number of hops the message
212
should travel), etc. Besides performing traffic differentiation and
supporting quality-of-service, message tags are also used for
buffer management purposes. For instance, a message that has
been stored pass its TTL would be discarded.
2.1 Overall Operation
Figure 2 illustrates MeDeHa’s overall operation. By default, a
node starts in idle state. It switches to receive state upon reception
of a message, or to forward state if it has some message to send.
This message can either be generated by this node, or can be the
message that the node has stored for some unavailable destination.
Thus, in forward state, if the destination is not found, the node
stores the message and goes back to idle. Later if the destination
is found, the node goes to forward state, delivers the message and
changes its state to idle.
Receive
Buffer Forward
I am not
Destination
Destination
Not Found
Message
Forwarded
Destination
Reached
Message
Received
Message
Buffered, or
Discarded
Idle
Message to Send / Relay
or Destination Found
Figure 2 MeDeHa’s Overall Operation
Forward: When a node has a message to send either as the
message originator or relay, it checks if it has a path to the
destination, and if so, it sends the message along that path and
switches to idle state. Otherwise, it tries to find a “suitable” relay.
If it does not succeed, it switches to buffer state to store the
message locally.
A number of heuristics can be used to select a relay for a
message-destination tuple including: (1) when the node last
encountered the destination (or age of last encounter), (2) how
frequent the destination was encountered, (3) how mobile a node
is, whether a node’s mobility is “local” or “global”, (4) how
“social” a node is, etc. In MeDeHa, when selecting relays, we also
account for the underlying heterogeneity among participating
nodes, e.g., the amount of available resources such as storage,
processing, and battery lifetime. In its current implementation,
MeDeHa favors APs as message relays.
Receive: When a node receives a message and it is not the
intended destination, it switches to forward state and follows the
steps described above.
Buffer: Storing messages at relay nodes is based upon traffic
differentiation and QoS requirements (e.g., message TTL,
message priorities). So, when a relay node has a message to store
and it doesn’t have space available, it drops the oldest, lowest
priority message. If all messages have the same priority, it then
drops the oldest stored message. Otherwise, if the incoming
packet has a priority, which is lower than all stored messages, the
incoming message is discarded.
2.2 The Protocol
2.2.1 Receive
MeDeHa’s receive functionality is shown in Figure 3. When a
node receives a message, it switches to receive state and checks if
it is the intended destination for the message. If so, it consumes
the message (ConsumeMessage()) and switches back to idle.
Otherwise, it switches to forward state.
Figure 3 Receive Function
2.2.2 Forward
The forward function is called either when a node has a message
to send, or when a node that carries messages for a destination
meets the destination, or meets another “suitable” relay node for
that destination. In order to search for destinations for any of the
stored messages, the forward function is called periodically. In
forward state, a node first consults its routing (or contact) table to
see if it has an entry for a destination. If the destination
information is found, the message is forwarded to the destination
(SendMessageToDestination()) and the node goes to idle
state. Having not found the destination information, the node tries
to find a route to the destination through its neighborhood
(SwitchNetworknCheckForDestInfo()). For this
purpose, the node may, for example, switch networks if it belongs
to multiple networks. The message is sent to the destination if a
route is found (SendMessageToDestination()).
Otherwise, if a “suitable” relay node that can carry the message to
the destination is found, the message is forwarded to the relay
node (ForwardMessageToRelay()), and the current node
changes its state to idle. If no information about the destination is
found or no relay is selected and the message is not already
buffered locally, the node changes its state to buffer and stores the
message (BufferMessage()). Pseudo code for the forward
function is shown in Figure 4.
2.2.3 Buffer
In this state, when a node has a message to store locally, it first
checks if there is available storage, and then stores the message
(StoreMessage()). In case the local buffer is full, the node
examines the priority of the incoming message
(CheckMessagePriority()). If the message has high
priority, the node deletes the oldest and lowest priority message
from its buffer. If all messages have the same priority, the oldest
message is removed. If the incoming message has low priority
and the buffer is already full with higher priority messages, the
incoming message is discarded and the state is changed to idle.
Figure 5 describes the pseudo code for the buffer function.
At the time of message origination, a TTL value (in seconds) is
assigned to each message by the source of the message, according
to its class of service. This TTL value indicates the amount of
time this message is allowed to remain buffered at the storing
213
node, and is used for buffer management. The storing node
discards the message when TTL for the message is expired. Note
that the TTL mechanism doesn’t require any synchronization
amongst different nodes, and is used in order to avoid messages to
remain buffered at nodes forever.
Figure 4 Forward Function
Figure 5 Buffer Function
2.2.4 Notification Protocol
In order to build its routing/contact tables, MeDeHa nodes use a
notification protocol by which they exchange topology and
content information. The main components of this protocol
include vicinity discovery, neighborhood and content information
exchange, and network switching. Neighborhood discovery is
performed via the exchange of “hello” messages in order to see
who is in the vicinity. This information could be provided by the
underlying link layer protocol. Nodes engage in this exchange
periodically so that if any node has messages stored for another
neighboring node, these messages are delivered. Nodes also
exchange “meta-information” about nodes they meet over time;
this information is used as heuristics for relay selection.
MeDeHa tries to make use of node and network heterogeneity.
For example, nodes that are able to participate in multiple
networks, switch between them attempting to find a path to a
destination or to find good relays. Switching is performed by
utilizing different frequencies for each network. This can be
achieved by making use of power-save mode, in case of IEEE
802.11. A similar kind of approach is proposed in [7]. MeDeHa
also attempts to take advantage of more powerful nodes whenever
available. For instance, MeDeHa’s current implementation targets
indoor scenarios (e.g. convention centers, exposition halls,
museums etc.) consisting of an internet with a backbone
connecting access points (APs). In these scenarios, the APs run
the notification protocol to exchange information about
associations and disassociations of mobile nodes.
2.3 Design Issues
MeDeHa raises a number of interesting design issues that are
critical to the correct and efficient operation of the protocol. We
discuss some of them here.
2.3.1 Relay node selection
As already mentioned in Section 2.1, several heuristics can be
considered when choosing a suitable relay node. These include
time since a node last saw the destination, frequency at which the
node encountered the destination, total meeting time with
destination, node’s mobility pattern, node’s social behavior,
resources available at node including battery, storage etc. Some of
these heuristics have already been reviewed in the literature
[15,16], but their application is highly dependent upon the target
environment. Messages can be replicated and carried by multiple
relays. While having more relay nodes increases the chances of
message delivery, it may cause buffer overflows and unnecessary
traffic in the network. Moreover, wherever an infrastructure is
available, MeDeHa gives preference to an infrastructure-based
node (e.g. AP) over other relay nodes, if the destination is not
directly reachable.
2.3.2 Buffer Management
Another important design decision is how to perform buffer
management including how much storage space to be utilized at
relay nodes, time to keep a message in storage at relay nodes,
when to discard stored messages, etc. The storage space
parameter depends upon the relay node’s storage capacity (i.e., its
storage and energy capabilities), as we can have heterogeneous
devices in the system. In Section 5, the impact of varying buffer
spaces is discussed and evaluated.
2.3.3 Switching Between Networks
For nodes that participate in more than one network (e.g.,
infrastructure and ad-hoc modes in IEEE 802.11), deciding when
to switch between different networks is important. It should
consider traffic demands, network and node capacity. Switching
can be periodic by default with a specific amount of time
dedicated to each mode. The switching operation can also be
forced by a specific event, like the urgency of finding a
214
destination for a high priority message. Moreover, mechanisms
can be built in order to adapt the switching time according to
varying network conditions.
3. MeDeHa’s CURRENT
IMPLEMENTATION
Our current implementation of MeDeHa performs message
delivery in an internet comprised of an infrastructure-based
wireless network where mobile nodes roam freely and become
temporarily disconnected from the network. This way, the
messages are stored in the network and they are delivered as soon
as a destination appears anywhere in the network. Moreover, a
source, when moves and finds itself in a region of no
connectivity, starts caching its messages for the destination. In
this way, the source stores messages at its end, and as soon as it
finds connectivity region, it starts forwarding the messages to the
associated AP.
In terms of buffering in the backbone, we define two different
strategies; centralized and distributed. In centralized buffering, we
dedicate a machine on the backbone network to be responsible for
data storage, whereas in distributed buffering, the responsibility
of storing messages for a particular destination is assigned to the
AP which has last seen the destination node. In this way, the
responsibility of buffering keeps on changing as a node moves
around in the network, which is a form of load balancing. While
storing at a specific station in the backbone network requires a
specific station to be dedicated for this purpose in the backbone,
we have implemented this scheme to address the case when the
APs are not capable of storing messages. Both these schemes
(centralized and distributed) have their own advantages and
disadvantages, and a comparison in between the two schemes is
presented in Section 5.
A notification mechanism is required in between APs in order to
send information about nodes’ association/disassociation. This
notification mechanism helps to find a route to a destination and
to update the buffer status. The APs also collect information from
their associated nodes and notify each other about the update
periodically. In the following, we explain how this notification
mechanism works between APs.
When a node is in the vicinity of an AP (associated with an AP),
the AP informs all other APs about the presence of the node in its
neighborhood by sending a NODE_PRESENT_NOTIF
notification. This message contains the node’s address, the AP’s
address, and notification ID. In this way, all other APs update
their routes to that particular node and record the association of
the node with the sending AP. Similarly, when a node leaves the
vicinity of an AP (the node is disassociated from an AP), the AP
updates all other APs by sending a NODE_LEAVE_NOTIF. The
format of this message is the same as for the
NODE_PRESENT_NOTIF.
After sending a NODE_PRESENT_NOTIF notification, the AP
does a pull by requesting all others to send any stored messages
for the node in question. This is done by sending a
FETCH_FRAMES_NOTIF notification. In case of centralized
buffering, this request is only sent to the central station, and for
distributed buffering, the request is broadcasted to all APs. Upon
reception of a FETCH_FRAMES_NOTIF request, the APs that
have stored frames for the particular destination start sending
messages. Message transmission is controlled by a timer so as to
avoid sending all the stored messages at once leading potentially
to congestion in the backbone. Note that it is possible that at a
given point in time, more than one APs have messages for a
single destination, in the case of distributed buffering mechanism.
This can happen if the responsibility of storing switches to
another AP before the previous AP delivers all the stored
messages to the destination (an AP stops sending stored messages
to a destination as soon as it receives a NODE_LEAVE_NOTIF
notification).
The storing of messages at a central station or at any particular
AP is done by sending a STORE_FRAMES_NOTIF notification.
This message contains notification ID, destination node’s address,
sending AP’s address, and the message to store. In case of
distributed buffering, when a node goes out of the range of an AP,
the AP declares itself responsible for storing messages for the
node by sending a NODE_LEAVE_NOTIF notification. So, all the
messages for the destination would be forwarded to this entity.
To be able to roam around, a node keeps on checking the received
power levels of beacon frames from APs, and triggers a new
association if the power level of other AP is 10% more than the
power level of the current associated AP. This threshold is set in
order to avoid oscillations of associations in between two APs. If
a station receives beacons from only one AP, it compares the
received power level with a threshold, and triggers an explicit
disassociation as soon as it touches the minimum received power
level. The management functionality of stations in IEEE 802.11 is
modified to perform this task. Also, an explicit disassociation
mechanism is incorporated in stations. In order to perform this
roaming, and to enable a node to keep on listening to the beacons,
all APs use the same channel and SSID.
4. EXPERIMENTAL METHODOLOGY
Today, network heterogeneity is not supported in most open-
source network simulators. We use OMNET++ simulator [17],
which provides basic network heterogeneity support. The version
of INET Framework of OMNET++ that we are using is an
extended version of the simulator
1
. We have added the handoff
process in the simulator. Also, we have developed an explicit
disassociation mechanism in which a station, before disconnecting
from an AP, sends a disassociation frame to the AP, and then
starts scanning all channels. This is done by comparing the
received power with a threshold that is just above the minimum
received power. The functionality of APs is also modified in
OMNET++ to enable them to buffer the messages for unavailable
destinations, and to implement notification protocol. We have
also implemented two types of buffering mechanisms, centralized
and distributed, as explained in Section 3.
The buffering policy is implemented to provide per flow and per
destination priority mechanism. In this way, when an AP’s buffer
is full, the oldest message with lower priority is dropped. In this
way, if a lower priority message is arrived and the buffer is full
with higher priority messages, the incoming message is discarded
(dropped).
5. RESULTS
We use packet delivery ratio (PDR) to show how MeDeHa
improves message delivery in heterogeneous internets subject to
1
The modified version of INET Framework of OMNET++ can be
found at http://planete.inria.fr/software/MeDeHa. Some scripts
to perform the simulations are also available online.
215
