Showing papers on "Overlay network published in 2006"

Network-Aware Operator Placement for Stream-Processing Systems

Abstract: To use their pool of resources efficiently, distributed stream-processing systems push query operators to nodes within the network. Currently, these operators, ranging from simple filters to custom business logic, are placed manually at intermediate nodes along the transmission path to meet application-specific performance goals. Determining placement locations is challenging because network and node conditions change over time and because streams may interact with each other, opening venues for reuse and repositioning of operators. This paper describes a stream-based overlay network (SBON), a layer between a stream-processing system and the physical network that manages operator placement for stream-processing systems. Our design is based on a cost space, an abstract representation of the network and on-going streams, which permits decentralized, large-scale multi-query optimization decisions. We present an evaluation of the SBON approach through simulation, experiments on PlanetLab, and an integration with Borealis, an existing stream-processing engine. Our results show that an SBON consistently improves network utilization, provides low stream latency, and enables dynamic optimization at low engineering cost.

Multi-path TCP: a joint congestion control and routing scheme to exploit path diversity in the internet

Abstract: We consider the problem of congestion-aware multi-path routing in the Internet. Currently, Internet routing protocols select only a single path between a source and a destination. However, due to many policy routing decisions, single-path routing may limit the achievable throughput. In this paper, we envision a scenario where multi-path routing is enabled in the Internet to take advantage of path diversity. Using minimal congestion feedback signals from the routers, we present a class of algorithms that can be implemented at the sources to stably and optimally split the flow between each source-destination pair. We then show that the connection-level throughput region of such multi-path routing/congestion control algorithms can be larger than that of a single-path congestion control scheme.

Eclipse Attacks on Overlay Networks: Threats and Defenses

Abstract: Overlay networks are widely used to deploy functionality at edge nodes without changing network routers. Each node in an overlay network maintains connections with a number of peers, forming a graph upon which a distributed application or service is implemented. In an “Eclipse” attack, a set of malicious, colluding overlay nodes arranges for a correct node to peer only with members of the coalition. If successful, the attacker can mediate most or all communication to and from the victim. Furthermore, by supplying biased neighbor information during normal overlay maintenance, a modest number of malicious nodes can eclipse a large number of correct victim nodes. This paper studies the impact of Eclipse attacks on structured overlays and shows the limitations of known defenses. We then present the design, implementation, and evaluation of a new defense, in which nodes anonymously audit each other’s connectivity. The key observation is that a node that mounts an Eclipse attack must have a higher than average node degree. We show that enforcing a node degree limit by auditing is an effective defense against Eclipse attacks. Furthermore, unlike most existing defenses, our defense leaves flexibility in the selection of neighboring nodes, thus permitting important overlay optimizations like proximity neighbor selection (PNS).

VON: a scalable peer-to-peer network for virtual environments

Abstract: The scalability of large-scale networked virtual environments (NVEs) such as today's massively multiplayer online games (MMOGs) faces inherent limits imposed by client-server architectures We identify an emerging research direction that applies peer-to-peer (P2P) networks in order to realize more scalable and affordable NVEs The central issue for P2P-based NVE (P2P-NVE) systems is to correctly and efficiently maintain the topology of all participating peers by solving the neighbor discovery problem We also propose the Voronoi-based overlay network (VON), a simple and efficient design that maintains the P2P topology in a fully-distributed, low-latency, and message-efficient manner Simulation results show that by bounding the per-node resource consumption, VON can be fundamentally more scalable than existing methods while achieving high topology consistency and reliability

Declarative networking: language, execution and optimization

Abstract: The networking and distributed systems communities have recently explored a variety of new network architectures, both for application-level overlay networks, and as prototypes for a next-generation Internet architecture. In this context, we have investigated declarative networking: the use of a distributed recursive query engine as a powerful vehicle for accelerating innovation in network architectures [23, 24, 33]. Declarative networking represents a significant new application area for database research on recursive query processing. In this paper, we address fundamental database issues in this domain. First, we motivate and formally define the Network Datalog (NDlog) language for declarative network specifications. Second, we introduce and prove correct relaxed versions of the traditional semi-naive query evaluation technique, to overcome fundamental problems of the traditional technique in an asynchronous distributed setting. Third, we consider the dynamics of network state, and formalize the iheventual consistencyl. of our programs even when bursts of updates can arrive in the midst of query execution. Fourth, we present a number of query optimization opportunities that arise in the declarative networking context, including applications of traditional techniques as well as new optimizations. Last, we present evaluation results of the above ideas implemented in our P2 declarative networking system, running on 100 machines over the Emulab network testbed.

Hot or not: Revealing hidden services by their clock skew

Abstract: Location-hidden services, as offered by anonymity systems such as Tor, allow servers to be operated under a pseudonym. As Tor is an overlay network, servers hosting hidden services are accessible both directly and over the anonymous channel. Traffic patterns through one channel have observable effects on the other, thus allowing a service's pseudonymous identity and IP address to be linked. One proposed solution to this vulnerability is for Tor nodes to provide fixed quality of service to each connection, regardless of other traffic, thus reducing capacity but resisting such interference attacks. However, even if each connection does not influence the others, total throughput would still affect the load on the CPU, and thus its heat output. Unfortunately for anonymity, the result of temperature on clock skew can be remotely detected through observing timestamps. This attack works because existing abstract models of anonymity-network nodes do not take into account the inevitable imperfections of the hardware they run on. Furthermore, we suggest the same technique could be exploited as a classical covert channel and can even provide geolocation.

Dynamic Topology Configuration in Service Overlay Networks: A Study of Reconfiguration Policies

Abstract: The routing infrastructure of the Internet has become resistant to fundamental changes and the use of overlay networks has been proposed to provide additional flexibility and control. One of the most prominent configurable components of an overlay network is its topology, which can be dynamically reconfigured to accommodate communication requirements that vary over time. In this paper, we study the problem of determining dynamic topology reconfiguration for service overlay networks with dynamic communication requirement, and the ideal goal is to find the optimal reconfiguration policies that can minimize the potential overall cost of using an overlay. We start by observing the properties of the optimal reconfiguration policies through studies on small systems and find structures in the optimal reconfiguration policies. Based on these observations, we propose heuristic methods for constructing different flavors of reconfiguration policies, i.e., never-change policy, always-change policy and cluster-based policies, to mimic and approximate the optimal ones. Our experiments show that our policy construction methods are applicable to large systems and generate policies with good performance. Our work does not only provide solutions to practical overlay topology design problems, but also provides theoretical evidence for the advantage of overlay network due to its configurability.

Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Abstract: The present invention leverages an existing content delivery network infrastructure to provide a system that enhances performance for any application that uses the Internet Protocol (IP) as its underlying transport mechanism. An overlay network comprises a set of edge nodes, intermediate nodes, and gateway nodes. This network provides optimized routing of IP packets. Internet application users can use the overlay to obtain improved performance during normal network conditions, to obtain or maintain good performance where normal default BGP routing would otherwise force the user over congested or poorly performing paths, or to enable the user to maintain communications to a target server application even during network outages.

Method, apparatus and system for a location-based uniform resource locator

Abstract: An aspect of the present invention is a method for routing content information to a mobile user or client application. The method preferably comprises re-directing a user request to one or more gateway servers provided via an overlay network. In another aspect, the present invention is an apparatus that includes a proxy service that intercepts content information requests to the Internet and re-directs the content requests to an overlay. Another aspect of the present invention comprises a location-based Uniform Resource Locator that includes a protocol semantic portion and a location-based resolver address portion that identifies one or more resources on a network based on the geographical location of the resources.

Drafting behind Akamai (travelocity-based detouring)

Abstract: To enhance web browsing experiences, content distribution networks (CDNs) move web content "closer" to clients by caching copies of web objects on thousands of servers worldwide. Additionally, to minimize client download times, such systems perform extensive network and server measurements, and use them to redirect clients to different servers over short time scales. In this paper, we explore techniques for inferring and exploiting network measurements performed by the largest CDN, Akamai; our objective is to locate and utilize quality Internet paths without performing extensive path probing or monitoring.Our contributions are threefold. First, we conduct a broad measurement study of Akamai's CDN. We probe Akamai's network from 140 PlanetLab vantage points for two months. We find that Akamai redirection times, while slightly higher than advertised, are sufficiently low to be useful for network control. Second, we empirically show that Akamai redirections overwhelmingly correlate with network latencies on the paths between clients and the Akamai servers. Finally, we illustrate how large-scale overlay networks can exploit Akamai redirections to identify the best detouring nodes for one-hop source routing. Our research shows that in more than 50% of investigated scenarios, it is better to route through the nodes "recommended" by Akamai, than to use the direct paths. Because this is not the case for the rest of the scenarios, we develop lowoverhead pruning algorithms that avoid Akamai-driven paths when they are not beneficial.

Device and a method for communicating in a network

Abstract: A method of managing a communication network comprising a control plane and a network plane, the network comprising nodes and physical connections of the nodes, each of a plurality of nodes being a logical network device, supporting a control plane portion in the control plane and a network plane portion in the network plane, in which method, the control plane portions of the logical network devices form a logical network in a peer to peer fashion, and control data necessary for administering the communication network and/or for managing users of the communication network is contained in at least one database distributed between at least a plurality of control plane portions of the network devices forming the logical network.

Packet Switched vs. Time Multiplexed FPGA Overlay Networks

Abstract: Dedicated, spatially configured FPGA interconnect is efficient for applications that require high throughput connections between processing elements (PEs) but with a limited degree of PE interconnectivity (e.g. wiring up gates and datapaths). Applications which virtualize PEs may require a large number of distinct PE-to-PE connections (e.g. using one PE to simulate 100s of operators, each requiring input data from thousands of other operators), but with each connection having low throughput compared with the PE's operating cycle time. In these highly interconnected conditions, dedicating spatial interconnect resources for all possible connections is costly and inefficient. Alternatively, we can time share physical network resources by virtualizing interconnect links, either by statically scheduling the sharing of resources prior to runtime or by dynamically negotiating resources at runtime. We explore the tradeoffs (e.g. area, route latency, route quality) between time-multiplexed and packet-switched networks overlayed on top of commodity FPGAs. We demonstrate modular and scalable networks which operate on a Xilinx XC2V6000-4 at 166MHz. For our applications, time-multiplexed, offline scheduling offers up to a 63% performance increase over online, packet-switched scheduling for equivalent topologies. When applying designs to equivalent area, packet-switching is up to 2times faster for small area designs while time-multiplexing is up to 5times faster for larger area designs. When limited to the capacity of a XC2V6000, if all communication is known, time-multiplexed routing outperforms packet-switching; however when the active set of links drops below 40% of the potential links, packet-switched routing can outperform time-multiplexing

Sub-2-Sub: Self-Organizing Content-Based Publish and Subscribe for Dynamic and Large Scale Collaborative Networks

Abstract: In this paper, we address the problem of constructing scalable content-based publish/subscribe systems. Publish/subscribe systems are asynchronous event-notification systems in which a published event is forwarded to exactly those nodes that have previously subscribed for that event. Subscriptions can range from a simple specification of merely the type of an event to a specification of the value ranges that an event's attributes can have. Notably the latter poses potential scalability problems. Structured peer-to-peer systems can provide scalable solutions to publish/subscribe systems with simple subscription patterns. For complex subscription types their applicability is less obvious. In this paper, we present Sub-2-Sub, a collaborative self-organizing publish/subscribe system deploying an unstructured overlay network. Sub-2-Sub relies on an epidemic-based algorithm in which peers continuously exchange subscription information to get clustered to similar peers. In contrast to many existing approaches, Sub-2-Sub supports both value-based and interval-based subscriptions. Simulations of Sub-2-Sub on synthetic and reusable workloads convey its good properties in terms of routing efficiency, fairness, accuracy and efficiency.

Defending the Sybil attack in P2P networks: taxonomy, challenges, and a proposal for self-registration

Abstract: The robustness of peer-to-peer (P2P) networks, in particular of DHT-based overlay networks, suffers significantly when a Sybil attack is performed. We tackle the issue of Sybil attacks from two sides. First, we clarify, analyze, and classify the P2P identifier assignment process. By clearly separating network participants from network nodes, two challenges of P2P networks under a Sybil attack become obvious: i) stability over time, and ii) identity differentiation. Second, as a starting point for a quantitative analysis of time-stability of P2P networks under Sybil attacks and under some assumptions with respect to identity differentiation, we propose an identity registration procedure called self-registration that makes use of the inherent distribution mechanisms of a P2P network.

Measurement based analysis, modeling, and synthesis of the internet delay space

Abstract: Understanding the characteristics of the Internet delay space (i.e., the all-pairs set of static round-trip propagation delays among edge networks in the Internet) is important for the design of global-scale distributed systems. For instance, algorithms used in overlay networks are often sensitive to violations of the triangle inequality and to the growth properties within the Internet delay space. Since designers of distributed systems often rely on simulation and emulation to study design alternatives, they need a realistic model of the Internet delay space.Our analysis shows that existing models do not adequately capture important properties of the Internet delay space. In this paper, we analyze measured delays among thousands of Internet edge networks and identify key properties that are important for distributed system design. Furthermore, we derive a simple model of the Internet delay space based on our analytical findings. This model preserves the relevant metrics far better than existing models, allows for a compact representation, and can be used to synthesize delay data for simulations and emulations at a scale where direct measurement and storage are impractical.

Load balancing communications system comprising cellular overlay and ad hoc networks

Abstract: The present invention relates to a method and system for balancing load in a network environment comprising at least one ad hoc network and at least one overlay network (10) , wherein at least one wireless mobile node (50) of the ad hoc network is selected as a head node for collecting transmission related information received from other mobile nodes of said ad hoc network, which is reported to a load balancing function of the network environment. There, the reported transmission related information is analyzed and a connection link within the network environment is selected based on the result of said analyzing step.

IP over P2P: enabling self-configuring virtual IP networks for grid computing

Abstract: Peer-to-peer (P2P) networks have mostly focused on task oriented networking, where networks are constructed for single applications, i.e. file-sharing, DNS caching, etc. In this work, we introduce IPOP, a system for creating virtual IP networks on top of a P2P overlay. IPOP enables seamless access to grid resources spanning multiple domains by aggregating them into a virtual IP network that is completely isolated from the physical network. The virtual IP network provided by IPOP supports deployment of existing IP-based protocols over a robust, self-configuring P2P overlay. We present implementation details as well as experimental measurement results taken from LAN, WAN, and Planet-Lab tests.

On achieving maximum multicast throughput in undirected networks

Abstract: The transmission of information within a data network is constrained by the network topology and link capacities. In this paper, we study the fundamental upper bound of information dissemination rates with these constraints in undirected networks, given the unique replicable and encodable properties of information flows. Based on recent advances in network coding and classical modeling techniques in flow networks, we provide a natural linear programming formulation of the maximum multicast rate problem. By applying Lagrangian relaxation on the primal and the dual linear programs (LPs), respectively, we derive a) a necessary and sufficient condition characterizing multicast rate feasibility, and b) an efficient and distributed subgradient algorithm for computing the maximum multicast rate. We also extend our discussions to multiple communication sessions, as well as to overlay and ad hoc network models. Both our theoretical and simulation results conclude that, network coding may not be instrumental to achieve better maximum multicast rates in most cases; rather, it facilitates the design of significantly more efficient algorithms to achieve such optimality.

Utility-Lifetime Trade-off in Self-regulating Wireless Sensor Networks: A Cross-Layer Design Approach

Abstract: The performance of wireless sensor network applications is typically a function of the amount of data collected by the individual sensors and delivered to a set of sinks through multihop routing within the network. However, the energy-constrained nature of the nodes limits the operational lifetime of the network since energy is dissipated both in sensing and in communicating data across the network. There is thus an inherent trade-off in simultaneously maximizing the network lifetime and the application performance (characterized here by a network utility function). In this paper, we characterize this trade-off by considering a cross-layer design problem in a wireless sensor network with orthogonal link transmissions. We compute an optimal set of source rates, network flows, and radio resources at the transport, network, and radio resource layers respectively, while jointly maximizing the network utility and lifetime. Using dual decomposition techniques, we show that the cross-layer optimization problem decomposes vertically into three subproblems - a joint transport and routing problem, a radio resource allocation problem, and a network lifetime maximization problem, all of which interact through the dual prices for capacities of links and battery capacities of nodes.

Distributed k-ary System: Algorithms for Distributed Hash Tables

Abstract: This dissertation presents algorithms for data structures called distributed hash tables (DHT) or structured overlay networks, which are used to build scalable self-managing distributed systems. The provided algorithms guarantee lookup consistency in the presence of dynamism: they guarantee consistent lookup results in the presence of nodes joining and leaving. Similarly, the algorithms guarantee that routing never fails while nodes join and leave. Previous algorithms for lookup consistency either suffer from starvation, do not work in the presence of failures, or lack proof of correctness. Several group communication algorithms for structured overlay networks are presented. We provide an overlay broadcast algorithm, which unlike previous algorithms avoids redundant messages, reaching all nodes in O(log n) time, while using O(n) messages, where n is the number of nodes in the system. The broadcast algorithm is used to build overlay multicast. We introduce bulk operation, which enables a node to efficiently make multiple lookups or send a message to all nodes in a specified set of identifiers. The algorithm ensures that all specified nodes are reached in O(log n) time, sending maximum O(log n) messages per node, regardless of the input size of the bulk operation. Moreover, the algorithm avoids sending redundant messages. Previous approaches required multiple lookups, which consume more messages and can render the initiator a bottleneck. Our algorithms are used in DHT-based storage systems, where nodes can do thousands of lookups to fetch large files. We use the bulk operation algorithm to construct a pseudo-reliable broadcast algorithm. Bulk operations can also be used to implement efficient range queries. Finally, we describe a novel way to place replicas in a DHT, called symmetric replication, that enables parallel recursive lookups. Parallel lookups are known to reduce latencies. However, costly iterative lookups have previously been used to do parallel lookups. Moreover, joins or leaves only require exchanging O(1) messages, while other schemes require at least log(f) messages for a replication degree of f. The algorithms have been implemented in a middleware called the Distributed k-ary System (DKS), which is briefly described.

A peer-to-peer architecture for massive multiplayer online games

Abstract: Massive Multiplayer Online Games with their virtual gaming worlds grow in user numbers as well as in the size of the virtual worlds. With this growth comes a significant increase of the requirements for server hardware. Today an MMOG provider usually faces the problem of serving thousands of users with entire server clusters. Peer-to-Peer networks with their high scalability and flexibility meet the requirements of connecting hundreds of thousands of people all over the world without a central server. In doing so the network bandwidth requirements remain at a reasonable level.In this work we propose to combine MMOGs with a Peer-to-Peer network. We introduce a game architecture capable of exploiting the flexibility and scalability of P2P networks. A P2P architecture based on an overlay network using distributed hash tables with support for persistent object storage and event distribution has been developed to meet MMOG requirements.

A virtual network (ViNe) architecture for grid computing

Abstract: This paper describes a virtual networking approach for grids called ViNe. It enables symmetric connectivity among grid resources and allows existing applications to run unmodified. Novel features of the ViNe architecture include: easy virtual networking administration; support for physical private networks and support for multiple independent virtual networks in the same infrastructure. The requirements of an application-friendly virtual network environment are presented and it is shown how the proposed solution meets them. Qualitative arguments are provided to justify all design decisions. Also presented is an experimental evaluation of the round-trip latencies and bandwidths achieved by a reference implementation. Measurements are reported for WAN-scenarios involving three different institutions. Under favorable conditions, ViNe bandwidths are within 90 to 100% of the available physical network bandwidth.

Biologically inspired self-adaptive multi-path routing in overlay networks

Abstract: In this paper we propose a multi-path routing scheme based on a biologically inspired attractor selection model. The advantage of this approach is that it is highly noise-tolerant and capable to operate in a very robust manner under varying environmental conditions. Further- more, the route selection is performed in accordance to

Cross-Layer Optimized Video Streaming Over Wireless Multihop Mesh Networks

Abstract: The proliferation of wireless multihop communication infrastructures in office or residential environments depends on their ability to support a variety of emerging applications requiring real-time video transmission between stations located across the network. We propose an integrated cross-layer optimization algorithm aimed at maximizing the decoded video quality of delay-constrained streaming in a multihop wireless mesh network that supports quality-of-service. The key principle of our algorithm lays in the synergistic optimization of different control parameters at each node of the multihop network, across the protocol layers-application, network, medium access control, and physical layers, as well as end-to-end, across the various nodes. To drive this optimization, we assume an overlay network infrastructure, which is able to convey information on the conditions of each link. Various scenarios that perform the integrated optimization using different levels ("horizons") of information about the network status are examined. The differences between several optimization scenarios in terms of decoded video quality and required streaming complexity are quantified. Our results demonstrate the merits and the need for cross-layer optimization in order to provide an efficient solution for real-time video transmission using existing protocols and infrastructures. In addition, they provide important insights for future protocol and system design targeted at enhanced video streaming support across wireless mesh networks

A distributed protocol for dynamic address assignment in mobile ad hoc networks

Abstract: A Mobile Ad hoc NETwork (MANET) is a group of mobile nodes that form a multihop wireless network. The topology of the network can change randomly due to unpredictable mobility of nodes and propagation characteristics. Previously, it was assumed that the nodes in the network were assigned IP addresses a priori. This may not be feasible as nodes can enter and leave the network dynamically. A dynamic IP address assignment protocol like DHCP requires centralized servers that may not be present in MANETs. Hence, we propose a distributed protocol for dynamic IP address assignment to nodes in MANETs. The proposed solution guarantees unique IP address assignment under a variety of network conditions including message losses, network partitioning and merging. Simulation results show that the protocol incurs low latency and communication overhead for an IP address assignment.

LION: Layered Overlay Multicast With Network Coding

Abstract: Recent advances in information theory show that the throughput of a multicast session can be improved using network coding. In overlay networks, the available bandwidth between sender and different receivers are different. In this paper, we propose a solution to improve the throughput of an overlay multicast session with heterogeneous receivers by organizing the receivers into layered data distribution meshes and sending substreams to each mesh using layered coding. Our solutions utilize alternative paths and network coding in each mesh. We first formulate the problem into a mathematical programming, whose optimal solution requires global information. We therefore present a distributed heuristic algorithm. The heuristic progressively organizes the receivers into layered meshes. Each receiver can subscribe to a proper number of meshes to maximize its throughput by fully utilizing its available bandwidth. The benefits of organizing the topology into layered mesh and using network coding are demonstrated through extensive simulations. Numerical results indicate that the average throughput of a multicast session is significantly improved (up to 50% to 60%) with only slightly higher delay and network resource consumption

WOW: Self-Organizing Wide Area Overlay Networks of Virtual Workstations

Abstract: This paper describes WOW, a distributed system that combines virtual machine, overlay networking and peer-to-peer techniques to create scalable wide-area networks of virtual workstations for high-throughput computing. The system is architected to: facilitate the addition of nodes to a pool of resources through the use of system virtual machines (VMs) and self-organizing virtual network links; to maintain IP connectivity even if VMs migrate across network domains; and to present to end-users and applications an environment that is functionally identical to a local-area network or cluster of workstations. We describe a novel, extensible user-level decentralized technique to discover, establish and maintain overlay links to tunnel IP packets over different transports (including UDP and TCP) and across firewalls. We also report on several experiments conducted on a testbed WOW deployment with 118 P2P router nodes over PlanetLab and 33 VMware-based VM nodes distributed across six firewalled domains. Experiments show that the latency in joining a WOW network is of the order of seconds: in a set of 300 trials, 90% of the nodes self-configured P2P routes within 10 seconds, and more than 99% established direct connections to other nodes within 200 seconds. Experiments also show that the testbed delivers good performance for two unmodified, representative benchmarks drawn from the life-sciences domain. The testbed WOW achieves an overall throughput of 53 jobs/minute for PBS-scheduled executions of the MEME application (with average single-job sequential running time of 24.1s) and a parallel speedup of 13.5 for the PVM-based fastDNAml application. Experiments also demonstrate that the system is capable of seamlessly maintaining connectivity at the virtual IP layer for typical client/server applications (NFS, SSH, PBS) when VMs migrate across a WAN