This paper presents the design and evaluation of Pastry, a scalable, distributed object location and routing substrate for wide-area peer-to-peer ap- plications. Pastry performs application-level routing and object location in a po- tentially very large overlay network of nodes connected via the Internet. It can be used to support a variety of peer-to-peer applications, including global data storage, data sharing, group communication and naming. Each node in the Pastry network has a unique identifier (nodeId). When presented with a message and a key, a Pastry node efficiently routes the message to the node with a nodeId that is numerically closest to the key, among all currently live Pastry nodes. Each Pastry node keeps track of its immediate neighbors in the nodeId space, and notifies applications of new node arrivals, node failures and recoveries. Pastry takes into account network locality; it seeks to minimize the distance messages travel, according to a to scalar proximity metric like the number of IP routing hops. Pastry is completely decentralized, scalable, and self-organizing; it automatically adapts to the arrival, departure and failure of nodes. Experimental results obtained with a prototype implementation on an emulated network of up to 100,000 nodes confirm Pastry's scalability and efficiency, its ability to self-organize and adapt to node failures, and its good network locality properties.

/pdf/pastry-scalable-decentralized-object-location-and-routing-1u2equbbbp.pdf

Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems

A case for end system multicast

In today’s chaotic network, data and services are mobile and replicated widely for availability, durability, and locality. Components within this infrastructure interact in rich and complex ways, greatly stressing traditional approaches to name service and routing. This paper explores an alternative to traditional approaches called Tapestry. Tapestry is an overlay location and routing infrastructure that provides location-independent routing of messages directly to the closest copy of an object or service using only point-to-point links and without centralized resources. The routing and directory information within this infrastructure is purely soft state and easily repaired. Tapestry is self-administering, faulttolerant, and resilient under load. This paper presents the architecture and algorithms of Tapestry and explores their advantages through a number of experiments.

/pdf/tapestry-an-infrastructure-for-fault-tolerant-wide-area-3mweaivgsc.pdf

Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and Routing

A Resilient Overlay Network (RON) is an architecture that allows distributed Internet applications to detect and recover from path outages and periods of degraded performance within several seconds, improving over today's wide-area routing protocols that take at least several minutes to recover. A RON is an application-layer overlay on top of the existing Internet routing substrate. The RON nodes monitor the functioning and quality of the Internet paths among themselves, and use this information to decide whether to route packets directly over the Internet or by way of other RON nodes, optimizing application-specific routing metrics.Results from two sets of measurements of a working RON deployed at sites scattered across the Internet demonstrate the benefits of our architecture. For instance, over a 64-hour sampling period in March 2001 across a twelve-node RON, there were 32 significant outages, each lasting over thirty minutes, over the 132 measured paths. RON's routing mechanism was able to detect, recover, and route around all of them, in less than twenty seconds on average, showing that its methods for fault detection and recovery work well at discovering alternate paths in the Internet. Furthermore, RON was able to improve the loss rate, latency, or throughput perceived by data transfers; for example, about 5% of the transfers doubled their TCP throughput and 5% of our transfers saw their loss probability reduced by 0.05. We found that forwarding packets via at most one intermediate RON node is sufficient to overcome faults and improve performance in most cases. These improvements, particularly in the area of fault detection and recovery, demonstrate the benefits of moving some of the control over routing into the hands of end-systems.

/pdf/resilient-overlay-networks-2see5sqvzl.pdf

Resilient overlay networks

/pdf/resilient-overlay-networks-e0tv6oh3hj.pdf

Overcast is an application-level multicasting system that can be incrementally deployed using today's Internet infrastructure. These properties stem from Overcast's implementation as an overlay network. An overlay network consists of a collection of nodes placed at strategic locations in an existing network fabric. These nodes implement a network abstraction on top of the network provided by the underlying substrate network.Overcast provides scalable and reliable single-source multicast using a simple protocol for building efficient data distribution trees that adapt to changing network conditions. To support fast joins, Overcast implements a new protocol for efficiently tracking the global status of a changing distribution tree.Results based on simulations confirm that Overcast provides its added functionality while performing competitively with IP Multicast. Simulations indicate that Overcast quickly builds bandwidth-efficient distribution trees that, compared to IP Multicast, provide 70%-100% of the total bandwidth possible, at a cost of somewhat less than twice the network load. In addition, Overcast adapts quickly to changes caused by the addition of new nodes or the failure of existing nodes without causing undue load on the multicast source.

Overcast: reliable multicasting with on overlay network

The present invention relates to techniques for controlling transfers of information in computer networks. One technique involves transmitting from a server computer to a client computer a document containing a channel object corresponding to a communication service, and storing an access ticket that indicates that a user of the client computer permits the information source computer to communicate with the user over a specified channel. Another technique involves transmitting smart digital offers based on information such as coupons and purchasing histories stored at the computer receiving the offer. Another technique involves transmitting from a server computer to a client computer a request for a user's personal profile information, and activating a client avatar that compares the request for personal profile information with a security profile of the user limiting access to personal profile information. Another technique involves transmitting from a server computer to a client computer a document containing an embedded link, activating the embedded link at the client computer and recording activation of the embedded link in a metering log.

Controlled transfer of information in computer networks

This thesis presents a new architecture for information discovery based on a hierarchy of content routers that provide both browsing and search services to end users. Content routers catalog information servers, which may in turn be other content routers. The resulting hierarchy of content routers and leaf servers provides a rich set of services to end users for locating information, including query re nement and query routing. Query re nement helps a user improve a query fragment to describe the user's interests more precisely. Once a query has been re ned and describes a manageable result set, query routing automatically forwards the query to relevant servers. These services make use of succinct descriptions of server contents called content labels. A unique contribution of this research is the demonstration of a scalable discovery architecture based on a hierarchical approach to routing. Results from four prototype content routing systems, based on both le system (NFS) and information system (HTTP) protocols, illustrate the practicality and utility of the content routing architecture. The experimental systems built using these prototypes provide access to local databases and le systems as well as over 500 Internet WAIS servers. Experimental work shows that system supported query re nement services provide users with an essential tool to manage the large number of matches that naive queries routinely generate in a large information system. Experience with the prototype systems suggests that the content routing architecture may scale to very large networks. Thesis Supervisor: David K. Gi ord Title: Professor of Computer Science

Content routing: a scalable architecture for network-based information discovery

: This report gives a defining description of the programming language FX-91. The FX (short for FX-91) programming language is designed to support the parallel implementation of applications that perform both symbolic and scientific computations. The unique features of FX include: (1) An effect system, to discover expression scheduling constraints. An effect is a static description of the side-effects an expression may perform when it is evaluated. Just as a type describes what an expression computes, an effect describes how an expression computes; (2) Abstraction over any kind of description, thus permitting first-class type and effect polymorphism. Effect polymorphism makes the FX effect system more powerful than previous approaches to side-effect analysis in the presence of first-class subroutines; (3) Type and effect inference, so that declaration free programs can be statically type and effect checked. FX also permits explicitly typed programs, and programs that use explicit types only for first-class polymorphic values and modules; and (4) First-class modules, which permit FX to serve as its own configuration language. It also includes an architecture independent module of parallel vector operators.

Report on the FX-91 Programming Language

We describe the first concurrent compacting garbage collector for a persistent heap. Client threads read and write the heap in primary memory, and can independently commit or about their write operations. When write operations are committed they are preserved in stab,.e storage and thus survive system failures. Clients can freely access the heap during a garbage collection because a replica of the heap is created by the stable replica collector. A log is maintained to capture client write operations. This log is used to support both the transaction system and the replication-based garbage collection algorithm. Experimental data from our implementations was obtained from a transactional version of the SML/.NJ compiler and modified versions of the TPC-B and OO1 database benchmarks. The pause time latency results show that the prototype implementation provides significantly better latencies than stop-and-copy collection. For small transactions, throughput is limited by the logging bandwidth of the underlying log manager. The results provide strong evidence that the replication copying algorithm imposes less overhead on transaction commit operations than other algorithms.

/pdf/concurrent-garbage-collection-of-persistent-heaps-2xy7gk2o2w.pdf

David K. Gifford

Papers

Overcast: reliable multicasting with on overlay network

Controlled transfer of information in computer networks

Content routing: a scalable architecture for network-based information discovery

Report on the FX-91 Programming Language

Concurrent garbage collection of persistent heaps