A fundamental problem that confronts peer-to-peer applications is to efficiently locate the node that stores a particular data item. This paper presents Chord, a distributed lookup protocol that addresses this problem. Chord provides support for just one operation: given a key, it maps the key onto a node. Data location can be easily implemented on top of Chord by associating a key with each data item, and storing the key/data item pair at the node to which the key maps. Chord adapts efficiently as nodes join and leave the system, and can answer queries even if the system is continuously changing. Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes.

Chord: A scalable peer-to-peer lookup service for internet applications

Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems.This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.

/pdf/dynamo-amazon-s-highly-available-key-value-store-2ozfk43ou4.pdf

Dynamo: amazon's highly available key-value store

A fundamental problem that confronts peer-to-peer applications is the efficient location of the node that stores a desired data item. This paper presents Chord, a distributed lookup protocol that addresses this problem. Chord provides support for just one operation: given a key, it maps the key onto a node. Data location can be easily implemented on top of Chord by associating a key with each data item, and storing the key/data pair at the node to which the key maps. Chord adapts efficiently as nodes join and leave the system, and can answer queries even if the system is continuously changing. Results from theoretical analysis and simulations show that Chord is scalable: Communication cost and the state maintained by each node scale logarithmically with the number of Chord nodes.

/pdf/chord-a-scalable-peer-to-peer-lookup-protocol-for-internet-3kpdq1okoa.pdf

Chord: a scalable peer-to-peer lookup protocol for Internet applications

Cassandra is a distributed storage system for managing very large amounts of structured data spread out across many commodity servers, while providing highly available service with no single point of failure. Cassandra aims to run on top of an infrastructure of hundreds of nodes (possibly spread across different data centers). At this scale, small and large components fail continuously. The way Cassandra manages the persistent state in the face of these failures drives the reliability and scalability of the software systems relying on this service. While in many ways Cassandra resembles a database and shares many design and implementation strategies therewith, Cassandra does not support a full relational data model; instead, it provides clients with a simple data model that supports dynamic control over data layout and format. Cassandra system was designed to run on cheap commodity hardware and handle high write throughput while not sacrificing read efficiency.

/pdf/cassandra-a-decentralized-structured-storage-system-xo5eclgx6x.pdf

Cassandra: a decentralized structured storage system

http://emamispnu.persiangig.com/Projects/Cloud%20Computing/science_2.pdf

Review: A survey on security issues in service delivery models of cloud computing

We describe a family of caching protocols for distrib-uted networks that can be used to decrease or eliminate the occurrence of hot spots in the network. Our protocols are particularly designed for use with very large networks such as the Internet, where delays caused by hot spots can be severe, and where it is not feasible for every server to have complete information about the current state of the entire network. The protocols are easy to implement using existing network protocols such as TCP/IP, and require very little overhead. The protocols work with local control, make efficient use of existing resources, and scale gracefully as the network grows. Our caching protocols are based on a special kind of hashing that we call consistent hashing. Roughly speaking, a consistent hash function is one which changes minimally as the range of the function changes. Through the development of good consistent hash functions, we are able to develop caching protocols which do not require users to have a current or even consistent view of the network. We believe that consistent hash functions may eventually prove to be useful in other applications such as distributed name servers and/or quorum systems.

/pdf/consistent-hashing-and-random-trees-distributed-caching-3h34vd88pi.pdf

Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web

The invention relates to a method and apparatus for distributing a request to one of a plurality of resources. A request is mapped to a location in mathematical mapping space. Each of the plurality of resources is mapped to a respective location or locations in the mathematical mapping space. The request is allocated to one of the resources based on a mathematical relationship between the request location and the resource location in the mathematical mapping space. An apparatus for distributing a request to one of a plurality of resources includes the plurality of resources and an input receiving a resource request. A mapper in communication with the input maps the request to a request location in a mathematical mapping space and maps each of the plurality of resources to at least one resource locations in the mathematical mapping space. A distributor distributes the request to one of the plurality of resources in response to a mathematical relationship between the locations of the request and resources in the mathematical mapping space. The invention also relates to a method and apparatus for requesting data from one of a plurality of servers. An virtual network of nodes is generated in response to a data request. A first node on the virtual network of nodes is chosen randomly, and a path generated from the first node to the root on the virtual network. At least one node on the path is mapped to a respective one of the plurality of servers and data is requested from a server. The node may be mapped to the server using the method and apparatus for distributing a request to one of a plurality of resources of the present invention.

Method and apparatus for distributing requests among a plurality of resources

Recently, several new algorithms have been developed for the minimum cut problem that substantially improve worst-case time bounds for the problem. These algorithms are very different from the earlier ones and from each other. We conduct an experimental evaluation of the relative performance of these algorithms. In the process, we develop heuristics and data structures that substantially improve practical performance of the algorithms. We also develop problem families for testing minimum cut algorithms. Our work leads to a better understanding of practical performance of the minimum cut algorithms and produces very efficient codes for the problem.

/pdf/experimental-study-of-minimum-cut-algorithms-4vsgabv8xs.pdf

Experimental study of minimum cut algorithms

Consider an n-vertex, m-edge, undirected graph with maximum flow value v. We give a method to find augmenting paths in such a graph in amortized sub-linear (O(npv)) time per path. This lets us improve the time bound of the classic augmenting path algorithm to O(m+ nv3=2) on simple graphs. The addition of a blocking flow subroutine gives a simple, deterministic O(nm2=3v1=6)-time algorithm. We also use our technique to improve known randomized algorithms, giving O(m+nv5=4)-time and O(m+n11=9v)-time algorithms for capacitated undirected graphs. For simple graphs, in which v n, the last bound is O(n2:2), improving on the best previous bound of O(n2:5), which is also the best known time bound for bipartite matching.

/pdf/finding-maximum-flows-in-undirected-graphs-seems-easier-than-1xiwg49vmi.pdf

Finding maximum flows in undirected graphs seems easier than bipartite matching

Consider an n-vertex, m-edge, undirected graph with maximum flow value v. We give a new O(m+nv)-time maximum flow algorithm based on finding augmenting paths in random samples of the edges of residual graphs. After assigning certain special sampling probabilities to edges in O(m) time, our algorithm is very simple: repeatedly find an augmenting path in a random sample of edges from the residual graph.

/pdf/random-sampling-in-residual-graphs-27qs9hfmx8.pdf

Matthew S. Levine

Papers

Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web

Method and apparatus for distributing requests among a plurality of resources

Experimental study of minimum cut algorithms

Finding maximum flows in undirected graphs seems easier than bipartite matching

Random sampling in residual graphs