Cassandra is a distributed storage system for managing very large amounts of structured data spread out across many commodity servers, while providing highly available service with no single point of failure. Cassandra aims to run on top of an infrastructure of hundreds of nodes (possibly spread across different data centers). At this scale, small and large components fail continuously. The way Cassandra manages the persistent state in the face of these failures drives the reliability and scalability of the software systems relying on this service. While in many ways Cassandra resembles a database and shares many design and implementation strategies therewith, Cassandra does not support a full relational data model; instead, it provides clients with a simple data model that supports dynamic control over data layout and format. Cassandra system was designed to run on cheap commodity hardware and handle high write throughput while not sacrificing read efficiency.

/pdf/cassandra-a-decentralized-structured-storage-system-xo5eclgx6x.pdf

Cassandra: a decentralized structured storage system

The increasing popularity of blockchain-based cryptocurrencies has made scalability a primary and urgent concern. We analyze how fundamental and circumstantial bottlenecks in Bitcoin limit the ability of its current peer-to-peer overlay network to support substantially higher throughputs and lower latencies. Our results suggest that reparameterization of block size and intervals should be viewed only as a first increment toward achieving next-generation, high-load blockchain protocols, and major advances will additionally require a basic rethinking of technical approaches. We offer a structured perspective on the design space for such approaches. Within this perspective, we enumerate and briefly discuss a number of recently proposed protocol ideas and offer several new ideas and open challenges.

On Scaling Decentralized Blockchains

The 2008 LADIS workshop on Large Scale Distributed Systems brought together leaders from the commercial cloud computing community with researchers working on a variety of topics in distributed computing. The dialog yielded some surprises: some hot research topics seem to be of limited near-term importance to the cloud builders, while some of their practical challenges seem to pose new questions to us as systems researchers. This brief note summarizes our impressions.

/pdf/toward-a-cloud-computing-research-agenda-21jy7qmaks.pdf

Toward a cloud computing research agenda

We consider the problem of eff- ciently reconciling two similar sets held by different hosts while minimizing the communication complex- ity. This type of problem arises naturally from gossip protocols used for the distribution of information, but has other applications as well. We describe an ap- proach to such reconciliation based on the encoding of sets as polynomials. The resulting protocols ex- hibit tractable computational complexity and nearly optimal communication complexity. Moreover, these protocols can be adapted to work over a broadcast channel, allowing many clients to reconcile with one host based on a single broadcast.

Set Reconciliation with N early Optimal Communication Complexity

A blockchain is a distributed ledger in which participants that do not fully trust each other agree on the ledger's content by running a consensus algorithm. This technology is raising a lot of interest both in academia and industry, but the lack of tools to evaluate design and implementation decisions may hamper fast progress. To address this issue, this paper presents a discrete-event simulator that is flexible enough to evaluate different blockchain implementations. These blockchains can be rapidly modeled and simulated by extending existing models. Running Bitcoin and Ethereum simulations allowed us to change conditions and answer different questions about their performance. For example, we concluded that doubling the number of transactions per block has a low impact on the block propagation delay (10ms) and that encrypting communication has a high impact in that delay (more than 25%).

BlockSim: Blockchain Simulator

The paper shows that anti-entropy protocols can process only a limited rate of updates, and proposes and evaluates a new state reconciliation mechanism as well as a flow control scheme for anti-entropy protocols.

/pdf/efficient-reconciliation-and-flow-control-for-anti-entropy-2gxv4s4d6u.pdf

Efficient reconciliation and flow control for anti-entropy protocols

Systems and methods provide logic for decentralized request routing. In one implementation, a computer-implemented method routes a request. The method includes receiving, by a client server, the request. The client server evaluates feedback information received by the client server from at least one of a plurality of host servers. The client server selects one of the plurality of host servers to process the request, based on the evaluation, and transmits the request to the selected one of the plurality of host servers.

Decentralized request routing

Systems and methods provide automated provisioning of load balancing. In one implementation, a computer-implemented method configures an access point. A request to create the access point may be received and data for the access point may be stored in a data store. The method may invoke a workflow engine. The workflow engine may assign the access point to a load balancer, configure the load balancer, assign a network address to the load balancer, and publish the network address for a plurality of clients.

Automated load balancing architecture

A load balancer obtains respective workload metrics of various worker nodes of a distributed system. In response to determining that a work request is to be assigned, the load balancer generates a workload variation estimate associated with the worker nodes. Using the workload variation estimate, the load balancer determines a size of a candidate pool of worker nodes for the work request. The load balancer assigns the work request to a selected worker node from the candidate pool.

Christopher C. Thomas

Papers

Efficient reconciliation and flow control for anti-entropy protocols

Decentralized request routing

Automated load balancing architecture

Distributed load balancing with imperfect workload information