There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

This paper describes a new replication algorithm that is able to tolerate Byzantine faults. We believe that Byzantinefault-tolerant algorithms will be increasingly important in the future because malicious attacks and software errors are increasingly common and can cause faulty nodes to exhibit arbitrary behavior. Whereas previous algorithms assumed a synchronous system or were too slow to be used in practice, the algorithm described in this paper is practical: it works in asynchronous environments like the Internet and incorporates several important optimizations that improve the response time of previous algorithms by more than an order of magnitude. We implemented a Byzantine-fault-tolerant NFS service using our algorithm and measured its performance. The results show that our service is only 3% slower than a standard unreplicated NFS.

/pdf/practical-byzantine-fault-tolerance-58gb0de0hd.pdf

Practical Byzantine fault tolerance

Fabric is a modular and extensible open-source system for deploying and operating permissioned blockchains and one of the Hyperledger projects hosted by the Linux Foundation (www.hyperledger.org). Fabric is the first truly extensible blockchain system for running distributed applications. It supports modular consensus protocols, which allows the system to be tailored to particular use cases and trust models. Fabric is also the first blockchain system that runs distributed applications written in standard, general-purpose programming languages, without systemic dependency on a native cryptocurrency. This stands in sharp contrast to existing block-chain platforms that require "smart-contracts" to be written in domain-specific languages or rely on a cryptocurrency. Fabric realizes the permissioned model using a portable notion of membership, which may be integrated with industry-standard identity management. To support such flexibility, Fabric introduces an entirely novel blockchain design and revamps the way blockchains cope with non-determinism, resource exhaustion, and performance attacks. This paper describes Fabric, its architecture, the rationale behind various design decisions, its most prominent implementation aspects, as well as its distributed application programming model. We further evaluate Fabric by implementing and benchmarking a Bitcoin-inspired digital currency. We show that Fabric achieves end-to-end throughput of more than 3500 transactions per second in certain popular deployment configurations, with sub-second latency, scaling well to over 100 peers.

https://dl.acm.org/doi/pdf/10.1145/3190508.3190538

Hyperledger fabric: a distributed operating system for permissioned blockchains

Our growing reliance on online services accessible on the Internet demands highly available systems that provide correct service without interruptions. Software bugs, operator mistakes, and malicious attacks are a major cause of service interruptions and they can cause arbitrary behavior, that is, Byzantine faults. This article describes a new replication algorithm, BFT, that can be used to build highly available systems that tolerate Byzantine faults. BFT can be used in practice to implement real services: it performs well, it is safe in asynchronous environments such as the Internet, it incorporates mechanisms to defend against Byzantine-faulty clients, and it recovers replicas proactively. The recovery mechanism allows the algorithm to tolerate any number of faults over the lifetime of the system provided fewer than 1/3 of the replicas become faulty within a small window of vulnerability. BFT has been implemented as a generic program library with a simple interface. We used the library to implement the first Byzantine-fault-tolerant NFS file system, BFS. The BFT library and BFS perform well because the library incorporates several important optimizations, the most important of which is the use of symmetric cryptography to authenticate messages. The performance results show that BFS performs 2p faster to 24p slower than production implementations of the NFS protocol that are not replicated. This supports our claim that the BFT library can be used to build practical systems that tolerate Byzantine faults.

/pdf/practical-byzantine-fault-tolerance-and-proactive-recovery-41bx19pxe1.pdf

Practical byzantine fault tolerance and proactive recovery

Wireless networks are fundamentally limited by the intensity of the received signals and by their interference. Since both of these quantities depend on the spatial location of the nodes, mathematical techniques have been developed in the last decade to provide communication-theoretic results accounting for the networks geometrical configuration. Often, the location of the nodes in the network can be modeled as random, following for example a Poisson point process. In this case, different techniques based on stochastic geometry and the theory of random geometric graphs -including point process theory, percolation theory, and probabilistic combinatorics-have led to results on the connectivity, the capacity, the outage probability, and other fundamental limits of wireless networks. This tutorial article surveys some of these techniques, discusses their application to model wireless networks, and presents some of the main results that have appeared in the literature. It also serves as an introduction to the field for the other papers in this special issue.

/pdf/stochastic-geometry-and-random-graphs-for-the-analysis-and-1e671n4af3.pdf

Stochastic geometry and random graphs for the analysis and design of wireless networks

A stable property in a distributed system is a global property which once true, remains true forever. This paper refines this notion by formally introducing the concept of strong stable properties. A strong stable property has the nice property that it can be correctly evaluated on the consistent part of uncoordinated snapshots. Termination and deadlock are shown to be strong stable properties, whereas distributed garbage is not. We also show how to derive a simple generic algorithm for the detection of a strong stable property. The generic algorithm is illustrated by two examples: termination detection and deadlock detection. Incidentally the paper presents a very simple algorithm for termination detection.

Strong stable properties in distributed systems

This paper addresses the problem of atomic multicasting messages in asynchronous systems with unreliable failure detection. First, we give a characterisation of the notion of algorithms tolerating unreliable failure detection. Second, we give a characterisation of the notion of genuine atomic multicast. This characterisation leads to a better understanding of the difference between atomic multicast and atomic broadcast, and to a clear distinction between genuine atomic multicast algorithms and non-genuine atomic multicast algorithms. Third, we show that in a system with at least three processes, the genuine atomic multicast problem cannot be solved with unreliable failure detection, if at least one process may crash. We mention however two ways of circumventing this impossibility result.

Genuine Atomic Multicast

Replication is a key mechanism for developing fault-tolerant and highly available applications. In this paper, we present a replication framework for replicating CORBA objects that combines the active and passive replication techniques. We show how we have used axiomatic properties of a consensus protocol together with a generic architectural framework to bridge the gap between active and passive replication. Our framework makes it possible to dynamically associate replication techniques to individual operations of a replicated object, without requiring the client to even know that the object is replicated. Augmenting CORBA with a replication service is a big step towards reliable and interoperable distributed systems.

/pdf/replicating-corba-objects-a-marriage-between-active-and-1wk4cyeaym.pdf

Replicating CORBA objects: a marriage between active and passive replication

A shared memory built on top of a distributed system constitutes a distributed shared memory (DSM). If a lot of protocols implementing DSMS in various contexts have been proposed, no set of homogeneous definitions has been given for the many semantics offered by these implementations. This paper provides a suite of such definitions for atomic, sequential, causal, PRAM and a few others consistency criteria. These definitions are based on a unique framework : a parallel computation is defined as a partial order on the set of read and write operations invoked by processes, and a consistency criterion is defined as a constraint on this partial order. Such an approach provides a simple classification of consistency criteria, from the more to the less constrained one. This paper can also be considered as a survey on consistency criteria for DSMS.

/pdf/a-suite-of-definitions-for-consistency-criteria-in-3b8eifaf1x.pdf

A suite of definitions for consistency criteria in distributed shared memories

We give a lower bound of $\Omega(n)$ rounds for broadcasting in a mobile ad hoc network, where $n$ is the number of nodes in the network. A round is the time taken by a node to successfully transmit a message to all its neighbors. It has been shown by Bruschi et al. and Chlebus et al. that a minimum of $\Omega(\log n)$ time-slots are required in a round to propagate the broadcast message by one hop. In static networks, this gives us the lower bound for network-wide broadcast to be $\Omega(D \log n)$ time-slots, where $D$ is its diameter. Although this lower bound is valid for a mobile network, we obtain a tighter lower bound of $\Omega(n \log n)$ time-slots by considering explicit node mobility. This result is valid even when $D \ll n$ and the network diameter never exceeds $D$. This shows that the dominating factor in the complexity of broadcasting in a mobile network is the number of nodes in the network and not its diameter.

André Schiper

Papers

Strong stable properties in distributed systems

Genuine Atomic Multicast

Replicating CORBA objects: a marriage between active and passive replication

A suite of definitions for consistency criteria in distributed shared memories

A Lower Bound for Broadcasting in Mobile Ad Hoc Networks