Horus: a flexible group communication system
TL;DR: The Horus system offers flexible group communication support for distributed applications, allowing applications to only pay for services they use, and for groups with different communication needs to coexist in a single system.
Abstract: The Horus system offers flexible group communication support for distributed applications. It is extensively layered and highly reconfigurable, allowing applications to only pay for services they use, and for groups with different communication needs to coexist in a single system. The approach encourages experimentation with new communication properties and incremental extension of the system, and enables us to support a variety of application-oriented interfaces.
Citations
More filters
Proceedings Article•
23 Jun 2010TL;DR: ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all requests that change the ZooKeeper state to enable the implementation of a high performance processing pipeline with read requests being satisfied by local servers.
Abstract: In this paper, we describe ZooKeeper, a service for coordinating processes of distributed applications Since ZooKeeper is part of critical infrastructure, ZooKeeper aims to provide a simple and high performance kernel for building more complex coordination primitives at the client It incorporates elements from group messaging, shared registers, and distributed lock services in a replicated, centralized service The interface exposed by Zoo-Keeper has the wait-free aspects of shared registers with an event-driven mechanism similar to cache invalidations of distributed file systems to provide a simple, yet powerful coordination service
The ZooKeeper interface enables a high-performance service implementation In addition to the wait-free property, ZooKeeper provides a per client guarantee of FIFO execution of requests and linearizability for all requests that change the ZooKeeper state These design decisions enable the implementation of a high performance processing pipeline with read requests being satisfied by local servers We show for the target workloads, 2:1 to 100:1 read to write ratio, that ZooKeeper can handle tens to hundreds of thousands of transactions per second This performance allows ZooKeeper to be used extensively by client applications
1,637 citations
Cites background from "Horus: a flexible group communicati..."
...Horus [30] and Ensemble [31] are systems that evolved from ISIS....
[...]
TL;DR: The specification framework presented in this article will help builders of group communication systems understand andspecify their service semantics; the extensive survey will allow them to compare their service to others, and serve as a unified framework for the classification, analysis, and comparison of group Communication systems.
Abstract: View-oriented group communication is an important and widely used building block for many distributed applications. Much current research has been dedicated to specifying the semantics and services of view-oriented group communication systems (GCSs). However, the guarantees of different GCSs are formulated using varying terminologies and modeling techniques, and the specifications vary in their rigor. This makes it difficult to analyze and compare the different systems. This survey provides a comprehensive set of clear and rigorous specifications, which may be combined to represent the guarantees of most existing GCSs. In the light of these specifications, over 30 published GCS specifications are surveyed. Thus, the specifications serve as a unifying framework for the classification, analysis, and comparison of group communication systems. The survey also discusses over a dozen different applications of group communication systems, shedding light on the usefulness of the presented specifications. This survey is aimed at both system builders and theoretical researchers. The specification framework presented in this article will help builders of group communication systems understand and specify their service semantics; the extensive survey will allow them to compare their service to others. Application builders will find a guide here to the services provided by a large variety of GCSs, which could help them choose the GCS appropriate for their needs. The formal framework may provide a basis for interesting theoretical work, for example, analyzing relative strengths of different properties and the costs of implementing them.
734 citations
Cites background from "Horus: a flexible group communicati..."
...All the GCSs that we are aware of satisfy Self Delivery; some examples are: Isis, Transis, Totem, Horus, and Newtop....
[...]
...Several group communication systems (e.g., Ensemble, Horus, and RMP) provide a reliable FIFO service type that satis.es Property 6.2 and does not impose additional ordering constraints. xAMp provides several service levels that satisfy Property 6.1 but vary by their reliability guarantees....
[...]
...Instead, the Horus STABLE layer maintains a more general stability matrix at each process....
[...]
...Strong Total Order is provided by Totem and by some of the implementations of totally ordered multicast in Transis, Ensemble, Phoenix, RMP, and Horus....
[...]
...Horus does not deliver safe pre.x noti.cations....
[...]
[...]
TL;DR: This article introduces the protocol, provides a theoretical analysis of its behavior, review experimental results, and discusses some candidate applications, confirming that bimodal multicast is reliable, scalable, and that the protocol provides remarkably stable delivery throughput.
Abstract: There are many methods for making a multicast protocol “reliable.” At one end of the spectrum, a reliable multicast protocol might offer tomicity guarantees, such as all-or-nothing delivery, delivery ordering, and perhaps additional properties such as virtually synchronous addressing. At the other are protocols that use local repair to overcome transient packet loss in the network, offering “best effort” reliability. Yet none of this prior work has treated stability of multicast delivery as a basic reliability property, such as might be needed in an internet radio, television, or conferencing application. This article looks at reliability with a new goal: development of a multicast protocol which is reliable in a sense that can be rigorously quantified and includes throughput stability guarantees. We characterize this new protocol as a “bimodal multicast” in reference to its reliability model, which corresponds to a family of bimodal probability distributions. Here, we introduce the protocol, provide a theoretical analysis of its behavior, review experimental results, and discuss some candidate applications. These confirm that bimodal multicast is reliable, scalable, and that the protocol provides remarkably stable delivery throughput.
693 citations
Cites background or methods from "Horus: a flexible group communicati..."
...In addition to the work reported here, our group at Cornell also explored other uses of gossip, such as gossip-based membership tracking [van Renesse et al. 1996] and gossipbased stability detection [Guo 1998]....
[...]
...Ensemble supports group-communication protocol stacks that are constructed by composing microprotocols, an idea that originated in the Horus project [Birman 1997; van Renesse et al. 1996]....
[...]
...Spinglass uses gossip to track membership as well as to do communication [van Renesse et al. 1996], but the behavior of the bimodal protocol is unaffected (formal analysis of the combined gossip mechanisms is, however, beyond our current ability)....
[...]
07 Mar 2004
TL;DR: This work has designed scalable mechanisms to distribute the game state to the participating players and to maintain consistency in the face of node failures, and has implemented a simple game called SimMud, and experimented with up to 4000 players to demonstrate the applicability of this approach.
Abstract: We present an approach to support massively multiplayer games on peer-to-peer overlays. Our approach exploits the fact that players in MMGs display locality of interest, and therefore can form self-organizing groups based on their locations in the virtual world. To this end, we have designed scalable mechanisms to distribute the game state to the participating players and to maintain consistency in the face of node failures. The resulting system dynamically scales with the number of online players. It is more flexible and has a lower deployment cost than centralized games servers. We have implemented a simple game we call SimMud, and experimented with up to 4000 players to demonstrate the applicability of this approach.
578 citations
Cites background from "Horus: a flexible group communicati..."
...0 20 40 60 80 100 Percentage of nodes [0-10] [10-20] [20-30] [30-40] [40-50] [50-60] [60-70] [70-80] [80-90] [90-100] [100-110] [110-120] [120-130] [130-140] [140-150] [150-160] [160-170] [170-180] [180-190] [190-200] [200-210] [210-220]...
[...]
...Fault tolerant consistent data services can also be built on top of view-synchronous group communication [15], [30] using...
[...]
...0 10 20 30 40 Percentage of nodes [0-10] [10-20] [20-30] [30-40] [40-50] [50-60] [60-70] [70-80] [80-90] [90-100] [100-110] [110-120]...
[...]
[...]
TL;DR: In this article, the authors present a survey of the field of process migration by summarizing the key concepts and giving an overview of the most important implementations, including MOSIX, Sprite, Mach, and Load Sharing Facility.
Abstract: Process migration is the act of transferring a process between two machines. It enables dynamic load distribution, fault resilience, eased system administration, and data access locality. Despite these goals and ongoing research efforts, migration has not achieved widespread use. With the increasing deployment of distributed systems in general, and distributed operating systems in particular, process migration is again receiving more attention in both research and product development. As high-performance facilities shift from supercomputers to networks of workstations, and with the ever-increasing role of the World Wide Web, we expect migration to play a more important role and eventually to be widely adopted.This survey reviews the field of process migration by summarizing the key concepts and giving an overview of the most important implementations. Design and implementation issues of process migration are analyzed in general, and then revisited for each of the case studies described: MOSIX, Sprite, Mach, and Load Sharing Facility. The benefits and drawbacks of process migration depend on the details of implementation and, therefore, this paper focuses on practical matters. This survey will help in understanding the potentials of process migration and why it has not caught on.
551 citations
References
More filters
01 Aug 1990
TL;DR: This paper identifies two new design principles, Application Level Framing and Integrated Layer Processing, and identifies the presentation layer as a key aspect of overall protocol performance.
Abstract: The current generation of protocol architectures, such as TCP/IP or the ISO suite, seem successful at meeting the demands of todays networks. However, a number of new requirements have been proposed for the networks of tomorrow, and some innovation in protocol structuring may be necessary. In this paper, we review some key requirements for tomorrow's networks, and propose some architectural principles to structure a new generation of protocols. In particular, this paper identifies two new design principles, Application Level Framing and Integrated Layer Processing. Additionally, it identifies the presentation layer as a key aspect of overall protocol performance.
1,083 citations
01 Oct 1995
TL;DR: An adaptive algorithm that uses the results of previous loss recovery events to adapt the control parameters used for future loss recovery is demonstrated, and the reliable multicast delivery algorithm provides good performance over a wide range of underlying topologies.
Abstract: This paper describes SRM (Scalable Reliable Multicast), a reliable multicast framework for application level framing and light-weight sessions. The algorithms of this framework are efficient, robust, and scale well to both very large networks and very large sessions. The framework has been prototyped in wb, a distributed whiteboard application, and has been extensively tested on a global scale with sessions ranging from a few to more than 1000 participants. The paper describes the principles that have guided our design, including the IP multicast group delivery model, an end-to-end, receiver-based model of reliability, and the application level framing protocol model. As with unicast communications, the performance of a reliable multicast delivery algorithm depends on the underlying topology and operational environment. We investigate that dependence via analysis and simulation, and demonstrate an adaptive algorithm that uses the results of previous loss recovery events to adapt the control parameters used for future loss recovery. With the adaptive algorithm, our reliable multicast delivery algorithm provides good performance over a wide range of underlying topologies.
753 citations
IBM1
TL;DR: A probabilistic method is proposed for reading remote clocks in distributed systems subject to unbounded random communication delays and can achieve clock synchronization precisions superior to those attainable by previously published clock synchronization algorithms.
Abstract: A probabilistic method is proposed for reading remote clocks in distributed systems subject to unbounded random communication delays. The method can achieve clock synchronization precisions superior to those attainable by previously published clock synchronization algorithms. Its use is illustrated by presenting a time service which maintains externally (and hence, internally) synchronized clocks in the presence of process, communication and clock failures.
620 citations
TL;DR: In this article, the authors describe protocols to implement a fault-tolerant computing system, which augment the hypervisor of a virtual machine manager and coordinate a primary virtual machine with its backup.
Abstract: Protocols to implement a fault-tolerant computing system are described. These protocols augment the hypervisor of a virtual-machine manager and coordinate a primary virtual machine with its backup. No modifications to the hardware, operating system, or application programs are required. A prototype system was constructed for HP's PA-RISC instruction-set architecture. Even though the prototype was not carefully tuned, it ran programs about a factor of 2 slower than a bare machine would.
480 citations