Technological progress in integrated, low-power, CMOS communication devices and sensors makes a rich design space of networked sensors viable. They can be deeply embedded in the physical world and spread throughout our environment like smart dust. The missing elements are an overall system architecture and a methodology for systematic advance. To this end, we identify key requirements, develop a small device that is representative of the class, design a tiny event-driven operating system, and show that it provides support for efficient modularity and concurrency-intensive operation. Our operating system fits in 178 bytes of memory, propagates events in the time it takes to copy 1.25 bytes of memory, context switches in the time it takes to copy 6 bytes of memory and supports two level scheduling. The analysis lays a groundwork for future architectural advances.

/pdf/system-architecture-directions-for-networked-sensors-dm8wnczmkp.pdf

System architecture directions for networked sensors

This paper describes a new replication algorithm that is able to tolerate Byzantine faults. We believe that Byzantinefault-tolerant algorithms will be increasingly important in the future because malicious attacks and software errors are increasingly common and can cause faulty nodes to exhibit arbitrary behavior. Whereas previous algorithms assumed a synchronous system or were too slow to be used in practice, the algorithm described in this paper is practical: it works in asynchronous environments like the Internet and incorporates several important optimizations that improve the response time of previous algorithms by more than an order of magnitude. We implemented a Byzantine-fault-tolerant NFS service using our algorithm and measured its performance. The results show that our service is only 3% slower than a standard unreplicated NFS.

/pdf/practical-byzantine-fault-tolerance-58gb0de0hd.pdf

Practical Byzantine fault tolerance

Multiple applications request data from multiple storage units over a computer network. The data is divided into segments and each segment is distributed randomly on one of several storage units, independent of the storage units on which other segments of the media data are stored. At least one additional copy of each segment also is distributed randomly over the storage units, such that each segment is stored on at least two storage units. This random distribution of multiple copies of segments of data improves both scalability and reliability. When an application requests a selected segment of data, the request is processed by the storage unit with the shortest queue of requests. Random fluctuations in the load applied by multiple applications on multiple storage units are balanced nearly equally over all of the storage units. This combination of techniques results in a system which can transfer multiple, independent high-bandwidth streams of data in a scalable manner in both directions between multiple applications and multiple storage units.

Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner

Distributed operating systems have many aspects in common with centralized ones, but they also differ in certain ways This paper is intended as an introduction to distributed operating systems, and especially to current university research about them After a discussion of what constitutes a distributed operating system and how it is distinguished from a computer network, various key design issues are discussed Then several examples of current research projects are examined in some detail, namely, the Cambridge Distributed Computing System, Amoeba, V, and Eden

https://cds.cern.ch/record/400319/files/p101.pdf

Distributed Operating Systems

Multiple applications request data from multiple storage units over a computer network. The data is divided into segments and each segment is distributed randomly on one of several storage units, independent of the storage units on which other segments of the media data are stored. Redundancy information corresponding to each segment also is distributed randomly over the storage units. The redundancy information for a segment may be a copy of the segment, such that each segment is stored on at least two storage units. The redundancy information also may be based on two or more segments. This random distribution of segments of data and corresponding redundancy information improves both scalability and reliability. When a storage unit fails, its load is distributed evenly over to remaining storage units and its lost data may be recovered because of the redundancy information. When an application requests a selected segment of data, the request may be processed by the storage unit with the shortest queue of requests. Random fluctuations in the load applied by multiple applications on multiple storage units are balanced nearly equally over all of the storage units. Small data files also may be stored on storage units that combine small files into larger segments of data using a log structured file system. This combination of techniques results in a system which can transfer both multiple, independent high-bandwidth streams of data and small data files in a scalable manner in both directions between multiple applications and multiple storage units.

Computer system and process for transferring streams of data between multiple storage units and multiple applications in a scalable and reliable manner

As we approach nation-wide integration of computer systems, it is clear that le replication will play a key role, both to improve data availability in the face of failures, and to improve performance by locating data near where it will be used. We expect that future le systems will have an extensible, modular structure in which features such as replication can be \slipped in" as a transparent layer in a stackable layered architecture. We introduce the Ficus replicated le system for NFS and show how it is layered on top of existing le systems. The Ficus le system di ers from previous le replication services in that it permits update during network partition if any copy of a le is accessible. File and directory updates are automatically propagated to accessible replicas. Con icting updates to directories are detected and automatically repaired; con icting updates to ordinary les are detected and reported to the owner. The frequency of communications outages rendering inaccessible some replicas in a large scale network and the relative rarity of con icting updates make this optimistic scheme attractive. Stackable layers facilitate the addition of new features to an existing le system without reimplementing existing functions. This is done in a manner analogous to object-oriented programming with inheritance. By structuring the le This work was sponsored by DARPA under contract number F29601-87-C-0072. This author is also associated with Locus Computing Corporation. system as a stack of modules, each with the same interface, modules which augment existing services can be added transparently. This paper describes the implementation of the Ficus le system using the layered architecture.

Implementation of the Ficus Replicated File System.

This research proposes and tests an approach to engineering distributed file systems that are aimed at wide-scale, Internet-based use. The premise is that replication is essential to deliver performance and availability, yet the traditional conservative replica consistency algorithms do not scale to this environment. Our Ficus replicated file system uses a single-copy availability, optimistic update policy with reconciliation algorithms that reliably detect concurrent updates and automatically restore the consistency of directory replicas. The system uses the peer-to-peer model in which all machines are architectural equals but still permits configuration in a client-server arrangement where appropriate. Ficus has been used for six years at several geographically scattered installations. This paper details and evaluates the use of optimistic replica consistency, automatic update conflict detection and repair, the peer-to-peer (as opposed to client-server) interaction model, and the stackable file system architecture in the design and construction of Ficus. The paper concludes with a number of lessons learned from the experience of designing, building, measuring, and living with an optimistically replicated file system. © 1998 John Wiley & Sons, Ltd.

Perspectives on optimistically replicated, peer-to-peer filing

Ficus is a flexible replication facility with optimistic concurrency control designed to span a wide range of scales and network environments. Support for partitioned operation is fundamental to the Ficus design but was not widely exercised in early Ficus use. This paper reports recent experiences using Ficus in settings where some replicas are only occasionally connected to a network, and hence partitioned operation is the rule rather than the exception. The authors conclude that with some tuning, Ficus adapted quite well to primarily disconnected operation. >

Primarily disconnected operation: experiences with Ficus

Ficus is a replicated general filing environment for Unix intended to scale to very large (nationwide) networks. The system uses an optimistic one copy availability model in which conflicting updates to the file system's directory information are automatically reconciled, while conflicting file updates are reliably detected and reported. The system architecture is based on a stackable layers methodology which permits a high degree of modularity and extensibility of file system services. The motivations for replication are presented, and the case for optimistic concurrency control for large-scale distributed file systems is summarized. A brief description of the Ficus file system is presented and a number of outstanding issues which must be addressed are outlined. >

/pdf/replication-in-ficus-distributed-file-systems-3c0x219s6e.pdf

Thomas W. Page

Papers

Implementation of the Ficus Replicated File System.

Perspectives on optimistically replicated, peer-to-peer filing

Primarily disconnected operation: experiences with Ficus

Replication in Ficus distributed file systems

Transactions and synchronization in a distributed operating system