Fault tolerance can allow processes executing in a computer system to survive failures within the system. This thesis addresses the theory and practice of transparent fault-tolerance methods using message logging and checkpointing in distributed systems. A general model for reasoning about the behavior and correctness of these methods is developed, and the design, implementation, and performance of two new low-overhead methods based on this model are presented. No specialized hardware is required with these new methods.
The model is independent of the protocols used in the system. Each process state is represented by a dependency vector, and each system state is represented by a dependency matrix showing a collection of process states. The set of system states that have occurred during any single execution of a system forms a lattice, with the sets of consistent and recoverable system states as sublattices. There is thus always a unique maximum recoverable system state.
The first method presented uses a new pessimistic message logging protocol called sender-based message logging. Each message is logged in the local volatile memory of the machine from which it was sent, and the order in which the message was received is returned to the sender as a receive sequence number. Message logging overlaps execution of the receiver, until the receiver attempts to send a new message. Implemented in the V-System, the maximum measured failure-free overhead on distributed application programs was under 16 percent, and average overhead measured 2 percent or less, depending on problem size and communication intensity.
Optimistic message logging can outperform pessimistic logging, since message logging occurs asynchronously. A new optimistic message logging system is presented that guarantees to find the maximum possible recoverable system state, which is not ensured by previous optimistic methods. All logged messages and checkpoints are utilized, and thus some messages received by a process before it was checkpointed may not need to be logged. Although failure recovery using optimistic message logging is more difficult, failure-free application overhead using this method ranged form only a maximum of under 4 percent to much less than 1 percent.

https://www.cs.rice.edu/~dbj/pubs/Rice-COMP-TR89-101.pdf

Distributed system fault tolerance using message logging and checkpointing

A network file system called Multifile is described. It meets response, availability, and stability requirements as primitive functions. Multifile has a high degree of responsiveness because its component parts compete among themselves to service file requests; it has high availability because it maintains multiple copies of files; and it exhibits stable behavior over wise range of system parameters. The responsiveness of Multifile to read requests improves as the number of pages per request rises, implying that read ahead pages can profitably be cached at client sites. The throughput of Multifile improves as the request size increases and as the number of clients increases. As server load increases, the responsiveness of Multifile to read requests is stable in most configurations. The throughput of writes is unstable as the number of pages in the wire request rises, implying that write back pages should not be cached at client sites. The scale of events in file service is dominated by disk activity, so lost message exceptions do not occur frequently enough to affect file service; however, duplicate message exceptions are a factor in performance. >

Stability, availability, and response in network file service

A distributed facility for load sharing and parallel processing among workstations

A kernel is described, called the link kernel, for high-performance interprocess communication among shared-memory multiprocessors using an Ethernet. The link kernel provides asynchronous multicast communication service without protocol overhead. Links are created by listeners, and processes may be simultaneously talkers and listeners. Processes access links through a global link table. Multicast addresses are site-independent, so communication is independent of process locations in the network. Communication is asynchronous, so much of the overhead of message management is masked by parallel computation. >

A kernel for high-performance multicast communications

The authors describe the software architecture of a distributed facility called REM (remote execution manager), which has been developed to enable the user of an engineering workstation to benefit from the idle processing power of other workstations in a local network. REM has been implemented at application layer to make it portable without any need to modify the underlying kernel. The system provides the facility for load sharing as well as parallel processing within a local network of workstations. The REMs are grouped into functional units called suites, which are responsible for execution of different jobs submitted by the user. Several suites of REMs may exist in a network. Process replication is used to achieve fault tolerance in case a station crashes or is withdrawn from the pool of processors by its owner. >

http://ieeexplore.ieee.org/iel2/197/538/00011809.pdf

Jason Gait

Papers

A Distributed Process Manager with Transparent Continuation.

Highly Available, Enhanced Response File Service in Network Computers.