GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters. GPFS is used on many of the largest supercomputers in the world. GPFS was built on many of the ideas that were developed in the academic community over the last several years, particularly distributed locking and recovery technology. To date it has been a matter of conjecture how well these ideas scale. We have had the opportunity to test those limits in the context of a product that runs on the largest systems in existence. While in many cases existing ideas scaled well, new approaches were necessary in many key areas. This paper describes GPFS, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.

/pdf/gpfs-a-shared-disk-file-system-for-large-computing-clusters-35v610gyrf.pdf

GPFS: A Shared-Disk File System for Large Computing Clusters

Farsite is a secure, scalable file system that logically functions as a centralized file server but is physically distributed among a set of untrusted computers. Farsite provides file availability and reliability through randomized replicated storage; it ensures the secrecy of file contents with cryptographic techniques; it maintains the integrity of file and directory data with a Byzantine-fault-tolerant protocol; it is designed to be scalable by using a distributed hint mechanism and delegation certificates for pathname translations; and it achieves good performance by locally caching file data, lazily propagating file updates, and varying the duration and granularity of content leases. We report on the design of Farsite and the lessons we have learned by implementing much of that design.

/pdf/farsite-federated-available-and-reliable-storage-for-an-3zqapv0sn7.pdf

Farsite: federated, available, and reliable storage for an incompletely trusted environment

We collected file system content data from 857 desktop computers at Microsoft over a span of 4 weeks. We analyzed the data to determine the relative efficacy of data deduplication, particularly considering whole-file versus block-level elimination of redundancy. We found that whole-file deduplication achieves about three quarters of the space savings of the most aggressive block-level deduplication for storage of live file systems, and 87p of the savings for backup images. We also studied file fragmentation, finding that it is not prevalent, and updated prior file system metadata studies, finding that the distribution of file sizes continues to skew toward very large unstructured files.

/pdf/a-study-of-practical-deduplication-27hbsh4ogr.pdf

A study of practical deduplication

HiStar is a new operating system designed to minimize the amount of code that must be trusted. HiStar provides strict information flow control, which allows users to specify precise data security policies without unduly limiting the structure of applications. HiStar's security features make it possible to implement a Unix-like environment with acceptable performance almost entirely in an untrusted user-level library. The system has no notion of superuser and no fully trusted code other than the kernel. HiStar's features permit several novel applications, including privacy-preserving, untrusted virus scanners and a dynamic Web server with only a few thousand lines of trusted code.

/pdf/making-information-flow-explicit-in-histar-2czr309b8o.pdf

Making information flow explicit in HiStar

HiStar is a new operating system designed to minimize the amount of code that must be trusted. HiStar provides strict information flow control, which allows users to specify precise data security policies without unduly limiting the structure of applications. HiStar's security features make it possible to implement a Unix-like environment with acceptable performance almost entirely in an untrusted user-level library. The system has no notion of superuser and no fully trusted code other than the kernel. HiStar's features permit several novel applications, including an entirely untrusted login process, separation of data between virtual private networks, and privacy-preserving, untrusted virus scanners.

/pdf/making-information-flow-explicit-in-histar-2scxsrw63s.pdf

In this paper we describe the architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system It is a general purpose file system for use on both workstations and servers The focus of the paper is on the mechanisms used by XFS to scale capacity and performance in supporting very large file systems The large file system support includes mechanisms for managing large files, large numbers of files, large directories, and very high performance I/O

In discussing the mechanisms used for scalability we include both descriptions of the XFS on-disk data structures and analyses of why they were chosen We discuss in detail our use of B+ trees in place of many of the more traditional linear file system structures

XFS has been shipping to customers since December of 1994 in a version of IRIX 53, and we are continuing to improve its performance and add features in upcoming releases We include performance results from running on the latest version of XFS to demonstrate the viability of our design

/pdf/scalability-in-the-xfs-file-system-59t6hnu8xf.pdf

Mike Nishimoto

Papers

Scalability in the XFS file system