Open AccessProceedings Article
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank B. Schmuck,Roger L. Haskin +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.Citations
More filters
Proceedings ArticleDOI
Fault Tolerance and Recovery of Scientific Workflows on Computational Grids
TL;DR: Results from real usage scenarios in LEAD show that the failure rate of individual steps in workflows decreases from about 30% to 5% by using the design and implementation of two mechanisms for fault-tolerance and recovery for complex scientific workflows on computational grids.
Proceedings ArticleDOI
Scaling parallel I/O performance through I/O delegate and caching system
TL;DR: A portable MPI-IO layer is proposed where certain tasks, such as file caching, consistency control, and collective I/O optimization are delegated to a small set of compute nodes, collectively termed asI/O Delegate nodes, which alleviates the lock contention at I/o servers.
Ceph: reliable, scalable, and high-performance distributed storage
Scott A. Brandt,Sage A. Weil +1 more
TL;DR: This dissertation shows that device intelligence can be leveraged to provide reliable, scalable, and high-performance file service in a dynamic cluster environment and presents a distributed metadata management architecture that provides excellent performance and scalability by adapting to highly variable system workloads while tolerating arbitrary node crashes.
Journal ArticleDOI
Trends in Data Locality Abstractions for HPC Systems
Didem Unat,Anshu Dubey,Torsten Hoefler,John Shalf,Mark Abraham,Mauro Bianco,Bradford L. Chamberlain,Romain E. Cledat,H. Carter Edwards,Hal Finkel,Karl Fuerlinger,Frank Hannig,Emmanuel Jeannot,Amir Kamil,Jeff Keasler,Paul H. J. Kelly,Vitus J. Leung,Hatem Ltaief,Naoya Maruyama,Chris J. Newburn,Miquel Pericas +20 more
TL;DR: This paper examines trends and identifies commonalities that can combine various locality concepts to develop a comprehensive approach to expressing and managing data locality on future large-scale high-performance computing systems.
Patent
Systems and methods for replicating data
TL;DR: In this article, a system facilitates the distribution and redistribution of chunks of data among multiple servers by identifying servers to store a replica of the data based on at least one of utilization of the servers and prior data distribution involving the servers, and failure correlation properties associated with the servers.
References
More filters
Book ChapterDOI
Notes on Data Base Operating Systems
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI
Petal: distributed virtual disks
TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI
Extendible hashing—a fast access method for dynamic files
TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI
Frangipani: a scalable distributed file system
TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article
Scalability in the XFS file system
TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.