Open AccessProceedings Article
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank B. Schmuck,Roger L. Haskin +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.Citations
More filters
Proceedings ArticleDOI
A failure recovery mechanism for distributed metadata servers in DCFS2
Zhihua Fan,Jin Xiong,Jie Ma +2 more
TL;DR: A journal-based failure-recovery mechanism for distributed metadata servers in the dawning cluster file system-DCFS2, which exploits a modified two-phase commit protocol which ensures consistent metadata updates on multiple metadata servers even in case of one server's failure.
Proceedings ArticleDOI
PageRank Pipeline Benchmark: Proposal for a Holistic System Benchmark for Big-Data Platforms
TL;DR: The PageRank pipeline benchmark builds on existing prior scalable benchmarks (Graph500, Sort, and PageRank) to create a holistic benchmark with multiple integrated kernels that can be run together or independently.
Topology-aware data aggregation for intensive I/O on large-scale supercomputers
TL;DR: This work takes into account the network topology for mapping aggregators and proposes an optimized buffering system in order to reduce the aggregation cost and shows improvements up to 15× faster for I/O operations compared to a standard implementation of MPI I/ O.
Book ChapterDOI
Towards a High Performance Implementation of MPI-IO on the Lustre File System
Phillip M. Dickens,Jeremy Logan +1 more
TL;DR: Experimental results are provided showing that the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO do not apply in Lustre, and new approaches that appear to provide significantly better performance are explored.
Proceedings ArticleDOI
Harnessing grid resources to enable the dynamic analysis of large astronomy datasets
TL;DR: A Web Services-based system, AstroPortal, that uses grid computing to federate large computing and storage resources for dynamic analysis of large datasets and a first analysis, "stacking," that sums multiple regions of the sky, a function that can help both identify variable sources and detect faint objects.
References
More filters
Book ChapterDOI
Notes on Data Base Operating Systems
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI
Petal: distributed virtual disks
TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI
Extendible hashing—a fast access method for dynamic files
TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI
Frangipani: a scalable distributed file system
TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article
Scalability in the XFS file system
TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.