Open AccessProceedings Article
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank B. Schmuck,Roger L. Haskin +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.Citations
More filters
Journal ArticleDOI
Scalable network file systems with load balancing and fault tolerance for web services
TL;DR: This paper uses a variable number of objects within a bucket to decrease internal fragmentation in small files and proposes a free space and access load-balancing mechanism to balance overall loading on the bucket servers, which can effectively improve scalable network file system performance for large web services.
Data Analysis Challenges
D. Meiron,S. Cazares,P. Dimotakis,F. Dyson,D. Eardley,S. Keller-McNulty,D. Long,F. Perkins,W. Press,R. Schwitters,C. Stubbs,J. Tonry,P. Weinberger +12 more
TL;DR: JASON finds that DOD/IC data requirements are certainly significant, but not unmanageable given the capabilities of current and projected storage technology.
Journal ArticleDOI
High-performance metadata indexing and search in petascale data storage systems
TL;DR: It is shown that Spyglass enables important functionality that can aid data management for scientists and storage administrators, and can achieve search performance up to several thousand times faster than existing database solutions.
Journal ArticleDOI
MARIANE: Using MApReduce in HPC environments
TL;DR: The components and trade-offs necessary for this model are identified, and the performance gains exhibited by the approach in HPC environments over Apache Hadoop in a data intensive setting at the National Energy Research Scientific Computing Center (NERSC) are quantified.
Proceedings ArticleDOI
Parallel I/O Performance for Application-Level Checkpointing on the Blue Gene/P System
TL;DR: This study shows that rbIO and coIO result in 100vó improvement over previous checkpointing approaches on up to 65,536 processors of the Blue Gene/P using the GPFS, and demonstrates a 25vó production performance improvement for NekCEM.
References
More filters
Book ChapterDOI
Notes on Data Base Operating Systems
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI
Petal: distributed virtual disks
TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI
Extendible hashing—a fast access method for dynamic files
TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI
Frangipani: a scalable distributed file system
TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article
Scalability in the XFS file system
TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.