Open AccessProceedings Article
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank B. Schmuck,Roger L. Haskin +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.Citations
More filters
Dissertation
Scalable data-management systems for Big Data
TL;DR: This thesis proposes and implements a versioning-based mechanism that can be leveraged to offer isolation for non-contiguous I/O operations in a scalable fashion without the need to perform expensive synchronizations, and introduces Pyramid, a large-scale, array-oriented storage system.
Characterizing HEC Storage Systems at Rest (CMU-PDL-08-109)
TL;DR: This paper reports on the statistics of supercomputing file systems at rest from a variety of national resource computing sites, contrast these to studies of the 80s and 90s of academic and software development campuses and observes the most interesting characteristics in this novel data.
Proceedings ArticleDOI
Adapting MapReduce for HPC environments
TL;DR: This work proposes a MapReduce implementation and design choices directly suitable for HPC environments, which use high performance distributed file systems.
Journal ArticleDOI
Managing very large distributed datasets on a data grid
TL;DR: The motivation for the work is the ATLAS experiment for the Large Hadron Collider, where the authors have been involved in the development of the data management middleware, called DQ2, which has been used for the last several years for shipping petabytes of data to research centres and universities worldwide.
Journal ArticleDOI
Can We Group Storage? Statistical Techniques to Identify Predictive Groupings in Storage System Accesses
Avani Wildani,Ethan L. Miller +1 more
TL;DR: This work proposes a series of methods to derive groupings from data that have predictive value, informing layout decisions for data on disk and intends for this work to provide a launchpad for future specialized system design using groupings in combination with caching policies and architectural distinctions to create the next generation of scalable storage systems.
References
More filters
Book ChapterDOI
Notes on Data Base Operating Systems
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI
Petal: distributed virtual disks
TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI
Extendible hashing—a fast access method for dynamic files
TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI
Frangipani: a scalable distributed file system
TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article
Scalability in the XFS file system
TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.