Open AccessProceedings Article
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank B. Schmuck,Roger L. Haskin +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.Citations
More filters
Proceedings ArticleDOI
SOSS: smart object-based storage system
TL;DR: Some methods adopted in SOSS to achieve intelligence, such as pattern recognition method, predicting object properties method and an adaptive cache replacement policy (all based on object attribute), are studied.
Journal ArticleDOI
A distributed data management system to support large-scale data analysis
TL;DR: A round-random partitioning scheme to represent a big dataset as a set of non-overlapping data blocks; each block is a random sample of the whole dataset.
Proceedings ArticleDOI
/scratch as a cache: rethinking HPC center scratch storage
TL;DR: This paper proposes a new perspective, where the HPC scratch space is treated as a cache, and data population, retention, and eviction tools are integrated with scratch management, and a new job-workflow-aware caching policy that leverages user-supplied hints for managing the cache.
Proceedings ArticleDOI
LLMapReduce: Multi-level map-reduce for high performance data analysis
Chansup Byun,Jeremy Kepner,William Arcand,David Bestor,Bill Bergeron,Vijay Gadepally,Matthew Hubbell,Peter Michaleas,Julie Mullen,Andrew Prout,Antonio Rosa,Charles Yee,Albert Reuther +12 more
TL;DR: LLMapReduce dramatically simplifies map- reduce programming by providing simple parallel programming capability in one line of code, and can overcome scaling limits in the map-reduce parallel programming model via options that allow the user to switch to the more efficient single-program-multiple-data (SPMD) parallel Programming model.
Journal ArticleDOI
Cooperative Client-Side File Caching for MPI Applications
TL;DR: This paper proposes a client-side file caching system for MPI applications that perform parallel I/O operations on shared files and shows a significant improvement over traditional methods that use either byte-range file locking or rely on coherentI/O provided by the file system.
References
More filters
Book ChapterDOI
Notes on Data Base Operating Systems
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI
Petal: distributed virtual disks
TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI
Extendible hashing—a fast access method for dynamic files
TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI
Frangipani: a scalable distributed file system
TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article
Scalability in the XFS file system
TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.