scispace - formally typeset
Open AccessProceedings Article

GPFS: A Shared-Disk File System for Large Computing Clusters

Frank B. Schmuck, +1 more
- pp 231-244
Reads0
Chats0
TLDR
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.
Abstract
GPFS is IBM's parallel, shared-disk file system for cluster computers, available on the RS/6000 SP parallel supercomputer and on Linux clusters. GPFS is used on many of the largest supercomputers in the world. GPFS was built on many of the ideas that were developed in the academic community over the last several years, particularly distributed locking and recovery technology. To date it has been a matter of conjecture how well these ideas scale. We have had the opportunity to test those limits in the context of a product that runs on the largest systems in existence. While in many cases existing ideas scaled well, new approaches were necessary in many key areas. This paper describes GPFS, and discusses how distributed locking and recovery techniques were extended to scale to large clusters.

read more

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI

Optimization of Ceph Reads/Writes Based on Multi-threaded Algorithms

Ke Zhan, +1 more
TL;DR: The results show that the performance of multi-threaded algorithm for downloading/uploading small files and large files are improved by 7624, 2827% and 58.95%, 170.36% respectively compared to the method of command line.
Proceedings ArticleDOI

Scaling a global file system to the greatest possible extent, performance, capacity, and number of users

TL;DR: IBM's GPFS file system, with extensions developed by the San Diego Supercomputer Center in collaboration with IBM, is used, both theoretically and by demonstration, in scaling file storage to the very widest possible extents.
BookDOI

Euro-Par 2015: Parallel Processing

TL;DR: This paper presents and illustrates two important notions associated with concurrent objects, the first of which is the notion of a hybrid implementation and the second one, which is related to their definition, is the idea of an abortable object.
Proceedings ArticleDOI

Toward a unified object storage foundation for scalable storage systems

TL;DR: This paper investigates a number of popular data models used in cloud storage, big data, and high-performance computing (HPC) storage and describes the unique features that distinguish them and proposes a proposed data model.
Proceedings ArticleDOI

StoRM: A flexible solution for Storage Resource Manager in grid

TL;DR: StoRM service, an implementation of the Storage Resource Manager Interface version 2.2, is described, designed to foster the adoption of cluster file systems, and thanks to the marked flexibility, StoRM can be used in small data centre with human resource deficiency to administer an other grid service and, at the same time, capable to grow in terms of storage managed and workload.
References
More filters
Book ChapterDOI

Notes on Data Base Operating Systems

Jim Gray
TL;DR: This paper is a compendium of data base management operating systems folklore and focuses on particular issues unique to the transaction management component especially locking and recovery.
Proceedings ArticleDOI

Petal: distributed virtual disks

TL;DR: The design, implementation, and performance of Petal is described, a system that attempts to approximate this ideal in practice through a novel combination of features.
Journal ArticleDOI

Extendible hashing—a fast access method for dynamic files

TL;DR: This work studies, by analysis and simulation, the performance of extendible hashing and indicates that it provides an attractive alternative to other access methods, such as balanced trees.
Proceedings ArticleDOI

Frangipani: a scalable distributed file system

TL;DR: Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.
Proceedings Article

Scalability in the XFS file system

TL;DR: The architecture and design of a new file system, XFS, for Silicon Graphics' IRIX operating system is described, and the use of B+ trees in place of many of the more traditional linear file system structures are discussed.
Trending Questions (1)
What is GPAIS?

Not addressed in the paper.