Grid-based metadata services
Ewa Deelman,Gurmeet Singh,Malcolm Atkinson,Ann L. Chervenak,N. P. Chue Hong,Carl Kesselman,Sonal Patil,Laura Pearlman,Mei-Hui Su +8 more
- pp 393-402
TLDR
This paper presents a data model that can capture the complexity of the data publication and discovery process through the use of descriptive metadata, and identifies a set of interfaces and operations that need to be provided to support metadata management.Abstract:
Data sets being managed in grid environments today are growing at a rapid rate, expected to reach 100s of petabytes in the near future. Managing such large data sets poses challenges for efficient data access, data publication and data discovery. In this paper we focus on the data publication and discovery process through the use of descriptive metadata. This metadata describe the properties of individual data items and collections. We discuss issues of metadata services in service rich environments, such as the grid. We describe the requirements and the architecture for such services in the context of grid and the available grid services. We present a data model that can capture the complexity of the data publication and discovery process. Based on that model we identify a set of interfaces and operations that need to be provided to support metadata management. We present a particular implementation of a grid metadata service, basing it on existing grid services technologies. Finally we examine alternative implementations of that service.read more
Citations
More filters
Journal ArticleDOI
Pegasus: A framework for mapping complex scientific workflows onto distributed systems
Ewa Deelman,Gurmeet Singh,Mei-Hui Su,Jim Blythe,Yolanda Gil,Carl Kesselman,Gaurang Mehta,Karan Vahi,G. Bruce Berriman,John C. Good,Anastasia C. Laity,Joseph C. Jacob,Daniel S. Katz +12 more
TL;DR: The results of improving application performance through workflow restructuring which clusters multiple tasks in a workflow into single entities are presented.
Proceedings Article
Provenance-aware storage systems
TL;DR: It is shown that with reasonable overhead, a Provenance-Aware Storage System can provide useful functionality not available in today's file systems or provenance management systems.
Proceedings ArticleDOI
Data Management Challenges of Data-Intensive Scientific Workflows
Ewa Deelman,Ann L. Chervenak +1 more
TL;DR: This paper examines some of the issues in the area of data management related to workflow creation, execution, and result management in the context of the entire workflow lifecycle.
Proceedings ArticleDOI
Introducing secure provenance: problems and challenges
TL;DR: The secure provenance problem is defined and it is argued that it is of vital importance in numerous applications and the issues related to ensuring the privacy and integrity of provenance information are discussed.
Journal ArticleDOI
A subscribable peer-to-peer RDF repository for distributed metadata management
TL;DR: A scalable peer-to-peer RDF repository, named RDFPeers, which stores each triple in a multi-attribute addressable network by applying globally known hash functions, and enables users to selectively subscribe to RDF content.
References
More filters
Journal ArticleDOI
The Anatomy of the Grid: Enabling Scalable Virtual Organizations
TL;DR: The authors present an extensible and open Grid architecture, in which protocols, services, application programming interfaces, and software development kits are categorized according to their roles in enabling resource sharing.
The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration
TL;DR: This presentation complements an earlier foundational article, “The Anatomy of the Grid,” by describing how Grid mechanisms can implement a service-oriented architecture, explaining how Grid functionality can be incorporated into a Web services framework, and illustrating how the architecture can be applied within commercial computing as a basis for distributed system integration.
Proceedings Article
A resource management architecture for metacomputing systems.
Krzysztof Czajkowski,Ian Foster,Nicholas T. Karonis,Carl Kesselman,Stuart Martin,Warren Smith,Steven Tuecke +6 more
TL;DR: This work describes a resource management architecture that distributes the resource management problem among distinct local manager, resource broker, and resource co-allocator components and defines an extensible resource specification language to exchange information about requirements.
Book ChapterDOI
A Resource Management Architecture for Metacomputing Systems
Krzysztof Czajkowski,Ian Foster,Nicholas T. Karonis,Carl Kesselman,Stuart Martin,Warren Smith,Steven Tuecke +6 more
TL;DR: The Globus metacomputing toolkit as discussed by the authors describes a resource management architecture that distributes the resource management problem among distinct local manager, resource broker, and resource co-allocator components.
Journal ArticleDOI
Data management and transfer in high-performance computational grid environments
Bill Allcock,Joe Bester,John Bresnahan,Ann L. Chervenak,Ian Foster,Carl Kesselman,Sam Meder,Veronika Nefedova,Darcy Quesnel,Steven Tuecke +9 more
TL;DR: A high-speed transport service that extends the popular FTP protocol with new features required for Data Grid applications, such as striping and partial file access and a replica management service that integrates a replica catalog with GridFTP transfers to provide for the creation, registration, location, and management of dataset replicas.