Managing rapidly-evolving scientific workflows
Juliana Freire,Cláudio T. Silva,Steven P. Callahan,Emanuele Santos,Carlos Scheidegger,Huy T. Vo +5 more
- pp 10-18
TLDR
An overview of VisTrails, a system that provides an infrastructure for systematically capturing detailed provenance and streamlining the data exploration process, which simplifies data exploration by allowing scientists to easily navigate through the space of workflows and parameter settings for an exploration task.Abstract:
We give an overview of VisTrails, a system that provides an infrastructure for systematically capturing detailed provenance and streamlining the data exploration process. A key feature that sets VisTrails apart from previous visualization and scientific workflow systems is a novel action-based mechanism that uniformly captures provenance for data products and workflows used to generate these products. This mechanism not only ensures reproducibility of results, but it also simplifies data exploration by allowing scientists to easily navigate through the space of workflows and parameter settings for an exploration task.read more
Citations
More filters
Proceedings ArticleDOI
Provenance and scientific workflows: challenges and opportunities
Susan B. Davidson,Juliana Freire +1 more
TL;DR: This tutorial provides an overview of research issues in provenance for scientific workflows, with a focus on recent literature and technology in this area, aimed at a general database research audience and at people who work with scientific data and workflows.
Journal ArticleDOI
Provenance for Computational Tasks: A Survey
TL;DR: The authors give an overview of important concepts related to provenance management, so that potential users can make informed decisions when selecting or designing a provenance solution.
Journal ArticleDOI
The ALPS project release 2.0: open source software for strongly correlated systems
Bela Bauer,Lincoln D. Carr,Hans Gerd Evertz,Adrian E. Feiguin,Juliana Freire,Sebastian Fuchs,Lukas Gamper,Jan Gukelberger,Emanuel Gull,S. Guertler,Andreas Hehn,R. Igarashi,Sergei V. Isakov,David Koop,Ping Ma,Phillip Mates,Phillip Mates,H. Matsuo,Olivier Parcollet,G. Pawłowski,J. D. Picon,Lode Pollet,Lode Pollet,Emanuele Santos,Vito Scarola,Ulrich Schollwöck,Cláudio T. Silva,Brigitte Surer,Synge Todo,Simon Trebst,Matthias Troyer,Michael L. Wall,Philipp Werner,Stefan Wessel,Stefan Wessel +34 more
TL;DR: The ALPS libraries provide a powerful framework for programmers to develop their own applications, which, for instance, greatly simplify the steps of porting a serial code onto a parallel, distributed memory machine.
Book ChapterDOI
Provenance collection support in the kepler scientific workflow system
TL;DR: A complete framework for data and process provenance in the Kepler Scientific Workflow System is described and how generic provenance capture can be facilitated in Kepler's actor-oriented workflow environment is introduced.
BookDOI
Reproducibility and Replicability in Science
TL;DR: The National Academies of Sciences, Engineering, and Medicine conducted a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research as mentioned in this paper.
References
More filters
Journal ArticleDOI
Scientific Workflow Management and the Kepler System
Bertram Ludäscher,Bertram Ludäscher,Ilkay Altintas,Chad Berkley,Dan Higgins,Efrat Jaeger,Matthew B. Jones,Edward A. Lee,Jing Tao,Yang Zhao +9 more
TL;DR: Kepler as mentioned in this paper is a scientific workflow system, which is currently under development across a number of scientific data management projects and is a community-driven, open source project, and always welcome related projects and new contributors to join.
Journal ArticleDOI
A survey of data provenance in e-science
TL;DR: The main aspect of the taxonomy categorizes provenance systems based on why they record provenance, what they describe, how they represent and storeprovenance, and ways to disseminate it.
Proceedings ArticleDOI
Chimera: a virtual data system for representing, querying, and automating data derivation
TL;DR: The Chimera virtual data system is developed, which combines avirtual data catalog for representing data derivation procedures and derived data, with a virtual data language interpreter that translates user requests into data definition and query operations on the database.
Proceedings ArticleDOI
VisTrails: enabling interactive multiple-view visualizations
Louis Bavoil,Steven P. Callahan,Patricia Crossno,Juliana Freire,Carlos Scheidegger,Cláudio T. Silva,Huy T. Vo +6 more
TL;DR: The design and implementation of VisTrails are described, the effectiveness of the system is shown, and its effectiveness in different application scenarios is shown.
Proceedings ArticleDOI
SCIRun: A Scientific Programming Environment for Computational Steering
TL;DR: This paper presents the design, implementation and application of SCIRun, a scientific programming environment that allows the interactive construction, debugging and steering of large scale scientific computations, and identifies ways to avoid the excessive memory use inherent in standard dataflow implementations.