Scripts are widely used to design and run scientific experiments. Scripting languages are easy to learn and use, and they allow complex tasks to be specified and executed in fewer steps than with traditional programming languages. However, they also have important limitations for reproducibility and data management. As experiments are iteratively refined, it is challenging to reason about each experiment run (or trial), to keep track of the association between trials and experiment instances as well as the differences across trials, and to connect results to specific input data and parameters. Approaches have been proposed that address these limitations by collecting, managing, and analyzing the provenance of scripts. In this article, we survey the state of the art in provenance for scripts. We have identified the approaches by following an exhaustive protocol of forward and backward literature snowballing. Based on a detailed study, we propose a taxonomy and classify the approaches using this taxonomy.

https://dl.acm.org/doi/pdf/10.1145/3311955

A Survey on Collecting, Managing, and Analyzing Provenance from Scripts

Deterministic record and replay systems have widely been used in software debugging, failure diagnosis, and intrusion detection. In order to detect the Advanced Persistent Threat (APT), online execution needs to be recorded with acceptable runtime overhead; then, investigators can analyze the replayed execution with heavy dynamic instrumentation. While most record and replay systems rely on kernel module or OS virtualization, those running at user space are favoured for being lighter weight and more portable without any of the changes needed for OS/Kernel virtualization. On the other hand, higher level provenance data at a higher level provides dynamic analysis with system causalities and hugely increases its efficiency. Considering both benefits, we propose a provenance-aware user space record and replay system, called RecProv. RecProv is designed to provide high provenance fidelity; specifically, with versioning files from the recorded trace logs and integrity protection to provenance data through real-time trace isolation. The collected provenance provides the high-level system dependency that helps pinpoint suspicious activities where further analysis can be applied. We show that RecProv is able to output accurate provenance in both visualized graph and W3C standardized PROV-JSON formats.

Provenance and Annotation of Data and Processes

Until not long ago, manually capturing and storing provenance from scientific experiments were constant concerns for scientists. With the advent of computational experiments (modeled as scientific workflows) and Scientific Workflow Management Systems, produced and consumed data, as well as the provenance of a given experiment, are automatically managed, so provenance capturing and storing in such a context is no longer a major concern. Similarly to several existing big data problems, the bottom line is now on how to analyze the large amounts of provenance data generated by workflow executions and how to be able to extract useful knowledge of this data. In this context, this article surveys the current state of the art on provenance analytics by presenting the key initiatives that have been taken to support provenance data analysis. We also contribute by proposing a taxonomy to classify elements related to provenance analytics.

Provenance Analytics for Workflow-Based Computational Experiments: A Survey

The use of games in software engineering education is not new. However, recent technologies have provided new opportunities for using games and their elements to enhance learning and student engagement. The goal of this paper is twofold. First, we discuss how game related methods have been used in the context of software engineering education by means of a systematic mapping study. Second, we investigate how these game related methods support specific knowledge areas from software engineering. The systematic mapping study identified 106 primary studies describing the use of serious games, gamification and game development in software engineering education. Based on this mapping, we aimed to track the learning goals of each primary study to the knowledge areas defined in ACM/IEEE curricular recommendations. As a result, we observed that "Software Process", "Software Design" and "Profession Practice" are the most recurring knowledge areas explored by game related approaches in software engineering education. We also uncover possible research opportunities for game-related education methods.

Games for Learning: Bridging Game-Related Education Methods to Software Engineering Knowledge Areas

Data provenance is a fundamental concept in scientific experimentation. However, for their proper understanding and use, efficient and user-friendly mechanisms are needed. Research in software visualization, ontologies and complex networks can help in this process. This paper presents a framework to assist in the understanding and use of data provenance using visualization techniques, ontologies and complex networks. The framework capture the provenance data and generates new information using ontologies and provenance graph analysis. The graph is analyzed through complex networks techniques and provide some metrics to help in each node analyzes. The visualization presents and highlights the inferences and results. The framework was used in the E-SECO scientific ecosystem to support the scientific experimentation.

A Framework for Provenance Analysis and Visualization

The analysis of provenance data for an experiment is often crucial to understand the achieved results. For long-running experiments or when provenance is captured at a low granularity, this analysis process can be overwhelming to the user due to the large volume of provenance data. In this paper we introduce, Prov Viewer, a provenance visualization tool that enables users to interactively explore provenance data. Among the visualization and exploratory features, we can cite zooming, filtering, and coloring. Moreover, we use of other properties such as shape and size to distinguish visual elements. These exploratory features are linked to the provenance semantics to ease the comprehension process. We also introduce collapsing and filtering strategies, allowing different levels of granularity exploration and analysis. We describe case studies that show how Prov Viewer has been successfully used to explore provenance in different domains, including games and urban data.

Prov Viewer: A Graph-Based Visualization Tool for Interactive Exploration of Provenance Data

Software Engineering is an area of computer science that focuses on practical aspects of the software production. The Undergraduate courses of Computer Science have disciplines of Software Engineering, but they are usually taught in a theoretic way and with only a few implementation exercises using the learned techniques and tools. A practical approach for the concepts studied during the Software Engineering classes would help the student in understanding the reason for using the presented concepts. Due to that, we introduce Software Development Manager, a novel simulation game where the player owns a software development company that counts with the help of a team, which is administered by the player, to develop products desired by customers. The purpose of this game is to assist in learning the knowledge of Software Engineering in a way that takes advantage of the benefits of fun and entertainment.

/pdf/sdm-an-educational-game-for-software-engineering-3uz6v2e9x0.pdf

SDM - An Educational Game for Software Engineering

Winning or losing a game session is the final consequence of a series of decisions and actions made during the game. The analysis and understanding of events, mistakes, and fluxes of a concrete game play may be useful for different reasons: understanding problems related to gameplay, data mining of specific situations, and even understanding educational and learning aspects in serious games. We introduce a novel approach based on provenance concepts in order to model and represent a game flux. We model the game data and map it to provenance to generate a provenance graph for analysis. As an example, we also instantiated our proposed conceptual framework and graph generation in a serious game, allowing developers and designers to identify possible mistakes and failures in gameplay design by analyzing the generated provenance graph from collected gameplay data.

Game Flux Analysis with Provenance

The usage of provenance data drastically increases the potential for game data mining since it is able to record causes, effects and relationships of events and objects during a game session. However, it commonly requires modifications in the game engine in order to collect such provenance data. The modifications in the game engine may be unviable in commercial (and not open source) systems. In this paper, we propose a novel and non-intrusive approach for collecting provenance data in digital games. Our proposal collects provenance data using image processing mechanisms and pre-defined image patterns, thus avoiding accessing and modifying the source code of the game. Using our approach, we are able to generate, analyze and visualize game design features based on the gameplay flow using provenance data. Furthermore, we evaluated our proposal with a well known commercial 2D game, called "Super Mario World".

A Non-intrusive Approach for 2D Platform Game Design Analysis Based on Provenance Data Extracted from Game Streaming

Software engineering is focused on practical and theoretical aspects of the software production Teaching software engineering is traditionally done through theoretical classes with some practical exercises Recently, games and simulators were introduced as a ludic alternative for software engineering learning, where decisions and interactions become key factors to transmit and acquire knowledge However, mistakes made by wrong decisions may jeopardize the learning process, especially when reproducing its effects is not a viable option due to the non-deterministic nature of games With this in mind, in a previous work we proposed a novel approach based on provenance concepts in order to present the decisions and effects of such decisions when learning through games In this work, we present an experimental evaluation of that approach with undergraduate students The obtained results show that the use of provenance leads to faster and more accurate answers from students, including learning aspects that could not be achieved by a traditional educational game

Troy C. Kohwalter

Papers

Prov Viewer: A Graph-Based Visualization Tool for Interactive Exploration of Provenance Data

SDM - An Educational Game for Software Engineering

Game Flux Analysis with Provenance

A Non-intrusive Approach for 2D Platform Game Design Analysis Based on Provenance Data Extracted from Game Streaming

Reinforcing Software Engineering Learning through Provenance