Journal ArticleDOI
Increasing Validity of Simulation Models Through Metamorphic Testing
Megan Olsen,Mohammad Raunak +1 more
Reads0
Chats0
TLDR
This work proposes taking the metamorphic testing (MT) approach from the realm of software verification and applying it for validating executable simulation models, and demonstrates the successful application of MT toward validating three different simulation models: an ABM of gossip propagation, anABM of cancer, and a DES of airport check-in and security.Abstract:
Simulation validation, demonstrating that a simulation accurately represents the system it studies, poses a particularly potent form of the oracle problem. A simulation must be validated for its results to be reliable, but often no oracle exists due to the nature of simulations. We propose taking the metamorphic testing (MT) approach from the realm of software verification and applying it for validating executable simulation models. By establishing pseudo-oracles based on metamorphic relations between parameters and behaviors within an executable model, we can create a methodical approach for validating simulation models. We propose an overall framework and guidelines to apply MT for simulation validation, with details for two prevalent simulation approaches: agent-based simulation models (ABM), and discrete-event simulation (DES) modeling. Through three case studies, we demonstrate the successful application of MT toward validating three different simulation models: an ABM of gossip propagation, an ABM of cancer, and a DES of airport check-in and security.read more
Citations
More filters
Journal ArticleDOI
METTLE: A METamorphic Testing Approach to Assessing and Validating Unsupervised Machine Learning Systems
TL;DR: In this paper, the authors developed a MET amorphic approach to assess and validate unsupervised machine learning systems, abbreviated as mettle, by explicitly considering the specific expectations and requirements of these systems from individual users' perspectives.
Journal ArticleDOI
TEA- Cloud : A Formal Framework for Testing Cloud Computing Systems
TL;DR: The aim of the framework is to provide a complete methodology to help users to model both software and hardware parts of cloud systems and automatically test the validity of these clouds using a cost-effective approach.
Journal ArticleDOI
METTLE: a METamorphic testing approach to assessing and validating unsupervised machine LEarning systems
TL;DR: In this article, the authors developed a metamorphic approach to assess and validate unsupervised machine learning systems, abbreviated as METTLE, by explicitly considering the specific expectations and requirements of these systems from individual users' perspectives.
Journal ArticleDOI
Metamorphic testing of OpenStreetMap
TL;DR: This work provides a general metamorphic testing framework that will support the analysis of the information provided in OpenStreetMap with the goal of detecting faulty information and shows the usefulness of this framework to identify potential issues in the information appearing in Open StreetMap.
References
More filters
Journal ArticleDOI
Tutorial on agent-based modelling and simulation
TL;DR: A brief introduction to ABMS is provided, the main concepts and foundations are illustrated, some recent applications across a variety of disciplines are discussed, and methods and toolkits for developing agent models are identified.
Book
Introduction to Software Testing
Paul Ammann,Jeff Offutt +1 more
TL;DR: In this paper, the authors define testing as the process of applying a few well-defined, general-purpose test criteria to a structure or model of the software, and present an innovative approach to explaining the process.
MonographDOI
Introduction to Software Testing: List of Figures
Paul Ammann,Jeff Offutt +1 more
TL;DR: The structure of the text directly reflects the pedagogical approach and incorporates the latest innovations in testing, including techniques to test modern types of software such as OO, web applications and embedded software.
Proceedings ArticleDOI
Software Testing Research: Achievements, Challenges, Dreams
TL;DR: A consistent roadmap of the most relevant challenges to be addressed in software testing research is proposed, constituted by some important past achievements, while the destination consists of four identified goals to which research ultimately tends, but which remain as unreachable as dreams.
Journal ArticleDOI
The Oracle Problem in Software Testing: A Survey
TL;DR: This paper provides a comprehensive survey of current approaches to the test oracle problem and an analysis of trends in this important area of software testing research and practice.