Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact
Reads0
Chats0
TLDR
The infrastructure that is being designed and constructed to support controlled experimentation with testing and regression testing techniques is described and the impact that this infrastructure has had and can be expected to have.Abstract:
Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.read more
Citations
More filters
Journal ArticleDOI
Trace anomalies as precursors of field failures: an empirical study
TL;DR: This work empirically investigates the effectiveness of various simple anomaly detection schemes to identify the conditions that precede failures in deployed software, and exposes the tradeoffs between different anomaly detection algorithms applied to several types of observable attributes under varying levels of in-house testing.
Journal ArticleDOI
Analyzing the Effects of Bugs on Software Interfaces
TL;DR: The results point out that the traditional error models, as used so far, do not accurately emulate software faults, but that richer interface errors need to be injected, by: injecting both fail-stop behaviors and data corruptions; targeting larger amounts of corrupted data structures.
Proceedings ArticleDOI
Finding Causes of Software Failure Using Ridge Regression and Association Rule Generation Methods
TL;DR: Recursion ridge regression method is applied to detect those predicates which are most often observed with bug predictors in faulty executions, and empirical results imply that the proposed approach could detect main causes of program failure with more accuracy.
Journal ArticleDOI
Generating Test Cases for Programs that Are Coded against Interfaces and Annotations
Mainul Islam,Christoph Csallner +1 more
TL;DR: The technique consists of collecting constraints on interfaces, annotations, and reflection, combining them with program constraints collected during dynamic symbolic execution, encoding them in a constraint system, solving them with an off-the-shelf constraint solver, and mapping constraint solutions to test cases and custom mock classes.
Journal ArticleDOI
KDFC-ART: a KD-tree approach to enhancing Fixed-size-Candidate-set Adaptive Random Testing
TL;DR: Three enhanced algorithms for fixed-size-candidate-set ART (FSCS-ART) are proposed based on the $k$-dimensional tree (KD-tree) structure, with results showing that these algorithms significantly reduce the computation time of the original F SCSS-ART for low dimensions and for the case of high dimensions with low failure rates.
References
More filters
Book
Case Study Research: Design and Methods
TL;DR: In this article, buku ini mencakup lebih dari 50 studi kasus, memberikan perhatian untuk analisis kuantitatif, membahas lebah lengkap penggunaan desain metode campuran penelitian, and termasuk wawasan metodologi baru.
Book Chapter
Case study research
TL;DR: The comprehensive and accessible nature of this collection will make it an essential and lasting handbook for researchers and students studying organizations.
Book
Research methods knowledge base
TL;DR: The Research Methods Knowledge Base is a comprehensive web-based textbook that addresses all of the topics in a typical introductory undergraduate or graduate course in social research methods and uses an informal, conversational style to engage both the newcomer and the more experienced student of research.
Book
Experimentation in Software Engineering: An Introduction
TL;DR: The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering, and to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering.
Related Papers (5)
Empirical evaluation of the tarantula automatic fault-localization technique
James A. Jones,Mary Jean Harrold +1 more