Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact
TLDR
The infrastructure that is being designed and constructed to support controlled experimentation with testing and regression testing techniques is described and the impact that this infrastructure has had and can be expected to have.Abstract:
Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.read more
Citations
More filters
Proceedings ArticleDOI
Test Benchmarks -- what is the question?
TL;DR: In the automotive press, there are lots of benchmarks on crash resistance and the power and the torque of the engine benchmarked, but rarely it is noticed whether the power is delivered at revs which are useful in daily driving or at top revs.
Proceedings ArticleDOI
Modeling the Diagnostic Efficiency of Regression Test Suites
TL;DR: An analytical model and a simulation model are presented to predict the diagnostic efficiency of test suites when prioritized with the information gain algorithm and show that, besides the size of the system itself, an optimal coverage density and uniform coverage distribution are needed to achieve an efficient diagnosis.
Proceedings ArticleDOI
Low-overhead and fully automated statistical debugging with abstraction refinement
TL;DR: This paper forms the technique as an instance of abstraction refinement, where efficient abstract-level profiling is first applied to the whole program and its execution brings information that can pinpoint suspicious coarse-grained entities that need to be refined.
Proceedings ArticleDOI
An Empirical Study of Bug Isolation on the Effectiveness of Multiple Fault Localization
Zheng Li,Yonghao Wu,Yong Liu +2 more
TL;DR: A large quantitative empirical study is conducted on 12786 version programs with multiple faults, in which the misgrouping are simulated with different accuracy by a controlled experiment, and results indicate that the effect of fault localization will become worse as the accuracy of clustering decreases.
Journal ArticleDOI
Optimal prediction of synchronization-preserving races
TL;DR: In this article, the authors introduce the notion of sync-hronization-preserving races, where synchronization operations (e.g., acquisition and release of locks) appear in the same order as in the trace.
References
More filters
Book
Case Study Research: Design and Methods
TL;DR: In this article, buku ini mencakup lebih dari 50 studi kasus, memberikan perhatian untuk analisis kuantitatif, membahas lebah lengkap penggunaan desain metode campuran penelitian, and termasuk wawasan metodologi baru.
Book Chapter
Case study research
TL;DR: The comprehensive and accessible nature of this collection will make it an essential and lasting handbook for researchers and students studying organizations.
Book
Research methods knowledge base
TL;DR: The Research Methods Knowledge Base is a comprehensive web-based textbook that addresses all of the topics in a typical introductory undergraduate or graduate course in social research methods and uses an informal, conversational style to engage both the newcomer and the more experienced student of research.
Book
Experimentation in Software Engineering: An Introduction
TL;DR: The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering, and to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering.
Related Papers (5)
Empirical evaluation of the tarantula automatic fault-localization technique
James A. Jones,Mary Jean Harrold +1 more