Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact
Reads0
Chats0
TLDR
The infrastructure that is being designed and constructed to support controlled experimentation with testing and regression testing techniques is described and the impact that this infrastructure has had and can be expected to have.Abstract:
Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.read more
Citations
More filters
Journal ArticleDOI
Regression testing minimization, selection and prioritization: a survey
Shin Yoo,Mark Harman +1 more
TL;DR: This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
Proceedings ArticleDOI
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
TL;DR: Defects4J, a database and extensible framework providing real bugs to enable reproducible studies in software testing research, and provides a high-level interface to common tasks in softwareTesting research, making it easy to con- duct and reproduce empirical studies.
Proceedings ArticleDOI
Software Testing Research: Achievements, Challenges, Dreams
TL;DR: A consistent roadmap of the most relevant challenges to be addressed in software testing research is proposed, constituted by some important past achievements, while the destination consists of four identified goals to which research ultimately tends, but which remain as unreachable as dreams.
Journal ArticleDOI
Search Algorithms for Regression Test Case Prioritization
TL;DR: The paper addresses the problems of choice of fitness metric, characterization of landscape modality, and determination of the most suitable search technique to apply, and sheds light on the nature of the regression testing search space, indicating that it is multimodal.
Proceedings ArticleDOI
SemFix: program repair via semantic analysis
TL;DR: This paper presents an automated repair method based on symbolic execution, constraint solving and program synthesis, which reports a higher success-rate than genetic programming based repair, and produces a repair faster.
References
More filters
Journal ArticleDOI
Regression testing in an industrial environment
TL;DR: A multilevel regression testing framework that can be easily adapted into asoftware development and maintenance process in which regression testing can play a key role in improving and ensuring consistent software quality is proposed.
Journal ArticleDOI
Interface Mutation: an approach for integration testing
TL;DR: Comparing the error revealing effectiveness of these Interface Mutation-based test sets with same size randomly generated test sets, it is observed that in most cases Interface Mutations based test sets are superior.
Journal ArticleDOI
Semantics guided regression test cost reduction
TL;DR: Two complimentary algorithms for reducing the cost of regression testing are presented, one of which produces a program called Differences that captures the semantic change between Certified, a previously tested program, and Modified, a changed version of Certified.
Book
Software Test Automation: Effective Use of Test Execution Tools
Mark Fewster,Dorothy Graham +1 more
TL;DR: This book is published as part of ACM Press Books a collaboration between the Association for Computing (ACM) and Addison Wesley Longman Limited.
Proceedings ArticleDOI
Improving test suites via operational abstraction
TL;DR: The operational difference technique for generating, augmenting, and minimizing test suites is presented, which is analogous to structural code coverage techniques, but it operates in the semantic domain of program properties rather than the syntacticdomain of program text.
Related Papers (5)
Empirical evaluation of the tarantula automatic fault-localization technique
James A. Jones,Mary Jean Harrold +1 more