Test case prioritization: a family of empirical studies
TLDR
In this article, the authors empirically compared the effectiveness of fine granularity and coarse granularity prioritization techniques using both controlled experiments and case studies, and found that the incorporation of measures of fault proneness into prioritization technique improves their effectiveness.Abstract:
To reduce the cost of regression testing, software testers may prioritize their test cases so that those which are more important, by some measure, are run earlier in the regression testing process. One potential goal of such prioritization is to increase a test suite's rate of fault detection. Previous work reported results of studies that showed that prioritization techniques can significantly improve rate of fault detection. Those studies, however, raised several additional questions: 1) Can prioritization techniques be effective when targeted at specific modified versions; 2) what trade-offs exist between fine granularity and coarse granularity prioritization techniques; 3) can the incorporation of measures of fault proneness into prioritization techniques improve their effectiveness? To address these questions, we have performed several new studies in which we empirically compared prioritization techniques using both controlled experiments and case studies.read more
Citations
More filters
Journal ArticleDOI
Regression testing minimization, selection and prioritization: a survey
Shin Yoo,Mark Harman +1 more
TL;DR: This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
Journal ArticleDOI
Search Algorithms for Regression Test Case Prioritization
TL;DR: The paper addresses the problems of choice of fitness metric, characterization of landscape modality, and determination of the most suitable search technique to apply, and sheds light on the nature of the regression testing search space, indicating that it is multimodal.
Proceedings ArticleDOI
Are mutants a valid substitute for real faults in software testing
TL;DR: This paper investigates whether mutants are indeed a valid substitute for real faults, i.e., whether a test suite’s ability to detect mutants is correlated with its able to detect real faults that developers have fixed, and shows a statistically significant correlation between mutant detection and real fault detection, independently of code coverage.
Proceedings ArticleDOI
TimeAware test suite prioritization
TL;DR: Experimental results indicate that the prioritization approach frequently yields higher average percentage of faults detected (APFD) values, for two case study applications, when basic block level coverage is used instead of method level coverage.
Proceedings ArticleDOI
Pareto efficient multi-objective test case selection
Shin Yoo,Mark Harman +1 more
TL;DR: The concept of Pareto efficiency to test case selection takes multiple objectives such as code coverage, past fault-detection history and execution cost, and constructs a group of non-dominating, equivalently optimal test case subsets.
References
More filters
Book
Experimental Design: Procedures for the Behavioral Sciences
TL;DR: This chapter discusses research strategies and the Control of Nuisance Variables, as well as randomly Randomized Factorial Design with Three or More Treatments and Randomized Block Factorial design, and Confounded Factorial Designs: Designs with Group-Interaction Confounding.
Book
Software Metrics: A Rigorous and Practical Approach
TL;DR: The Second Edition of Software Metrics provides an up-to-date, coherent, and rigorous framework for controlling, managing, and predicting software development processes.
Journal ArticleDOI
Hints on Test Data Selection: Help for the Practicing Programmer
TL;DR: In many cases tests of a program that uncover simple errors are also effective in uncovering much more complex errors, so-called coupling effect can be used to save work during the testing process.
Journal ArticleDOI
Prioritizing test cases for regression testing
TL;DR: Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal as discussed by the authors, such as rate of fault detection, a measure of how quickly faults are detected within the testing process.