Incorporating varying test costs and fault severities into test case prioritization
Sebastian Elbaum,Alexey G. Malishevsky,Gregg Rothermel +2 more
- pp 329-338
Reads0
Chats0
TLDR
This study presents a new metric for assessing the rate of fault detection of prioritized test cases that incorporates varying test case and fault costs and presents the results of a case study illustrating the application of the metric.Abstract:
Test case prioritization techniques schedule test cases for regression testing in an order that increases their ability to meet some performance goal. One performance goal, rate of fault detection, measures how quickly faults are detected within the testing process. In previous work (S. Elbaum et al., 2000; G. Rothermel et al., 1999), we provided a metric, APFD, for measuring rate of fault detection, and techniques for prioritizing test cases to improve APFD, and reported the results of experiments using those techniques. This metric and these techniques, however, applied only in cases in which test costs and fault severity are uniform. We present a new metric for assessing the rate of fault detection of prioritized test cases that incorporates varying test case and fault costs. We present the results of a case study illustrating the application of the metric. This study raises several practical questions that might arise in applying test case prioritization; we discuss how practitioners could go about answering these questions.read more
Citations
More filters
Journal ArticleDOI
Regression testing minimization, selection and prioritization: a survey
Shin Yoo,Mark Harman +1 more
TL;DR: This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
Journal ArticleDOI
Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact
TL;DR: The infrastructure that is being designed and constructed to support controlled experimentation with testing and regression testing techniques is described and the impact that this infrastructure has had and can be expected to have.
Journal ArticleDOI
Search Algorithms for Regression Test Case Prioritization
TL;DR: The paper addresses the problems of choice of fitness metric, characterization of landscape modality, and determination of the most suitable search technique to apply, and sheds light on the nature of the regression testing search space, indicating that it is multimodal.
Journal ArticleDOI
Test-suite reduction and prioritization for modified condition/decision coverage
James A. Jones,Mary Jean Harrold +1 more
TL;DR: New algorithms for test-suite reduction and prioritization that can be tailored effectively for use with modified condition/decision coverage (MC/DC) adequate are presented.
Journal ArticleDOI
Effectively prioritizing tests in development environment
TL;DR: Echelon is a test prioritization system, which prioritizes the application's given set of tests, based on what changes have been made to the program, and proposes a practical binary code based approach that scales well to large systems.
References
More filters
Proceedings ArticleDOI
Prioritizing test cases for regression testing
TL;DR: Can prioritization techniques be effective when aimed at specific modified versions; what tradeoffs exist between fine granularity and coarse granularity prioritized techniques; and can the incorporation of measures of fault proneness into prioritization technique improve their effectiveness?
Proceedings ArticleDOI
Test case prioritization: an empirical study
TL;DR: Several techniques for prioritizing test cases are described and the empirical results measuring the effectiveness of these techniques for improving rate of fault detection are reported, providing insights into the tradeoffs among various techniques for test case prioritization.
Book
Software Reliability Engineering
TL;DR: Software reliability engineering (SRE) assesses how well software-based products and services meet users' operational needs and uses quantitative methods based on reliability measures to do this assessment, to maximize customer satisfaction.
Proceedings ArticleDOI
A study of effective regression testing in practice
TL;DR: This proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions to identify a representative subset of all test cases that may result in different output behavior on the new software version.
Proceedings ArticleDOI
Investigating quality factors in object-oriented designs: an industrial case study
TL;DR: Results show that many of the findings are consistent across systems, despite the very disparate nature of the systems under study, and there are also differences across studies which illustrate the fact that quality does not follow universal laws and that quality models must be developed locally, wherever needed.