Journal ArticleDOI
Coverage-based regression test case selection, minimization and prioritization: a case study on an industrial system
Reads0
Chats0
TLDR
This paper presents a case study of coverage‐based regression testing techniques on a real world industrial system with real regression faults, and shows that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates.Abstract:
This paper presents a case study of coverage-based regression testing techniques on a real world industrial system with real regression faults. The study evaluates four common prioritization techniques, a test selection technique, a test suite minimization technique and a hybrid approach that combines selection and minimization. The study also examines the effects of using various coverage criteria on the effectiveness of the studied approaches. The results show that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information in prioritization techniques does not significantly enhance fault detection rates. The results show that test selection does not provide significant savings in execution cost <2%, which might be attributed to the nature of the changes made to the system. Test suite minimization using finer grained coverage criteria could provide significant savings in execution cost 79.5% while maintaining a fault detection capability level above 70%, thus representing a possible trade-off. The hybrid technique did not provide a significant improvement over traditional minimization techniques. Copyright © 2015John Wiley & Sons, Ltd.read more
Citations
More filters
Journal ArticleDOI
Test case prioritization approaches in regression testing: A systematic literature review
TL;DR: This review examines and classify the current test case prioritization approaches in TCP based on the articulated research questions and found that variations in the starting point of TCP process among the approaches provide a different timeline and benefit to project manager to choose which approaches suite with the project schedule and available resources.
Proceedings ArticleDOI
Reinforcement learning for automatic test case prioritization and selection in continuous integration
TL;DR: In this article, the Retecs method uses reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure history, in a constantly changing environment.
Proceedings ArticleDOI
Hybrid regression test selection
TL;DR: The experimental results on 2707 revisions of 32 projects, totalling over 124 Million LoC, demonstrate that HyRTS outperforms state-of-the-art FRTS significantly in terms of selected test ratio and the offline testing time.
Journal ArticleDOI
Effective Regression Test Case Selection: A Systematic Literature Review
TL;DR: This systematic literature review presents state-of-the-art research in effective regression test case selection techniques and observed that 70% of the studies being analyzed used cost as the effectiveness measure compared to 31% that use fault-detection capability and 16% that used coverage.
Proceedings ArticleDOI
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
TL;DR: This paper introduces Retecs, a new method for automatically learning test case selection and prioritization in CI with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases.
References
More filters
Book
Experimentation in Software Engineering: An Introduction
TL;DR: The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering, and to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering.
Journal ArticleDOI
Regression testing minimization, selection and prioritization: a survey
Shin Yoo,Mark Harman +1 more
TL;DR: This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
Journal ArticleDOI
Prioritizing test cases for regression testing
TL;DR: Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal as discussed by the authors, such as rate of fault detection, a measure of how quickly faults are detected within the testing process.
Proceedings ArticleDOI
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
TL;DR: An experimental study investigating the effectiveness of two code-based test adequacy criteria for identifying sets of test cases that detect faults found that tests based respectively on control-flow and dataflow criteria are frequency complementary in their effectiveness.
Journal ArticleDOI
Test case prioritization: a family of empirical studies
TL;DR: In this article, the authors empirically compared the effectiveness of fine granularity and coarse granularity prioritization techniques using both controlled experiments and case studies, and found that the incorporation of measures of fault proneness into prioritization technique improves their effectiveness.