Estimating software testing complexity
TLDR
The definition of a new measure of the difficulty for a computer to generate test cases, called Branch Coverage Expectation (BCE), which is much more correlated with the code coverage than the existing complexity measures.Abstract:
Context: Complexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase. Objective: The aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite. Method: The definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs. Results: The results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures. Conclusion: The high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.read more
Citations
More filters
Book ChapterDOI
Probability, Random Processes, and Statistical Analysis: Moment-generating function and characteristic function
Proceedings ArticleDOI
How high will it be? Using machine learning models to predict branch coverage in automated testing
TL;DR: The first steps towards the definition of machine learning models to predict the branch coverage achieved by test data generation tools are taken, considering well known code metrics as a features.
Journal ArticleDOI
Branch coverage prediction in automated testing
TL;DR: It is argued that knowing a priori the branch coverage that can be achieved with test‐data generation tools can help developers into taking informed decision about issues and it is investigated the possibility to use source‐code metrics to predict the coverage achieved by test‐ data generation tools.
Journal ArticleDOI
Investigating software testing and maintenance reports: Case study
Paweł Janczarek,Janusz Sosnowski +1 more
TL;DR: Fine-grained problem handling reports provide a wider scope of possible measures to assess the relevant development processes and provide significant enhancement in evaluation and refinement of software development processes as well as in reliability prediction.
Proceedings ArticleDOI
An adaptive fitness function based on branch hardness for search based testing
Xiong Xu,Ziming Zhu,Li Jiao +2 more
TL;DR: This paper proposes an adaptive fitness function based on branch hardness, evaluated by the variation coefficient of the fitness function, of generating test data can be minimized and is more flexible than the traditional counterparts.
References
More filters
Journal ArticleDOI
On the ability of complexity metrics to predict fault-prone classes in object-oriented systems
TL;DR: The experimental results reveal that many metrics exhibit moderate or almost moderate ability in discriminating between fault-prone and not fault- prone classes, and LOC and WMC are indeed better fault-proneness predictors than SDMC and AMC; and the explanatory power of other complexity metrics in addition to LOC is limited.
Proceedings ArticleDOI
Third time charm: Stronger prediction of programmer performance by software complexity metrics
TL;DR: In this paper, the authors investigated characteristics of software which are related to its psychological complexity and found that Halstead's E was the best predictor of performance, followed by McCabe's v (G) and the number of lines of code.
Journal ArticleDOI
Evolutionary repair of faulty software
TL;DR: This paper proposes to model this task of fixing faults as a search problem, and hence to use for example evolutionary algorithms to solve it, and gives directions for future work.
Book
Probability, Random Processes, and Statistical Analysis: Applications to Communications, Signal Processing, Queueing Theory and Mathematical Finance
TL;DR: In this article, the authors present a broad range of advanced topics and applications, including Bayesian vs. frequentist statistics, time series and spectral representation, inequalities, bound and approximation, maximum-likelihood estimation and the expectation-maximization (EM) algorithm, geometric Brownian motion and Ito process.
Journal ArticleDOI
Evolutionary algorithms for the multi-objective test data generation problem
TL;DR: The results indicate that the oracle cost can be properly optimized; however, the full branch coverage of the system poses a great challenge, so the direct multi‐objective approach is mainly compared.