Xiong Xu

Bio: Xiong Xu is an academic researcher from Chinese Academy of Sciences. The author has contributed to research in topics: Fitness function & Search algorithm. The author has an hindex of 3, co-authored 7 publications receiving 21 citations.

An adaptive fitness function based on branch hardness for search based testing

Abstract: Search based software testing has received great attention as a means of automating the test data generation, and the goal is to improve various criteria. There are different types of coverage criteria. In this paper, we deal with the path coverage. Concretely, we focus on the path that is the most difficult to cover. One major limitation of search based testing is the inefficient and insufficiently informed fitness function. To address this problem, we propose an adaptive fitness function based on branch hardness. The branch hardness is measured by the expected number of visits of each branch in the program, which is modeled by an absorbing discrete time Markov chain. By tuning the parameters of branch hardness heuristically, the search hardness, evaluated by the variation coefficient of the fitness function, of generating test data can be minimized. Therefore, this new fitness function is more flexible than the traditional counterparts. In addition, we point out that the present definition of branch distance and the use of normalizing functions are problematic, and propose some improvements. Finally, the empirical study reveals the promising result of our proposal in this paper.

Improved evolutionary generation of test data for multiple paths in search-based software testing

Abstract: Search-based software testing has achieved great attention recently, but the efficiency is still the bottleneck of it. This paper focuses on improving the efficiency of generating test data for multiple paths. Genetic algorithms are chosen as the heuristic algorithms in search-based software testing in this paper. First, we propose an improved grouping strategy of target paths to balance the load of each calculation resource. This work makes a contribution to the parallel execution in search-based software testing. Then, common constraints of the target paths in the same group are collected to reduce the search space of test data. Symbolic execution technique is used in this phase. Based on the reduced search space, we can accelerate the convergence of search process and improve the efficiency of search-based software testing. Finally, our method is applied to some study cases to compare with other methods.

Combining Search-Based Testing and Dynamic Symbolic Execution by Evolvability Metric

Abstract: In the area of software testing, search-based software testing (SBST) and dynamic symbolic execution (DSE) are two efficient testing techniques for test cases generation. However, both of the two approaches have their own drawbacks: The efficiency of SBST depends on the guidance of the fitness landscape. When the fitness landscape has some plateaus with no gradient for directing search process, SBST may degenerate into random testing. DSE relies on the capability of constraint solvers. It may struggle to generate test cases with constraints that are difficult to be solved. In this paper, we combine the strengths of both techniques. SBST is used to help DSE for solving difficult constraints and DSE is used to improve the efficiency and capability of SBST. Evolvability metric is introduced for measuring when the software is not suitable for SBST. A novel switch mechanism based on the evolvability metric between SBST and DSE is proposed in this paper to help to choose the proper technique at the proper time. Experiments on several benchmarks reveal the promising results of our proposal.

A dynamic fitness function for search based software testing

Abstract: Search Based Software Testing (SBST) formulates testing as an optimization problem, hence some search algorithms can be used to tackle it. The goal of SBST is to improve various test adequacy criteria. There are different types of coverage criteria, and in this paper, we deal with branch coverage, as it is the mostly used criterion. However, the state of the art fitness function definitions used for branch coverage testing have changed little. To fill this gap, this paper proposes a novel fitness function for branch coverage. Concretely, we first use a negative exponential model to evaluate the hardness of covering essential branches of the program, based on which we approximately evaluate the distance of a test candidate to the target branch. Finally, the experiment reveals the promising results of our proposal.

Boosting Search Based Software Testing by Using Ensemble Methods

Abstract: Search Based Software Testing (SBST) formulates testing as an optimization problem, hence some search algorithms (e.g., Genetic Algorithms) can be used to tackle it. There are different types of coverage criteria, and the goal of SBST is to improve various test adequacy criteria. However, the major limitation of SBST is the insufficiently informed fitness functions and the inefficient search algorithms. Besides, although there are various fitness functions and search algorithms for SBST, there is little guidance on when to use one fitness function (resp., search algorithm) over another. To address these problems, we propose an ensemble strategy to boost the performance of SBST. In this paper, we deal with path coverage. Concretely, by combining multiple weak fitness functions, the heuristic information of the problem instances can be expressed more sufficiently, and therefore, a stronger fitness function can be obtained. On the other hand, by combining multiple complementary search algorithms, a hyper-heuristic search algorithm is generated and the search performance can be improved. The empirical study reveals the promising results of our proposal. Especially, for the paths that are very difficult to be covered, our ensemble method proposed in this paper outperforms other approaches significantly.

Branch coverage prediction in automated testing

Abstract: This is the peer reviewed version which has been published in final form at [DOI]. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.

A systematic mapping addressing Hyper-Heuristics within Search-based Software Testing

Abstract: Context Search-based Software Testing (SBST) is a research field where testing a software product is formulated as an optimization problem. It is an active sub-area of Search-based Software Engineering (SBSE) where many studies have been published and some reviews have been carried out. The majority of studies in SBST has been adopted meta-heuristics while hyper-heuristics have a long way to go. Moreover, there is still a lack of studies to perceive the state-of-the-art of the use of hyper-heuristics within SBST. Objective The objective of this work is to investigate the adoption of hyper-heuristics for Software Testing highlighting the current efforts and identifying new research directions. Method A Systematic mapping study was carried out with 5 research questions considering papers published up to may/2019, and 4 different bases. The research questions aims to find out, among other things, what are the hyper-heuristics used in the context of Software Testing, for what problems hyper-heuristics have been applied, and what are the objective functions in the scope of Software Testing. Results A total of 734 studies were found via the search strings and 164 articles were related to Software Testing. However, from these, only 26 papers were actually in accordance with the scope of this research and 3 more papers were considered due to snowballing or expert’s suggestion, totalizing 29 selected papers. Few different problems and application domains where hyper-heuristics have been considered were identified. Conclusion Differently from other communities (Operational Research, Artificial Intelligence), SBST has little explored the benefits of hyper-heuristics which include generalization and less difficulty in parameterization. Hence, it is important to further investigate this area in order to alleviate the effort of practitioners to use such an approach in their testing activities.

Recovering fitness gradients for interprocedural Boolean flags in search-based testing

Abstract: In Search-based Software Testing (SBST), test generation is guided by fitness functions that estimate how close a test case is to reach an uncovered test goal (e.g., branch). A popular fitness function estimates how close conditional statements are to evaluating to true or false, i.e., the branch distance. However, when conditions read Boolean variables (e.g., if(x && y)), the branch distance provides no gradient for the search, since a Boolean can either be true or false. This flag problem can be addressed by transforming individual procedures such that Boolean flags are replaced with numeric comparisons that provide better guidance for the search. Unfortunately, defining a semantics-preserving transformation that is applicable in an interprocedural case, where Boolean flags are passed around as parameters and return values, is a daunting task. Thus, it is not yet supported by modern test generators. This work is based on the insight that fitness gradients can be recovered by using runtime information: Given an uncovered interprocedural flag branch, our approach (1) calculates context-sensitive branch distance for all control flows potentially returning the required flag in the called method, and (2) recursively aggregates these distances into a continuous value. We implemented our approach on top of the EvoSuite framework for Java, and empirically compared it with state-of-the-art testability transformations on non-trivial methods suffering from interprocedural flag problems, sampled from open source Java projects. Our experiment demonstrates that our approach achieves higher coverage on the subject methods with statistical significance and acceptable runtime overheads.

Scheduling sequence selection for generating test data to cover paths of MPI programs

Abstract: Context: As one of key tasks in software testing, test data generation has been receiving widespread attention in recent years. Message-passing Interface (MPI) programs, which are one representative type of parallel programs, have the characteristic of non-determinism, which is reflected by the non-deterministic execution under different scheduling sequences against the same program input. Previous studies have shown that different difficulties are raised in generating test data under different scheduling sequences, suggesting that selecting appropriate scheduling sequences is beneficial to a high efficiency. Objective: We propose a method of selecting a superior and feasible scheduling sequence for generating test data in the criterion of path coverage against each target path of an MPI program. Method: In the proposed method, a number of program inputs are first sampled by Latin hypercube sampling. Then, the program is executed against each sample under each scheduling sequence, and all the scheduling sequences are sorted according to the similarities between the paths traversed by these samples and the target one. Finally, the feasibility of a scheduling sequence with the best quality is investigated based on the symbolic execution. Results: We apply the proposed method to seven typical MPI programs and compare it with the random one. The experimental results show that test data covering the target path can be generated under the selected scheduling sequence with high success rate and low time consumption. Conclusion: The proposed method takes the influence of scheduling sequences on generating test data into consideration, thus providing a competent way to test parallel programs.