The need for test adequacy criteria is widely recognized. Several criteria have been proposed for the assessment of adequacy of tests at the unit level. However, there remains a lack of criteria for the assessment of the adequacy of tests generated during integration testing. We present a mutation based interprocedural criterion, named Interface Mutation (IM), suitable for use during integration testing. A case study to evaluate the proposed criterion is reported. In the study, the UNIX sort utility was seeded with errors and Interface Mutation evaluated by measuring the cost of its application and its error revealing effectiveness. Alternative IM criteria using different sets of Interface Mutation operators were also evaluated. While comparing the error revealing effectiveness of these Interface Mutation-based test sets with same size randomly generated test sets, we observed that in most cases Interface Mutation based test sets are superior. The results suggest that Interface Mutation offers a viable test adequacy criteria for use at the integration level.

Interface Mutation: an approach for integration testing

Mutation testing has been used mostly at the unit level. To support its application few tools have been developed and used, mainly in the academic environment. Interface Mutation has been proposed aiming at applying mutation at the integration level. A tool named Proteum/IM was implemented to support such criterion. With the definition of the Interface Mutation criterion the tester has the possibility of applying mutation testing concepts throughout the software development. It seems mandatory to have a single, integrated environment that would support mutation-based unit and integration testing. Such environment, which provides facilities to investigate low-cost and incremental testing strategies, is the focus of this paper.

Proteum/IM 2.0: An Integrated Mutation Testing Environment

One key to several quality factors of software is the way components are connected. Software coupling can be used to estimate a number of quality factors, including maintainability, complexity, and reliability. Object-oriented languages are designed to reduce the number of dependencies among classes, which encourages separation of concerns and should reduce the amount of coupling. At the same time, the object-oriented language features change the way the connections are made, how they must be analyzed, and how they are measured. This paper discusses software couplings based on object-oriented relationships between classes, specifically focusing on types of couplings that are not available until after the implementation is completed, and presents a static analysis tool that measures couplings among classes in Java packages. Data from evaluating the tool on several open-source projects are provided. The coupling measurement is based on source code, which has the advantage of being quantitative and more precise than previous measures, but the disadvantage of not being available before implementation, and thus not useful for some predictive efforts.

/pdf/quantitatively-measuring-object-oriented-couplings-4f8259zt83.pdf

Quantitatively measuring object-oriented couplings

Component-based software engineering (CBSE) has emerged as an approach that offers rapid development of system using fewer resources and effort. The core idea of reuse and cutting down the development costs can be achieved if the components offer reliable services. Thus, integration components and testing become an important phase in CBSE. Integration of components is a very important activity. This involves understanding communication and coordination between the components. Developers are not provided with sufficient information on these components. As a result of this, understanding data flow while integrating these components is a challenge. Component-based software facilitates development of complex systems by allowing integration of reusable components. Testing components is a challenging area of research. There have been troubles integrating the components. This in turn affects the quality and reliability of the software. Our research aims at finding the existing integration testing and challenges in CBSE. The systematic literature survey is based on 49 articles collected from multiple-stage selection process. These articles have been published within the time span of 1995–2009.

A systematic literature survey of integration testing in component-based software engineering

An experiment was conducted to evaluate an inter-procedural test adequacy criterion named Interface Mutation. Program SPACE, developed for the European Space Agency (ESA), was used in this experiment. The development record available for this program was used to find the faults uncovered during its development. Using this information the test process was reproduced starting with a version of SPACE containing several faults and then applying Interface Mutation. Thus we could evaluate the fault revealing effectiveness of Interface Mutation. Results from the experiment suggest that (a) the application of Interface Mutation favors the selection of fault revealing test cases when they exist and (b) Interface Mutation tends to select fault revealing test cases more efficiently than in the case where random selection is used.

Interface Mutation Test Adequacy Criterion: An Empirical Evaluation

Integration testing is an important part of the testing process, but few integration testing techniques have been systematically studied or defined. This paper presents an integration testing technique based on couplings between software components. The coupling-based testing technique is described, and 12 coverage criteria are defined. The coupling-based technique is also compared with the category-partition method on a case study, which found that the coupling-based technique detected more faults with fewer test cases than category-partition. This modest result indicates that the coupling-based testing approach can benefit practitioners who are performing integration testing on software. While it is our intention to develop algorithms to fully automate this technique, it is relatively easy to apply by hand.

A.J. Offutt

Papers

Integration testing based on software couplings