Testing is a trade-off between increased confidence in the correctness of the implementation under test and constraints on the amount of time and effort that can be spent in testing. Therefore, the coverage, or adequacy of the test suite, becomes a very important issue. In this paper, we analyze basic ideas underlying the techniques for fault coverage analysis and assurance mainly developed in the context of protocol conformance testing based on finite state models. Special attention is paid to parameters which determine the testability of a given specification and influence the length of a test suite which guarantees complete fault coverage. We also point out certain issues which need further study.

On fault coverage of tests for finite state specifications

The aim of this work is to explore empirically the relationship between lack of cohesion metrics and testability of classes in object-oriented systems. We addressed testability from the perspective of unit testing. We performed an empirical analysis using data collected from two Java software systems for which JUnit test cases exist. To capture testability of classes, we used different metrics to measure some characteristics of the corresponding JUnit test cases. In order to evaluate the capability of lack of cohesion metrics to predict testability, we used statistical analysis techniques using correlation and logistic regression. The performance of the predicted model was evaluated using Receiver Operating Characteristic (ROC) analysis. The achieved results provide evidence that there exist a relationship between lack of cohesion and testability.

An Empirical Analysis of Lack of Cohesion Metrics for Predicting Testability of Classes

In this paper, we investigate empirically the relationship between object-oriented design metrics and testability of classes. We address testability from the point of view of unit testing effort. We collected data from three open source Java software systems for which JUnit test cases exist. To capture the testing effort of classes, we used metrics to quantify the corresponding JUnit test cases. Classes were classified, according to the required unit testing effort, in two categories: high and low. In order to evaluate the relationship between object-oriented design metrics and unit testing effort of classes, we used logistic regression methods. We used the univariate logistic regression analysis to evaluate the individual effect of each metric on the unit testing effort of classes. The multivariate logistic regression analysis was used to explore the combined effect of the metrics. The performance of the prediction models was evaluated using Receiver Operating Characteristic analysis. The results indicate that: 1) complexity, size, cohesion and (to some extent) coupling were found significant predictors of the unit testing effort of classes and 2) multivariate regression models based on object-oriented design metrics are able to accurately predict the unit testing effort of classes.

/pdf/empirical-analysis-of-object-oriented-design-metrics-for-mv3wmd1xnc.pdf

Empirical Analysis of Object-Oriented Design Metrics for Predicting Unit Testing Effort of Classes

Context Software testability is the degree to which a software system or a unit under test supports its own testing. To predict and improve software testability, a large number of techniques and metrics have been proposed by both practitioners and researchers in the last several decades. Reviewing and getting an overview of the entire state-of-the-art and state-of-the-practice in this area is often challenging for a practitioner or a new researcher. Objective Our objective is to summarize the body of knowledge in this area and to benefit the readers (both practitioners and researchers) in preparing, measuring and improving software testability. Method To address the above need, the authors conducted a survey in the form of a systematic literature mapping (classification) to find out what we as a community know about this topic. After compiling an initial pool of 303 papers, and applying a set of inclusion/exclusion criteria, our final pool included 208 papers (published between 1982 and 2017). Results The area of software testability has been comprehensively studied by researchers and practitioners. Approaches for measurement of testability and improvement of testability are the most-frequently addressed in the papers. The two most often mentioned factors affecting testability are observability and controllability. Common ways to improve testability are testability transformation, improving observability, adding assertions, and improving controllability. Conclusion This paper serves for both researchers and practitioners as an “index” to the vast body of knowledge in the area of testability. The results could help practitioners measure and improve software testability in their projects. To assess potential benefits of this review paper, we shared its draft version with two of our industrial collaborators. They stated that they found the review useful and beneficial in their testing activities. Our results can also benefit researchers in observing the trends in this area and identify the topics that require further investigation.

A survey on software testability

The study presented in this paper aims at exploring empirically the relationship between lack of cohesion and testability of classes in object-oriented systems. We investigated testability from the perspective of unit testing. We designed and conducted an empirical study using two Java software systems for which JUnit test cases exist. To capture testability of classes, we used different metrics to measure some characteristics of the corresponding JUnit test cases. We used also some lack of cohesion metrics. In order to evaluate the capability of lack of cohesion metrics to predict testability, we performed statistical tests using correlation. The achieved results provide evidence that (lack of) cohesion may be associated with (low) testability.

Exploring Empirically the Relationship between Lack of Cohesion and Testability in Object-Oriented Systems

Communication software has become more complex and therefore more difficult to test. In order to handle the complexity of tests for communication protocols, a research topic known as the design for testability (DFT) has emerged. The main objective of DFT is to reduce the cost and the complexity of tests. Testability activity and its analysis necessitate the use of estimation techniques or measures. The early estimation of testability in the design phase helps designers to identify parts of the specification that are hard to test; then appropriate transformations can be proposed to enhance the testability of the end product.

Specification transformations and design for testability

To deal with the increased complexity related to the testing of communications software, we propose the integration and application of finite state machine based specification transformations and testability measures early in the communications software development process. Based on this integration, the testability of a given design is estimated and appropriate specification transformations are defined and applied iteratively to enhance the testability of the product implementation.

Communications software design for testability: specification transformations and testability measures

Design For Testability (DFT) is understood as the process of introducing some features into a communication software that facilitate the testing process of protocol implementations. DFT at the implementation level deals with a particular realization on a given platform, whereas DFT at the specification level affects all possible implementations regardless of the implementation process. In this paper, we propose in the first part, a generic framework of design for testability where DFT activity is integrated in the communication software development process. A generic model for testability transformation based on modification of a given specification and testability measurement is explained. In the second part, we address one particular problem of DFT, the problem of analyzing the testability of a given module specified as a finite state machine. Transformations in this paper are not the main objective, but they are used to illustrate how testability can be enhanced by finding an augmentation of the given protocol behavior such that a newly obtained specification is more testable than the original one A measure of testability of a protocol entity is assumed to be based on the shortest length of a test suite needed to achieve guaranteed coverage of certain faults. We consider the FSM model of a protocol machine and propose a classification method based on the formalized notions of controllability and observability.

https://link.springer.com/content/pdf/10.1007%2F978-0-387-34988-6_15.pdf

Towards testable communication software

Design For Testability (DFT) is understood as the process of introducing some features into a protocol entity that facilitate the testing process of protocol implementations. DFT at the implementation level deals with a particular realization on a given platform, whereas DFT at the specification level affects all possible implementations regardless of the implementation process. The fact that protocols are usually specified only partially, facilitates DEC at the specification level. In this paper, we address one particular problem of DFT, the problem of finding a minimal augmentation of the given protocol behavior (an FSM) such that a newly obtained specification is more testable than the original one, while maintaining sets of defined states and events. We propose an approach to augmenting a partially specified FSM such that a test suite for the resulting FSM with guaranteed fault coverage is shorter than that for the original FSM.

/pdf/on-the-design-for-testability-of-communication-protocols-3o7pnmyh4j.pdf

On the design for testability of communication protocols

Publisher Summary This chapter discusses the problem of augmenting a partially specified extended finite-state machine, extracted from a Specification Description Language (SDL) specification, to have shorter tests. Conformance testing is used to verify whether an implementation under test exhibits a permissible behavior. Test cases derived from a reference specification are applied to an implementation under test to compare its behavior with the reference one. Length of test cases, facility of test generation, and analysis of results of test application are important factors that affect the cost of testing. Various formal description techniques (FDTs)—such as SDL, LOTOS or ESTELLE—are used for communication protocols. Most of the FDTs possess some practical tools for automatic verification and simulation of a protocol to detect and correct some classes of faults before implementing the protocol. Some unspecified transitions—such as those that can be interpreted as “don't care” transitions—to improve the testability of protocol's implementations are used in the chapter.

K. Karoui

Papers

Specification transformations and design for testability

Communications software design for testability: specification transformations and testability measures

Towards testable communication software

On the design for testability of communication protocols

Design for Testability of Communication Protocols Based on SDL Specification