Lex Heerink

Bio: Lex Heerink is an academic researcher from Philips. The author has contributed to research in topics: Promela & Conformance testing. The author has an hindex of 2, co-authored 2 publications receiving 180 citations.

Formal Test Automation: A Simple Experiment

Abstract: In this paper we study the automation of test derivation and execution in the area of conformance testing. The test scenarios are derived from multiple specification languages: LOTOS, Promela and SDL. A central theme of this study is the usability of batch-oriented and on-the-fly testing approaches. To facilitate the derivation from multiple formal description techniques and the different test execution approaches, an open, generic environment called TorX is introduced. TorX enables plugging in existing or dedicated tools. We have carried out several experiments in testing a conference protocol, resulting in requirements on automated testing and benchmarking criteria.

Formal Test Automation: The Conference Protocol with PHACT

Abstract: We discuss a case study of automatic test generation and test execution based on formal methods The case is the Conference Protocol, a simple, chatbox-like protocol, for which (formal) specifications and multiple implementations are publicly available and which is also used in other case study experiments The tool used for test generation and test execution is Phact, the PHilips Automated Conformance Tester The formal method is (Extented) Finite State Machines which is the input language for Phact The experiment consists of developing a Finite State Machine specification for the Conference Protocol, generating 82 tests in TTCN with Phact, and executing these tests against 28 different implementations of the Conference Protocol, both correct and erroneous ones The result is that some erroneous implementations are not detected by the test cases These results are analysed, the merits of Extented Finite State Machines for specification are discussed, and the achievements of Phact are assessed Moreover, the results are compared with a previous experiment in which the same 28 implementations were tested based on specifications in LOTOS and Promela

Model-based testing in practice

Abstract: Model-based testing is a new and evolving technique for generating a suite of test cases from requirements. Testers using this approach concentrate on a data model and generation infrastructure instead of hand-crafting individual tests. Several relatively small studies have demonstrated how combinatorial test generation techniques allow testers to achieve broad coverage of the input domain with a small number of tests. We have conducted several relatively large projects in which we applied these techniques to systems with millions of lines of code. Given the complexity of testing, the model-based testing approach was used in conjunction with test automation harnesses. Since no large empirical study has been conducted to measure efficacy of this new approach, we report on our experience with developing tools and methods in support of model-based testing. The four case studies presented here offer details and results of applying combinatorial test-generation techniques on a large scale to diverse applications. Based on the four projects, we offer our insights into what works in practice and our thoughts about obstacles to transferring this technology into testing organizations.

CADP 2011: a toolbox for the construction and analysis of distributed processes

Abstract: CADP (Construction and Analysis of Distributed Processes) is a comprehensive software toolbox that implements the results of concurrency theory. Started in the mid 80s, CADP has been continuously developed by adding new tools and enhancing existing ones. Today, CADP benefits from a worldwide user community, both in academia and industry. This paper presents the latest release, CADP 2011, which is the result of a considerable development effort spanning the last five years. The paper first describes the theoretical principles and the modular architecture of CADP, which has inspired several other recent model checkers. The paper then reviews the main features of CADP 2011, including compilers for various formal specification languages, equivalence checkers, model checkers, compositional verification tools, performance evaluation tools, and parallel verification tools running on clusters and grids. Finally, the paper surveys some significant case studies.

Testing Concurrent Systems: A Formal Approach

Abstract: This paper discusses the use of formal methods in testing of concurrent systems. It is argued that formal methods and testing can be mutually profitable and useful. A framework for testing based on formal specifications is presented. This framework is elaborated for labelled transition systems, providing formal definitions of conformance, test execution and test derivation. A test derivation algorithm is given and its tool implementation is briefly discussed.

Testing real-time systems using UPPAAL

Abstract: This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

TorX: Automated Model-Based Testing

Abstract: Systematic testing is very important for assessing and improving the quality of software systems. Yet, testing turns out to be expensive, laborious, time-consuming and error-prone. The Dutch research and development project Cote de Resyste worked on methods, techniques and tools for automating specification based testing using formal methods. The main achievement of the project is a test tool, baptized TorX, which integrates automatic test generation, test execution, and test analysis in an on-the-fly manner. On the one hand, TorX is based on well-defined theory, viz. the ioco-test theory, which has its roots in the theory of testing- and refusal-equivalences for transition systems. On the other hand, the applicability of TorX has been demonstrated by testing several academic and industrial case studies. This paper summarizes the main results of the project Cote de Resyste.