The current increased demand for distributed applications in domains such as web services and cloud computing has significantly increased interest in concurrent programming. This demand in turn has resulted in new testing methodologies for such systems, which take account of the challenges necessary to test these applications. This paper presents a systematic review of the published research related to concurrent testing approaches, bug classification and testing tools. A systematic review is a process of collection, assessment and interpretation of the published papers related to a specific search question, designed to provide a background for further research. The results include information about the research relationships and research teams that are working in the different areas of concurrent programs testing.

/pdf/research-in-concurrent-software-testing-a-systematic-review-t47wrayio1.pdf

Research in concurrent software testing: a systematic review

Object-oriented programs involve many unique features that are not present in their conventional counterparts. Examples are message passing, synchronization, dynamic binding, object instantiation, persistence, encapsulation, inheritance, and polymorphism. Integration testing for such programs is, therefore, more difficult than that for conventional programs. In this paper, we present an overview of current work on integration testing for object-oriented and/or concurrent programs, with a view to identifying areas for future research. We cover state-based testing, eventbased testing, fault-based testing, deterministic and reachability techniques, and formal and semiformal techniques.

/pdf/an-overview-of-integration-testing-techniques-for-object-1csrjggli0.pdf

An Overview of Integration Testing Techniques for Object-Oriented Programs

This paper presents a testing method for an agent system against its specification described with Statecharts. Here, an agent system means an implementation of an agent application. Its testing method generates test sequences from the specification and checks whether the agent system behaves in accordance with the specification by executing the test sequences. For specification-based testing, we have extended Statecharts in order to describe the behavior of an agent system according to the concept of Agent UML. Then, we propose an approach to generating test sequences from the extended Statecharts. Our approach makes effective use of partial order methods for considering representative sequences of equivalent classes divided from all possible sequences. Therefore, we can efficiently manage a large number of possible sequences caused by the agents’ autonomy. As a result, we can reduce the number of test sequences and the number of executions to be tested.

Modeling and testing agent systems based on statecharts

Previous studies on estimating software reliability employed statistical functions for next system failure prediction. These models used parameters based on assumptions regarding the nature of software faults and debugging process. However, none of the existing models, attempted on ensuring reliable runtime system operation. To serve the current demand of autonomous, reliable, service-oriented software, we present a novel approach for runtime reliability estimation of executable software. The approach can help control software execution at runtime by monitoring software state-to-state transition at runtime. The approach involves representing executable software as an automata using opcode extracted from executable code. The extracted opcode is then used to learn stochastic finite state machine (SFSM) representation of executable software which is later employed to trace software state-to-state transition at each runtime instance. An evaluation of our approach on Java-based Chart generator application is also discussed to explain how we can ensure reliable software execution and prevent software failures at runtime with the proposed approach.

/pdf/estimating-software-reliability-by-monitoring-software-5e336mu04v.pdf

Estimating Software Reliability by Monitoring Software Execution through OpCode

This paper presents an approach to specification-based testing of concurrent programs with representative test sequences generated from Statecharts. Representative test sequences are a subset of all possible interleavings of concurrent events that define the behaviors of a concurrent program. Because a program's correctness may be determined by checking whether a program implemented all behaviors in its specification or not, the program can be regarded as being correct if it can supply an alternative execution that has the same effects as the program's behavior with each representative test sequence. Based on this observation, we employ each representative test sequence as a seed to generate an automaton that accepts its equivalent sequences to reveal the same behavior. In order to effectively test a concurrent program, the automaton such generated accepts all sequences equivalent to the representative test sequence and it is used to control test execution. This paper describes an automated process of generating automata from a Statecharts specification and shows how the proposed approach works on Statecharts specifications through some examples.

Generating Test Sequences from Statecharts for Concurrent Program Testing

Deterministic execution testing has been considered a promising method for concurrent program testing because of its reproducibility. However, since deterministic execution requires that a synchronization sequence to be replayed is feasible and valid, it is not directly applicable to a situation in which synchronization sequences, being valid but infeasible, are taken into account. To resolve this problem, we proposed automata-based testing in previous work, where a concurrent program is executed according to one sequence accepted by the automaton recognizing all sequences semantically equivalent to a given sequence. In this paper, we present the automata-based testing environment for Java multi-thread programs, and design and implement key components-automata generator, program transformer and replay controller. Algorithms for generating the equivalence automaton of a given sequence are presented and a program transformation method is suggested in order to guide a program to be executed according to the sequence accepted by the automaton. The replay controller is also redesigned and implemented to adopt the automaton. By illustrating automata-based testing procedures with the gas station example, we show how the proposed approach works in Java multi-threaded programs.

/pdf/the-design-and-implementation-of-automata-based-testing-30p0obpurb.pdf

Yong Rae Kwou

Papers

The design and implementation of automata-based testing environment for Java multi-thread programs