GUIDiVa: Automated Discovery and Validation of State-Based GUI Invariants

TLDR: This paper presents an automated black box framework that reveals an important set of GUI invariants as state-based event constraints, and an iterative algorithm that enumerates all possible constraint violations as potential reasons of a test case failure, on the failed event of the infeasible test case.

Abstract: Applications with a Graphical User Interface (GUI) front-end are ubiquitous, thus it is of great importance to assure the quality of such systems. Model-based GUI testing provides an effective mechanism for automated testing of applications with a GUI. However, most existing model-based GUI test generation techniques produce many infeasible (i.e., Broken) event sequences which are used as test cases. This happens primarily because they ignore behavioral specifications of the GUI under test. In this paper, we present an automated black box framework that reveals an important set of GUI invariants as state-based event constraints. GUIDiVa, an iterative algorithm at the core of our framework, enumerates all possible constraint violations as potential reasons of a test case failure, on the failed event of the infeasible test case. It further selects and adds the most promising constraints of each iteration to a final set based on our notion of constraint Validity Weight. The results of empirical studies on both seeded and nine non-trivial study subjects show that our framework is capable of capturing important aspects of GUI behavior, while considerably reducing the number of insfeasible test cases. Additionally, a human oracle is used to further validate the accuracy of the discovered constraints.