https://onlinelibrary.wiley.com/doi/pdf/10.1002/stvr.322

The Art of Software Testing, Second edition. Glenford J. Myers. Revised and updated by Tom Badgett and Todd M. Thomas, with Corey Sandler. John Wiley and Sons, New Jersey, U.S.A., 2004. ISBN: 0-471-46912-2, pp 234: Book Reviews

Since their introduction into software testing in the mid-1980s, combinatorial methods for test design gathered popularity as a testing best practice and as a prominent software testing research area. This chapter reviews recent advances in combinatorial testing, with special focus on the research since 2011. It provides a brief background on the theory behind combinatorial testing and on its use in practice. Requirements from industry usage have led to advances in various areas examined in this chapter, including constraints handling in combinatorial algorithms, support for the combinatorial modeling process, and studies on metrics to support the effectiveness of combinatorial testing. We also highlight recent case studies describing novel use cases for test and field quality improvement in the context of system test, and for optimization of test data. Finally, we examine recent developments in advanced topics such as utilization of existing tests, test case prioritization, fault localization, and evolution of combinatorial models.

Advances in Combinatorial Testing

Property-based testing uses randomly generated inputs to validate high-level program specifications. It can be shockingly effective at finding bugs, but it often requires generating a very large number of inputs to do so. In this paper, we apply ideas from combinatorial testing, a powerful and widely studied testing methodology, to modify the distributions of our random generators so as to find bugs with fewer tests. The key concept is combinatorial coverage, which measures the degree to which a given set of tests exercises every possible choice of values for every small combination of input features.

/pdf/do-judge-a-test-by-its-cover-3fr3khuntm.pdf

Do Judge a Test by its Cover

One of the main challenges of software testing research is the automated addition of oracles to the generated test cases: Whereas the automated generation of operation sequences (which is one of the essential components of test cases) is in practice a solved problem, the automated addition of the oracle (another indispensable element) is still an important problem and an open research question. This article proposes an approach to get executable test suites composed by complete test cases (i.e., they include the oracle). The core of the method is based on annotated regular expressions. The test generation process, which is supported by a tool, follows three steps: (1) creation of annotated regular expressions, where each regular expression describes a set of sequences of operations to be executed against the system under test; (2) expansion of the regular expressions to get sequences of operations, which still do not have parameter values; and (3) generation of the executable test cases with oracle. In this third step, each test case is generated with the suitable oracle, depending on the conditions specified in the regular expression.

/pdf/automated-generation-of-oracled-test-cases-with-regular-j93d68hshr.pdf

Automated generation of oracled test cases with regular expressions and combinatorial techniques

The internal control system must make an important contribution to the organization's performance by the ability to minimize the occurrence of risks associated with the processes. This article describesthe procedure used to improve the management of risks in transportation processes considered as a case study. The management and improvement processes are based on the use of mappingtechniques for the selection of risk sources, the Kendall method and the objective metrics against actors for the selection of processes.The result of the application of each of the stages was: the determination of the risk factors, the sources of risks, and the quantification of the risks classified as relevant with a high frequency of occurrence that require the taking of preventive actions and / or corrective measures to minimize their possible manifestations and the impact on the organization.

/pdf/priorizacion-de-casos-de-prueba-en-entornos-de-desarrollo-382v0dz5q4.pdf

Priorización de casos de prueba en entornos de desarrollo ágil

A test case describes a specific execution scenario of the system under test (SUT). Its goal is to discover errors by means of its oracle, that emits a pass or fail verdict depending on the SUT behavior. The test case has a sequence of calls to SUT's operations with specific test data, which may come from the application of a combinatorial algorithm. This paper describes a method to describe generic test scenarios by means of regular expressions, whose symbols point to a SUT operation. The tester assigns values to each operation's parameter. A further step expands the regular expression and produces a set of operation sequences, which are then passed to a combinatorial algorithm to generate actual test cases. Regular expressions are annotated with a set of when clauses, that are processed by the combinatorial algorithm to include the oracle in the generated test cases.

Francisco Ruiz Romero

Papers

Test Case Generation with Regular Expressions and Combinatorial Techniques