Test strategy

About: Test strategy is a research topic. Over the lifetime, 4208 publications have been published within this topic receiving 89763 citations. The topic is also known as: testing approach.

Practical Model-Based Testing: A Tools Approach

Abstract: This book gives a practical introduction to model-based testing, showing how to write models for testing purposes and how to use model-based testing tools to generate test suites. It is aimed at testers and software developers who wish to use model-based testing, rather than at tool-developers or academics. The book focuses on the mainstream practice of functional black-box testing and covers different styles of models, especially transition-based models (UML state machines) and pre/post models (UML/OCL specifications and B notation). The steps of applying model-based testing are demonstrated on examples and case studies from a variety of software domains, including embedded software and information systems. From this book you will learn: * The basic principles and terminology of model-based testing * How model-based testing differs from other testing processes * How model-based testing fits into typical software lifecycles such as agile methods and the Unified Process * The benefits and limitations of model-based testing, its cost effectiveness and how it can reduce time-to-market * A step-by-step process for applying model-based testing * How to write good models for model-based testing * How to use a variety of test selection criteria to control the tests that are generated from your models * How model-based testing can connect to existing automated test execution platforms such as Mercury Test Director, Java JUnit, and proprietary test execution environments * Presents the basic principles and terminology of model-based testing * Shows how model-based testing fits into the software lifecycle, its cost-effectiveness, and how it can reduce time to market * Offers guidance on how to use different kinds of modeling techniques, useful test generation strategies, how to apply model-based testing techniques to real applications using case studies

Operational profiles in software-reliability engineering

Abstract: A systematic approach to organizing the process of determining the operational profile for guiding software development is presented. The operational profile is a quantitative characterization of how a system will be used that shows how to increase productivity and reliability and speed development by allocating development resources to function on the basis of use. Using an operational profile to guide testing ensures that if testing is terminated and the software is shipped because of schedule constraints, the most-used operations will have received the most testing and the reliability level will be the maximum that is practically achievable for the given test time. For guiding regression testing, it efficiently allocates test cases in accordance with use, so the faults most likely to be found, of those introduced by changes, are the ones that have the most effect on reliability. >

Test case prioritization: an empirical study

Abstract: Test case prioritization techniques schedule test cases for execution in an order that attempts to maximize some objective function. A variety of objective functions are applicable; one such function involves rate of fault detection-a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during regression testing can provide faster feedback on a system under regression test and let debuggers begin their work earlier than might otherwise be possible. In this paper we describe several techniques for prioritizing test cases and report our empirical results measuring the effectiveness of these techniques for improving rate of fault detection. The results provide insights into the tradeoffs among various techniques for test case prioritization.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks

Abstract: This paper describes a testing methodology for quantitatively assessing the risk that rare or unique training-data sequences are unintentionally memorized by generative sequence models---a common type of machine-learning model. Because such models are sometimes trained on sensitive data (e.g., the text of users' private messages), this methodology can benefit privacy by allowing deep-learning practitioners to select means of training that minimize such memorization. In experiments, we show that unintended memorization is a persistent, hard-to-avoid issue that can have serious consequences. Specifically, for models trained without consideration of memorization, we describe new, efficient procedures that can extract unique, secret sequences, such as credit card numbers. We show that our testing strategy is a practical and easy-to-use first line of defense, e.g., by describing its application to quantitatively limit data exposure in Google's Smart Compose, a commercial text-completion neural network trained on millions of users' email messages.