Architecture-based approach to reliability assessment of software systems

Statistical testing of software establishes a basis for statistical inference about a software system's expected field quality. This paper describes a method for statistical testing based on a Markov chain model of software usage. The significance of the Markov chain is twofold. First, it allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques. Analytical results associated with Markov chains facilitate informative analysis of the sequences before they are generated, indicating how the test is likely to unfold. Second, the test input sequences generated from the chain and applied to the software are themselves a stochastic model and are used to create a second Markov chain to encapsulate the history of the test, including any observed failure information. The influence of the failures is assessed through analytical computations on this chain. We also derive a stopping criterion for the testing process based on a comparison of the sequence generating properties of the two chains. >

/pdf/a-markov-chain-model-for-statistical-software-testing-26830n6gtt.pdf

A Markov chain model for statistical software testing

Software safety issues become important when computers are used to control real-time, safety-critical processes. This survey attempts to explain why there is a problem, what the problem is, and what is known about how to solve it. Since this is a relatively new software research area, emphasis is placed on delineating the outstanding issues and research topics.

/pdf/software-safety-why-what-and-how-44olmi9ii7.pdf

Software safety: why, what, and how

One of the motivations for specifying software architectures explicitly is the use of high level structural design information for improved control and prediction of software system quality attributes. In this paper, we present an approach for determining the reliability of component-based software architectures.Our method is based on rich architecture definition language (RADL) oriented towards modem industrial middleware platforms, such as Microsoft's. NET and Sun's EJB. Our methods involve parameterised contractual specifications based on state machines and thus permits efficient static analysis.We show how RADL allows software architects to predict component reliability through compositional analysis of usage profiles and of environment component reliability. We illustrate our approach with an e-commerce example and report about empirical measurements which confirm our analytical reliability prediction through monitoring in our reliability test-bed. Our evaluation confirms that prediction accuracy for software components necessitates modelling the behaviour of binary components and the dependency of provided services on required components. Fortunately, our measurements also show that an abstract protocol view of that behaviour is sufficient to predict reliability with high accuracy. The reliability of a component most strongly depends on its environment. Therefore, we advocate a reliability model parameterized by required component reliability in a deployment context.

Reliability prediction for component-based software architectures

It is essential to assess the reliability of digital computer systems used for critical real-time control applications (e.g., nuclear power plant safety control systems). This involves the assessment of the design correctness of the combined hardware/software system as well as the reliability of the hardware. In this paper we survey methods of determining the design correctness of systems as applied to computer programs.

Software Reliability&#8212;Status and Perspectives

A user-oriented reliability model has been developed to measure the reliability of service that a system provides to a user community. It has been observed that in many systems, especially software systems, reliable service can be provided to a user when it is known that errors exist, provided that the service requested does not utilize the defective parts. The reliability of service, therefore, depends both on the reliability of the components and the probabilistic distribution of the utilization of the components to provide the service. In this paper, a user-oriented software reliability figure of merit is defined to measure the reliability of a software system with respect to a user environment. The effects of the user profile, which summarizes the characteristics of the users of a system, on system reliability are discussed. A simple Markov model is formulated to determine the reliability of a software system based on the reliability of each individual module and the measured intermodular transition probabilities as the user profile. Sensitivity analysis techniques are developed to determine modules most critical to system reliability. The applications of this model to develop cost-effective testing strategies and to determine the expected penalty cost of failures are also discussed. Some future refinements and extensions of the model are presented.

A User-Oriented Software Reliability Model

A user-oriented reliability model has been developed to measure the reliability of service that a system provides to a user community. It has been observed that in many systems, especially software systems, reliable service can be provided to a user when it is known that errors exist, provided that the service requested does not utilize the defective parts. The reliability of service, therefore, depends both on the reliability of the components and the probabilistic distribution of ulitization of the components to provide the service. In this paper a user-oriented reliability figure of merit is defined to measure the reliability of a software system with respect to a user environment. The effects of the user profile, which summarizes the characteristics of the users of a system, on system reliability is discussed. A simple Markov model is formulated to determine the reliability of a software system based on the reliability of each individual module and the measured inter-modular transition probabilities as the user profile. Sensitivity analysis techniques are developed to determine modules most critical to system reliability. The applications of this model to develop cost-effective testing strategies and to determine the expected penalty cost of failure are also discussed. Some future refinements and extensions of the model are presented.

A user-oriented software reliability model

Controlling a telephone switching system is a difficult software problem involving many concurrent processes and complex patterns of inter-process communication. This paper describes the functional multiprocessing architecture used in the control of an Experimental Digital Switching office. The software system is partitioned into two subsystems: Peripheral Control and Call Processing. The Peripheral Control Subsystem maintains the status of the switching resources, passes customer inputs to the Call Processing Subsystem, and carries out all the hardware control functions. The Call Processing Subsystem performs all the decision making functions to provide telephone features. By choosing a proper interface between them, the Peripheral Control Subsystem hides the hardware details of the peripherals to provide a high-level abstract switching machine. By isolating feature processing from hardware control, new services and peripherals can be introduced to the system easily. The authors' experience with this system is presented.

R.C. Cheung

Papers

A User-Oriented Software Reliability Model

A user-oriented software reliability model

Functional multiprocessing in an experimental digital switching office