The development of techniques for quantitative, model-based evaluation of computer system dependability has a long and rich history. A wide array of model-based evaluation techniques is now available, ranging from combinatorial methods, which are useful for quick, rough-cut analyses, to state-based methods, such as Markov reward models, and detailed, discrete-event simulation. The use of quantitative techniques for security evaluation is much less common, and has typically taken the form of formal analysis of small parts of an overall design, or experimental red team-based approaches. Alone, neither of these approaches is fully satisfactory, and we argue that there is much to be gained through the development of a sound model-based methodology for quantifying the security one can expect from a particular design. In this work, we survey existing model-based techniques for evaluating system dependability, and summarize how they are now being extended to evaluate system security. We find that many techniques from dependability evaluation can be applied in the security domain, but that significant challenges remain, largely due to fundamental differences between the accidental nature of the faults commonly assumed in dependability evaluation, and the intentional, human nature of cyber attacks.

/pdf/model-based-evaluation-from-dependability-to-security-50ybd7tqdv.pdf

Model-based evaluation: from dependability to security

This paper demonstrates a practical approach to the study of the failure behavior of computer systems. Particular attention is devoted to the analysis of permanent failures. A number of important techniques, which may have general applicability in both failure and workload analysis, are brought together in this presentation. These include: smeared averaging of the workload data, clustering of like failures, and joint analysis of workload and failures. Approximately 17 percent of all failures affecting the CPU were estimated to be permanent. The manifestation of a permanent failure was found to be strongly correlated with the level and type of workload prior to the failure. Although, in strict terms, the results only relate to the manifestation of permanent failures and not to their occurrence, there are strong indications that permanent failures are both caused and discovered by increased activity. More measurements and experiments are necessary to determine their respective contributions to the measured workload/failure relationship.

Measurement and modeling of computer reliability as affected by system activity

In computer design, it is essential to know the effectiveness of different design options in improving performance and dependability. Various software tools have been created to evaluate these parameters, applying both analytic and simulation techniques, and this paper reviews those related primarily to reliability, availability, and serviceability. The purpose, type of models used, type of systems modeled, inputs, and outputs are given for each package. Examples of some of the key modeling elements such as Markov chains, fault trees, and Petri nets are discussed. The information is compiled to facilitate recognition of similarities and differences between various models and tools and can be used to aid in selecting models and tools for a particular application or designing tools for future needs. Tools included in the evaluation are CARE-III, ARIES-82, SAVE, MARKl, HARP, SHARPE, GRAMP, SURF, SURE, ASSIST, METASAN, METFAC, ARM, and SUPER. Modeling tools, such as REL70, RELCOMP, CARE, CARSRA, and CAST, that were forerunners to some of the current tools are noted for their contributions. Modeling elements that have gained widespread use for general systems, as well as fault-tolerant systems, are included. Tools capable of modeling both repairable and nonrepairable systems, accepting constant or time varying failure rates, and predicting reliability, availability, and serviceability parameters are surveyed.

Survey of software tools for evaluating reliability, availability, and serviceability

The feasibility of integrating failure modes and effects analysis (FMEA) and sneak circuit analysis (SCA) into a comprehensive reliability analysis technique is examined especially from the perspective of automation. FMEA looks at a system's strengths and weaknesses; SCA looks for latent circuit conditions which may lead to unplanned or unexpected modes of operation. The goals of the two techniques complement each other and their combination results in a more comprehensive analysis than either technique alone can achieve. The rich collection of heuristics used in SCA can be applied to design validation and also used as guidelines at various stages of system design. At both the functional level and the component level, the combined analysis is done using the same circuit representation as for the SCA and for the FMEA, and draws on the same database. >

Combining sneak circuit analysis and failure modes and effects analysis

Provides a comprehensive and evaluative account of product reliability across a number of different dimensions. Through utilizing secondary sources of data and supported by some preliminary qualitative research findings, attempts to decompose those elements which have a direct relevance to product reliability improvement. To this end, provides a conceptual framework which can be integrated within the whole business interaction entities to enable reliability to be adequately considered throughout the life cycle phases.

J. A. Roberts

Papers

Reliability and Maintainability of Electronic Systems