The quantitative evaluation of computer performance is needed during the entire life cycle of a computer system. We survey the major quantitative methods used in computer performance evaluation, focusing on post-1970 developments and emphasizing trends and challenges. We divide the methods used into three main areas, namely performance measurement, analytic performance modeling, and simulation performance modeling, which we survey in the three main sections of the paper. Although we concentrate on the methods per se, rather than on the results of applying the methods, numerous application examples are cited. The methods to be covered have been applied across the entire spectrum of computer systems from personal computers to large mainframes and supercomputers, including both centralized and distributed systems. The application of these methods has certainly not decreased over the years and we anticipate their continued use as well as their enhancement when needed to evaluate future systems.

Computer Performance Evaluation Methodology

Equivalence and decomposition in queueing systems—A unified approach

The purpose of this paper is to review, unify, and extend previous work on sample-path analysis of queues. Our main interest is in the asymptotic behavior of a discrete-state, continuous-time process with an imbedded point process. We present a sample-path analogue of the renewal-reward theorem, which we callY=λX. We then applyY=λX to derive several relations involving the transition rates and the asymptotic (long-run) state frequencies at an arbitrary point in time and at the points of the imbedded point process. Included are sample-path versions of the rate-conservation principle, the global-balance conditions, and the insensitivity of the asymptotic frequency distribution to the distribution of processing time in a LCFS-PR service facility. We also provide a natural sample-path characterization of the PASTA property.

Sample-path analysis of processes with imbedded point processes

In this paper we provide a survey of sample-path methods in queueing theory, particularly in connection with “distribution-free” analysis such as (i) relations between customer averages and time averages, such as L = λW (Little’s formula); (ii) relations between the stationary distribution of a process and an imbedded process; and (iii) the phenomenon of insensitivity.

Sample-Path Analysis of Queues

Operational analysis of stochastic closed queueing networks

Operational analysis replaces certain classical queueing theory assumptions with the conditions of "homogeneous service times" and "on-line= off-line behavior." In this paper we explore the relationship between the operational and classical concepts for the sample paths of an M/G/1 queueing system. The primary results are that the sample paths can have these operational properties with nonzero probability if and only if the service time is exponential. We also state dual results for interarrival times in G/M/l. Additionally, we show that open, feedforward networks of single server queues can have product form solutions valid across a range of system arrival rates if and only if all of the service times are exponential. Finally, we consider the relationship between the operational quantities S(n) and the mean service time in M/G/1. This relationship is shown to depend on the form of the service time distribution. It follows that using operational analysis to predict the performance of an M/G/1 queueing system will be most successful when the service time is exponential. Simulation evidence is presented which supports this claim.

R.M. Bryant

Papers

On Homogeneity and On Line=Off-Line Behavior in M/G/1 Queueing Systems