Three major purposes for evaluating the hardware and software performance of computer systems--selection evaluation, performance projection, and performance monitoring--are described. Eight techniques that have been used or suggested for evaluating performance are discussed. Each of these techniques is rated on its suitability for the three purposes of evaluation. Recommendations are made on the most appropriate technique for each evaluation purpose. These suggestions include the development of a comprehensive set of synthetic programs on an industry-wide basis for selection evaluation purposes. Simulation is recommended as the most suitable technique for performance projection. Finally, a number of hardware and software monitors are available for performance monitoring.

Performance Evaluation and Monitoring

After an introduction which discusses the significance of a trend to the design of parallel processing systems, the paper describes some of the results obtained to date in a project which aims to develop and evaluate a unified hardware-software parallel processing computing system and the techniques for its use.

A survey of problems and preliminary results concerning parallel processing and parallel processors

In this paper, the following definitions have been followed:1 Multiprogramming---the time-sharing of a processor by many programs operating sequentially Many programs are available and in memory but only one program is actually being executed at a given time Control of object programs is provided by a supervisory control program Thruput is highest when many programs can be interleaved to use hardware most efficiently In general, the time required to complete a selected program will be increased over single program operation2 Multiprocessing---independent and simultaneous processing accomplished by the use of several duplicate hardware units Specifically, duplicate logical and arithmetic units are assumed, although systems with separate input-output channels can also be said to be multiprocessors Note that "processors" do not include storage units while "computers" do (Table 122)3 Scheduling---is the determination of the sequence in which job programs will use the available facilities Scheduling assignments are based on the availability of all required facilities, the priority of the job program and the relative priorities of other programs Scheduling algorithms aim to optimize performance of the system with respect to chosen goals4 Allocation---is the assignment of particular facilities: core memory, tapes, disk files to a job program5 Interrupt and Trapping are considered synonymous Both mean the ability, provided by hardware, to monitor particular conditions in the system during execution of all other operations and to provide an alarm signal which can interrupt a processor to obtain required action Program interrupts or Intentional interrupts are really branching operations which sometimes use the alarm signal hardware

Generalized multiprocessing and multiprogramming systems

A historical summary of the growing complexities in computing systems and the effect of these complexities on system performance evalution are presented.

The paper outlines basic approach. It considers the application of test results and the test data itself. Two general approaches to gathering performance data are discussed.

A perspective on system performance evaluation

The author assumes that the reader is familiar with the content of the preceding paper, "A General Purpose Systems Simulator," by G. Gordon. Two dissimilar examples are provided to illustrate various aspects of simulation in the systems engineering process. One example involves the study of an IBM 7040--IBM 7090 computer complex for scientific applications. The other concerns an IBM 1410 Tele-processing system for a stock brokerage house.

In addition to illustrating the paper, both examples are of intrinsic interest. The first presents a new philosophy of multiprocessing. The second examines a method of integrating communication facilities with an information processor.

Simulation in systems engineering

This paper examines the machine utilization and job turnaround problems of a large computer center by analyzing the information handling and queuing problems occurring between jobs.

A system designed to overcome these difficulties is described and the results of simulating the system are reported. The system design includes the interconnection of input-output computers with large scale processors by means of commonly shared disk files.

Although this paper deals with a study which is not yet completed, the techniques developed and the results obtained to date are of general interest.

C. B. Poland

Papers

A multiprocessing approach to a large computer system