The most exciting development in parallel computer architecture is the convergence of traditionally disparate approaches on a common machine structure. This book explains the forces behind this convergence of shared-memory, message-passing, data parallel, and data-driven computing architectures. It then examines the design issues that are critical to all parallel architecture across the full range of modern design, covering data access, communication performance, coordination of cooperative work, and correct implementation of useful semantics. It not only describes the hardware and software techniques for addressing each of these issues but also explores how these techniques interact in the same system. Examining architecture from an application-driven perspective, it provides comprehensive discussions of parallel programming for high performance and of workload-driven evaluation, based on understanding hardware-software interactions.


* synthesizes a decade of research and development for practicing engineers, graduate students, and researchers in parallel computer architecture, system software, and applications development

* presents in-depth application case studies from computer graphics, computational science and engineering, and data mining to demonstrate sound quantitative evaluation of design trade-offs 

* describes the process of programming for performance, including both the architecture-independent and architecture-dependent aspects, with examples and case-studies

* illustrates bus-based and network-based parallel systems with case studies of more than a dozen important commercial designs

Table of Contents

1 Introduction 
2 Parallel Programs 
3 Programming for Performance 
4 Workload-Driven Evaluation 
5 Shared Memory Multiprocessors 
6 Snoop-based Multiprocessor Design 
7 Scalable Multiprocessors
8 Directory-based Cache Coherence
9 Hardware-Software Tradeoffs 
10 Interconnection Network Design 
11 Latency Tolerance 
12 Future Directions 
APPENDIX A Parallel Benchmark Suites

Parallel Computer Architecture: A Hardware/Software Approach

An Internet access device (100) uses an automatic configuration process (600) to handle the task of configuring the Internet access device at a customer site for communication with the Internet (10). Once configured, the customer has electronic mail and other access to the Internet from his local area network. A not yet configured Internet access device is shipped directly to a customer without having to be manually configured first. The customer enters a registration identification number (326) and a telephone number onto the Internet access device. The Internet access device then automatically connects to the Internet, downloads configuration data from a configuration server (410) containing customer site specific configuration data, and then automatically configures itself for communication with the Internet. The Internet access device is simple to install for a customer and provides valuable features such as a router (240), firewall, e-mail gateway (212), web server (220), and other servers (222). The Internet access device initially connects to the Internet through an Internet service provider (14) over a standard analog telephone line using a standard modem (52) and using a dynamic IP address. Once automatically configured, the Internet access device may then communicate with the Internet using any suitable connection including an analog telephone line, or a higher-speed line such as an ISDN line or a frame relay circuit and is assigned a static IP address and a range of IP addresses for other devices on its local area network.

Automatic configuration for internet access device

Method and apparatus for establishing restricted broadcast groups in a switched network. The method assigns different virtual LAN identifiers (VLAN-IDs) to different subsets of associated end systems or access ports. Tables are maintained for mapping the VLAN-IDs with associated end systems and access ports. When a broadcast packet is received at a first switch, it is encapsulated with a VLAN header, including the VLAN-IDs, and sent out a multicast channel to all other switches in the network (domain). The original packet is sent out the other access ports of the receiving switch for the designated VLAN-IDs. The switches receiving the VLAN packet remove the header and send the original packet out access ports associated with the VLAN-IDs extracted from the header. The method provides a mechanism for forwarding broadcast packets of a protocol not supported by the switching mechanism, as well as multicast packets and unicast packets from undiscovered end systems.

Method for establishing restricted broadcast groups in a switched network

This paper examines the plausibility of using a network of workstations (NOW) for a mixture of parallel and sequential jobs. Through simulations, our study examines issues that arise when combining these two workloads on a single platform. Starting from a dedicated NOW just for parallel programs, we incrementally relax uniprogramming restrictions until we have a multi-programmed, multi-user NOW for both interactive sequential users and parallel programs. We show that a number of issues associated with the distributed NOW environment (e.g., daemon activity, coscheduling skew) can have a small but noticeable effect on parallel program performance. We also find that efficient migration to idle workstations is necessary to maintain acceptable parallel application performance. Furthermore, we present a methodology for deriving an optimal delay time for recruiting idle machines for use by parallel programs; this recruitment threshold was just 3 minutes for the research cluster we measured. Finally, we quantify the effects of the additional parallel load upon interactive users by keeping track of the potential number of user delays in our simulations. When we limit the maximum number of delays per user, we can still maintain acceptable parallel program performance. In summary, we find that for our workloads a 2:1 rule applies: a NOW cluster of approximately 60 machines can sustain a 32-node parallel workload in addition to the sequential load placed upon it by interactive users.

The interaction of parallel and sequential workloads on a network of workstations

Orca is a language for parallel programming on distributed systems. Communication in Orca is based on shared data-objects, which is a form of distributed shared memory. The performance of Orca programs depends strongly on how shared dataobjects are distributed among the local physical memories of the processors. This paper studies a new and efficient solution to this problem, based on an integration of compile-time and run-time techniques. The Orca compiler has been extended to determine the access patterns of processes to shared objects. The compiler passes a summary of this information to the run-time system, which uses it to make good decisions about which objects to replicate and where to store nonreplicated objects. Measurements show that the new system gives better overall performance than any previous implementation of Orca. 3333333333333333 1 This research was supported in part by a PIONIER grant from the Netherlands Organization for Scientific Research (N.W.O.). 2 This research was done while the author was at the Vrije Universiteit.

Object distribution in Orca using Compile-Time and Run-Time techniques

The adaptor cards and driver software for workstations and local asynchronous transfer mode (ATM) switches and switch control software used in an ATM local area network (LAN) system are discussed. It is shown that the ATM hardware and software components together provide services that are essential for ATM to be considered a realistic alternative to current shared-media LANs. These services include: completely transparent support for the TCP/IP protocol suite, an application programming interface for full access to the underlying ATM capabilities, support for AAL5, AAL3/4, and the null AAL, both connection-oriented and connectionless service, dynamic connection establishment or switched virtual circuits, resource reservation of guaranteed bandwidth and quality of service, full-bandwidth multicast and broadcast, virtual path and channel routing among multiple switches, automatic configuration and failure recovery, dynamic address assignment and internetwork address resolution, and network management via the simple network management protocol (SNMP). >

E. Cooper

Papers

Designing a practical ATM LAN