One of the greatest challenges faced by designers of digital systems is optimizing the communication and interconnection between system components. Interconnection networks offer an attractive and economical solution to this communication crisis and are fast becoming pervasive in digital systems. Current trends suggest that this communication bottleneck will be even more problematic when designing future generations of machines. Consequently, the anatomy of an interconnection network router and science of interconnection network design will only grow in importance in the coming years.

This book offers a detailed and comprehensive presentation of the basic principles of interconnection network design, clearly illustrating them with numerous examples, chapter exercises, and case studies. It incorporates hardware-level descriptions of concepts, allowing a designer to see all the steps of the process from abstract design to concrete implementation.

·Case studies throughout the book draw on extensive author experience in designing interconnection networks over a period of more than twenty years, providing real world examples of what works, and what doesn't.

·Tightly couples concepts with implementation costs to facilitate a deeper understanding of the tradeoffs in the design of a practical network.

·A set of examples and exercises in every chapter help the reader to fully understand all the implications of every design decision.

Table of Contents


Chapter 1 Introduction to Interconnection Networks 
1.1 Three Questions About Interconnection Networks 
1.2 Uses of Interconnection Networks 
1.3 Network Basics 
1.4 History 
1.5 Organization of this Book 

Chapter 2 A Simple Interconnection Network 
2.1 Network Specifications and Constraints 
2.2 Topology 
2.3 Routing 
2.4 Flow Control 
2.5 Router Design 
2.6 Performance Analysis 
2.7 Exercises 

Chapter 3 Topology Basics 
3.1 Nomenclature 
3.2 Traffic Patterns 
3.3 Performance 
3.4 Packaging Cost 
3.5 Case Study: The SGI Origin 2000 
3.6 Bibliographic Notes 
3.7 Exercises 

Chapter 4 Butterfly Networks 
4.1 The Structure of Butterfly Networks 
4.2 Isomorphic Butterflies 
4.3 Performance and Packaging Cost 
4.4 Path Diversity and Extra Stages 
4.5 Case Study: The BBN Butterfly 
4.6 Bibliographic Notes 
4.7 Exercises 

Chapter 5 Torus Networks 
5.1 The Structure of Torus Networks 
5.2 Performance 
5.3 Building Mesh and Torus Networks 
5.4 Express Cubes 
5.5 Case Study: The MIT J-Machine 
5.6 Bibliographic Notes 
5.7 Exercises 
Chapter 6 Non-Blocking Networks 
6.1 Non-Blocking vs. Non-Interfering Networks 
6.2 Crossbar Networks 
6.3 Clos Networks 
6.4 Benes Networks 
6.5 Sorting Networks 
6.6 Case Study: The Velio VC2002 (Zeus) Grooming Switch 
6.7 Bibliographic Notes 
6.8 Exercises 

Chapter 7 Slicing and Dicing 
7.1 Concentrators and Distributors 
7.2 Slicing and Dicing 
7.3 Slicing Multistage Networks 
7.4 Case Study: Bit Slicing in the Tiny Tera 
7.5 Bibliographic Notes 
7.6 Exercises 

Chapter 8 Routing Basics 
8.1 A Routing Example 
8.2 Taxonomy of Routing Algorithms 
8.3 The Routing Relation 
8.4 Deterministic Routing 
8.5 Case Study: Dimension-Order Routing in the Cray T3D 
8.6 Bibliographic Notes 
8.7 Exercises 

Chapter 9 Oblivious Routing 
9.1 Valiant's Randomized Routing Algorithm 
9.2 Minimal Oblivious Routing 
9.3 Load-Balanced Oblivious Routing 
9.4 Analysis of Oblivious Routing 
9.5 Case Study: Oblivious Routing in the
Avici Terabit Switch Router(TSR) 
9.6 Bibliographic Notes 
9.7 Exercises 

Chapter 10 Adaptive Routing 
10.1 Adaptive Routing Basics 
10.2 Minimal Adaptive Routing 
10.3 Fully Adaptive Routing 
10.4 Load-Balanced Adaptive Routing 
10.5 Search-Based Routing 
10.6 Case Study: Adaptive Routing in the
Thinking Machines CM-5 
10.7 Bibliographic Notes 
10.8 Exercises 

Chapter 11 Routing Mechanics 
11.1 Table-Based Routing 
11.2 Algorithmic Routing 
11.3 Case Study: Oblivious Source Routing in the
IBM Vulcan Network 
11.4 Bibliographic Notes 
11.5 Exercises 

Chapter 12 Flow Control Basics 
12.1 Resources and Allocation Units 
12.2 Bufferless Flow Control 
12.3 Circuit Switching 
12.4 Bibliographic Notes 
12.5 Exercises 

Chapter 13 Buffered Flow Control 
13.1 Packet-Buffer Flow Control 
13.2 Flit-Buffer Flow Control 
13.3 Buffer Management and Backpressure 
13.4 Flit-Reservation Flow Control 
13.5 Bibliographic Notes 
13.6 Exercises 

Chapter 14 Deadlock and Livelock 
14.1 Deadlock 
14.2 Deadlock Avoidance 
14.3 Adaptive Routing 
14.4 Deadlock Recovery 
14.5 Livelock 
14.6 Case Study: Deadlock Avoidance in the Cray T3E 
14.7 Bibliographic Notes 
14.8 Exercises 

Chapter 15 Quality of Service 
15.1 Service Classes and Service Contracts 
15.2 Burstiness and Network Delays 
15.3 Implementation of Guaranteed Services 
15.4 Implementation of Best-Effort Services 
15.5 Separation of Resources 
15.6 Case Study: ATM Service Classes 
15.7 Case Study: Virtual Networks in the Avici TSR 
15.8 Bibliographic Notes 
15.9 Exercises 

Chapter 16 Router Architecture 
16.1 Basic Router Architecture 
16.2 Stalls 
16.3 Closing the Loop with Credits 
16.4 Reallocating a Channel 
16.5 Speculation and Lookahead 
16.6 Flit and Credit Encoding 
16.7 Case Study: The Alpha 21364 Router 
16.8 Bibliographic Notes 
16.9 Exercises 

Chapter 17 Router Datapath Components 
17.1 Input Buffer Organization 
17.2 Switches 
17.3 Output Organization 
17.4 Case Study: The Datapath of the IBM Colony
Router 
17.5 Bibliographic Notes 
17.6 Exercises 

Chapter 18 Arbitration 
18.1 Arbitration Timing 
18.2 Fairness 
18.3 Fixed Priority Arbiter 
18.4 Variable Priority Iterative Arbiters 
18.5 Matrix Arbiter 
18.6 Queuing Arbiter 
18.7 Exercises 

Chapter 19 Allocation 
19.1 Representations
19.2 Exact Algorithms
19.3 Separable Allocators 
19.4 Wavefront Allocator 
19.5 Incremental vs. Batch Allocation 
19.6 Multistage Allocation 
19.7 Performance of Allocators 
19.8 Case Study: The Tiny Tera Allocator 
19.9 Bibliographic Notes 
19.10 Exercises

Chapter 20 Network Interfaces 
20.1 Processor-Network Interface 
20.2 Shared-Memory Interface 
20.3 Line-Fabric Interface 
20.4 Case Study: The MIT M-Machine Network Interface 
20.5 Bibliographic Notes 
20.6 Exercises 

Chapter 21 Error Control 411
21.1 Know Thy Enemy: Failure Modes and Fault Models 
21.2 The Error Control Process: Detection, Containment,
and Recovery 
21.3 Link Level Error Control 
21.4 Router Error Control 
21.5 Network-Level Error Control 
21.6 End-to-end Error Control 
21.7 Bibliographic Notes 
21.8 Exercises 

Chapter 22 Buses 
22.1 Bus Basics 
22.2 Bus Arbitration 
22.3 High Performance Bus Protocol 
22.4 From Buses to Networks 
22.5 Case Study: The PCI Bus 
22.6 Bibliographic Notes 
22.7 Exercises 

Chapter 23 Performance Analysis 
23.1 Measures of Interconnection Network Performance 
23.2 Analysis 
23.3 Validation
23.4 Case Study: Efficiency and Loss in the
BBN Monarch Network 
23.5 Bibliographic Notes 
23.6 Exercises 

Chapter 24 Simulation 
24.1 Levels of Detail 
24.2 Network Workloads 
24.3 Simulation Measurements 
24.4 Simulator Design 
24.5 Bibliographic Notes 
24.6 Exercises 

Chapter 25 Simulation Examples 495
25.1 Routing
25.2 Flow Control Performance 
25.3 Fault Tolerance 

Appendix A Nomenclature 
Appendix B Glossary 
Appendix C Network Simulator

Principles and Practices of Interconnection Networks

Author(s): Asanovic, K; Bodik, R; Catanzaro, B; Gebis, J; Husbands, P; Keutzer, K; Patterson, D; Plishker, W; Shalf, J; Williams, SW | Abstract: The recent switch to parallel microprocessors is a milestone in the history of computing. Industry has laid out a roadmap for multicore designs that preserves the programming paradigm of the past via binary compatibility and cache coherence. Conventional wisdom is now to double the number of cores on a chip with each silicon generation. A multidisciplinary group of Berkeley researchers met nearly two years to discuss this change. Our view is that this evolutionary approach to parallel hardware and software may work from 2 or 8 processor systems, but is likely to face diminishing returns as 16 and 32 processor systems are realized, just as returns fell with greater instruction-level parallelism. We believe that much can be learned by examining the success of parallelism at the extremes of the computing spectrum, namely embedded computing and high performance computing. This led us to frame the parallel landscape with seven questions, and to recommend the following: • The overarching goal should be to make it easy to write programs that execute efficiently on highly parallel computing systems • The target should be 1000s of cores per chip, as these chips are built from processing elements that are the most efficient in MIPS (Million Instructions per Second) per watt, MIPS per area of silicon, and MIPS per development dollar. • Instead of traditional benchmarks, use 13 “Dwarfs” to design and evaluate parallel programming models and architectures. (A dwarf is an algorithmic method that captures a pattern of computation and communication.) • “Autotuners” should play a larger role than conventional compilers in translating parallel programs. • To maximize programmer productivity, future programming models must be more human-centric than the conventional focus on hardware or applications. • To be successful, programming models should be independent of the number of processors. • To maximize application efficiency, programming models should support a wide range of data types and successful models of parallelism: task-level parallelism, word-level parallelism, and bit-level parallelism. 1 The Landscape of Parallel Computing Research: A View From Berkeley • Architects should not include features that significantly affect performance or energy if programmers cannot accurately measure their impact via performance counters and energy counters. • Traditional operating systems will be deconstructed and operating system functionality will be orchestrated using libraries and virtual machines. • To explore the design space rapidly, use system emulators based on Field Programmable Gate Arrays (FPGAs) that are highly scalable and low cost. Since real world applications are naturally parallel and hardware is naturally parallel, what we need is a programming model, system software, and a supporting architecture that are naturally parallel. Researchers have the rare opportunity to re-invent these cornerstones of computing, provided they simplify the efficient programming of highly parallel systems.

/pdf/the-landscape-of-parallel-computing-research-a-view-from-1i5pswggnf.pdf

The Landscape of Parallel Computing Research: A View from Berkeley

The most exciting development in parallel computer architecture is the convergence of traditionally disparate approaches on a common machine structure. This book explains the forces behind this convergence of shared-memory, message-passing, data parallel, and data-driven computing architectures. It then examines the design issues that are critical to all parallel architecture across the full range of modern design, covering data access, communication performance, coordination of cooperative work, and correct implementation of useful semantics. It not only describes the hardware and software techniques for addressing each of these issues but also explores how these techniques interact in the same system. Examining architecture from an application-driven perspective, it provides comprehensive discussions of parallel programming for high performance and of workload-driven evaluation, based on understanding hardware-software interactions.


* synthesizes a decade of research and development for practicing engineers, graduate students, and researchers in parallel computer architecture, system software, and applications development

* presents in-depth application case studies from computer graphics, computational science and engineering, and data mining to demonstrate sound quantitative evaluation of design trade-offs 

* describes the process of programming for performance, including both the architecture-independent and architecture-dependent aspects, with examples and case-studies

* illustrates bus-based and network-based parallel systems with case studies of more than a dozen important commercial designs

Table of Contents

1 Introduction 
2 Parallel Programs 
3 Programming for Performance 
4 Workload-Driven Evaluation 
5 Shared Memory Multiprocessors 
6 Snoop-based Multiprocessor Design 
7 Scalable Multiprocessors
8 Directory-based Cache Coherence
9 Hardware-Software Tradeoffs 
10 Interconnection Network Design 
11 Latency Tolerance 
12 Future Directions 
APPENDIX A Parallel Benchmark Suites

Parallel Computer Architecture: A Hardware/Software Approach

The Wisconsin Multifacet Project has created a simulation toolset to characterize and evaluate the performance of multiprocessor hardware systems commonly used as database and web servers. We leverage an existing full-system functional simulation infrastructure (Simics [14]) as the basis around which to build a set of timing simulator modules for modeling the timing of the memory system and microprocessors. This simulator infrastructure enables us to run architectural experiments using a suite of scaled-down commercial workloads [3]. To enable other researchers to more easily perform such research, we have released these timing simulator modules as the Multifacet General Execution-driven Multiprocessor Simulator (GEMS) Toolset, release 1.0, under GNU GPL [9].

/pdf/multifacet-s-general-execution-driven-multiprocessor-18gajjgh1d.pdf

Multifacet's general execution-driven multiprocessor simulator (GEMS) toolset

This paper examines the effect of technology scaling and microarchitectural trends on the rate of soft errors in CMOS memory and logic circuits. We describe and validate an end-to-end model that enables us to compute the soft error rates (SER) for existing and future microprocessor-style designs. The model captures the effects of two important masking phenomena, electrical masking and latching-window masking, which inhibit soft errors in combinational logic. We quantify the SER due to high-energy neutrons in SRAM cells, latches, and logic circuits for feature sizes from 600 nm to 50 nm and clock periods from 16 to 6 fan-out-of-4 inverter delays. Our model predicts that the SER per chip of logic circuits will increase nine orders of magnitude from 1992 to 2011 and at that point will be comparable to the SER per chip of unprotected memory elements. Our result emphasizes that computer system designers must address the risks of soft errors in logic circuits for future designs.

/pdf/modeling-the-effect-of-technology-trends-on-the-soft-error-7cmde8sa08.pdf

Modeling the effect of technology trends on the soft error rate of combinational logic

Single-event upsets from particle strikes have become a key challenge in microprocessor design. Techniques to deal with these transients faults exist, but come at a cost. Designers clearly require accurate estimates of processor error rates to make appropriate cost/reliability tradeoffs. This paper describes a method for generating these estimates. A key aspect of this analysis is that some single-bit faults (such as those occurring in the branch predictor) do not produce an error in a program's output. We define a structure's architectural vulnerability factor (AVF) as the probability that a fault in that particular structure do not result in an error. A structure's error rate is the product of its raw error rate, as determined by process and circuit technology, and the AVF. Unfortunately, computing AVFs of complex structures, such as the instruction queue, can be quite involved. We identify numerous cases, such as prefetches, dynamically dead code, and wrong-path instructions, in which a fault do not affect, correct execution. We instrument a detailed 1A64 processor simulator to map bit-level microarchitectural state to these cases, generating per-structure AVF estimates. This analysis shows AVFs of 28% and 9% for the instruction queue and execution units, respectively, averaged across dynamic sections of the entire CPU2000 benchmark suite.

/pdf/a-systematic-methodology-to-compute-the-architectural-gnuelfzd79.pdf

A systematic methodology to compute the architectural vulnerability factors for a high-performance microprocessor

Smaller feature sizes, reduced voltage levels, higher transistor counts, and reduced noise margins make future generations of microprocessors increasingly prone to transient hardware faults. Most commercial fault-tolerant computers use fully replicated hardware components to detect microprocessor faults. The components are lockstepped (cycle-by-cycle synchronized) to ensure that, in each cycle, they perform the same operation on the same inputs, producing the same outputs in the absence of faults. Unfortunately, for a given hardware budget, full replication reduces performance by statically partitioning resources among redundant operations.We demonstrate that a Simultaneous and Redundantly Threaded (SRT) processor—derived from a Simultaneous Multithreaded (SMT) processor—provides transient fault coverage with significantly higher performance. An SRT processor provides transient fault coverage by running identical copies of the same program simultaneously as independent threads. An SRT processor provides higher performance because it dynamically schedules its hardware resources among the redundant copies. However, dynamic scheduling makes it difficult to implement lockstepping, because corresponding instructions from redundant threads may not execute in the same cycle or in the same order.This paper makes four contributions to the design of SRT processors. First, we introduce the concept of the sphere of replication, which abstracts both the physical redundancy of a lockstepped system and the logical redundancy of an SRT processor. This framework aids in identifying the scope of fault coverage and the input and output values requiring special handling. Second, we identify two viable spheres of replication in an SRT processor, and show that one of them provides fault detection while checking only committed stores and uncached loads. Third, we identify the need for consistent replication of load values, and propose and evaluate two new mechanisms for satisfying this requirement. Finally, we propose and evaluate two mechanisms—slack fetch and branch outcome queue—that enhance the performance of an SRT processor by allowing one thread to prefetch cache misses and branch results for the other thread. Our results with 11 SPEC95 benchmarks show that an SRT processor can outperform an equivalently sized, on-chip, hardware-replicated solution by 16% on average, with a maximum benefit of up to 29%.

Transient fault detection via simultaneous multithreading

Exponential growth in the number of on-chip transistors, coupled with reductions in voltage levels, makes each generation of microprocessors increasingly vulnerable to transient faults. In a multithreaded environment, we can detect these faults by running two copies of the same program as separate threads, feeding them identical inputs, and comparing their outputs, a technique we call Redundant Multithreading (RMT).This paper studies RMT techniques in the context of both single- and dual-processor simultaneous multithreaded (SMT) single-chip devices. Using a detailed, commercial-grade, SMT processor design we uncover subtle RMT implementation complexities, and find that RMT can be a more significant burden for single-processor devices than prior studies indicate. However, a novel application of RMT techniques in a dual-processor device, which we term chip-level redundant threading (CRT), shows higher performance than lockstepping the two cores, especially on multithreaded workloads.

https://csserver.evansville.edu/~mr56/cs838/Paper20.pdf

Detailed design and evaluation of redundant multithreading alternatives

Radiation-induced soft errors have emerged as a key challenge in computer system design. If the industry is to continue to provide customers with the level of reliability they expect, microprocessor architects must address this challenge directly. This effort has two parts. First, architects must understand the impact of soft errors on their designs. Second, they must select judiciously from among available techniques to reduce this impact in order to meet their reliability targets with minimum overhead. To provide a foundation for these efforts, this paper gives a broad overview of the soft error problem from an architectural perspective. We start with basic definitions, followed by a description of techniques to compute the soft error rate. Then, we summarize techniques used to reduce the soft error rate. This paper also describes problems with double-bit errors. Finally, this paper outlines future directions for architecture research in soft errors.

/pdf/the-soft-error-problem-an-architectural-perspective-5dqyw2nwhh.pdf

The soft error problem: an architectural perspective

Transient faults due to neutron and alpha particle strikes posea significant obstacle to increasing processor transistor counts infuture technologies. Although fault rates of individual transistorsmay not rise significantly, incorporating more transistors into adevice makes that device more likely to encounter a fault. Hence,maintaining processor error rates at acceptable levels will requireincreasing design effort.This paper proposes two simple approaches to reduce errorrates and evaluates their application to a microprocessor instructionqueue. The first technique reduces the time instructions sit invulnerable storage structures by selectively squashing instructionswhen long delays are encountered. A fault is less likely to cause anerror if the structure it affects does not contain valid instructions.We introduce a new metric, MITF (Mean Instructions To Failure),to capture the trade-off between performance and reliability introducedby this approach.The second technique addresses false detected errors. In theabsence of a fault detection mechanism, such errors would nothave affected the final outcome of a program. For example, a faultaffecting the result of a dynamically dead instruction would notchange the final program output, but could still be flagged by thehardware as an error. To avoid signalling such false errors, wemodify a pipeline's error detection logic to mark affected instructionsand data as possibly incorrect rather than immediately signalingan error. Then, we signal an error only if we determine laterthat the possibly incorrect value could have affected the program'soutput.

/pdf/techniques-to-reduce-the-soft-error-rate-of-a-high-2dixq22522.pdf

Shubhendu S. Mukherjee

Papers

A systematic methodology to compute the architectural vulnerability factors for a high-performance microprocessor

Transient fault detection via simultaneous multithreading

Detailed design and evaluation of redundant multithreading alternatives

The soft error problem: an architectural perspective

Techniques to Reduce the Soft Error Rate of a High-Performance Microprocessor