There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

One of the greatest challenges faced by designers of digital systems is optimizing the communication and interconnection between system components. Interconnection networks offer an attractive and economical solution to this communication crisis and are fast becoming pervasive in digital systems. Current trends suggest that this communication bottleneck will be even more problematic when designing future generations of machines. Consequently, the anatomy of an interconnection network router and science of interconnection network design will only grow in importance in the coming years.

This book offers a detailed and comprehensive presentation of the basic principles of interconnection network design, clearly illustrating them with numerous examples, chapter exercises, and case studies. It incorporates hardware-level descriptions of concepts, allowing a designer to see all the steps of the process from abstract design to concrete implementation.

·Case studies throughout the book draw on extensive author experience in designing interconnection networks over a period of more than twenty years, providing real world examples of what works, and what doesn't.

·Tightly couples concepts with implementation costs to facilitate a deeper understanding of the tradeoffs in the design of a practical network.

·A set of examples and exercises in every chapter help the reader to fully understand all the implications of every design decision.

Table of Contents


Chapter 1 Introduction to Interconnection Networks 
1.1 Three Questions About Interconnection Networks 
1.2 Uses of Interconnection Networks 
1.3 Network Basics 
1.4 History 
1.5 Organization of this Book 

Chapter 2 A Simple Interconnection Network 
2.1 Network Specifications and Constraints 
2.2 Topology 
2.3 Routing 
2.4 Flow Control 
2.5 Router Design 
2.6 Performance Analysis 
2.7 Exercises 

Chapter 3 Topology Basics 
3.1 Nomenclature 
3.2 Traffic Patterns 
3.3 Performance 
3.4 Packaging Cost 
3.5 Case Study: The SGI Origin 2000 
3.6 Bibliographic Notes 
3.7 Exercises 

Chapter 4 Butterfly Networks 
4.1 The Structure of Butterfly Networks 
4.2 Isomorphic Butterflies 
4.3 Performance and Packaging Cost 
4.4 Path Diversity and Extra Stages 
4.5 Case Study: The BBN Butterfly 
4.6 Bibliographic Notes 
4.7 Exercises 

Chapter 5 Torus Networks 
5.1 The Structure of Torus Networks 
5.2 Performance 
5.3 Building Mesh and Torus Networks 
5.4 Express Cubes 
5.5 Case Study: The MIT J-Machine 
5.6 Bibliographic Notes 
5.7 Exercises 
Chapter 6 Non-Blocking Networks 
6.1 Non-Blocking vs. Non-Interfering Networks 
6.2 Crossbar Networks 
6.3 Clos Networks 
6.4 Benes Networks 
6.5 Sorting Networks 
6.6 Case Study: The Velio VC2002 (Zeus) Grooming Switch 
6.7 Bibliographic Notes 
6.8 Exercises 

Chapter 7 Slicing and Dicing 
7.1 Concentrators and Distributors 
7.2 Slicing and Dicing 
7.3 Slicing Multistage Networks 
7.4 Case Study: Bit Slicing in the Tiny Tera 
7.5 Bibliographic Notes 
7.6 Exercises 

Chapter 8 Routing Basics 
8.1 A Routing Example 
8.2 Taxonomy of Routing Algorithms 
8.3 The Routing Relation 
8.4 Deterministic Routing 
8.5 Case Study: Dimension-Order Routing in the Cray T3D 
8.6 Bibliographic Notes 
8.7 Exercises 

Chapter 9 Oblivious Routing 
9.1 Valiant's Randomized Routing Algorithm 
9.2 Minimal Oblivious Routing 
9.3 Load-Balanced Oblivious Routing 
9.4 Analysis of Oblivious Routing 
9.5 Case Study: Oblivious Routing in the
Avici Terabit Switch Router(TSR) 
9.6 Bibliographic Notes 
9.7 Exercises 

Chapter 10 Adaptive Routing 
10.1 Adaptive Routing Basics 
10.2 Minimal Adaptive Routing 
10.3 Fully Adaptive Routing 
10.4 Load-Balanced Adaptive Routing 
10.5 Search-Based Routing 
10.6 Case Study: Adaptive Routing in the
Thinking Machines CM-5 
10.7 Bibliographic Notes 
10.8 Exercises 

Chapter 11 Routing Mechanics 
11.1 Table-Based Routing 
11.2 Algorithmic Routing 
11.3 Case Study: Oblivious Source Routing in the
IBM Vulcan Network 
11.4 Bibliographic Notes 
11.5 Exercises 

Chapter 12 Flow Control Basics 
12.1 Resources and Allocation Units 
12.2 Bufferless Flow Control 
12.3 Circuit Switching 
12.4 Bibliographic Notes 
12.5 Exercises 

Chapter 13 Buffered Flow Control 
13.1 Packet-Buffer Flow Control 
13.2 Flit-Buffer Flow Control 
13.3 Buffer Management and Backpressure 
13.4 Flit-Reservation Flow Control 
13.5 Bibliographic Notes 
13.6 Exercises 

Chapter 14 Deadlock and Livelock 
14.1 Deadlock 
14.2 Deadlock Avoidance 
14.3 Adaptive Routing 
14.4 Deadlock Recovery 
14.5 Livelock 
14.6 Case Study: Deadlock Avoidance in the Cray T3E 
14.7 Bibliographic Notes 
14.8 Exercises 

Chapter 15 Quality of Service 
15.1 Service Classes and Service Contracts 
15.2 Burstiness and Network Delays 
15.3 Implementation of Guaranteed Services 
15.4 Implementation of Best-Effort Services 
15.5 Separation of Resources 
15.6 Case Study: ATM Service Classes 
15.7 Case Study: Virtual Networks in the Avici TSR 
15.8 Bibliographic Notes 
15.9 Exercises 

Chapter 16 Router Architecture 
16.1 Basic Router Architecture 
16.2 Stalls 
16.3 Closing the Loop with Credits 
16.4 Reallocating a Channel 
16.5 Speculation and Lookahead 
16.6 Flit and Credit Encoding 
16.7 Case Study: The Alpha 21364 Router 
16.8 Bibliographic Notes 
16.9 Exercises 

Chapter 17 Router Datapath Components 
17.1 Input Buffer Organization 
17.2 Switches 
17.3 Output Organization 
17.4 Case Study: The Datapath of the IBM Colony
Router 
17.5 Bibliographic Notes 
17.6 Exercises 

Chapter 18 Arbitration 
18.1 Arbitration Timing 
18.2 Fairness 
18.3 Fixed Priority Arbiter 
18.4 Variable Priority Iterative Arbiters 
18.5 Matrix Arbiter 
18.6 Queuing Arbiter 
18.7 Exercises 

Chapter 19 Allocation 
19.1 Representations
19.2 Exact Algorithms
19.3 Separable Allocators 
19.4 Wavefront Allocator 
19.5 Incremental vs. Batch Allocation 
19.6 Multistage Allocation 
19.7 Performance of Allocators 
19.8 Case Study: The Tiny Tera Allocator 
19.9 Bibliographic Notes 
19.10 Exercises

Chapter 20 Network Interfaces 
20.1 Processor-Network Interface 
20.2 Shared-Memory Interface 
20.3 Line-Fabric Interface 
20.4 Case Study: The MIT M-Machine Network Interface 
20.5 Bibliographic Notes 
20.6 Exercises 

Chapter 21 Error Control 411
21.1 Know Thy Enemy: Failure Modes and Fault Models 
21.2 The Error Control Process: Detection, Containment,
and Recovery 
21.3 Link Level Error Control 
21.4 Router Error Control 
21.5 Network-Level Error Control 
21.6 End-to-end Error Control 
21.7 Bibliographic Notes 
21.8 Exercises 

Chapter 22 Buses 
22.1 Bus Basics 
22.2 Bus Arbitration 
22.3 High Performance Bus Protocol 
22.4 From Buses to Networks 
22.5 Case Study: The PCI Bus 
22.6 Bibliographic Notes 
22.7 Exercises 

Chapter 23 Performance Analysis 
23.1 Measures of Interconnection Network Performance 
23.2 Analysis 
23.3 Validation
23.4 Case Study: Efficiency and Loss in the
BBN Monarch Network 
23.5 Bibliographic Notes 
23.6 Exercises 

Chapter 24 Simulation 
24.1 Levels of Detail 
24.2 Network Workloads 
24.3 Simulation Measurements 
24.4 Simulator Design 
24.5 Bibliographic Notes 
24.6 Exercises 

Chapter 25 Simulation Examples 495
25.1 Routing
25.2 Flow Control Performance 
25.3 Fault Tolerance 

Appendix A Nomenclature 
Appendix B Glossary 
Appendix C Network Simulator

Principles and Practices of Interconnection Networks

A great variety of systems in nature, society and technology -- from the web of sexual contacts to the Internet, from the nervous system to power grids -- can be modeled as graphs of vertices coupled by edges The network structure, describing how the graph is wired, helps us understand, predict and optimize the behavior of dynamical systems In many cases, however, the edges are not continuously active As an example, in networks of communication via email, text messages, or phone calls, edges represent sequences of instantaneous or practically instantaneous contacts In some cases, edges are active for non-negligible periods of time: eg, the proximity patterns of inpatients at hospitals can be represented by a graph where an edge between two individuals is on throughout the time they are at the same ward Like network topology, the temporal structure of edge activations can affect dynamics of systems interacting through the network, from disease contagion on the network of patients to information diffusion over an e-mail network In this review, we present the emergent field of temporal networks, and discuss methods for analyzing topological and temporal structure and models for elucidating their relation to the behavior of dynamical systems In the light of traditional network theory, one can see this framework as moving the information of when things happen from the dynamical system on the network, to the network itself Since fundamental properties, such as the transitivity of edges, do not necessarily hold in temporal networks, many of these methods need to be quite different from those for static networks

Temporal Networks

This paper presents control and coordination algorithms for groups of vehicles. The focus is on autonomous vehicle networks performing distributed sensing tasks where each vehicle plays the role of a mobile tunable sensor. The paper proposes gradient descent algorithms for a class of utility functions which encode optimal coverage and sensing policies. The resulting closed-loop behavior is adaptive, distributed, asynchronous, and verifiably correct.

Coverage control for mobile sensing networks

Anarchy, state, and utopia

http://www.cs.otago.ac.nz/staffpriv/mike/Papers/BooleanMatrixMult/BooleanMatrix.pdf

A practical algorithm for Boolean matrix multiplication

Consider a tree network that has been contaminated by a persistent and active virus: when infected, a network site will continuously attempt to spread the virus to all its neighbours. The decontamination problem is that of disinfecting the entire network using a team of mobile antiviral system agents, called cleaners, avoiding any recontamination of decontaminated areas. A cleaner is able to decontaminate any infected node it visits; once the cleaner departs, the decontaminated node is immune for t ≥ 0 time units to viral attacks from infected neighbours. After the immunity time t is elapsed, re-contamination can occur. The primary research objective is to determine the minimum team size, as well as the solution strategy, that is the protocol that would enable such a minimal team of cleaners to perform the task. The network decontamination problem has been extensively investigated in the literature, and a very large number of studies exist on the subject. However, all the existing work is limited to the special case t = 0. In this paper we examine the tree decontamination problem for any value t ≥ 0. We determine the minimum team size necessary to disinfect any given tree with immunity time t. Further we show how to compute for all nodes of the tree the minimum team size and implicitly the solution strategy starting from each starting node; these computations use a total of θ(n) time (serially) or θ(n) messages (distributively). We then provide a complete structural characterization of the class of trees that can be decontaminated with k agents and immunity time t; we do so by identifying the forbidden subgraphs and analyzing their properties. Finally, we consider generic decontamination algorithms, i.e. protocols that work unchanged in a large class of trees, with little knowledge of their topological structure. We prove that, for each immunity time t ≥ 0, all trees of height at most h can be decontaminated by a team of $k=\lfloor {{2 h} \over t+2 }\rfloor$ agents whose only knowledge of the tree is the bound h. The proof is constructive.

/pdf/tree-decontamination-with-temporary-immunity-c4zr9eco4q.pdf

Tree Decontamination with Temporary Immunity

A black hole is a highly harmful host that disposes of visiting agents upon their arrival without any observable trace of the destruction. The problem of locating the black hole in asynchronous ring network is known to be solvable by a team of mobile agents if each node is equipped with a whiteboard. A simpler and less expensive inter-communication and synchronization mechanism is provided by tokens: each agent has available a bounded number of tokens that can be carried, placed in a node or/and on a port of the node, or removed. All tokens are identical and no other form of communication or coordination is available to the agents. It is known that locating the black hole in an anonymous ring network using tokens is feasible when the team of agents is initially collocated (i.e. they all start from the same host). Recently, the more difficult case when the agents are scattered (i.e., when the agents do not start from the same host) has also been examined and solutions requiring only O(1) tokens per agent but using a total of O(n2) moves have been presented. The number of moves can be reduced to O(kn + n log n) if the number k of agents is known. In this paper, we study the impact of orientation and knowledge of team size on the cost of black hole location by scattered agents with tokens. We prove that, in oriented rings, the number of moves can be reduced from O(n2) to the optimal Theta(nlogn) using only O(1) tokens per agent, without any knowledge of the team size. This result holds even if both agents and nodes are anonymous. Interestingly, the proposed algorithm solves, with the same cost, also the leader election problem and the rendezvous problem for the scattered agents despite the presence of a BH.

/pdf/scattered-black-hole-search-in-an-oriented-ring-using-tokens-2h4rxt0sns.pdf

Scattered Black Hole Search in an Oriented Ring using Tokens

In mobile sensor networks, the coverage improvement problem, i.e., maximizing and/or maintaining overall sensing coverage, is a fundamental research issue attracting many researchers. Existing coverage improvement algorithms such as sensor self-deployment algorithms and sensor relocation protocols enhance coverage with limitations due to their specialized design purposes. In this paper, we propose an integrated self-deployment and coverage maintenance scheme, which solves the coverage improvement problem in a complete sense. The proposed scheme is an integration of four algorithms: a node redundancy determination algorithm, a sensor self-deployment algorithm, a sensor relocation protocol, and a sensor replenishment protocol. By this scheme, redundant sensors are placed together with non-redundant ones in the target field at random; non-redundant sensors autonomously scatter to form a network with maximal coverage after initial placement; all the sensors collaborate to compensate coverage loss throughout network lifetime. Mentionably, we notice that no existing scheme besides ours take into account the impact on coverage from nodal sensing range diminishment. At the end, we summarize the paper and discuss our future work.

/pdf/an-integrated-self-deployment-and-coverage-maintenance-k7l65i0tzl.pdf

An Integrated Self-deployment and Coverage Maintenance Scheme for Mobile Sensor Networks

In the graph exploration problem, a team of mobile computational entities, called agents, arbitrarily positioned at some nodes of a graph, must cooperate so that each node is eventually visited by at least one agent. In the literature, the main focus has been on graphs that are static; that is, the topology is either invariant in time or subject to localized changes. The few studies on exploration of dynamic graphs have been almost all limited to the centralized case (i.e., assuming complete a priori knowledge of the changes and the times of their occurrence). We investigate the decentralized exploration of dynamic graphs (i.e., when the agents are unaware of the location and timing of the changes) focusing, in this paper, on dynamic systems whose underlying graph is a ring. We first consider the fully-synchronous systems traditionally assumed in the literature; i.e., all agents are active at each time step. We then introduce the notion of semi-synchronous systems, where only a subset of agents might be active at each time step (the choice of the subset is made by an adversary); this model is common in the context of mobile agents in continuous spaces but has never been studied before for agents moving in graphs. Our main focus is on the impact that the level of synchrony as well as other factors such as anonymity, knowledge of the size of the ring, and chirality (i.e., common orientation) have on the solvability of the problem, focusing on the minimum number of agents necessary. We draw an extensive map of feasibility, and of complexity in terms of minimum number of agent movements. All our sufficiency proofs are constructive, and almost all our solution protocols are asymptotically optimal.

Nicola Santoro

Papers

A practical algorithm for Boolean matrix multiplication

Tree Decontamination with Temporary Immunity

Scattered Black Hole Search in an Oriented Ring using Tokens

An Integrated Self-deployment and Coverage Maintenance Scheme for Mobile Sensor Networks

Live Exploration of Dynamic Rings