The theoretical background for the design of deadlock-free adaptive routing algorithms for wormhole networks is developed. The author proposes some basic definitions and two theorems. These create the conditions to verify that an adaptive algorithm is deadlock-free, even when there are cycles in the channel dependency graph. Two design methodologies are also proposed. The first supplies algorithms with a high degree of freedom, without increasing the number of physical channels. The second methodology is intended for the design of fault-tolerant algorithms. Some examples are given to show the application of the methodologies. Simulations show the performance improvement that can be achieved by designing the routing algorithms with the new theory. >

A new theory of deadlock-free adaptive routing in wormhole networks

The performance of most digital systems today is limited by their communication or interconnection, not by their logic or memory. As designers strive to make more efficient use of scarce interconnection bandwidth, interconnection networks are emerging as a nearly universal solution to the system-level communication problems for modern digital systems. 


Interconnection networks have become pervasive in their traditional application as processor-memory and processor-processor interconnect. Point-to-point interconnection networks have replaced buses in an ever widening range of applications that include on-chip interconnect, switches and routers, and I/O systems. 


In this book, the authors present in a structured way the basic underlying concepts of most interconnection networks and provide representative solutions that have been implemented in the industry or proposed in the research literature.

* Gives a coherent, comprehensive treatment of the entire field
* Presents a formal statement of the basic concepts, alternative design choices, and design trade-offs
* Provides thorough classifications, clear descriptions, accurate definitions, and unified views to structure the knowledge on interconnection networks
* Focuses on issues critical to designers

Table of Contents

Foreword
Foreword to the First Printing
Preface
Chapter 1 - Introduction
Chapter 2 - Message Switching Layer
Chapter 3 - Deadlock, Livelock, and Starvation
Chapter 4 - Routing Algorithms
Chapter 5 - CollectiveCommunicationSupport
Chapter 6 - Fault-Tolerant Routing
Chapter 7 - Network Architectures
Chapter 8 - Messaging Layer Software
Chapter 9 - Performance Evaluation
Appendix A - Formal Definitions for Deadlock Avoidance
Appendix B - Acronyms
References
Index

Interconnection Networks

This paper describes the interconnection network used in the Cray T3E multiprocessor. The network is a bidirectional 3D torus with fully adaptive routing, optimized virtual channel assignments, integrated barrier synchronization support and considerable fault tolerance. The routers are built with LSI’s 500K ASIC technology with custom transmitters/ receivers driving low-voltage differential signals at 375 MHz, for a link data payload capacity of approximately 500 MB/s.

The Cray T3E Network: Adaptive Routing in a High Performance 3D Torus

Deadlock avoidance is a key issue in wormhole networks. A first approach by W.J. Dally and C.L. Seitz (1987) consists of removing the cyclic dependencies between channels. Many deterministic and adaptive routing algorithms have been proposed based on that approach. Although the absence of cyclic dependencies is a necessary and sufficient condition for deadlock-free deterministic routing, it is only a sufficient condition for deadlock-free adaptive routing. A more powerful approach by J. Duato (1991) only requires the absence of cyclic dependencies on a connected channel subset. The remaining channels can be used in almost any way. In this paper, we show that the previously mentioned approach is also a sufficient condition. Moreover, we propose a necessary and sufficient condition for deadlock-free adaptive routing. This condition is the key for the design of fully adaptive routing algorithms with minimum restrictions, An example shows the application of the new theory. >

A necessary and sufficient condition for deadlock-free adaptive routing in wormhole networks

Multicast communication, in which the same message is delivered from a source node to an arbitrary number of destination nodes, is being increasingly demanded in parallel computing. System supported multicast services can potentially offer improved performance, increased functionality, and simplified programming, and may in turn be used to support various higher-level operations for data movement and global process control. This paper presents efficient algorithms to implement multicast communication in wormhole-routed direct networks, in the absence of hardware multicast support, by exploiting the properties of the switching technology. Minimum-time multicast algorithms are presented for n-dimensional meshes and hypercubes that use deterministic, dimension-ordered routing of unicast messages. Both algorithms can deliver a multicast message to m-1 destinations in [log/sub 2/ m] message passing steps, while avoiding contention among the constituent unicast messages. Performance results of implementations on a 64-node nCUBE-2 hypercube and a 168-node Symult 2010 2-D mesh are given. >

Unicast-based multicast communication in wormhole-routed networks

In order to provide an arbitrary and fully dynamic connectivity in a network of processors, transport mechanisms must be implemented, which provide the propagation of data from processor to processor, based on addresses contained within a packet of data. Such data transport mechanisms must satisfy a number of requirements - deadlock and livelock freedom, good hot-spot performance, high throughput and low latency. This paper proposes a solution to these problems, which allows deadlock free, adaptive, high throughput packet routing to be implemented on networks of processors. Examples are given which illustrate the technique for 2-D array and toroidal networks. An implementation of this scheme on arrays of transputers is described. The scheme also serves as a basis for a very low latency routing strategy named the mad postman, a detailed implementation of which is described here as well.

P. R. Miller

Papers

High Performance Communications In Processor Networks

The mad-postman network chip