Static scheduling of a program represented by a directed task graph on a multiprocessor system to minimize the program completion time is a well-known problem in parallel processing. Since finding an optimal schedule is an NP-complete problem in general, researchers have resorted to devising efficient heuristics. A plethora of heuristics have been proposed based on a wide spectrum of techniques, including branch-and-bound, integer-programming, searching, graph-theory, randomization, genetic algorithms, and evolutionary methods. The objective of this survey is to describe various scheduling algorithms and their functionalities in a contrasting fashion as well as examine their relative merits in terms of performance and time-complexity. Since these algorithms are based on diverse assumptions, they differ in their functionalities, and hence are difficult to describe in a unified context. We propose a taxonomy that classifies these algorithms into different categories. We consider 27 scheduling algorithms, with each algorithm explained through an easy-to-understand description followed by an illustrative example to demonstrate its operation. We also outline some of the novel and promising optimization approaches and current research trends in the area. Finally, we give an overview of the software tools that provide scheduling/mapping functionalities.

http://charm.cs.uiuc.edu/users/arya/docs/6.pdf

Static scheduling algorithms for allocating directed task graphs to multiprocessors

The performance of most digital systems today is limited by their communication or interconnection, not by their logic or memory. As designers strive to make more efficient use of scarce interconnection bandwidth, interconnection networks are emerging as a nearly universal solution to the system-level communication problems for modern digital systems. 


Interconnection networks have become pervasive in their traditional application as processor-memory and processor-processor interconnect. Point-to-point interconnection networks have replaced buses in an ever widening range of applications that include on-chip interconnect, switches and routers, and I/O systems. 


In this book, the authors present in a structured way the basic underlying concepts of most interconnection networks and provide representative solutions that have been implemented in the industry or proposed in the research literature.

* Gives a coherent, comprehensive treatment of the entire field
* Presents a formal statement of the basic concepts, alternative design choices, and design trade-offs
* Provides thorough classifications, clear descriptions, accurate definitions, and unified views to structure the knowledge on interconnection networks
* Focuses on issues critical to designers

Table of Contents

Foreword
Foreword to the First Printing
Preface
Chapter 1 - Introduction
Chapter 2 - Message Switching Layer
Chapter 3 - Deadlock, Livelock, and Starvation
Chapter 4 - Routing Algorithms
Chapter 5 - CollectiveCommunicationSupport
Chapter 6 - Fault-Tolerant Routing
Chapter 7 - Network Architectures
Chapter 8 - Messaging Layer Software
Chapter 9 - Performance Evaluation
Appendix A - Formal Definitions for Deadlock Avoidance
Appendix B - Acronyms
References
Index

Interconnection Networks

This paper describes capabilities, evolution, performance, and applications of the Global Arrays (GA) toolkit. GA was created to provide application programmers with an inteface that allows them to distribute data while maintaining the type of global index space and programming syntax similar to that available when programming on a single processor. The goal of GA is to free the programmer from the low level management of communication and allow them to deal with their problems at the level at which they were originally formulated. At the same time, compatibility of GA with MPI enables the programmer to take advatage of the existing MPI software/libraries when available and appropriate. The variety of applications that have been implemented using Global Arrays attests to the attractiveness of using higher level abstractions to write parallel code.

/pdf/advances-applications-and-performance-of-the-global-arrays-6j5g16c9x3.pdf

Advances, Applications and Performance of the Global Arrays Shared Memory Programming Toolkit

A global server load balancing (GSLB) switch serves as a proxy to an authoritative DNS communicates with numerous site switches which are coupled to host servers serving specific applications. The GSLB switch receives from site switches operational information regarding host servers within the site switches neighborhood. When a client program requests a resolution of a host name, the GSLB switch, acting as a proxy of an authoritative DNS, returns one or more ordered IP addresses for the host name. The IP addresses are ordered using metrics that include the information collected from the site switches. In one instance, the GSLB switch places the address that is deemed “best” at the top of the list.

Global server load balancing

Bandwidth optimal all-reduce algorithms for clusters of workstations

A reliable multicast algorithm, called RMA, for mobile ad hoc networks is presented that is based on a new cost criterion, called link lifetime, for determining the optimal path between a pair of nodes. The algorithm has the characteristics of using an undirected graph for its routing operations rather than a fixed structure like a tree or a mesh. Previously proposed routing metrics for mobile ad hoc networks were designed for use in wired environments, where link stability is not a concern. We propose a new metric, called the lifetime, which is more appropriate for mobile ad hoc networks. The lifetime metric is dependent on the predicted future life of the link under consideration. We developed a simulator for the mobile ad hoc networks, which is portable and scalable to a large number of nodes. Using the simulator, we carried out a simulation study to analyze the effectiveness of the routing metrics and the performance of the proposed reliable multicast algorithm. The simulation results show that the lifetime metric helps achieve better performance in mobile ad hoc environments than the hop count metric.

/pdf/a-reliable-multicast-algorithm-for-mobile-ad-hoc-networks-5e4s5enkqs.pdf

A reliable multicast algorithm for mobile ad hoc networks

Popular algorithms proposed in the literature for doing Barrier and Allreduce in clusters, such as pair-wise exchange, dissemination and gather-broadcast do not give an optimal performance when there is skew among the nodes in the cluster. In pair-wise exchange and dissemination, all the nodes must arrive for the completion of each step. The gather-broadcast algorithm assumes a fixed tree topology. We propose to use hardware multicast of InfiniBand in the design of an adaptive algorithm that performs well in the presence of skew. In this approach, the topology of the tree is not fixed but adapts depending on the skew. The last arriving node becomes the root of the tree if the skew is sufficiently large. We have carried out in-depth evaluation of our scheme and use synchronization delay as the performance metric for Barrier and Allreduce in the presence of skew. Our performance evaluation shows that our design scales very well with system size. Our designs can reduce the synchronization delay by a factor of 2.28 for Barrier and by a factor of 2.18 in the case of Allreduce. We have examined different skew scenarios and showed that the adaptive design performs either better or comparably to the existing schemes.

Efficient Barrier and Allreduce on Infiniband clusters using multicast and adaptive algorithms

Modern processors have multiple cores on a chip to overcome power consumption and heat dissipation issues. As more and more compute cores become available on a single node, it is expected that node-local communication will play an increasingly greater role in overall performance of parallel applications such as MPI applications. It is therefore crucial to optimize intra-node communication paths utilized by MPI libraries. In this paper, we propose a novel design of a kernel extension, called LiMIC2, for high-performance MPI intra-node communication over multi-core systems. LiMIC2 can minimize the communication overheads by implementing lightweight primitives and provide portability across different interconnects and flexibility for performance optimization. Our performance evaluation indicates that LiMIC2 can attain 80% lower latency and more than three times improvement in bandwidth. Also the experimental results show that LiMIC2 can deliver bidirectional bandwidth greater than 11GB/s.

Lightweight kernel-level primitives for high-performance MPI intra-node communication over multi-core systems

Current generation distributed shared memory (DSM) systems use point-to-point (unicast) messages for cache invalidations. This incurs a large number of control messages, heavy network traffic, and high occupancy at home nodes. This paper introduces a new approach to reduce these overheads by using multidestination-based reservation and gather worms for distributing invalidation requests and collecting acknowledgments. Different grouping schemes to generate multidestination worms on networks supporting deterministic (e-cube) or adaptive (turn-model) routing are investigated to implement the fully-mapped cache-coherence protocol. For different applications on a 2D mesh system, our simulation results indicate that up to 15% reduction in overall execution time can be achieved by using multidestination messages.

Reducing cache invalidation overheads in wormhole routed DSMs using multidestination message passing

The paper describes software architecture for supporting remote memory operations on clusters equipped with high-performance networks such as Myrinet and Giganet/Emulex cLAN. It presents protocols and strategies that bridge the gap between user-level API requirements and low-level network-specific interfaces such as GM and VIA. In particular, the issues of memory registration, management of network resources and memory consumption on the host, are discussed and solved to achieve an efficient implementation.

D.K. Panda

Papers

A reliable multicast algorithm for mobile ad hoc networks

Efficient Barrier and Allreduce on Infiniband clusters using multicast and adaptive algorithms

Lightweight kernel-level primitives for high-performance MPI intra-node communication over multi-core systems

Reducing cache invalidation overheads in wormhole routed DSMs using multidestination message passing

Protocols and strategies for optimizing performance of remote memory operations on clusters