There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Communications of the ACM (CACM for short, not the best sounding acronym around) is the ACM’s flagship magazine. Started in 1957, CACM is handy for keeping up to date on current research being carried out across all topics of computer science and realworld applications. CACM has had an illustrious past with many influential pieces of work and debates started within its pages. These include Hoare’s presentation of the Quicksort algorithm; Rivest, Shamir and Adleman’s description of the first publickey cryptosystem RSA; and Dijkstra’s famous letter against the use of GOTO. In addition to the print edition, which is released monthly, there is a fantastic website (http://cacm.acm. org/) that showcases not only the most recent edition but all previous CACM articles as well, readable online as well as downloadable as a PDF. In addition, the website lets you browse for articles by subject, a handy feature if you want to focus on a particular topic. CACM is really essential reading. Pretty much guaranteed to contain content that is interesting to anyone, it keeps tabs on the latest in computer science. It is a valuable asset for us students, who tend to delve deep into a particular area of CS and forget everything that is happening around us. — Daniel Gooch U ndergraduate research is like a box of chocolates: You never know what kind of project you will get. That being said, there are still a few things you should know to get the most out of the experience.

Communications of the ACM

This book makes a clear presentation of the traditional topics included in a course of undergraduate parallel programming. As explained by the authors, it was developed from their own experience in classrooms, introducing their students to parallel programming. It can be used almost directly to teach basic parallel programming.

/pdf/parallel-programming-techniques-and-applications-using-1g2pyrr6g4.pdf

Parallel programming: techniques and applications using networked workstations and parallel computers. Barry Wilkinson, C. Michael Allen

Kai Hwang and Zhiwei Xu  McGraw-Hill, Boston, 1998, 802 pp. ISBN 0-07-031798-4, $97.30 This text is an in depth introduction to the concepts of Parallel Computing. Designed for use in university level computer science courses, the text covers scalable architecture and parallel programming of symmetric muli-processors, clusters of workstations, massively parallel processors, and Internet-based metacomputing platforms. Hwang and Xu give an excellent overview in these topics while keeping the text easily comprehensible. The text is organized into four parts. Part I covers scalability and clustering. Part II deals with the technology used to construct a parallel system. Part III pertains to the architecture of scalable systems. Finally, Part IV presents methods of parallel programming on various platforms and languages. The first chapter presents different models on scalability as divided into resources, applications, and technology. It defines three abstract models (PRAM, BSP, and phase parallel models) and five physical models (PVP, SMP, MPP, COW, and MPP systems). Chapter 2 introduces the ideas behind parallel programming including processes, tasks, threads and environments. Chapter 3 introduces performance issues and metrics. As an introduction to Part II, Chapter 4 introduces the history of microprocessor types and their applications in the architectures of current systems. Chapter 5 deals with the issues of distributed memory. It discusses several models such as UMA, NORMA, CC-NUMA, COMA, and DSM. Chapter 6 presents gigabit networks, switched interconnects, and other various high-speed networking architectures to construct clusters. Chapter 7 discusses the overheads created by parallel computing, such as threads, synchronization, and efficient communication between nodes. Part III, Chapter 8, 9 and 11, give comparisons between various types of scalable systems (SMP, CC-NUMA, Clusters, and MPP). The comparisons are based on hardware architecture, the system software, and special features that make each system unique. Chapter 10 compares various research and commercial clusters with an in depth study of the Berkeley NOW, IBM SP2, and Digital TruCluster systems. Chapter 12 introduces the concepts of Part IV with details into parallel programming paradigms. Chapter 13 discusses communications between the processors using message passing programming (such as MPI and PVM libraries). Chapter 14 studies the data parallel approach with an emphasis in Fortran 90 and HPF. With attention to detail through examples, Hwang and Xu have created a well-written introduction to Parallel Computing. The authors are distinguished for their contributions in this field. This text is based on cutting-edge research, providing the current theories that are in use in industry today. Bin Cong, Shawn Morrison and Michael Yorg, Department of Computer Science  California Polytechnic State University at San Luis Obispo

Scalable Parallel Computing: Technology, Architecture, Programming

Load balancing mechanisms and techniques in the cloud environments

The paper describes methods for using Extremal Optimization (EO) for processor load balancing during execution of distributed applications. A load balancing algorithm for clusters of multicore processors is presented and discussed. In this algorithm the EO approach is used to periodically detect the best tasks as candidates for migration and for a guided selection of the best computing nodes to receive the migrating tasks. To decrease the complexity of selection for migration, the embedded EO algorithm assumes a two-step stochastic selection during the solution improvement based on two separate fitness functions. The functions are based on specific models which estimate relations between the programs and the executive hardware. The proposed load balancing algorithm is assessed by experiments with simulated load balancing of distributed program graphs. The algorithm is compared against a greedy fully deterministic approach, a genetic algorithm and an EO-based algorithm with random placement of migrated tasks.

Extremal Optimization applied to load balancing in execution of distributed programs

Look-Ahead Dynamic Reconfiguration of Link Connections in Multi-Processor Architectures.

New architectural solutions for parallel systems built of bus-based shared memory processor clusters are presented. A new paradigm is proposed for interprocessor communication, called communication on the fly. With this paradigm, processors can be dynamically switched between clusters at program run-time to bring in their caches data that can be read by many processors in a cluster at the same time they are written to the cluster memory. A cache controlled macro data flow program execution paradigm is also proposed. Programs are structured into tasks for which all required data are brought to the processor data cache before task execution. A. new graph representation of programs is introduced, which enables modeling of functioning of data caches, memories, bus arbiters, processor switching between clusters and parallel reads of data on the fly. This representation is used for realistic simulation of a numerical algorithm execution based on distribution of parallel tasks between dynamic SMP clusters and on communication on the fly. Performance evaluation results are presented for different configurations of the programs and shared memory clusters in the system.

Communication on the fly and program execution control in a system of dynamically configurable SMP clusters

An extension of the graphical parallel program design system P-GRADE towards specification of program execution control based on global application state monitoring is presented. De-coupled structured specifications of computational and control elements of parallel programs are assumed. Special synchronizer processes collect process state messages supplied with time interval timestamps and construct strongly consistent application states. Control predicates are evaluated on these states by synchronizers. As a result, control signals can be sent to application processes to stimulate desired reactions to the predicates. The signals can cause asynchronous computation activation or cancellation. Implementation of a parallel program of Traveling Salesman Problem (TSP) solved by branch-and-bound (B&B) method is described to illustrate properties of the new system.

Graphical Design of Parallel Programs with Control Based on Global Application States Using an Extended P-Grade System

The paper presents a new architecture for systems based on run-time reconfigured shared memory processor clusters meant for implementation using network on chip technology. Clusters constitute local data exchange sub-networks, which dynamically connect processors with shared memory modules. The sub-networks enable exposure of data from one processor's data cache for reading by other processors to their data caches. This inter-processor data exchange paradigm, called "communication on the fly", enables direct communication between processor data caches. Dual-ported data caches are assumed to enable parallel reading and writing data between the caches and memory modules. In the proposed architecture, programs are executed according to a cache-controlled macro data flow execution model. Computational tasks are so defined, as to eliminate re-loading of data caches during task execution. A special program macro-data flow graph representation enables modeling of program behaviour for different architectural and program structure assumptions. Simulation results of symbolic execution of program graphs of matrix multiplication are presented in the paper. They show suitability of the proposed architecture for very fine grain parallel computations.

Marek Tudruj

Papers

Extremal Optimization applied to load balancing in execution of distributed programs

Look-Ahead Dynamic Reconfiguration of Link Connections in Multi-Processor Architectures.

Communication on the fly and program execution control in a system of dynamically configurable SMP clusters

Graphical Design of Parallel Programs with Control Based on Global Application States Using an Extended P-Grade System

Dynamic SMP clusters with communication on the fly in NoC technology for very fine grain computations