From the leading minds in the field, Distributed and Cloud Computing is the first modern, up-to-date distributed systems textbook Starting with an overview of modern distributed models, the book exposes the design principles, systems architecture, and innovative applications of parallel, distributed, and cloud computing systems It will teach you how to create high-performance, scalable, reliable systems, providing comprehensive coverage of distributed and cloud computing, including: Facilitating management, debugging, migration, and disaster recovery through virtualization Clustered systems for research or ecommerce applications Designing systems as web services Social networking systems using peer-to-peer computing Principles of cloud computing using examples from open-source and commercial applications Using examples from open-source and commercial vendors, the text describes cloud-based systems for research, e-commerce, social networking and more Complete coverage of modern distributed computing technology including clusters, the grid, service-oriented architecture, massively parallel processors, peer-to-peer networking, and cloud computing Includes case studies from the leading distributed computing vendors: Amazon, Microsoft, Google, and more Designed to meet the needs of students taking a distributed systems course, each chapter includes exercises and further reading, with lecture slides and solutions available online

Distributed and Cloud Computing: From Parallel Processing to the Internet of Things

Predicting grid performance is a complex task because heterogeneous resource nodes are involved in a distributed environment. Long execution workload on a grid is even harder to predict due to heavy load fluctuations. In this paper, we use Kalman filter to minimize the prediction errors. We apply Savitzky-Golay filter to train a sequence of confidence windows. The purpose is to smooth the prediction process from being disturbed by load fluctuations. We present a new adaptive hybrid method (AHModel) for load prediction guided by trained confidence windows. We test the effectiveness of this new prediction scheme with real-life workload traces on the AuverGrid and Grid5000 in France. Both theoretical and experimental results are reported in this paper. As the lookahead span increases from 10 to 50 steps (5 minutes per step), the AHModel predicts the grid workload with a mean-square error (MSE) of 0.04-0.73 percent, compared with 2.54-30.2 percent in using the static point value autoregression (AR) prediction method. The significant gain in prediction accuracy makes the new model very attractive to predict Grid performance. The model was proved especially effective to predict large workload that demands very long execution time, such as exceeding 4 hours on the Grid5000 over 5,000 processors. With minor changes of some system parameters, the AHModel can apply to other computational grids as well. At the end, we discuss extended research issues and tool development for Grid performance prediction.

Adaptive Workload Prediction of Grid Performance in Confidence Windows

We assess the state-of-the-art technology in massively parallel processors (MPPs) and their variations in different architectural platforms. Architectural and programming issues are identified in using MPPs for time-critical applications such as adaptive radar signal processing. We review the enabling technologies. These include high-performance CPU chips and system interconnects, distributed memory architectures, and various latency hiding mechanisms. We characterize the concept of scalability in three areas: resources, applications, and technology. Scalable performance attributes are analytically defined. Then we compare MPPs with symmetric multiprocessors (SMPs) and clusters of workstations (COWs). The purpose is to reveal their capabilities, limits, and effectiveness in signal processing. We evaluate the IBM SP2 at MHPCC, the Intel Paragon at SDSC, the Gray T3D at Gray Eagan Center, and the Gray T3E and ASCI TeraFLOP system proposed by Intel. On the software and programming side, we evaluate existing parallel programming environments, including the models, languages, compilers, software tools, and operating systems. Some guidelines for program parallelization are provided. We examine data-parallel, shared-variable, message-passing, and implicit programming models. Communication functions and their performance overhead are discussed. Available software tools and communication libraries are also introduced.

/pdf/scalable-parallel-computers-for-real-time-signal-processing-1ab9pmar44.pdf

Scalable parallel computers for real-time signal processing

List of Figures. List of Tables. Preface. 1. Introduction. 2. Speedup. 3. Scalability. 4. Parallel Performance Measurement and Analysis. 5. Parallel Performance Prediction. 6. Parallel Performance Visualization. 7. Parallel Performance Debugging. 8. Performance Evaluation of Interconnection Networks for Parallel systems. Index.

Performance Evaluation, Prediction and Visualization of Parallel Systems

This paper evaluates the IBM SP2 architecture, the AIX parallel programming environment, and the IBM message-passing library (MPL) through STAP (Space-Time Adaptive Processing) benchmark experiments. Only coarse-grain parallelism was exploited on the SP2 due to its high communication overhead. A new parallelization scheme is developed for programming message passing multicomputers. Parallel STAP benchmark structures are illustrated with domain decomposition, efficient mapping of partitioned programs, and optimization of collective communication operations. We measure the SP2 performance in terms of execution time, Gflop/s rate, speedup over a single SP2 node, and overall system utilization. With 256 nodes, the Maul SP2 demonstrated the best performance of 23 Gflop/s in executing the High-Order Post-Doppler program, corresponding to a 34% system utilization. We have conducted a scalability analysis to reveal the performance growth rate as a function of machine size and STAP problem size. Important lessons learned from these parallel processing benchmark experiments are discussed in the context of real-time, adaptive, radar signal processing on massively parallel processors (MPP).

Benchmark evaluation of the IBM SP2 for parallel signal processing

The performance of Massively Parallel Processors (MPPs) is attributed to a large number of machine and program factors. Software development for MPP applications is often very costly. The high cost is partially caused by a lack of early prediction of MPP performance. The program development cycle may iterate many times before achieving the desired performance level. In this paper, we present an early prediction scheme we have developed at the University of Southern California for reducing the cost of application software development. Using workload analysis and overhead estimation, our scheme optimizes the design of parallel algorithm before entering the tedious coding, debugging, and testing cycle of the applications. The scheme is generally applied at user/programmer level, not tied to any particular machine platform or any specific software environment. We have tested the effectiveness of this early performance prediction scheme by running the MIT/STAP benchmark programs on a 400-node IBM SP2 system at the Maui High-Performance Computing Center (MHPCC), on a 400-node Intel Paragon system at the San Diego Supercomputing Center (SDSC), and on a 128-node Cray T3D at the Cray Research Eagan Center in Wisconsin. Our prediction shows to be rather accurate compared with the actual performance measured on these machines. We use the SP2 data to illustrate the early prediction scheme. The main contribution of this work lies in providing a systematic procedure to estimate the computational work-load, to determine the application attributes, and to reveal the communication overhead in using these MPPs. These results can be applied to develop any MPP applications other than the STAP benchmarks by which this prediction scheme was developed.

/pdf/early-prediction-of-mpp-performance-the-sp2-t3d-and-paragon-4m0qgfzdoy.pdf

Kai Hwang

Papers

Early prediction of MPP performance: the SP2, T3D, and Paragon experiences