OPTIMIZATION AND PERFORMANCE MODELING OF STENCIL COMPUTATIONS ON MODERN MICROPROCESSORS ‡ KAUSHIK DATTA † SHOAIB KAMIL ∗† SAMUEL WILLIAMS ∗† LEONID OLIKER ∗ JOHN SHALF ∗ KATHERINE YELICK ∗† Abstract. Stencil-based kernels constitute the core of many important scientiﬁc applications on block-structured grids. Unfortunately, these codes achieve a low fraction of peak performance, due primarily to the disparity between processor and main memory speeds. In this paper, we explore the impact of trends in memory subsystems on a variety of stencil optimization techniques and develop performance models to analytically guide our optimizations. Our work targets cache reuse methodologies across single and multiple stencil sweeps, examining cache-aware algorithms as well as cache-oblivious techniques on the Intel Itanium2, AMD Opteron, and IBM Power5. Additionally, we consider stencil computations on the heterogeneous multi-core design of the Cell processor, a machine with an explicitly-managed memory hierarchy. Overall our work represents one of the most extensive analyses of stencil optimizations and performance modeling to date. Results demonstrate that recent trends in memory system organization have reduced the eﬃcacy of traditional cache- blocking optimizations. We also show that a cache-aware implementation is signiﬁcantly faster than a cache-oblivious approach, while the explicitly managed memory on Cell enables the highest overall eﬃciency: Cell attains 88% of algorithmic peak while the best competing cache-based processor only achieves 54% of algorithmic peak performance. Key words. Stencil computations, cache blocking, time-skewing, cache-oblivious algorithms, performance modeling, performance evaluation, Intel Itanium2, AMD Opteron, IBM Power5, STI Cell. AMS subject classiﬁcations. 65Y10, 65Yxx, 35R99, 68M20 1. Introduction. Partial diﬀerential equation (PDE) solvers constitute a large fraction of scientiﬁc applications in such diverse areas as heat diﬀusion, electromag- netics, and ﬂuid dynamics. These applications are often implemented using itera- tive ﬁnite-diﬀerence techniques, which sweep over a spatial grid, performing nearest neighbor computations called stencils. In a stencil operation, each point in a multidi- mensional grid is updated with weighted contributions from a subset of its neighbors in both time and space — thereby representing the coeﬃcients of the PDE for that data element. These operations are then used to build solvers that range from simple Jacobi iterations to complex multigrid and adaptive mesh reﬁnement methods [3]. Stencil calculations perform global sweeps through data structures that are typ- ically much larger than the capacity of the available data caches. As a result, these computations generally achieve a low fraction of theoretical peak performance, since data from main memory cannot be transferred fast enough to avoid stalling the com- putational units on modern microprocessors. Reorganizing these computations to take full advantage of memory hierarchies has been the subject of much investigation over the years. These have principally focused on tiling optimizations [11, 16, 17] that attempt to exploit locality by performing operations on cache-sized blocks of data before moving on to the next block. Whereas many tiling optimizations use domain decomposition to improve spatial locality, more recent studies have focused attention on exploiting locality in the time dimension [6, 13, 19, 24]. In this work, we re-examine stencil computations on current microprocessors in light of the growing performance gap between processors and memory, as well as ‡ Preliminary ∗ CRD/NERSC, versions of this article appeared in [8, 9]. Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA, 94720. † Computer Science Department, University of California, Berkeley, CA, 94720.

Optimization and Performance Modeling of Stencil Computations on Modern Microprocessors

Modern high-performance and warehouse computing centers show strong interest in minimizing system power consumption while satisfying customer requested quality of service (QoS). Dynamic voltage and frequency scaling (DVFS) is effective for achieving this goal. Nevertheless, automating the process online is challenging due to the great complexity — today’s hardware components (e.g., CPUs, GPUs, memory, network, etc.) can be configured in several or dozens of frequency/voltage states for satisfying divergent system demands. Given their combination and the emergence of heterogeneity, searching the optimal configuration in the design space online can be time consuming. Existing work relies on detailed knowledge of workloads, either obtained from extensive offline profiling or intrusive code transformation for fine-grain profiling, unable to transparently support unknown workloads. This work bridges the gap and targets GPU-accelerated systems and applications.We focus on applications exhibiting an interesting feature - iterative or periodic, which is common among conventional HPC and emerging machine learning workloads. We propose an online dynamic power-performance (ODPP) management framework. ODPP runs on the host and dynamically adjusts GPU DVFS configurations to meet performance and power objectives without any code annotation or modification. Particularly, ODPP extracts the performance and power indicators for applications from easily obtained GPU resource utilization profiles in a short episode. The extraction method uses Fourier Transform and incurs minimal cost. ODPP further automatically constructs an accurate model that infers from the indicators how the application’s performance and power vary with GPU core and memory frequencies. Aided with the model, for both seen and unseen applications, ODPP can quickly determine the most appropriate DVFS configuration for their execution. We evaluate ODPP on an NVIDIA GPU using multiple exascale computing (ECP) and deep learning applications. Evaluation results show that ODPP can improve energy efficiency by over 30% under different QoS and improve performance by more than 8% under different power bounds.

Indicator-Directed Dynamic Power Management for Iterative Workloads on GPU-Accelerated Systems

Considering the increasing complexity and scale of HPC architecture and software, the performance modeling of parallel applications on large-scale HPC platforms has become increasingly important. It plays an important role in many areas, such as performance analysis, job management, and resource estimation. In this work, we propose a multi-parameter performance modeling and prediction framework called MPerfPred, which utilizes basic block frequencies as features and uses machine learning algorithms to automatically construct multi-parameter performance models with high generalization ability. To reduce the prediction overhead, we propose some feature-filtering strategies to reduce the number of features in the training stage and build a serial program called BBF collector for each target application to quickly collect feature values in the prediction stage. We demonstrate the use of MPerfPred on the TianHe-2 supercomputer with six parallel applications. Results show that MPerfPred with SVR achieves better prediction than other input parameter-based modeling methods. The average prediction error and average standard deviation of prediction errors of MPerfPred are 8.42% and 6.09%, respectively. In the prediction stage, the average prediction overhead of MPerfPred is less than 0.13% of the total execution time.

/pdf/multi-parameter-performance-modeling-based-on-machine-73558iqx8d.pdf

Multi-Parameter Performance Modeling Based on Machine Learning with Basic Block Features

Performance tuning, software/hardware co-design, and job scheduling are among the many tasks that rely on models to predict application performance. We propose and evaluate low rank tensor decomposition for modeling application performance. We discretize the input and configuration domain of an application using regular grids. Application execution times mapped within grid-cells are averaged and represented by tensor elements. We show that low-rank canonical-polyadic (CP) tensor decomposition is effective in approximating these tensors. We further show that this decomposition enables accurate extrapolation of unobserved regions of an application's parameter space. We then employ tensor completion to optimize a CP decomposition given a sparse set of observed runtimes. We consider alternative piecewise/grid-based models and supervised learning models for six applications and demonstrate that CP decomposition optimized using tensor completion offers higher prediction accuracy and memory-efficiency for high-dimensional applications.

Multi-Parameter Performance Modeling via Tensor Completion

Understanding the computational requirements of scientific applications and their relation to power consumption is a fundamental task to overcome the current barriers to achieve the computational exascale. However, this imposes some challenging tasks, such as to monitor a wide range of parameters in heterogeneous environments, to enable fine grained profiling and power consumed across different components, to be language independent and to avoid code instrumentation. Considering these challenges, this work proposes the SMCis, an application monitoring tool developed with the goal of collecting all these aspects in an effective and accurate way, as well as to correlate these data graphically, with the environment of analysis and visualization. In addition, SMCis integrates and facilitates the use of Machine Learning tools for the development of predictive runtime and power consumption models.

SMCis: Scientific Applications Monitoring and Prediction for HPC Environments

To understand and predict the performance of scientific applications, several analytical and machine learning approaches have been proposed, each having its advantages and disadvantages. In this paper, we propose and validate a hybrid approach for performance modeling and prediction, which combines analytical and machine learning models. The proposed hybrid model aims to minimize prediction cost while providing reasonable prediction accuracy. Our validation results show that the hybrid model is able to learn and correct the analytical models to better match the actual performance. Furthermore, the proposed hybrid model improves the prediction accuracy in comparison to pure machine learning techniques while using small training datasets, thus making it suitable for hardware and workload changes.

Oliver Dobon

Papers

Learning with Analytical Models

Learning with Analytical Models