This dissertation presents an automated system to generate highly efficient, platform-adapted implementations of sparse matrix kernels. We show that conventional implementations of important sparse kernels like sparse matrix-vector multiply (SpMV) have historically run at 10% or less of peak machine speed on cache-based superscalar architectures. Our implementations of SpMV, automatically tuned using a methodology based on empirical-search, can by contrast achieve up to 31% of peak machine speed, and can be up to 4× faster. 
Given a matrix, kernel, and machine; our approach to selecting a fast implementation consists of two steps: (1) we identify and generate a space of reasonable implementations, and then (2) search this space for the fastest one using a combination of heuristic models and actual experiments (i.e., running and timing the code). We build on the SPARSITY system for generating highly-tuned implementations of the SpMV kernel y ← y + Ax, where A is a sparse matrix and x, y are dense vectors. We extend SPARSITY to support tuning for a variety of common non-zero patterns arising in practice, and for additional kernels like sparse triangular solve (SpTS) and computation of ATA·x (or AAT·x) and A ρ·x. 
We develop new models to compute, for particular data structures and kernels, the best absolute performance (e.g., Mflop/s) we might expect on a given matrix and machine. These performance upper bounds account for the cost of memory operations at all levels of the memory hierarchy, but assume ideal instruction scheduling and low-level tuning. We evaluate our performance with respect to such bounds, finding that the generated and tuned implementations of SpMV and SpTS achieve up to 75% of the performance bound. This finding places limits on the effectiveness of additional low-level tuning (e.g., better instruction selection and scheduling). (Abstract shortened by UMI.)

/pdf/automatic-performance-tuning-of-sparse-matrix-kernels-76ql9zn9l5.pdf

Automatic performance tuning of sparse matrix kernels

Standard benchmarking provides to run-times for given programs on given machines, but fails to provide insight as to why those results were obtained (either in terms of machine or program characteristics) and fails to provide run-times for that program on some other machine, or some other programs on that machine. We have developed a machine-imdependent model of program execution to characterize both machine performance and program execution. By merging these machine and program characterizations, we can estimate execution time for arbitrary machine/program combinations. Our technique allows us to identify those operations, either on the machine or in the programs, which dominate the benchmark results. This information helps designers in improving the performance of future machines and users in tuning their applications to better utilize the performance of existing machines. Here we apply our methodology to characterize benchmarks and predict their execution times. We present extensive run-time statistics for a large set of benchmarks including the SPEC and Perfect Club suites. We show how these statistics can be used to identify important shortcoming in the programs. In addition, we give execution time estimates for a large sample of programs and machines and compare these against benchmark results. Finally, we develop a metric for program similarity that makes it possible to classify benchmarks with respect to a large set of characteristics.

Analysis of benchmark characteristics and benchmark performance prediction

Memory access efficiency is a key factor in fully utilizing the computational power of graphics processing units (GPUs). However, many details of the GPU memory hierarchy are not released by GPU vendors. In this paper, we propose a novel fine-grained microbenchmarking approach and apply it to three generations of NVIDIA GPUs, namely Fermi, Kepler, and Maxwell, to expose the previously unknown characteristics of their memory hierarchies. Specifically, we investigate the structures of different GPU cache systems, such as the data cache, the texture cache and the translation look-aside buffer (TLB). We also investigate the throughput and access latency of GPU global memory and shared memory. Our microbenchmark results offer a better understanding of the mysterious GPU memory hierarchy, which will facilitate the software optimization and modelling of GPU architectures. To the best of our knowledge, this is the first study to reveal the cache properties of Kepler and Maxwell GPUs, and the superiority of Maxwell in shared memory performance under bank conflict.

Dissecting GPU Memory Hierarchy Through Microbenchmarking

The authors consider whether SPECmarks, the figures of merit obtained from running the SPEC benchmarks under certain specified conditions, accurately indicate the performance to be expected from real, live work loads. Miss ratios for the entire set of SPEC92 benchmarks are measured. It is found that instruction cache miss ratios in general, and data cache miss ratios for the integer benchmarks, are quite low. Data cache miss ratios for the floating-point benchmarks are more in line with published measurements for real work loads. >

/pdf/cache-performance-of-the-spec92-benchmark-suite-28jv6mleml.pdf

Cache performance of the SPEC92 benchmark suite

Power and energy management is crucial for mobile devices that rely on battery power. In addition to voice recognition, image understanding is an important class of applications for mobile environments. We propose a new compilation strategy for remote task mapping, and report experimental results for a face detection and recognition system.

Our compilation strategy generates two versions of the input program, one to be executed on the mobile device (client), and the other on a machine connected to the mobile device via a wireless network (server). Compiler supported checkpointing is used to allow the client to monitor program progress on the server, and to request checkpoint data in case of anticipated server and/or network failure. The reported results have been obtained by actual power measurements, not simulation. Experiments show energy savings of up to one order of magnitude on the mobile machine. A prototype implementation of the discussed compilation framework is underway, and preliminary results are reported.

A compilation framework for power and energy management on mobile computers

Runs of a benchmark or a suite of benchmarks are inadequate either to characterize a given machine or to predict the running time of some benchmark not included in the suite. Further, the observed results are quite sensitive to the nature of the benchmarks, and the relative performance of two machines can vary greatly depending on the benchmarks used. In this dissertation we propose and investigate a new approach to CPU performance evaluation. The main idea is to represent machine performance and program execution in terms of a high level abstract machine model. The model is machine-independent and thus is valid on any uniprocessor. We have developed tools to measure the performance of a variety of machines, from workstations to supercomputers. We have also characterized the execution of many large applications, including the SPEC and Perfect benchmark suites. By merging these machine and program characterizations, we can estimate execution times quite accurately for arbitrary machine-program combinations. Another aspect of the research has consisted in characterizing the effectiveness of optimizing compilers.
Another contribution of this dissertation is to propose and investigate new metrics for machine and program similarity and the information that can be derived from them. We define the concept of pershapes, which represent the level of performance of a machine for different types of computation. We introduce a metric based on pershapes that provides a quantitative way of measuring how similar two machines are in terms of their performance distributions. This metric is related to the extent to which pairs of machines have varying relative performance levels depending on which benchmark is used. A similar metric for programs allows us to compare and cluster them according to their dynamic behavior. All this information helps to identify those parameters in machines and programs which are the most important in determining their execution times. Further, it provides a way for designers and users to identify potential bottlenecks in machines, compilers, and applications.

Rafael Hector Saavedra-Barrera

Papers

CPU performance evaluation and execution time prediction using narrow spectrum benchmarking