Loop tiling is a well-known loop transformation generally used to expose coarse-grain parallelism and to exploit data reuse at the cache level. Tiling can also be used to exploit data reuse at the register level and to improve a program's ILP. However, previous proposals in the literature (as well as commercial compilers) are only able to perform multidimensional tiling for the register level when the iteration space is rectangular. In this article we present a new general algorithm to perform multidimensional tiling for the register level in both rectangular and nonrectangular iteration spaces. We also propose a simple heuristic to determine the tiling parameters at this level. Finally, we evaluate our method using as benchmarks typical linear algebra algorithms having nonrectangular iteration spaces and compare our proposal against hand-optimized vendor-supplied numerical libraries and against commercial compilers able to perform optimizing code transformations such as inner unrolling, unroll-and-jam, and software pipelining. Measurements were taken on three different superscalar microprocessors. Results will show that our method outperforms the native compilers (showing speedups of 2.5 in average) and matches the performance of vendor-supplied numerical libraries. The general conclusion is that compiler technology can make it possible for nonrectangular loop nests to achieve as high performance as hand-optimized codes.

https://hpc.ac.upc.edu/PDFs/dir24/file000198.pdf

Register tiling in nonrectangular iteration spaces

Exact and efficient data dependence testing is a key to success of loop-parallelizing compiler for computationally intensive programs. A number of algorithms has been created to test array references contained in parameter loops for dependence but most of them are unable to answer the following question correctly: Are references C(i1 + 10j1) and C(i2 + 5), 0 ≤ i1, i2 ≤ 4, 0 ≤ j1,j2 ≤ 9 independent? The technique introduced in this paper recognizes that i1, i2 and j1, j2 make different order contributions to the subscript index, and breaks dependence equation i1 + 10j1 = i2 + 10j2 + 5 into two equations i1 = i2 and 10j1 = 10j2 which then can be solved independently. Since resulting equations contain less variables it is less expensive to solve them. We call this technique delinearization because it is reverse of the linearization much discussed in the literature.In the introduction we demonstrate that linearized references are used not infrequently in scientific FORTRAN and C codes. Then we present a theorem on which delinearization algorithm is based and the algorithm itself. The algorithm is fairly simple and inexpensive. As a byproduct it tests equations it produces for independence as exactly as it is done by GCD-test and Banerjee inequalities combined. The algorithm has been implemented at Moscow State University in a vectorizer named VIC.

Delinearization: an efficient way to break multiloop dependence equations

The authors present a general approach of self-scheduling a non-uniform parallel loop on a distributed-memory machine. The approach has two phases: a static scheduling phase and a dynamic scheduling phase. In addition to reduce scheduling overhead, using the static scheduling phase allows the data needed by the statically scheduled iterations to be prefetched. The dynamic scheduling phase balances the workload. Data distribution methods for self-scheduling are also the focus of this paper. The authors classify the data distribution methods into four categories and present partial duplication, a method that allows the problem size to grow linearly in the number of processors. The experiments conducted on a 64-node NCUBE show that as much as 79% improvement is achieved over static scheduling on the generation of a false-color image.

Self-scheduling on distributed-memory machines

: This paper presents a formal mathematical framework which unifies the existing loop transformations. This framework also includes more general classes of loop transformations, which can extract more parallelism from a class of programs than the existing techniques. We classify schedules into three classes: uniform, subdomain-variant, and statement-variant. Viewing from the degree of parallelism to be gained by loop transformation, the schedules can also be classified as single-sequential level, multiple-sequential level, and mixed schedules. We also illustrate the usefulness of the more general loop transformation with an example program.

A unified framework for systematic loop transformations

In this paper we generalize the framework of linear loop transformations: we consider loop alignment as a new component in the transformation process The aim is to exploit the additional inherent statement-level parallelism and reduce the amount of interprocessor synchronization and communication when a coarse-grain MIMD execution model is considered The transformation process is modelled with non-singular matrices and we use the ideas recently proposed in this field to generate an efficient transformed code However, additional aspects have to be studied when statements are considered in the process We try to reduce the overhead due to conditionals that appear in the loop body of the transformed loops

Partitioning the statement per iteration space using non-singular matrices

: This paper presents new loop transformation techniques that can extract more parallelism from a class of programs than existing techniques. A formal mathematical framework which unifies the existing loop transformation techniques is given. We classify schedules of a loop transformation into three classes: uniform, subdomain-variant, and statement-variant. New algorithms for generating these schedules are given. Viewing from the degree of parallelism to be gained by loop transformation, the schedules can also be classified as single-sequential level, multiple-sequential level, and mixed schedule. We described iterative and recursive algorithms to obtain multiple-sequential level and mixed schedules respectively based on the algorithms for single-sequential level schedules.

http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA249325&Location=U2&doc=GetTRDoc.pdf

Marina Chen

Papers

New Loop Transformation Techniques for Massive Parallelism