OpenMP has been very successful in exploiting structured parallelism in applications. With increasing application complexity, there is a growing need for addressing irregular parallelism in the presence of complicated control structures. This is evident in various efforts by the industry and research communities to provide a solution to this challenging problem. One of the primary goals of OpenMP 3.0 is to define a standard dialect to express and efficiently exploit unstructured parallelism. This paper presents the design of the OpenMP tasking model by members of the OpenMP 3.0 tasking sub-committee which was formed for this purpose. The paper summarizes the efforts of the sub-committee (spanning over two years) in designing, evaluating and seamlessly integrating the tasking model into the OpenMP specification. In this paper, we present the design goals and key features of the tasking model, including a rich set of examples and an in-depth discussion of the rationale behind various design choices. We compare a prototype implementation of the tasking model with existing models, and evaluate it on a wide range of applications. The comparison shows that the OpenMP tasking model provides expressiveness, flexibility, and huge potential for performance and scalability.

The Design of OpenMP Tasks

Large multidimensional arrays are widely used in scientific and engineering database applications. The authors present methods of organizing arrays to make their access on secondary and tertiary memory devices fast and efficient. They have developed four techniques for doing this: (1) storing the array in multidimensional "chunks" to minimize the number of blocks fetched, (2) reordering the chunked array to minimize seek distance between accessed blocks, (3) maintaining redundant copies of the array, each organized for a different chunk size and ordering and (4) partitioning the array onto platters of a tertiary memory device so as to minimize the number of platter switches. The measurements on real data obtained from global change scientists show that accesses on arrays organized using these techniques are often an order of magnitude faster than on the unoptimized data. >

/pdf/efficient-organization-of-large-multidimensional-arrays-35e9mxpo9u.pdf

Efficient organization of large multidimensional arrays

Traditional parallel applications have exploited regular parallelism, based on parallel loops. Only a few applications exploit sections parallelism. With the release of the new OpenMP specification (3.0), this programming model supports tasking. Parallel tasks allow the exploitation of irregular parallelism, but there is a lack of benchmarks exploiting tasks in OpenMP. With the current (and projected) multicore architectures that offer many more alternatives to execute parallel applications than traditional SMP machines, this kind of parallelism is increasingly important. And so, the need to have some set of benchmarks to evaluate it. In this paper, we motivate the need of having such a benchmarks suite, for irregular and/or recursive task parallelism. We present our proposal, the Barcelona OpenMP Tasks Suite (BOTS), with a set of applications exploiting regular and irregular parallelism, based on tasks. We present an overall evaluation of the BOTS benchmarks in an Altix system and we discuss some of the different experiments that can be done with the different compilation and runtime alternatives of the benchmarks.

/pdf/barcelona-openmp-tasks-suite-a-set-of-benchmarks-targeting-dliajczcpt.pdf

Barcelona OpenMP Tasks Suite: A Set of Benchmarks Targeting the Exploitation of Task Parallelism in OpenMP

In this paper we introduce a model of Hierarchical Memory with Block Transfer (BT for short). It is like a random access machine, except that access to location x takes time f(x), and a block of consecutive locations can be copied from memory to memory, taking one unit of time per element after the initial access time. We first study the model with f(x) = xα for 0 ≪ α ≪ 1. A tight bound of θ(n log log n) is shown for many simple problems: reading each input, dot product, shuffle exchange, and merging two sorted lists. The same bound holds for transposing a √n × √n matrix; we use this to compute an FFT graph in optimal θ(n log n) time. An optimal θ(n log n) sorting algorithm is also shown. Some additional issues considered are: maintaining data structures such as dictionaries, DAG simulation, and connections with PRAMs. Next we study the model f(x) = x. Using techniques similar to those developed for the previous model, we show tight bounds of θ(n log n) for the simple problems mentioned above, and provide a new technique that yields optimal lower bounds of Ω(n log2n) for sorting, computing an FFT graph, and for matrix transposition. We also obtain optimal bounds for the model f(x)= xα with α ≫ 1. Finally, we study the model f(x) = log x and obtain optimal bounds of θ(n log*n) for simple problems mentioned above and of θ(n log n) for sorting, computing an FFT graph, and for some permutations.

Hierarchical memory with block transfer

In this paper we report on the development of an efficient and portable implementation of Strassen's matrix multiplication algorithm for matrices of arbitrary size. Our technique for defining the criterion which stops the recursions is more detailed than those generally used, thus allowing enhanced performance for a larger set of input sizes. In addition, we deal with odd matrix dimensions using a method whose usefulness had previously been in question and had not so far been demonstrated. Our memory requirements have also been reduced, in certain cases by 40 to more than 70 percent over other similar implementations. We measure performance of our code on the IBM RS/6000, CRAY YMP C90, and CRAY T3D single processor, and offer comparisons to other codes. Finally, we demonstrate the usefulness of our implementation by using it to perform the matrix multiplications in a large application code.

Implementation of Strassen's Algorithm for Matrix Multiplication

A, set of basic procedures for constructing matrix multiplication algorithms is defined. Five classes of composite matrix multiplication algorithms are considered and an optimal strategy is presented for each class. Instances are given of improvements in arithmetic cost over Strassen’s method for multiplying square matrices. Best and worst case cost coefficients for matrix multiplication are given.

Efficient Procedures for Using Matrix Algorithms

The procedure TANGENTS (P, m, p) accepts as its inputs a point p and a convex polygon P, given as a sequence of vertices beginning with m, also explicitly given. P is represented by means of an A VL tree T(P) modified so that each vertex Vi stores a pointer NEXT[vi] to the address of its successor Vi+l on the boundary of P. This modification is prompted by the following considerations. The \"case classification task\" performed by TANGENTS uses the two vertices m and M; to ensure that this task be completed in constant time, both M (the root) and m (the first member) of T(P) must be available. Thus when either the left or the right subtrees of T(P) are chosen for a recursive call of TANGENTS, their first elements must be also supplied and this is expediently done by means of the pointer NEXT. (An update of NEXT occurs only when a vertex pi is inserted by RESTRUCTURE and it is easily seen that it only involves two pointers, associated with t and pi respectively.) Below, u is a Boolean parameter which is set to 1 in cases 1, 3, 5, 7 (recursive calls of TANGENTS) and is set to 0 otherwise.

Storage reorganization techniques for matrix computation in a paging environment

In order to minimize the number of page fetches required when multiplying matrices occupying many pages of virtual storage, we consider adapting Strassen-like recursive methods to a paging environment. An algorithm with a theoretically better rate of growth results. Also presented is an algorithm for efficiently converting matrices from row storage form to sub-matrix storage form, thus making more accessible the benefits of algorithms based on sub-matrix storage form which were presented in [5].

A note on matrix multiplication in a paging environment

Commutativity, non-commutativity, and bilinearity

These papers indicate the diversity of the theoretical area of computer science. The first explores programming language concepts in terms of Hoare's formal assignment axiom. The second is well described by its title. The third is a contribution to formal language theory, and the last paper adapts a “divide and conquer” technique to a paging environment.

Robert L. Probert

Papers

Efficient Procedures for Using Matrix Algorithms

Storage reorganization techniques for matrix computation in a paging environment

A note on matrix multiplication in a paging environment

Commutativity, non-commutativity, and bilinearity

SIGACT (Paper Session)