Run-time scheduling and execution of loops on message passing machines

For parallelism to become tractable for mass programmers, shared-memory languages and environments must evolve to enforce disciplined practices that ban "wild shared-memory behaviors;'' e.g., unstructured parallelism, arbitrary data races, and ubiquitous non-determinism. This software evolution is a rare opportunity for hardware designers to rethink hardware from the ground up to exploit opportunities exposed by such disciplined software models. Such a co-designed effort is more likely to achieve many-core scalability than a software-oblivious hardware evolution. This paper presents DeNovo, a hardware architecture motivated by these observations. We show how a disciplined parallel programming model greatly simplifies cache coherence and consistency, while enabling a more efficient communication and cache architecture. The DeNovo coherence protocol is simple because it eliminates transient states -- verification using model checking shows 15X fewer reachable states than a state-of-the-art implementation of the conventional MESI protocol. The DeNovo protocol is also more extensible. Adding two sophisticated optimizations, flexible communication granularity and direct cache-to-cache transfers, did not introduce additional protocol states (unlike MESI). Finally, DeNovo shows better cache hit rates and network traffic, translating to better performance and energy. Overall, a disciplined shared-memory programming model allows DeNovo to seamlessly integrate message passing-like interactions within a global address space for improved design complexity, performance, and efficiency.

/pdf/denovo-rethinking-the-memory-hierarchy-for-disciplined-32e5kmy7ka.pdf

DeNovo: Rethinking the Memory Hierarchy for Disciplined Parallelism

Linear algebra algorithms based on the BLAS or ex tended BLAS do not achieve high performance on mul tivector processors with a hierarchical memory system because of a lack of data locality. For such machines, block linear algebra algorithms must be implemented in terms of matrix-matrix primitives BLAS3. Designing ef ficient linear algebra algorithms for these architectures requires analysis of the behavior of the matrix-matrix primitives and the resulting block algorithms as a func tion of certain system parameters. The analysis must identify the limits of performance improvement possible via blocking and any contradictory trends that require trade-off consideration. We propose a methodology that facilitates such an analysis and use it to analyze the per formance of the BLAS3 primitives used in block methods. A similar analysis of the block size-perfor mance relationship is also performed at the algorithm level for block versions of the LU decomposition and the Gram-Schmidt orthogonalization procedures.

Impact of Hierarchical Memory Systems On Linear Algebra Algorithm Design

Private data caches have not been as effective in reducing the average memory delay in multiprocessors as in uniprocessors due to data spreading among the processors, and due to the cache coherence problem. A wide variety of mechanisms have been proposed for maintaining cache coherence in large-scale shared memory multiprocessors making it difficult to compare their performance and implementation implications. To help the computer architect understand some of the trade-offs involved, this paper surveys current cache coherence mechanisms, and identifies several issues critical to their design. These design issues include: 1) the coherence detection strategy, through which possibly incoherent memory accesses are detected either statically at compile-time, or dynamically at run-time; 2) the coherence enforcement strategy, such as updating or invalidating, that is used to ensure that stale cache entries are never referenced by a processor; 3) how the precision of block sharing information can be changed to trade-off the implementation cost and the performance of the coherence mechanism; and 4) how the cache block size affects the performance of the memory system. Trace-driven simulations are used to compare the performance and implementation impacts of these different issues. In addition, hybrid strategies are presented that can enhance the performance of the multiprocessor memory system by combining several different coherence mechanisms into a single system.

/pdf/cache-coherence-in-large-scale-shared-memory-multiprocessors-1jgx4cvbtr.pdf

Cache coherence in large-scale shared-memory multiprocessors: issues and comparisons

Software-assisted cache coherence enforcement schemes for large multiprocessor systems with shared global memory and interconnection network have gained increasing attention. Proposed software-assisted approaches rely on either indiscriminate invalidation or selective invalidation to invalidate stale cache lines. The indiscriminate approach combined with advanced memory hardware can quickly invalidate the entire cache but may result in lower hit ratios. The selective approach may achieve a better hit ratio. However, sequential selection and invalidation of cache or TLB entries is time consuming. We propose a new solution that offers the fast operation of the indiscriminate invalidation approach and can selectively invalidate cache items without extensive run-time book-keeping and checking. The solution relies on the combination of compile-time reference tagging and individual invalidation of potentially stale cache lines only when referenced. Performance improvement over an indiscriminate invalidation approach is presented.

A cache coherence scheme with fast selective invalidation

A version control approach to maintain cache coherence is proposed for large-scale shared-memory multiprocessor systems with interconnection networks. The new approach, unlike existing approaches for such class of systems, makes it possible to exploit temporal locality across synchronization boundaries. As with the other software-directed approaches, each processor independently manages its cache, i.e., there is no interprocessor communication involved in maintaining cache coherence. The hardware required per processor in the version control approach stays constant as the number of processors increases; hence, it scales up to larger systems. Furthermore, the new approach incurs low overhead. The simulated results of several schemes for large-scale systems show that the new approach achieves a data cache hit ratio closest to maximum possible.

A version control approach to Cache coherence

Stale Data Detection and Coherence Enforcement Using Flow Analysis.

The Chief project provides a environment for analyzing parallel systems. The project aims to employ the Perfect club benchmarks to evaluate performance. A tool (MaxPar) determines the maximum available parallelism in a program by instrumenting each program; the results are computed by running the compiled result. Three different parallel trace generators convert benchmark programs into trace input files for simulation. Parallel simulation kernels based upon event-driven and hybrid time- and event-driven models run on multiprocessors such as the Alliant FX/8. Simulation components are designed in a high-level language, and their modular design encourages reconfiguring existing components into new simulators to explore architectural variations. A graphical user-interface provides a powerful tool for simulator configuration, simulator debugging, and result visualization. >

Chief: a parallel simulation environment for parallel systems

In recent years interest has grown in multiprocessor architectures that can have several hundred processors sharing memory and all working on solving a single problem. Such multiprocessors are characterized by a long memory access time, which makes use of cache memories very important. However, the cache coherence problem makes the use of private caches difficult. The proposed solutions to the cache coherence problem are not suitable for a large-scale multiprocessor. We proposed a different solution that relies on a compiler to manage the caches during the execution of a parallel program. In this paper we discuss the performance evaluation of private cache memories for multiprocessors in general and present the results of performance improvement for parallel numerical programs using the caches managed with compiler assistance. The effect of cache organization and other system parameters such as cache block size, cache size, and the number of processors in the system performance is shown. 18 refs., 2 figs., 7 tabs.

The performance of software-managed multiprocessor caches on parallel numerical programs

We discuss three different software-assisted cache coherence enforcement schemes for large shared-memory multiprocessor systems using interconnection networks. All three rely on a compiler to detect potential coherence problems and generate code to enforce coherence in a parallel program. The main goals are to maintain coherence without any interprocessor communication and to keep coherence enforcement overhead low. The former is achieved by using compile-time knowledge of the parallelism and data dependences in a program. The latter is achieved by using special hardware to invalidate stale cache blocks in time independent of the number of such blocks. Cache words are allowed to become inconsistent with memory as long as the compiler decided it is safe to do so. This allows invalidation to be delayed beyond the time a new copy of a cache word has been generated till the time the word has to be invalidated. The three schemes differ in the complexity and the power of the compiler detection algorithms, the complexity of the additional hardware, and the run-time support the hardware provides for deciding what to invalidate. Each scheme improves over the previous one in terms of the amount of unnecessary invalidation and achieves higher hit ratios.

Hoichi Cheong

Papers

A version control approach to Cache coherence

Stale Data Detection and Coherence Enforcement Using Flow Analysis.

Chief: a parallel simulation environment for parallel systems

The performance of software-managed multiprocessor caches on parallel numerical programs

Software-directed Cache Management in Multiprocessors