Prefetching is one approach to reducing the latency of memory operations in modern computer systems. In this paper, we describe the Markov prefetcher. This prefetcher acts as an interface between the on-chip and off-chip cache, and can be added to existing computer designs. The Markov prefetcher is distinguished by prefetching multiple reference predictions from the memory subsystem, and then prioritizing the delivery of those references to the processor.This design results in a prefetching system that provides good coverage, is accurate and produces timely results that can be effectively used by the processor. In our cycle-level simulations, the Markov Prefetcher reduces the overall execution stalls due to instruction and data memory operations by an average of 54% for various commercial benchmarks while only using two thirds the memory of a demand-fetch cache organization.

Prefetching using Markov predictors

Software-controlled data prefetching offers the potential for bridging the ever-increasing speed gap between the memory subsystem and today's high-performance processors. While prefetching has enjoyed considerable success in array-based numeric codes, its potential in pointer-based applications has remained largely unexplored. This paper investigates compiler-based prefetching for pointer-based applications---in particular, those containing recursive data structures. We identify the fundamental problem in prefetching pointer-based data structures and propose a guideline for devising successful prefetching schemes. Based on this guideline, we design three prefetching schemes, we automate the most widely applicable scheme (greedy prefetching) in an optimizing research compiler, and we evaluate the performance of all three schemes on a modern superscalar processor similar to the MIPS R10000. Our results demonstrate that compiler-inserted prefetching can significantly improve the execution speed of pointer-based codes---as much as 45% for the applications we study. In addition, the more sophisticated algorithms (which we currently perform by hand, but which might be implemented in future compilers) can improve performance by as much as twofold. Compared with the only other compiler-based pointer prefetching scheme in the literature, our algorithms offer substantially better performance by avoiding unnecessary overhead and hiding more latency.

/pdf/compiler-based-prefetching-for-recursive-data-structures-1fqcvs6pcb.pdf

Compiler-based prefetching for recursive data structures

Effective data prefetching requires accurate mechanisms to predict both “which” cache blocks to prefetch and “when” to prefetch them. This paper proposes the Dead-Block Predictors (DBPs), trace-based predictors that accurately identify “when” an Ll data cache block becomes evictable or “dead”. Predicting a dead block significantly enhances prefetching lookahead and opportunity, and enables placing data directly into Ll, obviating the need for auxiliary prefetch buffers. This paper also proposes Dead-Block Correlating Prefetchers (DBCPs), that use address correlation to predict “which” subsequent block to prefetch when a block becomes evictable. A DBCP enables effective data prefetching in a wide spectrum of pointer-intensive, integer, and floating-point applications.We use cycle-accurate simulation of an out-of-order superscalar processor and memory-intensive benchmarks to show that: (1) dead-block prediction enhances prefetching lookahead at least by an order of magnitude as compared to previous techniques, (2) a DBP can predict dead blocks on average with a coverage of 90% only mispredicting 4% of the time, (3) a DBCP offers an address prediction coverage of 86% only mispredicting 3% of the time, and (4) DBCPs improve performance by 62% on average and 282% at best in the benchmarks we studied.

/pdf/dead-block-prediction-dead-block-correlating-prefetchers-4bhjncfwyp.pdf

Dead-block prediction & dead-block correlating prefetchers

The expanding gap between microprocessor and DRAM performance has necessitated the use of increasingly aggressive techniques designed to reduce or hide the latency of main memory access. Although large cache hierarchies have proven to be effective in reducing this latency for the most frequently used data, it is still not uncommon for many programs to spend more than half their run times stalled on memory requests. Data prefetching has been proposed as a technique for hiding the access latency of data referencing patterns that defeat caching strategies. Rather than waiting for a cache miss to initiate a memory fetch, data prefetching anticipates such misses and issues a fetch to the memory system in advance of the actual memory reference. To be effective, prefetching must be implemented in such a way that prefetches are timely, useful, and introduce little overhead. Secondary effects such as cache pollution and increased memory bandwidth requirements must also be taken into consideration. Despite these obstacles, prefetching has the potential to significantly improve overall program execution time by overlapping computation with memory accesses. Prefetching strategies are diverse, and no single strategy has yet been proposed that provides optimal performance. The following survey examines several alternative approaches, and discusses the design tradeoffs involved when implementing a data prefetch strategy.

Data prefetch mechanisms

Various systems and methods are provided that display various interactive user interfaces containing interactive maps. The user may use the interactive map to perform searches for data objects, toggle different layers associated with different types of objects for display, generate heatmaps based upon a grid and/or data object shapes, copy data between different interactive maps, and/or define filters to filter the displayed data. In response to user interaction with the map at a client system, tile layers are generated using server-side components, assembled into map tiles, and transmitted to client-side components to be displayed to the user. The tile layers may include a base tile layer, vector tile layer, selection tile layer, and inactive tile layer.

Interactive geospatial map

There are provided a processing unit that processes received requests, a storage unit that stores order information added to the last-processed request among the requests processed by the processing unit as final request information, a determination unit that determines whether the received request has been processed with reference to the final request information based on the order information added to the received request, and a control unit that, when the determination unit determines that the received request has been processed, prevents the received request from being processed by the processing unit, thereby preventing overlapped execution of a command.

Processor, computer readable recording medium recording program therein, and processing system

Previous research on hiding memory latencies has tended to focus on regular numerical programs. This paper presents a latency-hiding compiler technique that is applicable to general-purpose C programs. By assuming a lock-up free cache and instruction score-boarding, our technique 'preloads' the data that are likely to cause a cache-miss before they are used, and thereby hiding the cache miss latency. We have developed simple compiler heuristics to identify load instructions that are likely to cause a cache-miss. Experimentation with a set of SPEC92 benchmarks shows that our heuristics are successful in identifying 85% of cache misses. We have also developed an algorithm that flexibly schedules the selected load instruction and instructions that use the loaded data to hide memory latency. Our simulation suggests that our technique is successful in hiding memory latency and improves the overall performance.

Cache miss heuristics and preloading techniques for general-purpose programs

A cache use priority of a cache memory is designated by a cache use priority designation unit. In response to an access demand issued from a processing unit, a cache control unit controls the access to the cache memory in accordance with the cache use priority. In the case of a set associative cache for example, the number of ways and the number of blocks to be used within one set are varied based on the cache use priority of a process.

Multiprocess execution system that designates cache use priority based on process priority

A control unit shifts a boundary between a range of hash values allocated to a first node and a range of hash values allocated to a second node from a first hash value to a second hash value to thereby expand the range of hash values allocated to the first node. The control unit moves data which is part of data stored in the second node and in which hash values calculated from associated keys belong to a range between the first hash value and the second hash value, from the second node to the first node.

Storage control method and information processing apparatus

A heap is a memory resource managed in units of cells and it is used in units of cells by the execution of an application program. A full garbage collection unit collects free cells based on the check result of the state of use of cells. A partial garbage collection unit collects free cells from cells that are used after the check of the state of use of cells recently made by the full garbage collection unit, based on the check result of the state of use of cells. A full/partial control unit improves the process efficiency of parallel type garbage collection by making either the full garbage collection unit or the partial garbage collection unit perform a subsequent collection of free cells based on the state of collections made in the past.

Toshihiro Ozawa

Papers

Processor, computer readable recording medium recording program therein, and processing system

Cache miss heuristics and preloading techniques for general-purpose programs

Multiprocess execution system that designates cache use priority based on process priority

Storage control method and information processing apparatus

Device and method for managing memory resources