Newness and distinctiveness is claimed in the features of ornamentation as shown inside the broken line circle in the accompanying representation.

Display screen or portion thereof with graphical user interface

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.

PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory

The explosion of digital data and the ever-growing need for fast data analysis have made in-memory big-data processing in computer systems increasingly important. In particular, large-scale graph processing is gaining attention due to its broad applicability from social science to machine learning. However, scalable hardware design that can efficiently process large graphs in main memory is still an open problem. Ideally, cost-effective and scalable graph processing systems can be realized by building a system whose performance increases proportionally with the sizes of graphs that can be stored in the system, which is extremely challenging in conventional systems due to severe memory bandwidth limitations. In this work, we argue that the conventional concept of processing-in-memory (PIM) can be a viable solution to achieve such an objective. The key modern enabler for PIM is the recent advancement of the 3D integration technology that facilitates stacking logic and memory dies in a single package, which was not available when the PIM concept was originally examined. In order to take advantage of such a new technology to enable memory-capacity-proportional performance, we design a programmable PIM accelerator for large-scale graph processing called Tesseract. Tesseract is composed of (1) a new hardware architecture that fully utilizes the available memory bandwidth, (2) an efficient method of communication between different memory partitions, and (3) a programming interface that reflects and exploits the unique hardware design. It also includes two hardware prefetchers specialized for memory access patterns of graph processing, which operate based on the hints provided by our programming model. Our comprehensive evaluations using five state-of-the-art graph processing workloads with large real-world graphs show that the proposed architecture improves average system performance by a factor of ten and achieves 87% average energy reduction over conventional systems.

/pdf/a-scalable-processing-in-memory-accelerator-for-parallel-23yasy5934.pdf

A scalable processing-in-memory accelerator for parallel graph processing

The high accuracy of deep neural networks (NNs) has led to the development of NN accelerators that improve performance by two orders of magnitude. However, scaling these accelerators for higher performance with increasingly larger NNs exacerbates the cost and energy overheads of their memory systems, including the on-chip SRAM buffers and the off-chip DRAM channels.This paper presents the hardware architecture and software scheduling and partitioning techniques for TETRIS, a scalable NN accelerator using 3D memory. First, we show that the high throughput and low energy characteristics of 3D memory allow us to rebalance the NN accelerator design, using more area for processing elements and less area for SRAM buffers. Second, we move portions of the NN computations close to the DRAM banks to decrease bandwidth pressure and increase performance and energy efficiency. Third, we show that despite the use of small SRAM buffers, the presence of 3D memory simplifies dataflow scheduling for NN computations. We present an analytical scheduling scheme that matches the efficiency of schedules derived through exhaustive search. Finally, we develop a hybrid partitioning scheme that parallelizes the NN computations over multiple accelerators. Overall, we show that TETRIS improves mthe performance by 4.1x and reduces the energy by 1.5x over NN accelerators with conventional, low-power DRAM memory systems.

https://dl.acm.org/doi/pdf/10.1145/3037697.3037702

TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory

Processing-in-memory (PIM) is rapidly rising as a viable solution for the memory wall crisis, rebounding from its unsuccessful attempts in 1990s due to practicality concerns, which are alleviated with recent advances in 3D stacking technologies. However, it is still challenging to integrate the PIM architectures with existing systems in a seamless manner due to two common characteristics: unconventional programming models for in-memory computation units and lack of ability to utilize large on-chip caches. In this paper, we propose a new PIM architecture that (1) does not change the existing sequential programming models and (2) automatically decides whether to execute PIM operations in memory or processors depending on the locality of data. The key idea is to implement simple in-memory computation using compute-capable memory commands and use specialized instructions, which we call PIM-enabled instructions, to invoke in-memory computation. This allows PIM operations to be interoperable with existing programming models, cache coherence protocols, and virtual memory mechanisms with no modification. In addition, we introduce a simple hardware structure that monitors the locality of data accessed by a PIM-enabled instruction at runtime to adaptively execute the instruction at the host processor (instead of in memory) when the instruction can benefit from large on-chip caches. Consequently, our architecture provides the illusion that PIM operations are executed as if they were host processor instructions. We provide a case study of how ten emerging data-intensive workloads can benefit from our new PIM abstraction and its hardware implementation. Evaluations show that our architecture significantly improves system performance and, more importantly, combines the best parts of conventional and PIM architectures by adapting to data locality of applications.

/pdf/pim-enabled-instructions-a-low-overhead-locality-aware-4gcijahtny.pdf

PIM-enabled instructions: a low-overhead, locality-aware processing-in-memory architecture

A method is described that includes creating a first data pattern access record for a region of system memory in response to a cache miss at a host side cache for a first memory access request. The first memory access request specifies an address within the region of system memory. The method includes fetching a previously existing data access pattern record for the region from the system memory in response to the cache miss. The previously existing data access pattern record identifies blocks of data within the region that have been previously accessed. The method includes pre-fetching the blocks from the system memory and storing the blocks in the cache.

Method and apparatus for pre-fetching data in a system having a multi-level system memory

—Recent years have seen a dramatic increase in the microarchitectural complexity of processors. This increase in complexity presents a twofold challenge for the ﬁeld of computer architecture. First, no individual architect can fully comprehend the complexity of the entire microarchitecture of the core. This leads to increasingly specialized architects, who treat parts of the core outside their particular expertise as black boxes. Sec- ond, with increasing complexity, the ﬁeld becomes decreasingly accessible to new students of the ﬁeld. When learning core microarchitecture, new students must ﬁrst learn the big picture of how the system works in order to understand how the pieces all ﬁt together. The tools used to study microarchitecture experience a similar struggle. As with the microarchitectures they simulate, an increase in complexity reduces accessibility to new users. In this work, we present ChampSim. ChampSim uses a modular design and conﬁgurable structure to achieve a low barrier to entry into the ﬁeld of microarchitecural simulation. ChampSim has shown itself to be useful in multiple areas of research, competition, and education. In this way, we seek to promote access and inclusion despite the increasing complexity of the ﬁeld of computer architecture.

The Championship Simulator: Architectural Simulation for Education and Competition

Systems, apparatuses and methods may provide for a chip that includes a memory array having a plurality of rows corresponding to neurons in a spiking neural network (SNN) and a row decoder coupled to the memory array, wherein the row decoder activates a row in the memory array in response to a pre-synaptic spike in a neuron associated with the row. Additionally, the chip may include a sense amplifier coupled to the memory array, wherein the sense amplifier determines post-synaptic information corresponding to the activated row. In one example, the chip includes a processor to determine a state of a plurality of neurons in the SNN based at least in part on the post-synaptic information and conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons.

In-memory spiking neural networks for memory array architectures

A processor includes a set associative cache and a cache controller. The cache controller makes an initial association between first and second groups of sampled sets in the cache and first and second cache replacement policies. Follower sets in the cache are initially associated with the more conservative of the two policies. Following cache line insertions in a first epoch, the associations between the groups of sampled sets and cache replacement policies are swapped for the next epoch. If the less conservative policy outperforms the more conservative policy during two consecutive epochs, the follower sets are associated with the less conservative policy for the next epoch. Subsequently, if the more conservative policy outperforms the less conservative policy during any epoch, the follower sets are again associated with the more conservative policy. Performance may be measured based the number of cache misses associated with each policy.

System and method for cache replacement using conservative set dueling

An apparatus is described. The apparatus includes a last level cache and a memory controller to interface to a multi-level system memory. The multi-level system memory has a caching level. The apparatus includes a first prediction unit to predict unneeded blocks in the last level cache. The apparatus includes a second prediction unit to predict unneeded blocks in the caching level of the multi-level system memory.

Seth H. Pugsley

Papers

Method and apparatus for pre-fetching data in a system having a multi-level system memory

The Championship Simulator: Architectural Simulation for Education and Competition

In-memory spiking neural networks for memory array architectures

System and method for cache replacement using conservative set dueling

Method and apparatus for unneeded block prediction in a computing system having a last level cache and a multi-level system memory