Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.

PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory

IEEE transactions on computer-aided design of integrated circuits and systems : a publication of the IEEE Circuits and Systems Society

Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost To address this cost, a number of quantization schemes have been proposed - but most of these techniques focused on quantizing weights, which are relatively smaller in size compared to activations This paper proposes a novel quantization scheme for activations during training - that enables neural networks to work well with ultra low precision weights and activations without any significant accuracy degradation This technique, PArameterized Clipping acTivation (PACT), uses an activation clipping parameter $\alpha$ that is optimized during training to find the right quantization scale PACT allows quantizing activations to arbitrary bit precisions, while achieving much better accuracy relative to published state-of-the-art quantization schemes We show, for the first time, that both weights and activations can be quantized to 4-bits of precision while still achieving accuracy comparable to full precision networks across a range of popular models and datasets We also show that exploiting these reduced-precision computational units in hardware can enable a super-linear improvement in inferencing performance due to a significant reduction in the area of accelerator compute engines coupled with the ability to retain the quantized model and activation data in on-chip memories

/pdf/pact-parameterized-clipping-activation-for-quantized-neural-544x8orkhr.pdf

PACT: Parameterized Clipping Activation for Quantized Neural Networks

Convolution neural networks (CNNs) are the heart of deep learning applications. Recent works PRIME [1] and ISAAC [2] demonstrated the promise of using resistive random access memory (ReRAM) to perform neural computations in memory. We found that training cannot be efficiently supported with the current schemes. First, they do not consider weight update and complex data dependency in training procedure. Second, ISAAC attempts to increase system throughput with a very deep pipeline. It is only beneficial when a large number of consecutive images can be fed into the architecture. In training, the notion of batch (e.g. 64) limits the number of images can be processed consecutively, because the images in the next batch need to be processed based on the updated weights. Third, the deep pipeline in ISAAC is vulnerable to pipeline bubbles and execution stall. In this paper, we present PipeLayer, a ReRAM-based PIM accelerator for CNNs that support both training and testing. We analyze data dependency and weight update in training algorithms and propose efficient pipeline to exploit inter-layer parallelism. To exploit intra-layer parallelism, we propose highly parallel design based on the notion of parallelism granularity and weight replication. With these design choices, PipeLayer enables the highly pipelined execution of both training and testing, without introducing the potential stalls in previous work. The experiment results show that, PipeLayer achieves the speedups of 42.45x compared with GPU platform on average. The average energy saving of PipeLayer compared with GPU implementation is 7.17x.

PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning

High-performance parallel computer architecture and systems have been improved at a phenomenal rate. In the meantime, VLSI computer-aided design (CAD) software for multibillion-transistor IC design has become increasingly complex and requires prohibitively high computational resources. Recent studies have shown that, numerous CAD problems, with their high computational complexity, can greatly benefit from the fast-increasing parallel computation capabilities. However, parallel programming imposes big challenges for CAD applications. Fully exploiting the computational power of emerging general-purpose and domain-specific multicore/many-core processor systems, calls for fundamental research and engineering practice across every stage of parallel CAD design, from algorithm exploration, programming models, design-time and run-time environment, to CAD applications, such as verification, optimization, and simulation. This journal special section will cover recent progress on parallel CAD research, including algorithm foundations, programming models, parallel architectural-specific optimization, and verification. More specifically, papers with in-depth and extensive coverage of the following topics will be considered, as well as other topics relevant to the design of parallel CAD algorithms and software tools. 1. Parallel algorithm design and specification for CAD applications 2. Parallel programming models and languages of particular use in CAD 3. Runtime support and performance optimization for CAD applications 4. Parallel architecture-specific design and optimization for CAD applications 5. Parallel program debugging and verification techniques particularly relevant for CAD The papers should be submitted via the Manuscript Central website and should adhere to standard ACM TODAES formatting requirements (http://todaes.acm.org/). The page count limit is 25.

Parallel CAD: Algorithm Design and Programming Special Section Call for Papers TODAES: ACM Transactions on Design Automation of Electronic Systems

In this article, we propose Aggregation-in-Memory (AIM), a new processing-in-memory system designed for energy efficiency and near-term adoption. In order to efficiently perform aggregation, we implement simple aggregation operations in main memory and develop a locality-adaptive host architecture for in-memory aggregation, called cache-conscious aggregation. Through this, AIM executes aggregation at the most energy-efficient location among all levels of the memory hierarchy. Moreover, AIM minimally changes existing sequential programming models and provides fully automated compiler toolchain, thereby allowing unmodified legacy software to use AIM. Evaluations show that AIM greatly improves the energy efficiency of main memory and the system performance.

https://dl.acm.org/doi/pdf/10.1145/2994149

AIM: Energy-Efficient Aggregation Inside the Memory Hierarchy

This brief presents an approach to reduce static power consumption in peripheral circuits of spin-transfer torque RAM (STT-RAM) instruction caches. It is based on the key observation that only a small set of instructions is accessed inside a program loop. We propose to add a small static RAM cache called loop cache between the processor and the L1 instruction cache made of STT-RAM. When the loop cache has an entire loop cached, the L1 instruction cache can be turned off to save energy during the execution of the loop. Experimental results show that the proposed approach achieves 49% reduction in energy consumption over the STT-RAM baseline.

LASIC: Loop-Aware Sleepy Instruction Caches Based on STT-RAM Technology

In mobile systems, such as smartphones, most of storage writes are incurred by the SQLite database (DB) system. These writes consist of two parts: writes to original data (e.g., SQLite DB file) and journaling-induced writes. In this paper, we first report our observations on the characteristics of these writes: (1) journaling writes dominate storage write, (2) overwrites are frequent in mobile storage, and most importantly, and (3) those overwrites change only a small portion of data. Based on these observations, we propose to employ small capacitor-backed nonvolatile write buffer and use it to durably store small writes without journaling. This allows us to completely avoid journal writes for those small writes, which contributes to significant reduction in journaling-induced writes while still providing the same level of data consistency and durability. In our mechanism, it is critical to minimize the capacity of nonvolatile write buffer so that the backing capacitor does not violate stringent resource/size requirements of mobile systems. Therefore, we propose three optimizations that make the best use of small write buffer: (1) write buffer that manages only the difference between old and new data, (2) a dynamic method to determine which data to store in the write buffer, and (3) an incremental flush policy, which controls the number of write buffer entries to be flushed. The experimental results show 92.4% and 85.1% storage write reductions on average when running single and multiple mobile applications, respectively, with only 8 KB write buffer and a tiny capacitor.

Nonvolatile Write Buffer-Based Journaling Bypass for Storage Write Reduction in Mobile Devices

Extensible processors provide an efficient mechanism to boost the performance of the whole system without losing much flexibility. However, due to the intense demand of low cost and power consumption, customizing an embedded system has been more difficult than ever. In this paper, we present a framework for custom instruction generation considering both area constraints and resource sharing. We also present how we can speed up the process through pruning and library-based design space exploration.

Fast Generation of Multiple Custom Instructions under Area Constraints

A semiconductor memory device including a non-volatile memory and a cache memory and a computer system comprising the same are disclosed. The cache memory according to an embodiment of the present invention comprises: a first storage area including a plurality of data blocks for storing data; and a second storage area for including m number of ECC blocks corresponding to n number of data blocks, wherein each of the first and the second storage areas includes non-volatile memory cells. At least one ECC block stores first information indicating an ECC parity for correcting an error bit of the corresponding data block and a position of the corresponding data block.

Junwhan Ahn

Papers

AIM: Energy-Efficient Aggregation Inside the Memory Hierarchy

LASIC: Loop-Aware Sleepy Instruction Caches Based on STT-RAM Technology

Nonvolatile Write Buffer-Based Journaling Bypass for Storage Write Reduction in Mobile Devices

Fast Generation of Multiple Custom Instructions under Area Constraints

Semiconductor memory device including non-volatile memory and cache memory and computer systam having the same