Data movement between the processing units and the memory in traditional von Neumann architecture is creating the “memory wall” problem. To bridge the gap, two approaches, the memory-rich processor (more on-chip memory) and the compute-capable memory (processing-in-memory) have been studied. However, the first one has strong computing capability but limited memory capacity/bandwidth, whereas the second one is the exact the opposite.To address the challenge, we propose DRISA, a DRAM-based Reconfigurable In-Situ Accelerator architecture, to provide both powerful computing capability and large memory capacity/bandwidth. DRISA is primarily composed of DRAM memory arrays, in which every memory bitline can perform bitwise Boolean logic operations (such as NOR). DRISA can be reconfigured to compute various functions with the combination of the functionally complete Boolean logic operations and the proposed hierarchical internal data movement designs. We further optimize DRISA to achieve high performance by simultaneously activating multiple rows and subarrays to provide massive parallelism, unblocking the internal data movement bottlenecks, and optimizing activation latency and energy. We explore four design options and present a comprehensive case study to demonstrate significant acceleration of convolutional neural networks. The experimental results show that DRISA can achieve 8.8× speedup and 1.2× better energy efficiency compared with ASICs, and 7.7× speedup and 15× better energy efficiency over GPUs with integer operations.CCS CONCEPTS• Hardware → Dynamic memory; • Computer systems organization → reconfigurable computing; Neural networks;

DRISA: a DRAM-based Reconfigurable In-Situ Accelerator

A semiconductor memory device comprises memory cells, a bitline connected to the memory cells, a read circuit including a precharge circuit, and a first transistor connected between the bitline and the read circuit, wherein a first voltage is applied to a gate of the first transistor when the precharge circuit precharges the bitline, and a second voltage which is different from the first voltage is applied to the gate of the first transistor when the read circuit senses a change in a voltage of the bitline.

Semiconductor memory device.

The unprecedented growth in deep neural networks (DNN) size has led to massive amounts of data movement from off-chip memory to on-chip processing cores in modern machine learning (ML) accelerators. Compute-in-memory (CIM) designs performing analog DNN computations within a memory array, along with peripheral mixed-signal circuits, are being explored to mitigate this memory-wall bottleneck: consisting of memory latency and energy overhead. Embedded-dynamic random-access memory (eDRAM) [1], [2], which integrates the 1T1C (T=Transistor, C=Capacitor) DRAM bitcell monolithically along with high-performance logic transistors and interconnects, can enable custom CIM designs. It offers the densest embedded bitcell, a low pJ/bit access energy, a low soft error rate, high-endurance, high-performance, and high-bandwidth: all desired attributes for ML accelerators. In addition, the intrinsic charge sharing operation during a dynamic memory access can be used effectively to perform analog CIM computations: by reconfiguring existing eDRAM columns as charge domain circuits, thus, greatly minimizing peripheral circuit area and power overhead. Configuring a part of eDRAM as a CIM engine (for data conversion, DNN computations, and weight storage) and retaining the remaining part as a regular memory (for inputs, gradients during training, and non-CIM workload data) can help to meet the layer/kernel dependent variable storage needs during a DNN inference/training step. Thus, the high cost/bit of eDRAM can be amortized by repurposing part of existing large capacity, level-4 eDRAM caches [7] in high-end microprocessors, into large-scale CIM engines.

16.2 eDRAM-CIM: Compute-In-Memory Design with Reconfigurable Embedded-Dynamic-Memory Array Realizing Adaptive Data Converters and Charge-Domain Computing

Linear algebra is ubiquitous across virtually every field of science and engineering, from climate modeling to macroeconomics. This ubiquity makes linear algebra a prime candidate for hardware acceleration, which can improve both the run time and the energy efficiency of a wide range of scientific applications. Recent work on memristive hardware accelerators shows significant potential to speed up matrix-vector multiplication (MVM), a critical linear algebra kernel at the heart of neural network inference tasks. Regrettably, the proposed hardware is constrained to a narrow range of workloads: although the eight- to 16-bit computations afforded by memristive MVM accelerators are acceptable for machine learning, they are insufficient for scientific computing where high-precision floating point is the norm. This paper presents the first proposal to enable scientific computing on memristive crossbars. Three techniques are explored---reducing overheads by exploiting exponent range locality, early termination of fixed-point computation, and static operation scheduling---that together enable a fixed-point memristive accelerator to perform high-precision floating point without the exorbitant cost of naive floating-point emulation on fixed-point hardware. A heterogeneous collection of crossbars with varying sizes is proposed to efficiently handle sparse matrices, and an algorithm for mapping the dense subblocks of a sparse matrix to an appropriate set of crossbars is investigated. The accelerator can be combined with existing GPU-based systems to handle datasets that cannot be efficiently handled by the memristive accelerator alone. The proposed optimizations permit the memristive MVM concept to be applied to a wide range of problem domains, respectively improving the execution time and energy dissipation of sparse linear solvers by 10.3x and 10.9x over a purely GPU-based system.

Enabling scientific computing on memristive accelerators

Recently proposed DRAM based memory-centric architectures have demonstrated their great potentials in addressing the memory wall challenge of modern computing systems. Such architectures exploit charge sharing of multiple rows to enable in-memory bitwise operations. However, existing designs rely heavily on reserved rows to implement computation, which introduces high data movement overhead, large operation latency, large energy consumption, and low operation reliability. In this paper, we propose ELP2IM, an efficient and low power processing in-memory architecture, to address the above issues. ELP2IM utilizes two stable states of sense amplifiers in DRAM subarrays so that it can effectively reduce the number of intra-subarray data movements as well as the number of concurrently opened DRAM rows, which exhibits great performance and energy consumption advantages over existing designs. Our experimental results show that the power efficiency of ELP2IM is more than 2X improvement over the state-of-the-art DRAM based memory-centric designs in real application.

ELP2IM: Efficient and Low Power Bitwise Operation Processing in DRAM

A 1.1 Mb embedded DRAM macro (eDRAM), for next-generation IBM SOI processors, employs 14 nm FinFET logic technology with $\hbox{0.0174}~\mu\hbox{m}^{2}$ deep-trench capacitor cell. A Gated-feedback sense amplifier enables a high voltage gain of a power-gated inverter at mid-level input voltage, while supporting 66 cells per local bit-line. A dynamic-and-gate-thin-oxide word-line driver that tracks standard logic process variation improves the eDRAM array performance with reduced area. The 1.1 $~$ Mb macro composed of 8 $\times$ 2 72 Kb subarrays is organized with a center interface block architecture, allowing 1 ns access latency and 1 ns bank interleaving operation using two banks, each having 2 ns random access cycle. 5 GHz operation has been demonstrated in a system prototype, which includes 6 instances of 1.1 Mb eDRAM macros, integrated with an array-built-in-self-test engine, phase-locked loop (PLL), and word-line high and word-line low voltage generators. The advantage of the 14 nm FinFET array over the 22 nm array was confirmed using direct tester control of the 1.1 Mb eDRAM macros integrated in 16 Mb inline monitor.

A 14 nm 1.1 Mb Embedded DRAM Macro With 1 ns Access

A method and circuit for adjusting the read margin of a self-timed memory array. The electronic circuit, including: a memory cell array including a sense amplifier self-timed decode circuit adapted to set a base read time delay of the memory cell array; and a read delay adjustment circuit coupled to the memory cell array, the read delay adjustment circuit adapted to adjust the base read time delay of the memory array based on an operating frequency of the memory cell array.

Method and circuit for dynamic read margin control of a memory array

A memory device includes a plurality of memory cells arranged in rows and columns. The memory cells are divided into a plurality of sub-arrays. The memory cell further includes a plurality of word lines connecting rows of the memory cells, and a plurality of bit line pairs connecting columns of the memory cells. An address transition detect (ATD) circuit detects an address transition for a selected memory cell and generates an ATD pulse in response thereto. A respective bit line precharge circuit is associated with each of the plurality of sub-arrays. An ATD pulse distribution circuit distributes the ATD pulse to only a selected sub-array containing the selected memory cell to activate only the bit line precharge circuit of the selected sub-array and not activate precharge circuits of other non-selected sub-arrays.

Memory device having reduced power requirements and associated methods

IBM introduced trench capacitor eDRAM into its high performance microprocessors beginning with 45nm and Power 7 [1] to provide a higher density cache without chip crossings. Whereas the 45 and 32nm designs employ a micro sense amplifier [2] and three-level bitline hierarchy, the design implemented for 22nm utilizes a higher gain sense amplifier and two-level bitline architecture that together provide significant reductions in area, latency, and power. This 22nm design style has been migrated into a 14nm FinFET [3] learning vehicle, complete with an ABIST engine, wordline charge pumps (VPP and VWL), and padcage interface circuitry.

Dongho Lee

Papers

A 14 nm 1.1 Mb Embedded DRAM Macro With 1 ns Access

Method and circuit for dynamic read margin control of a memory array

Memory device having reduced power requirements and associated methods

17.4 A 14nm 1.1Mb embedded DRAM macro with 1ns access