Large-scale matrix-vector multiplications, which dominate in deep neural networks (DNNs), are limited by data movement in modern VLSI technologies. This paper addresses data movement via an in-memory-computing accelerator that employs charged-domain mixed-signal operation for enhancing compute SNR and, thus, scalability. The architecture supports analog/binary input activation (IA)/weight first layer (FL) and binary/binary IA/weight hidden layers (HLs), with batch normalization and input–output (IO) (buffering) circuitry to enable cascading, if desired, for realizing different DNN layers. The architecture is arranged as $8\times 8=64$ in-memory-computing neuron tiles, supporting up to 512, $3\times 3\times 512$ -input HL neurons and 64, $3\times 3\times 3$ -input FL neurons, configurable via tile-level clock gating. In-memory computing is achieved using an 8T bit cell with overlaying metal-oxide-metal (MOM) capacitor, yielding a structure having $1.8\times $ the area of a standard 6T bit cell. Implemented in 65-nm CMOS, the design achieves HLs/FL energy efficiency of 866/1.25 TOPS/W and throughput of 18876/43.2 GOPS (1498/3.43 GOPS/mm2), when implementing convolution layers; and 658/0.95 TOPS/W, 9438/10.47 GOPS (749/0.83 GOPS/mm2), when implementing convolution followed by batch normalization layers. Several large-scale neural networks are demonstrated, showing performance on standard benchmarks (MNIST, CIFAR-10, and SVHN) equivalent to ideal digital computing.

A 64-Tile 2.4-Mb In-Memory-Computing CNN Accelerator Employing Charge-Domain Compute

This article presents C3SRAM, an in-memory-computing SRAM macro. The macro is an SRAM module with the circuits embedded in bitcells and peripherals to perform hardware acceleration for neural networks with binarized weights and activations. The macro utilizes analog-mixed-signal (AMS) capacitive-coupling computing to evaluate the main computations of binary neural networks, binary-multiply-and-accumulate operations. Without the need to access the stored weights by individual row, the macro asserts all its rows simultaneously and forms an analog voltage at the read bitline node through capacitive voltage division. With one analog-to-digital converter (ADC) per column, the macro realizes fully parallel vector–matrix multiplication in a single cycle. The network type that the macro supports and the computing mechanism it utilizes are determined by the robustness and error tolerance necessary in AMS computing. The C3SRAM macro is prototyped in a 65-nm CMOS. It demonstrates an energy efficiency of 672 TOPS/W and a speed of 1638 GOPS (20.2 TOPS/mm2), achieving 3975 $\times $ better energy–delay product than the conventional digital baseline performing the same operation. The macro achieves 98.3% accuracy for MNIST and 85.5% for CIFAR-10, which is among the best in-memory computing works in terms of energy efficiency and inference accuracy tradeoff.

C3SRAM: An In-Memory-Computing SRAM Macro Based on Robust Capacitive Coupling Computing Mechanism

Through virtualization, single physical data planes can logically support multiple networking contexts. We propose HyPer4 as a portable virtualization solution. HyPer4 provides a general purpose program, written in the P4 dataplane programming language, that may be dynamically configured to adopt behavior that is functionally equivalent to other P4 programs. HyPer4 extends, through software, the following features to diverse P4-capable devices: the ability to logically store multiple programs and either run them in parallel (network slicing) or as hot-swappable snapshots; and virtual networking between programs (supporting program composition or multi-tenant service interaction). HyPer4 permits modifying the set of programs, as well as the virtual network connecting them, at runtime, without disrupting currently active programs. We show that realistic ASICs-based hardware would be capable of running HyPer4 today.

https://dl.acm.org/doi/pdf/10.1145/2999572.2999607

HyPer4: Using P4 to Virtualize the Programmable Data Plane

Energy efficient hardware implementation of artificial neural network is challenging due the "memory-wall" bottleneck. Neuromorphic computing promises to address this challenge by eliminating data movement to and from off-chip memory devices. Emerging non-volatile memory devices that exhibit gradual changes in resistivity are a key enabler of in-memory computing - a type of neuromorphic computing. In this paper, we present a review of some of the non-volatile memory devices (RRAM, CBRAM, PCM) commonly used in neuromorphic application. The review focuses on the trade-off between device parameters such as retention, endurance, device-to-device variation, speed and resistance levels, and the interplay with target applications. This work aims at providing guidance for finding the optimized resistive memory devices material stack suitable for neuromorphic application.

https://iopscience.iop.org/article/10.1088/1361-6463/aaf784/pdf

Device and materials requirements for neuromorphic computing

Racetrack memory (RTM) is a novel spintronic memory-storage technology that has the potential to overcome fundamental constraints of existing memory and storage devices. It is unique in that its core differentiating feature is the movement of data, which is composed of magnetic domain walls (DWs), by short current pulses. This enables more data to be stored per unit area compared to any other current technologies. On the one hand, RTM has the potential for mass data storage with unlimited endurance using considerably less energy than today’s technologies. On the other hand, RTM promises an ultrafast nonvolatile memory competitive with static random access memory (SRAM) but with a much smaller footprint. During the last decade, the discovery of novel physical mechanisms to operate RTM has led to a major enhancement in the efficiency with which nanoscopic, chiral DWs can be manipulated. New materials and artificially atomically engineered thin-film structures have been found to increase the speed and lower the threshold current with which the data bits can be manipulated. With these recent developments, RTM has attracted the attention of the computer architecture community that has evaluated the use of RTM at various levels in the memory stack. Recent studies advocate RTM as a promising compromise between, on the one hand, power-hungry, volatile memories and, on the other hand, slow, nonvolatile storage. By optimizing the memory subsystem, significant performance improvements can be achieved, enabling a new era of cache, graphical processing units, and high capacity memory devices. In this article, we provide an overview of the major developments of RTM technology from both the physics and computer architecture perspectives over the past decade. We identify the remaining challenges and give an outlook on its future.

/pdf/magnetic-racetrack-memory-from-physics-to-the-cusp-of-5brh05qnrb.pdf

Magnetic Racetrack Memory: From Physics to the Cusp of Applications Within a Decade

In this paper, we pave a novel way towards the concept of bit-wise In-Memory Convolution Engine (IMCE) that could implement the dominant convolution computation of Deep Convolutional Neural Networks (CNN) within memory. IMCE employs parallel computational memory sub-array as a fundamental unit based on our proposed Spin Orbit Torque Magnetic Random Access Memory (SOT-MRAM) design. Then, we propose an accelerator system architecture based on IMCE to efficiently process low bit-width CNNs. This architecture can be leveraged to greatly reduce energy consumption dealing with convolutional layers and also accelerate CNN inference. The device to architecture co-simulation results show that the proposed system architecture can process low bit-width AlexNet on ImageNet data-set favorably with 785.25μJ/img, which consumes ∼3× less energy than that of recent RRAM based counterpart. Besides, the chip area is ∼4× smaller.

https://par.nsf.gov/servlets/purl/10059775

IMCE: energy-efficient bit-wise in-memory convolution engine for deep neural network

In this paper, a key-controlled hybrid spin-CMOS polymorphic logic gate using a novel 5 terminal magnetic domain wall motion device is proposed. The proposed hybrid polymorphic gate is able to perform a full set of 2-input Boolean logic functions (i.e. AND/NAND, OR/NOR, NOT, XOR/XNOR) by configuring the applied keys. The SPICE device-circuit co-simulation indicates that a full adder design using our proposed polymorphic logic gate shows 74.23% power reduction and 7.14% transistor count reduction compared with traditional CMOS full adder design. Our proposed polymorphic gate could be a promising hardware security primitive to address IC counterfeiting or reverse engineering by logic locking and polymorphic transformation. To summarize, by providing zero leakage power, low dynamic power consumption, compactness and polymorphism to logic circuits, our proposed design can thrive a new paradigm for future power efficient and secured computing platform.

Hybrid Polymorphic Logic Gate with 5-Terminal Magnetic Domain Wall Motion Device

This paper presents a new Reconfigurable dualmode In-Memory Processing Architecture based on spin Hall effect-driven domain wall motion device called RIMPA. In this architecture, a portion of spintronic memory array can be reconfigured to either non-volatile memory or in-memory logic. Accordingly, computation can be performed within memory without long distance data transfer or large in-memory logic area overhead concerning conventional Von-Neumann or in-memory computing architecture, respectively. The device to architecture simulation results show that, with 17% area increase, RIMPA improves the operating energy by 72.2% as compared with the conventional non-volatile in-memory logic schemes. We show that the Advanced Encryption Standard (AES) algorithm which is widely used in secure big data storage, can be efficiently mapped to RIMPA with 68.8% and 20.8% energy saving in comparison to CMOS-ASIC and recent DW-AES implementations, respectively.

RIMPA: A New Reconfigurable Dual-Mode In-Memory Processing Architecture with Spin Hall Effect-Driven Domain Wall Motion Device

In this paper we propose a Highly Flexible In-Memory (HieIM) computing platform using STT MRAM, which can be leveraged to implement Boolean logic functions without sacrificing memory functionality. It could pre-process data within memory to further reduce power hungry long distance communication between memory and processing units as in Von-Neumann computing system. HieIM can implement all the Boolean logic functions (AND/NAND, OR/NOR, XOR/XNOR) between any two cells in the same memory array, thus overcoming the 'operand locality' problem in contemporary in-memory computing platform designs. To investigate the performance of HieIM, we test in-memory bulk bit-wise Boolean logic operations using different vector datasets, which shows ∼ 8× energy saving and ∼ 5× speedup compared to recent DRAM based in-memory computing platform. We further implement an in-memory data encryption engine design based on HieIM as another case study. With AES algorithm, it shows 51.5% and 68.9% lower energy consumption compared to CMOS-ASIC and CMOL based implementations, respectively.

https://par.nsf.gov/servlets/purl/10059776

HieIM: highly flexible in-memory computing using STT MRAM

In this paper, we propose a novel Spin Orbit Torque Magnetic Random Access Memory (SOT-MRAM) array design that could simultaneously work as non-volatile memory and implement a reconfigurable in-memory logic (AND, OR) without add-on logic circuits to memory chip as in traditional logic-in-memory designs. The computed logic output could be simply read out like a normal MRAM bit-cell using the shared memory peripheral circuits. Such intrinsic in-memory logic could be used to process data within memory to greatly reduce power-hungry and long distance data communication in conventional Von-Neumann computing systems. We further employ in-memory data encryption using Advanced Encryption Standard (AES) algorithm as a case study to demonstrate the efficiency of the proposed design. The device to architecture co-simulation results show that the proposed design can achieve 70.15% and 80.87% lower energy consumption compared to CMOS-ASIC and CMOL-AES implementations, respectively. It offers almost similar energy consumption as recent DW-AES implementation, but with 60.65% less area overhead.

Farhana Parveen

Papers

IMCE: energy-efficient bit-wise in-memory convolution engine for deep neural network

Hybrid Polymorphic Logic Gate with 5-Terminal Magnetic Domain Wall Motion Device

RIMPA: A New Reconfigurable Dual-Mode In-Memory Processing Architecture with Spin Hall Effect-Driven Domain Wall Motion Device

HieIM: highly flexible in-memory computing using STT MRAM

Low power in-memory computing based on dual-mode SOT-MRAM