Guided by brain-like ‘spiking’ computational frameworks, neuromorphic computing—brain-inspired computing for machine intelligence—promises to realize artificial intelligence while reducing the energy requirements of computing platforms. This interdisciplinary field began with the implementation of silicon circuits for biological neural routines, but has evolved to encompass the hardware implementation of algorithms with spike-based encoding and event-driven representations. Here we provide an overview of the developments in neuromorphic computing for both algorithms and hardware and highlight the fundamentals of learning and hardware frameworks. We discuss the main challenges and the future prospects of neuromorphic computing, with emphasis on algorithm–hardware codesign. The authors review the advantages and future prospects of neuromorphic computing, a multidisciplinary engineering concept for energy-efficient artificial intelligence with brain-inspired functionality.

Towards spike-based machine intelligence with neuromorphic computing.

Traditional von Neumann computing systems involve separate processing and memory units. However, data movement is costly in terms of time and energy and this problem is aggravated by the recent explosive growth in highly data-centric applications related to artificial intelligence. This calls for a radical departure from the traditional systems and one such non-von Neumann computational approach is in-memory computing. Hereby certain computational tasks are performed in place in the memory itself by exploiting the physical attributes of the memory devices. Both charge-based and resistance-based memory devices are being explored for in-memory computing. In this Review, we provide a broad overview of the key computational primitives enabled by these memory devices as well as their applications spanning scientific computing, signal processing, optimization, machine learning, deep learning and stochastic computing. This Review provides an overview of memory devices and the key computational primitives for in-memory computing, and examines the possibilities of applying this computing approach to a wide range of applications.

Memory devices and applications for in-memory computing

Many important applications trigger bulk bitwise operations, i.e., bitwise operations on large bit vectors. In fact, recent works design techniques that exploit fast bulk bitwise operations to accelerate databases (bitmap indices, BitWeaving) and web search (BitFunnel). Unfortunately, in existing architectures, the throughput of bulk bitwise operations is limited by the memory bandwidth available to the processing unit (e.g., CPU, GPU, FPGA, processing-in-memory).To overcome this bottleneck, we propose Ambit, an Accelerator-in-Memory for bulk bitwise operations. Unlike prior works, Ambit exploits the analog operation of DRAM technology to perform bitwise operations completely inside DRAM, thereby exploiting the full internal DRAM bandwidth. Ambit consists of two components. First, simultaneous activation of three DRAM rows that share the same set of sense amplifiers enables the system to perform bitwise AND and OR operations. Second, with modest changes to the sense amplifier, the system can use the inverters present inside the sense amplifier to perform bitwise NOT operations. With these two components, Ambit can perform any bulk bitwise operation efficiently inside DRAM. Ambit largely exploits existing DRAM structure, and hence incurs low cost on top of commodity DRAM designs (1% of DRAM chip area). Importantly, Ambit uses the modern DRAM interface without any changes, and therefore it can be directly plugged onto the memory bus.Our extensive circuit simulations show that Ambit works as expected even in the presence of significant process variation. Averaged across seven bulk bitwise operations, Ambit improves performance by 32X and reduces energy consumption by 35X compared to state-of-the-art systems. When integrated with Hybrid Memory Cube (HMC), a 3D-stacked DRAM with a logic layer, Ambit improves performance of bulk bitwise operations by 9.7X compared to processing in the logic layer of the HMC. Ambit improves the performance of three real-world data-intensive applications, 1) database bitmap indices, 2) BitWeaving, a technique to accelerate database scans, and 3) bit-vector-based implementation of sets, by 3X-7X compared to a state-of-the-art baseline using SIMD optimizations. We describe four other applications that can benefit from Ambit, including a recent technique proposed to speed up web search. We believe that large performance and energy improvements provided by Ambit can enable other applications to use bulk bitwise operations.CCS CONCEPTS• Computer systems organization → Single instruction, multiple data; • Hardware → Hardware accelerator; • Hardware → Dynamic memory;

/pdf/ambit-in-memory-accelerator-for-bulk-bitwise-operations-2t2ntwty69.pdf

Ambit: in-memory accelerator for bulk bitwise operations using commodity DRAM technology

This paper presents a machine-learning classifier where computations are performed in a standard 6T SRAM array, which stores the machine-learning model. Peripheral circuits implement mixed-signal weak classifiers via columns of the SRAM, and a training algorithm enables a strong classifier through boosting and also overcomes circuit nonidealities, by combining multiple columns. A prototype 128 $\times $ 128 SRAM array, implemented in a 130-nm CMOS process, demonstrates ten-way classification of MNIST images (using image-pixel features downsampled from 28 $\times $ 28 = 784 to 9 $\times $ 9 = 81, which yields a baseline accuracy of 90%). In SRAM mode (bit-cell read/write), the prototype operates up to 300 MHz, and in classify mode, it operates at 50 MHz, generating a classification every cycle. With accuracy equivalent to a discrete SRAM/digital-MAC system, the system achieves ten-way classification at an energy of 630 pJ per decision, 113 times lower than a discrete system with standard training algorithm and 13 times lower than a discrete system with the proposed training algorithm.

In-Memory Computation of a Machine-Learning Classifier in a Standard 6T SRAM Array

This paper presents the Compute Cache architecturethat enables in-place computation in caches. ComputeCaches uses emerging bit-line SRAM circuit technology to repurpose existing cache elements and transforms them into active very large vector computational units. Also, it significantlyreduces the overheads in moving data between different levelsin the cache hierarchy. Solutions to satisfy new constraints imposed by ComputeCaches such as operand locality are discussed. Also discussedare simple solutions to problems in integrating them into aconventional cache hierarchy while preserving properties suchas coherence, consistency, and reliability. Compute Caches increase performance by 1.9× and reduceenergy by 2.4× for a suite of data-centric applications, includingtext and database query processing, cryptographic kernels, and in-memory checkpointing. Applications with larger fractionof Compute Cache operations could benefit even more, asour micro-benchmarks indicate (54× throughput, 9× dynamicenergy savings).

Compute Caches

Conventional content addressable memory (BCAM and TCAM) uses specialized 10T/16T bit cells that are significantly larger than 6T SRAM cells. A new BCAM/TCAM is proposed that can operate with standard push-rule 6T SRAM cells, reducing array area by 2–5× and allowing reconfiguration of the SRAM as a CAM. In this way, chip area and overall capacitance can be reduced, leading to higher energy efficiency for search operations. In addition, the configurable memory can perform bit-wise logical operations: “AND” and “NOR” on two or more words stored within the array. Thus, the configurable memory with CAM and logical function capability can be used to off-load specific computational operations to the memory, improving system performance and efficiency. Using a 6T 28 nm FDSOI SRAM bit cell, the 64×64 (4 kb) BCAM achieves 370 MHz at 1 V and consumes 0.6 fJ/search/bit. A logical operation between two 64 bit words achieves 787 MHz at 1 V.

A 28 nm Configurable Memory (TCAM/BCAM/SRAM) Using Push-Rule 6T Bit Cell Enabling Logic-in-Memory

This paper presents a 4+2T SRAM for embedded searching and in-memory-computing applications. The proposed SRAM cell uses the n-well as the write wordline to perform write operations and eliminate the write access transistors, achieving 15% area saving compared with conventional 8T SRAM. The decoupled differential read paths significantly improve read noise margin, and therefore reliable multi-word activation can be enabled to perform in-memory Boolean logic functions. Reconfigurable differential sense amplifiers are employed to realize fast normal read or multi-functional logic operations. Moreover, the proposed 4 + 2T SRAM can be reconfigured as binary content-addressable memory (BCAM) or ternary content-addressable memory (TCAM) for searching operations, achieving 0.13 fJ/search/bit at 0.35 V. The chip is fabricated in 55-nm deeply depleted channel technology. The area efficiency is 65% for a $128 \times 128$ pushed-rule array including all peripherals such as column-wise sense amplifier for read/logic and row-wise sense amplifier for BCAM/TCAM operations. Forty dies across five wafers in different corners are measured, showing a worst-case read/write $V_{\mathrm {DDmin}}$ of 0.3 V.

A 4 + 2T SRAM for Searching and In-Memory Computing With 0.3-V $V_{\mathrm {DDmin}}$

We present a discontinuous harvesting approach for switch capacitor dc–dc converters that enables ultralow-power energy harvesting. Smart sensor applications rely on ultralow-power energy harvesters to scavenge energy across a wide range of ambient power levels and charge the battery. Based on the key observation that energy source efficiency is higher than charge pump efficiency, we present a discontinuous harvesting technique that decouples the two efficiencies for a better tradeoff. By slowly accumulating charge on an input capacitor and then transferring it to a battery in burst mode, dc–dc converter switching and leakage losses can be optimally traded off with the loss incurred by nonideal maximum power point tracking operation. Harvester duty cycle is automatically modulated instead of charge pump operating frequency to match with the energy source input power level. The harvester uses a hybrid structure called a moving-sum charge pump for low startup energy upon a mode switch, an automatic conversion ratio modulator based on conduction loss optimization for fast conversion ratio increment, and a 40% end-to-end efficiency from 113 pW to 1.5 $\mu \text{W}$ with 20-pW minimum harvestable input power.

A 20-pW Discontinuous Switched-Capacitor Energy Harvester for Smart Sensor Applications

A 4+2T SRAM is proposed that offers searching and logic functions. The cell uses the N-well as the write wordline (WL) and eliminates the access transistors. Decoupled read paths enable reliable multi-word activation for in-memory Boolean logic functions. The SRAM can reconfigure to BCAM/TCAM for searching operations, with 0.13fJ/search/bit at 0.35V. Forty test chips in 55nm deeply depleted channel (DDC) technology achieve worst-case 0.3 V VDDmin.

Supreet Jeloka

Papers

Compute Caches

A 28 nm Configurable Memory (TCAM/BCAM/SRAM) Using Push-Rule 6T Bit Cell Enabling Logic-in-Memory

A 4 + 2T SRAM for Searching and In-Memory Computing With 0.3-V $V_{\mathrm {DDmin}}$

A 20-pW Discontinuous Switched-Capacitor Energy Harvester for Smart Sensor Applications

A 0.3V VDDmin 4+2T SRAM for searching and in-memory computing using 55nm DDC technology