Computing-in-memory (CIM) architectures have paved the way for energy-efficient artificial intelligence (AI) systems while outperforming von Neumann architectures. In particular, resistive RAM (RRAM)-based CIM has drawn attention due to high cell density, non-volatility, and compatibility with a CMOS process. RRAM also exhibits the feasibility of high-capacity CIM with multi-bit encoding per cell exploiting an appropriate ON/OFF resistance ratio. However, the prior work regarding multi-level RRAM cells mainly focused on achieving higher bit resolution in write without consideration of CIM performance. Thus, the circuit solution to achieve multi-bit encoding per cell dedicated to RRAM-based CIM (RCIM) is of importance to support high-capacity AI systems with reliable CIM performance. This article presents a 256 $\times $ 256 CIM multi-level RRAM macro featuring iterative write with verification to achieve reliable multi-bit encoding per cell and the voltage-sensing readout circuit to surmount the underlying logic ambiguity in RCIM architectures. In addition, we also demonstrate the key design space of a fabricated RRAM array in the write operation with extensive experiments. The test chip fabricated in a Taiwan Semiconductor Manufacturing Company (TSMC) 40-nm CMOS and RRAM process achieves a peak energy efficiency of 118.44 TOPS/W in the ternary-weight multiply-and-accumulate (MAC) operation and demonstrates the feasibility of multi-level RCIM with voltage-sensing RCIM.

A 40-nm 118.44-TOPS/W Voltage-Sensing Compute-in-Memory RRAM Macro With Write Verification and Multi-Bit Encoding

For the non-stop demands for a better and smarter society, the number of electronic devices keeps increasing exponentially; and the computation power, communication data rate, smart sensing capability and intelligence are always not enough. Hardware supports software, while the integrated circuit (IC) is the core of hardware. In this long review paper, we summarize and discuss recent trending IC design directions and challenges, and try to give the readers big/cool pictures on each selected small/hot topics. We divide the trends into the following six categories, namely, 1) machine learning and artificial intelligence (AI) chips, 2) communication ICs, 3) data converters, 4) power converters, 5) imagers and range sensors, 6) emerging directions. Hope you find this paper useful for your future research and works.

Trending IC design directions in 2022

Artificial intelligence (AI) and machine learning (ML) are revolutionizing many fields of study, such as visual recognition, natural language processing, autonomous vehicles, and prediction. Traditional von-Neumann computing architecture with separated processing elements and memory devices have been improving their computing performances rapidly with the scaling of process technology. However, in the era of AI and ML, data transfer between memory devices and processing elements becomes the bottleneck of the system. To address this data movement issue, memory-centric computing takes an approach of merging the memory devices with processing elements so that computations can be done in the same location without moving any data. Processing-In-Memory (PIM) has attracted research community’s attention because it can improve the energy efficiency of memory-centric computing systems substantially by minimizing the data movement. Even though the benefits of PIM are well accepted, its limitations and challenges have not been investigated thoroughly. This paper presents a comprehensive investigation of state-of-the-art PIM research works based on various memory device types, such as static-random-access-memory (SRAM), dynamic-random-access-memory (DRAM), and resistive memory (ReRAM). We will present the overview of PIM designs in each memory type, covering from bit cells, circuits, and architecture. Then, a new software stack standard and its challenges for incorporating PIM with the conventional computing architecture will be discussed. Finally, we will discuss various future research directions in PIM for further reducing the data conversion overhead, improving test accuracy, and minimizing intra-memory data movement.

An Overview of Processing-in-Memory Circuits for Artificial Intelligence and Machine Learning

Computing-in-memory (CIM) is a promising approach to reduce latency and improve the energy efficiency of the multiply-and-accumulate (MAC) operation under a memory wall constraint for artificial intelligence (AI) edge processors. This paper proposes an approach focusing on scalable CIM designs using a new ten-transistor (10T) static random access memory (SRAM) bit-cell. Using the proposed 10T SRAM bit-cell, we present two SRAM-based CIM (SRAM-CIM) macros supporting multibit and binary MAC operations. The first design achieves fully parallel computing and high throughput using 32 parallel binary MAC operations. Advanced circuit techniques such as an input-dependent dynamic reference generator and an input-boosted sense amplifier are presented. Fabricated in 28 nm CMOS process, this design achieves 409.6 GOPS throughput, 1001.7 TOPS/W energy efficiency, and a 169.9 TOPS/mm2 throughput area efficiency. The proposed approach effectively solves previous problems such as writing disturb, throughput, and the power consumption of an analog to digital converter (ADC). The second design supports multibit MAC operation (4-b weight, 4-b input, and 8-b output) to increase the inference accuracy. We propose an architecture that divides 4-b weight and 4-b input multiplication to four 2-b multiplication in parallel, which increases the signal margin by $16\times $ compared to conventional 4-b multiplication. Besides, the capacitive digital-to-analog converter (CDAC) area issue is effectively addressed using the intrinsic bit-line capacitance existing in the SRAM-CIM architecture. The proposed approach of realizing four 2-b parallel multiplication using the CDAC is successfully demonstrated with a modified LeNet-5 neural network. These results demonstrate that the proposed 10T bit-cell is promising for realizing robust and scalable SRAM-CIM designs, which is essential for realizing fully parallel edge computing.

/pdf/10t-sram-computing-in-memory-macros-for-binary-and-multibit-90akorw13z.pdf

10T SRAM Computing-in-Memory Macros for Binary and Multibit MAC Operation of DNN Edge Processors

Abstract Neuromorphic computing, an alternative for von Neumann architecture, requires synapse devices where the data can be stored and computed in the same place. The three-terminal synapse device is attractive for neuromorphic computing due to its high stability and controllability. However, high nonlinearity on weight update, low dynamic range, and incompatibility with conventional CMOS systems have been reported as obstacles for large-scale crossbar arrays. Here, we propose the CMOS compatible gate injection-based field-effect transistor employing thermionic emission to enhance the linear conductance update. The dependence of the linearity on the conduction mechanism is examined by inserting an interfacial layer in the gate stack. To demonstrate the conduction mechanism, the gate current measurement is conducted under varying temperatures. The device based on thermionic emission achieves superior synaptic characteristics, leading to high performance on the artificial neural network simulation as 93.17% on the MNIST dataset. 

/pdf/the-gate-injection-based-field-effect-synapse-transistor-7950i7wq.pdf

The gate injection-based field-effect synapse transistor with linear conductance update for online training

Computing-In-Memory (CIM) techniques which incorporate analog computing inside memory macros have shown significant advantages in computing efficiency for deep learning applications. While earlier CIM macros were limited by lower bit precision, e.g. binary weights in [1], recent works have shown 4-to-8b precision for the weights/inputs and up to 20b for the output values [2], [3]. Sparsity and application features have also been exploited at the system level to further improve the computation efficiency [4], [5]. To enable higher precision, bit-wise operations were commonly utilized [3], [4]. However, there are limitations in existing solutions using the bit-wise operations with SRAM cells. Fig. 15.3.1 shows the summary of challenges and solutions in this work. First, all existing solutions utilize 6T/8T/10T SRAM as a CIM cell, which fundamentally limits the size of the CIM array. In this work, we replace the commonly used SRAM cell with a 3-transistor (3T) analog memory cell, referred as dynamic-analog-RAM (DARAM) which represents a 4b weight value as an analog voltage. This leads to $\sim 10 \times$ reduction in transistor count and achieves an effective CIM single-bit area smaller than the foundry-supplied 6T SRAM cell. Secondly, as no bit-wise calculation is needed in this work, only single-phase MAC operations are performed, removing the throughput degradation associated with previous multi-phase approaches and digital accumulation in [3], [4]. Furthermore, analog linearity issues are mitigated by highly linear time-based activation, removal of matching requirements for critical multi-bit caps [4], [6], and a special read current compensation technique. Thirdly, to mitigate the power bottleneck of ADC or SA, this work applies analog sparsity-based low-power methods, which include a compute-adaptive ADC skipping operation when the analog MAC value is small (or “sparse”) and a special weight-shifting technique, leading to an additional $\sim 2 \times$ reduction in CIM-macro power. We demonstrate the proposed techniques using a 65nm CIM-based CNN accelerator showing state-of-art energy efficiency.

15.3 A 65nm 3T Dynamic Analog RAM-Based Computing-in-Memory Macro and CNN Accelerator with Retention Enhancement, Adaptive Analog Sparsity and 44TOPS/W System Energy Efficiency

Xi Chen

Papers

15.3 A 65nm 3T Dynamic Analog RAM-Based Computing-in-Memory Macro and CNN Accelerator with Retention Enhancement, Adaptive Analog Sparsity and 44TOPS/W System Energy Efficiency