Traditional von Neumann computing systems involve separate processing and memory units. However, data movement is costly in terms of time and energy and this problem is aggravated by the recent explosive growth in highly data-centric applications related to artificial intelligence. This calls for a radical departure from the traditional systems and one such non-von Neumann computational approach is in-memory computing. Hereby certain computational tasks are performed in place in the memory itself by exploiting the physical attributes of the memory devices. Both charge-based and resistance-based memory devices are being explored for in-memory computing. In this Review, we provide a broad overview of the key computational primitives enabled by these memory devices as well as their applications spanning scientific computing, signal processing, optimization, machine learning, deep learning and stochastic computing. This Review provides an overview of memory devices and the key computational primitives for in-memory computing, and examines the possibilities of applying this computing approach to a wide range of applications.

Memory devices and applications for in-memory computing

von Neumann architecture based computers isolate computation and storage (i.e. data is shuttled between computation blocks (processor) and memory blocks). The to-and-fro movement of data leads to a fundamental limitation of modern computers, known as the Memory wall. Logic in-Memory (LIM)/In-Memory Computing (IMC) approaches aim to address this bottleneck by directly computing inside memory units thereby eliminating energy-intensive and time-consuming data movement. Several recent works in literature, propose realization of logic function(s) directly using arrays of emerging resistive memory devices (example- memristors, RRAM/ReRAM, PCM, CBRAM, OxRAM, STT-MRAM etc.), rather than using conventional transistors for computing. The logic/embedded-side of digital systems (like processors, micro-controllers) can greatly benefit from such LIM realizations. However, the pure storage-side of digital systems (example SSDs, enterprise storage etc.) will not benefit much from such LIM approaches as when memory arrays are used for logic they lose their core functionality of storage. Thus, there is the need for an approach complementary to existing LIM techniques, that's more beneficial for the storage-side of digital systems; one that gives compute capability to memory arrays not at the cost of their existing stored states. Fundamentally, this would require memory nanodevice arrays that are capable of storing and computing simultaneously. In this paper, we propose a novel 'Simultaneous Logic in-Memory' (SLIM) methodology which is complementary to existing LIM approaches in literature. Through extensive experiments we demonstrate novel SLIM bitcells (1T-1R/2T-1R) comprising non-filamentary bilayer analog OxRAM devices with NMOS transistors. Proposed bitcells are capable of implementing both Memory and Logic operations simultaneously. Detailed programming scheme, array level implementation, and controller architecture are also proposed. Furthermore, to study the impact of proposed SLIM approach for real-world implementations, we performed analysis for two applications: (i) Sobel Edge Detection, and (ii) Binary Neural Network- Multi layer Perceptron (BNN-MLP). By performing all computations in SLIM bitcell array, huge Energy Delay Product (EDP) savings of ≈75× for 1T-1R (≈40× for 2T-1R) SLIM bitcell were observed for edge-detection application while EDP savings of ≈3.5× for 1T-1R (≈1.6× for 2T-1R) SLIM bitcell were observed for BNN-MLP application respectively, in comparison to conventional computing. EDP savings owing to reduction in data transfer between CPU ↔ memory is observed to be ≈780× (for both SLIM bitcells).

/pdf/slim-simultaneous-logic-in-memory-computing-exploiting-48akr75xhf.pdf

SLIM: Simultaneous Logic-in-Memory Computing Exploiting Bilayer Analog OxRAM Devices.

The rapid increase in information in the big-data era calls for changes to information-processing paradigms, which, in turn, demand new circuit-building blocks to overcome the decreasing cost-effectiveness of transistor scaling and the intrinsic inefficiency of using transistors in non-von Neumann computing architectures. Accordingly, resistive switching materials (RSMs) based on different physical principles have emerged for memories that could enable energy-efficient and area-efficient in-memory computing. In this Review, we survey the four physical mechanisms that lead to such resistive switching: redox reactions, phase transitions, spin-polarized tunnelling and ferroelectric polarization. We discuss how these mechanisms equip RSMs with desirable properties for representation capability, switching speed and energy, reliability and device density. These properties are the key enablers of processing-in-memory platforms, with applications ranging from neuromorphic computing and general-purpose memcomputing to cybersecurity. Finally, we examine the device requirements for such systems based on RSMs and provide suggestions to address challenges in materials engineering, device optimization, system integration and algorithm design. Resistive switching materials enable novel, in-memory information processing, which may resolve the von Neumann bottleneck. This Review focuses on how the switching mechanisms and the resultant electrical properties lead to various computing applications.

Resistive switching materials for information processing

The rapid development of artificial intelligence (AI) demands the rapid development of domain-specific hardware specifically designed for AI applications. Neuro-inspired computing chips integrate a range of features inspired by neurobiological systems and could provide an energy-efficient approach to AI computing workloads. Here, we review the development of neuro-inspired computing chips, including artificial neural network chips and spiking neural network chips. We propose four key metrics for benchmarking neuro-inspired computing chips — computing density, energy efficiency, computing accuracy, and on-chip learning capability — and discuss co-design principles, from the device to the algorithm level, for neuro-inspired computing chips based on non-volatile memory. We also provide a future electronic design automation tool chain and propose a roadmap for the development of large-scale neuro-inspired computing chips. This Review Article examines the development of neuro-inspired computing chips and their key benchmarking metrics, providing a co-design tool chain and proposing a roadmap for future large-scale chips.

Neuro-inspired computing chips

Reinforcement learning algorithms that use deep neural networks are a promising approach for the development of machines that can acquire knowledge and solve problems without human input or supervision. At present, however, these algorithms are implemented in software running on relatively standard complementary metal–oxide–semiconductor digital platforms, where performance will be constrained by the limits of Moore’s law and von Neumann architecture. Here, we report an experimental demonstration of reinforcement learning on a three-layer 1-transistor 1-memristor (1T1R) network using a modified learning algorithm tailored for our hybrid analogue–digital platform. To illustrate the capabilities of our approach in robust in situ training without the need for a model, we performed two classic control problems: the cart–pole and mountain car simulations. We also show that, compared with conventional digital systems in real-world reinforcement learning tasks, our hybrid analogue–digital computing system has the potential to achieve a significant boost in speed and energy efficiency. A reinforcement learning algorithm can be implemented on a hybrid analogue–digital platform based on memristive arrays for parallel and energy-efficient in situ training.

Reinforcement learning with analogue memristor arrays

Many artificial intelligence (AI) edge devices use nonvolatile memory (NVM) to store the weights for the neural network (trained off-line on an AI server), and require low-energy and fast I/O accesses. The deep neural networks (DNN) used by AI processors [1,2] commonly require p-layers of a convolutional neural network (CNN) and q-layers of a fully-connected network (FCN). Current DNN processors that use a conventional (von-Neumann) memory structure are limited by high access latencies, I/O energy consumption, and hardware costs. Large working data sets result in heavy accesses across the memory hierarchy, moreover large amounts of intermediate data are also generated due to the large number of multiply-and-accumulate (MAC) operations for both CNN and FCN. Even when binary-based DNN [3] are used, the required CNN and FCN operations result in a major memory I/O bottleneck for AI edge devices.

A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors

Non-volatile computing-in-memory (nvCIM) could improve the energy efficiency of edge devices for artificial intelligence applications. The basic functionality of nvCIM has recently been demonstrated using small-capacity memristor crossbar arrays combined with peripheral readout circuits made from discrete components. However, the advantages of the approach in terms of energy efficiency and operating speeds, as well as its robustness against device variability and sneak currents, have yet to be demonstrated experimentally. Here, we report a fully integrated memristive nvCIM structure that offers high energy efficiency and low latency for Boolean logic and multiply-and-accumulation (MAC) operations. We fabricate a 1 Mb resistive random-access memory (ReRAM) nvCIM macro that integrates a one-transistor–one-resistor ReRAM array with control and readout circuits on the same chip using an established 65 nm foundry complementary metal–oxide–semiconductor (CMOS) process. The approach offers an access time of 4.9 ns for three-input Boolean logic operations, a MAC computing time of 14.8 ns and an energy efficiency of 16.95 tera operations per second per watt. Applied to a deep neural network using a split binary-input ternary-weighted model, the system can achieve an inference accuracy of 98.8% on the MNIST dataset. A 1 Mb non-volatile computing-in-memory system, which integrates a resistive memory array with control and readout circuits using an established 65 nm foundry CMOS process, can offer high energy efficiency and low latency for Boolean logic and multiply-and-accumulation operations.

CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors

Many IoT and wearable devices require an on-chip small-to-mid-capacity nonvolatile memory (NVM) with a fast read-access time (T AC ) and reliable read operations: for applications including data-logging, configurable look-up tables (LUT), eFuse, and physically unclonable functions (PUF). STT-MRAM [1-4] is a good candidate for these applications due to its fast write speed, low-voltage write, and high endurance. However, STT-MRAM suffers from a small read-signal margin (RSM) due to a small tunnel magnetoresistance ratio (TMR: (R AP -R P )/R P ) between the cell resistance of parallel (P, R P ) and anti-parallel (AP, R AP ) states [1-6]. Moreover, the read-disturb behavior of STT-MRAM cells is sensitive to the BL read voltage (V BL_RD ) and the stress/read time. Compact 1T1MTJ arrays are suitable for high-density applications [5-6]; however, they use a power-hungry current-mode read scheme with a slow read speed due to the small RSM. Researchers have proposed 2T2MTJ (Fig. 30.3.1) arrays [1-4] with differential bitlines (BL and BLB) and a voltage-mode read scheme, with an enlarged RSM (V RSM ), for fast, low-power read operations. V RSM refers to the voltage difference between BL (V BL ) and BLB (V BLB ). 2T2MTJ STT-MRAM read operations still face the following challenges: (1) V BL and V BLB both drop from V BL_RD to 0V quite quickly due to the large cell read current (V P and V AP ) or low R-value in both R P and R AP , resulting in small sensing window (T SMW ), which is the period when V RSM >offset; (2) the maximum V RSM (V RSM_MAX ) occurs at different times (t RSM_MAX ) for different cells due to TMR (R AP /R P ) variation; and (3) a degraded V RSM due to the use of a low V BL_RD to avoid read disturbs for high data-reliability applications. (1) and (2) lead to a decrease in V RSM after reaching its peak (t RSM_MAX ), despite an increase in BL development time (t BL ). When using a conventional voltage-mode sense amplifier (CNV-VSA) with a common activated (SAEN=1) timing (f SAEN ) under the effects of (1)-(3), the signal to be amplified (ΔV IN „ RSM_MAX ) is subject to degradation at the VSA's differential inputs, resulting in a sensing failure at a low VBL_RD.

A 28nm 32Kb embedded 2T2MTJ STT-MRAM macro with 1.3ns read-access time for fast and reliable read applications

For the ultra fine pitch applications, solder paste printing acts as one of the most critical step in the SMT assembly process. A review of literature indicates that solder paste printing contributes around 60% of the total process defects. One of the key factors for the performance of solder paste deposition is stencil fabrication quality. Other important factors include stencil aperture design, paste printing parameters and stainless steel raw material. The incoming inspection of the stencil typically includes frame appearance examination and tension force measurement. Inspection for individual aperture under microscope is not feasible due to the large size of stencil. Related researches also shows that stencil aperture and wall roughness would impact solder paste deposition quality. This study uses measuring tools combined with statistical methods to investigate the effects of laser cutting taper angle and speed on the stencil quality. The stencil quality is defined by the amount of stainless steel residues after laser cutting and the roundness of stencil aperture for a circular aperture design. The solder paste inspection data was analyzed to investigate the influences of electro-polishing and stencil area ratio on the transfer ratio during solder paste deposition. Weighted ranking was adopted to indicate the variation in solder paste deposits for 01005 and 0.4/0.3mm pitch CSP pads due to stencil fabrication process. This study aims at determining the optimal parameters for stencil fabrication and corresponding performance in solder paste deposition. Results shows that the taper angle and laser speed have significant impacts to the amount of stainless steel residue and the roundness on laser entry side. Through EDS analysis, the residue material generated in laser cutting process is organic and it can't be effectively removed by laser electro-polishing process. Stencil printing quality will be affected by laser machine stability. If the laser fabrication process can be well controlled, 60% transfer ratio of solder deposition can be achieved with area ration of 0.5. Stencil thickness of 0.08mm might not be feasible for 01005 components. This is due to that the tension force of 0.1mm thick stencil is higher than that of 0.08mm stencil. Also, the anti-scrape ability of 0.1mm thick stencil is superior to that of 0.08mm. It is critical to meet the solder volume requirements for SMT connector and BGA components with coplanarity spec, of 0.1mm. Considering the consistency in solder deposition for 01005 components, the round stencil aperture is superior to the rectangle aperture. For the rectangle aperture of connector and gull-wing components, performance of vertical printing (with respect to the direction of stencil aperture) is superior to that of horizontal printing.

Stencil evaluation of ultra fine pitch solder paste printing process

This paper investigated the root cause of component fell-off from hand-held products after drop test. Failure modes are cracks at solder joints. Microstructure and fracture morphology of ENIG pad and solder joint were investigated through optical microscope (OM), scanning electron microscope (SEM), and energy dispersive spectrometer (EDS). The solder alloy used for assembly is SAC305 (SnAg3.0Cu0.5). The inter-metallic compound (IMC) at both PCB and component pads of the failed component is characterized through cross-section analysis. Through appearance observation on the failed components, fracture interface was smooth and appeared black and grey colour. Moreover, fracture contained high phosphors concentration (16–24 wt. %) and the flux residue covered on fracture surface. Compared with survived components, the failed samples show a thicker P-rich layer. Several obvious V-sharp spiked into Ni layer and some of them almost penetrated into 50% of the Ni layer thickness. Based on the results of drop test, appearance observation, composition analysis, and cross-section analysis, we concluded that the root cause of this failure maybe related to the black pad issue.

http://ieeexplore.ieee.org/iel5/5676708/5699462/05699643.pdf

K. C. Li

Papers

A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors

CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors

A 28nm 32Kb embedded 2T2MTJ STT-MRAM macro with 1.3ns read-access time for fast and reliable read applications

Stencil evaluation of ultra fine pitch solder paste printing process

Failure analysis of ENIG surface finish pad