A number of recent efforts have attempted to design accelerators for popular machine learning algorithms, such as those involving convolutional and deep neural networks (CNNs and DNNs). These algorithms typically involve a large number of multiply-accumulate (dot-product) operations. A recent project, DaDianNao, adopts a near data processing approach, where a specialized neural functional unit performs all the digital arithmetic operations and receives input weights from adjacent eDRAM banks.This work explores an in-situ processing approach, where memristor crossbar arrays not only store input weights, but are also used to perform dot-product operations in an analog manner. While the use of crossbar memory as an analog dot-product engine is well known, no prior work has designed or characterized a full-fledged accelerator based on crossbars. In particular, our work makes the following contributions: (i) We design a pipelined architecture, with some crossbars dedicated for each neural network layer, and eDRAM buffers that aggregate data between pipeline stages. (ii) We define new data encoding techniques that are amenable to analog computations and that can reduce the high overheads of analog-to-digital conversion (ADC). (iii) We define the many supporting digital components required in an analog CNN accelerator and carry out a design space exploration to identify the best balance of memristor storage/compute, ADCs, and eDRAM storage on a chip. On a suite of CNN and DNN workloads, the proposed ISAAC architecture yields improvements of 14.8×, 5.5×, and 7.5× in throughput, energy, and computational density (respectively), relative to the state-of-the-art DaDianNao architecture.

ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars

Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed

A Survey of Neuromorphic Computing and Neural Networks in Hardware.

Learning in multilayer neural networks (MNNs) relies on continuous updating of large matrices of synaptic weights by local rules. Such locality can be exploited for massive parallelism when implementing MNNs in hardware. However, these update rules require a multiply and accumulate operation for each synaptic weight, which is challenging to implement compactly using CMOS. In this paper, a method for performing these update operations simultaneously (incremental outer products) using memristor-based arrays is proposed. The method is based on the fact that, approximately, given a voltage pulse, the conductivity of a memristor will increment proportionally to the pulse duration multiplied by the pulse magnitude if the increment is sufficiently small. The proposed method uses a synaptic circuit composed of a small number of components per synapse: one memristor and two CMOS transistors. This circuit is expected to consume between 2% and 8% of the area and static power of previous CMOS-only hardware alternatives. Such a circuit can compactly implement hardware MNNs trainable by scalable algorithms based on online gradient descent (e.g., backpropagation). The utility and robustness of the proposed memristor-based circuit are demonstrated on standard supervised learning tasks.

Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training

Recent theoretical studies have shown that probabilistic spiking can be interpreted as learning and inference in cortical microcircuits. This interpretation creates new opportunities for building neuromorphic systems driven by probabilistic learning algorithms. However, such systems must have two crucial features: 1) the neurons should follow a specific behavioral model, and 2) stochastic spiking should be implemented efficiently for it to be scalable. This paper proposes a memristor-based stochastically spiking neuron that fulfills these requirements. First, the analytical model of the memristor is enhanced so it can capture the behavioral stochasticity consistent with experimentally observed phenomena. The switching behavior of the memristor model is demonstrated to be akin to the firing of the stochastic spike response neuron model, the primary building block for probabilistic algorithms in spiking neural networks. Furthermore, the paper proposes a neural soma circuit that utilizes the intrinsic nondeterminism of memristive switching for efficient spike generation. The simulations and analysis of the behavior of a single stochastic neuron and a winner-take-all network built of such neurons and trained on handwritten digits confirm that the circuit can be used for building probabilistic sampling and pattern adaptation machinery in spiking networks. The findings constitute an important step towards scalable and efficient probabilistic neuromorphic platforms.

Memristors Empower Spiking Neurons With Stochasticity

The nonvolatile memory property of a memristor enables the realization of new methods for a variety of computational engines ranging from innovative memristive-based neuromorphic circuitry through to advanced memory applications. The nanometer-scale feature of the device creates a new opportunity for realization of innovative circuits that in some cases are not possible or have inefficient realization in the present and established design domain. The nature of the boundary, the complexity of the ionic transport and tunneling mechanism, and the nanoscale feature of the memristor introduces challenges in modeling, characterization, and simulation of future circuits and systems. Here, a deeper insight is gained in understanding the device operation, leading to the development of practical models that can be implemented in current computer-aided design (CAD) tools.

/pdf/memristive-device-fundamentals-and-modeling-applications-to-501zi9ckiu.pdf

Memristive Device Fundamentals and Modeling: Applications to Circuits and Systems Simulation

The nanometer scale feature of memristor created a broad range of opportunities for innovative architectures. The nature of the boundary conditions, the complexity of the ionic transport and tunneling mechanism, and the nanoscale feature of the memristor introduces new challenges in modeling, characterization, and measurements for Memristor-MOS (M2) circuits. These new challenges can be addressed by a joint insight from the circuit designer and device engineers, which will dictate the needed modeling and layout rules to attain an accurate estimation of M2 circuit performance. In this paper, memristive behavior of titanium dioxide (TiO 2 ) is studied using a novel combination of electrodes, silver (Ag) and indium thin oxide (ITO). Fabrication method and a modeling approach are also explained. The ITO electrode provide (a) an excellent transparency in visible light, (b) improved functional reproducibility, and (c) non-volatile characteristics as well as a promising unique application of the M2 circuits in sensory applications. Furthermore, proposed modeling approach shows a good agreement between measurements and simulations of analog memory characteristics and reproducibility as well as long-term retention.

Fabrication and modeling of Ag/TiO 2 /ITO memristor

Sensory search engines in a variety of applications, including DNA sequencing, are beginning to influence the way new search nanoarchitectures are constructed. This paper presents a novel nanoarchitecture based on a complementary resistive switch (CRS) construct for smart sensor search engines. Detailed operational principles of CRS, CRS-based memory array, and image matching applications using the Hamming distance measure are presented along with backing simulation results. The proposed scheme draws upon the inherent windowing behavior of fabricated CRS and hence, the duty cycle thus leading to a power dissipation of 115 nW/b, which is a reduction in power consumption by almost 50% when compared with the more conventional architectures. The variation on the match line is 100 mV/Hamming distance, providing a stable reference source for comparison input and stored data.

Complementary Resistive Switch-Based Smart Sensor Search Engine

We present new computational building blocks based on memristive devices. These blocks, can be used to implement either supervised or unsupervised learning modules. This is achieved using a crosspoint architecture which is an efficient array implementation for nanoscale two-terminal mem-ristive devices. Based on these blocks and an experimentally verified SPICE macromodel for the memristor, we demonstrate that firstly, the Spike-Timing-Dependent Plasticity (STDP) can be implemented by a single memristor device and secondly, a memristor-based competitive Hebbian learning through STDP using a 1×1000 synaptic network. This is achieved by adjusting the memristor's conductance values (weights) as a function of the timing difference between presynaptic and postsynaptic spikes. These implementations have a number of shortcomings due to the memristor's characteristics such as memory decay, highly nonlinear switching behaviour as a function of applied voltage/current, and functional uniformity. These shortcomings can be addressed by utilising a mixed gates that can be used in conjunction with the analogue behaviour for biomimetic computation. The digital implementations in this paper use in-situ computational capability of the memristor.

/pdf/memristor-based-synaptic-networks-and-logical-operations-3ze6b3g060.pdf

Memristor-based synaptic networks and logical operations using in-situ computing

In this study, molybdenum tungsten/amorphous InGaZnO (a-IGZO)/TiO2/n-type Si-based resistive random access memory (ReRAM) is manufactured. After deposition of the a-IGZO, annealing was performed at 200, 300, 400, and 500 °C for approximately 1 h in order to analyze the effect of temperature change on the ReRAM after post annealing in a furnace. As a result of measuring the current-voltage curve, the a-IGZO/TiO2-based ReRAM annealed at 400 °C reached compliance current in a low-resistance state, and showed the most complete hysteresis curve. In the a-IGZO layer annealed at 400 °C, the O1/Ototal value increased most significantly, to approximately 78.2%, and the O3/Ototal value decreased the most, to approximately 2.6%. As a result, the a-IGZO/TiO2-based ReRAM annealed at 400 °C reduced conductivity and prevented an increase in leakage current caused by oxygen vacancies with sufficient recovery of the metal-oxygen bond. Scanning electron microscopy analysis revealed that the a-IGZO surface showed hillocks at a high post annealing temperature of 500 °C, which greatly increased the surface roughness and caused the surface area performance to deteriorate. Finally, as a result of measuring the capacitance-voltage curve in the a-IGZO/TiO2-based ReRAM in the range of -2 V to 4 V, the accumulation capacitance value of the ReRAM annealed at 400 °C increased most in a nonvolatile behavior.

/pdf/filamentary-resistive-switching-and-capacitance-voltage-3tbfjpj7bi.pdf

Filamentary Resistive Switching and Capacitance-Voltage Characteristics of the a-IGZO/TiO 2 Memory

Most of the existing copper indium gallium diselenide (CIGS) thin film solar cells are based on a cadmium sulfide (CdS) buffer layer fabricated using a chemical bath deposition (CBD) process. However, due to environmental pollution caused by material toxicity and the unique wet process's incompatibility with the vacuum process, many studies are now being actively carried out on nontoxic buffer layers. In this study, to replace CdS buffer layers, zinc sulfide (ZnS) buffer layers with a big band gap and a low optical loss at a short wavelength were fabricated using a magnetron sputtering system. For comparative analysis, this study also fabricated CdS buffer layers using the CBD process. Then, the conversion efficiency of CIGS thin film solar cells deposited with ZnS and CdS thin film as buffer layers was measured. The light conversion efficiency of ZnS buffer layer-based CIGS was measured at 14.44%, while that of the CdS buffer layer-based CIGS was measured at 15.71%. Given that both are higher than the minimum conversion efficiency required for commercialization (10%), ZnS buffer layer-based solar cells could have a competitive edge over the existing CdS buffer layer-based solar cells.

Sung-Jin Kim

Papers

Fabrication and modeling of Ag/TiO 2 /ITO memristor

Complementary Resistive Switch-Based Smart Sensor Search Engine

Memristor-based synaptic networks and logical operations using in-situ computing

Filamentary Resistive Switching and Capacitance-Voltage Characteristics of the a-IGZO/TiO 2 Memory

High-Efficiency Cu(In,Ga)Se₂ Thin Film Solar Cells Using ZnS and CdS Buffer Layers.