Showing papers by "Jean-Michel Portal published in 2020"

Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays.

Abstract: The brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergence of resistive memory technologies indeed provides an opportunity to tightly co-integrate logic and memory in hardware. In parallel, the recently proposed concept of a Binarized Neural Network, where multiplications are replaced by exclusive NOR (XNOR) logic gates, offers a way to implement artificial intelligence using very low precision computation. In this work, we therefore propose a strategy for implementing low-energy Binarized Neural Networks that employs brain-inspired concepts while retaining the energy benefits of digital electronics. We design, fabricate, and test a memory array, including periphery and sensing circuits, that is optimized for this in-memory computing scheme. Our circuit employs hafnium oxide resistive memory integrated in the back end of line of a 130-nm CMOS process, in a two-transistor, two-resistor cell, which allows the exclusive NOR operations of the neural network to be performed directly within the sense amplifiers. We show, based on extensive electrical measurements, that our design allows a reduction in the number of bit errors on the synaptic weights without the use of formal error-correcting codes. We design a whole system using this memory array. We show on standard machine learning tasks (MNIST, CIFAR-10, ImageNet, and an ECG task) that the system has inherent resilience to bit errors. We evidence that its energy consumption is attractive compared to more standard approaches and that it can use memory devices in regimes where they exhibit particularly low programming energy and high endurance. We conclude the work by discussing how it associates biologically plausible ideas with more traditional digital electronics concepts.

In-memory resistive RAM implementation of binarized neural networks for medical applications

Abstract: The advent of deep learning has considerably accelerated machine learning development The deployment of deep neural networks at the edge is however limited by their high memory and energy consumption requirements With new memory technology available, emerging Binarized Neural Networks (BNNs) are promising to reduce the energy impact of the forthcoming machine learning hardware generation, enabling machine learning on the edge devices and avoiding data transfer over the network In this work, after presenting our implementation employing a hybrid CMOS - hafnium oxide resistive memory technology, we suggest strategies to apply BNNs to biomedical signals such as electrocardiography and electroencephalography, keeping accuracy level and reducing memory requirements We investigate the memory-accuracy trade-off when binarizing whole network and binarizing solely the classifier part We also discuss how these results translate to the edge-oriented Mobilenet V1 neural network on the Imagenet task The final goal of this research is to enable smart autonomous healthcare devices

3D RRAMs with Gate-All-Around Stacked Nanosheet Transistors for In-Memory-Computing

Abstract: This paper explores a novel 3D one transistor / one RRAM (1T1R) memory cube. The proposed architecture integrates HfO 2 -based OxRAM with select junctionless (JL) transistors based on low-voltage Gate-All-Around (GAA) stacked NanoSheet (NS) technology. A bitcell size of 23.9×F2/N is achieved (‘N’ being the number of stacked-NS) as well as a very high write and read parallelism. Extensive characterization of JL transistors and OxRAMs is performed to show their ability to be co-integrated inside a same 1T1R memory cell. Electrical characterization of 4kbits OxRAM arrays shows a large memory window (HRS/LRS=20) up to 104 cycles with a current compliance of 150µA, compatible with the performances of our JL transistors. Then, we experimentally demonstrate scouting logic operations capability with 2 operands, which should be extended to 4 operands thanks to an original two cells/bit “double coding” scheme assessed by SPICE simulations. Finally, we evidenced that this computing scheme is 2 times more energy efficient than a write-verify approach.

Crosspoint Memory Arrays: Principle, Strengths and Challenges

Abstract: This paper presents an outlook of crosspoint memory arrays. We survey the characteristics, strengths and challenges of crosspoint array composed by a resistive memory device integrated in series with a back-end selector. We present the performances of our 1S1R technology composed of an HfO 2 OxRAM (1R) combined with a GeSeSbN based Ovonic Threshold Switch (1S). We benchmark our results with a detailed collection of experimental data reported in the literature.

Write Termination circuits for RRAM : A Holistic Approach From Technology to Application Considerations

Abstract: While Resistive Random Access Memories (RRAM) are perceived nowadays as a promising solution for the future of computing, these technologies suffer from intrinsic variability regarding programming voltage, switching speed and achieved resistance values. Write Termination (WT) circuits are a potential solution to solve these issues. However, previously reported WT circuits do not demonstrate sufficient reliability. In this work, we propose an industrially-ready WT circuit that was simulated with a RRAM model calibrated on real measurements. We perform extensive CMOS and RRAM variability simulations to extract the actual performances of the proposed WT circuit. Finally, we simulate the effects of the proposed WT circuit with memory traces extracted from real Edge-level data-intensive applications. Overall, we demonstrate 2× to 40× of energy gains at bit level. Moreover, we show from 1.9× to 16.2× energy gains with real applications running depending on the application memory access pattern thanks to the proposed WT circuit.

Optimization of RRAM and OTS selector for advanced low voltage CMOS compatibility

Abstract: Al 2 O 3 based Conductive Bridge RAM (CBRAM) is co-integrated with an optimized Ge-Se-Sb-N based back-end selector in 1S1R memory arrays for low voltage and advanced CMOS compatibility. Electrical characterization is performed to extract device features, showing forming free behavior, ~3.5V maximum operating voltage, s table 2 decades for I on /I off during 107 cycles, ~1V reading voltage margin and moderate leakage current. Compatible with 100kbit sector size, this optimized stack is found very promising for embedded applications on advanced process nodes. Indeed, design exploration on a 28nm core process shows that peripherals can be designed with middle voltage MOS, fitting best with low voltage 1S1R cross-point array and leading to reduced overall memory area.

Low Power In-Memory Implementation of Ternary Neural Networks with Resistive RAM-Based Synapse

Abstract: The design of systems implementing low precision neural networks with emerging memories such as resistive random access memory (RRAM) is a major lead for reducing the energy consumption of artificial intelligence (AI). Multiple works have for example proposed in-memory architectures to implement low power binarized neural networks. These simple neural networks, where synaptic weights and neuronal activations assume binary values, can indeed approach state-of-the-art performance on vision tasks. In this work, we revisit one of these architectures where synapses are implemented in a differential fashion to reduce bit errors, and synaptic weights are read using precharge sense amplifiers. Based on experimental measurements on a hybrid 130 nm CMOS/RRAM chip and on circuit simulation, we show that the same memory array architecture can be used to implement ternary weights instead of binary weights, and that this technique is particularly appropriate if the sense amplifier is operated in near-threshold regime. We also show based on neural network simulation on the CIFAR-10 image recognition task that going from binary to ternary neural networks significantly increases neural network performance. These results highlight that AI circuits function may sometimes be revisited when operated in low power regimes.

Low Power In-Memory Implementation of Ternary Neural Networks with Resistive RAM-Based Synapse

Abstract: The design of systems implementing low precision neural networks with emerging memories such as resistive random access memory (RRAM) is a major lead for reducing the energy consumption of artificial intelligence (AI). Multiple works have for example proposed in-memory architectures to implement low power binarized neural networks. These simple neural networks, where synaptic weights and neuronal activations assume binary values, can indeed approach state-of-the-art performance on vision tasks. In this work, we revisit one of these architectures where synapses are implemented in a differential fashion to reduce bit errors, and synaptic weights are read using precharge sense amplifiers. Based on experimental measurements on a hybrid 130 nm CMOS/RRAM chip and on circuit simulation, we show that the same memory array architecture can be used to implement ternary weights instead of binary weights, and that this technique is particularly appropriate if the sense amplifier is operated in near-threshold regime. We also show based on neural network simulation on the CIFAR-10 image recognition task that going from binary to ternary neural networks significantly increases neural network performance. These results highlight that AI circuits function may sometimes be revisited when operated in low power regimes.

Implementation of Ternary Weights with Resistive RAM Using a Single Sense Operation per Synapse

Abstract: The design of systems implementing low precision neural networks with emerging memories such as resistive random access memory (RRAM) is a significant lead for reducing the energy consumption of artificial intelligence. To achieve maximum energy efficiency in such systems, logic and memory should be integrated as tightly as possible. In this work, we focus on the case of ternary neural networks, where synaptic weights assume ternary values. We propose a two-transistor/two-resistor memory architecture employing a precharge sense amplifier, where the weight value can be extracted in a single sense operation. Based on experimental measurements on a hybrid 130 nm CMOS/RRAM chip featuring this sense amplifier, we show that this technique is particularly appropriate at low supply voltage, and that it is resilient to process, voltage, and temperature variations. We characterize the bit error rate in our scheme. We show based on neural network simulation on the CIFAR-10 image recognition task that the use of ternary neural networks significantly increases neural network performance, with regards to binary ones, which are often preferred for inference hardware. We finally evidence that the neural network is immune to the type of bit errors observed in our scheme, which can therefore be used without error correction.

Embracing the Unreliability of Memory Devices for Neuromorphic Computing

Abstract: The emergence of resistive non-volatile memories opens the way to highly energy-efficient computation near- or in-memory. However, this type of computation is not compatible with conventional ECC, and has to deal with device unreliability. Inspired by the architecture of animal brains, we present a manufactured differential hybrid CMOS/RRAM memory architecture suitable for neural network implementation that functions without formal ECC. We also show that using low-energy but error-prone programming conditions only slightly reduces network accuracy.

In-Memory Resistive RAM Implementation of Binarized Neural Networks for Medical Applications

Abstract: The advent of deep learning has considerably accelerated machine learning development The deployment of deep neural networks at the edge is however limited by their high memory and energy consumption requirements With new memory technology available, emerging Binarized Neural Networks (BNNs) are promising to reduce the energy impact of the forthcoming machine learning hardware generation, enabling machine learning on the edge devices and avoiding data transfer over the network In this work, after presenting our implementation employing a hybrid CMOS - hafnium oxide resistive memory technology, we suggest strategies to apply BNNs to biomedical signals such as electrocardiography and electroencephalography, keeping accuracy level and reducing memory requirements We investigate the memory-accuracy trade-off when binarizing whole network and binarizing solely the classifier part We also discuss how these results translate to the edge-oriented Mobilenet~V1 neural network on the Imagenet task The final goal of this research is to enable smart autonomous healthcare devices

Embracing the Unreliability of Memory Devices for Neuromorphic Computing

Abstract: The emergence of resistive non-volatile memories opens the way to highly energy-efficient computation near- or in-memory. However, this type of computation is not compatible with conventional ECC, and has to deal with device unreliability. Inspired by the architecture of animal brains, we present a manufactured differential hybrid CMOS/RRAM memory architecture suitable for neural network implementation that functions without formal ECC. We also show that using low-energy but error-prone programming conditions only slightly reduces network accuracy.