Showing papers on "Neuromorphic engineering published in 2010"

Nanoscale Memristor Device as Synapse in Neuromorphic Systems

Abstract: A memristor is a two-terminal electronic device whose conductance can be precisely modulated by charge or flux through it. Here we experimentally demonstrate a nanoscale silicon-based memristor device and show that a hybrid system composed of complementary metal−oxide semiconductor neurons and memristor synapses can support important synaptic functions such as spike timing dependent plasticity. Using memristors as synapses in neuromorphic circuits can potentially offer both high connectivity and high density required for efficient computing.

A wafer-scale neuromorphic hardware system for large-scale neural modeling

Abstract: Modeling neural tissue is an important tool to investigate biological neural networks. Until recently, most of this modeling has been done using numerical methods. In the European research project "FACETS" this computational approach is complemented by different kinds of neuromorphic systems. A special emphasis lies in the usability of these systems for neuroscience. To accomplish this goal an integrated software/hardware framework has been developed which is centered around a unified neural system description language, called PyNN, that allows the scientist to describe a model and execute it in a transparent fashion on either a neuromorphic hardware system or a numerical simulator. A very large analog neuromorphic hardware system developed within FACETS is able to use complex neural models as well as realistic network topologies, i.e. it can realize more than 10000 synapses per neuron, to allow the direct execution of models which previously could have been simulated numerically only.

An Organic Nanoparticle Transistor Behaving as a Biological Spiking Synapse

Abstract: Molecule-based devices are envisioned to complement silicon devices by providing new functions or by implementing existing functions at a simpler process level and lower cost, by virtue of their self-organization capabilities. Moreover, they are not bound to von Neuman architecture and this feature may open the way to other architectural paradigms. Neuromorphic electronics is one of them. Here, a device made of molecules and nanoparticles-a nanoparticle organic memory field-effect transistor (NOMFET)—that exhibits the main behavior of a biological spiking synapse is demonstrated. Facilitating and depressing synaptic behaviors can be reproduced by the NOMFET and can be programmed. The synaptic plasticity for real-time computing is evidenced and described by a simple model. These results open the way to rate-coding utilization of the NOMFET in dynamical neuromorphic computing circuits.

Ultra Low Power Bioelectronics: Fundamentals, Biomedical Applications, and Bio-Inspired Systems

Abstract: Part I. Foundations: Ten chapters lay a foundation in device physics, noise, and feedback systems including nano scales in a highly original fashion, emphasizing intuitive thinking. This foundation is important in designing and analyzing ultra-low-power systems in both electronics and biology Part II. Low-Power Analog and Biomedical Circuits: Five chapters present building-block circuits that are useful for ultra-low-power biomedical electronics and analog electronic systems in general Part III. Low-Power RF and Energy-Harvesting Circuits for Biomedical Systems: Three chapters provide an in-depth description of energy-efficient power and data radio-frequency (RF) links that are fundamental to biomedical systems Part IV. Biomedical Electronic Systems: Two chapters provide an in-depth look at ultra-low-power implantable electronics and ultra-low-power noninvasive electronics for biomedical applications, respectively. Case studies for cochlear implants for the deaf, brain implants for the blind and paralyzed, wearable cardiac devices, and biomolecular sensing are provided Part V. Principles for Ultra-Low-Power Analog and Digital Design: Two chapters discuss principles for ultra-low-power digital design and ultra-low-power analog and mixed-signal design, respectively. The chapters identify ten fundamental principles that are common in both biology and electronics, analog and digital design Part VI. Bio-Inspired Systems: A chapter on neuromorphic electronics discusses electronics inspired by neurobiology followed by a chapter that discusses a novel form of electronics termed Cytomorphic Electronics, electronics inspired by cell biology. These chapters discuss applications of bio-inspired systems to engineering and medicine, deep connections between chemistry and electronics, and provide a unifying viewpoint of ultra-low-power design in biology and in electronics Part VII. Energy Sources: A chapter on batteries and electrochemistry discusses how batteries work from a unique circuit viewpoint. The last chapter discusses energy harvesting in biomedical systems at portable scales (vibration and body heat) and at larger scales (low-power cars and solar cells). Principles of low-power design are shown to extend from small scales in electronics to larger scales and to non-electrical systems. This book reveals the deep connections between energy use and energy generation, vital for sustainable energy systems of the future.

Neuromorphic, Digital and Quantum Computation with Memory Circuit Elements

Abstract: Memory effects are ubiquitous in nature and the class of memory circuit elements - which includes memristors, memcapacitors and meminductors - shows great potential to understand and simulate the associated fundamental physical processes. Here, we show that such elements can also be used in electronic schemes mimicking biologically-inspired computer architectures, performing digital logic and arithmetic operations, and can expand the capabilities of certain quantum computation schemes. In particular, we will discuss few examples where the concept of memory elements is relevant to the realization of associative memory in neuronal circuits, spike-timing-dependent plasticity of synapses, digital and field-programmable quantum computing.

Nanotube devices based crossbar architecture: toward neuromorphic computing

Abstract: Nanoscale devices such as carbon nanotube and nanowires based transistors, memristors and molecular devices are expected to play an important role in the development of new computing architectures. While their size represents a decisive advantage in terms of integration density, it also raises the critical question of how to efficiently address large numbers of densely integrated nanodevices without the need for complex multi-layer interconnection topologies similar to those used in CMOS technology. Two-terminal programmable devices in crossbar geometry seem particularly attractive, but suffer from severe addressing difficulties due to cross-talk, which implies complex programming procedures. Three-terminal devices can be easily addressed individually, but with limited gain in terms of interconnect integration. We show how optically gated carbon nanotube devices enable efficient individual addressing when arranged in a crossbar geometry with shared gate electrodes. This topology is particularly well suited for parallel programming or learning in the context of neuromorphic computing architectures.

Si Memristive devices applied to memory and neuromorphic circuits

Abstract: We report studies on nanoscale Si-based memristive devices for memory and neuromorphic applications. The devices are based on ion motion inside an insulating a-Si matrix. Digital devices show excellent performance metrics including scalability, speed, ON/OFF ratio, endurance and retention. High density non-volatile memory arrays based on a crossbar structure have been fabricated and tested. Devices inside a 1kb array can be individually addressed with excellent reproducibility and reliability. By adjusting the device and material structures, nanoscale analog memristor devices have also been demonstrated. The analog memristor devices exhibit incremental conductance changes that are controlled by the charge flown through the device. The performances of the digital and analog devices are thought to be determined by the formation of a dominant conducting filament and the continuous motion of a uniform conduction front, respectively.

Robust Bioinspired Architecture for Optical-Flow Computation

Abstract: Motion estimation from image sequences, called optical flow, has been deeply analyzed by the scientific community. Despite the number of different models and algorithms, none of them covers all problems associated with real-world processing. This paper presents a novel customizable architecture of a neuromorphic robust optical flow (multichannel gradient model) based on reconfigurable hardware with the properties of the cortical motion pathway, thus obtaining a useful framework for building future complex bioinspired real-time systems with high computational complexity. The presented architecture is customizable and adaptable, while emulating several neuromorphic properties, such as the use of several information channels of small bit width, which is the nature of the brain. This paper includes the resource usage and performance data, as well as a comparison with other systems. This hardware platform has many application fields in difficult environments due to its bioinspired nature and robustness properties, and it can be used as starting point in more complex systems.

Neural Dynamics in Reconfigurable Silicon

Abstract: A neuromorphic analog chip is presented that is capable of implementing massively parallel neural computations while retaining the programmability of digital systems. We show measurements from neurons with Hopf bifurcations and integrate and fire neurons, excitatory and inhibitory synapses, passive dendrite cables, coupled spiking neurons, and central pattern generators implemented on the chip. This chip provides a platform for not only simulating detailed neuron dynamics but also uses the same to interface with actual cells in applications such as a dynamic clamp. There are 28 computational analog blocks (CAB), each consisting of ion channels with tunable parameters, synapses, winner-take-all elements, current sources, transconductance amplifiers, and capacitors. There are four other CABs which have programmable bias generators. The programmability is achieved using floating gate transistors with on-chip programming control. The switch matrix for interconnecting the components in CABs also consists of floating-gate transistors. Emphasis is placed on replicating the detailed dynamics of computational neural models. Massive computational area efficiency is obtained by using the reconfigurable interconnect as synaptic weights, resulting in more than 50 000 possible 9-b accurate synapses in 9 mm2.

On neuromorphic spiking architectures for asynchronous STDP memristive systems

Abstract: Neuromorphic circuits and systems techniques have great potential for exploiting novel nanotechnology devices, which suffer from great parametric spread and high defect rate In this paper we explore some potential ways of building neural network systems for sophisticated pattern recognition tasks using memristors We will focus on spiking signal coding because of its energy and information coding efficiency, and concentrate on Convolutional Neural Networks because of their good scaling behavior, both in terms of number of synapses and temporal processing delay We propose asynchronous architectures that exploit memristive synapses with specially designed neurons that allow for arbitrary scalability as well as STDP learning We present some behavioral simulation results for small neural arrays using electrical circuit simulators, and system level spike processing results on human detection using a custom made event based simulator

Producing spike-timing dependent plasticity in a neuromorphic network utilizing phase change synaptic devices

Abstract: Embodiments of the invention relate to a neuromorphic network for producing spike-timing dependent plasticity. The neuromorphic network includes a plurality of electronic neurons and an interconnect circuit coupled for interconnecting the plurality of electronic neurons. The interconnect circuit includes plural synaptic devices for interconnecting the electronic neurons via axon paths, dendrite paths and membrane paths. Each synaptic device includes a variable state resistor and a transistor device with a gate terminal, a source terminal and a drain terminal, wherein the drain terminal is connected in series with a first terminal of the variable state resistor. The source terminal of the transistor device is connected to an axon path, the gate terminal of the transistor device is connected to a membrane path and a second terminal of the variable state resistor is connected to a dendrite path, such that each synaptic device is coupled between a first axon path and a first dendrite path, and between a first membrane path and said first dendrite path.

A review on memristive devices and applications

Abstract: Recent discovery of the memristor has sparked a new wave of enthusiasm and optimism in revolutionising circuit design, marking a new era for the advancement of neuromorphic and analogue applications. In this work, we consider practical applications in which the highly non-linear dynamic response of the memristor can be employed. It is shown that the device can be utilised as a non-volatile memory element and/or a programmable dynamic load, with particular emphasis given into bio-inspired analog implementations that typically exploit the ability of the memristor to support both logic and memory simultaneously. Finally, a novel concept is presented demonstrating the capacity of memristive networks in realising demanding image processing algorithms and more specifically edge detection.

Neuromorphic and synaptronic spiking neural network with synaptic weights learned using simulation

Abstract: Embodiments of the invention provide neuromorphic-synaptronic systems, including neuromorphic-synaptronic circuits implementing spiking neural network with synaptic weights learned using simulation. One embodiment includes simulating a spiking neural network to generate synaptic weights learned via the simulation while maintaining one-to-one correspondence between the simulation and a digital circuit chip. The learned synaptic weights are loaded into the digital circuit chip implementing a spiking neural network, the digital circuit chip comprising a neuromorphic-synaptronic spiking neural network including plural synapse devices interconnecting multiple digital neurons.

Neuromorphic spatiotemporal where-what machines

Abstract: In various embodiments, electronic apparatus, systems, and methods include a unified compact spatiotemporal method that provides a process for machines to deal with space and time and to deal with sensors and effectors. Additional apparatus, systems, and methods are disclosed.

Consistent recovery of sensory stimuli encoded with MIMO neural circuits

Abstract: We consider the problem of reconstructing finite energy stimuli encoded with a population of spiking leaky integrate-and-fire neurons. The reconstructed signal satisfies a consistency condition: when passed through the same neuron, it triggers the same spike train as the original stimulus. The recovered stimulus has to also minimize a quadratic smoothness optimality criterion. We formulate the reconstruction as a spline interpolation problem for scalar as well as vector valued stimuli and show that the recovery has a unique solution. We provide explicit reconstruction algorithms for stimuli encoded with single as well as a population of integrate-and-fire neurons. We demonstrate how our reconstruction algorithms can be applied to stimuli encoded with ON-OFF neural circuits with feedback. Finally, we extend the formalism to multi-input multi-output neural circuits and demonstrate that vector-valued finite energy signals can be efficiently encoded by a neural population provided that its size is beyond a threshold value. Examples are given that demonstrate the potential applications of our methodology to systems neuroscience and neuromorphic engineering.

Modeling the switching dynamics of programmable-metallization-cell (PMC) memory and its application as synapse device for a neuromorphic computation system

Abstract: A physical model is developed to investigate the switching dynamics of programmable-metallization-cell (PMC) memory Both “quasi-DC” and time-dependent transient characteristics of PMC are captured by this model in good agreement with the experimental data from Cu/SiO 2 and Ag/Ge 03 Se 07 cells For the first time, the time-dependent switching process of PMC is quantified, thus paving the way for a compact SPICE model for circuit simulation This model reveals that experimentally measured switching parameters such as threshold voltage and cell resistance are dynamic quantities that depend on the programming pulse shape and not the pulse amplitude alone Using this model, we show that the PMC has the potential to emulate the function of a biological synapse and exhibit the spike-timing-dependent plasticity (STDP) behavior for emerging neuromorphic computation system designs

A device mismatch compensation method for VLSI neural networks

Abstract: Device mismatch in neuromorphic VLSI implementations of spiking neural networks can be a serious and limiting problem. Classical engineering solutions can reduce the effect of mismatch, but require increasing layout sizes or using additional precious silicon real-estate. Here we propose a complementary strategy which exploits the Address-Event Representation used in neuromorphic systems and does not affect the device layout. We propose a method that selectively changes the connectivity profile in the neural network to normalize its response. We provide a theoretical analysis of the approach proposed and demonstrate its effectiveness with experimental data obtained from a VLSI Soft Winner-Take-All network.

Analysis of dynamic linear and non-linear memristor device models for emerging neuromorphic computing hardware design

Abstract: The value memristor devices offer to the neuromorphic computing hardware design community rests on the ability to provide effective device models that can enable large scale integrated computing architecture application simulations. Therefore, it is imperative to develop practical, functional device models of minimum mathematical complexity for fast, reliable, and accurate computing architecture technology design and simulation. To this end, various device models have been proposed in the literature seeking to characterize the physical electronic and time domain behavioral properties of memristor devices. In this work, we analyze some promising and practical non-quasi-static linear and non-linear memristor device models for neuromorphic circuit design and computing architecture simulation.

Functional Model of a Nanoparticle Organic Memory Transistor for Use as a Spiking Synapse

Abstract: Emerging synapse-like nanoscale devices such as memristive devices and synaptic transistors are of great interest to provide adaptability, high density, and robustness for the development of new bio-inspired circuits or systems. We have recently reported the nanoparticle organic memory field-effect transistor (NOMFET), which exhibits behaviors similar to a biological spiking synapse in neural network. It is considered as a promising nanocomponent to design neuromorphic adaptive computing circuits and systems. A functional model of NOMFET is presented in this paper, which allows the reliable conception and simulation of hybrid nano/complimentary metal-oxide-semiconductor circuits and architectures. Spice simulations of the model have demonstrated good agreement with the experimental results. By using the model, some complex neuromorphic functions such as synaptic gain control have been simulated. The model is developed in Verilog-A language and implemented on Cadence Virtuoso platform with Spectre 5.1.41 simulator. An iterative physical model and a number of experimental parameters have been integrated to improve the simulation accuracy. Special techniques and methods for dynamic behavior modeling have been developed, which could be extended to other nanodevices.

Neural dynamics in reconfigurable silicon

Abstract: A neuromorphic analog chip is presented that is capable of implementing massively parallel neural computations while retaining the programmability of digital systems. We show measurements from neurons with Hopf bifurcations and integrate and fire neurons, excitatory and inhibitory synapses, passive dendrite cables and central pattern generators implemented on the chip. This chip provides a platform for not only simulating detailed neuron dynamics but also using the same to interface with actual cells in applications like a dynamic clamp. The programmability is achieved using floating gate transistors with on-chip programming control. The switch matrix for interconnecting the components also consists of floating-gate transistors. Massive computational area efficiency is obtained by using the reconfigurable interconnect as synaptic weights.

Neuromorphic systems: past, present and future.

Abstract: Neuromorphic systems are implementations in silicon of elements of neural systems. The idea of electronic implementation is not new, but modern microelectronics has provided opportunities for producing systems for both sensing and neural modelling that can be mass produced straightforwardly. We review the the history of neuromorphic systems, and discuss the range of neuromorphic systems that have been developed. We discuss recent ideas for overcoming some of the problems, particularly providing effective adaptive synapses in large numbers.

Replicating experimental spike and rate based neural learning in CMOS

Abstract: The computational function of neural networks is thought to depend primarily on the learning/plasticity function carried out at the synapse. Neuromorphic circuit realizations have taken this into account by implementing a variety of synaptical processing functions, with most recent synapse circuits replicating some form of Spike Time Dependent Plasticity (STDP). However, STDP is being challenged by older rate-dependent learning rules as well as by biological experiments exhibiting more complex timing rules (e.g. spike triplets) as well as simultaneous rate- and timing dependent plasticity. In this paper, we present a circuit realization of a plasticity rule based on the postsynaptic neuron potential as well as the transmission profile of the presynaptic spike [1]. To the best of our knowledge, this is the first circuit realization of synaptical behaviour which moves significantly beyond STDP, replicating the triplet experiments of Froemke and Dan [2], the combined timing and rate experiments of Sjoestroem et al. [3], as well as conventional BCM behaviour [4].

Compensating Inhomogeneities of Neuromorphic VLSI Devices Via Short-Term Synaptic Plasticity.

Abstract: Recent developments in neuromorphic hardware engineering make mixed-signal VLSI neural network models promising candidates for neuroscientific research tools and massively parallel computing devices, especially for tasks which exhaust the computing power of software simulations. Still, like all analog hardware systems, neuromorphic models suffer from a constricted configurability and production-related fluctuations of device characteristics. Since also future systems, involving ever-smaller structures, will inevitably exhibit such inhomogeities on the unit level, self-regulation properties become a crucial requirement for their successful operation. By applying a cortically inspired self-adjusting network architecture, we show that the activity of generic spiking neural networks emulated on a neuromorphic hardware system can be kept within a biologically realistic firing regime and gain a remarkable robustness against transistor-level variations. As a first approach of this kind in engineering practice, the short-term synaptic depression and facilitation mechanisms implemented within an analog VLSI model of I&F neurons are functionally utilized for the purpose of network level stabilization. We present experimental data acquired both from the hardware model and from comparative software simulations which prove the applicability of the employed paradigm to neuromorphic VLSI devices.

Algorithm and software for simulation of spiking neural networks on the multi-chip SpiNNaker system

Abstract: This paper presents the algorithm and software developed for parallel simulation of spiking neural networks on multiple SpiNNaker universal neuromorphic chips. It not only describes approaches to simulating neural network models, such as dynamics, neural representations, and synaptic delays, but also presents the software design of loading a neural application and initial a simulation on the multi-chip SpiNNaker system. A series of sub-issues are also investigated, such as neuron-processor allocation, synapses distribution, and route planning. The platform is verified by running spiking neural applications on both the SoC Designer model and the physical SpiNNaker Test Chip. This work sums the problems we have solved and highlights those requiring further investigations, and therefore it forms the foundation of the software design on SpiNNaker, leading the future development towards a universal platform for real-time simulations of extreme large-scale neural systems.

Highly integrated packet-based AER communication infrastructure with 3Gevent/S throughput

Abstract: One of the main challenges in large scale neuromorphic VLSI systems is the design of the communication infrastructure. Traditionally, the neural communication has been done via parallel asynchronous transmission of Address-Event-Representations (AER) of pulses, while the configuration was achieved via off-the-shelf chip connect protocols. Recently, there has been a move towards greater event transmission speed via a serialization of the AER protocols, as well as an integration of both communication and configuration in the same interface. We present the PCB and FPGA design of such an interface for a newly developed waferscale neuromorphic system. The serial event communication of other current approaches has been refined into a packet based synchronous (rather than asynchronous) protocol, which offers better flexibility and bandwidth utilization. A factor 30–100 greater event transmission rate has been achieved. Compared to other approaches, the full communication bandwidth can also be employed for configuration. The system offers additional functionality, such as event storage and replay. Also, a very high degree of mechanical integration has been achieved.

Building blocks for spikes signals processing

Abstract: Neuromorphic engineers study models and implementations of systems that mimic neurons behavior in the brain. Neuro-inspired systems commonly use spikes to represent information. This representation has several advantages: its robustness to noise thanks to repetition, its continuous and analog information representation using digital pulses, its capacity of pre-processing during transmission time, …, Furthermore, spikes is an efficient way, found by nature, to codify, transmit and process information. In this paper we propose, design, and analyze neuro-inspired building blocks that can perform spike-based analog filters used in signal processing. We present a VHDL implementation for FPGA. Presented building blocks take advantages of the spike rate coded representation to perform a massively parallel processing without complex hardware units, like floating point arithmetic units, or a large memory. Those low requirements of hardware allow the integration of a high number of blocks inside a FPGA, allowing to process fully in parallel several spikes coded signals.

State-dependent sensory processing in networks of VLSI spiking neurons

Abstract: An increasing number of research groups develop dedicated hybrid analog/digital very large scale integration (VLSI) devices implementing hundreds of spiking neurons with bio-physically realistic dynamics. However, despite the significant progress in their design, there is still little insight in translating circuitry of neural assemblies into desired (non-trivial) function. In this work, we propose to use neural circuits implementing the soft Winner-Take-All (WTA) function. By showing that recurrently connected instances of them can have persistent activity states, which can be used as a form of working memory, we argue that such circuits can perform state-dependent computation. We demonstrate such a network in a distributed neuromorphic system consisting of two multi-neuron chips implementing soft WTA, stimulated by an event-based vision sensor. The resulting network is able to track and remember the position of a localized stimulus along a trajectory previously encoded in the system.

Neuromorphic models on a GPGPU cluster

Abstract: There is currently a strong push in the research community to develop biological scale implementations of neuron based vision models. Systems at this scale are computationally demanding and have generally utilized more accurate neuron models, such as the Izhikevich and Hodgkin- Huxley models, in favor of the integrate and fire model. This paper examines the feasibility of using a cluster of NVIDIA General Purpose Graphics Processing Units (GPGPUs) for accelerating a spiking neural network based character recognition network based on the Izhikevich and Hodgkin-Huxley models to enable such large scale systems. We utilized a 32 node cluster at NCSA containing an NVIDIA Tesla S1070 GPGPU on each node. Based on a thorough review of the literature, this is the first study examining the use of a cluster of GPGPUs for accelerating neuromorphic models. Our results show that the GPGPU can provide speedups of 24.6 and 177.0 times over a dual core 2.4 GHz AMD Opteron processor for the Izhikevich and Hodgkin-Huxley models respectively. Additionally, the MPI implementations of the models scaled almost linearly, with 16 GPGPUs providing throughputs of 14.1 and 15.9 times that of a single GPGPU for the Izhikevich and Hodgkin-Huxley models respectively. This indicates that clusters of GPGPUs are well suited for this application domain.