In order to understand how the mammalian neocortex is performing computations, two things are necessary; we need to have a good understanding of the available neuronal processing units and mechanisms, and we need to gain a better understanding of how those mechanisms are combined to build functioning systems. Therefore, in recent years there is an increasing interest in how spiking neural networks (SNN) can be used to perform complex computations or solve pattern recognition tasks. However, it remains a challenging task to design SNNs which use biologically plausible mechanisms (especially for learning new patterns), since most of such SNN architectures rely on training in a rate-based network and subsequent conversion to a SNN. We present a SNN for digit recognition which is based on mechanisms with increased biological plausibility, i.e. conductance-based instead of current-based synapses, spike-timing-dependent plasticity with time-dependent weight change, lateral inhibition, and an adaptive spiking threshold. Unlike most other systems, we do not use a teaching signal and do not present any class labels to the network. Using this unsupervised learning scheme, our architecture achieves 95% accuracy on the MNIST benchmark, which is better than previous SNN implementations without supervision. The fact that we used no domain-specific knowledge points toward the general applicability of our network design. Also, the performance of our network scales well with the number of neurons used and shows similar performance for four different learning rules, indicating robustness of the full combination of mechanisms, which suggests applicability in heterogeneous biological neural networks.

/pdf/unsupervised-learning-of-digit-recognition-using-spike-4aclr1f742.pdf

Unsupervised learning of digit recognition using spike-timing-dependent plasticity.

Deep learning in spiking neural networks

Deep neural networks such as Convolutional Networks (ConvNets) and Deep Belief Networks (DBNs) represent the state-of-the-art for many machine learning and computer vision classification problems To overcome the large computational cost of deep networks, spiking deep networks have recently been proposed, given the specialized hardware now available for spiking neural networks (SNNs) However, this has come at the cost of performance losses due to the conversion from analog neural networks (ANNs) without a notion of time, to sparsely firing, event-driven SNNs Here we analyze the effects of converting deep ANNs into SNNs with respect to the choice of parameters for spiking neurons such as firing rates and thresholds We present a set of optimization techniques to minimize performance loss in the conversion process for ConvNets and fully connected deep networks These techniques yield networks that outperform all previous SNNs on the MNIST database to date, and many networks here are close to maximum performance after only 20 ms of simulated time The techniques include using rectified linear units (ReLUs) with zero bias during training, and using a new weight normalization method to help regulate firing rates Our method for converting an ANN into an SNN enables low-latency classification with high accuracies already after the first output spike, and compared with previous SNN approaches it yields improved performance without increased training time The presented analysis and optimization techniques boost the value of spiking deep networks as an attractive framework for neuromorphic computing platforms aimed at fast and efficient pattern recognition

Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of is), very high dynamic range (140dB vs. 60dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

Event-based Vision: A Survey

A forward-facing vision system for a vehicle includes a forward-facing camera disposed in a windshield electronics module attached at a windshield of the vehicle and viewing through the windshield. A control includes a processor that, responsive to processing of captured image data, detects taillights of leading vehicles during nighttime conditions and, responsive to processing of captured image data, detects lane markers on a road being traveled by the vehicle. The control, responsive to lane marker detection and a determination that the vehicle is drifting out of a traffic lane, may control a steering system of the vehicle to mitigate such drifting, with the steering system manually controllable by a driver of the vehicle irrespective of control by the control. The processor, based at least in part on detection of lane markers via processing of captured image data, determines curvature of the road being traveled by the vehicle.

Vision system for vehicle

This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system’s most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.

/pdf/feedforward-categorization-on-aer-motion-events-using-cortex-3gvhvi8mr9.pdf

Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network

Ring oscillator (RO)-based physical unclonable function (PUF) is resilient against noise impacts, but its response is susceptible to temperature variations. This paper presents a low-power and small footprint hybrid RO PUF with a very high temperature stability, which makes it an ideal candidate for lightweight applications. The negative temperature coefficient of the low-power subthreshold operation of current starved inverters is exploited to mitigate the variations of differential RO frequencies with temperature. The new architecture uses conspicuously simplified circuitries to generate and compare a large number of pairs of RO frequencies. The proposed nine-stage hybrid RO PUF was fabricated using global foundry 65-nm CMOS technology. The PUF occupies only $250~ {\mu }\text{m}^{ {2}}$ of chip area and consumes only $32.3~ {\mu }\text{W}$ per challenge response pair at 1.2 V and 230 MHz. The measured average and worst-case reliability of its responses are 99.84% and 97.28%, respectively, over a wide range of temperature from −40 to 120 °C.

/pdf/a-low-power-hybrid-ro-puf-with-improved-thermal-stability-3n8fw1to45.pdf

A Low-Power Hybrid RO PUF With Improved Thermal Stability for Lightweight Applications

In the applications of biometric authentication and video surveillance, the image sensor is expected to provide certain degree of trust and resiliency. This paper presents a new low-cost CMOS image sensor based physical unclonable function (PUF) targeting a variety of security, privacy and trusted protocols that involves image sensor as a trusted entity. The proposed PUF exploits the intrinsic imperfection during the image sensor manufacturing process to generate unique and reliable digital signatures. The proposed differential readout stabilizes the response bits extracted from the random fixed pattern noises of selected pixel pairs determined by the applied challenge against supply voltage and temperature variations. The threshold of difference can be tightened to winnow out more unstable response bits from the challenge-response space offered by modern image sensors to enhance the reliability under harsher operating conditions and loosened to improve its resiliency against masquerade attacks in routine operating environment. The proposed design can be classified as a weak PUF which is resilient to modeling attacks, with direct access to its challenge-response pair restricted by the linear feedback shift register. Our experiments on the reset voltages extracted from a 64 $\times$ 64 image sensor fabricated in 180 nm 3.3 V CMOS technology demonstrated that robust and reliable challenge-response pairs can be generated with a uniqueness of 49.37% and a reliability of 99.80% under temperature variations of $15\sim 115~^{\circ}{\rm C}$ and supply voltage variations of $3\sim 3.6\ {\rm V}$ .

/pdf/cmos-image-sensor-based-physical-unclonable-function-for-hyxba8cogg.pdf

CMOS Image Sensor Based Physical Unclonable Function for Coherent Sensor-Level Authentication

This paper presents the architecture, algorithm, and VLSI hardware of image acquisition, storage, and compression on a single-chip CMOS image sensor. The image array is based on time domain digital pixel sensor technology equipped with nondestructive storage capability using 8-bit Static-RAM device embedded at the pixel level. The pixel-level memory is used to store the uncompressed illumination data during the integration mode as well as the compressed illumination data obtained after the compression stage. An adaptive quantization scheme based on fast boundary adaptation rule (FBAR) and differential pulse code modulation (DPCM) procedure followed by an online, least storage quadrant tree decomposition (QTD) processing is proposed enabling a robust and compact image compression processor. A prototype chip including 64×64 pixels, read-out and control circuitry as well as an on-chip compression processor was implemented in 0.35 μm CMOS technology with a silicon area of 3.2×3.0 mm2 and an overall power of 17 mW. Simulation and measurements results show compression figures corresponding to 0.6-1 bit-per-pixel (BPP), while maintaining reasonable peak signal-to-noise ratio levels.

A CMOS Image Sensor With On-Chip Image Compression Based on Predictive Boundary Adaptation and Memoryless QTD Algorithm

We demonstrate a new generation smart image sensor, CeleX-V. With 1280×800 pixels, 9.8um pitch, the sensor integrates several vision functions into one chip, such as full-array-parallel motion detection and on-chip optical flow extraction. CeleX-V is also capable of producing high-quality full-frame pictures and thus is compatible with traditional picture-based algorithms. The sensor supports both MIPI and parallel interface, with typical 400mW power consumption.

/pdf/live-demonstration-celex-v-a-1m-pixel-multi-mode-event-based-4kdochk09n.pdf

Shoushun Chen

Papers

Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network

A Low-Power Hybrid RO PUF With Improved Thermal Stability for Lightweight Applications

CMOS Image Sensor Based Physical Unclonable Function for Coherent Sensor-Level Authentication

A CMOS Image Sensor With On-Chip Image Compression Based on Predictive Boundary Adaptation and Memoryless QTD Algorithm

Live Demonstration: CeleX-V: A 1M Pixel Multi-Mode Event-Based Sensor