The convolutional neural network (CNN) is one of the most used deep learning models for image detection and classification, due to its high accuracy when compared to other machine learning algorithms. CNNs achieve better results at the cost of higher computing and memory requirements. Inference of convolutional neural networks is therefore usually done in centralized high-performance platforms. However, many applications based on CNNs are migrating to edge devices near the source of data due to the unreliability of a transmission channel in exchanging data with a central server, the uncertainty about channel latency not tolerated by many applications, security and data privacy, etc. While advantageous, deep learning on edge is quite challenging because edge devices are usually limited in terms of performance, cost, and energy. Reconfigurable computing is being considered for inference on edge due to its high performance and energy efficiency while keeping a high hardware flexibility that allows for the easy adaption of the target computing platform to the CNN model. In this paper, we described the features of the most common CNNs, the capabilities of reconfigurable computing for running CNNs, the state-of-the-art of reconfigurable computing implementations proposed to run CNN models, as well as the trends and challenges for future edge reconfigurable platforms.

A survey of convolutional neural networks on edge with reconfigurable computing

As the emerging field of machine learning, deep learning shows excellent ability in solving complex learning problems. However, the size of the networks becomes increasingly large scale due to the demands of the practical applications, which poses significant challenge to construct high performance implementations of deep learning neural networks. In order to improve the performance as well as to maintain the low power cost, in this paper we design deep learning accelerator unit (DLAU), which is a scalable accelerator architecture for large-scale deep learning networks using field-programmable gate array (FPGA) as the hardware prototype. The DLAU accelerator employs three pipelined processing units to improve the throughput and utilizes tile techniques to explore locality for deep learning applications. Experimental results on the state-of-the-art Xilinx FPGA board demonstrate that the DLAU accelerator is able to achieve up to 36.1× speedup comparing to the Intel Core2 processors, with the power consumption at 234 mW.

/pdf/dlau-a-scalable-deep-learning-accelerator-unit-on-fpga-38jbyl4y1b.pdf

DLAU: A Scalable Deep Learning Accelerator Unit on FPGA

): – requirements throughout surface waters of the N-limited North Pacific. Recent isotopic analysis of skeleton material from deep-sea corals near Hawaii also exhibit a decreasing trend over this time period, which has been interpreted as a signal of increasing N inputs from N2 fixation (36). However, because isotopic and stoichiometric signals of denitrification are transported from the anoxic zone into the subtropical gyre (37), the reported coral trends may originate partly from the OMZ. Any remaining signal attributable to N2 fixation would imply that the ecological niche of diazotrophs in the central gyre is uncoupled from the major N loss in the OMZ (38), and that a substantial imbalance of the Pacific N budget has persisted over the 20th century.

A million spiking-neuron integrated circuit with a scalable communication network and interface

In recent years, convolutional neural network (CNN)-based methods have been widely used for optical remote sensing object detection and have shown excellent performance. Some aerospace systems, such as satellites or aircrafts, need to adopt these methods to observe objects on the ground. Due to the limited budget of the logical resources and power consumption in these systems, an embedded device is a good choice to implement the CNN-based methods. However, it is still a challenge to strike a balance between performance and power consumption. In this paper, we propose an efficient hardware-implementation method for optical remote sensing object detection. Firstly, we optimize the CNN-based model for hardware implementation, which establishes a foundation for efficiently mapping the network on a field-programmable gate array (FPGA). In addition, we propose a hardware architecture for the CNN-based remote sensing object detection model. In this architecture, a general processing engine (PE) is proposed to implement multiple types of convolutions in the network using the uniform module. An efficient data storage and access scheme is also proposed, and it achieves low-latency calculations and a high memory bandwidth utilization rate. Finally, we deployed the improved YOLOv2 network on a Xilinx ZYNQ xc7z035 FPGA to evaluate the performance of our design. The experimental results show that the performance of our implementation on an FPGA is only 0.18% lower than that on a graphics processing unit (GPU) in mean average precision (mAP). Under a 200 MHz working frequency, our design achieves a throughput of 111.5 giga-operations per second (GOP/s) with a 5.96 W on-chip power consumption. Comparison with the related works demonstrates that the proposed design has obvious advantages in terms of energy efficiency and that it is suitable for deployment on embedded devices.

FPGA Implementation for CNN-Based Optical Remote Sensing Object Detection

Convolutional Neural Network (CNN) is one of the most effective neural network model for many classification tasks, such as voice recognition, computer vision and biological information processing. Unfortunately, Computation of CNN is both memory-intensive and computation-intensive, which brings a huge challenge to the design of the hardware accelerators. A large number of hardware accelerators for CNN inference are designed by the industry and the academia. Most of the engines are based on 32-bit floating point matrix multiplication, where the data precision is over-provisioned for inference job and the hardware cost are too high. In this paper, a 8-bit fixed-point LeNet inference engine (Laius) is designed and implemented on FPGA. In order to reduce the consumption of FPGA resource, we proposed a methodology to find the optimal bit-length for weight and bias in LeNet, which results in using 8-bit fixed point for most of the computation and using 16-bit fixed point for other computation. The PE (Processing Element) design is proposed. Pipelining and PE tiling technique is use to improve the performance of the inference engine. By theoretical analysis, we came to the conclusion that DSP resource in FPGA is the most critical resource, it should be carefully used during the design process. We implement the inference engine on Xilinx 485t FPGA. Experiment result shows that the designed LeNet inference engine can achieve 44.9 Gops throughput with 8-bit fixed-point operation after pipelining. Moreover, with only 1% loss of accuracy, the 8-bit fixed-point engine largely reduce 31.43% in latency, 87.01% in LUT consumption, 66.50% in BRAM consumption, 65.11% in DSP consumption and 47.95% reduction in power compared to a 32-bit fixed-point inference engine with the same structure.

Laius: An 8-Bit Fixed-Point CNN Hardware Inference Engine

Spike-timing dependent plasticity (STDP)-based spiking neural network (SNN) is a promising choice to realize unsupervised intelligent systems with a limited power budget. In addition to STDP, another two bio-inspired mechanisms of lateral inhibition and homeostasis are always implemented in the unsupervised training procedure of STDP-based SNNs. However, the existing methods to achieve lateral inhibition necessitate a great number of connections that are proportional to the square of the number of learning neurons, and the existing hardware solution of homeostasis demands complex circuits for each learning neuron, both of which challenge the hardware implementation of STDP-based SNNs. In this brief, we propose a novel SNN using memristor-based inhibitory synapses to realize the mechanisms of lateral inhibition and homeostasis with low hardware complexity. The proposed SNN can improve the network scalability by reducing the connection number for lateral inhibition from $N^{2}$ to $N$ and reduce the hardware overhead by leveraging the circuit of lateral inhibition to achieve homeostasis. Software simulations on the recognition task on MNIST dataset show that the proposed SNN achieves a ~ 2 times higher learning efficiency with comparable accuracy. In addition, the challenging properties of realistic memristor devices, including limited number of resistive states, intrinsic parameter variation, and permanent open device, are added in the simulation to evaluate the robustness of our proposed approach.

A Memristor-Based Spiking Neural Network With High Scalability and Learning Efficiency

Although Deep Neural Network (DNN) architectures have made some breakthroughs in computer vision tasks, they are not close to biological brain neurons. Spiking Neural Network (SNN) is highly expected to bridge the gap between artificial computing systems and bio-systems. And it also shows great potential in low power computing. This paper presents a low power hardware accelerator for SNN inference using systolic array, and a corresponding software framework for optimization. First, we give the hardware design which adopts systolic array inspired by explorations of SNN. Then we ensure correct data mapping for systolic array for the sake of computational correctness. Next, we use compression methods for decreasing both the runtime and memory footprint. Finally, we make the systolic array size-configurable to adapt to different input, so as to reduce computational overhead. We implement the accelerator on Xilinx FPGA V7 690T. The experimental results show that SNN inference on our scheme suffers little loss on accuracy (less than 0.1%) on MNIST and Fashion-MNIST, and the runtime of the time-consuming layers decreases. The total power of our scheme is 0.745 W at 100 MHz.

A Systolic SNN Inference Accelerator and its Co-optimized Software Framework

Neuromorphic computing is considered to be the future of machine learning, and it provides a new way of cognitive computing. Inspired by the excellent performance of spiking neural networks (SNNs) on the fields of low-power consumption and parallel computing, many groups tried to simulate the SNN with the hardware platform. However, the efficiency of training SNNs with neuromorphic algorithms is not ideal enough. Facing this, Michael et al. proposed a method which can solve the problem with the help of DNN (deep neural network). With this method, we can easily convert a well-trained DNN into an SCNN (spiking convolutional neural network). So far, there is a little of work focusing on the hardware accelerating of SCNN. The motivation of this paper is to design an SNN processor to accelerate SNN inference for SNNs obtained by this DNN-to-SNN method. We propose SIES (Spiking Neural Network Inference Engine for SCNN Accelerating). It uses a systolic array to accomplish the task of membrane potential increments computation. It integrates an optional hardware module of max-pooling to reduce additional data moving between the host and the SIES. We also design a hardware data setup mechanism for the convolutional layer on the SIES with which we can minimize the time of input spikes preparing. We implement the SIES on FPGA XCVU440. The number of neurons it supports is up to 4 000 while the synapses are 256 000. The SIES can run with the working frequency of 200 MHz, and its peak performance is 1.562 5 TOPS.

SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array

Spiking neural network (SNN), as the third generation of artificial neural networks, has been widely adopted in vision and audio tasks. Nowadays, many neuromorphic platforms support SNN simulation and adopt Network-on-Chips (NoC) architecture for multi-cores interconnection. However, a large volume and run-time communication on the interconnection has a significant effect on performance of the platform. In this paper, we propose a toolchain called SNEAP (Spiking NEural network mAPping toolchain) for mapping SNNs to neuromorphic platforms with multi-cores, which aims to reduce the energy and latency brought by spike communication on the interconnection. SNEAP includes two key steps: partitioning the SNN to reduce the spikes communicated between partitions, and mapping the partitions of SNN to the NoC to reduce average hop of spikes under the constraint of hardware resources. SNEAP effectively reduces the energy and latency on the NoC-based neuromorphic platform and spend less time than other toolchains. The experimental results show that SNEAP can achieve average 418X reduction in end-to-end execution time, and reduce energy consumption and spike latency, on average, by 23% and 51% respectively, compared with SpiNeMap.

/pdf/sneap-a-fast-and-efficient-toolchain-for-mapping-large-scale-h787c89iu9.pdf

Shasha Guo

Papers

Laius: An 8-Bit Fixed-Point CNN Hardware Inference Engine

A Memristor-Based Spiking Neural Network With High Scalability and Learning Efficiency

A Systolic SNN Inference Accelerator and its Co-optimized Software Framework

SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array

SNEAP: A Fast and Efficient Toolchain for Mapping Large-Scale Spiking Neural Network onto NoC-based Neuromorphic Platform