https://www.cse.iitk.ac.in/users/isaha/Publications/Journals/NC10.pdf/

Artificial neural networks in hardware: A survey of two decades of progress

The aim of this paper is to review the state-of-the-art of Field Programmable Gate Array (FPGA) technologies and their contribution to industrial control applications. Authors start by addressing various research fields which can exploit the advantages of FPGAs. The features of these devices are then presented, followed by their corresponding design tools. To illustrate the benefits of using FPGAs in the case of complex control applications, a sensorless motor controller has been treated. This controller is based on the Extended Kalman Filter. Its development has been made according to a dedicated design methodology, which is also discussed. The use of FPGAs to implement artificial intelligence-based industrial controllers is then briefly reviewed. The final section presents two short case studies of Neural Network control systems designs targeting FPGAs.

https://hal-centralesupelec.archives-ouvertes.fr/hal-01337007/document

FPGAs in Industrial Control Applications

Convolutional Neural Networks (ConvNets) are a powerful Deep Learning model, providing state-of-the-art accuracy to many emerging classification problems. However, ConvNet classification is a computationally heavy task, suffering from rapid complexity scaling. This paper presents fpgaConvNet, a novel domain-specific modelling framework together with an automated design methodology for the mapping of ConvNets onto reconfigurable FPGA-based platforms. By interpreting ConvNet classification as a streaming application, the proposed framework employs the Synchronous Dataflow (SDF) model of computation as its basis and proposes a set of transformations on the SDF graph that explore the performance-resource design space, while taking into account platform-specific resource constraints. A comparison with existing ConvNet FPGA works shows that the proposed fully-automated methodology yields hardware designs that improve the performance density by up to 1.62x and reach up to 90.75% of the raw performance of architectures that are hand-tuned for particular ConvNets.

/pdf/fpgaconvnet-a-framework-for-mapping-convolutional-neural-392m9gvr0d.pdf

fpgaConvNet: A Framework for Mapping Convolutional Neural Networks on FPGAs

This paper proposes DeepSecure, a novel framework that enables scalable execution of the state-of-the-art Deep Learning (DL) models in a privacy-preserving setting. DeepSecure targets scenarios in which neither of the involved parties including the cloud servers that hold the DL model parameters or the delegating clients who own the data is willing to reveal their information. Our framework is the first to empower accurate and scalable DL analysis of data generated by distributed clients without sacrificing the security to maintain efficiency. The secure DL computation in DeepSecure is performed using Yao's Garbled Circuit (GC) protocol. We devise GC-optimized realization of various components used in DL. Our optimized implementation achieves more than 58-fold higher throughput per sample compared with the best-known prior solution. In addition to our optimized GC realization, we introduce a set of novel low-overhead pre-processing techniques which further reduce the GC overall runtime in the context of deep learning. Extensive evaluations of various DL applications demonstrate up to two orders-of-magnitude additional runtime improvement achieved as a result of our pre-processing methodology. This paper also provides mechanisms to securely delegate GC computations to a third party in constrained embedded settings.

DeepSecure: Scalable Provably-Secure Deep Learning

Special attention must be paid to an efficient approximation of the sigmoid function in implementing FPGA-based reprogrammable hardware-based artificial neural networks. Four previously published piecewise linear and one piecewise second-order approximation of the sigmoid function are compared with SIG-sigmoid, a purely combinational approximation. The approximations are compared in terms of speed, required area resources and accuracy measured by average and maximum error. It is concluded that the best performance is achieved by SIG-sigmoid.

/pdf/efficient-digital-implementation-of-the-sigmoid-function-for-j9uxr4ftmo.pdf

Efficient digital implementation of the sigmoid function for reprogrammable logic

An efficient piecewise linear approximation of a nonlinear function (PLAN) is proposed. This uses a simple digital gate design to perform a direct transformation from X to Y, where X is the input and Y is the approximated sigmoidal output. This PLAN is then used within the outputs of an artificial neural network to perform the nonlinear approximation. The comparison of this technique with two other sigmoidal approximation techniques for digital circuits is presented and the results show that the fast and compact digital circuit proposed produces the closest approximation to the sigmoid function. The hardware implementation of PLAN has been verified by a VHDL simulation with Mentor Graphics running under the UNIX operating system.

Piecewise linear approximation applied to nonlinear function of a neural network

This paper presents the results of research into a scheme for overcoming the communications 'bottleneck' within hardware neural networks, utilising frequency division multiplexing of amplitude modulated neural signals. The introduction explains the problem of connectivity in neural networks and the need for multiplexing, followed by an introduction to the FDM solution. A comparison of FDM with TDM is then presented which shows that it is necessary to allow a moderate amount of band overlap in the FDM channel, to utilise the available bandwidth effectively. This approach is possible by virtue of the inherent error tolerance of the neural system.

Consideration of multiplexing in neural network hardware

Frequency division multiplexing has been studied as a means of communication between neural layers in an analogue multilayered perceptron neural network architecture, trained using the back-propagation learning algorithm. Simulation results on network learning and generalisation show that the neural network is tolerant to as much as 50% overlap of frequency responses of filters used in demultiplexing. Thus, the number of communication channels available is considerably increased.

Frequency division multiplexing in analogue neural network

A technique is presented which combines rule-based and neural network pattern recognition methods in an integrated system in order to perform learning and recognition of hand-written characters and gestures in realtime. The GesRec system is introduced which provides a framework for data acquisition, training, recognition, and gesture-to-speech transcription in a Windows environment. A recognition accuracy of 92.5% was obtained for the hybrid system, compared to 89.6% for the neural network only and 82.7% for rules only. Training and recognition times are given for an able-bodied and a disabled user.

A hybrid neural network/rule-based technique for on-line gesture and hand-written character recognition

Fractal transformations are an exciting new scale invariant shape recognition technique developed by the authors. They have been successfully used for spatially invariant shape recognition using a cumbersome rotational invariance algorithm. Within this paper a new rotational invariance algorithm is presented which makes the recognition process inherently rotationally invariant. A full discussion of fractal recognition is given and previous results discussed. The new algorithm is detailed and applied to a difficult recognition problem. The results of this application show that the new algorithm is successful in achieving the desired effect.

K.M. Curtis

Papers

Piecewise linear approximation applied to nonlinear function of a neural network

Consideration of multiplexing in neural network hardware

Frequency division multiplexing in analogue neural network

A hybrid neural network/rule-based technique for on-line gesture and hand-written character recognition

Shape recognition using a novel fractal technique