The article was published on 2017-05-01 and is currently open access. It has received 190 citations till now. The article focuses on the topics: Deep learning & Types of artificial neural networks.
TL;DR: A binary matrix multiplication GPU kernel is written with which it is possible to run the MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy.
TL;DR: DoReFa-Net, a method to train convolutional neural networks that have low bitwidth weights and activations using low bit width parameter gradients, is proposed and can achieve comparable prediction accuracy as 32-bit counterparts.
TL;DR: SuperSpike is derived, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns.
TL;DR: A survey on two types of network compression: pruning and quantization is provided, which compare current techniques, analyze their strengths and weaknesses, provide guidance for compressing networks, and discuss possible future compression techniques.
TL;DR: A review of the most essential topics on HPO, including the key hyper-parameters related to model training and structure, and a comparison between optimization algorithms, and prominent approaches for model evaluation with limited computational resources.
The paper covers the basic principles of Neural Networks ’ work. In addition, the article touches upon the main programming languages used to write software for Neural Networks.