Book ChapterDOI
GradientBased Learning Applied to Document Recognition
Simon Haykin,Bart Kosko +1 more
- pp 306-351
TLDR
Various methods applied to handwritten character recognition are reviewed and compared and Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.Abstract:
Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure. Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks. A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.read more
Citations
More filters
Proceedings ArticleDOI
Pointwise Convolutional Neural Networks
TL;DR: Pointwise convolution as discussed by the authors is a new convolution operator that can be applied at each point of a point cloud, which can yield competitive accuracy in both semantic segmentation and object recognition task.
Journal ArticleDOI
Audio-visual speech recognition using deep learning
TL;DR: A connectionist-hidden Markov model (HMM) system for noise-robust AVSR is introduced and it is demonstrated that approximately 65 % word recognition rate gain is attained with denoised MFCCs under 10 dB signal-to-noise-ratio (SNR) for the audio signal input.
Proceedings ArticleDOI
Appearance-based gaze estimation in the wild
TL;DR: An extensive evaluation of several state-of-the-art image-based gaze estimation algorithms on three current datasets, including the MPIIGaze dataset, which contains 213,659 images collected from 15 participants during natural everyday laptop use over more than three months.
Proceedings ArticleDOI
Deep Neural Decision Forests
TL;DR: Deep Neural Decision Forests as discussed by the authors proposes a stochastic and differentiable decision tree model, which steers the representation learning usually conducted in the initial layers of a (deep) convolutional network.
Posted Content
Federated Optimization in Heterogeneous Networks
TL;DR: FedProx as discussed by the authors is a generalization and re-parametrization of FedAvg, which is the state-of-the-art method for federated learning.