Book ChapterDOI
GradientBased Learning Applied to Document Recognition
Simon Haykin,Bart Kosko +1 more
- pp 306-351
TLDR
Various methods applied to handwritten character recognition are reviewed and compared and Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.Abstract:
Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure. Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks. A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.read more
Citations
More filters
Posted Content
Reliable Fidelity and Diversity Metrics for Generative Models
TL;DR: It is shown that even the latest version of the precision and recall metrics are not reliable yet, and density and coverage metrics provide more interpretable and reliable signals for practitioners than the existing metrics.
Journal ArticleDOI
Performance evaluation of pattern classifiers for handwritten character recognition
TL;DR: The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance.
Proceedings ArticleDOI
On the relationship between visual attributes and convolutional networks
TL;DR: This work characterize the nature of the relationship between abstract concepts learned by popular and high performing convolutional networks (conv-nets) and established mid-level representations used in computer vision and shows empirical evidence of the existence of Attribute Centric Nodes (ACNs) within a conv-net, which is trained to recognize objects (not attributes) in images.
Posted Content
Improving Adversarial Robustness via Promoting Ensemble Diversity
TL;DR: A new notion of ensemble diversity in the adversarial setting is defined as the diversity among non-maximal predictions of individual members, and an adaptive diversity promoting (ADP) regularizer is presented to encourage the diversity, which leads to globally better robustness for the ensemble by making adversarial examples difficult to transfer among individual members.
Posted Content
Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling
Spyros Gidaris,Nikos Komodakis +1 more
TL;DR: This work proposes a generic architecture that decomposes the label improvement task to three steps: detecting the initial label estimates that are incorrect, replacing the incorrect labels with new ones, and finally refining the renewed labels by predicting residual corrections w.r.t. them.