Neural networks are a powerful technology forclassification of visual inputs arising from documents.However, there is a confusing plethora of different neuralnetwork methods that are used in the literature and inindustry. This paper describes a set of concrete bestpractices that document analysis researchers can use toget good results with neural networks. The mostimportant practice is getting a training set as large aspossible: we expand the training set by adding a newform of distorted data. The next most important practiceis that convolutional neural networks are better suited forvisual document tasks than fully connected networks. Wepropose that a simple "do-it-yourself" implementation ofconvolution with a flexible architecture is suitable formany visual document problems. This simpleconvolutional neural network does not require complexmethods, such as momentum, weight decay, structure-dependentlearning rates, averaging layers, tangent prop,or even finely-tuning the architecture. The end result is avery simple yet general architecture which can yieldstate-of-the-art performance for document analysis. Weillustrate our claims on the MNIST set of English digitimages.

/pdf/best-practices-for-convolutional-neural-networks-applied-to-1r5hlmtzmr.pdf

Best practices for convolutional neural networks applied to visual document analysis

This paper introduces a pair of online and offline Chinese handwriting databases, containing samples of isolated characters and handwritten texts. The samples were produced by 1,020 writers using Anoto pen on papers for obtaining both online trajectory data and offline images. Both the online samples and offline samples are divided into six datasets, three for isolated characters (DB1.0-C1.2) and three for handwritten texts (DB2.0-C2.2). The (either online or offline) datasets of isolated characters contain about 3.9 million samples of 7,356 classes (7,185 Chinese characters and 171 symbols), and the datasets of handwritten texts contain about 5,090 pages and 1.35 million character samples. Each dataset is segmented and annotated at character level, and is partitioned into standard training and test subsets. The online and offline databases can be used for the research of various handwritten document analysis tasks.

/pdf/casia-online-and-offline-chinese-handwriting-databases-u25ema1eot.pdf

CASIA Online and Offline Chinese Handwriting Databases

A survey on off-line Cursive Word recognition

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.

/pdf/markov-models-for-offline-handwriting-recognition-a-survey-4ijdjn6mf0.pdf

Markov models for offline handwriting recognition: a survey

In this paper we present IAM-OnDB - a new large online handwritten sentences database. It is publicly available and consists of text acquired via an electronic interface from a whiteboard. The database contains about 86 K word instances from an 11 K dictionary written by more than 200 writers. We also describe a recognizer for unconstrained English text that was trained and tested using this database. This recognizer is based on hidden Markov models (HMMs). In our experiments we show that by using larger training sets we can significantly increase the word recognition rate. This recognizer may serve as a benchmark reference for future research.

IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard

Databases for character recognition algorithms are of fundamental interest for the training of statistics based recognition methods (neural networks, hidden Markov models) as well as for benchmarking existing recognition systems. Such databases currently exist, but none of them gives access to the online data (pen trajectory) and offline data (digital images) for the same writing signal. We have developed such a dual on/off database, named IRONOFF. Currently, it contains a large number of isolated characters, digits, and cursive words written by French writers. We have designed this database so that, given an online point, it can be mapped at the correct location in the corresponding scanned image, and conversely, each offline pixel can be temporally indexed. Since we think this database is of interest for a large part of the research community, it is publicly available.

The IRESTE On/Off (IRONOFF) dual handwriting database

On-line handwriting includes more information on the time order of the writing signal and on the dynamics of the writing process than off-line handwriting. Therefore, on-line recognition systems achieve higher recognition rates. This can be concluded from results reported in the literature, and has been demonstrated empirically as part of this work. We propose a new approach for recovering the time order of the off-line writing signal. Starting from an over-segmentation of the off-line handwriting into regular and singular parts, the time ordering of these parts and recognition of the word are performed simultaneously. This approach, termed “ OrdRec”, is based on a graph description of the handwriting signal and a recognition process using Hidden Markov Models (HMM). A complete omni-scriptor isolated word recognition system has been developed. Using a dynamic lexicon and models for upper and lower case characters, our system can process binary and gray value word images of any writing style (script, cursive, or mixed). Using a dual handwriting data base which features both the on-line and the off-line signal for each of the 30 000 words written by about 700 scriptors, we have shown experimentally that such an off-line recognition system, using the recovered time order information, can achieve recognition performances close to those of an on-line recognition system.

/pdf/from-off-line-to-on-line-handwriting-recognition-1huut5wtrm.pdf

From Off-line to On-line Handwriting Recognition

This paper describes an offline cursive handwritten word recognition system that combines hidden Markov models (HMM) and neural networks (NN). Using a fast left-right slicing method, we generate a segmentation graph that describes all possible ways to segment a word into letters. The NN computes the observation probabilities for each letter hypothesis in the segmentation graph. Then, the HMM compute the likelihood for each word in the lexicon by summing the probabilities over all possible paths through the graph. We present the preprocessing and the recognition process as well as the training procedure for the NN-HMM hybrid system. Another recognition system based on discrete HMM is also presented for performance comparison. The latter is also used for bootstrapping the NN-HMM hybrid system. Recognition performances of the two recognition systems using two image databases of French isolated words are presented. This paper is one of the first publications using the IRONOFF database, and thus can be used as a reference for future work on this database.

An offline cursive handwritten word recognition system

We describe an analytical handwritten word recognition system combining neural networks (NN) and hidden Markov models (HMM). Using a fast left-right slicing method, we generate a segmentation graph that describes all possible ways to segment a word into characters. The NN computes the observation probabilities for each character hypothesis in the segmentation graph. Then, using concatenated character HMMs, a likelihood is computed for each word in the lexicon by multiplying the observation probabilities over the best path through the graph. The role of the NN is to recognize characters and to reject non-characters. We present our approach to globally train the word recognizer using isolated word images. Using a maximum mutual information (MMI) cost function at the word level, the discriminant training updates the parameters of the NN within a global optimization process based on gradient descent. The recognizer is bootstrapped from a baseline recognition system, which is based on character level training. The recognition performance of the globally trained system is compared to the baseline system.

An analytical handwritten word recognition system with word-level discriminant training

This paper describes an approach to combine neural network (NN) and hidden Markov models (HMM) for solving the handwritten word recognition problem. The preprocessing involves generating a segmentation graph that describes all possible ways to segment a word into letters. To recognize a word, the NN computes the observation probabilities for each letter hypothesis in the segmentation graph. The HMMs then compute the likelihood for each word in the lexicon by summing the probabilities over all possible paths through the graph. One critical criterion for the NN-HMM hybrid system is that the NN character recognizer should be able to recognize non-characters or junk, apart from having the ability to distinguish between characters. In other words, the NN should give low probabilities for all character classes if junk is presented. We introduce the discriminant training to train the NN to recognize junk. We present a structural training scheme to improve the performance of the recognizer. An offline handwritten word recognizer is developed based on this approach and the recognition performance of the recognizer on three isolated word image databases, namely, IRONOFF, SRTP and AWS, are presented.

P.-M. Lallican

Papers

The IRESTE On/Off (IRONOFF) dual handwriting database

From Off-line to On-line Handwriting Recognition

An offline cursive handwritten word recognition system

An analytical handwritten word recognition system with word-level discriminant training

Offline handwritten word recognition using a hybrid neural network and hidden Markov model