Recurrent neural networks are powerful sequence learners. They are able to incorporate context information in a flexible way, and are robust to localised distortions of the input data. These properties make them well suited to sequence labelling, where input sequences are transcribed with streams of labels. The aim of this thesis is to advance the state-of-the-art in supervised sequence labelling with recurrent networks. Its two main contributions are (1) a new type of output layer that allows recurrent networks to be trained directly for sequence labelling tasks where the alignment between the inputs and the labels is unknown, and (2) an extension of the long short-term memory network architecture to multidimensional data, such as images and video sequences.

https://mediatum.ub.tum.de/doc/673554/file.pdf

Supervised Sequence Labelling with Recurrent Neural Networks

Current state-of-the-art approaches to offline Handwritten Text Recognition extensively rely on Multidimensional Long Short-Term Memory networks However, these architectures come with quite an expensive computational cost, and we observe that they extract features visually similar to those of convolutional layers, which are computationally cheaper This suggests that the two-dimensional long-term dependencies, which are potentially modeled by multidimensional recurrent layers, may not be essential to achieve a good recognition accuracy, at least in the lower layers of the architecture In this work, an alternative model is explored that relies only on convolutional and one-dimensional recurrent layers that achieves better or equivalent results than those of the current state-of-the-art architecture, and runs significantly faster In addition, we observe that using random distortions during training as synthetic data augmentation dramatically improves the accuracy of our model Thus, are multidimensional recurrent layers really necessary for Handwritten Text Recognition? Probably not

Are Multidimensional Recurrent Layers Really Necessary for Handwritten Text Recognition

Text independent writer recognition using redundant writing patterns with contour-based orientation and curvature features

Multidimensional long short-term memory recurrent neural networks achieve impressive results for handwriting recognition. However, with current CPU-based implementations, their training is very expensive and thus their capacity has so far been limited. We release an efficient GPU-based implementation which greatly reduces training times by processing the input in a diagonal-wise fashion. We use this implementation to explore deeper and wider architectures than previously used for handwriting recognition and show that especially the depth plays an important role. We outperform state of the art results on two databases with a deep multidimensional network.

Handwriting Recognition with Large Multidimensional Long Short-Term Memory Recurrent Neural Networks

Given an image of a handwritten word, a CNN is employed to estimate its n-gram frequency profile, which is the set of n-grams contained in the word. Frequencies for unigrams, bigrams and trigrams are estimated for the entire word and for parts of it. Canonical Correlation Analysis is then used to match the estimated profile to the true profiles of all words in a large dictionary. The CNN that is used employs several novelties such as the use of multiple fully connected branches. Applied to all commonly used handwriting recognition benchmarks, our method outperforms, by a very large margin, all existing methods.

/pdf/cnn-n-gram-for-handwritingword-recognition-3vyy6rqsl2.pdf

CNN-N-Gram for HandwritingWord Recognition

The paper presents the French RIMES (Reconnaissance et Indexation de donnees Manuscrites et de fac similES / Recognition and Indexing of handwritten documents and faxes) project, which is an evaluation campaign dedicated to automatic systems of recognition and indexing of handwritten letters such as those sent by postal mail or fax by individuals to companies or administrations. This project is intended to all the key players of the handwritten recognition and document analysis fields by proposing them to participate to evaluation tests, a first test having taken already place. In this framework, a new database of more than 5600 real mails completely annotated have been created as well as secondary databases of isolated characters, handwritten words (300 000 snippets) and logos. Beyond the evaluation, the other contribution of the RIMES project is to make available all these databases to the community after the last test

RIMES evaluation campaign for handwritten mail processing

This paper presents the results of the second test phase of the RIMES evaluation campaign. The latter is the first large-scale evaluation campaign intended to all the key players of the handwritten recognition and document analysis communities. It proposes various tasks around recognition and indexing of handwritten letters such as those sent by postal mail or fax by individuals to companies or administrations. In this second evaluation test, automatic systems have been evaluated on three themes: layout analysis, handwriting recognition and writer identification. The databases used are part of the RIMES database of 5605 real mails completely annotated as well as secondary databases of isolated characters and handwritten words (250,000 snippets). The paper reports on protocols and gives the results obtained in the campaign.(RIMES : Reconnaissance et Indexation de donnees Manuscrites et de fac similES / Recognition and Indexing of handwritten documents and faxes)

http://www.cvc.uab.es/icdar2009/papers/3725a941.pdf

Results of the RIMES Evaluation Campaign for Handwritten Mail Processing

This paper addresses the problem of layout analysis of handwritten letters using textural and spatial information with a bidimensional Markovian approach. In this framework, the layout extraction is viewed as a labeling problem which aims to find the optimal configuration of the Markov random field performed by the 2D dynamic programming method by E. Geoffrois (2003). Preliminary experiments have been led on a small part of the RIMES database by E.Augustine el al. (2006) for which an error rate of 15% has been achieved.

Preliminary experiments in layout analysis of handwritten letters based on textural and spatial information and a 2D Markovian approach

In this paper, we present a two-dimensional approach of the processing of handwriting. It combines a Markovian model, an efficient decoding algorithm, a windowed spectral features extraction scheme and a rigorous evaluation methodology. We applied this principle to a digit recognition task and to a word recognition task.

A generic 2D approach of handwriting recognition

Processing text components in multimedia contents remains a challenging issue for document indexing and retrieval. More specifically, handwritten characters processing is a very active field of pattern recognition. This paper describes an innovative two-dimensional approach for character recognition and segmentation. The method proposed combines Markovian modeling, efficient decoding algorithm together with a windowed spectral features extraction scheme. A rigorous evaluation methodology is achieved to analyse and discuss the performances obtained for digit and word recognition.

Edouard Geoffrois

Papers

RIMES evaluation campaign for handwritten mail processing

Results of the RIMES Evaluation Campaign for Handwritten Mail Processing

Preliminary experiments in layout analysis of handwritten letters based on textural and spatial information and a 2D Markovian approach

A generic 2D approach of handwriting recognition

2D Markovian modeling for character recognition and segmentation