Recurrent neural networks are powerful sequence learners. They are able to incorporate context information in a flexible way, and are robust to localised distortions of the input data. These properties make them well suited to sequence labelling, where input sequences are transcribed with streams of labels. The aim of this thesis is to advance the state-of-the-art in supervised sequence labelling with recurrent networks. Its two main contributions are (1) a new type of output layer that allows recurrent networks to be trained directly for sequence labelling tasks where the alignment between the inputs and the labels is unknown, and (2) an extension of the long short-term memory network architecture to multidimensional data, such as images and video sequences.

https://mediatum.ub.tum.de/doc/673554/file.pdf

Supervised Sequence Labelling with Recurrent Neural Networks

Recognizing lines of unconstrained handwritten text is a challenging task. The difficulty of segmenting cursive or overlapping characters, combined with the need to exploit surrounding context, has led to low recognition rates for even the best current recognizers. Most recent progress in the field has been made either through improved preprocessing or through advances in language modeling. Relatively little work has been done on the basic recognition algorithms. Indeed, most systems rely on the same hidden Markov models that have been used for decades in speech and handwriting recognition, despite their well-known shortcomings. This paper proposes an alternative approach based on a novel type of recurrent neural network, specifically designed for sequence labeling tasks where the data is hard to segment and contains long-range bidirectional interdependencies. In experiments on two large unconstrained handwriting databases, our approach achieves word recognition accuracies of 79.7 percent on online data and 74.1 percent on offline data, significantly outperforming a state-of-the-art HMM-based system. In addition, we demonstrate the network's robustness to lexicon size, measure the individual influence of its hidden layers, and analyze its use of context. Last, we provide an in-depth discussion of the differences between the network and HMMs, suggesting reasons for the network's superior performance.

/pdf/a-novel-connectionist-system-for-unconstrained-handwriting-yx43b6vvre.pdf

A Novel Connectionist System for Unconstrained Handwriting Recognition

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.

/pdf/markov-models-for-offline-handwriting-recognition-a-survey-4ijdjn6mf0.pdf

Markov models for offline handwriting recognition: a survey

Handwritten word-spotting using hidden Markov models and universal vocabularies

This paper reviews the state of the art in off-line Roman cursive handwriting recognition. The input provided to an off-line handwriting recognition system is an image of a digit, a word, or - more generally -some text, and the system produces, as output, an ASCII transcription of the input. This task involves a number of processing steps, some of which are quite difficult. Typically, preprocessing, normalization, feature extraction, classification, and postprocessing operations are required. We'll survey the state of the art, analyze recent trends, and try to identify challenges for future research in this field.

Recognition of cursive Roman handwriting: past, present and future

This paper proposes a sequential coupling of a hidden Markov model (HMM) recognizer for offline handwritten English sentences with a probabilistic bottom-up chart parser using stochastic context-free grammars (SCFG) extracted from a text corpus. Based on extensive experiments, we conclude that syntax analysis helps to improve recognition rates significantly.

/pdf/offline-grammar-based-recognition-of-handwritten-sentences-hc1qufm69c.pdf

Offline grammar-based recognition of handwritten sentences

In this paper we apply a novel non-uniform slant correction preprocessing technique to improve the recognition of offline handwritten text lines. The local slant correction is expressed as a global optimisation problem of the sequence of local slant angles. This is different to conventional slant removal techniques that rely on the average slant angle. Experiments based on a state-of-the-art handwritten text line recogniser show a significant gain in word level accuracy for the investigated preprocessing methods.

Non-Uniform Slant Correction for Handwritten Text Line Recognition

This work investigates three different rejection strategies for offline handwritten sentence recognition. The rejection strategies are implemented as a postprocessing step of a hidden Markov model based text recognition system and are based on confidence measures derived from a list of candidate sentences produced by the recognizer. The better performing confidence measures make use of the fact that the recognizer integrates a word bigram language model. Experimental results on extracted sentences from the IAM database validate the effectiveness of the proposed rejection strategies.

Rejection strategies for offline handwritten sentence recognition

This paper investigates the application of a probabilistic parser for natural language on the list of the N-best sentences produced by an offline recognition system for cursive handwritten sentences. For the generation of the N-best sentence list an HMM-based recognizer including a bigram language model is used. The parsing of the sentences is achieved by a bottom-up chart parser for stochastic context-free grammars which produces the parse tree of the input sentence as well as the word tags. From a collection of corpora we extract the linguistic resources to build the lexicon, a word bigram model and the stochastic context-free grammar. Results from experiments indicate an increase of the word and sentence recognition rate when using the proposed combination scheme.

M. Zimmermann

Papers

Offline grammar-based recognition of handwritten sentences

Non-Uniform Slant Correction for Handwritten Text Line Recognition

Rejection strategies for offline handwritten sentence recognition

Parsing N-best lists of handwritten sentences