Recognizing lines of unconstrained handwritten text is a challenging task. The difficulty of segmenting cursive or overlapping characters, combined with the need to exploit surrounding context, has led to low recognition rates for even the best current recognizers. Most recent progress in the field has been made either through improved preprocessing or through advances in language modeling. Relatively little work has been done on the basic recognition algorithms. Indeed, most systems rely on the same hidden Markov models that have been used for decades in speech and handwriting recognition, despite their well-known shortcomings. This paper proposes an alternative approach based on a novel type of recurrent neural network, specifically designed for sequence labeling tasks where the data is hard to segment and contains long-range bidirectional interdependencies. In experiments on two large unconstrained handwriting databases, our approach achieves word recognition accuracies of 79.7 percent on online data and 74.1 percent on offline data, significantly outperforming a state-of-the-art HMM-based system. In addition, we demonstrate the network's robustness to lexicon size, measure the individual influence of its hidden layers, and analyze its use of context. Last, we provide an in-depth discussion of the differences between the network and HMMs, suggesting reasons for the network's superior performance.

/pdf/a-novel-connectionist-system-for-unconstrained-handwriting-yx43b6vvre.pdf

A Novel Connectionist System for Unconstrained Handwriting Recognition

In this paper we describe a database that consists of handwritten English sentences. It is based on the Lancaster-Oslo/Bergen (LOB) corpus. This corpus is a collection of texts that comprise about one million word instances. The database includes 1,066 forms produced by approximately 400 different writers. A total of 82,227 word instances out of a vocabulary of 10,841 words occur in the collection. The database consists of full English sentences. It can serve as a basis for a variety of handwriting recognition tasks. However, it is expected that the database would be particularly useful for recognition tasks where linguistic knowledge beyond the lexicon level is used, because this knowledge can be automatically derived from the underlying corpus. The database also includes a few image-processing procedures for extracting the handwritten text from the forms and the segmentation of the text into lines and words.

The IAM-database: an English sentence database for offline handwriting recognition

Prior work in RF-based positioning has mainly focused on discovering the absolute location of an RF source, where state-of-the-art systems can achieve an accuracy on the order of tens of centimeters using a large number of antennas. However, many applications in gaming and gesture based interface see more benefits in knowing the detailed shape of a motion. Such trajectory tracing requires a resolution several fold higher than what existing RF-based positioning systems can offer. This paper shows that one can provide a dramatic increase in trajectory tracing accuracy, even with a small number of antennas. The key enabler for our design is a multi-resolution positioning technique that exploits an intrinsic tradeoff between improving the resolution and resolving ambiguity in the location of the RF source. The unique property of this design is its ability to precisely reconstruct the minute details in the trajectory shape, even when the absolute position might have an offset. We built a prototype of our design with commercial off-the-shelf RFID readers and tags and used it to enable a virtual touch screen, which allows a user to interact with a desired computing device by gesturing or writing her commands in the air, where each letter is only a few centimeters wide.

/pdf/rf-idraw-virtual-touch-screen-in-the-air-using-rf-signals-1z6hfms244.pdf

RF-IDraw: virtual touch screen in the air using RF signals

This paper demonstrates that it is possible to leverage WiFi signals from commodity mobile devices to enable hands-free drawing in the air. While prior solutions require the user to hold a wireless transmitter, or require custom wireless hardware, or can only determine a pre-defined set of hand gestures, this paper introduces WiDraw, the first hand motion tracking system using commodity WiFi cards, and without any user wearables. WiDraw harnesses the Angle-of-Arrival values of incoming wireless signals at the mobile device to track the user's hand trajectory. We utilize the intuition that whenever the user's hand occludes a signal coming from a certain direction, the signal strength of the angle representing the same direction will experience a drop. Our software prototype using commodity wireless cards can track the user's hand with a median error lower than 5 cm. We use WiDraw to implement an in-air handwriting application that allows the user to draw letters, words, and sentences, and achieves a mean word recognition accuracy of 91%.

/pdf/widraw-enabling-hands-free-drawing-in-the-air-on-commodity-3r4u320d9p.pdf

WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices

Since their first inception more than half a century ago, automatic reading systems have evolved substantially, thereby showing impressive performance on machine-printed text. The recognition of handwriting can, however, still be considered an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic offline handwriting recognition. However, so far, no standard procedures for building Markov-model-based recognizers could be established though trends toward unified approaches can be identified. It is therefore the goal of this survey to provide a comprehensive overview of the application of Markov models in the research field of offline handwriting recognition, covering both the widely used hidden Markov models and the less complex Markov-chain or n-gram models. First, we will introduce the typical architecture of a Markov-model-based offline handwriting recognition system and make the reader familiar with the essential theoretical concepts behind Markovian models. Then, we will give a thorough review of the solutions proposed in the literature for the open problems how to apply Markov-model-based approaches to automatic offline handwriting recognition.

/pdf/markov-models-for-offline-handwriting-recognition-a-survey-4ijdjn6mf0.pdf

Markov models for offline handwriting recognition: a survey

Due to large shape variations in human handwriting, recognition accuracy of cursive handwritten word is hardly satisfying using a single classifier. In this paper we introduce a framework to combine results of multiple classifiers and present an intuitive run-time weighted opinion pool combination approach for recognizing cursive handwritten words with a large size vocabulary. The individual classifiers are evaluated run-time dynamically. The final combination is weighted according to their local performance. For an open vocabulary recognition task, we use the ROVER algorithm to combine the different strings of characters provided by each classifier. Experimental results for recognizing cursive handwritten words demonstrate that our new approach achieves better recognition performance and reduces the relative error rate significantly.

Combination of multiple classifiers for handwritten word recognition

In this paper a system for on-line cursive handwriting recognition is described. The system is based on Hidden Markov Models (HMMs) using discrete and hybrid modeling techniques. Here, we focus on two aspects of the recognition system. First, we present different hybrid modeling techniques, whereas one depends on an information theory-based neural network (MMI-criterion) used as a vector quantizer and the other uses a neural net for estimating the a posteriori probabilities to replace the codebook of a tied-mixture HMM system. This is the first paper where we present this novel approach -called tied posteriors- for handwriting recognition. Second, we demonstrate the usage of a language model, that consists of character n-grams, as an alternative to the recognition with a large dictionary of German words. Our resulting system for character recognition yields significantly better recognition results using an unlimited vocabulary.

/pdf/off-line-handwriting-recognition-using-various-hybrid-2g38yd7m6f.pdf

Off-line handwriting recognition using various hybrid modeling techniques and character n-grams

This paper describes an online handwriting recognition system with focus on adaptation techniques. Our hidden Markov model (HMM)-based recognition system for cursive German script can be adapted to the writing style of a new writer using either a retraining depending on the EM (expectation maximization)-approach or an adaptation according to the MAP (maximum a posteriori) or MLLR (maximum likelihood linear regression)-criterion. The performance of the resulting writer-dependent system increases significantly even if the amount of adaptation data is very small (about 6 words). So this approach is also applicable for online systems in hand-held computers such as PDAs. Special attention was paid to the performance comparison of the different adaptation techniques with the availability of different amounts of adaptation data ranging from a few words tip to 100 words per writer.

Comparing adaptation techniques for on-line handwriting recognition

In this paper a recognition system, based on tied-mixture hidden Markov models, for handwritten address words is described, which makes use of a language model that consists of backoff character n-grams. For a dictionary-based recognition system it is essential that the structure of the address (name, street, city) is known. If the single parts of the address cannot be categorized, the used vocabulary is unknown and thus unlimited. The performance of this open vocabulary recognition using n-grams is compared to the use of dictionaries of different sizes. Especially, the confidence of recognition results and the possibility of a useful post-processing are significant advantages of language models.

Handwritten address recognition with open vocabulary using character n-grams

The paper deals with the performance evaluation of a novel hybrid approach to large vocabulary cursive handwriting recognition and contains various innovations. 1) It presents the investigation of a new hybrid approach to handwriting recognition, consisting of hidden Markov models (HMMs) and neural networks trained with a special information theory based training criterion. This approach has only been recently introduced successfully to online handwriting recognition and is now investigated for the first time for offline recognition. 2) The hybrid approach is extensively compared to traditional HMM modeling techniques and the superior performance of the new hybrid approach is demonstrated. 3) The data for the comparison has been obtained from a database containing online handwritten data which has been converted to offline data. Therefore, a multiple evaluation has been carried out, incorporating the comparison of different modeling techniques and the additional comparison of each technique for online and offline recognition, using a unique database. The results confirm that online recognition leads to better recognition results due to the dynamic information of the data, but also show that it is possible to obtain recognition rates for offline recognition that are close to the results obtained for online recognition. Furthermore, it can be shown that for both online and offline recognition, the new hybrid approach clearly outperforms the competing traditional HMM techniques. It is also shown that the new hybrid approach yields superior results for the offline recognition of machine printed multifont characters.

A. Brakensiek

Papers

Combination of multiple classifiers for handwritten word recognition

Off-line handwriting recognition using various hybrid modeling techniques and character n-grams

Comparing adaptation techniques for on-line handwriting recognition

Handwritten address recognition with open vocabulary using character n-grams

Performance evaluation of a new hybrid modeling technique for handwriting recognition using identical on-line and off-line data