Showing papers on "Handwriting recognition published in 2001"

Gradient-based learning applied to document recognition

Abstract: Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

GradientBased Learning Applied to Document Recognition

Abstract: Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure. Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks. A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.

Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition systems

Abstract: In this paper, a system for the reading of totally unconstrained handwritten text is presented. The kernel of the system is a hidden Markov model (HMM) for handwriting recognition. The HMM is enhanced by a statistical language model. Thus linguistic knowledge beyond the lexicon level is incorporated in the recognition process. Another novel feature of the system is that the HMM is applied in such a way that the difficult problem of segmenting a line of text into individual words is avoided. A number of experiments with various language models and large vocabularies have been conducted. The language models used in the system were also analytically compared based on their perplexity.

Online handwriting recognition: the NPen++ recognizer

Abstract: This paper presents the online handwriting recognition system NPen++ developed at the University of Karlsruhe and Carnegie Mellon University The NPen++ recognition engine is based on a multi-state time delay neural network and yields recognition rates from 96% for a 5,000 word dictionary to 934% on a 20,000 word dictionary and 912% for a 50,000 word dictionary The proposed tree search and pruning technique reduces the search space considerably without losing too much recognition performance compared to an exhaustive search This enables the NPen++ recognizer to be run in real-time with large dictionaries Initial recognition rates for whole sentences are promising and show that the MS-TDNN architecture is suited to recognizing handwritten data ranging from single characters to whole sentences

The role of holistic paradigms in handwritten word recognition

Abstract: The holistic paradigm in handwritten word recognition treats the word as a single, indivisible entity and attempts to recognize words from their overall shape, as opposed to their character contents. In this survey, we have attempted to take a fresh look at the potential role of the holistic paradigm in handwritten word recognition. The survey begins with an overview of studies of reading which provide evidence for the existence of a parallel holistic reading process,in both developing and skilled readers. In what we believe is a fresh perspective on handwriting recognition, approaches to recognition are characterized as forming a continuous spectrum based on the visual complexity of the unit of recognition employed and an attempt is made to interpret well-known paradigms of word recognition in this framework. An overview of features, methodologies, representations, and matching techniques employed by holistic approaches is presented.

Writer identification using text line based features

Abstract: We present a system for writer identification. From handwritten lines of text, twelve features are extracted which are used to recognize persons, based on their handwriting. The features extracted mainly correspond to visible characteristics of the writing, for example, the width, the slant and the height of the three main writing zones. Additionally, features based on the fractal behavior of the writing, which are correlated with the writing's legibility, are used. With these features two classifiers are applied: a k-nearest neighbor and a feedforward neural network classifier. In the experiments, 100 pages of text written by 20 different writers are used. By classifying individual text lines, an average recognition rate of 87.8% for the k-nearest neighbor and 90.7% for the neural network is measured. By a simple maximum ranking over all lines of a page, all texts are correctly assigned to the corresponding writers. Compared to these results, an average recognition rate of 98% was measured when humans assigned persons to the text lines.

Pattern Recognition: From Classical to Modern Approaches

Abstract: Pattern recognition - evolution of methodologies and data mining, A. Pal and S.K. Pal adaptive stochastic algorithms for pattern classification, M.A.L. Thathachar and P.S. Sastry shape in images, K.V. Mardia decision trees for classification - a review and some new results, R. Kothari and M. Dong syntactic pattern recognition, A.K. Majumder and A.K. Ray fuzzy sets as a logic canvas for pattern recognition, W. Pedrycz and N. Pizzi neural network based pattern recognition, V. David Sanchez networks of spiking neurons in data mining, K. Cios and D.M. Sala genetic algorithms, pattern classification and neural networks design, S. Bandyopadhyay et al rough sets in pattern recognition, A. Skowron and R. Swiniarski automated generation of qualitative representations of complex objects by hybrid soft-computing methods, E.H. Ruspini and I.S. Zwir) writing speed and writing sequence invariant on-line handwriting recognition, S-H Cha and S.N. Srihari tongue diagnosis based on biometric pattern recognition technology, K. Wang et al and other papers.

Individuality of handwriting: a validation study

Abstract: Motivated by several rulings in United States courts concerning expert testimony in general and handwriting testimony in particular, we undertook a study to objectively validate the hypothesis that handwriting is individualistic. Handwriting samples of 1500 individuals, representative of the US population with respect to gender, age, ethnic groups, etc., were obtained. Analyzing differences in handwriting was done by using computer algorithms for extracting features from scanned images of handwriting. Attributes characteristic of the handwriting were obtained, e.g., line separation, slant, character shapes, etc. These attributes, which are a subset of attributes used by expert document examiners, were used to quantitatively establish individuality by using machine learning approaches. Using global attributes of handwriting and very few characters in the writing, the ability to determine the writer with a high degree of confidence was established. The work is a step towards providing scientific support for admitting handwriting evidence in court. The mathematical approach and the resulting software also have the promise of aiding the expert document examiner.

In-place adaptive handwriting input method and system

Abstract: A system and method that displays a semi-transparent user input interface relative to an application's currently focused input field at times when handwritten input is appropriate. The semi-transparent user interface starts when a program's text input field receives focus, can grow as needed to receive input, or will disappear when not used for a time. Handwritten data is recognized and passed to the application as if it was typed in the focused field, and the application need not be aware of handwriting, as the system and method are external to the application. Pen events that are not handwriting, but comprise gestures directed to the program through the semi-transparent input user interface, are detected by a gesture detection engine and sent to the application. A user is thus guided to enter handwriting, while handwriting recognition appears to be built into applications, whether or not those applications are aware of handwriting.

Separating handwritten material from machine printed text using hidden Markov models

Abstract: In this paper, we address the problem of separating handwritten annotations from machine-printed text within a document. We present an algorithm that is based on the theory of hidden Markov models (HMMs) to distinguish between machine-printed and handwritten materials. No OCR results are required prior to or during the process, and the classification is performed at the word level. Handwritten annotations are not limited to marginal areas, as the approach can deal with document images having handwritten annotations overlaid on machine-printed text and it has been shown to be promising in our experiments. Experimental results show that the proposed method can achieve 72.19% recall for fully extracted handwritten words and 90.37% for partially extracted words. The precision of extracting handwritten words has reached 92.86%.

Substroke approach to HMM-based on-line Kanji handwriting recognition

Abstract: A new method is proposed for online handwriting recognition of Kanji characters. The method employs substroke HMM as minimum units to constitute Japanese Kanji characters and utilizes the direction of pen motion. The main motivation is to fully utilize the continuous speech recognition algorithm by relating sentence speech to Kanji character phonemes to substrokes, and grammar to Kanji structure. The proposed system consists input feature analysis, substroke HMM, a character structure dictionary and a decoder. The present approach has the following advantages over the conventional methods that employ whole character HMM. 1) Much smaller memory requirement for dictionary and models. 2) Fast recognition by employing efficient substroke network search. 3) Capability of recognizing characters not included in the training data if defined as a sequence of substrokes in the dictionary. 4) Capability of recognizing characters written by various different stroke orders with multiple definitions per one character in the dictionary. 5) Easiness in HMM adaptation to the user with a few sample character data.

Hidden Markov models: applications in computer vision

Abstract: Introduction - a simple complex in artificial intelligence and machine learning, B.H. Juang an introduction to hidden Markov models and Bayesian networks, Z. Chahramani multi-lingual machine printed OCR, P. Natarajan et al using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system, U.-V. Marti and H. Bunke a 2-D HMM method for offline handwritten character recognition, H.-S. Park et al data-driven design for HMM topology for online handwriting recognition, J.J. Lee et al hidden Markov models for modelling and recognizing gesture under variation, A.D. Wilson and A.F. Bobick sentence lipreading using hidden Markov model with integrated grammar, K. Yu et al tracking and surveillance in wide-area spatial environments using the abstract hidden Markov model, H.H. Bui et al shape tracking and production using hidden Markov models, T. Caelli et al an integrated approach to shape and colour-based image retrieval of rotated objects using hidden Markov models, S. Muller et al.

Establishing handwriting individuality using pattern recognition techniques

Abstract: We undertook a study to objectively validate the hypothesis that handwriting is individualistic. Handwriting samples of one thousand five hundred individuals, representative of the US population with respect to gender age, ethnic groups, etc., were obtained. Analyzing differences in handwriting was done by using computer algorithms for extracting features from scanned images of handwriting. Attributes characteristic of the handwriting were obtained, e.g., line separation, slant, character shapes, etc. These attributes, which are a subset of attributes used by expert document examiners, were used to quantitatively establish individuality by using machine learning approaches. Using global attributes of handwriting and very few characters in the writing, the ability to determine the writer with a high degree of confidence was established. The work is a step towards providing scientific support for admitting handwriting evidence in court. The mathematical approach and the resulting software also have the promise of aiding the expert document examiner.

Semi-transparent handwriting recognition UI

Abstract: A user interface of a handwriting recognition system intended for use in small electronic devices, such as PDAs, mobile Telephones and laptop computers. The user interface is a semi-transparent window that opens in response to a user-initiated manuscript input to any point on a touch-activated screen of a display of the electronic device. The semi-transparent window may be resized or moved, as desired by the user, and may be automatically sizable in response to the placement of the user's manuscript input on the touch-activated screen.

Text line segmentation and word recognition in a system for general writer independent handwriting recognition

Abstract: We present a system for recognizing unconstrained English handwritten text based on a large vocabulary. We describe the three main components of the system, which are preprocessing, feature extraction and recognition. In the preprocessing phase the handwritten texts are first segmented into lines. Then each line of text is normalized with respect to of skew, slant, vertical position and width. After these steps, text lines are segmented into single words. For this purpose distances between connected components are measured. Using a threshold, the distances are divided into distances within a word and distances between different words. A line of text is segmented at positions where the distances are larger than the chosen threshold. From each image representing a single word, a sequence of features is extracted. These features are input to a recognition procedure which is based on hidden Markov models. To investigate the stability of the segmentation algorithm the threshold that separates intra- and inter-word distances from each other is varied. If the threshold is small many errors are caused by over-segmentation, while for large thresholds under-segmentation errors occur. The best segmentation performance is 95.56% correctly segmented words, tested on 541 text lines containing 3899 words. Given a correct segmentation rate of 95.56%, a recognition rate of 73.45% on the word level is achieved.

Genetic algorithms for feature selection and weighting, a review and study

Abstract: Our aim is: a) to present a comprehensive survey of previous attempts at using genetic algorithms (GA) for feature selection in pattern recognition applications, with a special focus on character recognition; and b) to report on work that uses GA to optimize the weights of the classification module of a character recognition system. The main purpose of feature selection is to reduce the number of features, by eliminating irrelevant and redundant features, while simultaneously maintaining or enhancing classification accuracy. Many search algorithms have been used for feature selection. Among those, GA have proven to be an effective computational method, especially in situations where the search space is uncharacterized (mathematically), not fully understood, or/and highly dimensional.

Measuring HMM similarity with the Bayes probability of error and its application to online handwriting recognition

Abstract: We propose a novel similarity measure for hidden Markov models (HMMs). This measure calculates the Bayes probability of error for HMM state correspondences and propagates it along the Viterbi path in a similar way to the HMM Viterbi scoring. It can be applied as a tool to interpret misclassifications, as a stop criterion in iterative HMM training or as a distance measure for HMM clustering. The similarity measure is evaluated in the context of online handwriting recognition on lower case character models which have been trained from the UNIPEN database. We compare the similarities with experimental classifications. The results show that similar and misclassified class pairs are highly correlated. The measure is not limited to handwriting recognition, but can be used in other applications that use HMM based methods.

On-line handwritten signature verification using wavelets and back-propagation neural networks

Abstract: This paper investigates dynamic handwritten signature verification (HSV) using the wavelet transform with verification by the backpropagation neural network (NN). It is yet another avenue in the approach to HSV that is found to produce excellent results when compared with other methods of dynamic, or on-line, HSV. Using a database of dynamic signatures collected from 41 Chinese writers and 7 from Latin script we extract features (including pen pressure, x and y velocity, angle of pen movement and angular velocity) from the signature and apply the Daubechies-6 wavelet transform using coefficients as input to a NN which learns to verify signatures with a False Rejection Rate (FRR) of 0.0% and False Acceptance Rate (FAR) less of than 0.1.

Handwriting user interface for personal digital assistants and the like

Abstract: The present invention concerns a graphical handwriting user interface for a handheld device. As each handwritten word is entered (142) at a handwriting input area, the handwritten entry is checked for completeness (144) by selecting a designated key or by gesturing a writing instrument. When the handwritten entry is complete, a handwriting recognition engine matches (146) the handwritten input against words in a system dictionary as supplemented by a user dictionary. A confidence score is then attached (148) to the top scoring word. If the confidence level is high enough (154), then it is inserted in the input buffers as primary word choice for that handwritten word and the user may decide (156) whether the primary word is correct. If the confidence level is not high enough, then the user is prompted (158) with an indication that the recognition result is less reliable.

Measuring the Usability of Text Input Methods for Children

Abstract: This paper describes an experiment in which children aged between 6 and 10 entered text into a word processor using four different input methods, mouse, keyboard, speech recognition, and handwriting recognition. Several different measures of usability were made in an attempt to assess the suitability of the input methods in this situation. The paper describes and discusses the measures and their use with very young children.

Data-driven design of HMM topology for online handwriting recognition

Abstract: Although HMM is widely used for online handwriting recognition, there is no simple and well-established method of designing the HMM topology. We propose a data-driven systematic method to design HMM topology. Data samples in a single pattern class are structurally simplified into a sequence of straight-line segments. Then the resulting multiple models of the class are combined to form an architecture of a multiple parallel-path HMM, which behaves as single HMM. To avoid excessive growing of the number of the states, parameter trying is applied such that structural similarity among patterns is reflected. Experiments on online Hangul recognition showed about 19% of error reductions, compared to the intuitive deisgn method.

On-line signature verification algorithm incorporating pen position, pen pressure and pen inclination trajectories

Abstract: A new algorithm is proposed for pen-input on-line signature verification incorporating pen-position, pen-pressure and pen-inclinations trajectories. Preliminary experimental result looks encouraging.

On the influence of vocabulary size and language models in unconstrained handwritten text recognition

Abstract: In this paper we present a system for unconstrained handwritten text recognition The system consists of three components: preprocessing, feature extraction and recognition In the preprocessing phase, a page of handwritten text is divided into its lines and the writing is normalized by means of skew and slant correction, positioning and scaling From a normalized text line image, features are extracted using a sliding window technique From each position of the window nine geometrical features are computed The core of the system, the recognizes is based on hidden Markov models For each individual character, a model is provided The character models are concatenated to words using a vocabulary Moreover, the word models are concatenated to models that represent full lines of text Thus the difficult problem of segmenting a line of text into its individual words can be overcome To enhance the recognition capabilities of the system, a statistical language model is integrated into the hidden Markov model framework To preselect useful language models and compare them, perplexity is used Both perplexity as originally proposed and normalized perplexity are considered

Rotationally desensitized unistroke handwriting recognition

Abstract: A Unistrokes symbollogy in which strokes of like profile (i.e., strokes that are distinguished from each other by their rotational orientation) are rotationally offset from each other by at least 90° is provided. This provides a sufficient tolerance for disambiguating these strokes when they are written into hand-held pen computers and the like by users having widely divergent hand writing styles.

Off-line general handwritten word recognition using an approximate BEAM matching algorithm

Abstract: A recognition system for general isolated off-line handwritten words using an approximate segment-string matching algorithm is described. The fundamental paradigm employed is a character-based segment-then-recognize/match strategy. An additional user supplied contextual information in the form of a lexicon guides a graph search to estimate the most likely word image identity. This system is designed to operate robustly in the presence of document noise, poor handwriting, and lexicon errors. A pre-processing step is initially applied to the image to remove noise artifacts and normalize the handwriting. An oversegmentation approach is used to improve the likelihood of capturing the individual characters embedded in the word. A directed graph is constructed that contains many possible interpretations of the word image, many implausible. The most likely graph path and associated confidence is computed for each lexicon word to produce a final lexicon ranking. Experiments highlighting the characteristics of this algorithm are given.

Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters

Abstract: This paper describes an adaptive recognition system for isolated handwritten characters and the ex- periments carried out with it. The characters used in our experiments are alphanumeric characters, including both the upper- and lower-case versions of the Latin al- phabets and three Scandinavian diacriticals. The writers are allowed to use their own natural style of writing. The recognition system is based on the k-nearest neighbor rule. The six character similarity measures applied by the system are all based on dynamic time warping. The aim of the first experiments is to choose the best combi- nation of the simple preprocessing and normalization op- erations and the dissimilarity measure for a multi-writer system. However, the main focus of the workis on online adaptation. The purpose of the adaptations is to turn a writer-independent system into writer-dependent and increase recognition performance. The adaptation is car- ried out by modifying the prototype set of the classifier according to its recognition performance and the user's writing style. The ways of adaptation include: (1) adding new prototypes; (2) inactivating confusing prototypes; and (3) reshaping existing prototypes. The reshaping al- gorithm is based on the Learning Vector Quantization. Four different adaptation strategies, according to which the modifications of the prototype set are performed, have been studied both offline and online. Adaptation is carried out in a self-supervised fashion during normal use and thus remains unnoticed by the user.

Spatial sorting and formatting for handwriting recognition

Abstract: Systems and methods for reordering unconstrained handwriting data using both spatial and temporal interrelationships prior to recognition, and for spatially organizing and formatting machine recognized transcription results. The present invention allows a machine recognizer to generate and present a full and accurate transcription of unconstrained handwriting in its correct spatial context such that the transcription output can appear to “mirror” the corresponding handwriting.

An offline cursive handwritten word recognition system

Abstract: This paper describes an offline cursive handwritten word recognition system that combines hidden Markov models (HMM) and neural networks (NN). Using a fast left-right slicing method, we generate a segmentation graph that describes all possible ways to segment a word into letters. The NN computes the observation probabilities for each letter hypothesis in the segmentation graph. Then, the HMM compute the likelihood for each word in the lexicon by summing the probabilities over all possible paths through the graph. We present the preprocessing and the recognition process as well as the training procedure for the NN-HMM hybrid system. Another recognition system based on discrete HMM is also presented for performance comparison. The latter is also used for bootstrapping the NN-HMM hybrid system. Recognition performances of the two recognition systems using two image databases of French isolated words are presented. This paper is one of the first publications using the IRONOFF database, and thus can be used as a reference for future work on this database.

PenCalc: a novel application of on-line mathematical expression recognition technology

Abstract: Most of the calculator programs found in existing pen-based mobile computing devices, such as personal digital assistants (PDA) and other handheld devices, do not take full advantages of the pen technology offered by these devices. Instead, input of expressions is still done through a virtual keypad shown on the screen, and the stylus (i.e., electronic pen) is simply used as a pointing device. In this paper we propose an intelligent handwriting-based calculator program with which the user can enter expressions simply by writing them on the screen using a stylus. In addition, variables can be defined to store intermediate results for subsequent calculations, as in ordinary algebraic calculations. The proposed software is the result of a novel application of on-line mathematical expression recognition technology which has mostly been used by others only for some mathematical expression editor programs.

Sentence recognition through hybrid neuro-Markovian modeling

Abstract: This paper focuses on designing a handwriting recognition system dealing with on-line signal, i.e. temporal handwriting signal captured through an electronic pen or a digitalized tablet. We present here some new results concerning a hybrid on-line handwriting recognition system based on Hidden Markov Models (HMMs) and Neural Networks (NNs), which has already been presented in several contributions. In our approach, a letter-model is a Left-Right HMM, whose emission probability densities are approximated with mixtures of predictive multilayer perceptrons. The basic letter models are cascaded in order to build models for words and sentences. At the word level, recognition is performed thanks to a dictionary organized with a tree-structure. At the sentence level, a word-predecessor conditioned frame synchronous beam search algorithm allows to perform simultaneously segmentation into words and word recognition. It processes through the building of a word graph from which a set of candidate sentences may be extracted. Word and sentence recognition performances are evaluated on parts of the UNIPEN international database.