Showing papers by "Ching Y. Suen published in 2002"

Automatic recognition of handwritten numerical strings: a recognition and verification strategy

Abstract: A modular system to recognize handwritten numerical strings is proposed. It uses a segmentation-based recognition approach and a recognition and verification strategy. The approach combines the outputs from different levels such as segmentation, recognition, and postprocessing in a probabilistic model. A new verification scheme which contains two verifiers to deal with the problems of oversegmentation and undersegmentation is presented. A new feature set is also introduced to feed the oversegmentation verifier. A postprocessor based on a deterministic automaton is used and the global decision module makes an accept/reject decision. Finally, experimental results on two databases are presented: numerical amounts on Brazilian bank checks and NIST SD19. The latter aims at validating the concept of modular system and showing the robustness of the system using a well-known database.

Feature selection using multi-objective genetic algorithms for handwritten digit recognition

Abstract: Discusses the use of genetic algorithms for feature selection for handwriting recognition. Its novelty lies in the use of multi-objective genetic algorithms where sensitivity analysis and neural networks are employed to allow the use of a representative database to evaluate fitness and the use of a validation database to identify the subsets of selected features that provide a good generalization. Comprehensive experiments on the NIST database confirm the effectiveness of the proposed strategy.

A hybrid large vocabulary handwritten word recognition system using neural networks with hidden Markov models

Abstract: We present a hybrid recognition system that integrates hidden Markov models (HMM) with neural networks (NN) in a probabilistic framework. The input data is processed first by a lexicon-driven word recognizer based on HMMs to generate a list of the candidate N-best-scoring word hypotheses as well as the segmentation of such word hypotheses into characters. An NN classifier is used to generate a score for each segmented character and in the end, the scores from the HMM and the NN classifiers are combined to optimize performance. Experimental results show that for an 80,000-word vocabulary, the hybrid HMM/NN system improves by about 10% the word recognition rate over the HMM system alone.

Optimization of the SVM Kernels Using an Empirical Error Minimization Scheme

Abstract: We address the problem of optimizing kernel parameters in Support Vector Machine modelling, especially when the number of parameters is greater than one as in polynomial kernels and KMOD, our newly introduced kernel The present work is an extended experimental study of the framework proposed by Chapelle et al for optimizing SVM kernels using an analytic upper bound of the error However, our optimization scheme minimizes an empirical error estimate using a Quasi-Newton technique The method has shown to reduce the number of support vectors along the optimization process In order to assess our contribution, the approach is further used for adapting KMOD, RBF and polynomial kernels on synthetic data and NIST digit image database The method has shown satisfactory results with much faster convergence in comparison with the simple gradient descent methodFurthermore, we also experimented two more optimization schemes based respectively on the maximization of the margin and on the minimization of an approximated VC dimension estimate While both of the objective functions are minimized, the error is not The corresponding experimental results we carried out show this shortcoming

Empirical error based optimization of SVM kernels: application to digit image recognition

Abstract: We address the problem of optimizing kernel parameters in support vector machine modeling, especially when the number of parameters is greater than one as in polynomial kernels and KMOD, our newly introduced kernel. The present work is an extended experimental study of the framework proposed by Chapelle et al. (2001) for optimizing SVM kernels using an analytic upper bound of the error. However our optimization scheme minimizes an empirical error estimate using a quasi-Newton optimization method. To assess our method, the approach is further used for adapting KMOD, RBF and polynomial kernels on synthetic data and NIST database. The method shows a much faster convergence with satisfactory results in comparison with the simple gradient descent method.

A Fast SVM Training Algorithm

Abstract: A fast support vector machine (SVM) training algorithm is proposed under the decomposition framework of SVM's algorithm by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Extensive experiments on MNIST handwritten digit database have been conducted to show that the proposed algorithm is much faster than Keerthi et al.'s improved SMO, about 9 times. Combined with principal component analysis, the total training for ten one against the-rest classifiers on MNIST took just 0.77 hours. The promising scalability of the proposed scheme can make it possible to apply SVM to a wide variety of problems in engineering.

KMOD - a two-parameter SVM kernel for pattern recognition

Abstract: It has been shown that the support vector machine (SVM) theory optimizes a smoothness functional hypothesis through kernel applications. We present KMOD, a two-parameter SVM kernel with distinctive properties of good discrimination between patterns while preserving the data neighborhood information. In classification problems, the experiments we carried out on the breast cancer benchmark produced better performance than the RBF kernel and some state of the art classifiers. It also generated favorable results when subjected to a 10-class problem of recognizing handwritten digits in the NIST database.

Segmentation and recognition of handwritten dates

Abstract: Presents an HMM-MLP hybrid system to recognize complex date images written on Brazilian bank cheques. The system first segments implicitly a date image into sub-fields through the recognition process based on an HMM-based approach. Afterwards, the three obligatory date sub-fields are processed by the system (day, month and year). A neural approach has been adopted to work with strings of digits and a Markovian strategy to recognize and verify words. We also introduce the concept of meta-classes of digits, which is used to reduce the lexicon size of the day and year and improve the precision of their segmentation and recognition. Experiments show interesting results on date recognition.

Recognition of courtesy amounts on bank checks based on a segmentation approach

Abstract: A segmentation based courtesy amount recognition (CAR) system is presented in this paper. A two-stage segmentation module has been proposed, namely the global segmentation stage and the local segmentation stage. At the global segmentation stage, a courtesy amount is coarsely segmented into sub-images according to the spatial relationships of the connected components. These sub-images are then verified by the recognition module and the rejected sub-images are sequentially split using contour analysis at the local segmentation stage. Two neural network classifiers are combined into a recognition module. The isolated digit classifier divides the input patterns into ten numeral classes (0-9), while the holistic double zeros classifier recognizes the cursive and touching double zeros. Experimental results show that the system reads 66.5% bank checks correctly at 0% misreading rate.

Recognition of similar objects using 2-D wavelet-fractal feature extraction

Abstract: A new two dimensional (2-D) object recognition method is proposed to differentiate similar objects, detect defective objects, and recognize printed characters. First, a 2-D image is transformed to a weighted shape matrix to secure invariance in translation, scaling, rotation, and split into four dyadic subimages. Wavelet transformation is applied to each subimage in order to further explore its details in different directions and to achieve image subband decomposition. Finally, an efficient and effective 2-D image fractal algorithm is used to extract each subband coefficient as a feature for classification. A series of experiments were conducted on binary objects and character images for recognition and classification. The experimental results showed that the proposed method is especially effective in classifying similar objects and the recognition rate could be very high in the recognition of printed characters.

Recognition of handwritten month words on bank cheques

Abstract: This paper describes an off-line system which recognizes unconstrained handwritten month words extracted from Canadian bank cheques. A segmentation based grapheme level HMM (hidden Markov model) classifier and two multilayer perceptron classifiers with different architectures and different features have been developed in CENPARMI for the recognition of month words. In this paper, a combination method with an effective conditional topology is presented, and the most widely used combination rules including Vote, Sum and Product, are experimented. A new modified Product rule is also proposed, which has produced the best recognition rate of 85.36% when tested on a real-life standard Canadian bank cheque database.

Fast two-level Viterbi search algorithm for unconstrained handwriting recognition

Abstract: This paper describes a fast two-level Viterbi search algorithm for recognizing handwritten words as a sequence of characters concatenated according to a lexicon. The algorithm is based on hidden Markov model (HMM) representations of characters and it breaks up the computation of word likelihood scores into two levels: state level and character level. This enables the reuse of likelihood scores of characters to decode all words in the lexicon, avoiding repeated computation of state sequences. Experimental results with an 85,000-word vocabulary indicate that the computational cost of an off-line handwritten word recognition system may be reduced by more than a factor of 20 while not introducing search errors.

An HMM-MLP hybrid system to recognize handwritten dates

Abstract: Presents an HMM-MLP hybrid system to process complex date images written on Brazilian bank cheques. The system first segments implicitly a date image into sub-fields through the recognition process based on an HMM approach. Afterwards, a recognition and verification strategy is proposed to recognize the three obligatory date sub-fields (day, month and year) using different classifiers. Markovian and neural approaches have been adopted to recognize and verify words and strings of digits respectively. We also introduce the concept of meta-classes of digits, which is used to reduce the lexicon size of the day and year and improve the precision of their segmentation and recognition. Experiments show interesting results on date recognition.

Content analysis in document images: a scale space approach

Abstract: With the growing interest in automatic transformation of paper document to its electronic version, geometric and logical structures have become an active research area for a decade. Nowadays, kernel scale space has been widely adopted as the most promising multi-scale image document analysis method. Yet still, traditional methods using scale space approach has its limitations: they are useful mostly on character extraction and they carry a large computational load. In view of these limitations, this paper proposes a new approach using scale space in order to analyse the composite document content. In the proposed method, scale space transform is used to decompose an image into different scaled objects where the scale value is used for detecting progressively finer objects: text, line drawing, logo, and image, with encouraging results on real-life data.

Introducing termination probabilities to HMM

Abstract: HMM is very well suited to model sequential patterns. This paper introduces a new parameter, called the termination probability, to a hidden Markov model (HMM). The new parameter provides a better initialization for the backward variable during the training and evaluation phases. This improves the discriminatory power of HMM by allowing the system to judge the input observation sequence based on where it is completed. Experimental results show the improvement was achieved by this parameter.

A string length predictor to control the level building of HMMs for handwritten numeral recognition

Abstract: In this paper a two-stage HMM-based method for recognizing handwritten numeral strings is extended to work with handwritten numeral strings of unknown length. We have proposed a Bayesian-based string length predictor (SLP) to estimate the number of digits in a string taking into account its width in pixels. The top 3 decisions of the SLP module are used to control the maximum number of levels to be searched by the Level Building (LB) algorithm. On 12,802 handwritten numeral strings and 2,069 touching digit pairs, this strategy has shown a small loss. (0.91%) in terms of recognition performance compared to the results when the string length is considered as known.

Dynamic observations and dynamic state termination for off-line handwritten word recognition using HMM

Abstract: HMM has been successfully used to model 1D data, e.g. voice signals. Their use to model 2D patterns was not as successful due to a major difficulty, in describing the 2D data using 1D observation sequences. In this paper, we discuss the importance of this issue and present an improved method to extract 1D observations from the dynamics of off-line handwritten words. The method is based on pen trajectory estimation techniques. The paper also includes description of our HMM classifier which allows dynamic termination states to achieve enhanced discriminative power. Experimental results show the applicability and usefulness of the proposed method. As a result of using the termination probability in HMM modeling, the top 1/sup st/ recognition rate increased by 10%.

Efficient estimation of pen trajectory from off-line handwritten words

Abstract: This paper presents an easy and efficient method to estimate the pen trajectory based on minimizing the pen movement. Given start and end vertices, the complexity of the proposed algorithm is linear. In addition, the algorithm clearly identifies alternatives that do not affect the overall length of the pen trajectory, making enough room for other criteria, e.g. vision rules, to be applied.

Differentiation between alphabetic and numeric data using NN ensembles

Abstract: Presents an approach to differentiating handwritten alphabetic words from numeric strings, which is motivated by an application in a real-life bank cheque processing system. Neural networks are first considered with the focus on effective feature extraction and architecture selection. The combination of networks is then implemented to achieve highly reliable and accurate results.

An application of nonlinear wavelet approximation to face recognition

Abstract: In this paper we present an application of the nonlinear wavelet approximation to recognize faces. The advantages of the nonlinear wavelet approximation are compared with its linear counterpart. An efficient scheme in applying the nonlinear wavelet transform to face recognition is presented. Experiments using the Yale Face Library show very encouraging recognition results and demonstrate that this method has the desirable property of being insensitive to facial expressions.

Post-processing scheme for improving recognition performance of touching handwritten numeral strings

Abstract: This paper describes a post-processing scheme to improve recognition performance of touching handwritten numeral strings. Verification factors are defined to rectify the recognition results of digits that are segmented by the highest reliability values. Three kinds of verification factors from structural features and recognition probability are used to determine mis-segmented digits. Additional segment digit set is included in valid digit sets when it satisfies the reconsideration condition. A final optimal digit set is selected as the highest ranked segment digit set among all candidate segmented digit sets. Introduction of reconsideration condition and additional segment digit set improves the reliability of the recognition result as well as recognition performance. Experiments were carried out with touching handwritten numeral strings of NIST SD19 database and an encouraging recognition result was obtained.

Système hybride de reconnaissance de mots manuscrits sur un grand vocabulaire utilisant des réseaux neuronaux et des modèles de Markov cachés

Abstract: We present a hybrid recognition system that integrates hidden Markov models (HMM) and neural networks (NN) in a probabilistic framework. The words are firstly processed by a lexicon–driven handwritten word recognition system which is based on HMMs. A list with the N–best word hypotheses as well as the segmentation of such word hypotheses into characters is also produced. The NN classifier computes a score for each segmented character and the results of both HMM and NN classifiers are combined to optimize the recognition performance. Experimental results show that for an 80,000–word vocabulary, the hybrid HMM/NN recognition 2 CIFED. Volume X n◦ X/2002 system improves by 9% the recognition rate relatively to the HMM–based recognition system alone. MOTS-CLÉS :handwriting recognition, large vocabulary, hidden Markov models, neural net-

New algorithms for geometric transformations of digital images and patterns

Abstract: Nonlinear geometric transformations, such as the splitting-shooting method (SSM), the splitting-integrating method (SIM) and their combination (CSIM), and their advanced versions S~S~M, S~I~M, CS~I~M have been developed. This paper proposes new versions of such nonlinear transformations, S~S~/sup #/M, S~I~/sup #/M, CS~/sup #/I~/sup #/M, approximated by piecewise linear transformations to circumvent the necessity of finding the nonlinear solutions, and to obtain exact integration computationally. The absolute errors of pixel greyness are proven to be O(H), where H is the length of a pixel region. It is worth pointing out that the new algorithms in this paper do not produce any sequential errors as N/spl ges/N/sub 0/. Apart from this distinctive feature, the absolute error bound O(H) can be applied to all kinds of images with discontinuity, including fully isolated pixels.