Showing papers by "Ching Y. Suen published in 2003"

Large vocabulary off-line handwriting recognition: A survey

Abstract: Considerable progress has been made in handwriting recognition technology over the last few years. Thus far, handwriting recognition systems have been limited to small and medium vocabulary applications, since most of them often rely on a lexicon during the recognition process. The capability of dealing with large lexicons, however, opens up many more applications. This article will discuss the methods and principles that have been proposed to handle large vocabularies and identify the key issues affecting their future deployment. To illustrate some of the points raised, a large vocabulary off-line handwritten word recognition system will be described.

A methodology for feature selection using multiobjective genetic algorithms for handwritten digit string recognition

Abstract: In this paper a methodology for feature selection for the handwritten digit string recognition is proposed. Its novelty lies in the use of a multiobjective genetic algorithm where sensitivity analysis and neural network are employed to allow the use of a representative database to evaluate fitness and the use of a validation database to identify the subsets of selected features that provide a good generalization. Some advantages of this approach include the ability to accommodate multiple criteria such as number of features and accuracy of the classifier, as well as the capacity to deal with huge databases in order to adequately represent the pattern recognition problem. Comprehensive experiments on the NIST SD19 demonstrate the feasibility of the proposed methodology.

Unsupervised feature selection using multi-objective genetic algorithms for handwritten word recognition

Abstract: In this paper a methodology for feature selection in unsupervisedlearning is proposed. It makes use of a multi-objectivegenetic algorithm where the minimization of thenumber of features and a validity index that measures thequality of clusters have been used to guide the search towardsthe more discriminant features and the best numberof clusters. The proposed strategy is evaluated usingtwo synthetic data sets and then it is applied to handwrittenmonth word recognition. Comprehensive experimentsdemonstrate the feasibility and efficiency of the proposedmethodology.

Application of Support Vector Machines for Recognition of Handwritten Arabic/Persian Digits

Abstract: A new method for recognition of isolated handwritten Arabic/Persian digits is presented. This method is based on Support Vector Machines (SVMs), and a new approach of feature extraction. Each digit is considered from four different views, and from each view 16 features are extracted and combined to obtain 64 features. Using these features, multiple SVM classifiers are trained to separate different classes of digits. CENPARMI Indian (Arabic/Persian) handwritten digit database is used for training and testing of SVM classifiers. Based on this database, differences between Arabic and Persian digits in digit recognition are shown. This database provides 7390 samples for training and 3035 samples for testing from the real life samples. Experiments show that the proposed features can provide a very good recognition result using Support Vector Machines at a recognition rate 94.14%, compared with 91.25% obtained by MLP neural network classifier using the same features and test set.

A fast parallel optimization for training support vector machine

Abstract: A fast SVM training algorithm for multi-classes consisting of parallel and sequential optimizations is presented. The main advantage of the parallel optimization step is to remove most non-support vectors quickly, which dramatically reduces the training time at the stage of sequential optimization. In addition, some strategies such as kernel caching, shrinking and calling BLAS functions are effectively integrated into the algorithm to speed up the training. Experiments on MNIST handwritten digit database have shown that, without sacrificing the generalization performance, the proposed algorithm has achieved a speed-up factor of 110, when compared with Keerthi et al.'s modified SMO. Moreover, for the first time ever we investigated the training performance of SVM on handwritten Chinese database ETL9B with more than 3000 categories and about 500,000 training samples. The total training time is just 5.1 hours. The raw error rate of 1.1% on ETL9B has been achieved.

A fast svm training algorithm

Abstract: A fast support vector machine (SVM) training algorithm is proposed under SVM's decomposition framework by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Kernel caching plays a key role in reducing the number of kernel evaluations by maximal reusage of cached kernel elements. Extensive experiments have been conducted on a large handwritten digit database MNIST to show that the proposed algorithm is much faster than Keerthi et al.'s improved SMO, about nine times. Combined with principal component analysis, the total training for ten one-against-the-rest classifiers on MNIST took less than an hour. Moreover, the proposed fast algorithm speeds up SVM training without sacrificing the generalization performance. The 0.6% error rate on MNIST test set has been achieved. The promising scalability of the proposed scheme paves a new way to solve more large-scale learning problems in other domains such as data mining.

The recognition of handwritten numeral strings using a two-stage HMM-based method

Abstract: In this paper, a two-stage HMM-based recognition method allows us to compensate for the possible loss in terms of recognition performance caused by the necessary trade-off between segmentation and recognition in an implicit segmentation-based strategy. The first stage consists of an implicit segmentation process that takes into account some contextual information to provide multiple segmentation-recognition hypotheses for a given preprocessed string. These hypotheses are verified and re-ranked in a second stage by using an isolated digit classifier. This method enables the use of two sets of features and numeral models: one taking into account both the segmentation and recognition aspects in an implicit segmentation-based strategy, and the other considering just the recognition aspects of isolated digits. These two stages have been shown to be complementary, in the sense that the verification stage compensates for the loss in terms of recognition performance brought about by the necessary tradeoff between segmentation and recognition carried out in the first stage. The experiments on 12,802 handwritten numeral strings of different lengths have shown that the use of a two-stage recognition strategy is a promising idea. The verification stage brought about an average improvement of 9.9% on the string recognition rates. On touching digit pairs, the method achieved a recognition rate of 89.6%.

Feature selection for ensembles:a hierarchical multi-objective genetic algorithm approach

Abstract: Feature selection for ensembles has shown to be an effectivestrategy for ensemble creation. In this paper we presentan ensemble feature selection approach based on a hierarchicalmulti-objective genetic algorithm. The first level performsfeature selection in order to generate a set of goodclassifiers while the second one combines them to providea set of powerful ensembles. The proposed method is evaluatedin the context of handwritten digit recognition, usingthree different feature sets and neural networks (MLP) asclassifiers. Experiments conducted on NIST SD19 demonstratedthe effectiveness of the proposed strategy.

Analysis and recognition of Asian scripts-the state of the art

Abstract: This paper summarizes the research activities of the pastdecade on the recognition of handwritten scripts used inChina, Japan, and Korea. It presents the recognitionmethodologies, features explored, databases used, andclassification schemes investigated. In addition, it includes adescription of the performance of numerous recognitionsystems found in both academic and industrial researchlaboratories. Recent achievements and applications are alsopresented. A list of relevant references is attached togetherwith our remarks on this subject.

Automatic filter selection using image quality assessment

Abstract: We present a method for automatically selecting the best filter to treat poor quality printed documents using image quality assessment. We introduce five quality measures to obtain information about the quality of the images, and morphological filters to improve their quality. A training set of 370 images was used to develop the system. Experimental results on the test set show a significant improvement in the recognition rate from 73.24% using no filter at all to 93.09% after applying a filter that was automatically selected.

Lexicon-driven HMM decoding for large vocabulary handwriting recognition with multiple character models

Abstract: This paper presents a handwriting recognition system that deals with unconstrained handwriting and large vocabularies. The system is based on the segmentation-recognition paradigm where words are first loosely segmented into characters or pseudocharacters and the final segmentation is obtained during the recognition process, which is carried out with a lexicon. Characters are modeled by multiple hidden Markov models (HMMs), which are concatenated to build up word models. The lexicon is organized as a tree structure, and during the decoding words with similar prefixes share the same computation steps. To avoid an explosion of the search space due to the presence of multiple character models, a lexicon-driven level building algorithm (LDLBA) is used to decode the lexical tree and to choose at each level the more likely models. Bigram probabilities related to the variation of writing styles within the words are inserted between the levels of the LDLBA to improve the recognition accuracy. To further speed up the recognition process, some constraints are added to limit the search efforts to the more likely parts of the search space. Experimental results on a dataset of 4674 unconstrained words show that the proposed recognition system achieves recognition rates from 98% for a 10-word vocabulary to 71% for a 30,000-word vocabulary and recognition times from 9 ms to 18.4 s, respectively.

Automatic segmentation and recognition system for handwritten dates on Canadian bank cheques

Abstract: This paper describes a system being developed to recognizedate information handwritten on Canadian bankcheques. A segmentation based strategy is adopted in thissystem. In order to achieve high performances in terms ofefficiency and reliability, a knowledge-based module is proposedfor the date segmentation and a cursive month wordrecognition module is implemented based on a combinationof classifiers. The interaction between the segmentation andrecognition stages is properly established by using multi-hypothesesgeneration and evaluation modules. As a result,promising performance is obtained on a test set from a real-lifestandard cheque database.

Recognition of rotated characters by Eigen-space

Abstract: In this paper, we present a method of recognizinginclined, rotated characters. First we construct an eigensub-space for each category using the covariance matrixwhich is calculated from a sufficient number of rotatedcharacters. Next, we can obtain a locus by projectingtheir rotated characters onto the eigen sub-space andinterpolating between their projected points. An unknowncharacter is also projected onto the eigen sub-space ofeach category. Then, the verification is carried out bycalculating the distance between the projected point ofthe unknown character and the locus. In our experiment,we obtained quite good results for the CENTURY font of26 capital letters of the English alphabet (A, B, .... ,Z).This method has the added advantage of obtaining therecognition result (category) and angle of inclination atthe same time

Impacts of verification on a numeral string recognition system

Abstract: In this paper we discuss the use of high-level verification on handwritten numeral strings. First of all, we introduce the concept of levels of verification and present the baseline system used to carry out the experiments. Two different strategies were developed: absolute and one-to-one verifiers. A thorough error analysis is also presented in order to identify under which conditions high-level verification is more appropriate. Experimental results are presented on NIST SD19 database.

A recognition and verification strategy for handwritten word recognition

Abstract: In this paper a word recognition and verification scheme based on HMMs is presented. However, the main contribution of the current work lies in the validation of such a strategy. In order to perform this task, we carried out some experiments on word recognition using a legal amount database and then we compared the results reached with other study which makes use of the same database. The experiments demonstrate the efficiency of the strategy we developed for word recognition and verification.

Integration of contextual information in handwriting recognition systems

Abstract: This paper investigates different strategies allowing integrationof contextual information during the feature extractionstage of a cursive handwriting HMM-based recognitionsystem. First we propose to use linear discriminant analysis(LDA) in order to integrate the class information duringfeature set building. Secondly several zoning strategies areused to integrate local contextual information. Finally, aweighting technique is proposed in association with zoningwith the aim of integrating handwriting style. Some experimentswere carried out and the results show the interest ofthe proposed strategies.

Speed and accuracy: large-scale machine learning algorithms and their applications

Abstract: Over the past few years, considerable progress has been made in the area of machine learning. However, when these learning machines, including support vector machines (SVMs) and neural networks, are applied to massive sets of high-dimensional data, many challenging problems emerge, such as high computational cost and the way to adapt the structure of a learning system. Therefore, it is important to develop some new methods with computational efficiency and high accuracy such that learning algorithms can be applied more widely to areas such as data ruining, Optical Character Recognition (OCR) and bio-informatics. In this thesis, we mainly focus on three problems: methodologies to adapt the structure of a neural network learning system, speeding up SVM's training and facilitating test on huge data sets. For the first problem, a local learning framework is proposed to automatically construct the ensemble of neural networks, which are trained on local subsets so that the complexity and training time of the learning system can be reduced and its generalization performance can be enhanced. With respect to SVM's training on a very large data set with thousands of classes and high-dimensional input vectors, block diagonal matrices are used to approximate the original kernel matrix such that the original SVM optimization process can be divided into hundreds of sub-problems, which can be solved efficiently. Theoretically, the run-time complexity of the proposed algorithm linearly scales to the size of the data set, the dimension of input vectors and the number of classes. For the last problem, a fast iteration algorithm is proposed to approximate the reduced set vectors simultaneously based on the general kernel type so that the number of vectors in the decision function of each class can be reduced considerably and the testing speed is increased significantly. The main contributions of this thesis are to propose effective solutions to the above three problems. It is especially worth mentioning that the methods which are used to solve the last two problems are crucial in making support vector machines more competitive in tasks where both high accuracy and classification speed are required. The proposed SVM algorithm runs at a much higher training speed than the existing ones such as svm-light and libsvm when applied to a huge data set with thousands of classes. The total training time of SVM with the radial basis function kernel on Hanwang's handwritten Chinese database (2,144,489 training samples, 542,122 testing samples, 3755 classes and 392-dimensional input vectors) is 19 hours on P4. In addition, the proposed testing algorithm has also achieved a promising classification speed, 16,000 patterns per second, on MNIST database. Besides the efficient computation, the state-of-the-art generalization performances have also been achieved on several well-known public and commercial databases. Particularly, very low error rates of 0.38%, 0.5% and 1.0% have been reached on MNIST, Hanwang handwritten digit databases and ETL9B handwritten Chinese database.

A width-invariant property of curves based on wavelet transform with a novel wavelet function

Abstract: This paper is an improvement on the characterization of edges. Using a novel wavelet function, it is proven that the maximum moduli of the wavelet transform (MMWT) of a curve produces two new symmetrical curves on both sides of the original with the same direction. The distance between the two curves is shown to be independent of the width d of the original curve if the scale s of the wavelet transform satisfies s/spl ges/d. This property provides a novel method of obtaining the skeletons of the curves in an image.

Image Recognition Based on Nonlinear Wavelet Approximation

Abstract: This paper presents a novel approach to recognize images based on nonlinear wavelet approximation. The mathematical theory on nonlinear wavelet approximation is introduced, which shows that nonlinear approximation contains much more information of the original image than linear approximation. Based on this theory, a scheme to obtain the basic information of images with less data is provided. Experiments on face recognition produce effective matching rates.

Information Retrieval Based on OCR Errors in Scanned Documents

Abstract: An important proportion of documents are document images, i.e. scanned documents. For their retrieval, it is important to recognize their contents. Current technologies for optical character recognition (OCR) and document analysis do not handle such documents adequately because of the recognition errors. In this paper, we describe an approach that integrates the detection of errors in scanned texts without relying on a lexicon, and this detection is integrated in the research process. The proposed algorithm consists of two basic steps. In the first step, we apply editing operations on OCR words that generate a collection of error-grams and correction rules. The second step uses query terms, error-grams, and correction rules to create searchable keywords, identify appropriate matching terms, and determine the degree of relevance of retrieved document images. Algorithms has been tested on 979 document images provided by Media-team databases from Washington University, and the experimental results obtained show the effectiveness of our method and indicate improvement in comparison with the standard methods such as exact or partial matching, N-gram overlaps, and Q-gram distance.

Palmprints: a novel co-evolutionary algorithm for clustering finger images

Abstract: The purpose of this study is to explore an alternative means of hand image classification, one that requires minimal human intervention. The main tool for accomplishing this is a Genetic Algorithm (GA). This study is more than just another GA application; it introduces (a) a novel cooperative co-evolutionary clustering algorithm with dynamic clustering and feature selection; (b) an extended fitness function, which is particularly suited to an integrated dynamic clustering space. Despite its complexity, the results of this study are clear: the GA evolved an average clustering of 4 clusters, with minimal overlap between them.

Character recognition device, method and program thereof

Abstract: PROBLEM TO BE SOLVED: To provide a character recognizing device which precisely recognizes a rotated character, independently of its rotation angle by applying an inherent space method. SOLUTION: The character recognizing device is provided with a space storage 32 which stores an inherent space generated by a plurality of rotated character images, a locus storage part 33 which stores loci drawn by projection points, obtained by projecting a plurality of the rotated character images to the corresponding inherent space, an input part 1 which inputs the image of a character to be recognized, a distance-computing part 27 which computes the distance between the projection point obtained by projecting the image of the character to be recognized onto the inherent space and each of the loci of a plurality of character kinds, and a candidate-selecting part 28 which selects a candidate for the image of the character to be recognized from among a plurality of the character kinds, based on the distances. COPYRIGHT: (C)2005,JPO&NCIPI