Showing papers on "MNIST database published in 1999"

Fast training of support vector machines using sequential minimal optimization

Abstract: This chapter describes a new algorithm for training Support Vector Machines: Sequential Minimal Optimization, or SMO Training a Support Vector Machine (SVM) requires the solution of a very large quadratic programming (QP) optimization problem SMO breaks this large QP problem into a series of smallest possible QP problems These small QP problems are solved analytically, which avoids using a time-consuming numerical QP optimization as an inner loop The amount of memory required for SMO is linear in the training set size, which allows SMO to handle very large training sets Because large matrix computation is avoided, SMO scales somewhere between linear and quadratic in the training set size for various test problems, while a standard projected conjugate gradient (PCG) chunking algorithm scales somewhere between linear and cubic in the training set size SMO's computation time is dominated by SVM evaluation, hence SMO is fastest for linear SVMs and sparse data sets For the MNIST database, SMO is as fast as PCG chunking; while for the UCI Adult database and linear SVMs, SMO can be more than 1000 times faster than the PCG chunking algorithm

Recognition and verification of unconstrained handwritten numerals

Abstract: Despite the success of many recognition systems for handwritten numerals within constrained domains, the problem remains difficult when unconstrained inputs are involved. The gap between the state-of-the-art machine recognition reliability and high practical demand leads to this investigation of verification scheme in pattern recognition. A pattern verifier is an expert specially trained to reliably confirm or negate a pattern identity from the General Purpose Recognizer (GPR), with the intention to significantly improve the class-specific Precision Rates of the system. The main goal of this thesis is to study the promising and critical role of a verifier in a recognition system. Theoretical aspects of a verifier including its unique task and functionality, inherent requirement, evaluation measurement, design concern and control strategy are discussed throughout the thesis, focusing on the problems of recognizing Unconstrained Isolated Handwritten Numerals (UIHN) and Unconstrained Touching Handwritten Numerals (UTHN). For each problem, an integrated recognition and verification system is designed and evaluated by incorporating together the GPR and the verifier. The GPR for UIHN is a combination of three conventional neural approaches. In the design of class-specific verifier for UIHN, a new kind of neural network--Quantum Neural Network (QNN)--with better distinguishing ability along decision boundary, is embedded in an efficient way. Novel experiments have been designed for in-depth studies of applying the QNN to both real data and confusing images synthesized by morphing. CENPARMI database and MNIST database are used for evaluation. UTHN recognition is an important component for automatic document processing in applications such as cheque processing. However, it is a more difficult problem that has attained less attention, reflected by the mediocre performance of current systems and lack of benchmarking databases. Two databases IRIS-Bell'98 and NIST for UTHN are newly built by the researchers at CENPARMI and the author. They are used in this research and are intended to serve as standard databases in this field. A novel graph-based combination of segmentation and recognition schemes is used in GPR for UTHN. Effective domain specific strategies making use of touching type, touching location and structural information are applied in the verifier for UTHN. The recognition and verification system for UIHN achieved a precision rate of 99.1% on MNIST database while the one for UTHN reached a precision rate of 96.1% on NIST database. The two systems are also evaluated by hypothesis testing. The substantial improvement of system precision rates by verification scheme proves the effectiveness of the proposed systems and justifies the important role of verifiers in the OCR system.