Showing papers on "MNIST database published in 2002"

Training Invariant Support Vector Machines

Abstract: Practical experience has shown that in order to obtain the best possible performance, prior knowledge about invariances of a classification problem at hand ought to be incorporated into the training procedure. We describe and review all known methods for doing so in support vector machines, provide experimental results, and discuss their respective merits. One of the significant new results reported in this work is our recent achievement of the lowest reported test error on the well-known MNIST digit recognition benchmark task, with SVM training times that are also significantly faster than previous SVM methods.

Handwritten digit recognition using state-of-the-art techniques

Abstract: This paper presents the latest results of handwritten digit recognition on well-known image databases using the state-of-the-art feature extraction and classification techniques. The tested databases are CENPARMI, CEDAR, and MNIST. On the test dataset of each database, 56 recognition accuracies are given by combining 7 classifiers with 8 feature vectors. All the classifiers and feature vectors give high accuracies. Among the features, the chain-code feature and gradient feature show advantages, and the profile structure feature shows efficiency as a complementary feature. In comparison of classifiers, the support vector classifier with RBF kernel gives the highest accuracy but is extremely expensive in storage and computation. Among the non-SV classifiers, the polynomial classifier performs best, followed by a learning quadratic discriminant function classifier. The results are competitive compared to previous ones and they provide a baseline for evaluation of future works.

A Fast SVM Training Algorithm

Abstract: A fast support vector machine (SVM) training algorithm is proposed under the decomposition framework of SVM's algorithm by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Extensive experiments on MNIST handwritten digit database have been conducted to show that the proposed algorithm is much faster than Keerthi et al.'s improved SMO, about 9 times. Combined with principal component analysis, the total training for ten one against the-rest classifiers on MNIST took just 0.77 hours. The promising scalability of the proposed scheme can make it possible to apply SVM to a wide variety of problems in engineering.

Constructing a large node Chow-Liu tree based on frequent itemsets

Abstract: We present a novel approach to construct a kind of tree belief network, in which the "nodes" are subsets of variables of dataset We call this large node Chow-Liu tree (LNCLT) Similar to the Chow-Liu tree (1968), the LNCLT is also ideal for density estimation and classification applications This technique uses the concept of "frequent itemsets" as found in the database literature to guide the construction of the LNCLT Our LNCLT has a simpler structure while it maintains a good fitness over the dataset We detail the theoretical formulation of our approach Moreover, based on the MNIST hand-printed digit database, we conduct a series of digit recognition experiments to verify our approach From the result we find that both recognition rate and density estimation accuracy are improved with the LNCLT structure

Multi-modular architecture based on convolutional neural networks for online handwritten character recognition

Abstract: In this paper, several convolutional neural network architectures are investigated for online isolated handwritten character recognition (Latin alphabet). Two main architectures have been developed and optimised. The first one, a TDNN, processes online features extracted from the character. The second one, a SDNN, relies on the off-line bitmaps reconstructed from the trajectory of the pen. Moreover, an hybrid architecture called SDTDNN has been derived, it allows the combination of on-line and off-line recognisers. Such a combination seems to be very promising to enhance the character recognition rate. This type of shared weights neural networks introduces the notion of receptive field, local extraction and it allows to restrain the number of free parameters in opposition to classic techniques such as multi-layer perceptron. Results on UNIPEN and IRONOFF databases for online recognition are reported, while the MNIST database has been used for the off-line classifier.

Multi-Class SVM Classifier Based on Pairwise Coupling

Abstract: In this paper, a novel structure is proposed to extend standard support vector classifier to multi-class cases. For a K-class classification task, an array of K optimal pairwise coupling classifier (O-PWC) is constructed, each of which is the most reliable and optimal for the corresponding class in the sense of cross entropy or square error. The final decision will be produced through combining the results of these K O-PWCs. The accuracy rate is improved while the computational cost will not increase too much. Our approach is applied to two applications: handwritten digital recognition on MNIST database and face recognition on Cambridge ORL face database, experimental results reveal that our method is effective and efficient.

Incremental Gaussian Processes

Abstract: In this paper, we consider Tipping's relevance vector machine (RVM) [1] and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call Subspace EM (SSEM). Working with a subset of active basis functions, the sparsity of the RVM solution will ensure that the number of basis functions and thereby the computational complexity is kept low. We also introduce a mean field approach to the intractable classification model that is expected to give a very good approximation to exact Bayesian inference and contains the Laplace approximation as a special case. We test the algorithms on two large data sets with O(103 - 104) examples. The results indicate that Bayesian learning of large data sets, e.g. the MNIST database is realistic.

The MNIST Database of Handwritten upper-case letters

Abstract: The MNIST database of Handwritten upper-case letters is a subset derived from NIST Special Database 19 Handprinted Forms and Characters Database, and developed at IDIAP. It comprises the image files of handwritten upper-case letters, which have been size-normalized and centered in a fixed-size image. Generally the image files were written in SVM-Torch format, it's convenient for the Torch users, it's also easy to transfer it into other format. There is software attached with INIST, which used to get visiable image files.