Automatic classification is one of the basic tasks required in any pattern recognition and human computer interaction application. In this paper, we discuss training probabilistic classifiers with labeled and unlabeled data. We provide a new analysis that shows under what conditions unlabeled data can be used in learning to improve classification performance. We also show that, if the conditions are violated, using unlabeled data can be detrimental to classification performance. We discuss the implications of this analysis to a specific type of probabilistic classifiers, Bayesian networks, and propose a new structure learning algorithm that can utilize unlabeled data to improve classification. Finally, we show how the resulting algorithms are successfully employed in two applications related to human-computer interaction and pattern recognition: facial expression recognition and face detection.

/pdf/semisupervised-learning-of-classifiers-theory-algorithms-and-1m191xmbv6.pdf

Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction

The classical problem of learning a classifier relies on a set of labelled examples, without ever questioning the correctness of the provided label assignments. However, there is an increasing realisation that labelling errors are not uncommon in real situations. In this paper we consider a label-noise robust version of the logistic regression and multinomial logistic regression classifiers and develop the following contributions: (i) We derive efficient multiplicative updates to estimate the label flipping probabilities, and we give a proof of convergence for our algorithm. (ii) We develop a novel sparsity-promoting regularisation approach which allows us to tackle challenging high dimensional noisy settings. (iii) Finally, we throughly evaluate the performance of our approach in synthetic experiments and we demonstrate several real applications including gene expression analysis, class topology discovery and learning from crowdsourcing data.

/pdf/label-noise-robust-logistic-regression-and-its-applications-55hly070g3.pdf

Label-Noise robust logistic regression and its applications

(1990). Some recent research in the analysis of mixture distributions. Statistics: Vol. 21, No. 4, pp. 619-641.

Some recent research in the analysis of mixture distributions

Computer-aided imaging systems are now widely used in cytogenetic laboratories to reduce the tedium and labour-intensiveness of traditional methods of chromosome analysis. Automatic chromosome classification is an essential component of such systems, and we review here the statistical techniques that have contributed towards it. Although completely error-free classification has not been, nor is ever likely to be, achieved, error rates have been reduced to levels that are acceptable for many routine purposes. Further reductions are likely to be achieved through advances in basic biology rather than in statistical methodology. Nevertheless, the subject remains of interest to those involved in statistical classification, because of its intrinsic challenges and because of the large body of existing results with which to compare new approaches. Also, the existence of very large databases of correctly-classified chromosomes provides a valuable resource for empirical investigations of the statistical properties of classifiers.

Computer-aided classification of human chromosomes: a review

Foreword. Preface 1. INTRODUCTION. 1 Research Issues on Learning in Computer Vision. 2 Overview of the Book. 3 Contributions. 2. THEORY: PROBABILISTIC CLASSIFIERS. 1 Introduction. 2 Preliminaries and Notations. 3 Bayes Optimal Error and Entropy. 4 Analysis of Classification Error of Estimated (Mismatched)Distribution. 5 Density of Distributions. 6 Complex Probabilistic Models and Small Sample Effects. 7 Summary. 3. THEORY: GENERALIZATION BOUNDS. 1 Introduction. 2 Preliminaries. 3 A Margin Distribution Based Bound. 4 Analysis. 5 Summary. 4. THEORY: SEMI-SUPERVISED LEARNING. 1 Introduction.2 Properties of Classification. 3 Existing Literature. 4 Semi-supervised Learning Using Maximum Likelihood Estimation. 5 Asymptotic Properties of Maximum Likelihood Estimation with Labeled and Unlabeled Data. 6 Learning with Finite Data. 7 Concluding Remarks. 5. ALGORITHM: MAXIMUM LIKELIHOOD MINIMUM ENTROPY HMM. 1 Previous Work. 2 Mutual Information, Bayes Optimal Error, Entropy, and Conditional Probability. 3 Maximum Mutual Information HMMs. 4 Discussion. 5 Experimental Results. 6 Summary. 6. ALGORITHM: MARGIN DISTRIBUTION OPTIMIZATION. 1 Introduction. 2 A Margin Distribution Based Bound. 3 Existing Learning Algorithms. 4 The Margin Distribution Optimization (MDO) Algorithm. 5 Experimental Evaluation. 6 Conclusions. 7. ALGORITHM: LEARNING THE STRUCTURE OF BAYESIAN NETWORK CLASSIFIERS. 1 Introduction. 2 Bayesian Network Classifiers. 3 Switching between Models: Naive Bayes and TAN Classifiers. 4 Learning the Structure of Bayesian Network Classifiers: Existing Approaches. 5 Classification Driven Stochastic Structure Search. 6 Experiments. 7 Should Unlabeled Data Be Weighed Differently? 8 Active Learning. 9 Concluding Remarks. 8. APPLICATION: OFFICE ACTIVITY RECOGNITION. 1 Context-Sensitive Systems. 2 Towards Tractable and Robust Context Sensing. 3 Layered Hidden Markov Models (LHMMs). 4 Implementation of SEER. 5 Experiments. 6 Related Representations. 7Summary. 9. APPLICATION: MULTIMODAL EVENT DETECTION. 1 Fusion Models: A Review. 2 A Hierarchical Fusion Model. 3 Experimental Setup, Features, and Results. 4 Summary. 10. APPLICATION: FACIAL EXPRESSION RECOGNITION. 1 Introduction. 2 Human Emotion Research. 3 Facial Expression Recognition System. 4 Experimental Analysis. 5 Discussion. 11. APPLICATION: BAYESIAN NETWORK CLASSIFIERS FOR FACE DETECTION. 1 Introduction. 2 Related Work. 3 Applying Bayesian Network Classifiers to Face Detection. 4 Experiments. 5 Discussion. References. Index.

T. Krishnan

Papers

Efficiency of discriminant analysis when initial samples are classified stochastically

Efficiency of learning with imperfect supervision

Discriminant analysis with a stochastic supervisor

Pattern recognition with an imperfect supervisor

Efficiency of logistic-normal stochastic supervision