Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Currently, many applications have emerged from the implementation of software development and hardware use, known as the Internet of things. One of the most important application areas of this type of technology is in health care. Various applications arise daily in order to improve the quality of life and to promote an improvement in the treatments of patients at home that suffer from different pathologies. That is why there has emerged a line of work of great interest, focused on the study and analysis of daily life activities, on the use of different data analysis techniques to identify and to help manage this type of patient. This article shows the result of the systematic review of the literature on the use of the Clustering method, which is one of the most used techniques in the analysis of unsupervised data applied to activities of daily living, as well as the description of variables of high importance as a year of publication, type of article, most used algorithms, types of dataset used, and metrics implemented. These data will allow the reader to locate the recent results of the application of this technique to a particular area of knowledge.

https://www.mdpi.com/1424-8220/20/9/2702/pdf

Unsupervised Human Activity Recognition Using the Clustering Approach: A Review

We study the problem of selecting the best parameter values to use for a support vector machine (SVM) with RBF kernel. Our methods extend the well-known formulas for AIC and BIC, and we present two alternative approaches for calculating the necessary likelihood functions for these formulas. Our rst approach is based on using the distances of support vectors from the separating hyperplane. Our second approach estimates the probability that the SVM hyperplane coincides with the Bayes classier, by analysing the disposition of points in the kernel feature space. We experimentally compare our two approaches with several existing methods and show they are able to achieve good accuracy, whilst also having low running time.

/pdf/aic-and-bic-based-approaches-for-svm-parameter-value-3fnu8bkq24.pdf

AIC and BIC based approaches for SVM parameter value estimation with RBF kernels.

We propose a novel practical method for finding the optimal classifier parameter status corresponding to the Bayes error (minimum classification error probability) through the evaluation of estimated class boundaries from the perspective of Bayes boundary-ness. While traditional methods approach classifier optimality from the angle of minimization of the estimated classification error probabilities, we approach it from the angle of optimality of the estimated classification boundaries. The optimal classification boundary consists solely of uncertain samples, whose posterior probability is equal for the two classes separated by the boundary. We refer to this essential characteristic of the boundary as “Bayes boundary-ness”, and use it to measure how optimal the estimated boundary is. Our proposed method achieves the optimal parameter status using the training data only once, in contrast to such traditional methods as Cross-Validation (CV), which demand separate validation data and often require a number of repetitions of training and validation. Moreover, it can be directly applied to any type of classifier, and potentially to any type of sample. In this paper, we first elaborate on our proposed method that implements the Bayes boundary-ness with an entropy-based uncertainty measure. Next, we analyze the mathematical characteristics of the uncertainty measure adopted. Finally, we evaluate the method through a systematic experimental comparison with CV-based Bayes boundary estimation, which is known to be highly reliable in the Bayes error estimation. From the analysis, we rigorously show the theoretical validity of our adopted uncertainty measure. Moreover, from the experiment, we successfully demonstrate that our method can closely approximate the CV-based Bayes boundary estimate and its corresponding classifier parameter status with only a single-shot training over the data in hand.

/pdf/a-practical-method-based-on-bayes-boundary-ness-for-optimal-5bj8oacklr.pdf

A Practical Method Based on Bayes Boundary-Ness for Optimal Classifier Parameter Status Selection

Recently, we proposed a new method to select an optimal classifier parameter status (value) using our new criterion that is referred to as uncertainty measure and directly evaluates the Bayes boundary-ness of estimated boundaries. The utility of the method was shown in a task of selecting an optimal Gaussian kernel width, which closely approximates the linear Bayes boundary in the feature space produced by Support Vector Machine classifier. In this paper, we apply the method to two types of classifiers whose class boundaries are basically nonlinear: Multi-ProtoType (MPT) classifier and Multi-Layer Perceptron (MLP) classifier. From experiments using a synthetic dataset and four real-life datasets, we show that our method can provide an optimal (in size and value) classifier parameter status, which basically corresponds to the nonlinear Bayes boundary in given feature spaces, for MPT and MLP classifiers.

Optimal Classifier Parameter Status Selection Based on Bayes Boundary-ness for Multi-ProtoType and Multi-Layer Perceptron Classifiers

We propose a method to select the optimal parameter status for any classifier model. In the statistical pattern recognition framework, optimal classification is defined as achieving the minimum classification error probability (Bayes error). Although the error probability is defined on infinite data, in practice only a finite amount of data is available. Using the same finite data for classifier training and evaluation provides a serious underestimate of the Bayes error. Traditional solutions consist in holding out some of the available data for evaluation, which unavoidably decreases the data available for either training or evaluation. By contrast, our proposed method uses the same data for training and evaluation in a single training without splitting, which is made possible by evaluating the ideality of the classifier’s classification boundary instead of estimating the error probability. Here, ideal classification boundary (Bayes boundary) refers to the boundary that leads to the Bayes error. We use the fact that the Bayes boundary solely consists of uncertain samples, namely samples whose class posterior probability is equal for the two classes separated by the boundary. Tests on several real-life datasets and experimental comparison to Cross-Validation clearly show the potential of our method.

Optimal classifier model status selection using bayes boundary uncertainty

The ultimate goal of pattern classifier parameter training is to achieve its optimal status (value) that produces Bayes error or a corresponding Bayes boundary. To realize this goal without unrealistically long training repetitions and strict parameter assumptions, the Bayes Boundary-ness-based Selection (BBS) method was recently proposed and its effectiveness was clearly demonstrated. However, the BBS method remains cumbersome because it consists of two stages: the first generates many candidate sets of trained parameters by carefully controlling the training hyperparameters so that those candidate sets can include the optimal target parameter set; the second stage selects an optimal set from candidate sets. To resolve the BBS method's burden, we propose a new one-stage training method that directly optimizes a given classifier parameter set by maximizing its Bayes boundary-ness or increasing its accuracy during Bayes error estimation. We experimentally evaluate our proposed method in terms of its accuracy of Bayes error estimation over four synthetic or real-life datasets. Our experimental results clearly show that it successfully overcomes the drawbacks of the preceding BBS method and directly creates optimal classifier parameter status without generating too many candidate parameter sets.

Maximum Bayes Boundary-Ness Training For Pattern Classification

To increase the utility of Regularization, which is a common framework for avoiding the underestimation of ideal Bayes error, for speech recognizer training, we propose a new classifier training concept that incorporates a regularization term that represents the speech synthesis ability of classifier parameters. To implement our new concept, we first introduce a speech recognizer that embeds Line Spectral Pairs-Conjugate Structure-Algebraic Code Excited Linear Prediction (LSP-CS-ACELP) in a Multi-Prototype State-Transition-Model (MP-STM) classifier, define a regularization term that represents the speech synthesis ability by the distance between a training sample and its nearest MP-STM word model, and formalize a new Minimum Classification Error (MCE) training method for jointly minimizing a conventional smooth classification error count loss and the newly defined regularization term. We evaluated the proposed training method in an isolated-word, closed-vocabulary, and speaker-independent speech recognition task whose Bayes error is estimated to be about 20% and found that our method successfully produced an estimate of Bayes error (about 18.4%) with a single training run over a training dataset without such data resampling as Cross-Validation or the assumptions of sample distribution. Moreover, we investigated the quality of the synthesized speech using LSP parameters derived from the trained prototypes and found that the quality of the Bayes error estimation is clearly supported by the speech synthesis ability preserved in the training.

Miho Ohsaki

Papers

A Practical Method Based on Bayes Boundary-Ness for Optimal Classifier Parameter Status Selection

Optimal classifier model status selection using bayes boundary uncertainty

Optimal Classifier Parameter Status Selection Based on Bayes Boundary-ness for Multi-ProtoType and Multi-Layer Perceptron Classifiers

Maximum Bayes Boundary-Ness Training For Pattern Classification

Minimum Classification Error Training with Speech Synthesis-Based Regularization for Speech Recognition