Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining Practical Machine Learning Tools and Techniques

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate seriously. To improve the robustness of KF against impulsive noises, we propose in this work a new Kalman filter, called the maximum correntropy Kalman filter (MCKF), which adopts the robust maximum correntropy criterion (MCC) as the optimality criterion, instead of using the MMSE. Similar to the traditional KF, the state mean and covariance matrix propagation equations are used to give prior estimations of the state and covariance matrix in MCKF. A novel fixed-point algorithm is then used to update the posterior estimations. A sufficient condition that guarantees the convergence of the fixed-point algorithm is given. Illustration examples are presented to demonstrate the effectiveness and robustness of the new algorithm.

Maximum Correntropy Kalman Filter

Missing value imputation (MVI) has been studied for several decades being the basic solution method for incomplete dataset problems, specifically those where some data samples contain one or more missing attribute values. This paper aims at reviewing and analyzing related studies carried out in recent decades, from the experimental design perspective. Altogether, 111 journal papers published from 2006 to 2017 are reviewed and analyzed. In addition, several technical issues encountered during the MVI process are addressed, such as the choice of datasets, missing rates and missingness mechanisms, and the MVI techniques and evaluation metrics employed, are discussed. The results of analysis of these issues allow limitations in the existing body of literature to be identified based upon which some directions for future research can be gleaned.

Missing value imputation: a review and analysis of the literature (2006–2017)

Multi-kernel extreme learning machine based method is proposed for EEG classification.Supplementary information from different kernels are integrated for better accuracy.Extensive experimental comparison confirms superiority of the proposed method. One of the most important issues for the development of a motor-imagery based brain-computer interface (BCI) is how to design a powerful classifier with strong generalization capability. Extreme learning machine (ELM) has recently proven to be comparable or more efficient than support vector machine for many pattern recognition problems. In this study, we propose a multi-kernel ELM (MKELM)-based method for motor imagery electroencephalogram (EEG) classification. The kernel extension of ELM provides an elegant way to circumvent calculation of the hidden layer outputs and inherently encode it in a kernel matrix. We investigate effects of two different kernel functions (i.e., Gaussian kernel and polynomial kernel) on the performance of kernel ELM. The MKELM method is subsequently developed by integrating these two types of kernels with a multi-kernel learning strategy, which can effectively explore the supplementary information from multiple nonlinear feature spaces for more robust classification of EEG. An extensive experimental comparison with two public EEG datasets indicates that the MKELM method gives higher classification accuracy than those of the other competing algorithms. The experimental results confirm that superiority of the proposed MKELM-based method for accurate classification of EEG associated with motor imagery in BCI applications. Our method also provides a promising and generalized solution to investigate the complex and nonlinear information for various applications in the fields of expert and intelligent systems.

Multi-kernel extreme learning machine for EEG classification in brain-computer interfaces

Recursive projection twin support vector machine via within-class variance minimization

Recursive robust least squares support vector regression based on maximum correntropy criterion

Twin support vector regression (TSVR) was proposed recently as a novel regressor that tries to find a pair of nonparallel planes, i.e., e-insensitive up- and down-bounds, by solving two related SVM-type problems. However, it may incur suboptimal solution since its objective function is positive semi-definite and the lack of complexity control. In order to address this shortcoming, we develop a novel SVR algorithm termed as smooth twin SVR (STSVR). The idea is to reformulate TSVR as a strongly convex problem, which results in unique global optimal solution for each subproblem. To solve the proposed optimization problem, we first adopt a smoothing technique to convert the original constrained quadratic programming problems into unconstrained minimization problems, and then use the well-known Newton–Armijo algorithm to solve the smooth TSVR. The effectiveness of the proposed method is demonstrated via experiments on synthetic and real-world benchmark datasets.

Smooth twin support vector regression

This paper presents a framework of accident prediction with a new perspective. First, the new framework of Chain of Road Traffic Incident (CRTI) is proposed, in which the observed vehicle movement features are viewed as road traffic system’s external “performance” that, in essence, reflect the internal “health states” (safety states) of the system at a specific time. A two-stage modeling procedure of CRTI is then proposed using scenario-based strategy: 1) a support vector machine is utilized to classify leaving lane scene versus remaining in lane scene and 2) Gaussian-mixture-based hidden Markov models are developed to recognize accident versus non-accident pattern CRTI given the classified scene. Moreover, the application procedure of the CRTI framework to online collision prediction is proposed. Finally, a simulation test of a typical vehicle collision scene based on PreScan platform is designed and carried out for model training and validation, and has shown promising results in accident prediction using the proposed framework. The CRTI framework could provide a new foundation for developing early warning/intervention strategies in driver assistance system under complex traffic environments.

A New Framework of Vehicle Collision Prediction by Combining SVM and HMM

Low-rank matrix completion (LRMC) is a recently emerging technique which has achieved promising performance in many real-world applications, such as traffic data imputation. In order to estimate missing values, the current LRMC based methods optimize the rank of the matrix comprising the whole traffic data, potentially assuming that all traffic data is equally important. As a result, it puts more emphasis on the commonality of traffic data while ignoring its subtle but crucial difference due to different locations of loop detectors as well as dates of sampling. To handle this problem and further improve imputation performance, a novel correlation-based LRMC method is proposed in this paper. Firstly, LRMC is applied to get initial estimations of missing values. Then, a distance matrix containing pairwise distance between samples is built based on a weighted Pearson's correlation which strikes a balance between observed values and imputed values. For a specific sample, its most similar samples based on the distance matrix constructed are chosen by using an adaptive K-nearest neighboring (KNN) search. LRMC is then applied on these samples with much stronger correlation to obtain refined estimations of missing values. Finally, we also propose a simple but effective ensemble learning strategy to integrate multiple imputed values for a specific sample for further improving imputation performance. Extensive numerical experiments are performed on both traffic flow volume data as well as standard benchmark datasets. The results confirm that the proposed correlation-based LRMC and its ensemble learning version achieve better imputation performance than competing methods.

Jun Liang

Papers

Recursive projection twin support vector machine via within-class variance minimization

Recursive robust least squares support vector regression based on maximum correntropy criterion

Smooth twin support vector regression

A New Framework of Vehicle Collision Prediction by Combining SVM and HMM

Ensemble correlation-based low-rank matrix completion with applications to traffic data imputation