Open AccessJournal Article
Robust Gaussian Process Regression with a Student- t Likelihood
TLDR
This paper illustrates the situations where standard EP fails to converge and review different modifications and alternative algorithms for improving the convergence and demonstrates that convergence problems may occur during the type-II maximum a posteriori (MAP) estimation of the hyperparameters.Abstract:
This paper considers the robust and efficient implementation of Gaussian process regression with a Student-t observation model, which has a non-log-concave likelihood. The challenge with the Student-t model is the analytically intractable inference which is why several approximative methods have been proposed. Expectation propagation (EP) has been found to be a very accurate method in many empirical studies but the convergence of EP is known to be problematic with models containing non-log-concave site functions. In this paper we illustrate the situations where standard EP fails to converge and review different modifications and alternative algorithms for improving the convergence. We demonstrate that convergence problems may occur during the type-II maximum a posteriori (MAP) estimation of the hyperparameters and show that standard EP may not converge in the MAP values with some difficult data sets. We present a robust implementation which relies primarily on parallel EP updates and uses a moment-matching-based double-loop algorithm with adaptively selected step size in difficult cases. The predictive performance of EP is compared with Laplace, variational Bayes, and Markov chain Monte Carlo approximations.read more
Citations
More filters
Journal Article
GPstuff: Bayesian modeling with Gaussian processes
TL;DR: The GPstuff toolbox is a versatile collection of Gaussian process models and computational tools required for Bayesian inference, including various inference methods, sparse approximations and model assessment methods.
Journal ArticleDOI
Review and big data perspectives on robust data mining approaches for industrial process modeling with outliers and missing data
TL;DR: A systematic review of various state-of-the-art data preprocessing tricks as well as robust principal component analysis methods for process understanding and monitoring applications and big data perspectives on potential challenges and opportunities have been highlighted.
Journal ArticleDOI
Non-linear calibration models for near infrared spectroscopy.
TL;DR: This comprehensive comparative study presents a comparison of different nonlinear calibration techniques, suggesting that GPR and BANN are powerful and promising methods for handling linear as well as nonlinear systems, even when the data sets are moderately small.
Journal Article
Gaussian Kullback-Leibler approximate inference
Edward Challis,David Barber +1 more
TL;DR: Numerical results comparing G-KL and other deterministic Gaussian approximate inference methods are presented for: robust Gaussian process regression models with either Student-t or Laplace likelihoods, large scale Bayesian binary logistic regression models, and Bayesian sparse linear models for sequential experimental design.
Posted Content
Expectation propagation as a way of life
Aki Vehtari,Andrew Gelman,Tuomas Sivula,Pasi Jylänki,Dustin Tran,Swupnil Sahai,Paul Blomstedt,John P. Cunningham,David Schiminovich,Christian P. Robert +9 more
TL;DR: EP is revisited as a prototype for scalable algorithms that partition big datasets into many parts and analyze each part in parallel to perform inference of shared parameters to be particularly efficient for hierarchical models.
References
More filters
Book
Pattern Recognition and Machine Learning
TL;DR: Probability Distributions, linear models for Regression, Linear Models for Classification, Neural Networks, Graphical Models, Mixture Models and EM, Sampling Methods, Continuous Latent Variables, Sequential Data are studied.
Book
Bayesian Data Analysis
TL;DR: Detailed notes on Bayesian Computation Basics of Markov Chain Simulation, Regression Models, and Asymptotic Theorems are provided.
Book
Gaussian Processes for Machine Learning
TL;DR: The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification.
Book
Pattern Recognition and Machine Learning (Information Science and Statistics)
TL;DR: Looking for competent reading resources?
Journal ArticleDOI
Multivariate Adaptive Regression Splines
TL;DR: In this article, a new method is presented for flexible regression modeling of high dimensional data, which takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data.