Sparse bayesian learning and the relevance vector machine
Reads0
Chats0
TLDR
It is demonstrated that by exploiting a probabilistic Bayesian learning framework, the 'relevance vector machine' (RVM) can derive accurate prediction models which typically utilise dramatically fewer basis functions than a comparable SVM while offering a number of additional advantages.Abstract:
This paper introduces a general Bayesian framework for obtaining sparse solutions to regression and classification tasks utilising models linear in the parameters Although this framework is fully general, we illustrate our approach with a particular specialisation that we denote the 'relevance vector machine' (RVM), a model of identical functional form to the popular and state-of-the-art 'support vector machine' (SVM) We demonstrate that by exploiting a probabilistic Bayesian learning framework, we can derive accurate prediction models which typically utilise dramatically fewer basis functions than a comparable SVM while offering a number of additional advantages These include the benefits of probabilistic predictions, automatic estimation of 'nuisance' parameters, and the facility to utilise arbitrary basis functions (eg non-'Mercer' kernels) We detail the Bayesian framework and associated learning algorithm for the RVM, and give some illustrative examples of its application along with some comparative benchmarks We offer some explanation for the exceptional degree of sparsity obtained, and discuss and demonstrate some of the advantageous features, and potential extensions, of Bayesian relevance learningread more
Citations
More filters
Journal ArticleDOI
Sparse Coding From a Bayesian Perspective
TL;DR: This paper interprets sparse coding from a novel Bayesian perspective, which results in a new objective function through maximum a posteriori estimation, which can generate more stable results than the obtained solution of the objective function and smaller reconstruction errors than the l1 penalty.
Journal ArticleDOI
Paradigm free mapping with sparse regression automatically detects single-trial functional magnetic resonance imaging blood oxygenation level dependent responses.
Cesar Caballero Gaudes,Cesar Caballero Gaudes,Natalia Petridou,Natalia Petridou,Susan T. Francis,Ian L. Dryden,Penny A. Gowland +6 more
TL;DR: A novel PFM approach is developed using the Dantzig selector estimator, solved via an efficient homotopy procedure, along with statistical model selection criteria, which achieves high detection rates of the BOLD responses, comparable with a model‐based analysis, but requiring no information on the timing of the events and being robust against hemodynamic response function variability.
Journal ArticleDOI
An Improved Auto-Calibration Algorithm Based on Sparse Bayesian Learning Framework
TL;DR: An auto-calibration sparse Bayesian learning algorithm is proposed, in which signal and perturbation are iteratively estimated to achieve sparsity by leveraging a variational Bayesian expectation maximization technique.
Journal ArticleDOI
Computationally Efficient Sparse Bayesian Learning via Belief Propagation
TL;DR: It is proved that the messages in BP are Gaussian probability density functions and therefore, they only need to update their means and variances when the authors update the messages.
Journal ArticleDOI
Robust One-Bit Bayesian Compressed Sensing with Sign-Flip Errors
TL;DR: A robust Bayesian compressed sensing framework is introduced to account for sign flip errors and a variational expectation-maximization algorithm is developed to identify the sign-flip errors and recover the sparse signal simultaneously.
References
More filters
Book
The Nature of Statistical Learning Theory
TL;DR: Setting of the learning problem consistency of learning processes bounds on the rate of convergence ofLearning processes controlling the generalization ability of learning process constructing learning algorithms what is important in learning theory?
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Journal ArticleDOI
Pattern Classification and Scene Analysis.
Book
Pattern classification and scene analysis
Richard O. Duda,Peter E. Hart +1 more
TL;DR: In this article, a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition is provided, including Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.