The blind source separation problem is to extract the underlying source signals from a set of linear mixtures, where the mixing matrix is unknown. This situation is common in acoustics, radio, medical signal and image processing, hyperspectral imaging, and other areas. We suggest a two-stage separation process: a priori selection of a possibly overcomplete signal dictionary (for instance, a wavelet frame or a learned dictionary) in which the sources are assumed to be sparsely representable, followed by unmixing the sources by exploiting the their sparse representability. We consider the general case of more sources than mixtures, but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures. Experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques.

/pdf/blind-source-separation-by-sparse-decomposition-in-a-signal-35zj66n7yc.pdf

Blind Source Separation by Sparse Decomposition in a Signal Dictionary

Several MCMC methods have been proposed for estimating probabilities of models and associated ‘model-averaged’ posterior distributions in the presence of model uncertainty. We discuss, compare, develop and illustrate several of these methods, focussing on connections between them.

https://stat.duke.edu/~scs/Courses/Stat376/Papers/TransdimMCMC/Dellaportas2002.pdf

On Bayesian model and variable selection using MCMC

This chapter reviews the recent developments in Markov chain Monte Carlo simulation methods. These methods, which are concerned with the simulation of high dimensional probability distributions, have gained enormous prominence and revolutionized Bayesian statistics. The chapter, provides background on the relevant Markov chain theory and provides detailed information on the theory and practice of Markov chain sampling based on the Metropolis–Hastings and Gibbs sampling algorithms. Convergence diagnostics and strategies for implementation are also discussed. A number of examples drawn from Bayesian statistics are used to illustrate the ideas. The chapter also covers in detail the application of MCMC methods to the problems of prediction and model choice.

Markov chain monte carlo methods: computation and inference

The deviance information criterion (DIC) is widely used for Bayesian model comparison, despite the lack of a clear theoretical foundation. DIC is shown to be an approximation to a penalized loss function based on the deviance, with a penalty derived from a cross-validation argument. This approximation is valid only when the effective number of parameters in the model is much smaller than the number of independent observations. In disease mapping, a typical application of DIC, this assumption does not hold and DIC under-penalizes more complex models. Another deviance-based loss function, derived from the same decision-theoretic framework, is applied to mixture models, which have previously been considered an unsuitable application for DIC.

https://courses.washington.edu/b572/TALKS/2013/JoshKeller-3.pdf

Penalized loss functions for Bayesian model comparison.

A new book in the Econometric Exercises series, this volume contains questions and answers to provide students with useful practice, as they attempt to master Bayesian econometrics. In addition to many theoretical exercises, this book contains exercises designed to develop the computational tools used in modern Bayesian econometrics. The latter half of the book contains exercises that show how these theoretical and computational skills are combined in practice, to carry out Bayesian inference in a wide variety of models commonly used by econometricians. Aimed primarily at advanced undergraduate and graduate students studying econometrics, this book may also be useful for students studying finance, marketing, agricultural economics, business economics or, more generally, any field which uses statistics. The book also comes equipped with a supporting website containing all the relevant data sets and MATLAB computer programs for solving the computational exercises.

/pdf/bayesian-econometric-methods-229aundtrj.pdf

Bayesian Econometric Methods

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture. A sample from the full joint distribution of all unknown variables is thereby generated, and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution. The methodology is applied here to the analysis of univariate normal mixtures, using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context.

R. C. H. Cheng

Papers

On Bayesian analysis of mixtures with an unknown number of components. Discussion. Author's reply