Showing papers on "Latent Dirichlet allocation published in 2000"

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Abstract: This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.

Computational Aspects of Nonparametric Bayesian Analysis with Applications to the Modeling of Multiple Binary Sequences

Abstract: We consider Markov mixture models for multiple longitudinal binary sequences. Prior uncertainty in the mixing distribution is characterized by a Dirichlet process centered on a matrix beta measure. We use this setting to evaluate and compare the performance of three competing algorithms that arise more generally in Dirichlet process mixture calculations: sequential imputations, Gibbs sampling, and a predictive recursion, for which an extension of the sequential calculations is introduced. This facilitates the estimation of quantities related to clustering structure which is not available in the original formulation. A numerical comparison is carried out in three examples. Our findings suggest that the sequential imputations method is most useful for relatively small problems, and that the predictive recursion can be an efficient preliminary tool for more reliable, but computationally intensive, Gibbs sampling implementations.

Bayesian analysis for a generalised Dirichlet process prior

Abstract: A family of random probabilities is defined and studied. This family contains the Dirichlet process as a special case, corresponding to an inner point in the appropriate parameter space. The extension makes it possible to have.random means with larger or smaller skewnesses as compared to skewnesses under the Dirichlet prior, and also in other ways amounts to additional modelling flexibility. The usefulness of such random probabilities for use in nonparametric Bayesian statistics is discussed. The posterior distribution is complicated, but inference can nevertheless be carried out via simulation, and some exact formulae are derived for the case of random means. The class of nonparametric priors provides an instructive example where the speed with which the posterior forgets its prior with increasing data sample size depends on special aspects of the prior, which is a different situation from that of parametric inference.

Dirichlet Generalizations of Latent-Class Models

Abstract: With a latent-class model, each individual belongs to a single latent class, which determines the person's set of response probabilities for the observed, or manifest, variables. A more general model, proposed herein, adds a single parameter and involves drawing, separately and independently for each individual, a latent set of mixing weights from a Dirichlet distribution whose dispersion is governed by the added parameter. The person's set of response probabilities then consists of weighted averages of the probabilities for the classes, where the weights are the person's Dirichlet values. The posterior probabilities commonly used under latent-class models generalize under Dirichlet models to posterior expectations, which serve much the same function. We give examples of formulations of the Dirichlet model, along with numerical illustrations using published data. The first two model formulations involve Guttman scaling and panel analysis and effectively have no latent-class models that compete.

A Bayesian Analysis of Structural Changes with an Application to the Displacement Effect

Abstract: In this paper we propose a new approach to the problem of structural change from a Bayesian point of view. Our approach is based on a hierarchical model with a Dirichlet process prior. A notable feature of the Dirichlet process is its discreteness which is useful for detecting structural changes. The approach developed in the paper is illustrated using simulated and real data sets. For the real data set, we examine possible structural changes in government expenditure in Japan using the annual data from 1957 to 1995. It is shown that two structural breaks occurred, before and after the first oil crisis.