The Bayesian approach to statistical problems, though fruitful in many ways, has been rather unsuccessful in treating nonparametric problems. This is due primarily to the difficulty in finding workable prior distributions on the parameter space, which in nonparametric ploblems is taken to be a set of probability distributions on a given sample space. There are two desirable properties of a prior distribution for nonparametric problems. (I) The support of the prior distribution should be large--with respect to some suitable topology on the space of probability distributions on the sample space. (II) Posterior distributions given a sample of observations from the true probability distribution should be manageable analytically. These properties are antagonistic in the sense that one may be obtained at the expense of the other. This paper presents a class of prior distributions, called Dirichlet process priors, broad in the sense of (I), for which (II) is realized, and for which treatment of many nonparametric statistical problems may be carried out, yielding results that are comparable to the classical theory. In Section 2, we review the properties of the Dirichlet distribution needed for the description of the Dirichlet process given in Section 3. Briefly, this process may be described as follows. Let $\mathscr{X}$ be a space and $\mathscr{A}$ a $\sigma$-field of subsets, and let $\alpha$ be a finite non-null measure on $(\mathscr{X}, \mathscr{A})$. Then a stochastic process $P$ indexed by elements $A$ of $\mathscr{A}$, is said to be a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$ if for any measurable partition $(A_1, \cdots, A_k)$ of $\mathscr{X}$, the random vector $(P(A_1), \cdots, P(A_k))$ has a Dirichlet distribution with parameter $(\alpha(A_1), \cdots, \alpha(A_k)). P$ may be considered a random probability measure on $(\mathscr{X}, \mathscr{A})$, The main theorem states that if $P$ is a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$, and if $X_1, \cdots, X_n$ is a sample from $P$, then the posterior distribution of $P$ given $X_1, \cdots, X_n$ is also a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with a parameter $\alpha + \sum^n_1 \delta_{x_i}$, where $\delta_x$ denotes the measure giving mass one to the point $x$. In Section 4, an alternative definition of the Dirichlet process is given. This definition exhibits a version of the Dirichlet process that gives probability one to the set of discrete probability measures on $(\mathscr{X}, \mathscr{A})$. This is in contrast to Dubins and Freedman [2], whose methods for choosing a distribution function on the interval [0, 1] lead with probability one to singular continuous distributions. Methods of choosing a distribution function on [0, 1] that with probability one is absolutely continuous have been described by Kraft [7]. The general method of choosing a distribution function on [0, 1], described in Section 2 of Kraft and van Eeden [10], can of course be used to define the Dirichlet process on [0, 1]. Special mention must be made of the papers of Freedman and Fabius. Freedman [5] defines a notion of tailfree for a distribution on the set of all probability measures on a countable space $\mathscr{X}$. For a tailfree prior, posterior distribution given a sample from the true probability measure may be fairly easily computed. Fabius [3] extends the notion of tailfree to the case where $\mathscr{X}$ is the unit interval [0, 1], but it is clear his extension may be made to cover quite general $\mathscr{X}$. With such an extension, the Dirichlet process would be a special case of a tailfree distribution for which the posterior distribution has a particularly simple form. There are disadvantages to the fact that $P$ chosen by a Dirichlet process is discrete with probability one. These appear mainly because in sampling from a $P$ chosen by a Dirichlet process, we expect eventually to see one observation exactly equal to another. For example, consider the goodness-of-fit problem of testing the hypothesis $H_0$ that a distribution on the interval [0, 1] is uniform. If on the alternative hypothesis we place a Dirichlet process prior with parameter $\alpha$ itself a uniform measure on [0, 1], and if we are given a sample of size $n \geqq 2$, the only nontrivial nonrandomized Bayes rule is to reject $H_0$ if and only if two or more of the observations are exactly equal. This is really a test of the hypothesis that a distribution is continuous against the hypothesis that it is discrete. Thus, there is still a need for a prior that chooses a continuous distribution with probability one and yet satisfies properties (I) and (II). Some applications in which the possible doubling up of the values of the observations plays no essential role are presented in Section 5. These include the estimation of a distribution function, of a mean, of quantiles, of a variance and of a covariance. A two-sample problem is considered in which the Mann-Whitney statistic, equivalent to the rank-sum statistic, appears naturally. A decision theoretic upper tolerance limit for a quantile is also treated. Finally, a hypothesis testing problem concerning a quantile is shown to yield the sign test. In each of these problems, useful ways of combining prior information with the statistical observations appear. Other applications exist. In his Ph. D. dissertation [1], Charles Antoniak finds a need to consider mixtures of Dirichlet processes. He treats several problems, including the estimation of a mixing distribution, bio-assay, empirical Bayes problems, and discrimination problems.

/pdf/a-bayesian-analysis-of-some-nonparametric-problems-siibmg5pnf.pdf

A Bayesian Analysis of Some Nonparametric Problems

(2002). Modelling Extremal Events for Insurance and Finance. Journal of the American Statistical Association: Vol. 97, No. 457, pp. 360-360.

Modelling Extremal Events for Insurance and Finance

Non-Gaussian processes of Ornstein–Uhlenbeck (OU) type offer the possibility of capturing important distributional deviations from Gaussianity and for flexible modelling of dependence structures. This paper develops this potential, drawing on and extending powerful results from probability theory for applications in statistical analysis. Their power is illustrated by a sustained application of OU processes within the context of finance and econometrics. We construct continuous time stochastic volatility models for financial assets where the volatility processes are superpositions of positive OU processes, and we study these models in relation to financial data and theory.

Non-Gaussian Ornstein–Uhlenbeck-based models and some of their uses in financial economics

A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson–Dirichlet process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to fit posteriors of Bayesian hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Polya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies t...

/pdf/gibbs-sampling-methods-for-stick-breaking-priors-1apv545uj7.pdf

Gibbs sampling methods for stick-breaking priors

We test for asymmetry in a model of the dependence between the Deutsche mark and the yen, in the sense that a different degree of correlation is exhibited during joint appreciations against the U.S. dollar versus during joint depreciations. We consider an extension of the theory of copulas to allow for conditioning variables, and employ it to construct flexible models of the conditional dependence structure of these exchange rates. We find evidence that the mark‐dollar and yen‐dollar exchange rates are more correlated when they are depreciating against the dollar than when they are appreciating.

/pdf/modelling-asymmetric-exchange-rate-dependence-3ncll9uq7l.pdf

Modelling asymmetric exchange rate dependence

A Course in Large Sample Theory is presented in four parts. The first treats basic probabilistic notions, the second features the basic statistical tools for expanding the theory, the third contains special topics as applications of the general theory, and the fourth covers more standard statistical topics. Nearly all topics are covered in their multivariate setting.The book is intended as a first year graduate course in large sample theory for statisticians. It has been used by graduate students in statistics, biostatistics, mathematics, and related fields. Throughout the book there are many examples and exercises with solutions. It is an ideal text for self study.

A Course in Large Sample Theory

/pdf/a-representation-of-independent-increment-processes-without-2wygqpubnd.pdf

A Representation of Independent Increment Processes without Gaussian Components

An example is given of a family of distributions on [— 1, 1] with a continuous one-dimensional parameterization that joins the triangular distribution (when Θ = 0) to the uniform (when Θ = 1), for which the maximum likelihood estimates exist and converge strongly to Θ = 1 as the sample size tends to infinity, whatever be the true value of the parameter. A modification that satisfies Cramer's conditions is also given.

/pdf/an-inconsistent-maximum-likelihood-estimate-2k6y7jpbvx.pdf

An Inconsistent Maximum Likelihood Estimate

The authors show how Kendall's tau can be adapted to test against serial dependence in a univariate time series context. They provide formulas for the mean and variance of circular and noncircular versions of this statistic, and they prove its asymptotic normality under the hypothesis of independence. They present also a Monte Carlo study comparing the power and size of a test based on Kendall's tau with the power and size of competing procedures based on alternative parametric and nonparametric measures of serial dependence. In particular, their simulations indicate that Kendall's tau outperforms Spearman's rho in detecting first-order autoregressive dependence, despite the fact that these two statistics are asymptotically equivalent under the null hypothesis, as well as under local alternatives.

/pdf/kendall-s-tau-for-serial-dependence-dgp3v2unb0.pdf

Kendall's tau for serial dependence

Summary. A general theorem on the asymptotically optimal sequential selection of experiments is presented and applied to a Bayesian classification problem when the parameter space is a finite partially ordered set. The main results include establishing conditions under which the posterior probability of the true state converges to 1 almost surely and determining optimal rates of convergence. Properties of a class of experiment selection rules are explored.

/pdf/sequential-classification-on-partially-ordered-sets-93fak0pbdk.pdf

Thomas S. Ferguson

Papers

A Course in Large Sample Theory

A Representation of Independent Increment Processes without Gaussian Components

An Inconsistent Maximum Likelihood Estimate

Kendall's tau for serial dependence

Sequential classification on partially ordered sets