The Bayesian approach to statistical problems, though fruitful in many ways, has been rather unsuccessful in treating nonparametric problems. This is due primarily to the difficulty in finding workable prior distributions on the parameter space, which in nonparametric ploblems is taken to be a set of probability distributions on a given sample space. There are two desirable properties of a prior distribution for nonparametric problems. (I) The support of the prior distribution should be large--with respect to some suitable topology on the space of probability distributions on the sample space. (II) Posterior distributions given a sample of observations from the true probability distribution should be manageable analytically. These properties are antagonistic in the sense that one may be obtained at the expense of the other. This paper presents a class of prior distributions, called Dirichlet process priors, broad in the sense of (I), for which (II) is realized, and for which treatment of many nonparametric statistical problems may be carried out, yielding results that are comparable to the classical theory. In Section 2, we review the properties of the Dirichlet distribution needed for the description of the Dirichlet process given in Section 3. Briefly, this process may be described as follows. Let $\mathscr{X}$ be a space and $\mathscr{A}$ a $\sigma$-field of subsets, and let $\alpha$ be a finite non-null measure on $(\mathscr{X}, \mathscr{A})$. Then a stochastic process $P$ indexed by elements $A$ of $\mathscr{A}$, is said to be a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$ if for any measurable partition $(A_1, \cdots, A_k)$ of $\mathscr{X}$, the random vector $(P(A_1), \cdots, P(A_k))$ has a Dirichlet distribution with parameter $(\alpha(A_1), \cdots, \alpha(A_k)). P$ may be considered a random probability measure on $(\mathscr{X}, \mathscr{A})$, The main theorem states that if $P$ is a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha$, and if $X_1, \cdots, X_n$ is a sample from $P$, then the posterior distribution of $P$ given $X_1, \cdots, X_n$ is also a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with a parameter $\alpha + \sum^n_1 \delta_{x_i}$, where $\delta_x$ denotes the measure giving mass one to the point $x$. In Section 4, an alternative definition of the Dirichlet process is given. This definition exhibits a version of the Dirichlet process that gives probability one to the set of discrete probability measures on $(\mathscr{X}, \mathscr{A})$. This is in contrast to Dubins and Freedman [2], whose methods for choosing a distribution function on the interval [0, 1] lead with probability one to singular continuous distributions. Methods of choosing a distribution function on [0, 1] that with probability one is absolutely continuous have been described by Kraft [7]. The general method of choosing a distribution function on [0, 1], described in Section 2 of Kraft and van Eeden [10], can of course be used to define the Dirichlet process on [0, 1]. Special mention must be made of the papers of Freedman and Fabius. Freedman [5] defines a notion of tailfree for a distribution on the set of all probability measures on a countable space $\mathscr{X}$. For a tailfree prior, posterior distribution given a sample from the true probability measure may be fairly easily computed. Fabius [3] extends the notion of tailfree to the case where $\mathscr{X}$ is the unit interval [0, 1], but it is clear his extension may be made to cover quite general $\mathscr{X}$. With such an extension, the Dirichlet process would be a special case of a tailfree distribution for which the posterior distribution has a particularly simple form. There are disadvantages to the fact that $P$ chosen by a Dirichlet process is discrete with probability one. These appear mainly because in sampling from a $P$ chosen by a Dirichlet process, we expect eventually to see one observation exactly equal to another. For example, consider the goodness-of-fit problem of testing the hypothesis $H_0$ that a distribution on the interval [0, 1] is uniform. If on the alternative hypothesis we place a Dirichlet process prior with parameter $\alpha$ itself a uniform measure on [0, 1], and if we are given a sample of size $n \geqq 2$, the only nontrivial nonrandomized Bayes rule is to reject $H_0$ if and only if two or more of the observations are exactly equal. This is really a test of the hypothesis that a distribution is continuous against the hypothesis that it is discrete. Thus, there is still a need for a prior that chooses a continuous distribution with probability one and yet satisfies properties (I) and (II). Some applications in which the possible doubling up of the values of the observations plays no essential role are presented in Section 5. These include the estimation of a distribution function, of a mean, of quantiles, of a variance and of a covariance. A two-sample problem is considered in which the Mann-Whitney statistic, equivalent to the rank-sum statistic, appears naturally. A decision theoretic upper tolerance limit for a quantile is also treated. Finally, a hypothesis testing problem concerning a quantile is shown to yield the sign test. In each of these problems, useful ways of combining prior information with the statistical observations appear. Other applications exist. In his Ph. D. dissertation [1], Charles Antoniak finds a need to consider mixtures of Dirichlet processes. He treats several problems, including the estimation of a mixing distribution, bio-assay, empirical Bayes problems, and discrimination problems.

/pdf/a-bayesian-analysis-of-some-nonparametric-problems-siibmg5pnf.pdf

A Bayesian Analysis of Some Nonparametric Problems

process. This paper extends Ferguson's result to cases where the random measure is a mixing distribution for a parameter which determines the distribution from which observations are made. The conditional distribution of the random measure, given the observations, is no longer that of a simple Dirichlet process, but can be described as being a mixture of Dirichlet processes. This paper gives a formal definition for these mixtures and develops several theorems about their properties, the most important of which is a closure property for such mixtures. Formulas for computing the conditional distribution are derived and applications to problems in bio-assay, discrimination, regression, and mixing distributions are given.

/pdf/mixtures-of-dirichlet-processes-with-applications-to-4brx5ee622.pdf

Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems

SUMMARY We wish to test a simple hypothesis against a family of alternatives indexed by a one-dimensional parameter, 0. We use a test derived from the corresponding family of test statistics appropriate for the case when 0 is given. Davies (1977) introduced this problem when these test statistics had normal distributions. The present paper considers the case when their distribution is chi-squared. The results are applied to the detection of a discrete frequency component of unknown frequency in a time series. In addition quick methods for finding approximate significance probabilities are given for both the normal and chi-squared cases and applied to the two-phase regression problem in the normal case.

Hypothesis testing when a nuisance parameter is present only under the alternative

True randomness does not exist in classical physics, where randomness is necessarily a result of forces that may be unknown but exist. The quantum world, however, is intrinsically truly random. This is difficult to prove, as it is not readily distinguishable from noise and other uncontrollable factors. Now Pironio et al. present proof of a quantitative relationship between two fundamental concepts of quantum mechanics — randomness and the non-locality of entangled particles. They first show theoretically that the violation of a Bell inequality certifies the generation of new randomness, independently of any implementation details. To illustrate the approach, they then perform an experiment in which — as confirmed using the theoretical tools that they developed — 42 new random bits have been generated. As well as having conceptual implications, this work has practical implications for cryptography and for numerical simulation of physical and biological systems. Here it is shown, both theoretically and experimentally, that non-local correlations between entangled quantum particles can be used for a new cryptographic application — the generation of certified private random numbers — that is impossible to achieve classically. The results have implications for future device-independent quantum information experiments and for addressing fundamental issues regarding the randomness of quantum theory. Randomness is a fundamental feature of nature and a valuable resource for applications ranging from cryptography and gambling to numerical simulation of physical and biological systems. Random numbers, however, are difficult to characterize mathematically1, and their generation must rely on an unpredictable physical process2,3,4,5,6. Inaccuracies in the theoretical modelling of such processes or failures of the devices, possibly due to adversarial attacks, limit the reliability of random number generators in ways that are difficult to control and detect. Here, inspired by earlier work on non-locality-based7,8,9 and device-independent10,11,12,13,14 quantum information processing, we show that the non-local correlations of entangled quantum particles can be used to certify the presence of genuine randomness. It is thereby possible to design a cryptographically secure random number generator that does not require any assumption about the internal working of the device. Such a strong form of randomness generation is impossible classically and possible in quantum systems only if certified by a Bell inequality violation15. We carry out a proof-of-concept demonstration of this proposal in a system of two entangled atoms separated by approximately one metre. The observed Bell inequality violation, featuring near perfect detection efficiency, guarantees that 42 new random numbers are generated with 99 per cent confidence. Our results lay the groundwork for future device-independent quantum information experiments and for addressing fundamental issues raised by the intrinsic randomness of quantum theory.

/pdf/random-numbers-certified-by-bell-s-theorem-5chjbj1xv7.pdf

Random numbers certified by Bell's theorem.

Local realism is the worldview in which physical properties of objects exist independently of measurement and where physical influences cannot travel faster than the speed of light. Bell's theorem states that this worldview is incompatible with the predictions of quantum mechanics, as is expressed in Bell's inequalities. Previous experiments convincingly supported the quantum predictions. Yet, every experiment requires assumptions that provide loopholes for a local realist explanation. Here, we report a Bell test that closes the most significant of these loopholes simultaneously. Using a well-optimized source of entangled photons, rapid setting generation, and highly efficient superconducting detectors, we observe a violation of a Bell inequality with high statistical significance. The purely statistical probability of our results to occur under local realism does not exceed 3.74×10^{-31}, corresponding to an 11.5 standard deviation effect.

https://physics.aps.org/featured-article-pdf/10.1103/PhysRevLett.115.250401

Significant-Loophole-Free Test of Bell's Theorem with Entangled Photons

Testing and estimating ordered parameters of probability distribution

https://yorkspace.library.yorku.ca/xmlui/bitstream/handle/10315/921/3-Asymptotic.pdf?sequence=1

Asymptotic properties of maximum weighted likelihood estimators

Mathematical statistics and applications : festschrift for Constance van Eeden

Introduction: Some history and some examples.- A statement of the problem, the notation and some definitions.- (In)admissibility and dominators.- Minimax estimators and their admissibility.- Presence of nuisance parameters.- The linear model.- Other properties.- Existence of MLEs and algorithms to compute them.

Restricted Parameter Space Estimation Problems: Admissibility and Minimaxity Properties

Linearized estimates, as functions of the ranks, are proposed for the general linear hypothesis. These estimates can be computed after a single ranking of the "centered" observations. The asymptotic distribution of the estimates is shown to be the same as the maximum likelihood estimates for fairly general sequences of design matrices.

/pdf/linearized-rank-estimates-and-signed-rank-estimates-for-the-cspdnzrcon.pdf

Constance van Eeden

Papers

Testing and estimating ordered parameters of probability distribution

Asymptotic properties of maximum weighted likelihood estimators

Mathematical statistics and applications : festschrift for Constance van Eeden

Restricted Parameter Space Estimation Problems: Admissibility and Minimaxity Properties

Linearized Rank Estimates and Signed-Rank Estimates for the General Linear Hypothesis