The information deviation between any two finite measures cannot be increased by any statistical operations (Markov morphisms). It is invarient if and only if the morphism is sufficient for these two measures

On Information and Sufficiency

Let X 1 …, X n be n independent random vectors, X v = , and Φ(x 1 …, x m ) a function of m(≤n) vectors . A statistic of the form , where the sum ∑″ is extended over all permutations (α1 …, α m ) of different integers, 1 α≤ (αi≤ n, is called a U-statistic. If X 1, …, X n have the same (cumulative) distribution function (d.f.) F(x), U is an unbiased estimate of the population characteristic θ(F) = f … f Φ(x 1,…, x m ) dF(x 1) … dF(x m ). θ(F) is called a regular functional of the d.f. F(x). Certain optimal properties of U-statistics as unbiased estimates of regular functionals have been established by Halmos [9] (cf. Section 4)

/pdf/a-class-of-statistics-with-asymptotically-normal-2a2cgk96ay.pdf

A Class of Statistics with Asymptotically Normal Distribution

Bayesian statistics, a currently controversial viewpoint concerning statistical inference, is based on a definition of probability as a particular measure of the opinions of ideally consistent people. Statistical inference is modification of these opinions in the light of evidence, and Bayes’ theorem specifies how such modifications should be made. The tools of Bayesian statistics include the theory of specific distributions and the principle of stable estimation, which specifies when actual prior opinions may be satisfactorily approximated by a uniform distribution. A common feature of many classical significance tests is that a sharp null hypothesis is compared with a diffuse alternative hypothesis. Often evidence which, for a Bayesian statistician, strikingly supports the null hypothesis leads to rejection of that hypothesis by standard classical procedures. The likelihood principle emphasized in Bayesian statistics implies, among other things, that the rules governing when data collection stops are irrelevant to data interpretation. It is entirely appropriate to collect data until a point has been proven or disproven, or until the data collector runs out of time, money, or patience.

Bayesian statistical inference for psychological research.

1. Summary Bohnenblust, Shapley, and Sherman [2] have introduced a method of comparing two sampling procedures or experiments; essentially their concept is that one experiment a is more informative than a second experiment ,, a v ,S, if, for every possible risk function, any risk attainable with , is also attainable with a. If a is a sufficient statistic for a procedure equivalent to ,S, a >,, it is shown that a v j3. In the case of dichotomies, the converse is proved. Whether > and v are equivalent in general is not known. Various properties of > and n are obtained, such as the following: if a > , and y is independent of both, then the combination (a, -y) > (#, y). An application to a problem in 2 X 2 tables is discussed.

/pdf/comparison-of-experiments-1owlchcq3a.pdf

Comparison of Experiments

We extend the definition of coherent risk measures, as introduced by Artzner, Delbaen, Eber and Heath, to general probability spaces and we show how to define such measures on the space of all random variables. We also give examples that relates the theory of coherent risk measures to game theory and to distorted probability measures. The mathematics are based on the characterisation of closed convex sets Pσ of probability measures that satisfy the property that every random variable is integrable for at least one probability measure in the set Pσ.

/pdf/coherent-risk-measures-on-general-probability-spaces-1t7s2w9ugb.pdf

Coherent Risk Measures on General Probability Spaces

The body of this paper is written in terms of very general and abstract ideas which have been popular in pure mathematical work on the theory of probability for the last two or three decades. It seems to us that these ideas, so fruitful in pure mathematics, have something to contribute to mathematical statistics also, and this paper is an attempt to illustrate the sort of contribution we have in mind. The purpose of generality here is not to solve immediate practical problems, but rather to capture the logical essence of an important concept (sufficient statistic), and in particular to disentangle that concept from such ideas as Euclidean space, dimensionality, partial differentiation, and the distinction between continuous and discrete distributions, which seem to us extraneous. In accordance with these principles the center of the stage is occupied by a completely abstract sample space--that is a set $X$ of objects $x$, to be thought of as possible outcomes of an experimental program, distributed according to an unknown one of a certain set of probability measures. Perhaps the most familiar concrete example in statistics is the one in which $X$ is $n$ dimensional Cartesian space, the points of which represent $n$ independent observations of a normally distributed random variable with unknown parameters, and in which the probability measures considered are those induced by the various common normal distributions of the individual observations. A statistic is defined, as usual, to be a function $T$ of the outcome, whose values, however, are not necessarily real numbers but may themselves be abstract entities. Thus, in the concrete example, the entire set of $n$ observations, or, less trivially, the sequence of all sample moments about the origin are statistics with values in an $n$ dimensional and in an infinite dimensional space respectively. Another illuminating and very general example of a statistic may be obtained as follows. Suppose that the outcomes of two not necessarily statistically independent programs are thought of as one united outcome--then the outcome $T$ of the first program alone is a statistic relative to the united program. A technical measure theoretic result, known as the Radon-Nikodym theorem, is important in the study of statistics such as $T$. It is, for example, essential to the very definition of the basic concept of conditional probability of a subset $E$ of $X$ given a value $y$ of $T$. The statistic $T$ is called sufficient for the given set $\mathcal{M}$ of probability measures if (somewhat loosely speaking) the conditional probability of a subset $E$ of $X$ given a value $y$ of $T$ is the same for every probability measure in $\mathcal{M}$. It is, for instance, well known that the sample mean and variance together form a sufficient statistic for the measures described in the concrete example. The theory of sufficiency is in an especially satisfactory state for the case in which the set $\mathcal{M}$ of probability measures satisfies a certain condition described by the technical term dominated. A set $\mathcal{M}$ of probability measures is called dominated if each measure in the set may be expressed as the indefinite integral of a density function with respect to a fixed measure which is not itself necessarily in the set. It is easy to verify that both classical extremes, commonly referred to as the discrete and continuous cases, are dominated. One possible formulation of the principal result concerning sufficiency for dominated sets is a direct generalization to the abstract case of the well known Fisher-Neyman result: $T$ is sufficient if and only if the densities can be written as products of two factors, the first of which depends on the outcome through $T$ only and the second of which is independent of the unknown measure. Another way of phrasing this result is to say that $T$ is sufficient if and only if the likelihood ratio of every pair of measures in $\mathcal{M}$ depends on the outcome through $T$ only. The latter formulation makes sense even in the not necessarily dominated case but unfortunately it is not true in that case. The situation can be patched up somewhat by introducing a weaker notion called pairwise sufficiency. In ordinary statistical parlance one often speaks of a statistic sufficient for some of several parameters. The abstract results mentioned above can undoubtedly be extended to treat this concept.

/pdf/application-of-the-radon-nikodym-theorem-to-the-theory-of-g5wpveyah1.pdf

Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics

Let $F(P)$ be a real valued function defined on a subset $\mathscr{D}$ of the set $\mathscr{D}^\ast$ of all probability distributions on the real line. A function $f$ of $n$ real variables is an unbiased estimate of $F$ if for every system, $X_1, \cdots, X_n$, of independent random variables with the common distribution $P$, the expectation of $f(X_1 \cdots, X_n)$ exists and equals $F(P)$, for all $P$ in $\mathscr{D}$. A necessary and sufficient condition for the existence of an unbiased estimate is given (Theorem 1), and the way in which this condition applies to the moments of a distribution is described (Theorem 2). Under the assumptions that this condition is satisfied and that $\mathscr{D}$ contains all purely discontinuous distributions it is shown that there is a unique symmetric unbiased estimate (Theorem 3); the most general (non symmetric) unbiased estimates are described (Theorem 4); and it is proved that among them the symmetric one is best in the sense of having the least variance (Theorem 5). Thus the classical estimates of the mean and the variance are justified from a new point of view, and also, from the theory, computable estimates of all higher moments are easily derived. It is interesting to note that for $n$ greater than 3 neither the sample $n$th moment about the sample mean nor any constant multiple thereof is an unbiased estimate of the $n$th moment about the mean. Attention is called to a paradoxical situation arising in estimating such non linear functions as the square of the first moment.

/pdf/the-theory-of-unbiased-estimation-ks3z9ifsqn.pdf

Paul R. Halmos

Papers

Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics

The Theory of Unbiased Estimation