Showing papers in "Annals of Mathematical Statistics in 1965"

The Existence of Probability Measures with Given Marginals

Abstract: First an integral representation of a continuous linear functional dominated by a support function in integral form is given (Theorem 1). From this the theorem of Blackwell-Stein-Sherman-Cartier [2], [20], [4], is deduced as well as a result on capacities alternating of order 2 in the sense of Choquet [5], which includes Satz 4.3 of [23] and a result of Kellerer [10], [12], under somewhat stronger assumptions. Next (Theorem 7), the existence of probability distributions with given marginals is studied under typically weaker assumptions, than those which are required by the use of Theorem 1. As applications we derive necessary and sufficient conditions for a sequence of probability measures to be the sequence of distributions of a martingale (Theorem 8), an upper semi-martingale (Theorem 9) or of partial sums of independent random variables (Theorem 10). Moreover an alternative definition of Levy-Prokhorov's distance between probability measures in a complete separable metric space is obtained (corollary of Theorem 11). Section 6 can be read independently of the former sections.

A nonparametric estimate of a multivariate density function

Abstract: Let $x_1, \cdots, x_n$ be independent observations on a $p$-dimensional random variable $X = (X_1, \cdots, X_p)$ with absolutely continuous distribution function $F(x_1, \cdots, x_p)$. An observation $x_i$ on $X$ is $x_i = (x_{1i}, \cdots, x_{pi})$. The problem considered here is the estimation of the probability density function $f(x_1, \cdots, x_p)$ at a point $z = (z_1, \cdots, z_p)$ where $f$ is positive and continuous. An estimator is proposed and consistency is shown. The problem of estimating a probability density function has only recently begun to receive attention in the literature. Several authors [Rosenblatt (1956), Whittle (1958), Parzen (1962), and Watson and Leadbetter (1963)] have considered estimating a univariate density function. In addition, Fix and Hodges (1951) were concerned with density estimation in connection with nonparametric discrimination. Cacoullos (1964) generalized Parzen's work to the multivariate case. The work in this paper arose out of work on the nonparametric discrimination problem.

On the asymptotic theory of fixed-width sequential confidence intervals for the mean.

Abstract: Let x1, x2, • • • be a sequence of independent observations from some population. We want to find a confidence interval of prescribed width 2d and prescribed coverage probability a for the unknown mean µ of the population. If the variance σ2 of the population is known, and if d is small compared to σ 2, this can be done as follows. For any n ⪴ 1 define \({x_n} = {n^{ - 1}}\sum olimits_1^n {Xi,{I_n}} = \left[ {{x_n} - d,{x_n} + d} \right],\) and choose a to satisfy \(\left( {2\pi } \right){ - ^{{1 \over 2}}}{\int_{ - a}^a e ^{ - u2/2}}du = a\)

A Robust Version of the Probability Ratio Test

Abstract: A statistical procedure is called robust, if its performance is insensitive to small deviations of the actual situation from the idealized theoretical model. In particular, a robust procedure should be insensitive to the presence of a few "bad" observations; that is, a small minority of the observations should never be able to override the evidence of the majority. (But at the same time the discordant minority might be a prime source of information for improving the theoretical model!) The classical probability ratio test is not robust in this sense: a single factor $p_1(x_j)/p_0(x_j)$ equal (or almost equal) to 0 or $\infty$ may upset the test statistic $T(x) = \prod^n_1 p_1(x_j)/p_0(x_j)$. This leads to the conjecture that appropriate robust substitutes to both fixed sample size and sequential probability ratio tests might be obtained by censoring the single factors at some fixed numbers $c' < c''$. Thus, one would replace the test statistic by $T'(x) = \prod^n_1 \pi(x_j)$, where $\pi(x_j) = \max (c', \min (c'', p_1(x_j)/p_0(x_j)))$. The problem of robustly testing a simple hypothesis $P_0$ against a simple alternative $P_1$ may be formalized by assuming that the true underlying distribution lies in some neighborhood of either of the idealized model distributions $P_0$ or $P_1$. The present paper exhibits two different types of such neighborhoods for which the above mentioned test, to be called censored probability ratio test, is most robust in a well defined minimax sense. The problem solved here originated through the earlier paper Huber (1964), over the question how to test hypotheses about the mean of contaminated normal distributions.

An Introduction to Polyspectra

Abstract: The subject of this paper is the higher-order spectra or polyspectra of multivariate stationary time series. The intent is to derive (i) certain mathematical properties of polyspectra, (ii) estimates of polyspectra based on an observed stretch of time series, (iii) certain statistical properties of the proposed estimates and (iv) several applications of the results obtained.

Asymptotically Optimal Tests for Multinomial Distributions

Abstract: Tests of simple and composite hypotheses for multinomial distributions are considered. It is assumed that the size αN of a test tends to 0 as the sample size N increases. The main concern of this paper is to substantiate the following proposition: If a given test of size αN is “sufficiently different” from a likelihood ratio test then there is a likelihood ratio test of size ≦αN which is considerably more powerful than the given test at “most” points in the set of alternatives when N is large enough, provided that αN → 0 at a suitable rate. In particular, it is shown that chi-square tests of simple and of some composite hypotheses are inferior, in the sense described, to the corresponding likelihood ratio tests. Certain Bayes tests are shown to share the above-mentioned property of a likelihood ratio test.

On the Number of Successes in Independent Trials

Abstract: The distribution of the number of successes in independent trials is shown to be bell-shaped of every order. The most likely number of successes is "almost uniquely" determined from the mean number and from the mean plus the largest and smallest probability of success on any trial. Bounds on the distribution function of the number of successes are obtained and extended to an infinite number of trials, including the Poisson distribution.

Inequalities for the $r$th Absolute Moment of a Sum of Random Variables, $1 \leqq r \leqq 2$

Abstract: Let $X_1, X_2, \cdots, X_n$ be a sequence of random variables (r.v.'s) and put $S_m = \sum^m_{ u = 1} X_ u, 1 \leqq m \leqq n$. It is well-known that \begin{equation*}\tag{(1)}E|S_n|^r \leqq n^{r - 1} \sum^n_{ u = 1} E|X_ u|^r\quad r > 1,\end{equation*} $E|S_n|^r \leqq \sum^n_{ u = 1} E|X_|nu|^r,\quad r \leqq 1.$ However, if the r.v.'s satisfy the relations \begin{equation*}\tag{2}E(X_{m + 1} \mid S_m) = 0 \text{a.s.}\quad 1 \leqq m \leqq n - 1,\end{equation*} it is possible to improve the first inequality considerably. The case $r > 2$ with independent r.v.'s will be treated elsewhere by one of the authors, von Bahr. If $r = 2$, we have, under (2), \begin{equation*}\tag{3}ES^2_n = \sum^n_{ u = 1} EX^2_ u.\end{equation*} In the case $1 \leqq r \leqq 2$, we will show that under (2) \begin{equation*}\tag{(4)}E|S_n|^r \leqq C(r, n) \sum^n_{ u = 1}e|X_ u|^r,\end{equation*} where $C(r, n)$ is a bounded function of $r$ and $n$. In Theorem 2 we show that (4) is true with $C(r, n) = 2$. If the distribution of each $X_{m + 1}$ conditioned by $S_m$ is symmetric about zero, one can put $C(r, n) = 1$ (Theorem 1). Further, if the r.v.'s satisfy the following conditions \begin{equation*}\tag{(5)}E(X_i \mid R_{mi}) = 0\text{a.s.}\quad 1 \leqq i \leqq m + 1 \leqq n,\end{equation*} where $R_{mi} = \sum^{m + 1}_{ u = 1, u eq i} X_ u$ it is possible to put $C(r, n) = 2 - n^{-1}$. The conditions (2) and (5) are fulfilled if the r.v.'s are independent and have zero means. In this case, however, it is possible to make $C(r, n)$ dependent on $r$, so that $C(r, n) \rightarrow 1$ as $r \rightarrow 2$. It is possible to show by an example, that (4) is not generally true with $C(r, n) = 1$ even in this case. If $1 \leqq r < s \leqq 2$ and $E|X_ u|^s < \infty, 1 \leqq u \leqq n$, it is generally better not to use (4) directly, but to use it together with $E|S_n|^r \leqq (E|S_n|^s)^{r/s}$, so that $E|S_n|^r \leqq (C(s, n) \sum^n_{ u = 1} E|X_ u|^s)^{r/s}$. The case $r < 1$ is by (1) trivial.

Estimation of the Bispectrum

Abstract: Recently interest has arisen in statistical applications of the bispectrum of stationary random processes. (The bispectrum can be thought of as the Fourier transform of the third-order moment function of the process.) The principal area of statistical harmonic analysis to receive attention previous to this time has been second-order (i.e. spectral) theory on which there is a vast literature. However, the spectrum is most useful in problems of a "linear nature" (see discussion beginning on p. viii of Blackman and Tukey [2]) and provides insufficient information in nonlinear problems. A desire to study phenomena of a nonlinear character has attracted attention to the higher order theory. Such was the case, for example, in a recent study by Hasselmann, Munk and MacDonald [8] where the bispectrum is used in connection with oceanographic problems, among which, as the authors state, a number of interesting phenomena such as surf beats, wave breaking, and the energy transfer between wave components can be explained only by the nonlinearity of the wave motion. The bispectrum therefore provides a first glimpse at the nonlinear effects. It is the purpose of the present paper to discuss estimating the weighted and unweighted bispectral density given a set of observations of the process. The relevant properties (consistency and asymptotic unbiasedness) of the estimates are derived for certain general classes of processes.

Bernard Friedman's Urn

Abstract: The case $\beta = 0$ is the famous Polya (1931) Urn; a discussion of its elementary properties can be found in (Feller, 1960, Chapter IV) and (Frechet, 1943). These facts about the Polya Urn are a classical part of the oral tradition, although some have yet to appear in print (see Blackwell and Kendall, 1964). The fractions $(W_n + B_n)^{-1}W_n$ converge with probability 1 to a limiting random variable $Z$, which has a beta distribution with parameters $W_0/\alpha, B_0/\alpha$. Given $Z$, the successive differences $W_{n + 1} - W_n :n \geqq 0$ are conditionally independent and identically distributed, being $\alpha$ with probability $Z$ and 0 with probability $1 - Z$. Proofs are in Section 2. If $\beta > 0$, the situation is radically different. No matter how large $\alpha$ is in comparison with $\beta$, the fractions $(W_n + B_n)^{-1}W_n$ converge to $\frac{1}{2}$ with probability 1. This seemingly paradoxical result can be sharpened in several ways. Abbreviate $\rho$ for $(\alpha + \beta)^{-1}(\alpha - \beta)$. If $\rho > \frac{1}{2}$, it is proved in Section 3 that $(W_n + B_n)^{-\rho}. (W_n - B_n)$ converges with probability 1 to a nondegenerate limiting random variable. This result in turn fails for $\rho \leqq \frac{1}{2}$. If $0 0$. Suppose first $\alpha > \beta$. If $0 \leqq x \leqq 1$ and $P\lbrack\lim \sup (W_n + B_n)^{-1}W_n \leqq x\rbrack = 1$, by an easy variation of the Strong Law, with probability 1, in $N$ trials there will be at most $Nx + o(N)$ drawings of a white ball; so at least $N(1 - x) - o(N)$ drawings of black. Therefore, with probability 1, $\lim \sup (W_n + B_n)^{-1}B_n$ is bounded above by $\lim_{N\rightarrow\infty}\{\alpha\lbrack Nx + o(N)\rbrack + \beta\lbrack N(1 - x) - o(N)\rbrack\}/N(\alpha + \beta)$ or $(\alpha + \beta)^{-1}\lbrack\beta + (\alpha - \beta)x\rbrack$. Starting with $x = 1$ and iterating, $P\lbrack\lim \sup (W_n + B_n)^{-1} \leqq \frac{1}{2}\rbrack = 1$ follows. Interchange white and black to complete the proof for $\alpha > \beta$. If $\alpha < \beta$, and $P\lbrack\lim \sup (W_n + B_n)^{-1}W_n \leqq x\rbrack = 1$, then a similar argument shows $P\lbrack\lim \sup (W_n + B_n)^{-1}B_n \leqq (\alpha + \beta)^{-1}. (\alpha + (\beta - \alpha)x)\rbrack = 1$. The argument proceeds as before, except both colors must be considered simultaneously.

Classical Statistical Analysis Based on a Certain Multivariate Complex Gaussian Distribution

Abstract: N. R. Goodman [4] has discussed some aspects of the complex multivariate normal distribution, in particular, the analogue of the Wishart distribution and of multiple and partial correlations. We shall obtain maximum likelihood estimates for certain parameters and likelihood ratio tests of certain hypotheses arising in the study of such complex multivariate normal distributions. Although the general principles involved in the derivation of the distribution of the associated statistics are well known to mathematicians working on group representations (see, for example, Gelfand and Naimark [3], p. 24), it has seemed desirable to derive the needed results on this type in a way parallel to one method of obtaining the distributions of real multivariate analysis (Deemer and Olkin [2], Olkin [6] and Roy [7]). With the help of the results derived, we can handle the complex variates in the same manner as we do for real variates in the case of Gaussian distributions. Moreover, it can be noted that for every distributional result of classical multivariate Gaussian statistical analysis obtainable in closed (explicit) form, the counterpart analysis for complex Gaussian is also obtainable in closed (explicit) form with necessary changes. It may be pointed out that the non-central distributions in this connection were derived independently by A. T. James [5] with the help of zonal polynomials of hermitian matrices, but we feel that sometimes the derivation of distributions with the help of Jacobian transformations may be useful.

On Some Robust Estimates of Location

Abstract: 1. Summary. During the past 15 years various approaches have been proposed to deal with the lack of robustness of the sample mean as an estimate of the population mean when the distribution sampled is contaminated by gross errors, i.e., has heavier tails than the normal distribution. First, Tukey and the Statistical Research Group at Princeton in [9] suggested and investigated the properties of "trimmed" and "Winsorized" means. More recently, Hodges and Lehmarn [6], proposed estimates related to the well-known robust Wilcoxon and normal scores tests, among others. Finally Huber in [7] considered essentially the class of maximum likelihood estimates and found those members of this class which minimize the maximum variance over various classes of contaminated distributions. For a review of work in these directions in testing as well as estimation the reader is referred to Elashoff [3]. In Theoremns 3.1 and 3.2 we state the main results of the asymptotic theory of the Winsorized and trimmed means and outline the proof. An alternative method of trimming and Winsorizing (not equivalent to that of Tukey) which encompasses the efficient estimates proposed by Huber and which generalizes to higher dimensions is discussed in Section 2. The fourth section (Theorem 4.1) gives the miniinum efficiency with respect to the families of all symmetric and symmetric unimiodal distributions, of the Winsorized and trimmed means with respect to the mean. The lower bounds found for the trimmed means (for small trimming proportions) in the unimodal case compare well with that found by Hodges and Lehmann in [5] for the median of averages of pairs, the Hodges-Lehmann estimate. However, the Winsorized

Admissibility and bayes estimation in sampling finite populations. ii

Abstract: tion is extended by removing the restriction of unbiasedness, with the corresponding modification of the definition of admissibility: Now some other estimate is shown to remain admissible for all sampling designs. The result appears to have implications concerning the basic logic of sampling with varying probabilities. These however are not discussed here. 2. Notation. The notation used here is the same as that formulated in the Section 2 of the Part I of this paper and is not restated here. The definitions and preliminaries, as given in that section, also apply in the following discussion. In addition for convenience of discussion, here we assume that the units u of the population U are numbered, that is U = (Ui, . , UN), N being the total number of units u in U. As a result a sample s (Definition 2.2, Part I) can now be specified by the set of integers namely the serial numbers of the units u - s. Thus for Ur I s now we write r - s. Further, the variate value X(Ur) associated

Extreme Values in Uniformly Mixing Stationary Stochastic Processes

Abstract: Let $X_1, X_2, \cdots$ be a sequence of independent, identically distributed random variables, and write $Z_n$ for the maximum of $X_1, X_2, \cdots X_n$. Then there are two well known theorems concerning the limiting behaviour of the distribution of $Z_n$. (See, for example, Gumbel [7].) Firstly, if for some sequence of pairs of numbers $a_n, b_n$, the quantities $a^{-1}_n(Z_n - b_n)$ have a non-degenerate limiting distribution as $n \rightarrow \infty$, then this limit must take one of three forms. Secondly, if $c_n = c_n(\xi)$ is defined by $P\lbrack X > c_n\rbrack \leqq \xi/n \leqq P\lbrack X \geqq c_n\rbrack$, then $P\lbrack Z_n \leqq c_n\rbrack$ tends to $e^{-\xi}$ as $n \rightarrow \infty$. Suppose now that we drop the assumption of independence of the $X_j$, and require instead that the sequence $\{X_n\}$ be a stationary stochastic process: then it might be expected that similar results will hold, at least if $X_i$ and $X_j$ are nearly independent when $|i - j|$ is large. In Section 2 it will be shown that, if the process $\{X_n\}$ is uniformly mixing, then the only possible non-degenerate limit laws of $a^{-1}_n(Z_n - b_n)$ are just those that occur in the case of independence, and that the only possible limit laws of $P\lbrack Z_n \leqq c_n\rbrack$ are of the form $e^{-k\xi}, k$ being some positive constant not greater than one. The uniform mixing property is rather strong at first sight. It is however clear that some restriction is necessary, at least for example to ergodic processes, and the parallel which exists to a certain extent with normed sums of random variables suggests that uniform mixing hypotheses may be appropriate. (Cf. Rosenblatt [9]). In the independent case converse results hold, as we have already observed for the second problem, giving necessary and sufficient conditions for the existence of the limits. We give some results concerning this problem in Section 3, but they are not altogether satisfactory. Berman ([2] and especially [3]) has investigated the same problem, under somewhat different conditions. His results for the Gaussian case in [3], however, include those which can be obtained from Lemmas 1 and 2 of the present paper, and in consequence we do not reproduce them here. The second problem mentioned above was considered by Watson [10] for $m$-dependent stationary processes, and his paper did in fact suggest the present investigations. His results are contained in Section 3. Certain results were announced without proof by Chibisov [4], but these appear not to overlap our results.

Bayesian estimation in multivariate analysis

Abstract: : The Bayes approach to Multivariate Analysis taken previously by Geisser and Cornfield (JRSS Series B, 1963 No. 2, pp. 368-376) is extended and given a more comprehensive treatment. Posterior joint and marginal densities are derived for vector means, linear combinations of means; simple and partial variances; simple, partial and multiple correlation coefficients. Also discussed are the posterior distributions of the canonical correlations and of the principal components. For the general multivariate linear hypothesis, it is demonstrated that the joint Bayesian posterior region for the elements of the regression matrix is equivalent to the usual confidence region for these parameters. The joint predictive density of a set of future observations generated by the linear hypothesis is obtained thus enabling one to specify the probability that a set of future observations will be contained in a particular region. (Author)

A local limit theorem for nonlattice multi-dimensional distribution functions'

Abstract: stants in R and Rd respectively such that (1) limno-oFn(Bn(x + An)) = F(x), x Rd. Let f and f, denote the characteristic functions of F and F, respectively. We

Maximum Likelihood Estimation for Distributions with Monotone Failure Rate

Abstract: : Using the idea of maximum likelihood, we derive an estimator for a distribution function possessing an increasing (decreasing) failure rate and also obtain corresponding estimators for the density and the failure rate. We show that these estimators are consistent.

On the Convergence of Moments in the Central Limit Theorem

Abstract: Let $X_1, X_2, \cdots, X_n$ be a sequence of independent random variables (r.v.'s) with zero mean and finite standard deviation $\sigma_i, 1 \leqq i \leqq n$. According to the central limit theorem, the normed sum $Y_n = (1/s_n) \sum^n_{i=1} X_i,$ where $s_n = \sum^n_{i=1} \sigma^2_i$, is under certain additional conditions approximatively normally distributed. We will here examine the convergence of the moments and the absolute moments of $Y_n$ towards the corresponding moments of the normal distribution. The results in this general case are stated in Theorem 3 and Theorem 4, but, in order to avoid repetition and unnecessary complication, explicit proofs will only be given in the case of equally distributed random variables. (Theorem 1 and Theorem 2).

Nonlinear least squares estimation

Abstract: We are given a set of $N$ responses $Y_t$ which have arisen from a nonlinear regression model \begin{equation*}\tag{(1.1)}Y_t = f(x_t, \theta) + e_t; \quad t = 1, 2, \cdots, N.\end{equation*} Here $x_t$ denotes the $t$th fixed input vector of $k$ elements giving rise to $Y_t$, whilst $\theta$ is an $m$-element unknown parameter vector with elements $\theta_i$ and the $e_t$ are a set of $N$ independent error residuals from $N(0, \sigma^2)$ with $\sigma^2$ unknown. The expectations of the $Y_t$, are therefore the functions $f(x_t, \theta)$ which will be assumed to satisfy certain regularity conditions. The problem is to estimate $\theta$ notably by least squares. In this paper we shall develop an iterative method of solution of the least squares equations which has the following properties: (a) the computational procedure is convergent for finite $N$; (b) the resulting estimators are asymptotically $100{\tt\#}$ efficient as $N \rightarrow \infty$. In Sections 2-4 we give a survey of our results leaving the mathematical proofs to Sections 5-7 whilst in Section 8 we illustrate our method with an example. Although our theoretical development is oriented towards our specific goals certain results are proved in a somewhat more general form. Some of our theory will be seen to correspond to well known theorems on stochastic limits which have to be reproved because of certain modifications which we require.

Moments of Randomly Stopped Sums

Abstract: 1. Introduction. Let (Ω, F, P) be a probability space, let x 1 x 2, … be a sequence of random variables on Ω, and let F n be the σ-algebra generated by x 1, …, x n with F 0 = (φ,Ω). A stopping variable (of the sequence x 1, x2, …) is a random variable t on Ω with positive integer values such that the event [t = n] e F n for every n ≧ l.Let \( {S_n} = \Sigma _i^n{x_i};{\text{ then }}{S_t} = {S_{t(\omega )}}(\omega ) = \Sigma _1^t{x_i} \) is a randomly stopped sum. We shall always assume that $$E|{x_n}| < \infty , E({x_{n + 1}}|{\Im _n}) = 0, (n \geqslant 1).$$ (1)

Some Aspects of the Random Sequence

Abstract: This is primarily a review of combinatorial problems connected with ballot problems, runs, records, and amalgamation, but numerous new results and applications occur throughout the paper. The early history of the classical ballot problem is clarified, and many recent generalizations and applications are noted. For runs and records, the main emphasis is on the derivation of the null-hypothesis distributions, with only passing reference to asymptotics and non-null distributions. An appendix lists recent work on the Kolmogorov and Smirnov statistics. There are 183 references.

Admissible Bayes Character of $T^2-, R^2-$, and Other Fully Invariant Tests for Classical Multivariate Normal Problems

Abstract: In a variety of standard multivariate normal testing problems, it is shown that certain procedures, often fully invariant, similar, and/or likelihood ratio, are admissible Bayes procedures. The problems include the multivariate general linear hypothesis (where some of the procedures considered were previously shown to be admissible by other methods), the testing of independence of sets of variates (where the likelihood ratio test is shown, for the first time, to be admissible), tests about only some components of the means, classification procedures (for any number of populations), Behrens-Fisher problem, tests about values of or proportionality or equality of covariance matrices, etc. A general technique is developed for obtaining certain Bayes procedures for such problems from the corresponding Bayes procedures relative to a priori distributions of a certain type for problems where nuisance parameter means have been deleted.