Showing papers in "Annals of Statistics in 1974"

Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems

Abstract: process. This paper extends Ferguson's result to cases where the random measure is a mixing distribution for a parameter which determines the distribution from which observations are made. The conditional distribution of the random measure, given the observations, is no longer that of a simple Dirichlet process, but can be described as being a mixture of Dirichlet processes. This paper gives a formal definition for these mixtures and develops several theorems about their properties, the most important of which is a closure property for such mixtures. Formulas for computing the conditional distribution are derived and applications to problems in bio-assay, discrimination, regression, and mixing distributions are given.

Prior Distributions on Spaces of Probability Measures

Abstract: Methods of generating prior distributions on spaces of probability measures for use in Bayesian nonparametric inference are reviewed with special emphasis on the Dirichlet processes, the tailfree processes, and processes neutral to the right. Some applications are given.

General Equivalence Theory for Optimum Designs (Approximate Theory)

Abstract: For general optimality criteria $\Phi$, criteria equivalent to $\Phi$-optimality are obtained under various conditions on $\Phi$. Such equivalent criteria are useful for analytic or machine computation of $\Phi$-optimum designs. The theory includes that previously developed in the case of $D$-optimality (Kiefer-Wolfowitz) and $L$-optimality (Karlin-Studden-Fedorov), as well as $E$-optimality and criteria arising in response surface fitting and minimax extrapolation. Multiresponse settings and models with variable covariance and cost structure are included. Methods for verifying the conditions required on $\Phi$, and for computing the equivalent criteria, are illustrated.

A Large Sample Study of the Life Table and Product Limit Estimates Under Random Censorship

Abstract: Using the model of random censorship, a necessary and sufficient condition for the consistency of the standard (actuarial) life table estimate of a survival distribution is derived. We establish the asymptotic normality of this estimate, showing that Greenwood's variance formula is nearly correct. In the case of a continuous survival distribution we establish limiting normality for the product limit estimate and for the closely related cumulative hazard process. Some applications of these results are outlined.

Consistent Autoregressive Spectral Estimates

Abstract: We consider an autoregressive linear process $\{x_t\}$, a one-sided moving average, with summable coefficients, of independent identically distributed variables $\{e_t\}$ with zero mean and fourth moment, such that $\{e_t\}$ is expressible in terms of past values of $\{x_t\}$. The spectral density of $\{x_t\}$ is assumed bounded and bounded away from zero. Using data $x_1,\cdots, x_n$ from the process, we fit an autoregression of order $k$, where $k^3/n \rightarrow 0$ as $n \rightarrow \infty$. Assuming the order $k$ is asymptotically sufficient to overcome bias, the autoregression yields a consistent estimator of the spectral density of $\{x_t\}$. Furthermore, assuming $k$ goes to infinity so that the bias from using a finite autoregression vanishes at a sufficient rate, the autoregressive spectral estimates are asymptotically normal, uncorrelated at different fixed frequencies. The asymptotic variance is the same as for spectral estimates based on a truncated periodogram.

An Antipodally Symmetric Distribution on the Sphere

Abstract: The distribution $\Psi(\mathbf{x}; Z, M) = \operatorname{const}. \exp(\mathrm{tr} (ZM^T \mathbf{xx}^T M))$ on the unit sphere in three-space is discussed. It is parametrized by the diagonal shape and concentration matrix $Z$ and the orthogonal orientation matrix $M. \Psi$ is applicable in the statistical analysis of measurements of random undirected axes. Exact and asymptotic sampling distributions are derived. Maximum likelihood estimators for $Z$ and $M$ are found and their asymptotic properties elucidated. Inference procedures, including tests for isotropy and circular symmetry, are proposed. The application of $\Psi$ is illustrated by a numerical example.

Empirical Probability Plots and Statistical Inference for Nonlinear Models in the Two-Sample Case

Abstract: Let $X$ and $Y$ be two random variables with continuous distribution functions $F$ and $G$ and means $\mu$ and $\xi$. In a linear model, the crucial property of the contrast $\Delta = \xi - \mu$ is that $X + \Delta =_\mathscr{L} Y$, where $= _\mathscr{L}$ denotes equality in law. When the linear model does not hold, there is no real number $\Delta$ such that $X + \Delta = _\mathscr{L} Y$. However, it is shown that if parameters are allowed to be function valued, there is essentially only one function $\Delta(\bullet)$ such that $X + \Delta(X) = _\mathscr{L} Y$, and this function can be defined by $\Delta(x) = G^{-1}(F(x)) - x$. The estimate $\hat{\Delta}_N(x) = G_n^{-1}(F_m(x)) - x$ of $\Delta(x)$ is considered, where $G_n$ and $F_m$ are the empirical distribution functions. Confidence bands based on this estimate are given and the asymptotic distribution of $\hat{\Delta}_N(\bullet)$ is derived. For general models in analysis of variance, contrasts that can be expressed as sums of differences of means can be replaced by sums of functions of the above kind.

Probability Inequalities for the Sum in Sampling without Replacement

Abstract: Upper bounds are established for the probability that, in sampling without replacement from a finite population, the sample sum exceeds its expected value by a specified amount. These are obtained as corollaries of two main results. Firstly, a useful upper bound is derived for the moment generating function of the sum, leading to an exponential probability inequality and related moment inequalities. Secondly, maximal inequalities are obtained, extending Kolmogorov's inequality and the Hajek-Renyi inequality. Compared to sampling with replacement, the results incorporate sharpenings reflecting the influence of the sampling fraction, $n/N$, where $n$ denotes the sample size and $N$ the population size. We go somewhat beyond previous work by Hoeffding (1963) and Sen (1970). As in the latter reference, martingale techniques are exploited. Applications to simple linear rank statistics are noted, dealing with the two-sample Wilcoxon statistic as an example. Finally, the question of sharpness of the exponential bounds is considered.

On Large-Sample Estimation for the Mean of a Stationary Random Sequence

Abstract: For a wide class of stationary random sequences possessing a spectral density function, the variance of the best linear unbiased estimator for the mean is seen to depend asymptotically only on the behavior of the spectral density near the origin. Asymptotically efficient estimators based only on this behavior may be chosen. For spectral densities behaving like $\lambda^ u$ at the origin, $ u > -1$ a constant, the minimum variance decreases like $n^{- u-1}$, where $n$ is the sample size. Asymptotically efficient estimators depending on $ u$ are given. Finally, the consequences of over- or under-estimating the value of $ u$ in choosing an estimator are considered.

Improving on Equivariant Estimators

Abstract: Techniques for improving on equivariant estimators are described. They may be applied, although without assurance of success, whatever be the family of underlying distributions. The loss function is required to satisfy an intuitively reasonable condition but is otherwise arbitrary. One of these techniques amounts to a sample space, orbit-by-orbit analysis of the conditional expected loss given the orbit. It yields, when successful, a "testimator". A second technique obtains the limit of a certain sequence of "testimator-like" estimators. The result is "smoother" than a testimator and often identical to a generalized Bayes estimator over much of its domain. Applications are presented. In the first we extend results of Stein (1964) and obtain a minimax estimator which is generalized Bayes, and in a univariate subcase, admissible within the class of scale-equivariant estimators. In the second, we extend a result of Srivastava and Bancroft (1967).

Linear Functions of Order Statistics with Smooth Weight Functions

Abstract: This paper considers linear functions of order statistics of the form $S_n = n^{-1} \sum J(i/(n + 1))X_{(i)}$. The main results are that $S_n$ is asymptotically normal if the second moment of the population is finite and $J$ is bounded and continuous a.e. $F^{-1}$, and that this first result continues to hold even if the unordered observations are not identically distributed. The moment condition can be discarded if $J$ trims the extremes. In addition, asymptotic formulas for the mean and variance of $S_n$ are given for both the identically and non-identically distributed cases. All of the theorems of this paper apply to discrete populations, continuous populations, and grouped data, and the conditions on $J$ are easily checked (and are satisfied by most robust statistics of the form $S_n$). Finally, a number of applications are given, including the trimmed mean and Gini's mean difference, and an example is presented which shows that $S_n$ may not be asymptotically normal if $J$ is discontinuous.

Characterization of the Partial Autocorrelation Function

Abstract: The conditions $|\phi_k| \leqq 1$ for all $k = 1,2, \cdots$ and $|\phi_k| = 1$ implies $\phi_{k+1} = \phi_k$ are both necessary and sufficient for a sequence of real numbers $\{\phi_k; k = 1,2, \cdots\}$ to be the partial autocorrelation function for a real, discrete parameter, stationary time series. If all partial autocorrelations beyond the $p$th are zero, the series is an autoregression. If all beyond the $p$th have magnitude unity, the series satisfies a homogeneous stochastic difference equation. A stationary series is singular if and only if $\sum^N_1 \phi_k^2$ diverges with $N$. The likelihood function for the partial autocorrelation function is produced, assuming normality.

Matrix Derivatives with an Application to an Adaptive Linear Decision Problem

Abstract: A theory of matrix differentiation is presented which uses the concept of a matrix of derivative operators. This theory allows matrix techniques to be used in both the derivation and the description of results. Several new operations and identities are presented which facilitate the process of matrix differentiation. The derivative theorems and new operations are then applied to the problem of determining optimal policies in a linear decision model with unknown coefficients, a problem which would be cumbersome if not impossible to solve without these theorems and operations.

Convergence of Sample Paths of Normalized Sums of Induced Order Statistics

Abstract: The main result in this paper concerns the limiting behavior of normalized cumulative sums of induced order statistics obtained from $n$ independent two-dimensional random vectors, as $n$ increases indefinitely. By means of a Skorokhod-type embedding of these cumulative sums on Brownian Motion paths, it is shown that under certain conditions the sample paths of these normalized sums converge in a certain sense to a process obtained from the Brownian Motion by a transformation of the time-axis. This yields an invariance principle similar to Donsker's. In particular, the asymptotic distribution of the supremum of the absolute values of these normalized cumulative sums is obtained from a well-known result for the Brownian Motion. Large sample tests of a specifieds regression function are obtained from these results.

Estimation of Models of Autoregressive Signal Plus White Noise

Abstract: If $x(\bullet)$ is a time series which may be written as $x(t) = s(t) + n(t)$ where $t$ is an integer, $s(\bullet)$ an autoregressive signal of order $q$ and $n(\bullet)$ white noise, then the model has $q + 2$ parameters. These are (i) the $q$ autoregressive parameters (ii) the residual variance of the autoregressive scheme and (iii) the variance of the white noise. A method is proposed to estimate the $q + 2$ parameters. This method is based on analogies with regression theory and in the case of a normal series yields strongly consistent efficient estimators.

An Unbalanced Jackknife

Abstract: It is proved that the jackknife estimate $\tilde{\theta} = n\hat{\theta} - (n - 1)(\sum \hat{\theta}_{-i}/n)$ of a function $\theta = f(\beta)$ of the regression parameters in a general linear model $\mathbf{Y} = \mathbf{X\beta} + \mathbf{e}$ is asymptotically normally distributed under conditions that do not require $\mathbf{e}$ to be normally distributed. The jackknife is applied by deleting in succession each row of the $\mathbf{X}$ matrix and $\mathbf{Y}$ vector in order to compute $\hat{\mathbf{\beta}}_{-i}$, which is the least squares estimate with the $i$th row deleted, and $\hat{\theta}_{-i} = f(\hat\mathbf{\beta}_{-i})$. The standard error of the pseudo-values $\tilde{\theta}_i = n\hat{\theta} - (n - 1)\hat{\theta}_{-i}$ is a consistent estimate of the asymptotic standard deviation of $\tilde{\theta}$ under similar conditions. Generalizations and applications are discussed.

Asymptotically Efficient Adaptive Rank Estimates in Location Models

Abstract: This paper describes a new construction of uniformly asymptotically efficient rank estimates in the one and two-sample location models. The method adopted differs from van Eeden's (1970) earlier construction in three respects. First, the whole sample, rather than a vanishingly small fraction of the sample, is used in estimating the efficient score function. Secondly, a Fourier series estimator is used for the score function rather than a window estimator. Thirdly, the linearized rank estimates corresponding to the estimated score function provide the uniformly asymptotically efficient location estimates. These estimates are asymptotically efficient over a larger class of distributions than the van Eeden estimates and should approach their asymptotic behavior more rapidly.

Edgeworth Expansions in Nonparametric Statistics

Abstract: This is a survey of recent work on Edgeworth expansions for $(M)$ estimates, rank tests and some other statistics arising in nonparametric models. A Berry-Esseen theorem for $U$-statistics which seems to be new is also proved.

Repeated Games with Absorbing States

Abstract: A zero-sum two person game is repeatedly played. Some of the payoffs are "absorbing" in the sense that, once any of them is reached, all future payoffs remain unchanged. Let $v_n$ denote the value of the $n$-times repeated game, and let $v_\infty$ denote the value of the infinitely-repeated game. It is shown that $\lim v_n$ always exists. When the information structure is symmetric, $v_\infty$ also exists and $v_\infty = \lim v_n$.

On the Theory of Connected Designs: Characterization and Optimality

Abstract: Connectedness is an important property which every block design must possess if it is to provide an unbiased estimator for all elementary treatment contrasts under the usual linear additive model. We have classified the family of connected designs into three subclasses: locally connected, globally connected and pseudo-globally connected designs. Basically, a locally connected design is one in which not all the observations participate in the estimation. A globally connected design is one in which all observations participate in the estimation. Finally, a pseudo-globally connected design is a compromise between locally and globally connected designs. Theorems and corollaries are given which characterize the different classes of connected designs. In our discussion on the optimality of connected designs we show that there is much to be gained by partitioning the family of connected designs in the above fashion. Our optimality criteria are $S$-optimality suggested by Shah, which selects the design with minimum trace of the information matrix squared and $(M, S)$-optimality which selects the $S$ optimal design from the class of designs with maximum trace of the information matrix. Using these optimality criteria, we have been able to derive some new results which we hope to be of interest to the users and researchers in the field of optimum design theory. To be specific, let BD $\{v, b, (r_i), (k_u)\}$ denote a block design on a set of $v$ treatments with $b$ blocks of size $k_u, u = 1,2, \cdots, b$ and treatment $i$ is replicated $r_i$ times. Then we have shown that for the family of connected block designs BD $\{v, b, (r_i), k\}$ with (i) less than $k - 1$ treatments having replication equal to one and binary (0, 1) the $S$-optimum design is pseudo-globally connected; (ii) the $S$-optimum design is globally connected if $r_i > 1$ and the designs are binary; and (iii) at least one treatment with replication greater than $b$, then the $(M, S)$-optimum design is pseudo-globally connected. In the final part of this paper we mention some unsolved problems in this area.

Majorization in Multivariate Distributions

Abstract: In case the joint density $f$ of $X = (X_1, \cdots, X_n)$ is Schur-concave (is an order-reversing function for the partial ordering of majorization), it is shown that $P(X \in A + \theta)$ is a Schur-concave function of $\theta$ whenever $A$ has a Schur-concave indicator function. More generally, the convolution of Schur-concave functions is Schur-concave. The condition that $f$ is Schur-concave implies that $X_1, \cdots, X_n$ are exchangeable. With exchangeability, the multivariate normal and certain multivariate "$t$", beta, chi-square, "$F$" and gamma distributions have Schur-concave densities. These facts lead to a number of useful inequalities. In addition, the main result of this paper can also be used to show that various non-central distributions (chi-square, "$t$", "$F$") are Schur-concave in the noncentrality parameter.

Point and Confidence Estimation of a Common Mean and Recovery of Interblock Information

Abstract: Consider the problem of estimating a common mean of two independent normal distributions, each with unknown variances. Note that the problem of recovery of interblock information in balanced incomplete blocks designs is such a problem. Suppose a random sample of size $m$ is drawn from the first population and a random sample of size $n$ is drawn from the second population. We first show that the sample mean of the first population can be improved on (with an unbiased estimator having smaller variance), provided $m \geqq 2$ and $n \geqq 3$. The method of proof is applicable to the recovery of information problem. For that problem, it is shown that interblock information could be used provided $b \geqq 4$. Furthermore for the case $b = t = 3$, or in the common mean problem, where $n = 2$, it is shown that the prescribed estimator does not offer improvement. Some of the results for the common mean problem are extended to the case of $K$ means. Results similar to some of those obtained for point estimation, are also obtained for confidence estimation.

Incomplete Information in Markovian Decision Models

Abstract: If a set of states is given in a problem of dynamic programming in which each state can be observed only partially, the given model is generally transformed into a new model with completely observed states. In this article a method is introduced with which Markov models of dynamic programming can be transformed and which preserves the Markov property. The method applies to relatively general sets of states.

The Expected Sample Size of Some Tests of Power One

Abstract: Asymptotic approximations to the expected sample size are given for a class of tests of power one introduced in [10]. Comparisons are made with the method of mixtures of likelihood ratios, and an application is given to Breiman's gambling theory for favorable games.

Periodic Splines and Spectral Estimation

Abstract: The theory of periodic smoothing splines is presented, with application to the estimation of periodic functions. Several theorems relating the order of the differential operator defining the spline to the saturation (order of bias) of the estimator are proven. The linear operator which maps a function to its periodic continuous smoothing spline approximation is represented as a convolution operator with a given convolution kernel. This operator is shown to be the limit of a sequence of operators which map a function into the periodic version of the usual lattice smoothing spline. The convolution kernel above appears as the kernel in a kernel type estimate of the spectral density. Thus, it is shown that, a smoothing spline spectral density estimate, is also asymptotically a kernel type spectral density estimate. Some numerical results are presented.

Tests for Change of Parameter at Unknown Times and Distributions of Some Related Functionals on Brownian Motion

Abstract: Statistics are derived for testing a sequence of observations from an exponential-type distribution for no change in parameter against possible two-sided alternatives involving parameter changes at unknown points. The test statistic can be chosen to have high power against certain of a variety of alternatives. Conditions on functionals on $C\lbrack 0,1\rbrack$ are given under which one can assert that the large sample distribution of the test statistic under the null-hypothesis or an alternative from a range of interesting hypotheses is that of a functional on Brownian Motion. We compute and tabulate distributions for functionals defined by nonnegative weight functions of the form $\psi(s) = as^k, k > -2$. The functionals for $-1 \geqq k > -2$ are not continuous in the uniform topology on $C\lbrack 0, 1\rbrack$.

Control Charts Based on Weighted Sums

Abstract: In a continuous production process, samples of fixed size are taken at regular intervals of time and a statistic $X_n$ is computed from the $n$th sample, $n = 1,2, \cdots$. In this paper, we study process inspection schemes which stop the production and take corrective action with $N = \text{first} n \geqq 1$ such that $\sum^n_{i=1} c_{n-i} X_i \geqq h$, where $h$ is a preassigned constant and $c_0 \geqq c_1 \geqq \cdots \geqq c_{k-1} > 0 = c_k = c_{k+1} = \cdots$ is a suitably chosen sequence of weights. The average run length of such procedures is examined, and in the normal case, numerical comparisons with the average run length of the usual Shewhart Chart are given. In connection with the normal case, the first passage times of more general Gaussian sequences are studied and an asymptotic theorem is obtained. The first passage time $N$ for more general weighted sums, where the sequence $(c_n)$ is not assumed to be eventually zero but is assumed to be at least square summable, is also considered.

Log-Linear Models for Frequency Tables Derived by Indirect Observation: Maximum Likelihood Equations

Abstract: Frequency tables are examined in which some cells are not distinguishable. Log-linear models are proposed for these tables which lead to likelihood equations closely related to those associated with log-linear models for conventional frequency tables. Just as in conventional tables, the maximum likelihood equations are shown to be the same under Poisson or multinomial sampling. Applications are made to the problem of estimation of gene frequencies from observed phenotype frequencies.

Estimating Equations in the Presence of a Nuisance Parameter

Abstract: Estimating equations for a real parameter $\theta$ which indexes a family of densities $p(x, \theta)$ were considered in the note by Godambe (Ann. Math. Statist. 31 (1960) 1208-1211). An optimality property of the equation $\partial \log p/\partial \theta = 0$ among unbiased estimating equations was established. In this paper an analogous result is proved for estimation of a real parameter $\theta_1$ in the presence of a nuisance parameter $\theta_2$.

Note on Anderson's Sequential Procedures with Triangular Boundary

Abstract: Simple expressions are given for the probability of a correct decision for the Anderson sequential procedure with a triangular boundary, for some values of parameters. This can be used to improve bounds of the probability of correct decision for the Paulson sequential ranking procedures and change slightly the comparison of these with the Bechhofer-Kiefer-Sobel procedures.