Showing papers in "Annals of Mathematical Statistics in 1951"

A Stochastic Approximation Method

Abstract: Let M(x) denote the expected value at level x of the response to a certain experiment. M(x) is assumed to be a monotone function of x but is unknown to the experimenter, and it is desired to find the solution x = θ of the equation M(x) = α, where a is a given constant. We give a method for making successive experiments at levels x1, x2, ··· in such a way that xn will tend to θ in probability.

Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal Distributions

Abstract: In this paper linear restrictions on regression coefficients are studied. Let the $p \times q_2$ matrix of coefficients of regression of the $p$ dependent variates on $q_2$ of the independent variates be $\mathbf{\bar B}_2$. Maximum likelihood estimates of an $m \times p$ matrix $\Gamma$ satisfying $\Gamma'\mathbf{\bar B}_2 = 0$ and certain other conditions are found under the assumption that the rank of $\mathbf{\bar B}_2$ is $p - m$ and the dependent variates are normally distributed (Section 2). Confidence regions for $\Gamma$ under various conditions are obtained (Section 5). The likelihood ratio test of the hypothesis that the rank of $\mathbf{\bar B}_2$ is a given number is obtained (Section 3). A test of the hypothesis that $\Gamma$ is a certain matrix is given (Section 4). These results are applied to the "$q$-sample problem" (Section 7) and are extended for certain econometric models (Section 6).

The Asymptotic Distribution of the Range of Sums of Independent Random Variables

Abstract: The asymptotic distribution of the range and normalized range of the sum of $n$ independent variables is derived using the theory of Brownian motion.

Consistency and Unbiasedness of Certain Nonparametric Tests

Abstract: It is shown that there exist strictly unbiased and consistent tests for the univariate and multivariate two- and fc-sample problem, for the hypothesis of independence, and for the hypothesis of symmetry with respect to a given point. Certain new tests for the univariate two-sample problem are discussed. The large sample power of these tests and of the Mann-Whitney test are obtained by means of a theorem of Hoeffding. There’is a discussion of the problem of tied observations.

Ratios Involving Extreme Values

Abstract: Ratios of the form $(x_n - x_{n-j})/(x_n - x_i)$ for small values of $i$ and $j$ and $n = 3, \cdots, 30$ are discussed. The variables concerned are order statistics, i.e., sample values such that $x_1 < x_2 < \cdots < x_n$. Analytic results are obtained for the distributions of these ratios for several small values of $n$ and percentage values are tabled for these distributions for samples of size $n \leqq 30$.

Minimum Variance Estimation Without Regularity Assumptions

Abstract: Following the essential steps of the proof of the Cramer-Rao inequality [1,2] but avoiding the need to transform coordinates or to differentiate under integral signs, a lower bound for the variance of estimators is obtained which is (a) free from regularity assumptions and (b) at least equal to and in some cases greater than that given by the Cramer-Rao inequality. The inequality of this paper might also be obtained from Barankin’s general result [3]. Only the simplest case—that of unbiased estimation of a single real parameter—is considered here but the same idea can be applied to more general problems of estimation.

A Combinatorial Central Limit Theorem

Abstract: Let (Y n1,…,Y nn be a random vector which takes on the n! permutations of (1,…, n) with equal probabilities. Let c n(i,j), i,j = 1, …, n, be n real numbers. Sufficient conditions for the asymptotic normality of $$ S_n = \sum\limits_{i - 1}^n {c_n \left( {i,Y_{ni} } \right)} $$ are given (Theorem 3). For the special case c n(i,j) = a n(i)b n(j) a stronger version of a theorem of Wald, Wolfowitz and Noether is obtained (Theorem 4). A condition of Noether is simplified (Theorem 1).

One-Sided Confidence Contours for Probability Distribution Functions

Abstract: Let $F(x)$ be the continuous distribution function of a random variable $X,$ and $F_n(x)$ the empirical distribution function determined by a sample $X_1, X_2, \cdots, X_n$. It is well known that the probability $P_n(\epsilon)$ of $F(x)$ being everywhere majorized by $F_n(x) + \epsilon$ is independent of $F(x)$. The present paper contains the derivation of an explicit expression for $P_n(\epsilon)$, and a tabulation of the 10%, 5%, 1%, and 0.1% points of $P_n(\epsilon)$ for $n =$ 5, 8, 10, 20, 40, 50. For $n =$ 50 these values agree closely with those obtained from an asymptotic expression due to N. Smirnov.

On minimax statistical decision procedures and their admissibility

Abstract: This paper is concerned with the problem of making a decision on the basis of a sequence of observations on a random variable. Two loss functions, each depending on the distribution of the random variable, the number of observations taken, and the decision made, are assumed given. Minimax problems can be stated for weighted sums of the two loss functions, or for either one subject to an upper bound on the expectation of the other. Under suitable conditions it is shown that solutions of the first type of problem provide solutions for all problems of the latter types, and that admissibility for a problem of the first type implies admissibility for problems of the latter types. Two examples are given: Estimation of the mean of a random variable which is (1) normal with known variance, (2) rectangular with known range. The resulting minimax estimates are, with a small class of exceptions, proved admissible among the class of all procedures with continuous risk functions. The two loss functions are in each case the number of observations, and an arbitrary nondecreasing function of the absolute error of estimate. Extensions to a function of the number of observations for the first loss function are indicated, and two examples are given for the normal case where the sample size can or must be randomised among more than a consecutive pair of integers.

Nonparametric Estimation IV

Abstract: In the three papers, [1], [2], [3], entitled "Nonparametric estimation", Scheffe and Tukey generalized previous results on tolerance regions and extended them to cover all continuous and discontinuous distribution functions. This note contains four comments arising from these papers: first, on a method for giving bounds to the confidence level in the discontinuous case which can lower the probability that the end points need to have part, a random variable, of their probability neglected to maintain the given confidence level; second, on a correction of a statement of results in [2]; third, on a proof in [2] requiring a further statement; fourth, a necessary restatement of theorems in [3].

Elimination of Randomization in Certain Statistical Decision Procedures and Zero-Sum Two-Person Games

Abstract: The general existence of minimax strategies and other important properties proved in the theory of statistical decision functions (e.g., [3]) and the theory of games (e.g., [5]) depends upon the convexity of the space of decision functions and the convexity of the space of strategies. This convexity can be obtained by the use of randomized decision functions and mixed (randomized) strategies. In Section 2 of the present paper the authors state the extension (first announced in [1]) of a measure theoretical result known as Lyapunov's theorem [2]. This result is applied in Section 3 to the statistical decision problem where the number of distributions and decisions is finite. It is proved that when the distributions are continuous (more generally, "atomless," see footnote 7 below) randomization is unnecessary in the sense that every randomized decision function can be replaced by an equivalent nonrandomized decision function. Section 4 extends this result to the case when the decision space is compact. Section 5 extends the results of Section 3 to the sequential case. Sections 6 and 7 show, by counterexamples, that the results of Section 3 cannot be extended to the case of infinitely many distributions without new restrictions. Section 8 gives sufficient conditions for the elimination of randomization under maintenance of $\epsilon$-equivalence. Section 9 concludes with a restatement of the results in the language of the theory of games.

The Distribution of the Maximum Deviation Between two Sample Cumulative Step Functions

Abstract: Let $x_1 < x_2 M \cdots < x_n$ and $y_1 < y_2 < \cdot < y_m$ be the ordered results of two random samples from populations having continuous cumulative distribution functions $F(x)$ and $G(x)$ respectively. Let $S_n(x) = K/n$ when $k$ is the number of observed values of $X$ which are less than or equal to $x$, and similarly let $S'_m(y) = j/m$ where $j$ is the number of observed values of $Y$ which are less than or equal to $y$. The statistic $d = \max | S_n(x) - S'_m(x) |$ can be used to test the hypothesis $F(x) \equiv G(x)$, where the hypothesis would be rejected if the observed $d$ is significantly large. The limiting distribution of $d \sqrt{mn}{m + n}$ has been derived [1] and [4], and tabled [5]. In this paper a method of obtaining the exact distribution of $d$ for small samples is described, and a short table for equal size samples is included. The general technique is that used by the author for the single sample case [2]. There is a lower bound to the power of the test against any specified alternative, [3]. This lower bound approaches one as $n$ and $m$ approach infinity proving that the test is consistent.

On the Fundamental Lemma of Neyman and Pearson

Abstract: The following lemma proved by Neyman and Pearson [1] is basic in the theory of testing statistical hypotheses: LEMMA. Let $f_1(x), \cdots, f_{m+1}(x)$ be $m + 1$ Borel measurable functions defined over a finite dimensional Euclidean space $R$ such that $\int_R |f_i(x)|dx < \infty (i = 1, \cdots, m + 1)$. Let, furthermore, $c_1, \cdots, c_m$ be $m$ given constants and $\mathcal{S}$ the class of all Borel measurable subsets $S$ of $R$ for which (1.1) $\int_S f_i(x) dx = c_i \\ (i = 1, \cdots, m)$. Let, finally, $\mathcal{S}_0$ be the subclass of $\mathcal{S}$ consisting of all members $\mathcal{S}_0$ of $\mathcal{S}$ for which (1.2) $\int_{S_0} f_{m + 1}(x) dx \geqq \int_S f_{m+1}(x) dx \text{for all S in} \mathcal{S}$. If $S$ is a member of $\mathcal{S}$ and if there exist $m$ constants $k_1, \cdots, k_m$ such that (1.3) $f_{m + 1}(x) \geqq k_1f_1(x) + \cdots + k_mf_m(x) \text{when} x \epsilon S$, (1.4) $f_{m + 1}(x) \leqq k_1f_1(x) + \cdots + k_mf_m(x) \text{when} x ot\epsilon S$, then $S$ is a member of $\mathcal{S}_0$. The above lemma gives merely a sufficient condition for a member $S$ of $\mathcal{S}$ to be also a member of $\mathcal{S}_0$. Two important questions were left open by Neyman and Pearson: (1) the question of existence, that is, the question whether $\mathcal{S}_0$ is non-empty whenever $\mathcal{S}$ is non-empty; (2) the question of necessity of their sufficient condition (apart from the obvious weakening that (1.3) and (1.4) may be violated on a set of measure zero). The purpose of the present note is to answer the above two questions. It will be shown in Section 2 that $\mathcal{S}_0$ is not empty whenever $\mathcal{S}$ is not empty. In Section 3, a necessary and sufficient condition is given for a member of $\mathcal{S}$ to be also a member of $\mathcal{S}_0$. This necessary and sufficient condition coincides with the Neyman-Pearson sufficient condition under a mild restriction.

Extremal properties of extreme value distributions

Abstract: The upper and lower bounds for the expectation, the coefficient of variation, and the variance of the largest member of a sample from a symmetric population are discussed. The upper bound for the expectation (Table 1, Fig. 1), the lower bound for the C.V. (Table 2, Fig. 4) and the lower bound for the variance (Fig. 7) are actually achieved for the corresponding particular population distributions (Figs. 2, 3, 5, 6, equation (5.1)). The rest of the bounds are not actually achieved but approached as the limits, for example, for the three-point distribution (Section 3) by letting $p$ tend to zero.

On the Distribution of Wald's Classification Statistic

Abstract: In this paper we shall consider the exact distribution of Wald's classification statistic $V$ in the univariate case, some theoretical approximations in various multivariate cases, and an empirical distribution in a particular multivariate case. We shall also draw some conclusions as to the potential usefulness of the statistic $V$ and the work which remains to be done.

Exact Tests of Serial Correlation using Noncircular Statistics

Abstract: For testing the hypothesis that successive members of a series of observations are independent J. von Neumann [5] (see also B. I. Hart [4]) and R. L. Anderson [1] have proposed test statistics and tabulated their significance points. von Neumann's criterion seems well designed to detect deviations from the null hypothesis which might be encountered in practice but its exact distribution is unknown. On the other hand Anderson's statistic, while it has a known distribution, is based on a circular conception of the population which is rarely plausible in practice. In the present note certain noncircular statistics are proposed for which exact distributions can be obtained from Anderson's results. The statistics are derived from the usual noncircular statistics by sacrificing a small amount of relevant information. Their application is noted to certain regression problems for which no satisfactory tests are at present available. Finally, some general remarks are made about the choice of best statistics for the problems discussed.

On Certain Methods of Estimating the Linear Structural Relation

Abstract: The first part of this paper considers two methods of estimating the linear structural relation between two variables both of which are subject to "error"; the second part of the paper comments on a recently advanced procedure for constructing the confidence region for the slope of the structural relation. In 1940 Wald [1] initiated a certain procedure for estimating the linear structural relation between two variables both of which are subject to "error." Wald's idea was extended by Nair and Banerjee [2] and later by Bartlett [3]. These procedures require some knowledge about the values of certain non-observable variables. When this knowledge is not available there is a temptation to substitute information derived from observations. One such method was considered by Wald who found sufficient conditions for the consistency of the resulting estimate. The purpose of the first part of the present paper is to find the necessary and sufficient conditions for two procedures with reference to a slightly more general case, namely, when the "errors" in the two observable variables may be correlated. The results obtained indicate that the two procedures, applied in the case of no additional knowledge about the values of the non-observable variables, will lead to consistent estimates of the slope of the structural relation in very exceptional cases only. In 1949 Hemelrijk [4] described a novel procedure for constructing the confidence region of the slope of the linear structural relation in the case when the non-observable variables have unknown fixed values and the observations are made with "error" which has the same probability distribution at each point. The present paper considers this same procedure when there is no information about the fixed non-observable variables and also when these variables are random variables, and shows that the probability that the confidence region covers the true slope is the same as before but that the probability of covering any other slope is now the same as this probability of covering the true slope.

Designs for two-way elimination of heterogeneity

Abstract: Sometimes in a design the position within the block is important as a source of variation, and the experiment gains in efficiency by eliminating the positional effect. The classical example is due to Youden in his studies on the tobacco mosaic virus [1]. He found that the response to treatments also depends on the position of the leaf on the plant. If the number of leaves is sufficient so that every treatment can be applied to one leaf of a tree, then we get an ordinary Latin square, in which the trees are columns and the leaves belonging to the same position constitute the rows. But if the number of treatments is larger than the number of leaf positions available, then we must have incomplete columns. Youden used a design in which the columns constituted a balanced incomplete block design, whereas the rows were complete. These designs are known as Youden's squares, and can be used when two-way elimination of heterogeneity is desired. In Fisher and Yates statistical tables [2] balanced incomplete block designs in which the number of blocks $b$ is equal to the number of treatments $v$ have been used to obtain Youden's squares, and the authors state that "in all cases of practical importance" it has been found possible to convert balanced incomplete blocks of the above kind to a Youden's square by so ordering the varieties in the blocks that each variety occurs once in each position. F. W. Levi noted ([3], p. 6) that this reordering can always be done, in virtue of a theorem given by Konig [4] which states that an even regular graph of degree $m$ is the product of $m$ regular graphs of degree 1. Smith and Hartley [5] give a practical procedure for converting balanced incomplete blocks with $b = v$ into Youden's squares. In this paper I have considered some general classes of designs for two-way elimination of heterogeneity. In Section 3 balanced incomplete block designs for which $b = mv$ have been used to obtain two-way designs in which each treatment occurs in a given position $m$ times. The case $m = 1$ gives Youden's squares. In Section 4 it has been shown that balanced incomplete block designs for which $b$ is not an integral multiple of $v$ can be used to obtain designs for two-way elimination of heterogeneity in which there are two accuracies (i.e., some pairs of treatments are compared with one accuracy, while other pairs are compared with a different accuracy) as in the case of lattice designs for one-way elimination of heterogeneity. In Sections 5 and 6 partially balanced designs have been used to obtain two-way designs with two accuracies. In every case the method of analysis and tables of actual designs have been given.

On the Theory of Unbiased Tests of Simple Statistical Hypotheses Specifying the Values of Two or More Parameters

Abstract: Unbiased critical regions of type D for testing simple hypotheses specifying the values of several parameters are defined and their properties studied. These regions constitute a natural generalization of the Neyman-Pearson regions of type A for testing simple hypotheses specifying the value of one parameter. A theorem is obtained which plays the role of the Neyman-Pearson fundamental lemma in the type A case. Illustrative examples of type D regions are given.

A Bivariate Extension of the $U$ Statistic

Abstract: Let $x, y$, and $z$ be three random variables with continuous cumulative distribution functions $f, g$, and $h$. In order to test the hypothesis $f = g = h$ under certain alternatives two statistics $U, V$ based on ranks are proposed. Recurrence relations are given for determining the probability of a given $(U, V)$ in a sample of $l x$'s, $m y$'s, $n z$'s and the different moments of the joint distribution of $U$ and $V$. The means, second, and fourth moments of the joint distribution are given explicitly and the limit distribution is shown to be normal. As an illustration the joint distribution of $U, V$ is given for the case $(l, m, n) = (6, 3, 3)$ together with the values obtained by using the bivariate normal approximation. Tables of the joint cumulative distribution of $U, V$ have been prepared for all cases where $l + m + n \leqq 15$.

Relations Between Moments of Order Statistics

Abstract: Moments of order statistics multiplied by appropriate factors are called normalized moments. These normalized moments are shown to be successive differences of normalized moments of largest order statistics.

Sequentially Determined Statistically Equivalent Blocks

Abstract: In 1943 Wald [2] gave a method for constructing tolerance regions in the multivariate case. Tukey generalized Wald's procedure in [4] and the results were interpreted for discontinuous distributions in [5] and [6]. This paper presents a further generalization of the method so that statistically equivalent blocks can be determined sequentially; the particular function used to cut off a block may depend on the shape or structure of previously selected blocks. The results are also interpreted for the case of discontinuous distributions. Possible advantages of applying the method are discussed.

Normal Regression Theory in the Presence of Intra-Class Correlation

Abstract: In this paper we prove that certain estimators and tests of significance used in regression analysis when observations are independent are equally valid in the presence of intra-class correlation. An application of this result is presented for the situation in which several replications of the correlated set of observations are available. As a special case of this application, it is shown that the usual test of ``column effects'' in the analysis of variance for a two-way classification remains valid when rows are independent and columns are uniformly correlated. This latter fact is also pointed out in [3].