Simultaneous test procedures-some theory of multiple comparisons'

TLDR: The Simultaneous Test Procedure (STP) as discussed by the authors is a generalization of the notion of multiple comparisons for means, which was originally proposed by Aitchison and Roy.

Abstract: 1. Introduction and summary. When a hypothesis is tested by a significanice test and is not rejected, it is generally agreed that all hypotheses implied by that hypothesis (its "components") must also be considered as non-rejected. However, when the hypothesis is rejected the question remains as to which components may also be rejected. Various writers have given attention to this question and have proposed a variety of multiple comparisons methods based either on tests of each one of the components or on simultaneous confidence bounds on parametric functions related to the various hypotheses. An approach to such methods, apparently originally due to Tukey [27], is to test each component hypothesis by comparing its statistic with the a level critical value of the statistic for the overall hypothesis. This is called a Simultaneous Test Procedure (STP for short) as all hypotheses may be tested simultaneously and without reference to one another. An STP involves no stepwise testing of the kind employed by some other methods of multiple comparisons for means, in which subsets are tested for equality only if they are contained in sets which have already been found significant. (See 13], [4], [10], [18]). A general formalization of STP's is attempted in this paper. Section 2 introduces the requisite concepts of families of hypotheses and the implication relations between them, as well as the monotonicity relations between the related statistics. Section 3 defines STP's and shows conditions for coherence and consonance of their decisions, these properties being that hypothesis implication relations are preserved in the decisions of the STP. Section 4 discusses comparison of various STP's for the same hypotheses and shows the advantages of the unionintersection type of statistics and of reducing the family of hypotheses tested as much as possible. Section 5 translates all these results to simultaneous confidence statements after introducing the definitions necessary to allow such translation. The analogy between simultaneous test and confidence methods is of special importance as it brings a wide spectrum of methods within this framework, most of which was originally formulated in confidence region terms. This covers the original work of Tukey [27] and Scheff6 [25] and continues with that of Roy and his associates [21] and most recently Krishnaiah [12], [13]. A general discussion of this confidence approach has been given by Aitchison [1) since the first draft of the present paper. In view of the close analogies pointed out in Section 5, it is