scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Appropriate Post Hoc Comparisons for Interaction and Nested Hypotheses in Analysis of Variance Designs: The Elimination of Type IV Errors

01 May 1970-American Educational Research Journal (SAGE Publications)-Vol. 7, Iss: 3, pp 397-421
TL;DR: To err is human, and now that behavioral researchers are engaging in inferential activity with increasing frequency, it is more likely that the number of erroneous inferences is also increasing as mentioned in this paper.
Abstract: To err is human, and now that behavioral researchers are engaging in inferential activity with increasing frequency, it is more than likely that the number of erroneous inferences is also increasing. Familiar to all seasoned hypothesis testers are those twin gremlins, Type I and Type II errors. The first faux pas refers, of course, to the rejection of a true hypothesis, while the second is the non-rejection of a false hypothesis. The rightful originator of Type III errors is disputable, as is the commonly accepted definition of them: "In 1947, F. N. David, perhaps not entirely seriously, suggested that there was a third kind of error which might be committed in
Citations
More filters
BookDOI
15 Sep 1993
TL;DR: In this article, the authors discuss policy institutions and practices, policy discourse and the politics of Washington think tanks, Frank Fischer Discourse coalitions and the institutionalization of practice, Maarten Hajer Political judgement and the policy cycle -the case of ethnicity arguments in the Netherlands, Robert Hoppe Counsel and consensus -norm of argumentation in health policy, Bruce Jennings.
Abstract: Part 1 The argumentative turn: policy institutions and practices: Policy discourse and the politics of Washington think tanks, Frank Fischer Discourse coalitions and the institutionalization of practice - the case of acid rain in Great Britain, Maarten Hajer Political judgement and the policy cycle - the case of ethnicity arguments in the Netherlands, Robert Hoppe Counsel and consensus - norms of argumentation in health policy, Bruce Jennings. Part 2 Analytical concepts - frames, tropes, and narratives: Survey research as rhetorical trope - electric power planning arguments in Chicago, J.A. Throgmorton Frame reflective policy discourse, Martin Rein and Donald Schon Reading policy narratives - beginning, middle, and end, Thomas J. Kaplan Learning from practice stories - the priority of practical judgement, John Forester. Part 3 Theoretical perspectives: Policy anlysis and planning - from science to argumentation, John Dryzek Planning through debate - the communicative turn in planning theory, Patsy Healey Policy reforms as arguments, William Dunn Two worlds of policy discourse - consensual versus adversarial proposal selection, Duncan MacRae.

1,809 citations

Journal ArticleDOI
TL;DR: This article examined the use of data analysis tools by researchers in four research paradigms: between-subjects univariate, multivariate, repeated measures, and covariance designs, concluding that researchers rarely verify that validity assumptions are satisfied and that, accordingly, they typically use analyses that are nonrobust to assumption violations.
Abstract: Articles published in several prominent educational journals were examined to investigate the use of data analysis tools by researchers in four research paradigms: between-subjects univariate designs, between-subjects multivariate designs, repeated measures designs, and covariance designs. In addition to examining specific details pertaining to the research design (e.g., sample size, group size equality/inequality) and methods employed for data analysis, the authors also catalogued whether (a) validity assumptions were examined, (b) effect size indices were reported, (c) sample sizes were selected on the basis of power considerations, and (d) appropriate textbooks and/or articles were cited to communicate the nature of the analyses that were performed. The present analyses imply that researchers rarely verify that validity assumptions are satisfied and that, accordingly, they typically use analyses that are nonrobust to assumption violations. In addition, researchers rarely report effect size statistics, ...

571 citations

Journal ArticleDOI
TL;DR: In this paper, the authors show that the statistical results yielded by this type of analysis can easily be misinterpreted, since the score model underlying the analysis is not correct, leading to incorrect statements regarding treatment effects, completely redundant reanalyses of the same data, and problems with respect to post hoc investigations.
Abstract: The pretest-posttest control group design (or an extension of it) is a highly prestigious experimental design. A popular analytic strategy involves subjecting the data provided, by this design to a repeated measures analysis of variance (ANOVA). Unfortunately, the statistical results yielded by this type of analysis can easily be misinterpreted, since the score model underlying the analysis is not correct. Examples from recently published articles are used to demonstrate that this statistical procedure has led to (a) incorrect statements regarding treatment effects, (b) completely redundant reanalyses of the same data, and (c) problems with respect to post hoc investigations; Two alternative strategies—gain scores and covariance—are discussed and compared.

461 citations

Journal ArticleDOI
TL;DR: This paper found that failure reflected less on ability and shame was correspondingly reduced when students studied little, the same failure condition that subjects, in the role of teachers, punished most severely.
Abstract: Self-worth theory suggests that teachers and students often operate at crosspurposes: Teachers encourage achievement through effort, yet many students attempt to avoid the implication that they lack ability by not trying. To test these assertions undergraduates rated their affective reactions to hypothetical test failures under conditions of high or low effort and in the presence or absence of self-serving excuses. Then, in the role of teachers, they administered punishment to hypothetical students under the same failure conditions. Results indicated that inability attributions and negative affect were greatest when failure followed much effort. Conversely, failure reflected less on ability, and shame was correspondingly reduced when students studied little—the same failure condition that subjects, in the role of teachers, punished most severely. The virtues of hard work have long been extolled in America. Nowhere is this truer than in our schools, where it is widely held among educators and parents alike that while not all students are brilliant, at least everyone can try. The paramount importance of such a work ethic in the teacher's system of values has recently been demonstrated (see Weiner, 1972, 1974). In the typical procedure, teachers are asked to reward and punish a group of hypothetical students of varying ability levels (either high or low) for test performances that range from excellent to clear failure. These students also differ in the amount of effort they expend in preparing for the test (either high or low). While the results indicate that test outcome is the major determinant of classroom evaluation, teachers also reinforce effort. Students who are perceived as having expended effort are rewarded more in success and punished less in failure than those who do not try. Moreover, these evaluative reactions appear to be largely independent of student ability level. These same general results have been obtained repeatedly by investigators using various subject popula

394 citations

Journal ArticleDOI
TL;DR: This article examined the relationship between epistemological beliefs (quick learning, simple knowledge, certain knowledge, and innate ability) and learned helplessness and conceptual understanding and application reasoning in conceptual change learning (CCL).
Abstract: This study examined the relationship between two variable sets: (a) epistemological beliefs (quick learning, simple knowledge, certain knowledge, and innate ability) and learned helplessness and (b) conceptual understanding and application reasoning in conceptual change learning (CCL). Hypothetical dimensions underlying the Epistemological Belief Questionnaire and effects of different kinds of prior knowledge on CCL were explored with 212 students in Grades 9-12 in 13 science classes at a rural public high school in Georgia. Exploratory factor analyses revealed 3 factors underlying epistemological beliefs: Quick Learning, Simple-Certain Knowledge, and Innate Ability. Canonical correlation analyses show that beliefs about Simple-Certain Knowledge contribute the most to CCL, whereas beliefs about Innate Ability contribute the least. Beliefs about Simple-Certain Knowledge and Quick Learning are important factors in CCL

343 citations

References
More filters
Book
B. J. Winer1
01 Jan 1962
TL;DR: In this article, the authors introduce the principles of estimation and inference: means and variance, means and variations, and means and variance of estimators and inferors, and the analysis of factorial experiments having repeated measures on the same element.
Abstract: CHAPTER 1: Introduction to Design CHAPTER 2: Principles of Estimation and Inference: Means and Variance CHAPTER 3: Design and Analysis of Single-Factor Experiments: Completely Randomized Design CHAPTER 4: Single-Factor Experiments Having Repeated Measures on the Same Element CHAPTER 5: Design and Analysis of Factorial Experiments: Completely-Randomized Design CHAPTER 6: Factorial Experiments: Computational Procedures and Numerical Example CHAPTER 7: Multifactor Experiments Having Repeated Measures on the Same Element CHAPTER 8: Factorial Experiments in which Some of the Interactions are Confounded CHAPTER 9: Latin Squares and Related Designs CHAPTER 10: Analysis of Covariance

25,607 citations

Journal ArticleDOI
TL;DR: This chapter discusses design and analysis of single-Factor Experiments: Completely Randomized Design and Factorial Experiments in which Some of the Interactions are Confounded.

24,665 citations

Journal ArticleDOI
TL;DR: In this paper, the basic theory of analysis of variance by considering several different mathematical models is examined, including fixed-effects models with independent observations of equal variance and other models with different observations of variance.
Abstract: Originally published in 1959, this classic volume has had a major impact on generations of statisticians. Newly issued in the Wiley Classics Series, the book examines the basic theory of analysis of variance by considering several different mathematical models. Part I looks at the theory of fixed-effects models with independent observations of equal variance, while Part II begins to explore the analysis of variance in the case of other models.

5,728 citations

Book
01 Jan 1966
TL;DR: In this article, the authors presented a case of two means regression method for the family error rate, which was used to estimate the probability of a family having a nonzero family error.
Abstract: 1 Introduction.- 1 Case of two means.- 2 Error rates.- 2.1 Probability of a nonzero family error rate.- 2.2 Expected family error rate.- 2.3 Allocation of error.- 3 Basic techniques.- 3.1 Repeated normal statistics.- 3.2 Maximum modulus (Tukey).- 3.3 Bonferroni normal statistics.- 3.4 ?2 projections (Scheffe).- 3.5 Allocation.- 3.6 Multiple modulus tests (Duncan).- 3.7 Least significant difference test (Fisher).- 4 p-mean significance levels.- 5 Families.- 2 Normal Univariate Techniques.- 1 Studentized range (Tukey).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 F projections (Scheffe)48.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Bonferroni t statistics.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Studentized maximum modulus.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Many-one t statistics76.- 5.1 Method.- 5.2 Applications.- 5.3 Comparison.- 5.4 Derivation.- 5.5 Distributions and tables.- 6 Multiple range tests (Duncan).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Least significant difference test (Fisher).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Tukey's gap-straggler-variance test.- 8.2 Shortcut methods.- 8.3 Multiple F tests.- 8.4 Two-sample confidence intervals of predetermined length.- 8.5 An improved Bonferroni inequality.- 9 Power.- 10 Robustness.- 3 Regression Techniques.- 1 Regression surface confidence bands.- 1.1 Method.- 1.2 Comparison.- 1.3 Derivation.- 2 Prediction.- 2.1 Method.- 2.2 Comparison.- 2.3 Derivation.- 3 Discrimination.- 3.1 Method.- 3.2 Comparison.- 3.3 Derivation.- 4 Other techniques.- 4.1 Linear confidence bands.- 4.2 Tolerance intervals.- 4.3 Unlimited discrimination intervals.- 4 Nonparametric Techniques.- 1 Many-one sign statistics (Steel).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 k-sample sign statistics.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Many-one rank statistics (Steel).- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 k-sample rank statistics.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Signed-rank statistics.- 6 Kruskal-Wallis rank statistics (Nemenyi).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Friedman rank statistics (Nemenyi).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Permutation tests.- 8.2 Median tests (Nemenyi).- 8.3 Kolmogorov-Smirnov statistics.- 5 Multivariate Techniques.- 1 Single population covariance scalar unknown.- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 Single population covariance matrix unknown.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 k populations covariance matrix unknown.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Other techniques.- 4.1 Variances known covariances unknown.- 4.2 Variance-covariance intervals.- 4.3 Two-sample confidence intervals of predetermined length.- 6 Miscellaneous Techniques.- 1 Outlier detection.- 2 Multinomial populations.- 2.1 Single population.- 2.2 Several populations.- 2.3 Cross-product ratios.- 2.4 Logistic response curves.- 3 Equality of variances.- 4 Periodogram analysis.- 5 Alternative approaches: selection, ranking, slippage.- A Strong Law For The Expected Error Rate.- B TABLES.- I Percentage points of the studentized range.- II Percentage points of the Bonferroni t statistic.- III Percentage points of the studentized maximum modulus.- IV Percentage points of the many-one t statistics.- V Percentage points of the Duncan multiple range test.- VI Percentage points of the many-one sign statistics.- VIII Percentage points of the many-one rank statistics.- IX Percentage points of the k-sample rank statistics.- Developments in Multiple Comparisons 1966-).- 3.5 Allocation.- 3.6 Multiple modulus tests (Duncan).- 3.7 Least significant difference test (Fisher).- 4 p-mean significance levels.- 5 Families.- 2 Normal Univariate Techniques.- 1 Studentized range (Tukey).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 F projections (Scheffe)48.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Bonferroni t statistics.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Studentized maximum modulus.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Many-one t statistics76.- 5.1 Method.- 5.2 Applications.- 5.3 Comparison.- 5.4 Derivation.- 5.5 Distributions and tables.- 6 Multiple range tests (Duncan).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Least significant difference test (Fisher).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Tukey's gap-straggler-variance test.- 8.2 Shortcut methods.- 8.3 Multiple F tests.- 8.4 Two-sample confidence intervals of predetermined length.- 8.5 An improved Bonferroni inequality.- 9 Power.- 10 Robustness.- 3 Regression Techniques.- 1 Regression surface confidence bands.- 1.1 Method.- 1.2 Comparison.- 1.3 Derivation.- 2 Prediction.- 2.1 Method.- 2.2 Comparison.- 2.3 Derivation.- 3 Discrimination.- 3.1 Method.- 3.2 Comparison.- 3.3 Derivation.- 4 Other techniques.- 4.1 Linear confidence bands.- 4.2 Tolerance intervals.- 4.3 Unlimited discrimination intervals.- 4 Nonparametric Techniques.- 1 Many-one sign statistics (Steel).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 k-sample sign statistics.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Many-one rank statistics (Steel).- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 k-sample rank statistics.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Signed-rank statistics.- 6 Kruskal-Wallis rank statistics (Nemenyi).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Friedman rank statistics (Nemenyi).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Permutation tests.- 8.2 Median tests (Nemenyi).- 8.3 Kolmogorov-Smirnov statistics.- 5 Multivariate Techniques.- 1 Single population covariance scalar unknown.- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 Single population covariance matrix unknown.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 k populations covariance matrix unknown.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Other techniques.- 4.1 Variances known covariances unknown.- 4.2 Variance-covariance intervals.- 4.3 Two-sample confidence intervals of predetermined length.- 6 Miscellaneous Techniques.- 1 Outlier detection.- 2 Multinomial populations.- 2.1 Single population.- 2.2 Several populations.- 2.3 Cross-product ratios.- 2.4 Logistic response curves.- 3 Equality of variances.- 4 Periodogram analysis.- 5 Alternative approaches: selection, ranking, slippage.- A Strong Law For The Expected Error Rate.- B TABLES.- I Percentage points of the studentized range.- II Percentage points of the Bonferroni t statistic.- III Percentage points of the studentized maximum modulus.- IV Percentage points of the many-one t statistics.- V Percentage points of the Duncan multiple range test.- VI Percentage points of the many-one sign statistics.- VIII Percentage points of the many-one rank statistics.- IX Percentage points of the k-sample rank statistics.- Developments in Multiple Comparisons 1966-1976.- 1 Introduction.- 2 Papers of special interest.- 2.1 Probability inequalities.- 2.2 Methods for unbalanced ANOVA.- 2.3 Conditional confidence levels.- 2.4 Empirical Bayes approach.- 2.5 Confidence bands in regression.- 3 References.- 4 Bibliography 1966-1976.- 4.1 Survey articles.- 4.2 Probability inequalities.- 4.3 Tables.- 4.4 Normal multifactor methods.- 4.5 Regression.- 4.6 Categorical data.- 4.7 Nonparametric techniques.- 4.8 Multivariate methods.- 4.9 Miscellaneous.- 4.10 Pre-1966 articles missed in [6].- 4.11 Late additions.- 5 List of journals scanned.- Addendum New Table of the Studentized Maximum Modulus.- Table IIIA Percentage points of the studentized maximum modulus.- Author Index.

4,763 citations

Journal ArticleDOI
TL;DR: Campbell and Stanley's "Experimental and Quasi-Experimental Designs for Research on Teaching" (1963) gained the status of a classic exposition of experimentation in education.
Abstract: Shortly after publication, Donald T. Campbell and Julian C. Stanley's "Experimental and Quasi-Experimental Designs for Research on Teaching" (1963) gained the status of a classic exposition of experimentation in education. There have been few attempts to extend this pioneering work, as might be expected of a work so comprehensive in conception and so brilliant in execution. Webb et al. (1966) produced a work similar in purposethat being to identify sources of external invalidity which arise from the reactive effect of measurement. We know of no other

547 citations