Showing papers in "Understanding Statistics in 2004"
TL;DR: Partial least squares (PLS) analysis as mentioned in this paper is a generalization of covariance-based structural equation modeling (SEM), which is particularly suited for situations in which constructs are measured by a very large number of indicators and where maximum likelihood covariancebased SEM tools reach their limit.
Abstract: Since the introduction of covariance-based structural equation modeling (SEM) by Joreskog in 1973, this technique has been received with considerable interest among empirical researchers However, the predominance of LISREL, certainly the most well-known tool to perform this kind of analysis, has led to the fact that not all researchers are aware of alternative techniques for SEM, such as partial least squares (PLS) analysis Therefore, the objective of this article is to provide an easily comprehensible introduction to this technique, which is particularly suited to situations in which constructs are measured by a very large number of indicators and where maximum likelihood covariance-based SEM tools reach their limit Because this article is intended as a general introduction, it avoids mathematical details as far as possible and instead focuses on a presentation of PLS, which can be understood without an in-depth knowledge of SEM
1,458 citations
TL;DR: Several methods are outlined that describe how post hoc power analyses can be used to improve the design of independent replications to rule in/out rival explanations in the presence of statistically nonsignificant findings.
Abstract: This article advocates the use of post hoc power analyses. First, reasons for the nonuse of a priori power analyses are presented. Next, post hoc power is defined and its utility delineated. Third, a step-by-step guide is provided for conducting post hoc power analyses. Fourth, a heuristic example is provided to illustrate how post hoc power can help to rule in/out rival explanations in the presence of statistically nonsignificant findings. Finally, several methods are outlined that describe how post hoc power analyses can be used to improve the design of independent replications.
166 citations
TL;DR: In this paper, the authors of journal articles in psychology, behavioral neuroscience, and medicine were invited by email to visit a Web site and indicate on a figure where they judged replication means would plausibly fall.
Abstract: Confidence intervals (CIs) and standard error bars give information about replication, but do researchers have an accurate appreciation of that information? Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited by email to visit a Web site and indicate on a figure where they judged replication means would plausibly fall. Responses from 263 researchers suggest that many leading researchers in the 3 disciplines underestimate the extent to which future replications will vary. A 95% CI will on average capture 83.4% of future replication means. A majority of respondents, however, held the confidence-level misconception that a 95% CI will on average capture 95% of replication means. Better understanding of CIs is needed if they are to be successfully used more widely in psychology.
138 citations
TL;DR: In this article, MR codes equivalent to repeated contrasts in ANOVA are introduced and explicit ties between MR coding techniques and ANOVA contrasts should increase the availability of both sets of procedures to students of statistics.
Abstract: The use of categorical variable coding has a long history in the application of multiple regression (MR) techniques. The primary purpose of this article is to review and clarify existing MR coding techniques-such as dummy, effect, and contrast coding-and to discuss their properties with reference to analogous procedures in the traditional analysis of variance (ANOVA) model (such as simple, deviation, Helmert, and trend contrasts). Furthermore, in this article, I introduce MR codes equivalent to repeated contrasts in ANOVA. Such explicit ties between MR coding techniques and ANOVA contrasts should increase the availability of both sets of procedures to students of statistics.
95 citations
TL;DR: The authors discuss some limitations of some common quantitative methods used in psychology and attempt to challenge the "golden rule" aspect that such methods frequently acquire among researchers, with the aim of emphasizing that quantitative methods are tools with limitations that need evaluation, and with rationales that require justification, in each single scientific application.
Abstract: The aim of this article is to discuss some of the limitations of some common quantitative methods used in psychology. The main reason for so doing is to attempt to challenge the "golden rule" aspect that such methods frequently acquire among researchers, with the aim of emphasizing that quantitative methods are tools with limitations that need evaluation, and with rationales that require justification, in each single scientific application.
88 citations
TL;DR: A nontechnical introduction to the taxometric method for assessing latent structure and a number of refinements and extensions to taxometric methodology including a useful interpretive aid based on the parallel analysis of simulated taxonic and dimensional comparison data.
Abstract: Although social and behavioral scientists often presume that individual differences underlying measured variables consist of differences in degree rather than differences in kind, the distinction between taxonic (categorical) and dimensional (continuous) latent structure poses an empirical question with important implications for basic and applied science. In this article, we present a nontechnical introduction to the taxometric method for assessing latent structure. We outline unique features of the general approach and then describe and illustrate specific taxometric procedures, emphasizing the conceptual logic of each analytic technique, factors that can influence results and their interpretation, and decisions that must be made to implement each procedure most appropriately and powerfully. We present a number of refinements and extensions to taxometric methodology including a useful interpretive aid based on the parallel analysis of simulated taxonic and dimensional comparison data. We focus on practi...
65 citations
TL;DR: In this paper, the authors outline the importance of nonindependence to the design of research studies and interpretation of research results and provide a 2-step process for using both data analytic techniques in individual-, dyadic-, and group-level research.
Abstract: The purpose of this article is to outline the importance of nonindependence to the design of research studies and interpretation of research results. Nonindependence can influence the validity of research results at any level of analysis (i.e., individual, dyadic, group; Kenny & Judd, 1986). We define specifically what nonindependence is, suggesting some sources of nonindependence based on Kenny and Judd. Additionally, we contrast the issue of nonindependence with what James, Demaree, and Wolf (1984) referred to as within-group agreement. Each approach offers a unique advantage in specific situations. We provide an explanation of the steps involved in the tests of nonindependence and within-group agreement from a practical perspective. Finally, we provide a 2-step process for using both data analytic techniques in individual-, dyadic-, and group-level research.
64 citations
TL;DR: Regression methods based on using generalized linear models are discussed as well as their implementation with the SAS procedures PROC LOGISTIC and PROC GENMOD.
Abstract: Sensitivity and specificity summarize the performance of a diagnostic test with a positive/negative outcome determined by a gold standard When the test is quantitative, receiver operating characteristic (ROC) curves are used to display the performance of all possible cutpoints of the quantitative diagnostic marker The ROC curve offers a graphical interpretation of the trade-off between sensitivity and specificity of the range of possible cutpoints Various methods are used to estimate the ROC curve including empirical, parametric, semiparametric, and regression methods Similarly, various software packages provide ROC curve estimation Regression methods based on using generalized linear models are discussed as well as their implementation with the SAS procedures PROC LOGISTIC and PROC GENMOD Recent attention has been given to determining the optimal decision rule, also called the optimal operating point (OOP) Similar to the ROC curve, the OOP provides a graphical interpretation for decision making T
43 citations
TL;DR: In this paper, the authors point out that the ITSE method yields intervention effect estimates that are inconsistent with the logic of the design and that these estimates are likely to be quite misleading.
Abstract: The ITSE method (Gottman, 1981; Rushe & Gottman, 1993; Williams & Gottman, 1982, 1999) for analyzing the 2-phase interrupted time-series design was evaluated. I point out that the method yields intervention effect estimates that are inconsistent with the logic of the design and that these estimates are likely to be quite misleading. Furthermore, the omnibus F test and the subsequent t test on level change are inconsistent in purpose; the omnibus F statistic is shown not to be relevant to the stated hypothesis of equality of intercepts defined for the method. The omnibus F can be exceedingly large when subhypotheses regarding equality of slopes and intercepts are true; the converse can also occur. Related problems, such as inconsistencies in published simulation evaluations and problems with software, are explained. It is recommended that the ITSE method not be used regardless of sample size.
38 citations
TL;DR: In this article, Anderson and Schumacker showed that the robust estimator can have a relatively large standard error when the error term is heteroscedastic, even under normality.
Abstract: A serious practical problem with the ordinary least squares regression estimator is that it can have a relatively large standard error when the error term is heteroscedastic, even under normality. In practical terms, power can be poor relative to other regression estimators that might be used. This article illustrates the problem and summarizes strategies for dealing with it. Included are new results on the robust estimator recently studied by Anderson and Schumacker (2003).
26 citations
TL;DR: In this paper, a flexible discrete time event history model that incorporates individual level random effects was applied to the analysis of partnership episodes for adult members of the National Child Development Study followed up between the ages of 16 and 33.
Abstract: Event history or survival models are applicable to outcomes that are measures of duration, for example the length of employment periods or times to death after medical treatment. When individuals are grouped within institutions such as firms or clinics the resulting multilevel structure also needs to be incorporated into the model. An important application is where individuals are the "higher level" units and they experience repeated durations, such as lengths of partnerships. In this article we show how such repeated measures data can be modeled using a flexible discrete time event history model that incorporates individual level random effects. The model is applied to the analysis of partnership episodes for adult members of the National Child Development Study followed up between the ages of 16 and 33. The exposition will not assume a detailed knowledge of event history modeling.
TL;DR: The results of a simulation study on the performance of measures for the analysis of 2-2 tables are reported in this article. But the results of the simulation study are limited to the 2 -2 tables, and only five factors were considered: sampling distribution, sample size, strength of association, symmetry of the distribution, and nominal α.
Abstract: The results of a simulation study on the performance of measures for the analysis of 2 � 2 tables are reported. The simulations included 11 measures for 2 � 2 tables: (a) Pearson's α; (b) the standard normal z; (c) the log-odds ratio; (d) the log-linear interaction; (e) Goodman's (1991) weighted log-linear interaction; (f) Vogel's z; (g) the binomial test; (h) Lehmacher's (1981) asymptotic hypergeometric test; (i) Perli, Hommel, and Lehmacher's (1985) asymptotic test; (j) Lindner's (1984) exact hypergeometric test; and (k) Lautsch and von Weber's (2003) adaptation of Dunkl and von Eye's (1990) test. The factors varied in the simulations were (a) type of sampling distribution, (b) sample size, (c) strength of association in the 2 � 2 table, (d) symmetry of the distribution in the 2 � 2 table, and (e) the nominal α. Results suggest that the distribution of these 11 tests is very near the normal under all conditions. Of the 5 factors, only the type of the sampling distribution has no strong effects on the Ty...
TL;DR: This paper used the noncentral t distribution and methods presented by Cumming and Finch (2001) to construct a confidence interval for the observed effect size, which is then used to compute the limits of plausible replicability (post hoc power) values.
Abstract: Following a suggestion from Greenwald, Gonzalez, Harris, and Guthrie (1996), Posavac (2002) developed an approach to estimate the probability that replications of completed studies would be statistically significant. We argue that this approach is faulty because it is based on the unjustifiably restrictive assumption that the effect size observed in the initial study is the population effect size. Using the noncentral t distribution and methods presented by Cumming and Finch (2001), we construct a confidence interval for the observed effect size. Boundaries of that confidence interval are used to compute the limits of plausible replicability (post hoc power) values. It is demonstrated that the possible values span an impractically large range, rendering the post hoc analysis of replicability without merit. These issues are discussed in the broader context of post hoc power analyses.
TL;DR: In this article, it is suggested that an adaptive kernel density estimator be used when summarizing data or when dealing with effect size, which is consistent with the graphical perspective initially suggested by Cohen (1977).
Abstract: A fundamental goal when comparing 2 groups of participants is understanding how the groups differ. It is well known that p values are inadequate for this purpose. Numerical measures of effect size represent a step toward addressing this issue, but practical problems with the better known measures have been revealed in recent years. Kernel density estimators provide an alternative approach that is consistent with the graphical perspective initially suggested by Cohen (1977), but better known methods suffer from 2 practical problems. This article illustrates these problems and describes how they might be addressed. It is suggested that an adaptive kernel density estimator be used when summarizing data or when dealing with effect size.
TL;DR: This paper reviewed 12 introductory statistics textbooks for the behavioral sciences published since 1998 and identified how well selected recent developments in the statistics field were incorporated into these texts, including concepts such as effect size, confidence intervals, power and a priori sample size determination, and statistical assumptions.
Abstract: This study reviewed 12 introductory statistics textbooks for the behavioral sciences published since 1998. The intent was to identify how well selected recent developments in the statistics field were incorporated into these texts. Textbooks generally included concepts such as effect size, confidence intervals, power and a priori sample size determination, and statistical assumptions. However, no text presented a design and analysis approach that integrated all of these data analysis activities into the standard hypothesis testing procedures. Second, few texts appeared to incorporate recent research regarding the effects of nonnormality on analysis results. In addition, no text illustrated tests involving newly developed robust estimators or modern rank-based techniques even though such procedures can effectively handle problems associated with nonnormality that frequently arise in behavioral data. Finally, few authors of the basic statistics textbooks, as well as authors of some highly respected analysis...
TL;DR: In this article, the authors focus on alternative explanations for significant types identified using configural frequency analysis (CFA) and argue that other base models, such as a Markov chain model, may be more appropriate for prospective research.
Abstract: Configural Frequency Analysis (CFA) is a technique for discovering co-occurrence patterns in categorical data that are often interpreted as indicating the presence of some syndrome or "type." This article focuses on alternative explanations for significant types identified using CFA. Specifically, although significant patterns of co-occurrence in categorical data may signal presence of a syndrome or type, such a finding can also indicate that the observed contingency table is a mixture of 2 or more underlying populations with different baserates on at least 2 of the variables under consideration. Alternatively, the assumed base model of response for the data is incorrect, thereby producing significant residual cells that are falsely identified as a syndrome. It is argued that other base models, such as a Markov chain model, may be more appropriate for prospective research. Finally, correct identification of a CFA type is most likely to occur when the researcher analyzes only that subset of variables that ...