Showing papers on "Effect size published in 2016"
TL;DR: A large sample of real-world data was used to illustrate the base rate dependence of correlations when applied to dichotomous or ordinal data to recommend AUCs, Pearson/Thorndike adjusted correlations, Cohen’s d, or polychoric correlations should be considered as alternate effect size statistics in many contexts.
Abstract: Correlations are the simplest and most commonly understood effect size statistic in psychology. The purpose of the current paper was to use a large sample of real-world data (109 correlations with 60,415 participants) to illustrate the base rate dependence of correlations when applied to dichotomous or ordinal data. Specifically, we examined the influence of the base rate on different effect size metrics. Correlations decreased when the dichotomous variable did not have a 50 % base rate. The higher the deviation from a 50 % base rate, the smaller the observed Pearson’s point-biserial and Kendall’s tau correlation coefficients. In contrast, the relationship between base rate deviations and the more commonly proposed alternatives (i.e., polychoric correlation coefficients, AUCs, Pearson/Thorndike adjusted correlations, and Cohen’s d) were less remarkable, with AUCs being most robust to attenuation due to base rates. In other words, the base rate makes a marked difference in the magnitude of the correlation. As such, when using dichotomous data, the correlation may be more sensitive to base rates than is optimal for the researcher’s goals. Given the magnitude of the association between the base rate and point-biserial correlations (r = −.81) and Kendall’s tau (r = −.80), we recommend that AUCs, Pearson/Thorndike adjusted correlations, Cohen’s d, or polychoric correlations should be considered as alternate effect size statistics in many contexts.
71 citations
TL;DR: The use of effect-size estimators is becoming generalized, as well as the consideration of meta-analytic studies, however, several inadequate practices still persist and recommendations for improving statistical practice are made.
Abstract: Background: The statistical reform movement and the American Psychological Association (APA) defend the use of estimators of the effect size and its confidence intervals, as well as the interpretation of the clinical significance of the findings. Method: A survey was conducted in which academic psychologists were asked about their behavior in designing and carrying out their studies. The sample was composed of 472 participants (45.8% men). The mean number of years as a university professor was 13.56 years (SD= 9.27). Results: The use of effect-size estimators is becoming generalized, as well as the consideration of meta-analytic studies. However, several inadequate practices still persist. A traditional model of methodological behavior based on statistical significance tests is maintained, based on the predominance of Cohen’s d and the unadjusted R2/2, which are not immune to outliers or departure from normality and the violations of statistical assumptions, and the under-reporting of confidence intervals of effect-size statistics. Conclusion: The paper concludes with recommendations for improving statistical practice.
6 citations
01 Jan 2016
TL;DR: In this article, the difference between correlation and regression is discussed and statistical techniques for testing the strength of the relationship between variables are described, and a discussion is provided on how to graph both a correlation and a regression and how to report the results in a scientific manuscript.
Abstract: A correlation is a rather unusual statistical procedure that can be used to describe a relationship between two variables or as a method to make inferences. Often scientists misuse a correlation to imply a cause and effect relationship when in fact one may or may not exist. Only a regression analysis can be used for determination of cause and effect. This chapter describes the difference between correlation and regression and describes statistical techniques for testing the strength of the relationship between variables. A discussion is provided on how to graph both a correlation and a regression and how to report the results in a scientific manuscript.
1 citations
01 Jan 2016
TL;DR: In this paper, the expected value of R 2 is not zero under the null hypothesis that ρ, the population value of the multiple correlation coefficient, equals zero, which has implications both for significance testing and effect size estimation involving R 2.
Abstract: In linear multiple regression it is common practice to test whether the squared multiple correlation coefficient, R 2 , differs significantly from zero. Although frequently used, this test is misleading because the expected value of R 2 is not zero under the null hypothesis that ρ, the population value of the multiple correlation coefficient, equals zero. The non-zero expected value of R 2 has implications both for significance testing and effect size estimation involving the squared multiple correlation coefficient. In this paper we discuss and offer a freely available computer program that calculates the expected value of R 2 , an adjusted R 2 value and effect size measure that both incorporate the expected value of R2, and an F statistic that tests the significance of difference between the obtained R 2 and the expected value of R 2 under the null hypothesis that ρ = 0. The interactive, stand-alone program is written in FORTRAN 77 for a Windows environment. The user simply enters the value of a multiple correlation coefficient from a linear regression, the number of predictors, and the sample size. No knowledge of FORTRAN or any other statistical programming language is required.
TL;DR: In this paper, the expected value of R2 has been used to test whether the squared multiple correlation coefficient (R2) differs significantly from zero under the null hypothesis that ρ, the population value of the R2, equals zero.
Abstract: In linear multiple regression it is common practice to test whether the squared multiple correlation co efficient, R2, differs significantly from zero. Although frequently used, this test is misleading because the expected value of R2 is not zero under the null hypothesis that ρ, the population value of the multiple correlation coefficient, equals zero. The non-zero expected value of R2 has implications both for significance testing and effect size estimation involving the squared multiple correlation coefficient. In this paper we discuss and offer a freely available computer program that calculates the expected value of R2, an adjusted R2value and effect size measure that both incorporate the expected value of R2, and an F statistic that tests the significance of difference between the obtained R2 and the expected value ofR2 under the null hypothesis that ρ = 0. The interactive, stand-alone program is written in Fortran 77 for a Windows environment. The user simply enters the value of a multiple correlation coefficient from a linear regression, the number of predictors, and the sample size. No knowledge of FORTRAN or any other statistical programming language is required.