■ Abstract It is widely but incorrectly believed that the t-test and linear regression are valid only for Normally distributed outcomes. The t-test and linear regression compare the mean of an outcome variable for different subjects. While these are valid even in very small samples if the outcome variable is Normally distributed, their major usefulness comes from the fact that in large samples they are valid for any distribution. We demonstrate this validity by simulation in extremely non-Normal data. We discuss situations in which in other methods such as the Wilcoxon rank sum test and ordinal logistic regression (proportional odds model) have been recommended, and conclude that the t-test and linear regression often provide a convenient and practical alternative. The major limitation on the t-test and linear regression for inference about associations is not a distributional one, but whether detecting and estimating a difference in the mean of the outcome answers the scientific question at hand.

/pdf/the-importance-of-the-normality-assumption-in-large-public-3i15uzzxrs.pdf

The importance of the normality assumption in large public health data sets.

Background
Gene set analysis (GSA) is a widely used strategy for gene expression data analysis based on pathway knowledge. GSA focuses on sets of related genes and has established major advantages over individual gene analyses, including greater robustness, sensitivity and biological relevance. However, previous GSA methods have limited usage as they cannot handle datasets of different sample sizes or experimental designs.

/pdf/gage-generally-applicable-gene-set-enrichment-for-pathway-2k6a5h2pj3.pdf

GAGE: generally applicable gene set enrichment for pathway analysis

Likert questionnaires are widely used in survey research, but it is unclear whether the item data should be investigated by means of parametric or nonparametric procedures. This study compared the Type I and II error rates of the t test versus the Mann-Whitney-Wilcoxon (MWW) for five-point Likert items. Fourteen population distributions were defined and pairs of samples were drawn from the populations and submitted to the t test and the t test on ranks, which yields the same results as MWW. The results showed that the two tests had equivalent power for most of the pairs. MWW had a power advantage when one of the samples was drawn from a skewed or peaked distribution. Strong power differences between the t test and MWW occurred when one of the samples was drawn from a multimodal distribution. Notably, the Type I error rate of both methods was never more than 3% above the nominal rate of 5%, even not when sample sizes were highly unequal. In conclusion, for five-point Likert items, the t test and MWW generally have similar power, and researchers do not have to worry about finding a difference whilst there is none in the population.

Five-Point Likert Items: t test versus Mann-Whitney-Wilcoxon

Freedman [Adv. in Appl. Math. 40 (2008) 180-193; Ann. Appl. Stat. 2 (2008) 176-196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman's model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fixed. OLS adjustment cannot hurt asymptotic precision when a full set of treatment-covariate interactions is included. Asymptotically valid confidence intervals can be constructed with the Huber-White sandwich standard error estimator. Checks on the asymptotic approximations are illustrated with data from Angrist, Lang, and Oreopoulos's [Am. Econ. J.: Appl. Econ. 1:1 (2009) 136--163] evaluation of strategies to improve college students' achievement. The strongest reasons to support Freedman's preference for unadjusted estimates are transparency and the dangers of specification search.

Agnostic notes on regression adjustments to experimental data: Reexamining Freedman's critique

Freedman [Adv. in Appl. Math. 40 (2008) 180–193; Ann. Appl. Stat. 2 (2008) 176–196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman’s model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fixed. OLS adjustment cannot hurt asymptotic precision when a full set of treatment–covariate interactions is included. Asymptotically valid confidence intervals can be constructed with the Huber–White sandwich standard error estimator. Checks on the asymptotic approximations are illustrated with data from Angrist, Lang, and Oreopoulos’s [Am. Econ. J.: Appl. Econ. 1:1 (2009) 136–163] evaluation of strategies to improve college students’ achievement. The strongest reasons to support Freedman’s preference for unadjusted estimates are transparency and the dangers of specification search.

/pdf/agnostic-notes-on-regression-adjustments-to-experimental-3ztsaszjlo.pdf

Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique

Summary When the assumptions of parametric statistical tests for the difference between two means are violated, it is commonly advised that non-parametric tests are a more robust substitute. The history of the investigation of this issue is summarized. The robustness of the t -test was evaluated, by repeated computer testing for differences between samples from two populations of equal means but non-normal distributions and with different variances and sample sizes. Two common alternatives to t -Welch's approximate t and the Mann-Whitney U -test-were evaluated in the same way. The t -test is sufficiently robust for use in all likely cases, except when skew is severe or when population variances and sample sizes both differ. The Welch test satisfactorily addressed the latter problem, but was itself sensitive to departures from normality. Contrary to its popular reputation, the U -test showed a dramatic 'lack of robustness' in many cases-largely because it is sensitive to population differences other than b...

Guy J. Forrester

Papers

Robustness of the t and U tests under combined assumption violations