SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses -the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.

/pdf/controlling-the-false-discovery-rate-a-practical-and-431ouqffk6.pdf

Controlling the false discovery rate: a practical and powerful approach to multiple testing

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-seq, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data, using shrinkage estimation for dispersions and fold changes to improve stability and interpretability of estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression. The DESeq2 package is available at http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html
 .

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-3ywnbkcan0.pdf

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2

Whole-genome association studies (WGAS) bring new computational, as well as analytic, challenges to researchers. Many existing genetic-analysis tools are not designed to handle such large data sets in a convenient manner and do not necessarily exploit the new opportunities that whole-genome data bring. To address these issues, we developed PLINK, an open-source C/C++ WGAS tool set. With PLINK, large data sets comprising hundreds of thousands of markers genotyped for thousands of individuals can be rapidly manipulated and analyzed in their entirety. As well as providing tools to make the basic analytic steps computationally efficient, PLINK also supports some novel approaches to whole-genome data that take advantage of whole-genome coverage. We introduce PLINK and describe the five main domains of function: data management, summary statistics, population stratification, association analysis, and identity-by-descent estimation. In particular, we focus on the estimation and use of identity-by-state and identity-by-descent information in the context of population-based whole-genome studies. This information can be used to detect and correct for population stratification and to identify extended chromosomal segments that are shared identical by descent between very distantly related individuals. Analysis of the patterns of segmental sharing has the potential to map disease loci that contain multiple rare variants in a population-based linkage analysis.

http://ibgwww.colorado.edu/cdrom2009/ScriptsA/purcell/GWAS/boulder_09_plink_intro.pdf

PLINK: A Tool Set for Whole-Genome Association and Population-Based Linkage Analyses

limma is an R/Bioconductor software package that provides an integrated solution for analysing data from gene expression experiments. It contains rich features for handling complex experimental designs and for information borrowing to overcome the problem of small sample sizes. Over the past decade, limma has been a popular choice for gene discovery through differential expression analyses of microarray and high-throughput PCR data. The package contains particularly strong facilities for reading, normalizing and exploring such data. Recently, the capabilities of limma have been significantly expanded in two important directions. First, the package can now perform both differential expression and differential splicing analyses of RNA sequencing (RNA-seq) data. All the downstream analysis tools previously restricted to microarray data are now available for RNA-seq as well. These capabilities allow users to analyse both RNA-seq and microarray data with very similar pipelines. Second, the package is now able to go past the traditional gene-wise expression analyses in a variety of ways, analysing expression profiles in terms of co-regulated sets of genes or in terms of higher-order expression signatures. This provides enhanced possibilities for biological interpretation of gene expression differences. This article reviews the philosophy and design of the limma package, summarizing both new and historical features, with an emphasis on recent enhancements and features that have not been previously described.

/pdf/limma-powers-differential-expression-analyses-for-rna-4yl7ae3arh.pdf

limma powers differential expression analyses for RNA-sequencing and microarray studies

In comparative high-throughput sequencing assays, a fundamental task is the analysis of count data, such as read counts per gene in RNA-Seq data, for evidence of systematic changes across experimental conditions. Small replicate numbers, discreteness, large dynamic range and the presence of outliers require a suitable statistical approach. We present DESeq2, a method for differential analysis of count data. DESeq2 uses shrinkage estimation for dispersions and fold changes to improve stability and interpretability of the estimates. This enables a more quantitative analysis focused on the strength rather than the mere presence of differential expression and facilitates downstream tasks such as gene ranking and visualization. DESeq2 is available as an R/Bioconductor package.

/pdf/moderated-estimation-of-fold-change-and-dispersion-for-rna-59glcgpwlk.pdf

Genizi and Hochberg (1978) recommended using Contrast Set Preserving (CSP) procedures in the class of $T(Q)$ procedures for multiple comparisons in general unbalanced designs based on partial results. They did not, however, propose a general method for selecting a specific CSP procedure, or for replacing a given non-CSP procedure with a better CSP one. In this work we identify a certain orthogonal transformation of non-CSP procedures into CSP ones and give a sufficient condition for the uniform dominance (shorter confidence intervals for all contrasts) of the latter over the former. Two important implications of the given condition are: (i) Applying the given transformation to Spjotvoll and Stoline's (1973) $T$'-procedure in any unbalanced ANOVA gives a uniformly improved procedure. (ii) In any arbitrary design, our transformation gives uniform improvement if the original procedure is "nearly CSP."

/pdf/improving-some-multiple-comparison-procedures-408le2pikf.pdf

Improving Some Multiple Comparison Procedures

We investigate the optimality of the estimator θ(Qα ) when L(θ ) is the set of all normalized linear functions of θ and all elements in L(θ ) are of “equal importance.”

On optimal1iy criteria in simultaneous interval estimation

Two new estimators for calibrating unknowns from dose-response curves, in a system of quality-controlled assays, are examined. In contrast with the conventional estimator which uses only the results of the one assay in which the response of the unknown dose is measured, the new estimators also utilize the results of all other assays through the replications of the control samples in the system. The first estimator is based on maximizing the likelihood of the given system (with respect to the different dose-response parameters, the levels of the control samples and the levels of the unknowns) when response errors are normally distributed. The second estimator is a regression-like estimator obtained by subtracting from the conventional estimator its estimated regression on the deviation of the calibrated control levels in the given assay from their average values in the system. Evaluations of the reductions in bias and variance attained by the new estimators show when substantial reductions in mean square error can be expected. The new estimators are illustrated with a system of 22 hFSH radioimmunoassays.

On improved calibrations of unknowns in a system of quality-controlled assays

Some Miscellaneous Multiple Comparison Problems

In a comparison of the means of several samples of size n, the studentized range is the range of the sample means divided by the usual standard error of a single mean. It may be used in successive stages of a multiple comparison procedure. Two approaches are the Newman–Keuls method and Tukey's T-method. In a single sample of size n, the internally studentized range is the ratio of the sample range to the standard deviation, and may be used in a test of normality. Various applications are discussed.


Keywords:

multiple comparisons;
simultaneous inference;
Newman–Keuls;
Tukey;
normality

Yosef Hochberg

Papers

Improving Some Multiple Comparison Procedures

On optimal1iy criteria in simultaneous interval estimation

On improved calibrations of unknowns in a system of quality-controlled assays

Some Miscellaneous Multiple Comparison Problems

Studentized Range including Neuman‐Keuls and Tukey's T Methods