Showing papers on "Bonferroni correction published in 2010"

Local Spatial Autocorrelation Statistics: Distributional Issues and an Application

Abstract: The statistics Gi(d) and Gi*(d), introduced in Getis and Ord (1992) for the study of local pattern in spatial data, are extended and their properties further explored. In particular, nonbinary weights are allowed and the statistics are related to Moran's autocorrelation statistic, I. The correlations between nearby values of the statistics are derived and verified by simulation. A Bonferroni criterion is used to approximate significance levels when testing extreme values from the set of statistics. An example of the use of the statistics is given using spatial-temporal data on the AIDS epidemic centering on San Francisco. Results indicate that in recent years the disease is intensifying in the counties surrounding the city.

Multiple Comparisons Using R

Abstract: Introduction General Concepts Error rates and general concepts Construction methods Methods based on Bonferroni's inequality Methods based on Simes' inequality Multiple Comparisons in Parametric Models General linear models Extensions to general parametric models The multcomp package Applications Multiple comparisons with a control All pairwise comparisons Dose response analyses Variable selection in regression models Simultaneous confidence bands Multiple comparisons under heteroscedasticity Multiple comparisons in logistic regression models Multiple comparisons in survival models Multiple comparisons in mixed-effects models Further Topics Resampling-based multiple comparison procedures Group sequential and adaptive designs Combining multiple comparisons with modeling Bibliography Index

Accounting for multiple comparisons in a genome-wide association study (GWAS)

Abstract: Background: As we enter an era when testing millions of SNPs in a single gene association study will become the standard, consideration of multiple comparisons is an essential part of determining statistical significance. Bonferroni adjustments can be made but are conservative due to the preponderance of linkage disequilibrium (LD) between genetic markers, and permutation testing is not always a viable option. Three major classes of corrections have been proposed to correct the dependent nature of genetic data in Bonferroni adjustments: permutation testing and related alternatives, principal components analysis (PCA), and analysis of blocks of LD across the genome. We consider seven implementations of these commonly used methods using data from 1514 European American participants genotyped for 700,078 SNPs in a GWAS for AIDS. Results: A Bonferroni correction using the number of LD blocks found by the three algorithms implemented by Haploview resulted in an insufficiently conservative threshold, corresponding to a genome-wide significance level of a = 0.15 - 0.20. We observed a moderate increase in power when using PRESTO, SLIDE, and simpleℳ when compared with traditional Bonferroni methods for population data genotyped on the Affymetrix 6.0 platform in European Americans (a = 0.05 thresholds between 1 × 10 -7 and 7 × 10 -8 ). Conclusions: Correcting for the number of LD blocks resulted in an anti-conservative Bonferroni adjustment. SLIDE and simpleℳ are particularly useful when using a statistical test not handled in optimized permutation testing packages, and genome-wide corrected p-values using SLIDE, are much easier to interpret for consumers of GWAS studies.

The empirical Bayes approach as a tool to identify non-random species associations

Abstract: A statistical challenge in community ecology is to identify segregated and aggregated pairs of species from a binary presence-absence matrix, which often contains hundreds or thousands of such potential pairs. A similar challenge is found in genomics and proteomics, where the expression of thousands of genes in microarrays must be statistically analyzed. Here we adapt the empirical Bayes method to identify statistically significant species pairs in a binary presence-absence matrix. We evaluated the per- formance of a simple confidence interval, a sequential Bonferroni test, and two tests based on the mean and the confidence interval of an empirical Bayes method. Observed patterns were compared to patterns generated from null model randomizations that preserved matrix row and column totals. We evaluated these four methods with random matrices and also with random matrices that had been seeded with an additional segregated or aggregated species pair. The Bayes methods and Bonferroni correc- tions reduced the frequency of false-positive tests (type I error) in random matrices, but did not always correctly identify the non-random pair in a seeded matrix (type II error). All of the methods were vulnerable to identifying spurious secondary associations in the seeded matrices. When applied to a set of 272 published presence-absence matrices, even the most conservative tests indicated a fourfold increase in the frequency of perfectly segregated ''checkerboard'' species pairs compared to the null expectation, and a greater predominance of segregated versus aggregated species pairs. The tests did not reveal a large number of significant species pairs in the Vanuatu bird matrix, but in the much smaller Galapagos bird matrix they correctly identified a concentration of segregated species pairs in the genus Geospiza. The Bayesian methods provide for increased selectivity in identifying non-random species pairs, but the analyses will be most powerful if investigators can use a priori biological criteria to identify potential sets of interacting species.

Controlling false discoveries in multidimensional directional decisions, with applications to gene expression data on ordered categories.

Abstract: Microarray gene expression studies over ordered categories are routinely conducted to gain insights into biological functions of genes and the underlying biological processes. Some common experiments are time-course/dose-response experiments where a tissue or cell line is exposed to different doses and/or durations of time to a chemical. A goal of such studies is to identify gene expression patterns/profiles over the ordered categories. This problem can be formulated as a multiple testing problem where for each gene the null hypothesis of no difference between the successive mean gene expressions is tested and further directional decisions are made if it is rejected. Much of the existing multiple testing procedures are devised for controlling the usual false discovery rate (FDR) rather than the mixed directional FDR (mdFDR), the expected proportion of Type I and directional errors among all rejections. Benjamini and Yekutieli (2005, Journal of the American Statistical Association 100, 71-93) proved that an augmentation of the usual Benjamini-Hochberg (BH) procedure can control the mdFDR while testing simple null hypotheses against two-sided alternatives in terms of one-dimensional parameters. In this article, we consider the problem of controlling the mdFDR involving multidimensional parameters. To deal with this problem, we develop a procedure extending that of Benjamini and Yekutieli based on the Bonferroni test for each gene. A proof is given for its mdFDR control when the underlying test statistics are independent across the genes. The results of a simulation study evaluating its performance under independence as well as under dependence of the underlying test statistics across the genes relative to other relevant procedures are reported. Finally, the proposed methodology is applied to a time-course microarray data obtained by Lobenhofer et al. (2002, Molecular Endocrinology 16, 1215-1229). We identified several important cell-cycle genes, such as DNA replication/repair gene MCM4 and replication factor subunit C2, which were not identified by the previous analyses of the same data by Lobenhofer et al. (2002) and Peddada et al. (2003, Bioinformatics 19, 834-841). Although some of our findings overlap with previous findings, we identify several other genes that complement the results of Lobenhofer et al. (2002).

A hidden Markov random field model for genome-wide association studies

Abstract: Genome-wide association studies (GWAS) are increasingly utilized for identifying novel susceptible genetic variants for complex traits, but there is little consensus on analysis methods for such data. Most commonly used methods include single single nucleotide polymorphism (SNP) analysis or haplotype analysis with Bonferroni correction for multiple comparisons. Since the SNPs in typical GWAS are often in linkage disequilibrium (LD), at least locally, Bonferroni correction of multiple comparisons often leads to conservative error control and therefore lower statistical power. In this paper, we propose a hidden Markov random field model (HMRF) for GWAS analysis based on a weighted LD graph built from the prior LD information among the SNPs and an efficient iterative conditional mode algorithm for estimating the model parameters. This model effectively utilizes the LD information in calculating the posterior probability that an SNP is associated with the disease. These posterior probabilities can then be used to define a false discovery controlling procedure in order to select the disease-associated SNPs. Simulation studies demonstrated the potential gain in power over single SNP analysis. The proposed method is especially effective in identifying SNPs with borderline significance at the single-marker level that nonetheless are in high LD with significant SNPs. In addition, by simultaneously considering the SNPs in LD, the proposed method can also help to reduce the number of false identifications of disease-associated SNPs. We demonstrate the application of the proposed HMRF model using data from a case-control GWAS of neuroblastoma and identify 1 new SNP that is potentially associated with neuroblastoma.

Bonferroni and Gini indices for various parametric families of distributions

Abstract: The Bonferroni index (BI) and Bonferroni curve (BC) have assumed relief not only in economics to study income and poverty, but also in other fields like reliability, demography, insurance and medicine. Besides, the increasingly frequent comparison with the Lorenz curve (LC) and Gini index (GI) both in theoretical and applied studies has driven us to derive explicit expressions for BI, BC, GI and LC for some thirty five continuous distributions. It is expected that these expressions could provide a useful reference and encourage further research within the aforementioned fields.

Powerful Multi-marker Association Tests: Unifying Genomic Distance-Based Regression and Logistic Regression

Abstract: To detect genetic association with common and complex diseases, many statistical tests have been proposed for candidate gene or genome-wide association studies with the case-control design. Due to linkage disequilibrium (LD), multi-marker association tests can gain power over single-marker tests with a Bonferroni multiple testing adjustment. Among many existing multi-marker association tests, most target to detect only one of many possible aspects in distributional differences between the genotypes of cases and controls, such as allele frequency differences, while a few new ones aim to target two or three aspects, all of which can be implemented in logistic regression. In contrast to logistic regression, a genomic-distance based regression (GDBR) approach aims to detect some high-order genotypic differences between cases and controls. A recent study has confirmed the high power of GDBR tests. At this moment, the popular logistic regression and the emerging GDBR approaches are completely unrelated; for example, one has to choose between the two. In this article, we reformulate GDBR as logistic regression, opening a venue to constructing other powerful tests while overcoming some limitations of GDBR. For example, asymptotic distributions can replace time-consuming permutations for deriving p-values, and covariates, including gene-gene interactions, can be easily incorporated. Importantly, this reformulation facilitates combining GDBR with other existing methods in a unified framework of logistic regression. In particular, we show that Fisher’s p-value combining method can boost statistical power by incorporating information from allele frequencies, Hardy-Weinberg disequilibrium (HWD), LD patterns and other higher-order interactions among multi-markers as captured by GDBR.

Testing multiple outcomes in repeated measures designs.

Abstract: This study investigates procedures for controlling the familywise error rate (FWR) when testing hypotheses about multiple, correlated outcome variables in repeated measures (RM) designs. A content analysis of RM research articles published in 4 psychology journals revealed that 3 quarters of studies tested hypotheses about 2 or more outcome variables. Several procedures originally proposed for testing multiple outcomes in 2-group designs are extended to 2-group RM designs. The investigated procedures include 2 modified Bonferroni procedures that adjust the level of significance, alpha, for the effective number of outcomes and a permutation step-down (PSD) procedure. The FWR, any-variable power, and all-variable power are investigated in a Monte Carlo study. One modified Bonferroni procedure frequently resulted in inflated FWRs, whereas the PSD procedure controlled the FWR. The PSD procedure could be substantially more powerful than the conventional Bonferroni procedure, which does not account for dependencies among the outcome variables. However, the difference in power between the PSD procedure, which does account for these dependencies, and Hochberg's step-up procedure, which does not, were negligible. A numeric example illustrates implementation of these multiple-testing procedures.

Meta-analysis of genetic association studies and adjustment for multiple testing of correlated SNPs and traits.

Abstract: Meta-analysis has become a key component of well-designed genetic association studies due to the boost in statistical power achieved by combining results across multiple samples of individuals and the need to validate observed associations in independent studies. Meta-analyses of genetic association studies based on multiple SNPs and traits are subject to the same multiple testing issues as single-sample studies, but it is often difficult to adjust accurately for the multiple tests. Procedures such as Bonferroni may control the type-I error rate but will generally provide an overly harsh correction if SNPs or traits are correlated. Depending on study design, availability of individual-level data, and computational requirements, permutation testing may not be feasible in a meta-analysis framework. In this article, we present methods for adjusting for multiple correlated tests under several study designs commonly employed in meta-analyses of genetic association tests. Our methods are applicable to both prospective meta-analyses in which several samples of individuals are analyzed with the intent to combine results, and retrospective meta-analyses, in which results from published studies are combined, including situations in which (1) individual-level data are unavailable, and (2) different sets of SNPs are genotyped in different studies due to random missingness or two-stage design. We show through simulation that our methods accurately control the rate of type I error and achieve improved power over multiple testing adjustments that do not account for correlation between SNPs or traits. Genet. Epidemiol. 34:739–746, 2010. r 2010 Wiley-Liss, Inc.

A Rank-Based Test for Comparison of Multidimensional Outcomes.

Abstract: For comparison of multiple outcomes commonly encountered in biomedical research, Huang et al. (2005) improved O’Brien’s (1984) rank-sum tests through the replacement of the ad hoc variance by the asymptotic variance of the test statistics. The improved tests control the type I error rate at the desired level and gain power when the differences between the two comparison groups in each outcome variable lie in the same direction; however, they may lose power when the differences are in different directions (e.g., some are positive and some are negative). These tests and the popular Bonferroni correction failed to show important significant differences when applied to compare heart rates from a clinical trial to evaluate the effect of a procedure to remove the cardioprotective solution HTK. We propose an alternative test statistic, taking the maximum of the individual rank-sum statistics, which controls the type I error rate and maintains satisfactory power regardless of the direction of the differences. Sim...

Multiple testing in disease mapping and descriptive epidemiology.

Abstract: The problem of multiple testing is rarely addressed in disease mapping or descriptive epidemiology. This issue is relevant when a large number of small areas or diseases are analysed. Control of the family wise error rate (FWER), for example via the Bonferroni correction, is avoided because it leads to loss of statistical power. To overcome such difficulties, control of the false discovery rate (FDR), the expected proportion of false rejections among all rejected hypotheses, was proposed in the context of clinical trials and genomic data analysis. FDR has a Bayesian interpretation and it is the basis of the so called q-value, the Bayesian counterpart of the p-value. In the present work, we address the multiplicity problem in disease mapping and show the performance of the FDR approach with two real examples and a small simulation study. The examples consider testing multiple diseases for a given area or multiple areas for a given disease. Using unadjusted p-values for multiple testing, an inappropriately large number of areas or diseases at altered risk are identified, whilst FDR procedures are appropriate and more powerful than the control of the FWER with the Bonferroni correction. We conclude that the FDR approach is adequate to screen for high/low risk areas or for disease excess/deficit and useful as a complementary procedure to point estimates and confidence intervals.

On the link between the Bonferroni index and the measurement of inclusive growth

Abstract: In a recent paper Ali and Son (2007) suggested measuring the concept of "inclusive growth" via the use of what they called a "social opportunity function". The latter was assumed to depend on the average opportunities available in the population and to give greater weight to the opportunities enjoyed by the poor. On the basis of this approach Ali and Son (2007) then defined an "opportunity index" and an "opportunity curve". The present paper derives the link which exists between these concepts of "opportunity index" and "opportunity curve" and what is known in the literature as the Bonferroni index and the Bonferroni curve. It also defines what could be called a Bonferroni concentration index, a Bonferroni concentration curve, a Generalized Bonferroni curve and a Generalized Bonferroni concentration curve.

Genetic inheritance and Genome Wide Association statistical test performance

Abstract: The choice of a statistical method significantly affects the power profiles of Genome Wide Association (GWA) predictions. Previous simulation studies of a single synthetic phenotype marker determined that the gene model or mode of inheritance (MOI) was a major influence on power. In this paper, the authors compare the power profiles of GWA statistical methods that combine MOI specific methods into multiple test scenarios against individual methods that may or not assume a MOI gene model consistent with the marker that predicts the association. Combining recessive, additive and dominant individual tests, and using either the Bonferroni Correction method or the MAX test (Li et al., 2008) has power implications with respect to single test GWA-based methods. If the gene model behind the associated phenotype is not known, a multiple test procedure could have significant advantages with respect to single test procedures. Our findings do not provide a specific answer as to which statistical method is best. The best method depends on the MOI gene model associated with the phenotype (diagnosis) in question. However, our results do indicate that the common assumption that the MOI of the locus associated with the diagnosis is additive has consequences. Our results indicate that researchers should consider a multi-test procedure that combines the results of individual MOI-based core tests as a statistical method for conducting the initial screen in a GW study. The process for combining the core tests into a single operational test can occur in a number of ways. We identify two: the Bonferroni procedures and the MAX procedure, each of which produce very similar statistical power profiles.

Asymptotic Bayes optimality under sparsity for generally distributed effect sizes under the alternative

Abstract: Recent results concerning asymptotic Bayes-optimality under sparsity (ABOS) of multiple testing procedures are extended to fairly generally distributed effect sizes under the alternative. An asymptotic framework is considered where both the number of tests m and the sample size m go to infinity, while the fraction p of true alternatives converges to zero. It is shown that under mild restrictions on the loss function nontrivial asymptotic inference is possible only if n increases to infinity at least at the rate of log m. Based on this assumption precise conditions are given under which the Bonferroni correction with nominal Family Wise Error Rate (FWER) level alpha and the Benjamini- Hochberg procedure (BH) at FDR level alpha are asymptotically optimal. When n is proportional to log m then alpha can remain fixed, whereas when n increases to infinity at a quicker rate, then alpha has to converge to zero roughly like n^(-1/2). Under these conditions the Bonferroni correction is ABOS in case of extreme sparsity, while BH adapts well to the unknown level of sparsity. In the second part of this article these optimality results are carried over to model selection in the context of multiple regression with orthogonal regressors. Several modifications of Bayesian Information Criterion are considered, controlling either FWER or FDR, and conditions are provided under which these selection criteria are ABOS. Finally the performance of these criteria is examined in a brief simulation study.

The importance of site location for girth measurements

Abstract: The main aim of this study was to assess the effect of site location on various girth measurements by using a novel method of three-dimensional whole-body scanning. We also wished to identify interactions between distances from the criterion site (site variants), sex, and body mass index (BMI) categories. Two hundred participants were analysed across the sexes and all BMI categories. Girth measurements were extracted 5, 10, 15, and 20 mm distal and proximal to the criterion site. The criterion site was identified by an ISAK-accredited (Level 2) anthropometrist. Error was quantified using the technical error of measurement (TEM). A limit of TEM ≤ 1.0% was applied when determining the practical significance of this error at each site location. Analysis of variance was used to determine the interaction effects between site variants, sex, and BMI categories. Post hoc analysis was completed using t-tests with sequential Bonferroni correction to identify where the significant differences occurred. We f...

Rules for contrast sets

Abstract: In this paper we present a technique to derive rules describing contrast sets. Contrast sets are a formalism to represent groups differences. We propose a novel approach to describe directional contrasts using rules where the contrasting effect is partitioned into pairs of groups. Our approach makes use of a directional Fisher Exact Test to find significant differences across groups. We used a Bonferroni within-search adjustment to control type I errors and a pruning technique to prevent derivation of non significant contrast set specializations.

Controlling Type I Error Rate in Evaluating Differential Item Functioning for Four DIF Methods: Use of Three Procedures for Adjustment of Multiple Item Testing.

Abstract: CONTROLLING TYPE I ERROR RATE IN EVALUATING DIFFERENTIAL ITEM FUNCTIONING FOR FOUR DIF METHODS: USE OF THREE PROCEDURES FOR ADJUSTMENT OF MULTIPLE ITEM TESTING by Jihye Kim In DIF studies, a Type I error refers to the mistake of identifying non-DIF items as DIF items, and a Type I error rate refers to the proportion of Type I errors in a simulation study The possibility of making a Type I error in DIF studies is always present and high possibility of making such an error can weaken the validity of the assessment Therefore, the quality of a test assessment is related to a Type I error rate and to how to control such a rate Current DIF studies regarding a Type I error rate have found that the latter rate can be affected by several factors, such as test length, sample size, test group size, group mean difference, group standard deviation difference, and an underlying model This study focused on another undiscovered factor that may affect a Type I error rate; the effect of multiple testing DIF analysis conducts multiple significance testing of items in a test, and such multiple testing may increase the possibility of making a Type I error at least once The main goal of this dissertation was to investigate how to control a Type I error rate using adjustment procedures for multiple testing which have been widely used in applied statistics but rarely used in DIF studies In the simulation study, four DIF methods were performed under a total of 36 testing conditions; the methods were the Mantel-Haenszel method, the logistic regression procedure, the Differential Functioning Item and Test framework, and the Lord’s chisquare test Then the Bonferroni correction, the Holm’s procedure, and the BH method were applied as an adjustment of multiple significance testing The results of this study showed the effectiveness of three adjustment procedures in controlling a Type I error rate CONTROLLING TYPE I ERROR RATE IN EVALUATING DIFFERENTIAL ITEM FUNCTIONING FOR FOUR DIF METHODS: USE OF THREE PROCEDURES FOR ADJUSTMENT OF MULTIPLE ITEM TESTING by Jihye Kim

A comparison of methods for the construction of confidence interval for relative risk in stratified matched‐pair designs

Abstract: A stratified matched-pair study is often designed for adjusting a confounding effect or effect of different trails/centers/ groups in modern medical studies. The relative risk is one of the most frequently used indices in comparing efficiency of two treatments in clinical trials. In this paper, we propose seven confidence interval estimators for the common relative risk and three simultaneous confidence interval estimators for the relative risks in stratified matched-pair designs. The performance of the proposed methods is evaluated with respect to their type I error rates, powers, coverage probabilities, and expected widths. Our empirical results show that the percentile bootstrap confidence interval and bootstrap-resampling-based Bonferroni simultaneous confidence interval behave satisfactorily for small to large sample sizes in the sense that (i) their empirical coverage probabilities can be well controlled around the pre-specified nominal confidence level with reasonably shorter confidence widths; and (ii) the empirical type I error rates of their associated test statistics are generally closer to the pre-specified nominal level with larger powers. They are hence recommended. Two real examples from clinical laboratory studies are used to illustrate the proposed methodologies.

Correction for Multiplicity in Genetic Association Studies of Triads: The Permutational TDT

Abstract: New technology for large-scale genotyping has created new challenges for statistical analysis. Correcting for multiple comparison without discarding true positive results and extending methods to triad studies are among the important problems facing statisticians. We present a one-sample permutation test for testing transmission disequilibrium hypotheses in triad studies, and show how this test can be used for multiple single nucleotide polymorphism (SNP) testing. The resulting multiple comparison procedure is shown in the case of the transmission disequilibrium test to control the familywise error. Furthermore, this procedure can handle multiple possible modes of risk inheritance per SNP. The resulting permutational procedure is shown through simulation of SNP data to be more powerful than the Bonferroni procedure when the SNPs are in linkage disequilibrium. Moreover, permutations implicitly avoid any multiple comparison correction penalties when the SNP has a rare allele. The method is illustrated by analyzing a large candidate gene study of neural tube defects and an independent study of oral clefts, where the smallest adjusted p-values using the permutation procedure are approximately half those of the Bonferroni procedure. We conclude that permutation tests are more powerful for identifying disease-associated SNPs in candidate gene studies and are useful for analysis of triad studies.

Asymptotic Bayes optimality under sparsity of selection rules for general priors

Abstract: Recent results concerning asymptotic Bayes-optimality under sparsity (ABOS) of multiple testing procedures are extended to fairly generally distributed effect sizes under the alternative. An asymptotic framework is considered where both the number of tests m and the sample size m go to infinity, while the fraction p of true alternatives converges to zero. It is shown that under mild restrictions on the loss function nontrivial asymptotic inference is possible only if n increases to infinity at least at the rate of log m. Based on this assumption precise conditions are given under which the Bonferroni correction with nominal Family Wise Error Rate (FWER) level alpha and the Benjamini- Hochberg procedure (BH) at FDR level alpha are asymptotically optimal. When n is proportional to log m then alpha can remain fixed, whereas when n increases to infinity at a quicker rate, then alpha has to converge to zero roughly like n^(-1/2). Under these conditions the Bonferroni correction is ABOS in case of extreme sparsity, while BH adapts well to the unknown level of sparsity. In the second part of this article these optimality results are carried over to model selection in the context of multiple regression with orthogonal regressors. Several modifications of Bayesian Information Criterion are considered, controlling either FWER or FDR, and conditions are provided under which these selection criteria are ABOS. Finally the performance of these criteria is examined in a brief simulation study.

Detecting gene-gene interactions using support vector machines with L 1 penalty

Abstract: Interactions among multiple genetic variants are likely to affect risk for human complex disease. It is increasingly recognized that the identification of interactions will not only increase the power to detect disease-associated variants, but will also help elucidate biological pathways that underlie diseases. In this article, we propose a two-stage method for detecting gene-gene interactions. In the first stage, using a model selection method, that is, support vector machines (SVM) with L 1 penalty, we identify the most promising single-nucleotide polymorphisms (SNPs) and interactions. In the second stage, we apply logistic regression and ensure a valid type I error by excluding non-significant candidates after Bonferroni correction. We analyze a published case-control dataset where our method successfully identified an interaction term which was not discovered in previous studies

SCORE Study Report 8: Closed Tests for All Pairwise Comparisons of Means

Abstract: We compare five closed tests for strong control of family-wide type 1 error while making all pairwise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used P values from chi-squared tests to compare performance of a pairwise closed test described below to Bonferroni and Hochberg adjusted P values. Pairwise closed testing was more powerful than Hochberg’s method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for P values based on normal, logistic, and Poisson distributions. The power benefit of pairwise closed testing begins to disappear with five or more arms and with unbalanced designs. For trials with four or fewer arms and balanced designs, investigators should consider using pairwise closed testing in preference to Shaffer’s, Hommel’s, and Hochberg’s approaches when making all pairwise comparisons of means. If not all P values from the closed family are available, Shaffer’s method is a good choice.

Multiple Testing in Genetic Epidemiology

Abstract: If multiple statistical tests are performed simultaneously, we should take into account multiple testing to properly control the false positive rate. In association studies performing statistical tests for a large number of correlated markers, the traditional Sidak correction is overly conservative and the permutation test is inefficient. This article discusses recently proposed approaches for correcting for multiple testing in association studies. We first explain basic concepts of statistics such as the p-value, false-positive rate, corrected p-value and family wise error rate. Then we discuss recently proposed methods in three categories: methods using multivariate normal distribution, methods calculating the effective number of tests and methods increasing the efficiency of permutation test. We compare the relative performance of these methods. Many of the methods are shown to be highly efficient and accurate compared to the traditional approaches and can readily be applied to the genome-wide datasets. Key Concepts: In a statistical testing procedure performing multiple tests, multiple testing must be taken into account to properly control the false positive rate. In association studies, the traditional Sidak correction is overly conservative and the permutation test is inefficient. Recently proposed multiple testing correction methods are highly efficient and accurate and can be applied to the genome-wide datasets. Keywords: multiple testing; statistical test; p-value correction; association study; false positive; family wise error rate; false discovery rate; permutation test; Bonferroni correction; multivariate normal distribution

Efficient statistical analysis of large correlated multivariate datasets: a case study on brain connectivity matrices

Abstract: In neuroimaging, a large number of correlated tests are routinely performed to detect active voxels in single-subject experiments or to detect regions that differ between individuals belonging to different groups. In order to bound the probability of a false discovery of pair-wise differences, a Bonferroni or other correction for multiplicity is necessary. These corrections greatly reduce the power of the comparisons which means that small signals (differences) remain hidden and therefore have been more or less successful depending on the application. We introduce a method that improves the power of a family of correlated statistical tests by reducing their number in an orderly fashion using our a-priori understanding of the problem . The tests are grouped by blocks that respect the data structure and only one or a few tests per group are performed. For each block we construct an appropriate summary statistic that characterizes a meaningful feature of the block. The comparisons are based on these summary statistics by a block-wise approach. We contrast this method with the one based on the individual measures in terms of power. Finally, we apply the method to compare brain connectivity matrices. Although the method is used in this study on the particular case of imaging, the proposed strategy can be applied to a large variety of problems that involves multiple comparisons when the tests can be grouped according to attributes that depend on the specific problem. Keywords and phrases: Multiple comparisons ; Family-wise error rate; False discovery rate; Bonferroni procedure; Human brain connectivity; Brain connectivity matrices.

An Improved Ad-Hoc Multiplicity Adjustment Method for Correlated Response Variables

Abstract: Clinical trials and many other scientific studies perform statistical tests for multiple response variables for finding whether an experimental treatment of interest is better than a specified control for some of these variables. Multiplicity adjustments for such tests of multiple response variables, having varying degrees of correlations, have been a challenging task for achieving a meaningful control of the Type I error rate. A way out of this difficulty has been simply to ignore correlations among response variables and use methods such as the Bonferroni procedure or the Sidak (1967) adjustment formula under independence. This, however, can make the multiplicity adjustments conservative when the correlations are in the moderate to high range and reduce the power of the tests for finding beneficial treatment effects. On the other hand, Sidak’s adjustment formula under independence is quite attractive; it’s easy to use for multiplicity adjustments and for setting simultaneous confidence intervals. Ther...