Showing papers on "Bonferroni correction published in 2016"

Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies.

Abstract: Many psychologists do not realize that exploratory use of the popular multiway analysis of variance harbors a multiple-comparison problem. In the case of two factors, three separate null hypotheses are subject to test (i.e., two main effects and one interaction). Consequently, the probability of at least one Type I error (if all null hypotheses are true) is 14 % rather than 5 %, if the three tests are independent. We explain the multiple-comparison problem and demonstrate that researchers almost never correct for it. To mitigate the problem, we describe four remedies: the omnibus F test, control of the familywise error rate, control of the false discovery rate, and preregistration of the hypotheses.

Multiple Hypothesis Testing in Experimental Economics

Abstract: Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives—multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogeneous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate – the probability of one false rejection – and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.

Improving power and accuracy of genome-wide association studies via a multi-locus mixed linear model methodology

Abstract: Genome-wide association studies (GWAS) have been widely used in genetic dissection of complex traits. However, common methods are all based on a fixed-SNP-effect mixed linear model (MLM) and single marker analysis, such as efficient mixed model analysis (EMMA). These methods require Bonferroni correction for multiple tests, which often is too conservative when the number of markers is extremely large. To address this concern, we proposed a random-SNP-effect MLM (RMLM) and a multi-locus RMLM (MRMLM) for GWAS. The RMLM simply treats the SNP-effect as random, but it allows a modified Bonferroni correction to be used to calculate the threshold p value for significance tests. The MRMLM is a multi-locus model including markers selected from the RMLM method with a less stringent selection criterion. Due to the multi-locus nature, no multiple test correction is needed. Simulation studies show that the MRMLM is more powerful in QTN detection and more accurate in QTN effect estimation than the RMLM, which in turn is more powerful and accurate than the EMMA. To demonstrate the new methods, we analyzed six flowering time related traits in Arabidopsis thaliana and detected more genes than previous reported using the EMMA. Therefore, the MRMLM provides an alternative for multi-locus GWAS.

Weighting sequence variants based on their annotation increases power of whole-genome association studies

Abstract: The consensus approach to genome-wide association studies (GWAS) has been to assign equal prior probability of association to all sequence variants tested. However, some sequence variants, such as loss-of-function and missense variants, are more likely than others to affect protein function and are therefore more likely to be causative. Using data from whole-genome sequencing of 2,636 Icelanders and the association results for 96 quantitative and 123 binary phenotypes, we estimated the enrichment of association signals by sequence annotation. We propose a weighted Bonferroni adjustment that controls for the family-wise error rate (FWER), using as weights the enrichment of sequence annotations among association signals. We show that this weighted adjustment increases the power to detect association over the standard Bonferroni correction. We use the enrichment of associations by sequence annotation we have estimated in Iceland to derive significance thresholds for other populations with different numbers and combinations of sequence variants.

The Multiple Testing Issue in Geographically Weighted Regression

Abstract: This article describes the problem of multiple testing within a Geographically Weighted Regression framework and presents a possible solution to the problem which is based on a family-wise error rate for dependent processes. We compare the solution presented here to other solutions such as the Bonferroni correction and the Byrne, Charlton, and Fotheringham proposal which is based on the Benjamini and Hochberg False Discovery Rate. We conclude that our proposed correction is superior to others and that generally some correction in the conventional t-test is necessary to avoid false positives in GWR.

An Efficient Multiple-Testing Adjustment for eQTL Studies that Accounts for Linkage Disequilibrium between Variants

Abstract: Methods for multiple-testing correction in local expression quantitative trait locus (cis-eQTL) studies are a trade-off between statistical power and computational efficiency. Bonferroni correction, though computationally trivial, is overly conservative and fails to account for linkage disequilibrium between variants. Permutation-based methods are more powerful, though computationally far more intensive. We present an alternative correction method called eigenMT, which runs over 500 times faster than permutations and has adjusted p values that closely approximate empirical ones. To achieve this speed while also maintaining the accuracy of permutation-based methods, we estimate the effective number of independent variants tested for association with a particular gene, termed Meff, by using the eigenvalue decomposition of the genotype correlation matrix. We employ a regularized estimator of the correlation matrix to ensure Meff is robust and yields adjusted p values that closely approximate p values from permutations. Finally, using a common genotype matrix, we show that eigenMT can be applied with even greater efficiency to studies across tissues or conditions. Our method provides a simpler, more efficient approach to multiple-testing correction than existing methods and fits within existing pipelines for eQTL discovery.

Extensions of Atanassov's Intuitionistic Fuzzy Interaction Bonferroni Means and Their Application to Multiple-Attribute Decision Making

Abstract: The Bonferroni mean (BM) was originally presented by Bonferroni and had been generalized by many researchers on Atanassov's intuitionistic fuzzy sets (AIFSs) for its capacity to capture the interrelationship between input arguments. Nevertheless, the forms of the combinations of the newly proposed interaction theory on AIFSs with BM are very single, and the existing BMs on AIFSs are not consistent with aggregation operations on the ordinary fuzzy sets. As complements to the existing generalizations of BM under Atanassov's intuitionistic fuzzy environment, this paper develops the extended Atanassov's intuitionistic fuzzy interaction Bonferroni mean (EIFIBM) and the extended weighted Atanassov's intuitionistic fuzzy interaction Bonferroni mean, which can evolve into a series of BMs by taking different generator functions that reflect the different preference attitudes of the decision makers. In addition, some of the EIFIBMs are consistent with aggregation operations on the ordinary fuzzy sets, and some of the EIFIBMs consider the interactions between the membership and nonmembership functions of different Atanassov's intuitionistic fuzzy sets; thus, they can be used in more decision situations. We investigate the properties of these new extensions and apply them to multiple-attribute decision-making problems with admissible orders. Finally, numerical examples show the validity and feasibility of the new approaches.

Bonferroni means with distance measures and the adequacy coefficient in entrepreneurial group theory

Abstract: Bonferroni means with OWA operators and distance measures.Bonferroni means with the adequacy coefficient and the index of maximum and minimum levels.Bonferroni means with moving averages.Bonferroni means in group theory with Moore's families and Galois lattice.An application in an entrepreneurial decision making problem. The aim of the paper is to develop new aggregation operators using Bonferroni means, OWA operators and some distance measure. We introduce the BON-OWAAC and BON-OWAIMAM operators. We are able to include coefficient adequacy and the maximum and minimum levels in the same formulation with Bonferroni means and an OWA operator. The main advantages of using these operators are that they allow consideration of continuous aggregations, multiple comparisons between each argument and distance measures in the same formulation. An application is developed using these new algorithms in combination with Moore's families and Galois lattices to solve group decision-making problems. The professional and personal interests of the entrepreneurs who share co-working spaces are taken as an example for establishing relationships and groups. According to the professional and personal profile affinities for each entrepreneur, the results show dissimilarity and fuzzy relationships and the maximum similarity sub-relations to establish relationships and groups using Moore's families and Galois lattice. Finally, this new type of distance family can be used for applications in areas such as sports teams, strategy marketing and teamwork.

Multiple Hypothesis Testing in Experimental Economics

Abstract: Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives—multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogeneous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate – the probability of one false rejection – and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Steps to prevent SUDEP: the validity of risk factors in the SUDEP and seizure safety checklist: a case control study

Abstract: Our objectives were to compare people with epilepsy (PWE) who died of sudden unexpected death in epilepsy (SUDEP) with live controls using the risk factor items of the SUDEP and Seizure Safety Checklist. All 48 SUDEPs of 93 epilepsy deaths which occurred in Cornwall UK 2004–2012 were compared to 220 live controls using the SUDEP and Seizure Safety Checklist, an evidenced based tool used to communicate person centered risk of SUDEP to PWE. The odds ratio for having a specific factor in those who died was compared to controls and ranked according to P value using a sequential Bonferroni correction for multiple comparisons. Of the 17 modifiable and non-modifiable risk factors analyzed 9 were statistically significant of which 7 are potentially modifiable. Well known modifiable factors such as nocturnal monitoring, compliance and sleeping position featured prominently in the risk association. This is the first case control study exploring the risk factors for SUDEP since 2009. The findings are compared to the current considered risk factors as identified in a major recent review. The study further validates certain SUDEP risk factors. It highlights that the majority of risk factors strongly associated with SUDEP are potentially modifiable. There is an emerging profile to rank the risk factors. It furthers the evidence to use structured risk assessment and communication tools such as the SUDEP and Seizure Safety Checklist in daily clinical practice. It highlights key areas for a person centered discussion to empower PWE to mitigate risk.

On Generalized Extended Bonferroni Means for Decision Making

Abstract: The extended Bonferroni mean (EBM) recently proposed differs from the classical Bonferroni mean, as it aims to capture the heterogeneous interrelationship among the attributes instead of presupposing a homogeneous relation among them. In this study, we generalize the EBM to explicitly and profoundly understand its aggregation mechanism by defining a composite aggregation function. We adopt the approach of optimizing the choice of weighting vectors for the generalized EBM (GEBM) with respect to the least absolute deviation of residuals. We also investigate several desirable properties of the GEBM. Our special interest in this study is to investigate the ability of the GEBM to model mandatory requirements. Finally, the influence of replacing the conjunctive of the GEBM is analyzed to show how the change of the conjunctive affects the global andness and orness of the GEBM. Meanwhile, the aggregation mechanism of the EBM is specified and provided with quite intuitive interpretations for application.

Sampling distribution of the bonferroni inequality index from exponential population

Abstract: SUMMARY After having expressed the Bonferroni index as a ratio of two linear com-binations of order statistics, its exact sampling distribution from exponential population isdeduced 1 IntroductionOver the last twenty years or so, the study of the income inequality hasbecome more and more important and the international scientific communityhas contributed considerably to this topic The Gini concentration ratio (1914)and the Lorenz curve (1905) are undoubtedly the inequality measures whichhave attracted the most interest For an examination of the vast literature onthis subject, see Giorgi (1990, 1992) and Moothathu (1991) However, apartfrom these and other well known indices, there are some measures which havenot received due attention in spite of their having interesting characteristicsOne of these is the Bonferroni (1930) inequality index ( B ) which, unlike theGini ratio, is more sensitive at lower levels of income distribution in as muchas it gives “more weights to transfer among poor” as already shown by Nygardand Sandstr¨om (1981, p 276) This makes the Bonferroni index particularlysuitable for the study of an important aspect of income distribution, viz, themeasurement of the intensity of povertyWe shall here study some of the sampling aspects of the index

Moving Beyond Univariate Post-Hoc Testing in Exercise Science: A Primer on Descriptive Discriminate Analysis

Abstract: There has been a recent call to improve data reporting in kinesiology journals, including the appropriate use of univariate and multivariate analysis techniques. For example, a multivariate analysis of variance (MANOVA) with univariate post hocs and a Bonferroni correction is frequently used to investigate group differences on multiple dependent variables. However, this univariate approach decreases power, increases the risk for Type 1 error, and contradicts the rationale for conducting multivariate tests in the first place. Purpose: The purpose of this study was to provide a user-friendly primer on conducting descriptive discriminant analysis (DDA), which is a post-hoc strategy to MANOVA that takes into account the complex relationships among multiple dependent variables. Method: A real-world example using the Statistical Package for the Social Sciences syntax and data from 1,095 middle school students on their body composition and body image are provided to explain and interpret the results from DDA. Re...

An Efficient Approach to Screening Epigenome-Wide Data.

Abstract: Screening cytosine-phosphate-guanine dinucleotide (CpG) DNA methylation sites in association with some covariate(s) is desired due to high dimensionality. We incorporate surrogate variable analyses (SVAs) into (ordinary or robust) linear regressions and utilize training and testing samples for nested validation to screen CpG sites. SVA is to account for variations in the methylation not explained by the specified covariate(s) and adjust for confounding effects. To make it easier to users, this screening method is built into a user-friendly R package, ttScreening, with efficient algorithms implemented. Various simulations were implemented to examine the robustness and sensitivity of the method compared to the classical approaches controlling for multiple testing: the false discovery rates-based (FDR-based) and the Bonferroni-based methods. The proposed approach in general performs better and has the potential to control both types I and II errors. We applied ttScreening to 383,998 CpG sites in association with maternal smoking, one of the leading factors for cancer risk.

An efficient empirical Bayes method for genomewide association studies

Abstract: Linear mixed model (LMM) is one of the most popular methods for genomewide association studies (GWAS). Numerous forms of LMM have been developed; however, there are two major issues in GWAS that have not been fully addressed before. The two issues are (i) the genomic background noise and (ii) low statistical power after Bonferroni correction. We proposed an empirical Bayes (EB) method by assigning each marker effect a normal prior distribution, resulting in shrinkage estimates of marker effects. We found that such a shrinkage approach can selectively shrink marker effects and reduce the noise level to zero for majority of non-associated markers. In the meantime, the EB method allows us to use an 'effective number of tests' to perform Bonferroni correction for multiple tests. Simulation studies for both human and pig data showed that EB method can significantly increase statistical power compared with the widely used exact GWAS methods, such as GEMMA and FaST-LMM-Select. Real data analyses in human breast cancer identified improved detection signals for markers previously known to be associated with breast cancer. We therefore believe that EB method is a valuable tool for identifying the genetic basis of complex traits.

The precision relationships between eight GWAS-identified genetic variants and breast cancer in a Chinese population.

Abstract: Some of the new breast cancer susceptibility loci discovered in recent Genome-wide association studies (GWASs) have not been confirmed in Chinese populations. To determine whether eight novel Single-Nucleotide Polymorphisms (SNPs) have associations with breast cancer risk in women from southeast China, we conducted a case-control study of 1,156 breast cancer patients and 1,256 healthy controls. We first validated that the SNPs rs12922061, rs2290203, and rs2981578 were associated with overall breast cancer risk in southeast Chinese women, with the per-allele OR of 1.209 (95%CI: 1.064-1.372), 1.176 (95%CI: 1.048-1.320), and 0.852 (95%CI: 0.759-0.956), respectively. Rs12922061 and rs2290203 even passed the threshold for Bonferroni correction (P value: 0.00625). In stratified analysis, we found another three SNPs were significantly associated within different subgroups. However, after Bonferroni correction (P value: 0.000446), there were no statistically significant was observed. In gene-environment interaction analysis, we observed gene-environment interactions played a potential role of in the risk of breast cancer. These findings provide new insight into the associations between the genetic susceptibility and fine classifications of breast cancer. Based on these results, we encourage further large series studies and functional research to confirm these finding.

A Remedy for the Overzealous Bonferroni Technique for Multiple Statistical Tests

Abstract: Ecological research often involves multiple statistical tests. It is common practice to employ the Bonferroni technique or its more advanced sequential variant for such multiple tests. Indeed, Moran (Oikos, 100, 2003, 403) found that 13% of ecological papers apply this technique. The seminal paper by Rice (Evolution, 43, 1989, 223) that introduced this technique to the ecological community, is cited to date over 12 000 times. However, these techniques are conservative and some null hypotheses that should be rejected are not. Using order statistics we find that significant results are correlated even when the data consist of independent events. The Bonferroni methods assume independent significant results which results in Type II error with their application. We propose a simple approach, which we term the correlated Bonferroni technique, to rectify this shortcoming, which reduces rejection of significant results. Ecologists may be able to confirm the significance of their results while they are unable to confirm it using the original Bonferroni technique. Researchers may revisit their projects and find that significant results were mistakenly ignored. We provide an Excel file (see supplement) that researchers can easily use. We illustrate the correlated Bonferroni technique with an example.

Validity of the Hochberg procedure revisited for clinical trial applications.

Abstract: There is much interest in using the Hochberg procedure (HP) for statistical tests on primary endpoints of confirmatory clinical trials. The procedure is simple to use and enjoys more power than the Bonferroni and the Holm procedures. However, the HP is not assumption free like the other two procedures. It controls the familywise type I error rate when test statistics (used for statistical tests) are independent or if dependent satisfy a conditionally independent formulation. Otherwise, its properties for dependent tests at present are not fully understood. Consequently, its use for confirmatory trials, especially for their primary endpoints, remains worrisome. Confirmatory trials are typically designed with 1-2 primary endpoints. Therefore, a question was raised at the Food and Drug Administration as to whether the HP is a valid test for the simple case of performing treatment-to-control comparisons on two primary endpoints when their test statistics are not independent. Confirmatory trials for statistical tests normally use simple test statistics, such as the normal Z, student's t, and chi-square. The literature does include some work on the HP for dependent cases covering these test statistics, but concerns remain regarding its use for confirmatory trials for which endpoint tests are mostly of the dependent kind. The purpose of this paper is therefore to revisit this procedure and provide sufficient details for better understanding of its performance for dependent cases related to the aforementioned question. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

A Framework for Monte Carlo based Multiple Testing

Abstract: We are concerned with a situation in which we would like to test multiple hypotheses with tests whose p-values cannot be computed explicitly but can be approximated using Monte Carlo simulation. This scenario occurs widely in practice. We are interested in obtaining the same rejections and non-rejections as the ones obtained if the p-values for all hypotheses had been available. The present article introduces a framework for this scenario by providing a generic algorithm for a general multiple testing procedure. We establish conditions that guarantee that the rejections and non-rejections obtained through Monte Carlo simulations are identical to the ones obtained with the p-values. Our framework is applicable to a general class of step-up and step-down procedures, which includes many established multiple testing corrections such as the ones of Bonferroni, Holm, Sidak, Hochberg or Benjamini–Hochberg. Moreover, we show how to use our framework to improve algorithms available in the literature in such a way as to yield theoretical guarantees on their results. These modifications can easily be implemented in practice and lead to a particular way of reporting multiple testing results as three sets together with an error bound on their correctness, demonstrated exemplarily using a real biological dataset.

Exact confidence intervals for the average causal effect on a binary outcome.

Abstract: Based on the physical randomization of completely randomized experiments, in a recent article in Statistics in Medicine, Rigdon and Hudgens propose two approaches to obtaining exact confidence intervals for the average causal effect on a binary outcome. They construct the first confidence interval by combining, with the Bonferroni adjustment, the prediction sets for treatment effects among treatment and control groups, and the second one by inverting a series of randomization tests. With sample size n, their second approach requires performing O(n4 )randomization tests. We demonstrate that the physical randomization also justifies other ways to constructing exact confidence intervals that are more computationally efficient. By exploiting recent advances in hypergeometric confidence intervals and the stochastic order information of randomization tests, we propose approaches that either do not need to invoke Monte Carlo or require performing at most O(n2) randomization tests. We provide technical details and R code in the Supporting Information.

Using Scheffé projections for multiple outcomes in an observational study of smoking and periodontal disease

Abstract: In an observational study of the effects caused by treatments, a sensitivity analysis asks about the magnitude of bias from unmeasured covariates that would need to be present to alter the conclusions of a naive analysis that presumes adjustments for measured covariates remove all biases. When there are two or more outcomes in an observational study, these outcomes may be unequally sensitive to unmeasured biases, and the least sensitive finding may concern a combination of several outcomes. A method of sensitivity analysis is proposed using Scheffe projections that permits the investigator to consider all linear contrasts in two or more scored outcomes while controlling the family-wise error rate. In sufficiently large samples, the method will exhibit insensitivity to bias that is greater than or equal to methods, such as the Bonferroni–Holm procedure, that focus on individual outcomes; that is, Scheffe projections have larger design sensitivities. More precisely, if the least sensitive linear combination is a single one of the several outcomes, then the design sensitivity using Scheffe projections equals that using a Bonferroni correction, but if the least sensitive combination is a nontrivial combination of two or more outcomes, then Scheffe projections have larger design sensitivities. This asymptotic property is examined in terms of finite sample power of sensitivity analyses using simulation. The method is applied to a replication with recent data of a well-known study of the effects of smoking on periodontal disease. In the example, the comparison that is least sensitive to bias from unmeasured covariates combines results for lower and upper teeth, but emphasizes lower teeth. This pattern would be difficult to anticipate prior to examining the data, but Scheffe’s method permits use of this unanticipated pattern without fear of capitalizing on chance.

Classical Averaging Functions

Abstract: This chapter presents the classical means, starting with the weighted arithmetic and power means, and then continuing to the quasi-arithmetic means. The topics of generating functions, comparability and weights selection are covered. Several interesting classes of non-quasi-arithmetic means are presented, including Gini, Bonferroni, logarithmic and Bajraktarevic means. Methods of extension of symmetric bivariate means to the multivariate case are also discussed.

Intuitionistic Fuzzy Optimized Weighted Geometric Bonferroni Means and Their Applications in Group Decision Making

Abstract: The geometric Bonferroni mean (GBM) is an important aggrega tion technique which reflects the correlations of aggregated arguments. Based on th e GBM, in this paper, we develop the optimized weighted geometric Bonferroni mean (OWGBM) and t he generalized optimized weighted geometric Bonferroni mean (GOWGBM), whose characteristic s are to reflect the preference and interrelationship of the aggregated arguments. Furthermore , we develop the intuitionistic fuzzy optimized weighted geometric Bonferroni mean (IFOWGBM) and the generalized intuitionistic fuzzy optimized weighted geometric Bonferroni mean (GIFOWGBM), and study their desirable properties such as idempotency, commutativity, monotonicity and boundedness. Finally, based on the IFOWGBM and GIFOWGBM, we present an approach to multi-crite ria decision making and illustrate it with a practical example.

Triangular intuitionistic fuzzy triple bonferroni harmonic mean operators and application to multi-attribute group decision making

Abstract: As an special intuitionistic fuzzy set defined on the real number set, triangular intuitionistic fuzzy number (TIFN) is a fundamental tool for quantifying an ill-known quantity. In order to model the decision maker's overall preference with mandatory requirements, it is necessary to develop some Bonferroni harmonic mean operators for TIFNs which can be used to effectively intergrate the information of attribute values for multi-attribute group decision making (MAGDM) with TIFNs. The purpose of this paper is to develop some Bonferroni harmonic operators of TIFNs and apply to the MAGDM problems with TIFNs. The weighted possibility means of TIFN are firstly defined. Hereby, a new lexicographic approach is presented to rank TIFNs sufficiently considering the risk preference of decision maker. The sensitivity analysis on the risk preference parameter is made. Then, three kinds of triangular intuitionistic fuzzy Bonferroni harmonic aggregation operators are defined, including a triangular intuitionistic fuzzy triple weighted Bonferroni harmonic mean operator (TIFTWBHM) operator, a triangular intuitionistic fuzzy triple ordered weighted Bonferroni harmonic mean (TIFTOWBHM) operator and a triangular intuitionistic fuzzy triple hybrid Bonferroni harmonic mean (TIFTHBHM) operator. Some desirable properties for these operators are discussed in detail. By using the TIFTWBHM operator, we can obtain the individual overall attribute values of alternatives, which are further integrated into the collective ones by the TIFTHBHM operator. The ranking order of alternatives is generated according to the collective overall attribute values of alternatives. A real investment selection case study verifies the validity and applicability of the proposed method.

Comparing Performances of Multiple Comparison Methods in Commonly Used 2 × C Contingency Tables.

Abstract: This study aims at mentioning briefly multiple comparison methods such as Bonferroni, Holm–Bonferroni, Hochberg, Hommel, Marascuilo, Tukey, Benjamini–Hochberg and Gavrilov–Benjamini–Sarkar for contingency tables, through the data obtained from a medical research and examining their performances by simulation study which was constructed as the total 36 scenarios to 2 × 4 contingency table. As results of simulation, it was observed that when the sample size is more than 100, the methods which can preserve the nominal alpha level are Gavrilov–Benjamini–Sarkar, Holm–Bonferroni and Bonferroni. Marascuilo method was found to be a more conservative than Bonferroni. It was found that Type I error rate for Hommel method is around 2 % in all scenarios. Moreover, when the proportions of the three populations are equal and the proportion value of the fourth population is far at a level of ±3 standard deviation from the other populations, the power value for Unadjusted All-Pairwise Comparison approach is at least a bit higher than the ones obtained by Gavrilov–Benjamini–Sarkar, Holm–Bonferroni and Bonferroni. Consequently, Gavrilov–Benjamini–Sarkar and Holm–Bonferroni methods have the best performance according to simulation. Hommel and Marascuilo methods are not recommended to be used because they have medium or lower performance. In addition, we have written a Minitab macro about multiple comparisons for use in scientific research.

Family-Wise Separation Rates for multiple testing

Abstract: Starting from a parallel between some minimax adaptive tests of a single null hypothesis, based on aggregation approaches, and some tests of multiple hypotheses, we propose a new second kind error-related evaluation criterion, as the core of an emergent minimax theory for multiple tests. Aggregation-based tests are justified through their first kind error rate, which is controlled by the prescribed level on the one hand, and through their separation rates over various classes of alternatives, rates that are minimax on the other hand. We show that these tests can be viewed as the first steps of classical step-down multiple testing procedures, and ac-cordingly be evaluated from the multiple testing point of view also, through a control of their Family-Wise Error Rate (FWER). Conversely, many multiple testing procedures, from the historical ones of Bonferroni and Holm, to more recent ones like min-p procedures or randomized procedures, can be investigated from the minimax adaptive testing point of view. To this end, we extend the notion of separation rate to the multiple testing field, by defining the weak Family-Wise Separation Rate and its stronger counterpart, the Family-Wise Separation Rate (FWSR). As for non-parametric tests of a single null hypothesis, we prove that these new concepts allow an accurate analysis of the second kind error of a multiple testing procedure, leading to clear definitions of minimax and minimax adaptive multiple tests. Some illustrations in a classical Gaussian framework corroborate several expected results under particular conditions on the tested hypotheses, but also lead to more surprising results.