Topic
Statistical hypothesis testing
About: Statistical hypothesis testing is a research topic. Over the lifetime, 19580 publications have been published within this topic receiving 1037815 citations. The topic is also known as: statistical hypothesis testing & confirmatory data analysis.
Papers published on a yearly basis
Papers
More filters
06 Nov 2007
TL;DR: It is discovered that there is little practical difference between the randomization, bootstrap, and t tests and their use should be discontinued for measuring the significance of a difference between means.
Abstract: Information retrieval (IR) researchers commonly use three tests of statistical significance: the Student's paired t-test, the Wilcoxon signed rank test, and the sign test. Other researchers have previously proposed using both the bootstrap and Fisher's randomization (permutation) test as non-parametric significance tests for IR but these tests have seen little use. For each of these five tests, we took the ad-hoc retrieval runs submitted to TRECs 3 and 5-8, and for each pair of runs, we measured the statistical significance of the difference in their mean average precision. We discovered that there is little practical difference between the randomization, bootstrap, and t tests. Both the Wilcoxon and sign test have a poor ability to detect significance and have the potential to lead to false detections of significance. The Wilcoxon and sign tests are simplified variants of the randomization test and their use should be discontinued for measuring the significance of a difference between means.
728 citations
TL;DR: A test statistics suggested by Cox is employed to test the adequacy of some statistical models of DNA sequence evolution used in the phylogenetic inference method introduced by Felsentein.
Abstract: Penny et al. have written that "The most fundamental criterion for a scientific method is that the data must, in principle, be able to reject the model. Hardly any [phylogenetic] tree-reconstruction methods meet this simple requirement." The ability to reject models is of such great importance because the results of all phylogenetic analyses depend on their underlying models--to have confidence in the inferences, it is necessary to have confidence in the models. In this paper, a test statistic suggested by Cox is employed to test the adequacy of some statistical models of DNA sequence evolution used in the phylogenetic inference method introduced by Felsenstein. Monte Carlo simulations are used to assess significance levels. The resulting statistical tests provide an objective and very general assessment of all the components of a DNA substitution model; more specific versions of the test are devised to test individual components of a model. In all cases, the new analyses have the additional advantage that values of phylogenetic parameters do not have to be assumed in order to perform the tests.
725 citations
01 Jun 2018
TL;DR: Two One-Sided Tests (TOSTs) as discussed by the authors were used to test both for the presence of an effect and for the absence of a effect in a test set.
Abstract: Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against zero, researchers can use the two one-sided tests (TOST) proce...
721 citations
Book•
01 Jan 1994
TL;DR: In this article, the authors introduce the concept of hypothesis testing with means of samples and the t test for independent means, and make sense of statistical significance: effect size and statistical power.
Abstract: 1. Displaying the order in a group of numbers. 2. Central tendency and variability. 3. Some key ingredients for inferential statistics: Z scores, the normal curve, sample versus population, and probability. 4. Introduction to hypothesis testing. 5. Hypothesis testing with means of samples. 6. Making sense of statistical significance: Effect size and statistical power. 7. Introduction to the t test: Single sample and dependent means. 8. The t test for independent means. 9. Introduction to the analysis of variance. 10. Factorial analysis of variance. 11. Correlation. 12. Prediction. 13. Chi-square tests. 14. Strategies when population distributions are not normal: Data transformations and rank-order tests. 15. Integration and the general linear model. 16. Making sense of advanced statistical procedures in research articles.
718 citations
TL;DR: In this paper, the authors propose unit root tests for large n and T panels in which the cross-sectional units are correlated and derive their asymptotic distribution under the null hypothesis of a unit root and local alternatives.
717 citations