Topic
Statistical hypothesis testing
About: Statistical hypothesis testing is a research topic. Over the lifetime, 19580 publications have been published within this topic receiving 1037815 citations. The topic is also known as: statistical hypothesis testing & confirmatory data analysis.
Papers published on a yearly basis
Papers
More filters
TL;DR: The retrieval effort hypothesis as discussed by the authors states that difficult but successful retrieevals are better for memory than easier successful retrievals, and it has been shown that the difficulty of retrieval during practice increases, final test performance increases.
434 citations
TL;DR: This paper describes a method for statistical testing based on a Markov chain model of software usage that allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques.
Abstract: Statistical testing of software establishes a basis for statistical inference about a software system's expected field quality. This paper describes a method for statistical testing based on a Markov chain model of software usage. The significance of the Markov chain is twofold. First, it allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques. Analytical results associated with Markov chains facilitate informative analysis of the sequences before they are generated, indicating how the test is likely to unfold. Second, the test input sequences generated from the chain and applied to the software are themselves a stochastic model and are used to create a second Markov chain to encapsulate the history of the test, including any observed failure information. The influence of the failures is assessed through analytical computations on this chain. We also derive a stopping criterion for the testing process based on a comparison of the sequence generating properties of the two chains. >
433 citations
TL;DR: A general framework for assessing predictive stream learning algorithms and defends the use of prequential error with forgetting mechanisms to provide reliable error estimators, and proves that, in stationary data and for consistent learning algorithms, the holdout estimator, the preQUential error and the prequentially error estimated over a sliding window or using fading factors, all converge to the Bayes error.
Abstract: Most streaming decision models evolve continuously over time, run in resource-aware environments, and detect and react to changes in the environment generating data. One important issue, not yet convincingly addressed, is the design of experimental work to evaluate and compare decision models that evolve over time. This paper proposes a general framework for assessing predictive stream learning algorithms. We defend the use of prequential error with forgetting mechanisms to provide reliable error estimators. We prove that, in stationary data and for consistent learning algorithms, the holdout estimator, the prequential error and the prequential error estimated over a sliding window or using fading factors, all converge to the Bayes error. The use of prequential error with forgetting mechanisms reveals to be advantageous in assessing performance and in comparing stream learning algorithms. It is also worthwhile to use the proposed methods for hypothesis testing and for change detection. In a set of experiments in drift scenarios, we evaluate the ability of a standard change detection algorithm to detect change using three prequential error estimators. These experiments point out that the use of forgetting mechanisms (sliding windows or fading factors) are required for fast and efficient change detection. In comparison to sliding windows, fading factors are faster and memoryless, both important requirements for streaming applications. Overall, this paper is a contribution to a discussion on best practice for performance assessment when learning is a continuous process, and the decision models are dynamic and evolve over time.
432 citations
Book•
11 Aug 2008
TL;DR: This paper presents meta-analysis results of a large-scale study of meta-analyses of the effects of various treatments on the severity of depressive symptoms in patients with learning disabilities.
Abstract: Statistical meta-analysis with applications , Statistical meta-analysis with applications , کتابخانه مرکزی دانشگاه علوم پزشکی تهران
431 citations
TL;DR: This work proposes a novel schema for utilizing data to design uncertainty sets for robust optimization using statistical hypothesis tests, and shows that data-driven sets significantly outperform traditional robust optimization techniques whenever data is available.
Abstract: The last decade witnessed an explosion in the availability of data for operations research applications. Motivated by this growing availability, we propose a novel schema for utilizing data to design uncertainty sets for robust optimization using statistical hypothesis tests. The approach is flexible and widely applicable, and robust optimization problems built from our new sets are computationally tractable, both theoretically and practically. Furthermore, optimal solutions to these problems enjoy a strong, finite-sample probabilistic guarantee whenever the constraints and objective function are concave in the uncertainty. We describe concrete procedures for choosing an appropriate set for a given application and applying our approach to multiple uncertain constraints. Computational evidence in portfolio management and queueing confirm that our data-driven sets significantly outperform traditional robust optimization techniques whenever data are available.
430 citations