Stepwise Multiple Testing as Formalized Data Snooping
Michael Wolf,Joseph P. Romano +1 more
Reads0
Chats0
TLDR
In this paper, a stepwise multiple testing procedure is proposed to asymptotically control the familywise error rate at a desired level, which implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses.Abstract:
It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses. Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.read more
Citations
More filters
Journal ArticleDOI
The Model Confidence Set
TL;DR: The paper revisits the inflation forecasting problem posed by Stock and Watson (1999), and compute the model confidence set (MCS) for their set of inflation forecasts, and compares a number of Taylor rule regressions to determine the MCS of the best in terms of in-sample likelihood criteria.
Journal ArticleDOI
Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects
TL;DR: This article presented a de novo analysis of these experiments, focusing on two core issues that have received limited attention in previous analyses: treatment effect heterogeneity by gender and overrejection of the null hypothesis due to multiple inference.
Journal ArticleDOI
A Test for Superior Predictive Ability
TL;DR: In this article, a new test for superior predictive ability is proposed, which is more powerful and less sensitive to poor and irrelevant alternatives than the Reality Check (RC) for data snooping.
Journal ArticleDOI
Neural responses to ingroup and outgroup members' suffering predict individual differences in costly helping
TL;DR: It is concluded that empathy-related insula activation can motivate costly helping, whereas an antagonistic signal in nucleus accumbens reduces the propensity to help.
References
More filters
Journal ArticleDOI
Controlling the false discovery rate: a practical and powerful approach to multiple testing
Yoav Benjamini,Yosef Hochberg +1 more
TL;DR: In this paper, a different approach to problems of multiple significance testing is presented, which calls for controlling the expected proportion of falsely rejected hypotheses -the false discovery rate, which is equivalent to the FWER when all hypotheses are true but is smaller otherwise.
Journal ArticleDOI
A Simple Sequentially Rejective Multiple Test Procedure
TL;DR: In this paper, a simple and widely accepted multiple test procedure of the sequentially rejective type is presented, i.e. hypotheses are rejected one at a time until no further rejections can be done.
ReportDOI
A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix
Whitney K. Newey,Kenneth D. West +1 more
TL;DR: In this article, a simple method of calculating a heteroskedasticity and autocorrelation consistent covariance matrix that is positive semi-definite by construction is described.
Journal ArticleDOI
Bootstrap Methods: Another Look at the Jackknife
TL;DR: In this article, the authors discuss the problem of estimating the sampling distribution of a pre-specified random variable R(X, F) on the basis of the observed data x.
Journal ArticleDOI
The control of the false discovery rate in multiple testing under dependency
Yoav Benjamini,Daniel Yekutieli +1 more
TL;DR: In this paper, it was shown that a simple FDR controlling procedure for independent test statistics can also control the false discovery rate when test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses.