Bias reduction of maximum likelihood estimates
TL;DR: In this paper, the first-order term is removed from the asymptotic bias of maximum likelihood estimates by a suitable modification of the score function, and the effect is to penalize the likelihood by the Jeffreys invariant prior.
Abstract: SUMMARY It is shown how, in regular parametric problems, the first-order term is removed from the asymptotic bias of maximum likelihood estimates by a suitable modification of the score function. In exponential families with canonical parameterization the effect is to penalize the likelihood by the Jeffreys invariant prior. In binomial logistic models, Poisson log linear models and certain other generalized linear models, the Jeffreys prior penalty function can be imposed in standard regression software using a scheme of iterative adjustments to the data.
Citations
More filters
Posted Content•
TL;DR: It is shown that more efficient sampling designs exist for making valid inferences, such as sampling all available events and a tiny fraction of nonevents, which enables scholars to save as much as 99% of their (nonfixed) data collection costs or to collect much more meaningful explanatory variables.
Abstract: We study rare events data, binary dependent variables with dozens to thousands of times fewer ones (events, such as wars, vetoes, cases of political activism, or epidemiological infections) than zeros ("nonevents"). In many literatures, these variables have proven difficult to explain and predict, a problem that seems to have at least two sources. First, popular statistical procedures, such as logistic regression, can sharply underestimate the probability of rare events. We recommend corrections that outperform existing methods and change the estimates of absolute and relative risks by as much as some estimated effects reported in the literature. Second, commonly used data collection strategies are grossly inefficient for rare events data. The fear of collecting data with too few events has led to data collections with huge numbers of observations but relatively few, and poorly measured, explanatory variables, such as in international conflict data with more than a quarter-million dyads, only a few of which are at war. As it turns out, more efficient sampling designs exist for making valid inferences, such as sampling all variable events (e.g., wars) and a tiny fraction of nonevents (peace). This enables scholars to save as much as 99% of their (nonfixed) data collection costs or to collect much more meaningful explanatory variables. We provide methods that link these two results, enabling both types of corrections to work simultaneously, and software that implements the methods developed.
3,170 citations
TL;DR: The authors study rare events data, binary dependent variables with dozens to thousands of times fewer events than zeros (nonevents) and recommend corrections that outperform existing methods and change the estimates of absolute and relative risks by as much as some estimated effects reported in the literature.
Abstract: We study rare events data, binary dependent variables with dozens to thousands of times fewer ones (events, such as wars, vetoes, cases of political activism, or epidemiological infections) than zeros (“nonevents”). In many literatures, these variables have proven difficult to explain and predict, a problem that seems to have at least two sources. First, popular statistical procedures, such as logistic regression, can sharply underestimate the probability of rare events. We recommend corrections that outperform existing methods and change the estimates of absolute and relative risks by as much as some estimated effects reported in the literature. Second, commonly used data collection strategies are grossly inefficient for rare events data. The fear of collecting data with too few events has led to data collections with huge numbers of observations but relatively few, and poorly measured, explanatory variables, such as in international conflict data with more than a quarter-million dyads, only a few of which are at war. As it turns out, more efficient sampling designs exist for making valid inferences, such as sampling all available events (e.g., wars) and a tiny fraction of nonevents (peace). This enables scholars to save as much as 99% of their (nonfixed) data collection costs or to collect much more meaningful explanatory variables. We provide methods that link these two results, enabling both types of corrections to work simultaneously, and software that implements the methods developed.
2,962 citations
University of Maryland, Baltimore1, Centers for Disease Control and Prevention2, International Centre for Diarrhoeal Disease Research, Bangladesh3, Aga Khan University4, Medical Research Council5, University of Barcelona6, GlaxoSmithKline7, United States Department of Veterans Affairs8, Center for Drug Evaluation and Research9, University of Bergen10, Norwegian Institute of Public Health11, Royal Children's Hospital12
TL;DR: Interventions targeting five pathogens can substantially reduce the burden of moderate-to-severe diarrhoea and suggest new methods and accelerated implementation of existing interventions (rotavirus vaccine and zinc) are needed to prevent disease and improve outcomes.
2,766 citations
Book•
29 Mar 2012
TL;DR: The problem of missing data concepts of MCAR, MAR and MNAR simple solutions that do not (always) work multiple imputation in a nutshell and some dangers, some do's and some don'ts are covered.
Abstract: Basics Introduction The problem of missing data Concepts of MCAR, MAR and MNAR Simple solutions that do not (always) work Multiple imputation in a nutshell Goal of the book What the book does not cover Structure of the book Exercises Multiple imputation Historic overview Incomplete data concepts Why and when multiple imputation works Statistical intervals and tests Evaluation criteria When to use multiple imputation How many imputations? Exercises Univariate missing data How to generate multiple imputations Imputation under the normal linear normal Imputation under non-normal distributions Predictive mean matching Categorical data Other data types Classification and regression trees Multilevel data Non-ignorable methods Exercises Multivariate missing data Missing data pattern Issues in multivariate imputation Monotone data imputation Joint Modeling Fully Conditional Specification FCS and JM Conclusion Exercises Imputation in practice Overview of modeling choices Ignorable or non-ignorable? Model form and predictors Derived variables Algorithmic options Diagnostics Conclusion Exercises Analysis of imputed data What to do with the imputed data? Parameter pooling Statistical tests for multiple imputation Stepwise model selection Conclusion Exercises Case studies Measurement issues Too many columns Sensitivity analysis Correct prevalence estimates from self-reported data Enhancing comparability Exercises Selection issues Correcting for selective drop-out Correcting for non-response Exercises Longitudinal data Long and wide format SE Fireworks Disaster Study Time raster imputation Conclusion Exercises Extensions Conclusion Some dangers, some do's and some don'ts Reporting Other applications Future developments Exercises Appendices: Software R S-Plus Stata SAS SPSS Other software References Author Index Subject Index
2,156 citations
TL;DR: A procedure by Firth originally developed to reduce the bias of maximum likelihood estimates is shown to provide an ideal solution to separation and produces finite parameter estimates by means of penalized maximum likelihood estimation.
Abstract: The phenomenon of separation or monotone likelihood is observed in the fitting process of a logistic model if the likelihood converges while at least one parameter estimate diverges to +/- infinity. Separation primarily occurs in small samples with several unbalanced and highly predictive risk factors. A procedure by Firth originally developed to reduce the bias of maximum likelihood estimates is shown to provide an ideal solution to separation. It produces finite parameter estimates by means of penalized maximum likelihood estimation. Corresponding Wald tests and confidence intervals are available but it is shown that penalized likelihood ratio tests and profile penalized likelihood confidence intervals are often preferable. The clear advantage of the procedure over previous options of analysis is impressively demonstrated by the statistical analysis of two cancer studies.
1,628 citations
Cites background or methods from "Bias reduction of maximum likelihoo..."
...By using this modi cation Firth [19] showed that the O(n−1) bias of maximum likelihood...
[...]
...order to reduce the small sample bias of these estimates Firth [19] suggested basing estimation on modi ed score equations...
[...]
...Estimation of standard errors can be based on the roots of the diagonal elements of I(̂)−1, which is a rst-order approximation to {−92 log L∗=(9 )2}−1 (see Firth, reference [19], p....
[...]
...In the following section we rst review some principal ideas of Firth [19], then deal with their implementation in logistic regression (FL), and, nally, suggest con dence intervals based on the pro le penalized likelihood....
[...]
References
More filters
TL;DR: A class of statistical models that generalizes classical linear models-extending them to include many other models useful in statistical analysis, of particular interest for statisticians in medicine, biology, agriculture, social science, and engineering.
Abstract: Addresses a class of statistical models that generalizes classical linear models-extending them to include many other models useful in statistical analysis. Incorporates numerous exercises, both theoretical and data-analytic Discusses quasi-likelihood functions and estimating equations, models for dispersion effect, components of dispersion, and conditional likelihoods Holds particular interest for statisticians in medicine, biology, agriculture, social science, and engineering
5,678 citations
TL;DR: Generalized linear models, 2nd edn By P McCullagh and J A Nelder as mentioned in this paper, 2nd edition, New York: Manning and Hall, 1989 xx + 512 pp £30
Abstract: Generalized Linear Models, 2nd edn By P McCullagh and J A Nelder ISBN 0 412 31760 5 Chapman and Hall, London, 1989 xx + 512 pp £30
5,146 citations
2,463 citations
TL;DR: It is shown that a certain differential form depending on the values of the parameters in a law of chance is invariant for all transformations of the parameter when the law is differentiable with regard to all parameters.
Abstract: It is shown that a certain differential form depending on the values of the parameters in a law of chance is invariant for all transformations of the parameters when the law is differentiable with regard to all parameters. For laws containing a location and a scale parameter a form with a somewhat restricted type of invariance is found even when the law is not everywhere differentiable with regard to the parameters. This form has the properties required to give a general rule for stating the prior probability in a large class of estimation problems.
2,292 citations
TL;DR: In this paper, the authors propose a statisticalique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations, and donne les estimateurs du maximum de VRAISEMblance for les parametres de nuisance.
Abstract: On propose une statistique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations, etant donne les estimateurs du maximum de vraisemblance pour les parametres de nuisance
1,261 citations