scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Accurate confidence intervals for binomial proportion and Poisson rate estimation.

01 Nov 2003-Computers in Biology and Medicine (Elsevier)-Vol. 33, Iss: 6, pp 509-531
TL;DR: Accurate confidence interval estimators for proportions, rates, and their differences are described and MATLAB programs are made available and the resulting confidence intervals are validated and compared to common methods.
About: This article is published in Computers in Biology and Medicine.The article was published on 2003-11-01 and is currently open access. It has received 108 citations till now. The article focuses on the topics: Robust confidence intervals & CDF-based nonparametric confidence interval.
Citations
More filters
Journal ArticleDOI
TL;DR: In this paper, a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small to intermediate samples is presented, revealing the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g., matlab, mathematica, idl, python).
Abstract: I present a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small to intermediate samples. Population proportions arise frequently as quantities of interest in astronomical research; for instance, in studies aiming to constrain the bar fraction, active galactic nucleus fraction, supermassive black hole fraction, merger fraction, or red sequence fraction from counts of galaxies exhibiting distinct morphological features or stellar populations. However, two of the most widely-used techniques for estimating binomial confidence intervals — the ‘normal approximation’ and the Clopper & Pearson approach — are liable to misrepresent the degree of statistical uncertainty present under sampling conditions routinely encountered in astronomical surveys, leading to an ineffective use of the experimental data (and, worse, an inefficient use of the resources expended in obtaining that data). Hence, I provide here an overview of the fundamentals of binomial statistics with two principal aims: (i) to reveal the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g. r, matlab, mathematica, idl, python); and (ii) to demonstrate convincingly the major flaws of both the ‘normal approximation’ and the Clopper & Pearson approach for error estimation.

377 citations


Cites background from "Accurate confidence intervals for b..."

  • ...As explained eloquently by both Kraft et al. (1991) and Ross (2003), there is a fundamental difference between the ‘classical’ and ‘Bayesian’ definitions of the term ‘confidence interval’....

    [...]

  • ...…¼ k=n, for a given value of p is, of course, proportional to pkqn–k. Normalisation of this likelihood function over 0, p, 1 defines a ‘beta distribution’ with integer parameters a¼ kþ 1 and b¼ n kþ 1: Bða; bÞ ¼ ðaþ b 1Þ!ða 1Þ!ðb 1Þ! p a 1qb 1 ð2Þ where q¼ 1 p (e.g., Gelman et al. 2003; Ross 2003)....

    [...]

Book ChapterDOI
17 Mar 2009
TL;DR: This monograph deals with the analysis by simulation of ‘rare situations’ in systems of quite different types, that is, situations that happen very infrequently, but important enough to justify their study.
Abstract: This monograph deals with the analysis by simulation of ‘rare situations’ in systems of quite different types, that is, situations that happen very infrequently, but important enough to justify their study. A rare event is an event occurring with a very small probability, the definition of ‘small’ depending on the application domain. These events are of interest in many areas. Typical examples come, for instance, from transportation systems, where catastrophic failures must be rare enough. For instance, a representative specification for civil aircraft is that the probability of failure must be less than, say, 10−9 during an ‘average-length’ flight (a flight of about 8 hours). Transportation systems are called critical in the dependability area because of the existence of these types of failures, that is, failures that can lead to loss of human life if they occur. Aircraft, trains, subways, all these systems belong to this class. The case of cars is less clear, mainly because the probability of a catastrophic failure is, in many contexts, much higher. Security systems in nuclear plants are also examples of critical systems. Nowadays we also call critical other systems where catastrophic failures may lead to significant loss of money rather than human lives (banking information systems, for example). In telecommunications, modern networks often offer very high speed links. Since information travels in small units or messages (packets in the Internet world, cells in asynchronous transfer mode infrastructures, etc.), the saturation of the memory of a node in the network, even during a small amount of time, may induce a huge amount of losses (in most cases, any unit arriving at

220 citations


Cites methods from "Accurate confidence intervals for b..."

  • ...In this case, there exist specific confidence interval constructions that can replace the one obtained by using the central limit theorem (see for instance [7], typical examples being...

    [...]

Journal ArticleDOI
20 Sep 2012-Nature
TL;DR: The results show that the superior colliculus contributes to visual attention through mechanisms that are independent of the classic effects in the visual cortex, demonstrating that other processes must have key roles in visual attention.
Abstract: The ability to process relevant stimuli selectively is a fundamental function of the primate visual system. The best-understood correlate of this function is the enhanced response of neurons in the visual cortex to attended stimuli. However, recent results show that the superior colliculus (SC), a midbrain structure, also has a crucial role in visual attention. It has been assumed that the SC acts through the same well-known mechanisms in the visual cortex. Here we tested this hypothesis by transiently inactivating the SC during a motion-change-detection task and measuring responses in two visual cortical areas. We found that despite large deficits in visual attention, the enhanced responses of neurons in the visual cortex to attended stimuli were unchanged. These results show that the SC contributes to visual attention through mechanisms that are independent of the classic effects in the visual cortex, demonstrating that other processes must have key roles in visual attention.

219 citations

Posted Content
02 Dec 2010
TL;DR: In this article, a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small-to-intermediate samples is presented, revealing the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g. R, matlab, mathematica, IDL, python).
Abstract: I present a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small-to-intermediate samples. Population proportions arise frequently as quantities of interest in astronomical research; for instance, in studies aiming to constrain the bar fraction, AGN fraction, SMBH fraction, merger fraction, or red sequence fraction from counts of galaxies exhibiting distinct morphological features or stellar populations. However, two of the most widely-used techniques for estimating binomial confidence intervals--the 'normal approximation' and the Clopper & Pearson approach--are liable to misrepresent the degree of statistical uncertainty present under sampling conditions routinely encountered in astronomical surveys, leading to an ineffective use of the experimental data (and, worse, an inefficient use of the resources expended in obtaining that data). Hence, I provide here an overview of the fundamentals of binomial statistics with two principal aims: (i) to reveal the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g. R, matlab, mathematica, IDL, python); and (ii) to demonstrate convincingly the major flaws of both the 'normal approximation' and the Clopper & Pearson approach for error estimation.

211 citations

Journal ArticleDOI
TL;DR: Lower birth weight, shorter gestation, and intraventricular hemorrhage were risk factors for psychiatric problems in the very low birth weight group, and lower Apgar score increased the risk for autism spectrum symptoms and internalizing symptoms.
Abstract: :Objective:To study perinatal risk factors for psychiatric symptoms in adolescents born preterm with very low birth weight or at term, but small for gestational age (GA).Method:Mental health was assessed in 65 adolescents born with very low birth weight (VLBW) (birth weight ≤1500 g), 59 born

118 citations

References
More filters
Journal ArticleDOI

4,095 citations


"Accurate confidence intervals for b..." refers methods in this paper

  • ...6 is the error plot for method 5 (Clopper–Pearson)....

    [...]

  • ...Method 5, Clopper–Pearson CIs: Method 5 is based on Clopper and Pearson [11]....

    [...]

  • ...The implementations of method 5 (Clopper–Pearson based) and method 6 (Normal approximation based) are those of [13] for proportion di erences....

    [...]

  • ...The di erence CIEs (prop−di −ci and rate−di −ci) are much slower, taking tens of seconds in some cases, and the comparison with the Normal and Clopper–Pearson methods is limited to the results reported in [13]....

    [...]

  • ...The CIEs’ accuracies are reported based on a Monte Carlo validated integration of the posterior probability distribution and compared to the normal approximation and Clopper–Pearson methods....

    [...]

Book
01 Jan 1992
TL;DR: In this paper, the authors propose a family of Discrete Distributions, which includes Hypergeometric, Mixture, and Stopped-Sum Distributions (see Section 2.1).
Abstract: Preface. 1. Preliminary Information. 2. Families of Discrete Distributions. 3. Binomial Distributions. 4. Poisson Distributions. 5. Neggative Binomial Distributions. 6. Hypergeometric Distributions. 7. Logarithmic and Lagrangian Distributions. 8. Mixture Distributions. 9. Stopped-Sum Distributions. 10. Matching, Occupancy, Runs, and q-Series Distributions. 11. Parametric Regression Models and Miscellanea. Bibliography. Abbreviations. Index.

2,106 citations

Journal ArticleDOI

1,769 citations


"Accurate confidence intervals for b..." refers methods in this paper

  • ...An alternative Clopper–Pearson method [4] is also well known....

    [...]

Book
01 Jan 1970
TL;DR: The introduction to Probability 2nd Edition Problem Solutions and Probability An Introduction With Statistical Applications, as well as an Introduction to Basic Statistics And Probability.
Abstract: (1966). Introductory Probability and Statistical Applications. Technometrics: Vol. 8, No. 4, pp. 720-722.

351 citations


"Accurate confidence intervals for b..." refers background in this paper

  • ...From the “classical” or sampling theory perspective [2], there is some unknown, but “true” xed value of the estimated parameter....

    [...]