Showing papers on "Coverage probability published in 2010"

Construction of Optimal Prediction Intervals for Load Forecasting Problems

Abstract: Short-term load forecasting is fundamental for the reliable and efficient operation of power systems. Despite its importance, accurate prediction of loads is problematic and far remote. Often uncertainties significantly degrade performance of load forecasting models. Besides, there is no index available indicating reliability of predicted values. The objective of this study is to construct prediction intervals for future loads instead of forecasting their exact values. The delta technique is applied for constructing prediction intervals for outcomes of neural network models. Some statistical measures are developed for quantitative and comprehensive evaluation of prediction intervals. According to these measures, a new cost function is designed for shortening length of prediction intervals without compromising their coverage probability. Simulated annealing is used for minimization of this cost function and adjustment of neural network parameters. Demonstrated results clearly show that the proposed methods for constructing prediction interval outperforms the traditional delta technique. Besides, it yields prediction intervals that are practically more reliable and useful than exact point predictions.

Bootstrap Inference in Partially Identified Models Defined by Moment Inequalities: Coverage of the Identified Set

Abstract: This paper introduces a novel bootstrap procedure to perform inference in a wide class of partially identi…ed econometric models. We consider econometric models de…ned by …nitely many weak moment inequalities y , which encompass many applications of economic interest. The objective of our inferential procedure is to cover the identi…ed set with a prespeci…ed probability z . We compare our bootstrap procedure, a competing asymptotic approximation and subsampling proce- dures in terms of the rate at which they achieve the desired coverage level, also known as the error in the coverage probability. Under certain conditions, we show that our bootstrap procedure and the asymp- totic approximation have the same order of error in the coverage probability, which is smaller than the one obtained by using subsampling. This implies that inference based on our bootstrap and asymptotic approximation should eventually be more precise than inference based on subsampling. A Monte Carlo study con…rms this …nding in a small sample simulation.

Confidence intervals for random effects meta-analysis and robustness to publication bias

Abstract: The DerSimonian-Laird confidence interval for the average treatment effect in meta-analysis is widely used in practice when there is heterogeneity between studies. However, it is well known that its coverage probability (the probability that the interval actually includes the true value) can be substantially below the target level of 95 per cent. It can also be very sensitive to publication bias. In this paper, we propose a new confidence interval that has better coverage than the DerSimonian-Laird method, and that is less sensitive to publication bias. The key idea is to note that fixed effects estimates are less sensitive to such biases than random effects estimates, since they put relatively more weight on the larger studies and relatively less weight on the smaller studies. Whereas the DerSimonian-Laird interval is centred on a random effects estimate, we centre our confidence interval on a fixed effects estimate, but allow for heterogeneity by including an assessment of the extra uncertainty induced by the random effects setting. Properties of the resulting confidence interval are studied by simulation and compared with other random effects confidence intervals that have been proposed in the literature. An example is briefly discussed.

Simultaneous inference of linear models with time varying coefficients

Abstract: Summary. The paper considers construction of simultaneous confidence tubes for time varying regression coefficients in functional linear models. Using a Gaussian approximation result for non-stationary multiple time series, we show that the constructed simultaneous confidence tubes have asymptotically correct nominal coverage probabilities. Our results are applied to the problem of testing whether the regression coefficients are of certain parametric forms, which is a fundamental problem in the inference of functional linear models. As an application, we analyse an environmental data set and study the association between levels of pollutants and hospital admissions.

Bayesian bivariate meta‐analysis of diagnostic test studies using integrated nested Laplace approximations

Abstract: For bivariate meta-analysis of diagnostic studies, likelihood approaches are very popular. However, they often run into numerical problems with possible non-convergence. In addition, the construction of confidence intervals is controversial. Bayesian methods based on Markov chain Monte Carlo (MCMC) sampling could be used, but are often difficult to implement, and require long running times and diagnostic convergence checks. Recently, a new Bayesian deterministic inference approach for latent Gaussian models using integrated nested Laplace approximations (INLA) has been proposed. With this approach MCMC sampling becomes redundant as the posterior marginal distributions are directly and accurately approximated. By means of a real data set we investigate the influence of the prior information provided and compare the results obtained by INLA, MCMC, and the maximum likelihood procedure SAS PROC NLMIXED. Using a simulation study we further extend the comparison of INLA and SAS PROC NLMIXED by assessing their performance in terms of bias, mean-squared error, coverage probability, and convergence rate. The results indicate that INLA is more stable and gives generally better coverage probabilities for the pooled estimates and less biased estimates of variance parameters. The user-friendliness of INLA is demonstrated by documented R-code.

Bootstrap consistency for general semiparametric M-estimation

Abstract: Consider M-estimation in a semiparametric model that is characterized by a Euclidean parameter of interest and an infinite-dimensional nuisance parameter. As a general purpose approach to statistical inferences, the bootstrap has found wide applications in semiparametric M-estimation and, because of its simplicity, provides an attractive alternative to the inference approach based on the asymptotic distribution theory. The purpose of this paper is to provide theoretical justifications for the use of bootstrap as a semiparametric inferential tool. We show that, under general conditions, the bootstrap is asymptotically consistent in estimating the distribution of the M-estimate of Euclidean parameter; that is, the bootstrap distribution asymptotically imitates the distribution of the M-estimate. We also show that the bootstrap confidence set has the asymptotically correct coverage probability. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to a broad class of bootstrap methods with exchangeable bootstrap weights. This paper provides a first general theoretical study of the bootstrap in semiparametric models.

A prediction interval-based approach to determine optimal structures of neural network metamodels

Abstract: Neural networks have been widely used in literature for metamodeling of complex systems and often outperform their traditional counterparts such as regression-based techniques. Despite proliferation of their applications, determination of their optimal structure is still a challenge, especially if they are developed for prediction and forecasting purposes. Researchers often seek a tradeoff between estimation accuracy and structure complexity of neural networks in a trial and error basis. On the other hand, the neural network point prediction performance considerably drops as the level of complexity and amount of uncertainty increase in systems that data originates from. Motivated by these trends and drawbacks, this research aims at adopting a technique for constructing prediction intervals for point predictions of neural network metamodels. Space search for neural network structures will be defined and confined based on particular features of prediction intervals. Instead of using traditional selection criteria such as mean square error or mean absolute percentage error, prediction interval coverage probability and normalized mean prediction interval will be used for selecting the optimal network structure. The proposed method will be then applied for metamodeling of a highly detailed discrete event simulation model which is essentially a validated virtual representation of a large real world baggage handling system. Through a more than 77% reduction in number of potential candidates, optimal structure for neural networks is found in a manageable time. According to the demonstrated results, constructed prediction intervals using optimal neural network metamodel have a satisfactory coverage probability of targets with a low mean of length.

Exact Likelihood Inference for Two Exponential Populations Under Joint Progressive Type-II Censoring

Abstract: Comparative lifetime experiments are of great importance when the interest is in ascertaining the relative merits of two competing products with regard to their reliability. In this article, we consider two exponential populations and when joint progressive Type-II censoring is implemented on the two samples. We then derive the moment generating functions and the exact distributions of the maximum likelihood estimators (MLEs) of the mean lifetimes of the two exponential populations under such a joint progressive Type-II censoring. We then discuss the exact lower confidence bounds, exact confidence intervals, and simultaneous confidence regions. Next, we discuss the corresponding approximate results based on the asymptotic normality of the MLEs as well as those based on the Bayesian method. All these confidence intervals and regions are then compared by means of Monte Carlo simulations with those obtained from bootstrap methods. Finally, an illustrative example is presented in order to illustrate all the m...

A new tractable model for cellular coverage

Abstract: Cellular networks are usually modeled by placing the base stations according to a regular geometry such as a grid, with the mobile users scattered around the network either as a Poisson point process (i.e. uniform distribution) or deterministically. These models have been used extensively for cellular design and analysis but suffer from being both highly idealized and not very tractable. Thus, complex simulations are used to evaluate key metrics such as coverage probability for a specified target rate (equivalently, the outage probability) or average/sum rate. We develop general models for multi-cell signal-to-noise-plus-interference ratio (SINR) based on homogeneous Poisson point processes and derive the coverage probability, which is one minus the outage probability. Under very general assumptions, the resulting expressions for the SINR cumulative distribution function involve quickly computable integrals, and in some important special cases of practical interest these integrals can be simplified to common integrals (e.g., the Q-function) or even to exact and quite simple closed-form expressions. We compare our coverage predictions to the standard grid model and an actual base station deployment. We observe that the proposed model is pessimistic (a lower bound on coverage) whereas the grid model is optimistic. In addition to being more tractable, the proposed model may better capture the increasingly opportunistic and dense placement of base stations in urban cellular networks with highly variable coverage radii.

Estimation of aquifer scale proportion using equal area grids: assessment of regional scale groundwater quality

Abstract: [1] The proportion of an aquifer with constituent concentrations above a specified threshold (high concentrations) is taken as a nondimensional measure of regional scale water quality. If computed on the basis of area, it can be referred to as the aquifer scale proportion. A spatially unbiased estimate of aquifer scale proportion and a confidence interval for that estimate are obtained through the use of equal area grids and the binomial distribution. Traditionally, the confidence interval for a binomial proportion is computed using either the standard interval or the exact interval. Research from the statistics literature has shown that the standard interval should not be used and that the exact interval is overly conservative. On the basis of coverage probability and interval width, the Jeffreys interval is preferred. If more than one sample per cell is available, cell declustering is used to estimate the aquifer scale proportion, and Kish's design effect may be useful for estimating an effective number of samples. The binomial distribution is also used to quantify the adequacy of a grid with a given number of cells for identifying a small target, defined as a constituent that is present at high concentrations in a small proportion of the aquifer. Case studies illustrate a consistency between approaches that use one well per grid cell and many wells per cell. The methods presented in this paper provide a quantitative basis for designing a sampling program and for utilizing existing data.

Coverage probability of bootstrap confidence intervals in heavy-tailed frequency models, with application to precipitation data.

Abstract: Bootstrap, a technique for determining the accuracy of statistics, is a tool widely used in climatological and hydrological applications. The paper compares coverage probabilities of confidence intervals of high quantiles (5- to 200-year return values) constructed by the nonparametric and parametric bootstrap in frequency analysis of heavy-tailed data, typical for maxima of precipitation amounts. The simulation experiments are based on a wide range of models used for precipitation extremes (generalized extreme value, generalized Pareto, generalized logistic, and mixed distributions). The coverage probability of the confidence intervals is quantified for several sample sizes (n = 20, 40, 60, and 100) and tail behaviors. We show that both bootstrap methods underestimate the width of the confidence intervals but that the parametric bootstrap is clearly superior to the nonparametric one. Even a misspecification of the parametric model—often unavoidable in practice—does not prevent the parametric bootstrap from performing better in most cases. A tendency to narrower confidence intervals from the nonparametric than parametric bootstrap is demonstrated in the application to high quantiles of distributions of observed maxima of 1- and 5-day precipitation amounts; the differences increase with the return level. The results show that estimation of uncertainty based on nonparametric bootstrap is highly unreliable, especially for small and moderate sample sizes and for very heavy-tailed data.

Prediction of order statistics and record values from two independent sequences

Abstract: In this paper, we discuss the problem of predicting future order statistics based on observed record values and similarly, the prediction of future records based on observed order statistics. The coverage probabilities of these intervals are exact and are free of the parent distribution F. Finally, two data sets are used to illustrate the proposed procedures.

On construction of the smallest one-sided confidence interval for the difference of two proportions

Abstract: For any class of one-sided 1 - α confidence intervals with a certain monotonicity ordering on the random confidence limit, the smallest interval, in the sense of the set inclusion for the difference of two proportions of two independent binomial random variables, is constructed based on a direct analysis of coverage probability function. A special ordering on the confidence limit is developed and the corresponding smallest confidence interval is derived. This interval is then applied to identify the minimum effective dose (MED) for binary data in dose-response studies, and a multiple test procedure that controls the familywise error rate at level α is obtained. A generalization of constructing the smallest one-sided confidence interval to other discrete sample spaces is discussed in the presence of nuisance parameters.

Estimators and confidence intervals for the marginal odds ratio using logistic regression and propensity score stratification

Abstract: Propensity score methods are widely used to estimate treatment or exposure effects in observational studies. In studies with binary response the effect can be described as an odds ratio, and the Mantel-Haenszel estimator is traditionally used for stratified data. Although propensity score methods are designed for marginal treatment effects, it has been shown that the Mantel-Haenszel estimator stratified for propensity score is a questionable estimator for the marginal odds ratio, which describes the change in odds of response if everybody versus nobody were treated.We studied recently proposed alternative estimators for the marginal odds ratio, one stratified for propensity score, the other derived from logistic regression. Additionally, we adapted the methodology of the logistic regression based estimator for the derivation of a marginal odds ratio estimator to covariate adjustment by the propensity score. We also derived corresponding variance estimators using the Delta-method.The estimators were illustrated and compared to the inverse probability weighted estimator and the stratified Mantel-Haenszel estimator in a study dealing with respiratory tract infections in children in Germany. Furthermore, simulation studies that were carried out to investigate relative bias, variance and coverage probability showed reasonable performance of marginal odds ratio estimators if response rates or regression based approaches were used. Their variances were accurately estimated. In contrast, the stratified Mantel-Haenszel estimator was substantially biased in some situations due to problems of non-collapsibility and thus it is generally inappropriate for a reliable estimation of the marginal odds ratio.

Comparison of confidence intervals for the Poisson mean

Abstract: Performance of various methods of confidence intervals for the mean of a Poisson distribution is evaluated in terms of coverage probability, expected length (width), p-confidence and p-bias. Methods include an exact procedure, methods based on normal approximations to pivotals, methods based on variance stabilizing transformation and five of these procedures with their bootstrap versions, in stead of the usual normal approximation. Comparison has been made by considering different values of Poisson mean and sample sizes. The nominal confidence coefficient has been taken to be 95%. Comparison is based on simulation studies. Recommendations for use of an appropriate method have been made as per requirements of a user.

On construction of the smallest one-sided confidence interval for the difference of two proportions

Abstract: For any class of one-sided $1-\alpha$ confidence intervals with a certain monotonicity ordering on the random confidence limit, the smallest interval, in the sense of the set inclusion for the difference of two proportions of two independent binomial random variables, is constructed based on a direct analysis of coverage probability function. A special ordering on the confidence limit is developed and the corresponding smallest confidence interval is derived. This interval is then applied to identify the minimum effective dose (MED) for binary data in dose-response studies, and a multiple test procedure that controls the familywise error rate at level $\alpha$ is obtained. A generalization of constructing the smallest one-sided confidence interval to other discrete sample spaces is discussed in the presence of nuisance parameters.

Fixed-Width Sequential Confidence Intervals for a Proportion

Abstract: Fixed-sample-size confidence intervals for a proportion p have widths that vary depending on the observed number of successes. In this article, we develop sequential methods for obtaining fixed-width confidence intervals for p. These methods are exact, and the confidence intervals have the simple form [max(0, p−h), min(1, p+h)], where p is the observed proportion and h is a user-chosen half-width. We consider four possible stopping rules for obtaining the intervals, and we find that a rule based on estimating the variance of p seems to perform best in terms of average run length and coverage probability. The new sequential confidence interval methods provide a valid way to obtain the desired accuracy in sample surveys and Monte Carlo simulation studies without doing excessively many runs. Supplementary material is available online.

Skart: A skewness- and autoregression-adjusted batch-means procedure for simulation analysis

Abstract: Skart is an automated sequential batch-means procedure for constructing a skewness- and autoregression-adjusted confidence interval (CI) for the steady-state mean of a simulation output process either in discrete time (i.e., using observation-based statistics), or in continuous time (i.e., using time-persistent statistics). Skart delivers a CI designed to satisfy user-specified requirements concerning both the CI's coverage probability and its absolute or relative precision. Skart exploits separate adjustments to the classical batch-means CI to account for the effects on the distribution of the underlying Student's t-statistic arising from skewness and autocorrelation of the batch means. The skewness adjustment is based on a Cornish–Fisher expansion for the classical batch-means t-statistic, and the autocorrelation adjustment is based on a first-order autoregressive approximation to the batch-means autocorrelation function. Skart also delivers a point estimator for the steady-state mean that is approximat...

Comparison of Some Parametric and Nonparametric Type One Sample Confidence Intervals for Estimating the Mean of a Positively Skewed Distribution

Abstract: Several researchers considered various interval estimators for estimating the mean of a skewed distribution. Since they considered in different times and under different simulation conditions, their performance are not comparable as a whole. In this article, an attempt has been made to review some existing estimators and compare them under the same simulation condition. In particular, we consider and compare both classical (Student-t, Land-t, Cheb-t, Johnson-t, Chen-t, Hall-t, median-t, Zhou and Dinh, empirical likelihood, etc.) and nonparametric (bootstrap-t, nonparametric bootstrap, empirical likelihood bootstrap, bias corrected acceleration bootstrap, Hall bootstrap-t, empirical Hall bootstrap, etc.) interval estimators for estimating the mean of a positively skewed distribution. A simulation study has been made to compare the performance of the estimators. Both average widths and coverage probabilities are considered as a criterion of the good estimators. Under the large sample sizes, the performances of the estimators are not different. However, they differ significantly when the sample sizes are small and data are from a highly skewed distribution. Some real-life data have been analyzed to illustrate the findings of the article. Based on the simulation study, some possible good interval estimators have been recommended for the practitioners. This article will provide more choices for the practitioners to use best possible interval estimators among many that have been used by several researchers at different times and situations.

Large Sample Inference on the Ratio of Two Independent Binomial Proportions

Abstract: The asymptotic distribution for the ratio of sample pro- portions in two independent bernoulli populations is introduced. The presented method can be used to derive the asymptotic confldence in- terval and hypothesis testing for the ratio of population proportions. The performance of the new interval is comparable with similar confl- dence intervals in the large sample cases. Then the simulation study is provided to compare our confldence interval with some other meth- ods. The proposed confldence set has a good coverage probability with a shorter length.

Load Forecasting and Neural Networks: A Prediction Interval-Based Perspective

Abstract: Successfully determining competitive optimal schedules for electricity generation intimately hinges on the forecasts of loads. The nonstationarity and high volatility of loads make their accurate prediction somewhat problematic. Presence of uncertainty in data also significantly degrades accuracy of point predictions produced by deterministic load forecasting models. Therefore, operation planning utilizing these predictions will be unreliable. This paper aims at developing prediction intervals rather than producing exact point prediction. Prediction intervals are theatrically more reliable and practical than predicted values. The delta and Bayesian techniques for constructing prediction intervals for forecasted loads are implemented here. To objectively and comprehensively assess quality of constructed prediction intervals, a new index based on length and coverage probability of prediction intervals is developed. In experiments with real data, and through calculation of global statistics, it is shown that neural network point prediction performance is unreliable. In contrast, prediction intervals developed using the delta and Bayesian techniques are satisfactorily narrow, with a high coverage probability.

On the tail index of a heavy tailed distribution

Abstract: This paper proposes some new estimators for the tail index of a heavy tailed distribution when only a few largest values are observed within blocks. These estimators are proved to be asymptotically normal under suitable conditions, and their Edgeworth expansions are obtained. Empirical likelihood method is also employed to construct confidence intervals for the tail index. The comparison for the confidence intervals based on the normal approximation and the empirical likelihood method is made in terms of coverage probability and length of the confidence intervals. The simulation study shows that the empirical likelihood method outperforms the normal approximation method.

Sample size determination for confidence intervals of interaction effects in moderated multiple regression with continuous predictor and moderator variables.

Abstract: Moderated multiple regression (MMR) has been widely employed to analyze the interaction or moderating effects in behavior and related disciplines of social science. Much of the methodological literature in the context of MMR concerns statistical power and sample size calculations of hypothesis tests for detecting moderator variables. Notably, interval estimation is a distinct and more informative alternative to significance testing for inference purposes. To facilitate the practice of reporting confidence intervals in MMR analyses, the present article presents two approaches to sample size determinations for precise interval estimation of interaction effects between continuous moderator and predictor variables. One approach provides the necessary sample size so that the designated interval for the least squares estimator of moderating effects attains the specified coverage probability. The other gives the sample size required to ensure, with a given tolerance probability, that a confidence interval of moderating effects with a desired confidence coefficient will be within a specified range. Numerical examples and simulation results are presented to illustrate the usefulness and advantages of the proposed methods that account for the embedded randomness and distributional characteristic of the moderator and predictor variables.

Frequentist evaluation of intervals estimated for a binomial parameter and for the ratio of Poisson means

Abstract: Confidence intervals for a binomial parameter or for the ratio of Poisson means are commonly desired in high energy physics (HEP) applications such as measuring a detection efficiency or branching ratio. Due to the discreteness of the data, in both of these problems the frequentist coverage probability unfortunately depends on the unknown parameter. Trade-offs among desiderata have led to numerous sets of intervals in the statistics literature, while in HEP one typically encounters only the classic intervals of Clopper–Pearson (central intervals with no undercoverage but substantial over-coverage) or a few approximate methods which perform rather poorly. If strict coverage is relaxed, some sort of averaging is needed to compare intervals. In most of the statistics literature, this averaging is over different values of the unknown parameter, which is conceptually problematic from the frequentist point of view in which the unknown parameter is typically fixed. In contrast, we perform an (unconditional) average over observed data in the ratio-of-Poisson-means problem. If strict conditional coverage is desired, we recommend Clopper–Pearson intervals and intervals from inverting the likelihood ratio test (for central and non-central intervals, respectively). Lancaster's mid- P modification to either provides excellent unconditional average coverage in the ratio-of-Poisson-means problem.

Orthodox BLUP versus h-likelihood methods for inferences about random effects in Tweedie mixed models

Abstract: Recently, the orthodox best linear unbiased predictor (BLUP) method was introduced for inference about random effects in Tweedie mixed models. With the use of h-likelihood, we illustrate that the standard likelihood procedures, developed for inference about fixed unknown parameters, can be used for inference about random effects. We show that the necessary standard error for the prediction interval of the random effect can be computed from the Hessian matrix of the h-likelihood. We also show numerically that the h-likelihood provides a prediction interval that maintains a more precise coverage probability than the BLUP method.

Confidence sets based on penalized maximum likelihood estimators in Gaussian regression

Abstract: Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the ‘sparsity property’, the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the ‘sparse’ case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.