Showing papers in "International Statistical Review in 2012"

Short‐Term Wind Speed Forecasting for Power System Operations

Abstract: Summary The emphasis on renewable energy and concerns about the environment have led to large-scale wind energy penetration worldwide. However, there are also significant challenges associated with the use of wind energy due to the intermittent and unstable nature of wind. High-quality short-term wind speed forecasting is critical to reliable and secure power system operations. This article begins with an overview of the current status of worldwide wind power developments and future trends. It then reviews some statistical short-term wind speed forecasting models, including traditional time series approaches and more advanced space–time statistical models. It also discusses the evaluation of forecast accuracy, in particular, the need for realistic loss functions. New challenges in wind speed forecasting regarding ramp events and offshore wind farms are also presented.

Assessing the Performance of Classification Methods

Abstract: Summary A large number of measures have been developed for evaluating the performance of classification rules. Some of these have been developed to meet the practical requirements of specific applications, but many others—which here we call “classification accuracy” criteria—represent different ways of balancing the different kinds of misclassification which may be made. This paper reviews classification accuracy criteria. However, the literature is now so large and diverse that a comprehensive list, covering all the measures and their variants, would probably be impossible. Instead, this paper embeds such measures in general framework, spanning the possibilities, and draws attention to relationships between them. Important points to note are, firstly, that different performance measures, by definition, measure different aspects of performance; secondly, that one should therefore carefully choose a measure to match the objectives of one's study; and, thirdly, that empirical comparisons between instruments measuring different aspects are of limited value.

Evaluating, Comparing, Monitoring, and Improving Representativeness of Survey Response Through R-Indicators and Partial R-Indicators

Abstract: Summary Non-response is a common source of error in many surveys Because surveys often are costly instruments, quality-cost trade-offs play a continuing role in the design and analysis of surveys The advances of telephone, computers, and Internet all had and still have considerable impact on the design of surveys Recently, a strong focus on methods for survey data collection monitoring and tailoring has emerged as a new paradigm to efficiently reduce non-response error Paradata and adaptive survey designs are key words in these new developments Prerequisites to evaluating, comparing, monitoring, and improving quality of survey response are a conceptual framework for representative survey response, indicators to measure deviations thereof, and indicators to identify subpopulationsthatneedincreasedeffortInthispaper,wepresentanoverviewofrepresentativeness indicators or R-indicators that are fit for these purposes We give several examples and provide guidelines for their use in practice

On the Generalized Bootstrap for Sample Surveys with Special Attention to Poisson Sampling

Abstract: Resume Nous etudions la technique du bootstrap generalise pour des plans de sondage generaux Nous nous concentrons principalement sur l’estimation bootstrap de la variance mais nous etudions egalement les proprietes empiriques des intervalles de confiance bootstrap obtenus en utilisant la methode des percentiles Le bootstrap generalise consiste a generer aleatoirement des poids bootstrap de telle sorte que les deux (ou plus) premiers moments selon le plan de l’erreur d’echantillonnage soient approches par leurs moments correspondants selon le mecanisme bootstrap On peut voir la plupart des methodes bootstrap dans la litterature comme etant des cas particuliers du bootstrap generalise Nous discutons de considerations telles que le choix de la distribution utilisee pour generer les poids bootstrap, le choix du nombre de repliques bootstrap et la presence possible de poids bootstrap negatifs Nous decrivons d’abord le bootstrap generalise pour l’estimateur lineaire de Horvitz-Thompson et considerons ensuite les estimateurs non lineaires tels que ceux definis au moyen d’equations d’estimation Nous developpons egalement deux facons d’appliquer le bootstrap a l’estimateur par la regression generalisee du total d’une population Nous etudions plus en profondeur le cas de l’echantillonnage de Poisson qui est souvent utilise pour selectionner des echantillons dans les enquetes sur les indices de prix effectuees par les agences statistiques nationales dans le monde Pour l’echantillonnage de Poisson, nous considerons une approche par pseudo-population et montrons que les poids bootstrap qui en resultent capturent les trois premiers moments sous le plan de l’erreur d’echantillonnage Nous utilisons une etude par simulation et un exemple avec des donnees d’enquetes reelles pour illustrer la theorie Summary We study the generalized bootstrap technique under general sampling designs We focus mainly on bootstrap variance estimation but we also investigate the empirical properties of bootstrap confidence intervals obtained using the percentile method Generalized bootstrap consists of randomly generating bootstrap weights so that the first two (or more) design moments of the sampling error are tracked by the corresponding bootstrap moments Most bootstrap methods in the literature can be viewed as special cases We discuss issues such as the choice of the distribution used to generate bootstrap weights, the choice of the number of bootstrap replicates, and the potential occurrence of negative bootstrap weights We first describe the generalized bootstrap for the linear Horvitz-Thompson estimator and then consider non-linear estimators such as those defined through estimating equations We also develop two ways of bootstrapping the generalized regression estimator of a population total We study in greater depth the case of Poisson sampling, which is often used to select samples in Price Index surveys conducted by national statistical agencies around the world For Poisson sampling, we consider a pseudo-population approach and show that the resulting bootstrap weights capture the first three design moments of the sampling error A simulation study and an example with real survey data are used to illustrate the theory

On Invariant Coordinate System (ICS) Functionals

Abstract: Resume Les problemes d'equivariance et d'invariance sont frequents en analyse multivariee, o des adaptations sont parfois necessaires en vue de l'obtention de versions equivariantes ou invariantes des procedures utilisees. Ces adaptations reposent souvent sur un traitement preliminaire des donnees, standardisation ou transformation via un systeme de coordonnees invariantes (ICS). Dans cet article, les methodes de standardisation, ainsi que les caracteristiques des fonctionnelles et statistiques utilisees dans le cadre des ICS et leurs invariances sous l'action de certains groupes de transformations, sont examinees. En particulier, la methode de construction fondee sur l'utilisation de deux matrices de scatter, developpee par Tyler et al. (2009), ainsi que la methode directe suggeree par Chaudhuri & Sengupta (1993), sont examinees. Diverses applications sont discutees. Summary Equivariance and invariance issues often arise in multivariate statistical analysis. Statistical procedures have to be modified sometimes to obtain an affine equivariant or invariant version. This is often done by preprocessing the data, e.g., by standardizing the multivariate data or by transforming the data to an invariant coordinate system (ICS). In this paper, standardization of multivariate distributions and characteristics of ICS functionals and statistics are examined. Also, invariances up to some groups of transformations are discussed. Constructions of ICS functionals are addressed. In particular, the construction based on the use of two scatter matrix functionals presented by Tyler et al. (2009) and direct definitions based on the approach presented by Chaudhuri & Sengupta (1993) are examined. Diverse applications of ICS functionals are discussed.

Statistical Disclosure Risk: Separating Potential and Harm

Abstract: Statistical agencies are keen to devise ways to provide research access to data while protecting confidentiality. Although methods of statistical disclosure risk assessment are now well established in the statistical science literature, the integration of these methods by agencies into a general scientific basis for their practice still proves difficult. This paper seeks to review and clarify the role of statistical science in the conceptual foundations of disclosure risk assessment in an agency's decision making. Disclosure risk is broken down into disclosure potential, a measure of the ability to achieve true disclosure, and disclosure harm. It is argued that statistical science is most suited to assessing the former. A framework for this assessment is presented. The paper argues that the intruder's decision making and behaviour may be separated from this framework, provided appropriate account is taken of the nature of potential intruder attacks in the definition of disclosure potential

Model assisted estimation of a spatial population mean

Abstract: Resume Cet article traite de l'estimation de la moyenne d'une population spatiale. Dans le cadre d'une approche fondee sur un plan d'echantillonnage, un estimateur assiste par un modle de regression spline penalise est propose et etudie. Nous montrons que cet estimateur est convergent (dans le cadre du plan) et etablissons sa loi normale asymptotique. Une etude de simulation est menee afin d'etudier ses performances et l'estimation de sa variance, ainsi que les questions liees au biais relatif, a l'efficacite, et au taux de convergence des probabilites de couverture des intervalles de confiance correspondants. Ces simulations indiquent des gains d'efficacite considerables par rapport aux estimateurs decoulant des methodes d'echantillonnage classiques. Summary This paper deals with the estimation of the mean of a spatial population. Under a design-based approach to inference, an estimator assisted by a penalized spline regression model is proposed and studied. Proof that the estimator is design-consistent and has a normal limiting distribution is provided. A simulation study is carried out to investigate the performance of the new estimator and its variance estimator, in terms of relative bias, efficiency, and confidence interval coverage rate. The results show that gains in efficiency over standard estimators in classical sampling theory may be impressive.

A Method to Evaluate Mode Effects on the Mean and Variance of a Continuous Variable in Mixed-Mode Surveys

Abstract: Resume Les enquetes a mode de collecte mixte, dans lesquelles le choix du mode de reponse est laisse aux personnes interrogees, sont de plus en plus repandues. Cette possibilite du choix du mode de reponse peut induire une confusion entre deux sortes d'effets : les effets de selection et les effets de mesure. L'estimation separee de ces deux effets est impossible a partir des seules donnees d'une enquete a mode de collecte mixte. Dans cet article, nous montrons comment la meme estimation devient possible a partir d'une comparaison entre les resultats d'une enquete a mode de collecte mixte et ceux d'un enquete a mode de collecte unique basees sur le meme questionnaire. La methode que nous proposons permet d'estimer l'effet du mode de reponse sur la moyenne et la variance d'une variable de reponse continue. Elle est illustree par l'estimation des effets du mode de reponse sur six questions relatives aux opinions vis-a-vis des enquetes. Summary Mixed-mode surveys, in which different respondents complete the survey by different modes, become increasingly popular. That said, such surveys may lead to a confounding of two forms of mode effects, i.e., selection effects and measurement effects. Exclusive focus on mixed-mode data almost precludes disentangling both effects. In this paper, we show how this problem can be circumvented merely by comparing mixed-mode data with a comparable single-mode sample. The proposed method allows estimating the mode effects on the mean and variance of a continuous variable. As an illustration, the authors estimate mode effects on six items on opinions about surveys.

What Happened to Discrete Chaos, the Quenouille Process, and the Sharp Markov Property? Some History of Stochastic Point Processes

Abstract: The use of properties of a Poisson process to study the randomness of stars is traced back to a 1767 paper. The process was used and rediscovered many times, and we mention some of the early scientific areas. The name Poisson process was first used in print in 1940, and we believe the term was coined in the corridors of Stockholm University some time between 1936 and 1939. We follow the early developments of doubly stochastic processes and cluster processes, and describe different efforts to apply the Markov property to point processes.

Using complex surveys to estimate the L1-median of a functional variable: Application to electricity load curves

Abstract: Mean proles are widely used as indicators of the electricity consumption habits of customers. Currently, Electricit e De France (EDF), estimates class load proles by using point-wise mean function. Unfortunately, it is well known that the mean is highly sensitive to the presence of outliers, such as one or more consumers with unusually high-levels of consumption. In this paper, we propose an alternative to the mean prole: the L1-median prole which is more robust. When dealing with large datasets of functional data (load curves for example), survey sampling approaches are useful for estimating the median prole and avoid storing all of the data. We propose here estimators of the median trajectory using several sampling strategies and estimators. A comparison between them is illustrated by means of a test population. We develop a stratication based on the linearized variable which substantially improves the accuracy of the estimator compared to simple random sampling without replacement. We suggest also an improved estimator that takes into account auxiliary information. Some potential areas for future research are also highlighted.

Asymptotically Optimal Response‐Adaptive Designs for Allocating the Best Treatment: An Overview

Abstract: Resume Les plans d’experiences qui s’adaptent aux reponses deja acquises sont de plus en plus frequemment utilises dans les applications, et c’est tout particulierement vrai pour les essais cliniques en phase precoce. Cet article est une revue dediee a une classe de plans adaptatifs qui ont pour propriete de selectionner le meilleur traitement avec une probabilite tendant vers un. D’un point de vue ethique, ceci est une propriete souhaitable pour les essais cliniques. Pour de tels plans, le modele sous-jacent est un modele d’urne aleatoirement renforcee. Cet article presente un panorama des resultats concernant ces plans, allant de l’article seminal de Durham et Yu (1990) jusqu’au travail recent de Flournoy et al. (2010). Summary Response-adaptive designs are being used increasingly in applications, and this is especially so in early phase clinical trials. This paper reviews a particular class of response-adaptive designs that have the property of picking the superior treatment with probability tending to one. This is a desirable property from an ethical point of view in clinical trials. The model underlying such designs is a randomly reinforced urn. This paper provides an overview of results for these designs, starting from the early paper of Durham and Yu (1990) until the recent work by Flournoy, May, Moler and Plo (2010).

Modelling and Forecasting Gas Flow on Exits of Gas Transmission Networks

Abstract: Resume Les flux de gaz naturel dans un reseau de transport sont etudies dans une perspective d'optimisation. L'analyse des donnees reelles fournit une comprehension meilleure du comportement du gaz dans ces reseaux. Plusieurs modeles de la dependance entre flux quotidien maximal et temperature aux points de sortie sont proposes. Dans la classe des modeles parametriques, une regression sigmoide modifiee est selectionnee. Dans un cadre semi-parametrique, c'est un modele de regression fonde sur les splines penalises qui est choisi. La comparaison des modeles et des previsions qu'ils fournissent en basses temperatures est egalement consideree. L'application des resultats est discutee. Summary The flow of natural gas within a gas transmission network is studied with the aim to optimize such networks. The analysis of real data provides a deeper insight into the behaviour of gas in- and outflow. Several models for describing dependence between the maximal daily gas flow and the temperature on network exits are proposed. A modified sigmoidal regression is chosen from the class of parametric models. As an alternative, a semi-parametric regression model based on penalized splines is considered. The comparison of models and the forecast of gas loads for very low temperatures based on both approaches is included. The application of the obtained results is discussed.

A Special Gen(d)re of Statistics: Roots, Development and Methodological Prospects of Gender Statistics

Abstract: Summary The very expression Gender Statistics calls for a double interpretation. It accounts for the popular mix-up of statistical methodology with its typical products such as indexes, tables and graphs. At the same time it implies a broader and forward-looking perspective, which is inspired by the increasing demand of gender sensitive statistical information coming from society, official agencies, economy. Gender statistics stands as a proper independent field of statistics with its own objectives and a variety of applications in social, human and life science. Concurrently an emerging necessity of appropriate equipment of methods and dissemination tools is noticeable. The paper tracks the roots and the historical development of gender statistics, reviews critically the existing indexes and practice and outlines methodological needs and research prospects. Resume L'expression “Gender Statistics/Statistiques de Genres” suscite une double interpretation. Elle recouvre un melange de methodes statistiques courantes, dont les elements les plus representatifs sont des constructions d'indices, de tables et de graphes. En meme temps, la “Statistique de Genres” requiert une vision plus large, inspiree par la demande croissante, par les entreprises et les agences officielles, d'informations statistiques “sensibles” du point de vue des genres. La “Statistique de Genres”emerge ainsi comme un sujet independant, possedant ses propres objectifs et ses propres applications dans le domaine social et celui des sciences humaines en general. On assiste ainsi a l'apparition de methodes et d'outils de dissemination propres, dont cet article examine les origines et le developpement historique.

Bayesian Priors from Loss Matching

Abstract: Resume Cet article discute le probleme de la construction de lois de probabilite a priori dans un contexte ou les opinions ou les informations refletees par ces lois portent uniquement sur les quantites observables du probleme. Nous apportons une attention particuliere au cas ou le parametre est caracterise par la minimisation d'une mesure de divergence, et examinons l'application de la regle de Bayes dans cette perspective. Notre point de vue est fondamentalement non parametrique, et repose sur le “matching” de deux fonctions de perte, l'une liee au modele, l'autre a l'a priori. Summary This paper is concerned with the construction of prior probability measures for parametric families of densities where the framework is such that only beliefs or knowledge about a single observable data point is required. We pay particular attention to the parameter which minimizes a measure of divergence to the distribution providing the data. The prior distribution reflects this attention and we discuss the application of the Bayes rule from this perspective. Our framework is fundamentally non-parametric and we are able to interpret prior distributions on the parameter space using ideas of matching loss functions, one of which is coming from the data model and the other from the prior.

Gumbel’s Identity, Binomial Moments, and Bonferroni Sums

Abstract: Resume L'identite de Gumbel etablit l'egalite de la somme de Bonferroni Sk,n, k = 0, 1, 2, … , n et du moment binomial d'ordre k de la variable qui compte, dans un ensemble arbitraire de nevenements, le nombre Mn d'evenements se realisant. Nous presentons un traitement unifie de bornes bien connues obtenues dans ce contexte par Bonferroni, Galambos-Renyi, Dawson-Sankoff et Chung-Erdos, ainsi que de quelques bornes moins connues etablies par Frechet et Gumbel. Toutes font intervenir des sommes de Bonferroni. Notre demarche consiste a montrer que ces bornes apparaissent dans un cadre plus general comme les moments binomiaux d'une variable aleatoire a valeurs entieres particuliere. L'application de l'identite de Gumbel fournit alors la forme usuelle en termes de sommes de Bonferroni. Notre approche simplifie les preuves existantes, et permet d'etendre les resultats de Frechet et Gumbel au cas de la probabilite pour qu'au moins t, 1 ≤t≤n des nevenements consideres se realisent. Une derniere consequence de notre approche est l'amelioration d'une borne de Petrov qui elle-meme est la generalisation de la borne de Chung et Erdos. Summary Gumbel’s Identity equates the Bonferroni sum with the k-th binomial moment of the number of eventsMnwhich occur, out ofnarbitrary events. We provide a unified treatment of familiar probability bounds on a union of events by Bonferroni, Galambos–Renyi, Dawson–Sankoff, and Chung–Erdos, as well as less familiar bounds by Frechet and Gumbel, all of which are expressed in terms of Bonferroni sums, by showing that all these arise as bounds in a more general setting in terms of binomial moments of a general non-negative integer-valued random variable. Use of Gumbel’s Identity then gives the inequalities in familiar Bonferroni sum form. This approach simplifies existing proofs. It also allows generalization of the results of Frechet and Gumbel to give bounds on the probability that at leasttofnevents occur for anyA further consequence of the approach is an improvement of a recent bound of Petrov which itself generalizes the Chung–Erdos bound.

Subsampling Inference with K Populations and a Non-standard Behrens–Fisher Problem

Abstract: Resume Nous passons en revue la methodologie et le developpement historique des methodes de sous-echantillonnage, et explorons en detail leur utilisation dans le cadre des problemes de tests d'hypotheses, ou ces methodes ont eteetonnamment peu prises en consideration. Nous explorons, en particulier, le cas tres general d'un modele aKechantillons faisant intervenir un parametre de grande dimension. Nous mettons en evidence le role important que joue le centrage de la loi de sous-echantillonnage, et montrons que ce centrage augmente la puissance des tests consideres. Nous montrons egalement comment les methodes de sous-echantillonnage permettent de traiter une forme non standard du probleme de Behrens-Fisher dans laquelle les populations dont on desire comparer les moyennes ne different pas seulement par leurs variances (qui peuvent eventuellement etre infinies), mais peuvent presenter des lois completement heterogenes. Notre formulation est tres generale, et s'applique meme a des observations de nature fonctionnelle. Enfin, nous developpons une theorie pour les statistiques-U permettant d'etablir, dans le cadre tres general de Kechantillons, la validite asymptotique des intervalles de confiance et des tests fondes sur le sous-echantillonnage. Summary We revisit the methodology and historical development of subsampling, and then explore in detail its use in hypothesis testing, an area which has received surprisingly modest attention. In particular, the general set-up of a possibly high-dimensional parameter with data fromKpopulations is explored. The role of centring the subsampling distribution is highlighted, and it is shown that hypothesis testing with a data-centred subsampling distribution is more powerful. In addition we demonstrate subsampling’s ability to handle a non-standard Behrens–Fisher problem, i.e., a comparison of the means of two or more populations which may possess not only different and possibly infinite variances, but may also possess different distributions. However, our formulation is general, permitting even functional data and/or statistics. Finally, we provide theory forK-sampleU-statistics that helps establish the asymptotic validity of subsampling confidence intervals and tests in this very general setting.