Showing papers in "The American Statistician in 2006"

Multilevel and Longitudinal Modeling Using Stata

Abstract: (2006). Multilevel and Longitudinal Modeling Using Stata. The American Statistician: Vol. 60, No. 3, pp. 293-294.

The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant

Abstract: It is common to summarize statistical comparisons by declarations of statistical significance or nonsignificance. Here we discuss one problem with such declarations, namely that changes in statistical significance are often not themselves statistically significant. By this, we are not merely making the commonplace observation that any particular threshold is arbitrary—for example, only a small change is required to move an estimate from a 5.1% significance level to 4.9%, thus moving it into statistical significance. Rather, we are pointing out that even large changes in significance levels can correspond to small, nonsignificant changes in the underlying quantities.The error we describe is conceptually different from other oft-cited problems—that statistical significance is not the same as practical importance, that dichotomization into significant and nonsignificant results encourages the dismissal of observed differences in favor of the usually less interesting null hypothesis of no difference, and that...

A Lego System for Conditional Inference

Abstract: Conditioning on the observed data is an important and flexible design principle for statistical test procedures. Although generally applicable, permutation tests currently in use are limited to the treatment of special cases, such as contingency tables or K-sample problems. A new theoretical framework for permutation tests opens up the way to a unified and generalized view. This article argues that the transfer of such a theory to practical data analysis has important implications in many applications and requires tools that enable the data analyst to compute on the theoretical concepts as closely as possible. We reanalyze four datasets by adapting the general conceptual framework to these challenging inference problems and using the coin add-on package in the R system for statistical computing to show what one can gain from going beyond the “classical” test procedures.

On The So-Called “Huber Sandwich Estimator” and “Robust Standard Errors”

Abstract: The “Huber Sandwich Estimator” can be used to estimate the variance of the MLE when the underlying model is incorrect. If the model is nearly correct, so are the usual standard errors, and robustification is unlikely to help much. On the other hand, if the model is seriously in error, the sandwich may help on the variance side, but the parameters being estimated by the MLE are likely to be meaningless—except perhaps as descriptive statistics.

Calibrated Bayes : A Bayes/Frequentist Roadmap

Abstract: The lack of an agreed inferential basis for statistics makes life "interesting" for academic statisticians, but at the price of negative implications for the status of statistics in industry, science, and government. The practice of our discipline will mature only when we can come to a basic agreement about how to apply statistics to real problems. Simple and more general illustrations are given of the negative consequences of the existing schism between frequentists and Bayesians. An assessment of strengths and weaknesses of the frequentist and Bayes systems of inference suggests that calibrated Bayes-a compromise based on the works of Box, Rubin, and others-captures the strengths of both approaches and provides a roadmap for future advances. The approach asserts that inferences under a particular model should be Bayesian, but model assessment can and should involve frequentist ideas. This article also discusses some implications of this proposed compromise for the teaching and practice of statistics.

A Framework for Evaluating the Utility of Data Altered to Protect Confidentiality

Abstract: When releasing data to the public, statistical agencies and survey organizations typically alter data values in order to protect the confidentiality of survey respondents' identities and attribute values. To select among the wide variety of data alteration methods, agencies require tools for evaluating the utility of proposed data releases. Such utility measures can be combined with disclosure risk measures to gauge risk-utility tradeoffs of competing methods. This article presents utility measures focused on differences in inferences obtained from the altered data and corresponding inferences obtained from the original data. Using both genuine and simulated data, we show how the measures can be used in a decision-theoretic formulation for evaluating disclosure limitation procedures.

Selecting the Best Linear Mixed Model Under REML

Abstract: Restricted maximum likelihood (REML) estimation of the parameters of the mixed model has become commonplace, even becoming the default option in many statistical software packages. However, a review of the literature indicates a need to update and clarify model selection techniques under REML, as ambiguities exist on the appropriateness of existing information criteria in this setting. A simulation study as well as an application assisted in gaining an understanding of the performance of information criteria in selecting the best model when using REML estimation.

Revising the Pareto Chart

Abstract: The Pareto chart is a bar chart of frequencies sorted by frequency. The most popular incarnation of the chart puts the highest bars on the left and includes a line showing the scores produced by adding the heights in order from left to right. This chart is used widely in quality control settings to identify critical factors leading to failure or defects in a process. This article presents revisions that remedy problems with the chart and improve its usability in diagnostic settings.

Symmetrized Percent Change for Treatment Comparisons

Abstract: Treatment effects are commonly analyzed by relating post-treatment (F) measurements with corresponding baseline (B) measurements. As effect measures, absolute difference (D) and percent change (PC) are used more than symmetrized percent change (SPC). However, all these measures alter the dependency structure with B and their distributions can differ from B and F. To examine their interpretability and relative performance, we considered simulations under independence and additive and multiplicative correlation structures for parametric and nonparametric analyses. Under independence, nonparametric analysis on F had the greatest power. Elsewhere, simple ANOVA on SPC had power equal to or greater than alternative analysis methods.

Penalized Splines and Reproducing Kernel Methods

Abstract: Two data analytic research areas—penalized splines and reproducing kernel methods—have become very vibrant since the mid-1990s. This article shows how the former can be embedded in the latter via theory for reproducing kernel Hilbert spaces. This connection facilitates cross-fertilization between the two bodies of research. In particular, connections between support vector machines and penalized splines are established. These allow for significant reductions in computational complexity, and easier incorporation of special structure such as additivity.

Tukey's gh Distribution for Multiple Imputation

Abstract: Tukey proposed a class of distributions, the g-and-h family (gh family), based on a transformation of a standard normal variable to accommodate different skewness and elongation in the distribution of variables arising in practical applications. It is easy to draw values from this distribution even though it is hard to explicitly state the probability density function. Given this flexibility, the gh family may be extremely useful in creating multiple imputations for missing data. This article demonstrates how this family, as well as its generalizations, can be used in the multiple imputation analysis of incomplete data. The focus of this article is on a scalar variable with missing values. In the absence of any additional information, data are missing completely at random, and hence the correct analysis is the complete-case analysis. Thus, the application of the gh multiple imputation to the scalar cases affords comparison with the correct analysis and with other model-based multiple imputation methods. C...

A Spatial Analysis of Basketball Shot Chart Data

Abstract: Basketball coaches at all levels use shot charts to study shot locations and outcomes for their own teams as well as upcoming opponents. Shot charts are simple plots of the location and result of each shot taken during a game. Although shot chart data are rapidly increasing in richness and availability, most coaches still use them purely as descriptive summaries. However, a team's ability to defend a certain player could potentially be improved by using shot data to make inferences about the player's tendencies and abilities. This article develops hierarchical spatial models for shot-chart data, which allow for spatially varying effects of covariates. Our spatial models permit differential smoothing of the fitted surface in two spatial directions, which naturally correspond to polar coordinates: distance to the basket and angle from the line connecting the two baskets. We illustrate our approach using the 2003–2004 shot chart data for Minnesota Timberwolves guard Sam Cassell.

Investigating Concurrency in Online Auctions Through Visualization

Abstract: This article presents graphical techniques for visualizing concurrency in online auctions. These include rug plots, which allow for a compact view of many simultaneous auctions while preserving the structure of individual auctions. We also use box plots, moving statistics plots, and autocorrelation plots, supplemented by statistical tests. Together, these are used to study synchronous events and to surmise trends in the data, as well as to raise new research questions. We illustrate our methods on data from eBay.com.

The boxer, the wrestler, and the coin flip : A paradox of robust bayesian inference and belief functions

Abstract: Bayesian inference requires all unknowns to be represented by probability distributions, which awkwardly implies that the probability of an event for which we are completely ignorant (e.g., that the world's greatest boxer would defeat the world's greatest wrestler) must be assigned a particular numerical value such as 1/2, as if it were known as precisely as the probability of a truly random event (e.g., a coin flip).Robust Bayes and belief functions are two methods that have been proposed to distinguish ignorance and randomness. In robust Bayes, a parameter can be restricted to a range, but without a prior distribution, yielding a range of potential posterior inferences. In belief functions (also known as the Dempster-Shafer theory), probability mass can be assigned to subsets of parameter space, so that randomness is represented by the probability distribution and uncertainty is represented by large subsets, within which the model does not attempt to assign probabilities.Through a simple example involvi...

Power Priors and Their Use in Clinical Trials

Abstract: This article reviews power priors, a class of prior distributions for an unknown parameter that exploits information from results of previous, similar studies, a situation arising often in clinical trials. The article shows that, for independent and identically distributed historical data, a basic formulation of power priors (geometric priors) can be obtained as the result of a prior updating-and-combining process based on training samples of iid historical data. This formulation gives an operational justification to power priors. It also allows us to relate the discount scalar quantity controlling the influence of historical information on final inference to the size of training samples. Properties of power priors and their extension to more complex set-ups are discussed. Then several examples are provided of their use in the analysis of clinical trials data. The approach is shown to be appropriate for handling problems arising when information is combined from different studies, such as lack of exchange...

A Review of Three Directed Acyclic Graphs Software Packages: MIM, Tetrad, and WinMine

Abstract: This article offers a review of three software packages that estimate directed acyclic graphs (DAGs) from data. The three packages, MIM, Tetrad and WinMine, can help researchers discover underlying causal structure. Although each package uses a different algorithm, the results are to some extent similar. All three packages are free and easy to use. They are likely to be of interest to researchers who do not have strong theory regarding the causal structure in their data. DAG modeling is a powerful analytic tool to consider in conjunction with, or in place of, path analysis, structural equation modeling, and other statistical techniques.

The Two-Sample T Test With One Variance Unknown

Abstract: We consider the situation in two-sample testing when one variance is assumed to be known and the other variance is considered unknown. This situation arises, for example, when one is interested in comparing a standard treatment with a new treatment. Although this situation occurs relatively infrequently, our example discusses the important tool of moment matching and makes the classic two-sample Satterthwaite t approximation transparent.

A Comparison of Potential Playoff Systems for NCAA I-A Football

Abstract: This article discusses the properties of various knockout tournament designs and presents theoretical results. Potential playoff schemes for Division I-A football are examined via simulation studies. Several metrics are used to assess the relative merits of playoff scenarios, which differ in number, selection, and seeding of playoff teams. Most suggest that college football would benefit from a limited playoff system. Interestingly, for the class of playoff systems examined, the number of teams influences the performance far more than does the seeding procedure.

Statistical Aspects of Forecasting and Planning for Hurricanes

Abstract: Virtually every aspect of hurricane planning and forecasting involves (or should involve!) the science of statistics. The very active 2004 and 2005 Atlantic hurricane seasons—in particular the devastating landfall of Hurricane Katrina on the Gulf Coast—as well as concerns that climate change is altering hurricane frequency and intensity, provide many examples of the use and misuse of statistics. Although the massive news media coverage indicated the interest and importance of these stories, from a scientific standpoint much of the “information” in these media reports was of dubious accuracy, especially where statistics were concerned. These examples indicate many opportunities to advance the state of the art of hurricane forecasting and planning through the intelligent applications of statistical analyses. This article considers several issues related to hurricane planning and forecasting, including a review and discussion of progress since 1985, when The American Statistician published some articles on t...

Design of Experiments with MINITAB

Abstract: Chapter 7. The 50 pages in this chapter contain the classical basics of hypothesis testing and estimation (the Neyman-Pearson lemma and the Cramér-Rao lower bound) but also (and perhaps a little bit surprising in this neighborhood) a discussion of kernel density estimation. The next 200 pages contain an exposition of linear models, design of experiments, and nonlinear models (Chapters 8–10). Chapter 10 also discusses generalized linear models, log-linear models, and nonparametric regression (via local polynomials). The next chapter on Bayesian approaches gives an introduction to modern developments in Bayesian computation. The book concludes with a chapter on conditional and marginal likelihood approaches. An introduction to saddlepoint approximations is found here. I like this book a lot. It is really a pleasure to read. The 700 pages offer a good initial synopsis of what is going on in modern statistics. The book is lively, full of data, and packed with ideas. The author has put a lot of energy, effort, care, and intellectual input into the book. I would definitely recommend this text, both to students and to colleagues.

The Sectioned Density Plot

Abstract: Effective graphical presentation efficiently summarizes, exposes, and communicates patterns in data. Here we describe a new plot, the sectioned density plot, that compares full distributions across multiple groups. We designed the sectioned density plot to exploit the visual system's natural ability to interpret occlusion and intensity variation as changes in depth. By incorporating depth into the graphical display, we were able to combine the ability of the boxplot to display trends in variance and central tendency with the ability of a histogram/kernel density plot to present distribution shape.

Spreadsheets in Statistical Practice—Another Look

Abstract: Many authors have criticized the use of spreadsheets for statistical data processing and computing because of incorrect statistical functions, no log file or audit trail, inconsistent behavior of computational dialogs, and poor handling of missing values. Some improvements in some spreadsheet processors and the possibility of audit trail facilities suggest that the use of a spreadsheet for some statistical data entry and simple analysis tasks may now be acceptable. A brief outline of some issues and some guidelines for good practice are included.

On Bayesian Analysis of Generalized Linear Models Using the Jacobian Technique

Abstract: In this article, we obtain an estimator of the regression parameters for generalized linear models, using the Jacobian technique. We restrict ourselves to the natural exponential family for the response variable and choose the conjugate prior for the natural parameter. Using the Jacobian of transformation, we obtain the posterior distribution for the canonical link function and thereby obtain the posterior mode for the link. Under the full rank assumption for the covariate matrix, we then find an estimator for the regression parameters for the natural exponential family. Then the proposed estimator is specially derived for the Poisson model with log link function, and the binomial response model with the logit link function. We also discuss extensions to the binomial response model when covariates are all positive. Finally, an illustrative real-life example is given for the Poisson model with log link. In order to estimate the standard error of our estimators, we use the Bernstein-von Mises theorem. Final...

Using Moment Generating Functions to Derive Mixture Distributions

Abstract: Mixture models are important in theoretical and applied statistics. Mathematical statistics courses for undergraduate senior or first-year graduate students should expose students to some properties of the mixture model. In current textbooks, the predominant approach to obtain the mixture distribution is based on marginalization of the joint distribution defined by the mixture model. This article proposes the use of moment generating functions (mgf) to obtain the distribution of some mixtures. For mixtures that do not have a mgf, a generalization using characteristic functions is illustrated with an example. The mgf approach tends to be simple and it uses some of the tools already built into the course. Several examples are used to illustrate the proposed methodology.

A User-Friendly Introduction to Link-Probit-Normal Models

Abstract: Probit-normal models have attractive properties compared to logit-normal models. In particular, they allow for easy specification of marginal links of interest while permitting a conditional random effects structure. Moreover, programming fitting algorithms for probit-normal models can be trivial with the use of well-developed algorithms for approximating multivariate normal quantiles. In typical settings, data cannot distinguish between probit and logit conditional link functions. Therefore, if marginal interpretations are desired, the default conditional link should be the most convenient one. We refer to models with a probit conditional link, an arbitrary marginal link, and a normal random effect distribution as link-probit-normal models. In this article we outline these models and discuss appropriate situations for using multivariate normal approximations for estimation. Unlike other articles in this area that focus on very general situations and implement Markov chain or MCEM algorithms, we focus on ...

Tell me the method, I'll give you the mean

Abstract: Different procedures underlying the determination of the mean of a set of values impart different weights to these values. Thus, different methods result in different weighted means. In particular, when the weights are directly proportional, or in fact equal, to the (weighted) values, the self-weighted mean is obtained, and when they are inversely proportional to the values, the harmonic mean results. Generally, the former is greater and the latter is smaller than the arithmetic (uniformly weighted) mean. We illustrate cases of sampling procedures and of geometric requirements that give rise to the self-weighted and harmonic means. We call attention to some psychological difficulties and to the didactic value of dealing with cases of differential weighting.

The Victory of Least Squares and Orthogonality in Statistics

Abstract: This article gives a short history of the origin of least squares from a geometric perspective. It describes techniques used to deal with contradictory data in the second half of the eighteenth century, and their applications to problems in astronomy and geodesy. It is interesting that analogues of least squares—with maximum deviation and sums of absolute deviations, respectively, replacing sums of squared deviations—preceded least squares, which first appeared publicly in 1805, by 50 years. Geometry—specifically, an inner product being used to produce angles and orthogonality—is offered as the reason for least squares becoming preferable. More generally, we briefly outline how definitions and fundamental results in the general linear model, analysis of variance, conditional probability, independence, sufficiency, and time series can be unified and clarified as deriving from the inner product.