Statistical procedures for missing data have vastly improved, yet misconception and unsound practice still abound. The authors frame the missing-data problem, review methods, offer advice, and raise issues that remain unresolved. They clear up common misunderstandings regarding the missing at random (MAR) concept. They summarize the evidence against older procedures and, with few exceptions, discourage their use. They present, in both technical and practical language, 2 general approaches that come highly recommended: maximum likelihood (ML) and Bayesian multiple imputation (MI). Newer developments are discussed, including some for dealing with missing data that are not MAR. Although not yet in the mainstream, these procedures may eventually extend the ML and MI methods that currently represent the state of the art.

/pdf/missing-data-our-view-of-the-state-of-the-art-3xnwc88wmd.pdf

Missing data: Our view of the state of the art.

Structural Equations with Latent Variables.

Statistical Analysis with Missing Data

An account is given of a recently devised method of prediction based on sample reuse techniques. It is most useful in low structure data paradigms that involve minimal assumptions. A series of applications demonstrating the technique is presented.

The Predictive Sample Reuse Method with Applications

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1972.tb00885.x

Bayes Estimates for the Linear Model

and Notation.- and Notation.- The Kernel Method of Test Equating: Theory.- Data Collection Designs.- Kernel Equating: Overview, Pre-smoothing, and Estimation of r and s.- Kernel Equating: Continuization and Equating.- Kernel Equating: The SEE and the SEED.- Kernel Equating versus Other Equating Methods.- The Kernel Method of Test Equating: Applications.- The Equivalent-Groups Design.- The Single-Group Design.- The Counterbalanced Design.- The NEAT Design: Chain Equating.- The NEAT Design: Post-Stratification Equating.

The Kernel Method of Test Equating

The Mantel-Haenszel procedure is a noniterative contingency table method for estimating and testing a common two-factor association parameter in a 2×2×k table. As such it may be used to study “item bias” or differential item functioning in two groups of examinees. This technique is discussed in this context and compared to other related techniques as well as to item response theory methods.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.2330-8516.1986.tb00186.x

Differential item functioning and the mantel‐haenszel procedure

future observed status. DIF status is expressed in terms of the probabilities associated with each of the five DIF levels defined by the ETS classification system: C-, B-, A, B+, and C+. The EB methods yield more stable DIF estimates than do conventional methods, especially in small samples, which is advantageous in computer-adaptive testing. The EB approach may also convey information about DIF stability in a more useful way by representing the state of knowledge about an item's DIF status as probabilistic. The results of a Mantel-Haenszel (MH; Mantel & Haenszel, 1959) analysis of differential item functioning (DIF) typically include an index of the magnitude of DIF, along with an estimated standard error (see Holland & Thayer, 1988). In making decisions about whether to discard items or flag them for review, however, testing companies sometimes rely on categorical ratings of the severity of DIF. Educational Testing Service (ETS) has a system for categorizing DIF as negligible ("A"), slight to moderate ("B") or moderate to severe ("C") based on both the magnitude of the DIF index and the statistical significance of the results. A disadvantage of this classification system is that it sometimes conveys the notion that an item's DIF category is deterministic. A possible solution is to use a Bayesian approach to estimate the probabilities that the true DIF for an item falls into the A, B, or C categories (the True DIF method in our terminology) or to

An Empirical Bayes Approach to Mantel-Haenszel DIF Analysis.

The well-developed theory of exponential families of distributions is applied to the problem of fitting the univariate histograms and discrete bivariate frequency distributions that often arise in the analysis of test scores. These models are powerful tools for many forms of parametric data smoothing and are particularly well-suited to problems in which there is little or no theory to guide a choice of probability models, e.g., smoothing a distribution to eliminate roughness and zero frequencies in order to equate scores from different tests. Attention is given to efficient computation of the maximum likelihood estimates of the parameters using Newton's Method and to computationally efficient methods for obtaining the asymptotic standard errors of the fitted frequencies and proportions. We discuss tools that can be used to diagnose the quality of the fitted frequencies for both the univariate and the bivariate cases. Five examples, using real data, are used to illustrate the methods of this paper.

Univariate and Bivariate Loglinear Models for Discrete Test Score Distributions

The theory of log-linear models is developed for multinomially distributed data – i.e., frequency distributions and discrete multivariate distributions. Such models have applications to data smoothing and can be used in test equating. Attention is given to the computations used to find maximum likelihood estimates using Newton's method and to the computation of asymptotic standard errors for the fitted frequencies. Two examples, using real data, are used to illustrate the output of a computer program that implements these ideas.

Dorothy T. Thayer

Papers

The Kernel Method of Test Equating

Differential item functioning and the mantel‐haenszel procedure

An Empirical Bayes Approach to Mantel-Haenszel DIF Analysis.

Univariate and Bivariate Loglinear Models for Discrete Test Score Distributions

Notes on the use of log-linear models for fitting discrete probability distributions