Commonalities and Differences in IRT-Based Methods for Nonignorable Item Nonresponses

TLDR: In this article, the authors focus on the more challenging case of unplanned missing data, which pose not only a loss of efficiency, but potentially lead to biased estimation of item and person parameters in the measurement model.

Abstract: Missing data are an inevitable problem for applied researchers. They may occur for many different reasons. For example, participants may not be willing to participate in a study, leading to unit nonresponses, or participants may be unable or unwilling to answer all items of a test. Such item nonresponses typically result from omitted or not-reached items and are common in educational assessments. Furthermore, test takers provide answers that cannot be scored meaningfully, producing item nonresponses due to notcodable item responses. Unplanned missing data resulting from test takers' response behavior must be distinguished from planned missing data due to the design (Graham, Taylor, & Cumsille, 2001; Graham, Taylor, Olchowski, & Cumsille, 2006). Especially in large scale assessments (LSA), only subsets of items are assigned to test takers to reduce costs, participant burden, fatigue, or potential practice effects. With an appropriate test design, including randomized assignment of the different test forms, planned missing data does not pose a threat to validity. Therefore, we focus on the more challenging case of unplanned missing data, which pose not only a loss of efficiency, but potentially lead to biased estimation of item and person parameters in the measurement model. In large scale assessments (LSA), parameters of the structural model such as means, variances, covariances of latent variables are of primary interest instead of individual proficiency levels; however, these distributional parameters may also be biased due to item nonresponses.Many different approaches to handle missing values have been proposed. Weighting methods, such as inverse probability weighting, are commonly applied to account for unit nonresponses (Li, Shen, Li, & Robins, 2011; Wooldridge, 2007). The simplest approach for item nonresponses is listwise deletion, the inclusion of complete cases into the statistical analysis. Pairwise deletion was proposed as an alternative for models that are based on bivariate statistics, such as structural equation models (SEM) that use covariance matrices as input for parameter estimation. Single and multiple imputation methods rest upon the idea that one should replace missing values with predicted or plausible values in the first step (imputation phase). Next, the augmented data sets are analyzed with standard methods in the second step (analysis phase). In contrast, model-based approaches, such as full information maximum likelihood (FIML), allow for parameter estimation with incomplete data sets. The suitability of the different missing data handling methods depend on whether certain assumptions hold. These assumptions can be derived from Rubin's taxonomy of missing data (1976; 2002). He distinguishes between three missing data mechanisms: Missing completely at random (MCAR), missing at random (MAR), and not missing at random (NMAR). We will examine these mechanisms in greater detail later in this paper. So far it suffices to note that missing data that are MCAR and MAR are also called ignorable. In this case, missingness is either completely independent of the observed and unobserved variables under examination (MCAR), or conditionally stochastically independent of the unobserved variables given the observed variables (MAR). The stochastic independencies imply that missingness is not informative with respect to unobserved variables and underlying model parameters and can therefore be ignored. Almost all modern missing data methods rest upon the assumption that the missing data mechanism is ignorable. This is also true for methods like FIML and multiple imputation, which are regarded as state of the art methods for item nonresponses (Schafer & Graham, 2002). In contrast, missing data that are NMAR are termed nonignorable. In this case, missingness is not conditionally independent of the unobserved variables given the observed variables. Such missingness is also called informative with respect to unobserved variables. …