A class of multidimensional IRT models for testing unidimensionality and clustering items
TL;DR: A hierarchical clustering algorithm is proposed which can be used, when the dimensionality of the latent structure is completely unknown, for dividing items into groups referred to different latent traits and an asymptotically equivalent Wald test statistic is derived.
Abstract: We illustrate a class of multidimensional item response theory models in which the items are allowed to have different discriminating power and the latent traits are represented through a vector having a discrete distribution. We also show how the hypothesis of unidimensionality may be tested against a specific bidimensional alternative by using a likelihood ratio statistic between two nested models in this class. For this aim, we also derive an asymptotically equivalent Wald test statistic which is faster to compute. Moreover, we propose a hierarchical clustering algorithm which can be used, when the dimensionality of the latent structure is completely unknown, for dividing items into groups referred to different latent traits. The approach is illustrated through a simulation study and an application to a dataset collected within the National Assessment of Educational Progress, 1996.
Citations
More filters
TL;DR: Analyzing the variation in outcomes of three standardized user satisfaction scales when completed by users who had spent different amounts of time with a website strongly encourages further research to analyze the relationships of the three scales with levels of product exposure.
Abstract: Nowadays, practitioners extensively apply quick and reliable scales of user satisfaction as part of their user experience analyses to obtain well-founded measures of user satisfaction within time and budget constraints. However, in the human–computer interaction literature the relationship between the outcomes of standardized satisfaction scales and the amount of product usage has been only marginally explored. The few studies that have investigated this relationship have typically shown that users who have interacted more with a product have higher satisfaction. The purpose of this article was to systematically analyze the variation in outcomes of three standardized user satisfaction scales (SUS, UMUX, UMUX-LITE) when completed by users who had spent different amounts of time with a website. In two studies, the amount of interaction was manipulated to assess its effect on user satisfaction. Measurements of the three scales were strongly correlated and their outcomes were significantly affected by the amo...
104 citations
Cites background or methods from "A class of multidimensional IRT mod..."
...we conducted additional analyses with a special class of statistical models known as LC IRT models (Bacci et al., 2014; Bartolucci, 2007; Bartolucci et al., 2014)....
[...]
...emerge did we plan to follow up with a multidimensional latent class item response theory model (LC IRT) to more deeply test the dimensional structure of the scale (Bacci, Bartolucci, & Gnaldi, 2014; Bartolucci, 2007; Bartolucci, Bacci, & Gnaldi, 2014)....
[...]
...Only if this expected pattern did not emerge did we plan to follow up with a multidimensional latent class item response theory model (LC IRT) to more deeply test the dimensional structure of the scale (Bacci, Bartolucci, & Gnaldi, 2014; Bartolucci, 2007; Bartolucci, Bacci, & Gnaldi, 2014)....
[...]
...To verify the appropriateness of the one-component solution, we conducted additional analyses with a special class of statistical models known as LC IRT models (Bacci et al., 2014; Bartolucci, 2007; Bartolucci et al., 2014)....
[...]
TL;DR: In this article, an extension of the latent Markov Rasch model is described for the analysis of binary longitudinal data with covariates when subjects are collected in clusters, such as students clustered in classes.
Abstract: An extension of the latent Markov Rasch model is described for the analysis of binary longitudinal data with covariates when subjects are collected in clusters, such as students clustered in classes. For each subject, a latent process is used to represent the characteristic of interest (e.g., ability) conditional on the effect of the cluster to which he or she belongs. The latter effect is modeled by a discrete latent variable associated to each cluster. For the maximum likelihood estimation of the model parameters, an Expectation-Maximization algorithm is outlined. Through the analysis of a data set collected in the Lombardy Region (Italy), it is shown how the proposed model may be used for assessing the development of cognitive achievement. The data set is based on test scores in mathematics observed over 3 years on middle school students attending public and non-state schools.Manuscript received March 20, 2009 Revision received July 2, 2010 Accepted July 10, 2010
49 citations
TL;DR: In this paper, a class of multidimensional item response theory models for polytomously-scored items with ordinal response categories was proposed, which allows for different parameterizations for the conditional distribution of the response variables given the latent traits, which depend on the type of link function and the constraints imposed on the item parameters.
Abstract: We propose a class of multidimensional Item Response Theory models for polytomously-scored items with ordinal response categories. This class extends an existing class of multidimensional models for dichotomously-scored items in which the latent abilities are represented by a random vector assumed to have a discrete distribution, with support points corresponding to different latent classes in the population. In the proposed approach, we allow for different parameterizations for the conditional distribution of the response variables given the latent traits, which depend on the type of link function and the constraints imposed on the item parameters. Moreover, we suggest a strategy for model selection that is based on a series of steps consisting of selecting specific features, such as the dimension of the model (number of latent traits), the number of latent classes, and the specific parameterization. In order to illustrate the proposed approach, we analyze a dataset from a study on anxiety and depression...
48 citations
TL;DR: This paper contains a systematic study of the Markov properties and the way they can be used to distinguish spurious from genuine evidence of DIF and local dependence and proposes a strategy for initial item screening that will reduce the time needed to identify a graphical loglinear Rasch model that fits the item responses.
Abstract: In behavioural sciences, local dependence and DIF are common, and purification procedures that eliminate items with these weaknesses often result in short scales with poor reliability. Graphical loglinear Rasch models (Kreiner & Christensen, in Statistical Methods for Quality of Life Studies, ed. by M. Mesbah, F.C. Cole & M.T. Lee, Kluwer Academic, pp. 187–203, 2002) where uniform DIF and uniform local dependence are permitted solve this dilemma by modelling the local dependence and DIF. Identifying loglinear Rasch models by a stepwise model search is often very time consuming, since the initial item analysis may disclose a great deal of spurious and misleading evidence of DIF and local dependence that has to disposed of during the modelling procedure. Like graphical models, graphical loglinear Rasch models possess Markov properties that are useful during the statistical analysis if they are used methodically. This paper describes how. It contains a systematic study of the Markov properties and the way they can be used to distinguish spurious from genuine evidence of DIF and local dependence and proposes a strategy for initial item screening that will reduce the time needed to identify a graphical loglinear Rasch model that fits the item responses. The last part of the paper illustrates the item screening procedure on simulated data and on data on the PF subscale measuring physical functioning in the SF36 Health Survey inventory.
41 citations
Cites background from "A class of multidimensional IRT mod..."
...Other examples are discussed by Hagenaars (1998), Anderson and Bockenholt (2000), Humphreys and Titterington (2003), Bartolucci and Forcina (2005), Bartolucci (2007) , Anderson and Yu (2007), and Rijmen, Vansteelandt, and De Boeck (2008)....
[...]
TL;DR: In this article, a model-based approach was used to investigate whether item indicators of test-taking disengagement constitute a continuous latent person variable by domain, and the effects of person and item characteristics were jointly tested using explanatory item response models.
Abstract: A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals in order to better understand conditioning factors of test-taking disengagement: First, a model-based approach was used to investigate whether item indicators of disengagement constitute a continuous latent person variable by domain. Second, the effects of person and item characteristics were jointly tested using explanatory item response models. Analyses were based on the Canadian sample of Round 1 of the PIAAC, with N = 26,683 participants completing test items in the domains of literacy, numeracy, and problem solving. Binary item disengagement indicators were created by means of item response time thresholds. The results showed that disengagement indicators define a latent dimension by domain. Disengagement increased with lower educational attainment, lower cognitive skills, and when the test language was not the participant’s native language. Gender did not exert any effect on disengagement, while age had a positive effect for problem solving only. An item’s location in the second of two assessment modules was positively related to disengagement, as was item difficulty. The latter effect was negatively moderated by cognitive skill, suggesting that poor test-takers are especially likely to disengage with more difficult items. The negative effect of cognitive skill, the positive effect of item difficulty, and their negative interaction effect support the assumption that disengagement is the outcome of individual expectations about success (informed disengagement).
40 citations
Cites methods from "A class of multidimensional IRT mod..."
...One option would be to use a (multi-dimensional) latent class model (Bartolucci 2007) distinguishing between engaged and disengaged respondents by means of a categorical latent person variable measured by binary disengagement indicators....
[...]
References
More filters
49,597 citations
Book•
19 Jun 2013
TL;DR: The second edition of this book is unique in that it focuses on methods for making formal statistical inference from all the models in an a priori set (Multi-Model Inference).
Abstract: Introduction * Information and Likelihood Theory: A Basis for Model Selection and Inference * Basic Use of the Information-Theoretic Approach * Formal Inference From More Than One Model: Multi-Model Inference (MMI) * Monte Carlo Insights and Extended Examples * Statistical Theory and Numerical Results * Summary
36,993 citations
Proceedings Article•
01 Jan 1973
TL;DR: The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.
Abstract: In this paper it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion. This observation shows an extension of the principle to provide answers to many practical problems of statistical model fitting.
18,539 citations
"A class of multidimensional IRT mod..." refers methods in this paper
...Since our main interest here is on hypothesis testing, we then suggest choosing k on the basis of the Akaike information criterion (AIC; Akaike, 1973; Burnham & Anderson, 2002)....
[...]
01 Jan 1973
TL;DR: In this paper, it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion.
Abstract: In this paper it is shown that the classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion. This observation shows an extension of the principle to provide answers to many practical problems of statistical model fitting.
15,424 citations
Book•
01 Jan 1968
TL;DR: In this paper, the authors present a survey of test theory models and their application in the field of mental test analysis. But the focus of the survey is on test-score theories and models, and not the practical applications and limitations of each model studied.
Abstract: This is a reprint of the orginal book released in 1968. Our primary goal in this book is to sharpen the skill, sophistication, and in- tuition of the reader in the interpretation of mental test data, and in the construction and use of mental tests both as instruments of psychological theory and as tools in the practical problems of selection, evaluation, and guidance. We seek to do this by exposing the reader to some psychologically meaningful statistical theories of mental test scores. Although this book is organized in terms of test-score theories and models, the practical applications and limitations of each model studied receive substantial emphasis, and these discussions are presented in as nontechnical a manner as we have found possible. Since this book catalogues a host of test theory models and formulas, it may serve as a reference handbook. Also, for a limited group of specialists, this book aims to provide a more rigorous foundation for further theoretical research than has heretofore been available.One aim of this book is to present statements of the assumptions, together with derivations of the implications, of a selected group of statistical models that the authors believe to be useful as guides in the practices of test construction and utilization. With few exceptions we have given a complete proof for each major result presented in the book. In many cases these proofs are simpler, more complete, and more illuminating than those originally offered. When we have omitted proofs or parts of proofs, we have generally provided a reference containing the omitted argument. We have left some proofs as exercises for the reader, but only when the general method of proof has already been demonstrated. At times we have proved only special cases of more generally stated theorems, when the general proof affords no additional insight into the problem and yet is substantially more complex mathematically.
6,814 citations