Showing papers in "Psychometrika in 1989"

Latent variable modeling in heterogeneous populations

Abstract: Common applications of latent variable analysis fail to recognize that data may be obtained from several populations with different sets of parameter values. This article describes the problem and gives an overview of methodology that can address heterogeneity. Artificial examples of mixtures are given, where if the mixture is not recognized, strongly distorted results occur. MIMIC structural modeling is shown to be a useful method for detecting and describing heterogeneity that cannot be handled in regular multiple-group analysis. Other useful methods instead take a random effects approach, describing heterogeneity in terms of random parameter variation across groups. These random effects models connect with emerging methodology for multilevel structural equation modeling of hierarchical data. Examples are drawn from educational achievement testing, psychopathology, and sociology of education. Estimation is carried out by the LISCOMP program.

Weighted likelihood estimation of ability in item response theory

Abstract: Applications of item response theory, which depend upon its parameter invariance property, require that parameter estimates be unbiased. A new method, weighted likelihood estimation (WLE), is derived, and proved to be less biased than maximum likelihood estimation (MLE) with the same asymptotic variance and normal distribution. WLE removes the first order bias term from MLE. Two Monte Carlo studies compare WLE with MLE and Bayesian modal estimation (BME) of ability in conventional tests and tailored tests, assuming the item parameters are known constants. The Monte Carlo studies favor WLE over MLE and BME on several criteria over a wide range of the ability scale.

Alternative Test Criteria in Covariance Structure Analysis: A Unified Approach.

Abstract: In the context of covariance structure analysis, a unified approach to the asymptotic theory of alternative test criteria for testing parametric restrictions is provided. The discussion develops within a general framework that distinguishes whether or not the fitting function is asymptotically optimal, and allows the null and alternative hypothesis to be only approximations of the true model. Also, the equivalent of the information matrix, and the asymptotic covariance matrix of the vector of summary statistics, are allowed to be singular. When the fitting function is not asymptotically optimal, test statistics which have asymptotically a chi-square distribution are developed as a natural generalization of more classical ones. Issues relevant for power analysis, and the asymptotic theory of a testing related statistic, are also investigated.

A maximin model for test design with practical constraints

Abstract: A maximin model for IRT-based test design is proposed. In the model only the relative shape of the target test information function is specified. It serves as a constraint subject to which a linear programming algorithm maximizes the information in the test. In the practice of test construction, several demands as linear constraints in the model. A worked example of a text construction problem with practical constraints is presented. The paper concludes with a discussion of some alternative models of test construction.

A simulated annealing methodology for clusterwise linear regression

Abstract: In many regression applications, users are often faced with difficulties due to nonlinear relationships, heterogeneous subjects, or time series which are best represented by splines. In such applications, two or more regression functions are often necessary to best summarize the underlying structure of the data. Unfortunately, in most cases, it is not known a priori which subset of observations should be approximated with which specific regression function. This paper presents a methodology which simultaneously clusters observations into a preset number of groups and estimates the corresponding regression functions' coefficients, all to optimize a common objective function. We describe the problem and discuss related procedures. A new simulated annealing-based methodology is described as well as program options to accommodate overlapping or nonoverlapping clustering, replications per subject, univariate or multivariate dependent variables, and constraints imposed on cluster membership. Extensive Monte Carlo analyses are reported which investigate the overall performance of the methodology. A consumer psychology application is provided concerning a conjoint analysis investigation of consumer satisfaction determinants. Finally, other applications and extensions of the methodology are discussed.

Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations

Abstract: Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.

Clustering n objects into k groups under optimal scaling of variables

Abstract: We propose a method to reduce many categorical variables to one variable withk categories, or stated otherwise, to classifyn objects intok groups. Objects are measured on a set of nominal, ordinal or numerical variables or any mix of these, and they are represented asn points inp-dimensional Euclidean space. Starting from homogeneity analysis, also called multiple correspondence analysis, the essential feature of our approach is that these object points are restricted to lie at only one ofk locations. It follows that thesek locations must be equal to the centroids of all objects belonging to the same group, which corresponds to a sum of squared distances clustering criterion. The problem is not only to estimate the group allocation, but also to obtain an optimal transformation of the data matrix. An alternating least squares algorithm and an example are given.

Pseudo maximum likelihood estimation and a test for misspecification in mean and covariance structure models

Abstract: Using the theory of pseudo maximum likelihood estimation the asymptotic covariance matrix of maximum likelihood estimates for mean and covariance structure models is given for the case where the variables are not multivariate normal This asymptotic covariance matrix is consistently estimated without the computation of the empirical fourth order moment matrix Using quasi-maximum likelihood theory a Hausman misspecification test is developed This test is sensitive to misspecification caused by errors that are correlated with the independent variables This misspecification cannot be detected by the test statistics currently used in covariance structure analysis

Extensions of the partial credit model

Abstract: The partial credit model, developed by Masters (1982), is a unidimensional latent trait model for responses scored in two or more ordered categories. In the present paper some extensions of the model are presented. First, a marginal maximum likelihood estimation procedure is developed which allows for incomplete data and linear restrictions on both the item and the population parameters. Secondly, two statistical tests for evaluating model fit are presented: the former test has power against violation of the assumption about the ability distribution, the latter test offers the possibility of identifying specific items that do not fit the model.

Criterion-related construct validity

Abstract: Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.

An irt-based model for dichotomous longitudinal data

Abstract: The LLRA (linear logistic model with relaxed assumptions; Fischer, 1974, 1977a, 1977b, 1983a) was developed, within the framework of generalized Rasch models, for assessing change in dichotomous item score matrices between two points in time; it allows to quantify change on latent trait dimensions and to explain change in terms of treatment effects, treatment interactions, and a trend effect. A remarkable feature of the model is that unidimensionality of the item set is not required. The present paper extends this model to designs with any number of time points and even with different sets of items presented on different occasions, provided that one unidimensional subscale is available per latent trait. Thus unidimensionality assumptions within subscales are combined with multidimensionality of the item set. Conditional maximum likelihood methods for parameter estimation and hypothesis testing are developed, and a necessary and sufficient condition for unique identification of the model, given the data, is derived. Finally, a sample application is presented.

Item bias detection using loglinear IRT

Abstract: A method is proposed for the detection of item bias with respect to observed or unobserved subgroups. The method uses quasi-loglinear models for the incomplete subgroup × test score × Item 1 × ... × itemk contingency table. If subgroup membership is unknown the models are Haberman's incomplete-latent-class models. The (conditional) Rasch model is formulated as a quasi-loglinear model. The parameters in this loglinear model, that correspond to the main effects of the item responses, are the conditional estimates of the parameters in the Rasch model. Item bias can then be tested by comparing the quasi-loglinear-Rasch model with models that contain parameters for the interaction of item responses and the subgroups.

A latent trait theory via a stochastic learning theory for a knowledge space

Abstract: To capture the cognitive organization of a set of questions or problems pertaining to a body of information, Doignon and Falmagne have proposed, and analyzed in a number of papers, the concept of aknowledge space, that is, a distinguished collection of subsets of questions, representing the possibleknowledge states. This collection of sets is assumed to satisfy a number of conditions. Since this concept is a deterministic one, the problem of empirical testing arises. A stochastic version of a knowledge space is developed in this paper, in which the knowledge states are considered as possible epochs in a subject's learning history. The knowledge space is decomposed as a union of a number of possible learning paths, calledgradations. The model specifies how a subject is channelled through and progresses along a gradation. A probabilistic axiom of the “local indepencence” type relates the knowledge states to the observable responses. The predictions of this model are worked out in details in the case of parametric assumptions involving gamma distributions. An application of the model to artificial data is described, based on maximum likelihood methods. The statistical analysis is shown to be capable of revealing the combinatoric core of the model.

A Quasi-Nonmetric Method for Multidimensional Scaling via an Extended Euclidean Model.

Abstract: An Extended Two-Way Euclidean Multidimensional Scaling (MDS) model which assumes both common and specific dimensions is described and contrasted with the “standard” (Two-Way) MDS model. In this Extended Two-Way Euclidean model then stimuli (or other objects) are assumed to be characterized by coordinates onR common dimensions. In addition each stimulus is assumed to have a dimension (or dimensions) specific to it alone. The overall distance between objecti and objectj then is defined as the square root of the ordinary squared Euclidean distance plus terms denoting the specificity of each object. The specificity,s j , can be thought of as the sum of squares of coordinates on those dimensions specific to objecti, all of which have nonzero coordinatesonly for objecti. (In practice, we may think of there being just one such specific dimension for each object, as this situation is mathematically indistinguishable from the case in which there are more than one.) We further assume that δ ij =F(d ij ) +e ij where δ ij is the proximity value (e.g., similarity or dissimilarity) of objectsi andj,d ij is the extended Euclidean distance defined above, whilee ij is an error term assumed i.i.d.N(0, σ2).F is assumed either a linear function (in the metric case) or a monotone spline of specified form (in the quasi-nonmetric case). A numerical procedure alternating a modified Newton-Raphson algorithm with an algorithm for fitting an optimal monotone spline (or linear function) is used to secure maximum likelihood estimates of the paramstatistics) can be used to test hypotheses about the number of common dimensions, and/or the existence of specific (in addition toR common) dimensions. This approach is illustrated with applications to both artificial data and real data on judged similarity of nations.

Factor and ideal point analysis for interpersonally incomparable data

Abstract: Interpersonally incomparable responses pose a significant problem for survey researchers. If the manifest responses of individuals differ from their underlying true responses by monotonic transformations which vary from person to person, then the covariances of the manifest responses tools such as factor analysis may yield incorrect results. Satisfactory results for interpersonally incomparable ordinal responses can be obtained by assuming that rankings are based upon a set of multivariate normal latent variables which satisfy the factor or ideal point models of choice. Two statistical methods based upon these assumptions are described; their statistical properties are explored; and their computational feasibility is demonstrated in some simulations. We conclude that is possible to develop methods for factor and ideal point analysis of interpersonally incomparable ordinal data.

A stochastic multidimensional scaling vector threshold model for the spatial representation of "pick any/n" data

Abstract: This paper presents a new stochastic multidimensional scaling vector threshold model designed to analyze “pick any/n” choice data (e.g., consumers rendering buy/no buy decisions concerning a number of actual products). A maximum likelihood procedure is formulated to estimate a joint space of both individuals (represented as vectors) and stimuli (represented as points). The relevant psychometric literature concerning the spatial treatment of such binary choice data is reviewed. The nonlinear probit type model is described, as well as the conjugate gradient procedure used to estimate parameters. Results of Monte Carlo analyses investigating the performance of this methodology with synthetic choice data sets are presented. An application concerning consumer choices for eleven competitive brands of soft drinks is discussed. Finally, directions for future research are presented in terms of further applications and generalizing the model to accommodate three-way choice data.

Influential observations in principal factor analysis

Abstract: We propose a method for detecting influential observations in iterative principal factor analysis. For this purpose we derive the influence functionsI(x; LLT) andI(x; δ) for the common variance matrixT =LLT and the unique variance matrix Δ, respectively, in the common factor decomposition Σ =LLT + Δ. A numerical example is given for illustration.

Least-squares approximation of an improper correlation matrix by a proper one

Abstract: An algorithm is presented for the best least-squares fitting correlation matrix approximating a given missing value or improper correlation matrix. The proposed algorithm is based upon a solution for Mosier's oblique Procrustes rotation problem offered by ten Berge and Nevels. A necessary and sufficient condition is given for a solution to yield the unique global minimum of the least-squares function. Empirical verification of the condition indicates that the occurrence of non-optimal solutions with the proposed algorithm is very unlikely. A possible drawback of the optimal solution is that it is a singular matrix of necessity. In cases where singularity is undesirable, one may impose the additional nonsingularity constraint that the smallest eigenvalue of the solution be δ, where δ is an arbitrary small positive constant. Finally, it may be desirable to weight the squared errors of estimation differentially. A generalized solution is derived which satisfies the additional nonsingularity constraint and also allows for weighting. The generalized solution can readily be obtained from the standard “unweighted singular” solution by transforming the observed improper correlation matrix in a suitable way.

Robust Multidimensional Scaling.

Abstract: A method for multidimensional scaling that is highly resistant to the effects of outliers is described. To illustrate the efficacy of the procedure, some Monte Carlo simulation results are presented. The method is shown to perform well when outliers are present, even in relatively large numbers, and also to perform comparably to other approaches when no outliers are present.

Using item-specific instructional information in achievement modeling

Abstract: The problem of detecting instructional sensitivity (“item basis”) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.

NETSCAL: A Network Scaling Algorithm for Nonsymmetric Proximity Data.

Abstract: A simple property of networks is used as the basis for a scaling algorithm that represents nonsymmetric proximities as network distances. The algorithm determines which vertices are directly connected by an arc and estimates the length of each arc. Network distance, defined as the minimum pathlength between vertices, is assumed to be a generalized power function of the data. The derived network structure, however, is invariant across monotonic transformations of the data. A Monte Carlo simulation and applications to eight sets of proximity data support the practical utility of the algorithm.

A stochastic growth model applied to repeated tests of academic knowledge

Abstract: In the course of the medical program at the University of Limburg, students complete a total of 24 progress tests, consisting of items drawn from a constant itembank. A model is presented for the growth of knowledge reflected by these results. The Rasch model is used as a starting point, but both ability and difficulty parameters are taken to be random, and moreover the logistic distribution is replaced by the normal. Both individual and group abilities are estimated and explained through simple linear regression. Application to real data shows that the model fits very well.

Exploratory longitudinal factor analysis in multiple populations

Abstract: For multiple populatios, a longtidinal factor analytic model which is entirely exploratory, that is, no explicit identification constraints, is proposed. Factorial collapse and period/practice effects are allowed. An invariant and/or stationary factor pattern is permitted. This model is formulated stochastically. To implement this model a stagewise EM algorithm is developed. Finally a numerical illustration utilizing Nesselroade and Baltes' data is presented.

Incorporating Prior Theory in Covariance Structure Analysis: A Bayesian Approach.

Abstract: A Bayesian approach to the testing of competing covariance structures is developed. The method provides approximate posterior probablities for each model under consideration without prior specification of individual parameter distributions. The method is based on ayesian updating using cross-validated pseudo-likelihoods. Given that the observed variables are the samefor all competing models, the approximate posterior probabilities may be obtained easily from the chi square values and other known constants, using only a hand calculator. The approach is illustrated using and example which illustrates how the prior probabilities can alter the results concerning which model specification is preferred.

On the Partitioning of Squared Euclidean Distance and Its Applications in Cluster Analysis.

Abstract: The partitioning of squared Eucliean distance between two vectors in M-dimensional space into the sum of squared lengths of vectors in mutually orthogonal subspaces is discussed and applications given to specific cluster analysis problems. Examples of how the partitioning idea can be used to help describe and interpret derived clusters, derive similarity measures for use in cluster analysis, and to design Monte Carlo studies with carefully specified types and magnitudes of differences between the underlying population mean vectors are presented. Most of the example applications presented in this paper involve the clustering of longitudinal data, but their use in cluster analysis need not be limited to this arena.

An alternating least squares algorithm for fitting the two- and three-way dedicom model and the idioscal model

Abstract: The DEDICOM model is a model for representing asymmetric relations among a set of objects by means of a set of coordinates for the objects on a limited number of dimensions. The present paper offers an alternating least squares algorithm for fitting the DEDICOM model. The model can be generalized to represent any number of sets of relations among the same set of objects. An algorithm for fitting this three-way DEDICOM model is provided as well. Based on the algorithm for the three-way DEDICOM model an algorithm is developed for fitting the IDIOSCAL model in the least squares sense.

Comparing the Variances of Dependent Groups.

Abstract: Recently several new attempts have been made to find a robust method for comparing the variances ofJ dependent random variables. However, empirical studies have shown that all of these procedures can give unsatisfactory results. This paper examines several new procedures that are derived heuristically. One of these procedures was found to perform better than all of the robust procedures studied here, and so it is recommended for general use.

Simultaneous Analysis of Multivariate Polytomous Variates in Several Groups.

Abstract: The frequencies ofm independentp-way contingency tables are analyzed by a model that assumes that the ordinal categorical data in each ofm groups are generated from a latent continuous multivariate normal distribution. The parameters of these multivariate distributions and of the relations between the ordinal and latent variables are estimated by maximum likelihood. Goodness-of-fit statistics based on the likelihood ratio criterion and the Pearsonian chisquare are provided to test the hypothesis that the proposed model is correct, that is, it fits the observed sample data. Hypotheses on the invariance of means, variances, and polychoric correlations of the latent variables across populations are tested by Wald statistics. The method is illustrated on an example involving data on three five-point ordinal scales obtained from male and female samples.

Ordinal consistency and ordinal true scores

Abstract: This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.

Minimizing the Number of Observations: A Generalization of the Spearman-Brown Formula.

Abstract: A new method for determining the minimum number of observations per subject needed to achieve a specific generalizability coefficient is presented. This method, which consists of a branch-and-bound algorithm, allows for the employment of constraints specified by the investigator.