Showing papers on "Latent variable model published in 2007"

Mixed membership stochastic blockmodels

Abstract: Observations consisting of measurements on relationships for pairs of objects arise in many settings, such as protein interaction and gene regulatory networks, collections of author-recipient email, and social networks. Analyzing such data with probabilisic models can be delicate because the simple exchangeability assumptions underlying many boilerplate models no longer hold. In this paper, we describe a latent variable model of such data called the mixed membership stochastic blockmodel. This model extends blockmodels for relational data to ones which capture mixed membership latent relational structure, thus providing an object-specific low-dimensional representation. We develop a general variational inference algorithm for fast approximate posterior inference. We explore applications to social and protein interaction networks.

ltm: An R Package for Latent Variable Modeling and Item Response Analysis

Abstract: The R package ltm has been developed for the analysis of multivariate dichotomous and polytomous data using latent variable models, under the Item Response Theory approach. For dichotomous data the Rasch, the Two-Parameter Logistic, and Birnbaum's Three-Parameter models have been implemented, whereas for polytomous data Semejima's Graded Response model is available. Parameter estimates are obtained under marginal maximum likelihood using the Gauss-Hermite quadrature rule. The capabilities and features of the package are illustrated using two real data examples.

Hidden Conditional Random Fields

Abstract: We present a discriminative latent variable model for classification problems in structured domains where inputs can be represented by a graph of local observations. A hidden-state conditional random field framework learns a set of latent variables conditioned on local features. Observations need not be independent and may overlap in space and time.

New developments in latent variable panel analyses of longitudinal data.

Abstract: We review fundamental issues in one traditional structural equation modeling (SEM) approach to analyzing longitudinal data — cross-lagged panel designs. We then discuss a number of new developments in SEM that are applicable to analyzing panel designs. These issues include setting appropriate scales for latent variables, specifying an appropriate null model, evaluating factorial invariance in an appropriate manner, and examining both direct and indirect (mediated), effects in ways better suited for panel designs. We supplement each topic with discussion intended to enhance conceptual and statistical understanding.

The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis.

Abstract: In practice, the inclusion of correlated residuals in latent-variable models is often regarded as a statistical sleight of hand, if not an outright form of cheating. Consequently, researchers have tended to allow only as many correlated residuals in their models as are needed to obtain a good fit to the data. The current article demonstrates that this strategy leads to the underinclusion of residual correlations that are completely justified on the basis of measurement theory and research design. In many designs, the absence of such correlations will not substantially harm the fit of the model; however, failure to include them can change the meaning of the extracted latent variables and generate potentially misleading results. Recommendations include (a) returning to the full multitrait-multimethod design when measurement theory implies the existence of shared method variance and (b) abandoning the evil-but-necessary attitude toward correlated residuals when they reflect intended features of the research design.

Performance of Factor Mixture Models as a Function of Model Size, Covariate Effects, and Class-Specific Parameters.

Abstract: Factor mixture models are designed for the analysis of multivariate data obtained from a population consisting of distinct latent classes. A common factor model is assumed to hold within each of the latent classes. Factor mixture modeling involves obtaining estimates of the model parameters, and may also be used to assign subjects to their most likely latent class. This simulation study investigates aspects of model performance such as parameter coverage and correct class membership assignment and focuses on covariate effects, model size, and class-specific versus class-invariant parameters. When fitting true models, parameter coverage is good for most parameters even for the smallest class separation investigated in this study (0.5 SD between 2 classes). The same holds for convergence rates. Correct class assignment is unsatisfactory for the small class separation without covariates, but improves dramatically with increasing separation, covariate effects, or both. Model performance is not influe...

Latent Curve Models: A Structural Equation Approach

Abstract: Davison, A. C., and Hinkley, D. V. (1997), Bootstrap Methods and Their Application, Cambridge, U.K.: Cambridge University Press. Efron, B., and Tibshirani, R. (1993), An Introduction to the Bootstrap, New York: Chapman & Hall. Franke, J., and Härdle, W. (1992), “On Bootstrapping Kernel Estimates,” The Annals of Statistics, 20, 121–145. Rissanen, J. (1983), “A Universal Prior for Integers and Estimation by Minimum Description Length,” The Annals of Statistics, 11, 416–431.

Supervised and semi-supervised separation of sounds from single-channel mixtures

Abstract: In this paper we describe a methodology for model-based single channel separation of sounds. We present a sparse latent variable model that can learn sounds based on their distribution of time/ frequency energy. This model can then be used to extract known types of sounds from mixtures in two scenarios. One being the case where all sound types in the mixture are known, and the other being being the case where only the target or the interference models are known. The model we propose has close ties to non-negative decompositions and latent variable models commonly used for semantic analysis.

Predicting group-level outcome variables from variables measured at the individual level : A latent variable multilevel model

Abstract: In multilevel modeling, one often distinguishes between macro-micro and micro-macro situations. In a macro-micro multilevel situation, a dependent variable measured at the lower level is predicted or explained by variables measured at that lower or a higher level. In a micro-macro multilevel situation, a dependent variable defined at the higher group level is predicted or explained on the basis of independent variables measured at the lower individual level. Up until now, multilevel methodology has mainly focused on macro-micro multilevel situations. In this article, a latent variable model is proposed for analyzing data from micro-macro situations. It is shown that regression analyses carried out at the aggregated level result in biased parameter estimates. A method that uses the best linear unbiased predictors of the group means is shown to yield unbiased estimates of the parameters.

Modeling homophily and stochastic equivalence in symmetric relational data

Abstract: This article discusses a latent variable model for inference and prediction of symmetric relational data. The model, based on the idea of the eigenvalue decomposition, represents the relationship between two nodes as the weighted inner-product of node-specific vectors of latent characteristics. This "eigenmodel" generalizes other popular latent variable models, such as latent class and distance models: It is shown mathematically that any latent class or distance model has a representation as an eigenmodel, but not vice-versa. The practical implications of this are examined in the context of three real datasets, for which the eigenmodel has as good or better out-of-sample predictive performance than the other two models.

Latent variable analyses of age trends of cognition in the Health and Retirement Study, 1992-2004.

Abstract: The present study was conducted to better describe age trends in cognition among older adults in the longitudinal Health and Retirement Study (HRS) from 1992 to 2004 (N = 17,000). The authors used contemporary latent variable models to organize this information in terms of both cross-sectional and longitudinal inferences about age and cognition. Common factor analysis results yielded evidence for at least 2 common factors, labeled Episodic Memory and Mental Status, largely separable from vocabulary. Latent path models with these common factors were based on demographic characteristics. Multilevel models of factorial invariance over age indicated that at least 2 common factors were needed. Latent curve models of episodic memory were based on age at testing and showed substantial age differences and age changes, including impacts due to retesting as well as several time-invariant and time-varying predictors.

Modeling homophily and stochastic equivalence in symmetric relational data

Abstract: This article discusses a latent variable model for inference and prediction of symmetric relational data. The model, based on the idea of the eigenvalue decomposition, represents the relationship between two nodes as the weighted inner-product of node-specific vectors of latent characteristics. This ``eigenmodel'' generalizes other popular latent variable models, such as latent class and distance models: It is shown mathematically that any latent class or distance model has a representation as an eigenmodel, but not vice-versa. The practical implications of this are examined in the context of three real datasets, for which the eigenmodel has as good or better out-of-sample predictive performance than the other two models.

An Introduction to Latent Variable Growth Curve Modeling: Concepts, Issues, and Applications (2nd ed.)

Abstract: This book approaches latent growth modeling (LGM) from a practical, hands-on perspective. LGM, a set of techniques for analyzing change over time, is applied within a structural equation modeling (...

Gaussian process latent variable models for human pose estimation

Abstract: We describe a method for recovering 3D human body pose from silhouettes. Our model is based on learning a latent space using the Gaussian Process Latent Variable Model (GP-LVM) [1] encapsulating both pose and silhouette features Our method is generative, this allows us to model the ambiguities of a silhouette representation in a principled way. We learn a dynamical model over the latent space which allows us to disambiguate between ambiguous silhouettes by temporal consistency. The model has only two free parameters and has several advantages over both regression approaches and other generative methods. In addition to the application shown in this paper the suggested model is easily extended to multiple observation spaces without constraints on type.

Latent variable modelling: a survey

Abstract: Latent variable modelling has gradually become an integral part of mainstream statistics and is currently used for a multitude of applications in different subject areas. Examples of 'traditional' latent variable models include latent class models, item-response models, common factor models, structural equation models, mixed or random effects models and covariate measurement error models. Although latent variables have widely different interpretations in different settings, the models have a very similar mathematical structure. This has been the impetus for the formulation of general modelling frameworks which accommodate a wide range of models. Recent developments include multilevel structural equation models with both continuous and discrete latent variables, multiprocess models and nonlinear latent variable models. © 2007 Board of the Foundation of the Scandinavian Journal of Statistics.

Multilevel structural equation modeling

Abstract: In conventional structural equation models, all latent variables and indicators vary between units (typically subjects) and are assumed to be independent across units. The latter assumption is violated in multilevel settings where units are nested in clusters, leading to within-cluster dependence. Different approaches to extending structural equation models for such multilevel settings are examined. The most common approach is to formulate separate within-cluster and between-cluster models. An advantage of this set-up is that it allows software for conventional structural equation models to be ‘tricked’ into estimating the model. However, the standard implementation of this approach does not permit cross-level paths from latent or observed variables at a higher level to latent or observed variables at a lower level, and does not allow for indicators varying at higher levels. A multilevel regression (or path) model formulation is therefore suggested in which some of the response variables and some of the explanatory variables at the different levels are latent and measured by multiple indicators. The Generalized Linear Latent and Mixed Modeling (GLLAMM) framework allows such models to be specified by simply letting the usual structural part of the model include latent and observed variables varying at different levels. Models of this kind are applied to the U.S. sample of the Program for International Student Assessment (PISA) 2000 to investigate the relationship between the school-level latent variable ‘teacher excellence’ and the student-level latent variable ‘reading ability’, each measured by multiple ordinal indicators.

Semiparametric latent variable regression models for spatiotemporal modelling of mobile source particles in the greater Boston area

Abstract: Summary. Traffic particle concentrations show considerable spatial variability within a metropolitan area. We consider latent variable semiparametric regression models for modelling the spatial and temporal variability of black carbon and elemental carbon concentrations in the greater Boston area. Measurements of these pollutants, which are markers of traffic particles, were obtained from several individual exposure studies that were conducted at specific household locations as well as 15 ambient monitoring sites in the area. The models allow for both flexible non-linear effects of covariates and for unexplained spatial and temporal variability in exposure. In addition, the different individual exposure studies recorded different surrogates of traffic particles, with some recording only outdoor concentrations of black or elemental carbon, some recording indoor concentrations of black carbon and others recording both indoor and outdoor concentrations of black carbon. A joint model for outdoor and indoor exposure that specifies a spatially varying latent variable provides greater spatial coverage in the area of interest. We propose a penalized spline formulation of the model that relates to generalized kriging of the latent traffic pollution variable and leads to a natural Bayesian Markov chain Monte Carlo algorithm for model fitting. We propose methods that allow us to control the degrees of freedom of the smoother in a Bayesian framework. Finally, we present results from an analysis that applies the model to data from summer and winter separately.

Learning for Larger Datasets with the Gaussian Process Latent Variable Model

Abstract: In this paper we apply the latest techniques in sparse Gaussian process regression (GPR) to the Gaussian process latent variable model (GPLVM). We review three techniques and discuss how they may be implemented in the context of the GP-LVM. Each approach is then implemented on a well known benchmark data set and compared with earlier attempts to sparsify the model.

Predictive discrete latent factor models for large scale dyadic data

Abstract: We propose a novel statistical method to predict large scale dyadic response variables in the presence of covariate information. Our approach simultaneously incorporates the effect of covariates and estimates local structure that is induced by interactions among the dyads through a discrete latent factor model. The discovered latent factors provide a redictive model that is both accurate and interpretable. We illustrate our method by working in a framework of generalized linear models, which include commonly used regression techniques like linear regression, logistic regression and Poisson regression as special cases. We also provide scalable generalized EM-based algorithms for model fitting using both "hard" and "soft" cluster assignments. We demonstrate the generality and efficacy of our approach through large scale simulation studies and analysis of datasets obtained from certain real-world movie recommendation and internet advertising applications.

"Pre-conditioning" for feature selection and regression in high-dimensional problems

Abstract: We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a "pre-conditioned" response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the pre-conditioned response variable. In a number of simulated and real data examples, this two-step procedure outperforms forward stepwise selection or the usual LASSO (applied directly to the raw outcome). We also show that under a certain Gaussian latent variable model, application of the LASSO to the pre-conditioned response variable is consistent as the number of predictors and observations increases. Moreover, when the observational noise is rather large, the suggested procedure can give a more accurate estimate than LASSO. We illustrate our method on some real problems, including survival analysis with microarray data.

Estimating parameters of dichotomous and ordinal item response models with gllamm

Abstract: Item response theory models are measurement models for categorical responses. Traditionally, the models are used in educational testing, where re- sponses to test items can be viewed as indirect measures of latent ability. The test items are scored either dichotomously (correct{incorrect) or by using an ordinal scale (a grade from poor to excellent). Item response models also apply equally for measurement of other latent traits. Here we describe the one- and two-parameter logit models for dichotomous items, the partial-credit and rating scale models for ordinal items, and an extension of these models where the latent variable is re- gressed on explanatory variables. We show how these models can be expressed as generalized linear latent and mixed models and tted by using the user-written command gllamm.

Challenges in Nonlinear Structural Equation Modeling

Abstract: . Challenges in evaluating nonlinear effects in multiple regression analyses include reliability, validity, multicollinearity, and dichotomization of continuous variables. While reliability and validity issues are solved by employing nonlinear structural equation modeling, multicollinearity remains a problem which may even be aggravated when using latent variable approaches. Further challenges of nonlinear latent analyses comprise the distribution of latent product terms, a problem especially relevant for approaches using maximum likelihood estimation methods based on multivariate normally distributed variables, and unbiased estimates of nonlinear effects under multicollinearity. The only methods that explicitly take the nonnormality of nonlinear latent models into account are latent moderated structural equations (LMS) and quasi-maximum likelihood (QML). In a small simulation study both methods yielded unbiased parameter estimates and correct estimates of standard errors for inferential statistic...

1. On the Assignment of Individuals to Latent Classes

Abstract: Consider an m-way cross-classification table (for m = 3, 4, …) of m dichotomous variables that describes (1) the 2m possible response patterns to a set of m questions (where the response to each question is binary), and (2) the number of individuals whose responses to the m questions can be described by a particular response pattern, for each of the 2m possible response patterns. Consider the situation where the data in the cross-classification table are analyzed using a particular latent class model having T latent classes (for T = 2, 3, …), and where this model fits the data well. With this latent class model, it is possible to estimate, for an individual who has a particular response pattern, what is the conditional probability that this individual is in a particular latent class, for each of the T latent classes. In this article, the following question is considered: For an individual who has a particular response pattern, can we use the corresponding estimated conditional probabilities to assign this...