Showing papers on "Latent variable model published in 2010"

Partial least squares regression and projection on latent structure regression (PLS Regression)

Abstract: Partial least squares (PLS) regression (a.k.a. projection on latent structures) is a recent technique that combines features from and generalizes principal component analysis (PCA) and multiple linear regression. Its goal is to predict a set of dependent variables from a set of independent variables or predictors. This prediction is achieved by extracting from the predictors a set of orthogonal factors called latent variables which have the best predictive power. These latent variables can be used to create displays akin to PCA displays. The quality of the prediction obtained from a PLS regression model is evaluated with cross-validation techniques such as the bootstrap and jackknife. There are two main variants of PLS regression: The most common one separates the roles of dependent and independent variables; the second one—used mostly to analyze brain imaging data—gives the same roles to dependent and independent variables. Copyright © 2010 John Wiley & Sons, Inc. For further resources related to this article, please visit the WIREs website.

Comorbidity: a network perspective.

Abstract: The pivotal problem of comorbidity research lies in the psychometric foundation it rests on, that is, latent variable theory, in which a mental disorder is viewed as a latent variable that causes a constellation of symptoms. From this perspective, comorbidity is a (bi)directional relationship between multiple latent variables. We argue that such a latent variable perspective encounters serious problems in the study of comorbidity, and offer a radically different conceptualization in terms of a network approach, where comorbidity is hypothesized to arise from direct relations between symptoms of multiple disorders. We propose a method to visualize comorbidity networks and, based on an empirical network for major depression and generalized anxiety, we argue that this approach generates realistic hypotheses about pathways to comorbidity, overlapping symptoms, and diagnostic boundaries, that are not naturally accommodated by latent variable models: Some pathways to comorbidity through the symptom space are more likely than others; those pathways generally have the same direction (i.e., from symptoms of one disorder to symptoms of the other); overlapping symptoms play an important role in comorbidity; and boundaries between diagnostic categories are necessarily fuzzy.

Modeling relationship strength in online social networks

Abstract: Previous work analyzing social networks has mainly focused on binary friendship relations. However, in online social networks the low cost of link formation can lead to networks with heterogeneous relationship strengths (e.g., acquaintances and best friends mixed together). In this case, the binary friendship indicator provides only a coarse representation of relationship information. In this work, we develop an unsupervised model to estimate relationship strength from interaction activity (e.g., communication, tagging) and user similarity. More specifically, we formulate a link-based latent variable model, along with a coordinate ascent optimization procedure for the inference. We evaluate our approach on real-world data from Facebook and LinkedIn, showing that the estimated link weights result in higher autocorrelation and lead to improved classification accuracy.

A Latent Variable Model for Geographic Lexical Variation

Abstract: The rapid growth of geotagged social media raises new computational possibilities for investigating geographic linguistic variation. In this paper, we present a multi-level generative model that reasons jointly about latent topics and geographical regions. High-level topics such as "sports" or "entertainment" are rendered differently in each geographic region, revealing topic-specific regional distinctions. Applied to a new dataset of geotagged microblogs, our model recovers coherent topics and their regional variants, while identifying geographic areas of linguistic consistency. The model also enables prediction of an author's geographic location from raw text, outperforming both text regression and supervised topic models.

Latent Class Models

Abstract: A statistical model can be called a latent class (LC) or mixture model if it assumes that some of its parameters differ across unobserved subgroups, LCs, or mixture components. This rather general idea has several seemingly unrelated applications, the most important of which are clustering, scaling, density estimation, and random-effects modeling. This article describes simple LC models for clustering, restricted LC models for scaling, and mixture regression models for nonparametric random-effects modeling, as well as gives an overview of recent developments in the field of LC analysis. Moreover, attention is paid to topics such as maximum likelihood estimation, identification issues, model selection, and software.

Bayesian Gaussian Process Latent Variable Model

Abstract: We introduce a variational inference framework for training the Gaussian process latent variable model and thus performing Bayesian nonlinear dimensionality reduction. This method allows us to variationally integrate out the input variables of the Gaussian process and compute a lower bound on the exact marginal likelihood of the nonlinear latent variable model. The maximization of the variational lower bound provides a Bayesian training procedure that is robust to overfitting and can automatically select the dimensionality of the nonlinear latent space. We demonstrate our method on real world datasets. The focus in this paper is on dimensionality reduction problems, but the methodology is more general. For example, our algorithm is immediately applicable for training Gaussian process models in the presence of missing or uncertain inputs.

A Two-Tier Full-Information Item Factor Analysis Model with Applications

Abstract: Motivated by Gibbons et al.’s (Appl. Psychol. Meas. 31:4–19, 2007) full-information maximum marginal likelihood item bifactor analysis for polytomous data, and Rijmen, Vansteelandt, and De Boeck’s (Psychometrika 73:167–182, 2008) work on constructing computationally efficient estimation algorithms for latent variable models, a two-tier item factor analysis model is developed in this research. The modeling framework subsumes standard multidimensional IRT models, bifactor IRT models, and testlet response theory models as special cases. Features of the model lead to a reduction in the dimensionality of the latent variable space, and consequently significant computational savings. An EM algorithm for full-information maximum marginal likelihood estimation is developed. Simulations and real data demonstrations confirm the accuracy and efficiency of the proposed methods. Three real data sets from a large-scale educational assessment, a longitudinal public health survey, and a scale development study measuring patient reported quality of life outcomes are analyzed as illustrations of the model’s broad range of applicability.

Unsupervised Object Discovery: A Comparison

Abstract: The goal of this paper is to evaluate and compare models and methods for learning to recognize basic entities in images in an unsupervised setting. In other words, we want to discover the objects present in the images by analyzing unlabeled data and searching for re-occurring patterns. We experiment with various baseline methods, methods based on latent variable models, as well as spectral clustering methods. The results are presented and compared both on subsets of Caltech256 and MSRC2, data sets that are larger and more challenging and that include more object classes than what has previously been reported in the literature. A rigorous framework for evaluating unsupervised object discovery methods is proposed.

Dynamical binary latent variable models for 3D human pose tracking

Abstract: We introduce a new class of probabilistic latent variable model called the Implicit Mixture of Conditional Restricted Boltzmann Machines (imCRBM) for use in human pose tracking. Key properties of the imCRBM are as follows: (1) learning is linear in the number of training exemplars so it can be learned from large datasets; (2) it learns coherent models of multiple activities; (3) it automatically discovers atomic “movemes” and (4) it can infer transitions between activities, even when such transitions are not present in the training set. We describe the model and how it is learned and we demonstrate its use in the context of Bayesian filtering for multi-view and monocular pose tracking. The model handles difficult scenarios including multiple activities and transitions among activities. We report state-of-the-art results on the HumanEva dataset.

Latent variable graphical model selection via convex optimization

Abstract: Suppose we have samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of hidden components, and to learn a statistical model over the entire collection of variables? We address this question in the setting in which the latent and observed variables are jointly Gaussian, with the conditional statistics of the observed variables conditioned on the latent variables being specified by a graphical model. As a first step we give natural conditions under which such latent-variable Gaussian graphical models are identifiable given marginal statistics of only the observed variables. Essentially these conditions require that the conditional graphical model among the observed variables is sparse, while the effect of the latent variables is “spread out” over most of the observed variables. Next we propose a tractable convex program based on regularized maximum-likelihood for model selection in this latent-variable setting; the regularizer uses both the l 1 norm and the nuclear norm. Our modeling framework can be viewed as a combination of dimensionality reduction (to identify latent variables) and graphical modeling (to capture remaining statistical structure not attributable to the latent variables), and it consistently estimates both the number of hidden components and the conditional graphical model structure among the observed variables. These results are applicable in the high-dimensional setting in which the number of latent/observed variables grows with the number of samples of the observed variables. The geometric properties of the algebraic varieties of sparse matrices and of low-rank matrices play an important role in our analysis.

Sequential and Simultaneous Estimation of Hybrid Discrete Choice Models: Some New Findings

Abstract: The formulation of hybrid discrete choice models, including both observable alternative attributes and latent variables associated with attitudes and perceptions, has become a topic of discussion once more To estimate models integrating both kinds of variables, two methods have been proposed: the sequential approach, in which the latent variables are built before their integration with the traditional explanatory variables in the choice model and the simultaneous approach, in which both processes are done together, albeit with a sophisticated but fairly complex treatment Here both approaches are applied to estimate hybrid choice models by using two data sets: one from the Santiago Panel (an urban mode choice context with many alternatives) and another consisting of synthetic data Differences between both approaches were found as well as similarities not found in earlier studies Even when both approaches result in unbiased estimators, problems arise when valuations are obtained such as the value of tim

Bayesian Nonparametric Models.

Abstract: A Bayesian nonparametric model is a Bayesian model on an infinite-dimensional parameter space. The parameter space is typically chosen as the set of all possible solutions for a given learning problem. For example, in a regression problem the parameter space can be the set of continuous functions, and in a density estimation problem the space can consist of all densities. A Bayesian nonparametric model uses only a finite subset of the available parameter dimensions to explain a finite sample of observations, with the set of dimensions chosen depending on the sample, such that the effective complexity of the model (as measured by the number of dimensions used) adapts to the data. Classical adaptive problems, such as nonparametric estimation and model selection, can thus be formulated as Bayesian inference problems. Popular examples of Bayesian nonparametric models include Gaussian process regression, in which the correlation structure is refined with growing sample size, and Dirichlet process mixture models for clustering, which adapt the number of clusters to the complexity of the data. Bayesian nonparametric models have recently been applied to a variety of machine learning problems, including regression, classification, clustering, latent variable modeling, sequential modeling, image segmentation, source separation and grammar induction.

Comparing the relative fit of categorical and dimensional latent variable models using consistency tests.

Abstract: A number of recent studies have used Meehl's (1995) taxometric method to determine empirically whether one should model assessment-related constructs as categories or dimensions. The taxometric method includes multiple data-analytic procedures designed to check the consistency of results. The goal is to differentiate between strong evidence of categorical structure, strong evidence of dimensional structure, and ambiguous evidence that suggests withholding judgment. Many taxometric consistency tests have been proposed, but their use has not been operationalized and studied rigorously. What tests should be performed, how should results be combined, and what thresholds should be applied? We present an approach to consistency testing that builds on prior work demonstrating that parallel analyses of categorical and dimensional comparison data provide an accurate index of the relative fit of competing structural models. Using a large simulation study spanning a wide range of data conditions, we examine many critical elements of this approach. The results provide empirical support for what marks the first rigorous operationalization of consistency testing. We discuss and empirically illustrate guidelines for implementing this approach and suggest avenues for future research to extend the practice of consistency testing to other techniques for modeling latent variables in the realm of psychological assessment.

Evaluation of Scale Reliability With Binary Measures Using Latent Variable Modeling

Abstract: A method for interval estimation of scale reliability with discrete data is outlined. The approach is applicable with multi-item instruments consisting of binary measures, and is developed within the latent variable modeling methodology. The procedure is useful for evaluation of consistency of single measures and of sum scores from item sets following the 2-parameter logistic model or the 1-parameter logistic model. An extension of the method is described for constructing confidence intervals of change in reliability due to instrument revision. The proposed procedure is illustrated with an example.

Testing for Measurement Invariance With Latent Class Analysis

Abstract: This chapter introduces the basic multigroup latent class (LC) model, and discusses two important extensions of the basic model, an extension for dealing with ordinal indicators and for modeling the latent variables as ordinal variables. It analyses measurement invariance using multigroup LC models, discussing the general procedure as well as methods for parameter estimation and evaluation of model fit. The multigroup extension of the standard LC model has been developed for the analysis of latent structures of observed categorical variables across two or more groups. The chapter presents three parameterizations of the multigroup LC models: probabilistic, log-linear, and logistic parameterizations. Multigroup LC models assume the presence of three types of categorical variables: observed variables; an unobserved variable that accounts for the relationships between the observed variables. LC models are usually estimated by means of maximum-likelihood under the assumption of a multinomial distribution for the indicator variables in the model.

A Latent Variable Model for Generative Dependency Parsing.

Abstract: We propose a generative dependency parsing model which uses binary latent variables to induce conditioning features. To define this model we use a recently proposed class of Bayesian Networks for structured prediction, Incremental Sigmoid Belief Networks. We demonstrate that the proposed model achieves state-of-the-art results on three different languages. We also demonstrate that the features induced by the ISBN's latent variables are crucial to this success, and show that the proposed model is particularly good on long dependencies.

Advances in Latent Variable Mixture Models

Abstract: Gregory R Hancock & Karen M Samuelson (Eds) Charlotte, NC: Information Age, 2008, 370 pages, $3999 (paperback) This volume, although not precisely a conference proceedings, is the result of a

Changes in Intentions, Planning, and Self-efficacy Predict Changes in Behaviors An Application of Latent True Change Modeling

Abstract: Can latent true changes in intention, planning, and self-efficacy account for latent change in two health behaviors (physical activity as well as fruit and vegetable intake)? Baseline data on predictors and behaviors and corresponding follow-up data four weeks later were collected from 853 participants. Interindividual differences in change and change-change associations were analyzed using structural equation modeling. For both behaviors, similar prediction patterns were found: changes in intention and self-efficacy predicted changes in planning, which in turn corresponded to changes in behavior. This evidence confirms that change predicts change, which is an inherent precondition in behavior change theories.

Exploring the Latent Structures of Psychological Constructs in Social Development Using the Dimensional-Categorical Spectrum.

Abstract: This paper provides an introduction to a recently developed conceptual framework— the dimensional–categorical spectrum—for utilizing general factor mixture models to explore the latent structures of psychological constructs. This framework offers advantages over traditional latent variable models that usually employ either continuous latent factors or categorical latent class variables to characterize the latent structure and require an a priori assumption about the underlying nature of the construct as either purely dimension or purely categorical. The modeling process is discussed in detail and then illustrated with data on the delinquency items of Achenbach’s child behavior checklist from a sample of children in the National Adolescent and Child Treatment Study.

A Multilevel Mixture IRT Model With an Application to DIF

Abstract: Mixture item response theory models have been suggested as a potentially useful methodology for identifying latent groups formed along secondary, possibly nuisance dimensions. In this article, we d...

Inferring 3D shapes and deformations from single views

Abstract: In this paper we propose a probabilistic framework that models shape variations and infers dense and detailed 3D shapes from a single silhouette. We model two types of shape variations, the object phenotype variation and its pose variation using two independent Gaussian Process Latent Variable Models (GPLVMs) respectively. The proposed shape variation models are learnt from 3D samples without prior knowledge about object class, e.g. object parts and skeletons, and are combined to fully span the 3D shape space. A novel probabilistic inference algorithm for 3D shape estimation is proposed by maximum likelihood estimates of the GPLVM latent variables and the camera parameters that best fit generated 3D shapes to given silhouettes. The proposed inference involves a small number of latent variables and it is computationally efficient. Experiments on both human body and shark data demonstrate the efficacy of our new approach.

Five Steps in Latent Curve and Latent Change Score Modeling with Longitudinal Data

Abstract: This paper describes a set of applications of one class of longitudinal growth analysis - latent curve (LCM) and latent change score (LCS) analysis using structural equation modeling (SEM) techniques. These techniques are organized in five sections based on Baltes & Nesselroade (1979). (1) Describing the observed and unobserved longitudinal data. (2) Characterizing the developmental shape of both individuals and groups. (3) Examining the predictors of individual and group differences in developmental shapes. (4) Studying dynamic determinants among variables over time. (5) Studying group differences in dynamic determinants among variables over time. To illustrate all steps, we present SEM analyses of a relatively large set of data from the National Longitudinal Survey of Youth (NLSY). The inclusion of all five aspects of latent curve modeling is not often used in longitudinal analyses, so we discuss why more efforts to include all five are needed in developmental research.

Bayesian Semiparametric Structural Equation Models with Latent Variables

Abstract: Structural equation models (SEMs) with latent variables are widely useful for sparse covariance structure modeling and for inferring relationships among latent variables. Bayesian SEMs are appealing in allowing for the incorporation of prior information and in providing exact posterior distributions of unknowns, including the latent variables. In this article, we propose a broad class of semiparametric Bayesian SEMs, which allow mixed categorical and continuous manifest variables while also allowing the latent variables to have unknown distributions. In order to include typical identifiability restrictions on the latent variable distributions, we rely on centered Dirichlet process (CDP) and CDP mixture (CDPM) models. The CDP will induce a latent class model with an unknown number of classes, while the CDPM will induce a latent trait model with unknown densities for the latent traits. A simple and efficient Markov chain Monte Carlo algorithm is developed for posterior computation, and the methods are illustrated using simulated examples, and several applications.

Topic models with power-law using Pitman-Yor process

Abstract: One important approach for knowledge discovery and data mining is to estimate unobserved variables because latent variables can indicate hidden specific properties of observed data. The latent factor model assumes that each item in a record has a latent factor; the co-occurrence of items can then be modeled by latent factors. In document modeling, a record indicates a document represented as a "bag of words," meaning that the order of words is ignored, an item indicates a word and a latent factor indicates a topic. Latent Dirichlet allocation (LDA) is a widely used Bayesian topic model applying the Dirichlet distribution over the latent topic distribution of a document having multiple topics. LDA assumes that latent topics, i.e., discrete latent variables, are distributed according to a multinomial distribution whose parameters are generated from the Dirichlet distribution. LDA also models a word distribution by using a multinomial distribution whose parameters follows the Dirichlet distribution. This Dirichlet-multinomial setting, however, cannot capture the power-law phenomenon of a word distribution, which is known as Zipf's law in linguistics. We therefore propose a novel topic model using the Pitman-Yor(PY) process, called the PY topic model. The PY topic model captures two properties of a document; a power-law word distribution and the presence of multiple topics. In an experiment using real data, this model outperformed LDA in document modeling in terms of perplexity.

Structural Equation Models of Latent Interactions: An Appropriate Standardized Solution and Its Scale-Free Properties

Abstract: Standardized parameter estimates are routinely used to summarize the results of multiple regression models of manifest variables and structural equation models of latent variables, because they facilitate interpretation. Although the typical standardization of interaction terms is not appropriate for multiple regression models, straightforward alternatives are well known (Aiken & West, 1991; Friedrich, 1982). Whereas the analogous problem exists for the estimation of latent interactions in structural equation modeling (SEM), the problem is more complex and apparently has not been resolved. Here we demonstrate that the appropriate “standardized” parameter estimates are easily formulated from parameter estimates routinely available from existing SEM software packages. Some properties of the appropriate “standardized” solution are mathematically derived, including the demonstration that the main and interaction effects are scale-free, as are the factor loadings. These desirable properties of the standardized...

Latent mixture models for multivariate and longitudinal outcomes

Abstract: Repeated measures and multivariate outcomes are an increasingly common feature of trials. Their joint analysis by means of random effects and latent variable models is appealing but patterns of heterogeneity in outcome profile may not conform to standard multivariate normal assumptions. In addition, there is much interest in both allowing for and identifying sub-groups of patients who vary in treatment responsiveness. We review methods based on discrete random effects distributions and mixture models for application in this field.