Showing papers on "Latent variable model published in 2008"

An introduction to latent class growth analysis and growth mixture modeling.

Abstract: In recent years, there has been a growing interest among researchers in the use of latent class and growth mixture modeling techniques for applications in the social and psychological sciences, in part due to advances in and availability of computer software designed for this purpose (e.g., Mplus and SAS Proc Traj). Latent growth modeling approaches, such as latent class growth analysis (LCGA) and growth mixture modeling (GMM), have been increasingly recognized for their usefulness for identifying homogeneous subpopulations within the larger heterogeneous population and for the identification of meaningful groups or classes of individuals. The purpose of this paper is to provide an overview of LCGA and GMM, compare the different techniques of latent growth modeling, discuss current debates and issues, and provide readers with a practical guide for conducting LCGA and GMM using the Mplus software. Researchers in the fields of social and psychological sciences are often interested in modeling the longitudinal developmental trajectories of individuals, whether for the study of personality development or for better understanding how social behaviors unfold over time (whether it be days, months, or years). This usually requires an extensive dataset consisting of longitudinal, repeated measures of variables, sometimes including multiple cohorts, and analyzing this data using various longitudinal latent variable modeling techniques such as latent growth curve models (cf. MacCallum & Austin, 2000). The objective of these approaches is to capture information about interindividual differences in intraindividual change over time (Nesselroade, 1991). However, conventional growth modeling approaches assume that individuals come from a single population and that a single growth trajectory can adequately approximate an entire population. Also, it is assumed that covariates that affect the growth factors influence each individual in the same way. Yet, theoretical frameworks and existing studies often categorize individuals into distinct subpopulations (e.g., socioeconomic classes, age groups, at-risk populations). For example, in the field of alcohol research, theoretical literature suggests different classes

Gaussian Process Dynamical Models for Human Motion

Abstract: We introduce Gaussian process dynamical models (GPDMs) for nonlinear time series analysis, with applications to learning models of human pose and motion from high-dimensional motion capture data. A GPDM is a latent variable model. It comprises a low-dimensional latent space with associated dynamics, as well as a map from the latent space to an observation space. We marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings. This results in a nonparametric model for dynamical systems that accounts for uncertainty in the model. We demonstrate the approach and compare four learning algorithms on human motion capture data, in which each pose is 50-dimensional. Despite the use of small data sets, the GPDM learns an effective representation of the nonlinear dynamics in these spaces.

The Multilevel Latent Covariate Model: A New, More Reliable Approach to Group-Level Effects in Contextual Studies.

Abstract: In multilevel modeling (MLM), group-level (L2) characteristics are often measured by aggregating individual-level (L1) characteristics within each group so as to assess contextual effects (e.g., group-average effects of socioeconomic status, achievement, climate). Most previous applications have used a multilevel manifest covariate (MMC) approach, in which the observed (manifest) group mean is assumed to be perfectly reliable. This article demonstrates mathematically and with simulation results that this MMC approach can result in substantially biased estimates of contextual effects and can substantially underestimate the associated standard errors, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the sampling ratio (the percentage of cases within each group sampled), and the nature of the data. To address this pervasive problem, the authors introduce a new multilevel latent covariate (MLC) approach that corrects for unreliability at L2 and results in unbiased estimates of L2 constructs under appropriate conditions. However, under some circumstances when the sampling ratio approaches 100%, the MMC approach provides more accurate estimates. Based on 3 simulations and 2 real-data applications, the authors evaluate the MMC and MLC approaches and suggest when researchers should most appropriately use one, the other, or a combination of both approaches.

Latent Growth Curve Modeling

Abstract: About the Authors Series Editor Introduction Acknowledgements 1. Introduction 2. Applying LGM to Empirical Data 3. Specialized Extensions 4. Relationships Between LGM and Multilevel Modeling 5. Summary Appendix References

Latent class and finite mixture models for multilevel data sets

Abstract: An extension of latent class (LC) and finite mixture models is described for the analysis of hierarchical data sets. As is typical in multilevel analysis, the dependence between lower-level units within higher-level units is dealt with by assuming that certain model parameters differ randomly across higher-level observations. One of the special cases is an LC model in which group-level differences in the logit of belonging to a particular LC are captured with continuous random effects. Other variants are obtained by including random effects in the model for the response variables rather than for the LCs. The variant that receives most attention in this article is an LC model with discrete random effects: higher-level units are clustered based on the likelihood of their members belonging to the various LCs. This yields a model with mixture distributions at two levels, namely at the group and the subject level. This model is illustrated with three rather different empirical examples. The appendix describes ...

A New SAS Procedure for Latent Transition Analysis: Transitions in Dating and Sexual Risk Behavior

Abstract: The set of statistical methods available to developmentalists is continually being expanded, allowing for questions about change over time to be addressed in new, informative ways. Indeed, new developments in methods to model change over time create the possibility for new research questions to be posed. Latent transition analysis, a longitudinal extension of latent class analysis, is a method that can be used to model development in discrete latent variables, for example, stage processes, over 2 or more times. The current article illustrates this approach using a new SAS procedure, PROC LTA, to model change over time in adolescent and young adult dating and sexual risk behavior. Gender differences are examined, and substance use behaviors are included as predictors of initial status in dating and sexual risk behavior and transitions over time.

Latent variable theory

Abstract: This paper formulates a metatheoretical framework for latent variable modeling. It does so by spelling out the difference between observed and latent variables. This difference is argued to be purely epistemic in nature: We treat a variable as observed when the inference from data structure to variable structure can be made with certainty and as latent when this inference is prone to error. This difference in epistemic accessibility is argued to be directly related to the data-generating process, i.e., the process that produces the concrete data patterns on which statistical analyses are executed. For a variable to count as observed through a set of data patterns, the relation between variable structure and data structure should be (a) deterministic, (b) causally isolated, and (c) of equivalent cardinality. When any of these requirements is violated, (part of) the variable structure should be considered latent. It is argued that, on these criteria, observed variables are rare to nonexistent in psychology;...

Factorial Invariance and the Specification of Second-Order Latent Growth Models

Abstract: Latent growth modeling has been a topic of intense interest during the past two decades. Most theoretical and applied work has employed first-order growth models, in which a single manifest variable serves as indicator of trait level at each time of measurement. In the current paper, we concentrate on issues regarding second-order growth models, which have multiple indicators at each time of measurement. With multiple indicators, tests of factorial invariance of parameters across times of measurement can be tested. We conduct such tests using two sets of data, which differ in the extent to which factorial invariance holds, and evaluate longitudinal confirmatory factor, latent growth curve, and latent difference score models. We demonstrate that, if factorial invariance fails to hold, choice of indicator used to identify the latent variable can have substantial influences on the characterization of patterns of growth, strong enough to alter conclusions about growth. We also discuss matters related to the scaling of growth factors and conclude with recommendations for practice and for future research.

Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models

Abstract: Factor mixture models (FMM's) are latent variable models with categorical and continuous latent variables which can be used as a model-based approach to clustering. A previous paper covered the results of a simulation study showing that in the absence of model violations, it is usually possible to choose the correct model when fitting a series of models with different numbers of classes and factors within class. The response format in the first study was limited to normally distributed outcomes. The current paper has two main goals, firstly, to replicate parts of the first study with 5-point Likert scale and binary outcomes, and secondly, to address the issue of testing class invariance of thresholds and loadings. Testing for class invariance of parameters is important in the context of measurement invariance and when using mixture models to approximate non-normal distributions. Results show that it is possible to discriminate between latent class models and factor models even if responses are categorical. Comparing models with and without class-specific parameters can lead to incorrectly accepting parameter invariance if the compared models differ substantially with respect to the number of estimated parameters. The simulation study is complemented with an illustration of a factor mixture analysis of ten binary depression items obtained from a female subsample of the Virginia Twin Registry.

Classical latent variable models for medical research

Abstract: Latent variable models are commonly used in medical statistics, although often not referred to under this name. In this paper we describe classical latent variable models such as factor analysis, item response theory, latent class models and structural equation models. Their usefulness in medical research is demonstrated using real data. Examples include measurement of forced expiratory flow, measurement of physical disability, diagnosis of myocardial infarction and modelling the determinants of clients' satisfaction with counsellors' interviews.

A Discriminative Latent Variable Model for Statistical Machine Translation

Abstract: Large-scale discriminative machine translation promises to further the state-of-the-art, but has failed to deliver convincing gains over current heuristic frequency count systems. We argue that a principle reason for this failure is not dealing with multiple, equivalent translations. We present a translation model which models derivations as a latent variable, in both training and decoding, and is fully discriminative and globally optimised. Results show that accounting for multiple derivations does indeed improve performance. Additionally, we show that regularisation is essential for maximum conditional likelihood models in order to avoid degenerate solutions.

Multiple imputation of incomplete categorical data using latent class analysis

Abstract: We propose using latent class analysis as an alternative to log-linear analysis for the multiple imputation of incomplete categorical data. Similar to log-linear models, latent class models can be used to describe complex association structures between the variables used in the imputation model. However, unlike log-linear models, latent class models can be used to build large imputation models containing more than a few categorical variables. To obtain imputations reflecting uncertainty about the unknown model parameters, we use a nonparametric bootstrap procedure as an alternative to the more common full Bayesian approach. The proposed multiple imputation method, which is implemented in Latent GOLD software for latent class analysis, is illustrated with two examples. In a simulated data example, we compare the new method to well-established methods such as maximum likelihood estimation with incomplete data and multiple imputation using a saturated log-linear model. This example shows that the proposed me...

"Preconditioning" for feature selection and regression in high-dimensional problems

Abstract: We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a "preconditioned" response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the preconditioned response variable. In a number of simulated and real data examples, this two-step procedure outperforms forward stepwise selection or the usual LASSO (applied directly to the raw outcome). We also show that under a certain Gaussian latent variable model, application of the LASSO to the preconditioned response variable is consistent as the number of predictors and observations increases. Moreover, when the observational noise is rather large, the suggested procedure can give a more accurate estimate than LASSO. We illustrate our method on some real problems, including survival analysis with microarray data.

Flexible latent variable models for multi-task learning

Abstract: Given multiple prediction problems such as regression or classification, we are interested in a joint inference framework that can effectively share information between tasks to improve the prediction accuracy, especially when the number of training examples per problem is small. In this paper we propose a probabilistic framework which can support a set of latent variable models for different multi-task learning scenarios. We show that the framework is a generalization of standard learning methods for single prediction problems and it can effectively model the shared structure among different prediction tasks. Furthermore, we present efficient algorithms for the empirical Bayes method as well as point estimation. Our experiments on both simulated datasets and real world classification datasets show the effectiveness of the proposed models in two evaluation settings: a standard multi-task learning setting and a transfer learning setting.

Latent-Variable Modeling of String Transductions with Finite-State Methods

Abstract: String-to-string transduction is a central problem in computational linguistics and natural language processing. It occurs in tasks as diverse as name transliteration, spelling correction, pronunciation modeling and inflectional morphology. We present a conditional loglinear model for string-to-string transduction, which employs overlapping features over latent alignment sequences, and which learns latent classes and latent string pair regions from incomplete training data. We evaluate our approach on morphological tasks and demonstrate that latent variables can dramatically improve results, even when trained on small data sets. On the task of generating morphological forms, we outperform a baseline method reducing the error rate by up to 48%. On a lemmatization task, we reduce the error rates in Wicentowski (2002) by 38--92%.

Joint modelling of mixed outcome types using latent variables.

Abstract: After a brief review of the use of latent variables to accommodate the correlation among multiple outcomes of mixed types, through theoretical and numerical calculation, the consequences of such a construction are quantified. The effects of including latent variables on marginal inference in these models are contrasted with the situation for jointly normal outcomes. A simulation study illustrates the efficiency and reduction in bias gains possible in using joint models, and analysis of an example from the field of osteoarthritis illustrates potential practical differences.

On the meaning of the latent variables in the CT-C(M-1) model: a comment on Maydeu-Olivares and Coffman (2006).

Abstract: In a recent article, A. Maydeu-Olivares and D. L. Coffman (2006) presented a random intercept factor approach for modeling idiosyncratic response styles in questionnaire data and compared this approach with competing confirmatory factor analysis models. Among the competing models was the CT-C(M-1) model (M. Eid, 2000). In an application to the Life Orientation Test (M. F. Scheier & C. S. Carver, 1985), Maydeu-Olivares and Coffman found that results obtained from the CT-C(M-1) model were difficult to interpret. In particular, Maydeu-Olivares and Coffman challenged the asymmetry of the CT-C(M-1) model. In the present article, the authors show that the difficulties faced by Maydeu-Olivares and Coffman rest upon an improper interpretation of the meaning of the latent factors. The authors' aim is to clarify the meaning of the latent variables in the CT-C(M-1) model. The authors explain how to properly interpret the results from this model and introduce an alternative restricted model that is conceptually similar to the CT-C(M-1) model and nested within it. The fit of this model is invariant across different reference methods. Finally, the authors provide guidelines as to which model should be used in which research context.

Bayesian latent variable modelling of multivariate spatio-temporal variation in cancer mortality.

Abstract: In this article, three alternative Bayesian hierarchical latent factor models are described for spatially and temporally correlated multivariate health data. The fundamentals of factor analysis with ideas of space- time disease mapping to provide a flexible framework for the joint analysis of multiple-related diseases in space and time with a view to estimating common and disease-specific trends in cancer risk are combined. The models are applied to area-level mortality data on six diet-related cancers for Greece over the 20-year period from 1980 to 1999. The aim of this study is to uncover the spatial and temporal patterns of any latent factor(s) underlying the cancer data that could be interpreted as reflecting some aspects of the habitual diet of the Greek population.

Statistical Modelling for Social Researchers: Principles and Practice

Abstract: 1. Statistical Modelling: An Overview 2. Research Designs and Data 3. Statistical Preliminaries 4. Multiple Regression for Continuous Response Variables 5. Logistic Regression for Binary Response Variables 6. Multinomial Logistic Regression for Multinomial Response Variables 7. Loglinear Modelling 8. Ordinal Logistic Regression for Ordered Categorical Response Variables 9. Multilevel Modelling 10. Latent Variables and Factor Analysis 11. Causal Modelling: Simultaneous Equation and Structural Equation Models 12. Longitudinal Data Analysis 13. Event History Models

Locally dependent latent class models with covariates: an application to under-age drinking in the USA.

Abstract: Under-age drinking is a long-standing public health problem in the USA and the identification of underage drinkers suffering alcohol-related problems has been difficult by using diagnostic criteria that were developed in adult populations. For this reason, it is important to characterize patterns of drinking in adolescents that are associated with alcohol-related problems. Latent class analysis is a statistical technique for explaining heterogeneity in individual response patterns in terms of a smaller number of classes. However, the latent class analysis assumption of local independence may not be appropriate when examining behavioural profiles and could have implications for statistical inference. In addition, if covariates are included in the model, non-differential measurement is also assumed. We propose a flexible set of models for local dependence and differential measurement that use easily interpretable odds ratio parameterizations while simultaneously fitting a marginal regression model for the latent class prevalences. Estimation is based on solving a set of second-order estimating equations. This approach requires only specification of the first two moments and allows for the choice of simple 'working' covariance structures. The method is illustrated by using data from a large-scale survey of under-age drinking. This new approach indicates the effectiveness of introducing local dependence and differential measurement into latent class models for selecting substantively interpretable models over more complex models that are deemed empirically superior.

Ambiguity Modeling in Latent Spaces

Abstract: We are interested in the situation where we have two or more representations of an underlying phenomenon. In particular we are interested in the scenario where the representation are complementary. This implies that a single individual representation is not sufficient to fully discriminate a specific instance of the underlying phenomenon, it also means that each representation is an ambiguous representation of the other complementary spaces. In this paper we present a latent variable model capable of consolidating multiple complementary representations. Our method extends canonical correlation analysis by introducing additional latent spaces that are specific to the different representations, thereby explaining the full variance of the observations. These additional spaces, explaining representation specific variance, separately model the variance in a representation ambiguous to the other. We develop a spectral algorithm for fast computation of the embeddings and a probabilistic model (based on Gaussian processes) for validation and inference. The proposed model has several potential application areas, we demonstrate its use for multi-modal regression on a benchmark human pose estimation data set.

Context and observation driven latent variable model for human pose estimation

Abstract: Current approaches to pose estimation and tracking can be classified into two categories: generative and discriminative. While generative approaches can accurately determine human pose from image observations, they are computationally expensive due to search in the high dimensional human pose space. On the other hand, discriminative approaches do not generalize well, but are computationally efficient. We present a hybrid model that combines the strengths of the two in an integrated learning and inference framework. We extend the Gaussian process latent variable model (GPLVM) to include an embedding from observation space (the space of image features) to the latent space. GPLVM is a generative model, but the inclusion of this mapping provides a discriminative component, making the model observation driven. Observation Driven GPLVM (OD-GPLVM) not only provides a faster inference approach, but also more accurate estimates (compared to GPLVM) in cases where dynamics are not sufficient for the initialization of search in the latent space. We also extend OD-GPLVM to learn and estimate poses from parameterized actions/gestures. Parameterized gestures are actions which exhibit large systematic variation in joint angle space for different instances due to difference in contextual variables. For example, the joint angles in a forehand tennis shot are function of the height of the ball (Figure 2). We learn these systematic variations as a function of the contextual variables. We then present an approach to use information from scene/objects to provide context for human pose estimation for such parameterized actions.

Dimensionality Reduction of Clustered Data Sets

Abstract: We present a novel probabilistic latent variable model to perform linear dimensionality reduction on data sets which contain clusters. We prove that the maximum likelihood solution of the model is an unsupervised generalization of linear discriminant analysis. This provides a completely new approach to one of the most established and widely used classification algorithms. The performance of the model is then demonstrated on a number of real and artificial data sets.

Sparse and shift-invariant feature extraction from non-negative data

Abstract: In this paper we describe a technique that allows the extraction of multiple local shift-invariant features from analysis of non-negative data of arbitrary dimensionality. Our approach employs a probabilistic latent variable model with sparsity constraints. We demonstrate its utility by performing feature extraction in a variety of domains ranging from audio to images and video.

A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies

Abstract: We propose a solution to the challenge of the CoNLL 2008 shared task that uses a generative history-based latent variable model to predict the most likely derivation of a synchronous dependency parser for both syntactic and semantic dependencies. The submitted model yields 79.1% macro-average F1 performance, for the joint task, 86.9% syntactic dependencies LAS and 71.0% semantic dependencies F1. A larger model trained after the deadline achieves 80.5% macro-average F1, 87.6% syntactic dependencies LAS, and 73.1% semantic dependencies F1.

Employing Latent Class Regression Analysis to Examine Logistics Theory: An Application of Truck Driver Retention

Abstract: Multiple regression analysis assumes that one model or theory is relevant for the entire population, yet research has shown that this assumption is often false and may severely limit valid theory development and testing. Latent class regression analysis overcomes this limitation and allows the researcher to identify and develop regression models that are relevant for different segments within the same population. Latent class regression analysis is introduced and is used to analyze truck drivers' intentions to stay with the same firm. This article demonstrates the advantages of testing logistics theory with latent class regression analysis and provides numerous applications for practitioners.

Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models.

Abstract: The analysis of convergent and discriminant validity is an integral part of the construct validation process. Models for analyzing the convergent and discriminant validity have typically been developed for cross-sectional data. There exist, however, only a few approaches for longitudinal data that can be applied for analyzing the construct validity of fluctuating states. In this article, the authors show how models of latent state-trait theory can be combined with models of multitrait-multimethod analysis to develop a model that allows for analyzing convergent and discriminant validity in time: the multimethod latent state-trait model. The model allows for identifying different sources of variance (trait consistency, trait-method specificity, occasion-specific consistency, occasion-specific method specificity, and unreliability). It is applied to the repeated measurement of depression and anxiety in children, which was assessed by self and teacher reports (N = 375). The application shows that the proposed models fit the data well and allow a deeper understanding of method effects in clinical assessment.

Latent class models for diary method data: Parameter estimation by local computations

Abstract: The increasing use of diary methods calls for the development of appropriate statistical methods. For the resulting panel data, latent Markov models can be used to model both individual differences and temporal dynamics. The computational burden associated with these models can be overcome by exploiting the conditional independence relations implied by the model. This is done by associating a probabilistic model with a directed acyclic graph, and applying transformations to the graph. The structure of the transformed graph provides a factorization of the joint probability function of the manifest and latent variables, which is the basis of a modified and more efficient E-step of the EM algorithm. The usefulness of the approach is illustrated by estimating a latent Markov model involving a large number of measurement occasions and, subsequently, a hierarchical extension of the latent Markov model that allows for transitions at different levels. Furthermore, logistic regression techniques are used to incorporate restrictions on the conditional probabilities and to account for the effect of covariates. Throughout, models are illustrated with an experience sampling methodology study on the course of emotions among anorectic patients.