Showing papers in "Multivariate Behavioral Research in 2005"

Probing Interactions in Fixed and Multilevel Regression: Inferential and Graphical Techniques

Abstract: Many important research hypotheses concern conditional relations in which the effect of one predictor varies with the value of another. Such relations are commonly evaluated as multiplicative interactions and can be tested in both fixed- and random-effects regression. Often, these interactive effects must be further probed to fully explicate the nature of the conditional relation. The most common method for probing interactions is to test simple slopes at specific levels of the predictors. A more general method is the Johnson-Neyman (J-N) technique. This technique is not widely used, however, because it is currently limited to categorical by continuous interactions in fixed-effects regression and has yet to be extended to the broader class of random-effects regression models. The goal of our article is to generalize the J-N technique to allow for tests of a variety of interactions that arise in both fixed- and random-effects regression. We review existing methods for probing interactions, explicate the analytic expressions needed to expand these tests to a wider set of conditions, and demonstrate the advantages of the J-N technique relative to simple slopes with three empirical examples.

Using Parcels to Convert Path Analysis Models Into Latent Variable Models.

Abstract: The biasing effects of measurement error in path analysis models can be overcome by the use of latent variable models. In cases where path analysis is used in practice, it is often possible to use parcels as indicators of a latent variable. The purpose of the current study was to compare latent variable models in which parcels were used as indicators of the latent variables, path analysis models of the aggregated variables, and models in which reliability estimates were used to correct for measurement error in path analysis models. Results showed that point estimates of path coefficients were smallest for the path analysis models and largest for the latent variable models. It is concluded that, whenever possible, it is better to use a latent variable model in which parcels are used as indicators than a path analysis model using total scale scores.

Fit Indices Versus Test Statistics

Abstract: Model evaluation is one of the most important aspects of structural equation modeling (SEM). Many model fit indices have been developed. It is not an exaggeration to say that nearly every publication using the SEM methodology has reported at least one fit index. Most fit indices are defined through test statistics. Studies and interpretation of fit indices commonly assume that the test statistics follow either a central chi-square distribution or a noncentral chi-square distribution. Because few statistics in practice follow a chi-square distribution, we study properties of the commonly used fit indices when dropping the chi-square distribution assumptions. The study identifies two sensible statistics for evaluating fit indices involving degrees of freedom. We also propose linearly approximating the distribution of a fit index/statistic by a known distribution or the distribution of the same fit index/statistic under a set of different conditions. The conditions include the sample size, the distribution of the data as well as the base-statistic. Results indicate that, for commonly used fit indices evaluated at sensible statistics, both the slope and the intercept in the linear relationship change substantially when conditions change. A fit index that changes the least might be due to an artificial factor. Thus, the value of a fit index is not just a measure of model fit but also of other uncontrollable factors. A discussion with conclusions is given on how to properly use fit indices.

Statistical modeling of the individual: Rationale and application of multivariate stationary time series analysis

Abstract: Results obtained with interindividual techniques in a representative sample of a population are not necessarily generalizable to the individual members of this population. In this article the specific condition is presented that must be satisfied to generalize from the interindividual level to the intraindividual level. A way to investigate whether this condition is satisfied is by means of multivariate time series analysis. More generally, time series analysis can be used to investigate psychological processes situated within individuals. In this article we consider a well established class of multivariate stationary time series models that may be used to study the intraindividual covariance structure. We demonstrate the application of some of these models with an empirical example consisting of state measurements of behavior associated with the Five Factor Model of Personality. We illustrate how one can investigate whether individuals are similar with respect to their intraindividual structure of variat...

Exploratory Factor Analysis, Theory Generation, and Scientific Method.

Abstract: This Chapter examines the methodological foundations of exploratory factor analysis (EFA) and suggests that it is properly construed as a method for generating explanatory theories. In the first half of the chapter, it is argued that EFA should be understood as an abductive method of theory generation that exploits an important precept of scientific inference known as the principle of the common cause. This characterization of the inferential nature of EFA coheres well with its interpretation as a latent variable method. The second half of the chapter outlines a broad theory of scientific method in which abductive reasoning figures prominently. It then discusses a number of methodological features of EFA in the light of that method. It is concluded that EFA, as a useful method of theory generation that can be profitably employed in tandem with confirmatory factor analysis and other methods of theory evaluation.

A Comparison of Methods to Test for Mediation in Multisite Experiments.

Abstract: A Monte Carlo study extended the research of MacKinnon, Lockwood, Hoffman, West, and Sheets (2002) for single-level designs by examining the statistical performance of four methods to test for mediation in a multilevel experimental design. The design studied was a two-group experiment that was replicated across several sites, included a single intervening variable and outcome, and assumed that the effects of the treatment and mediator were constant across sites. The findings provide new evidence of the benefits of and further support for using the asymmetric confidence limits approach to test for mediation. In addition, the authors provide further support for using confidence intervals to assess if treatment effects are completely mediated, as using traditional hypothesis testing may lead to erroneous conclusions.

Further Empirical Results on Parametric Versus Non-Parametric IRT Modeling of Likert-Type Personality Data

Abstract: Chernyshenko, Stark, Chan, Drasgow, and Williams (2001) investigated the fit of Samejima's logistic graded model and Levine's non-parametric MFS model to the scales of two personality questionnaires and found that the graded model did not fit well. We attribute the poor fit of the graded model to small amounts of multidimensionality present in their data. To verify this conjecture, we compare the fit of these models to the Social Problem Solving Inventory-Revised, whose scales were designed to be unidimensional. A calibration and a cross-validation sample of new observations were used. We also included the following parametric models in the comparison: Bock's nominal model, Masters' partial credit model, and Thissen and Steinberg's extension of the latter. All models were estimated using full information maximum likelihood. We also included in the comparison a normal ogive model version of Samejima's model estimated using limited information estimation. We found that for all scales Samejima's model outper...

The Consequence of Ignoring a Level of Nesting in Multilevel Analysis: A Comment

Abstract: This short contribution is a comment on M. Moerbeek's exploration of consequences of ignoring a level of clustering in a multilevel model, which was published in the first issue of the 2004 volume of Multivariate Behavioral Research. After having recapitulated the framework and extended the results of Moerbeek's study, we formulate two critical notes. First, we point at the incompleteness of the conclusions drawn by Moerbeek from the analytical work. The second note is concerned with the limitations of the framework itself.

Mixed-Effects Logistic Regression Models for Indirectly Observed Discrete Outcome Variables

Abstract: A well-established approach to modeling clustered data introduces random effects in the model of interest. Mixed-effects logistic regression models can be used to predict discrete outcome variables when observations are correlated. An extension of the mixed-effects logistic regression model is presented in which the dependent variable is a latent class variable. This method makes it possible to deal simultaneously with the problems of correlated observations and measurement error in the dependent variable. As is shown, maximum likelihood estimation is feasible by means of an EM algorithm with an E step that makes use of the special structure of the likelihood function. The new model is illustrated with an example from organizational psychology.

Discrete latent Markov models for normally distributed response data

Abstract: Van de Pol and Langeheine (1990) presented a general framework for Markov modeling of repeatedly measured discrete data. We discuss analogical single indicator models for normally distributed responses. In contrast to discrete models, which have been studied extensively, analogical continuous response models have hardly been considered. These models are formulated as highly constrained multinormal finite mixture models (McLachlan & Peel, 2000). The assumption of conditional independence, which is often postulated in the discrete models, may be relaxed in the normal-based models. In these models, the observed correlation between two variables may thus be due to the presence of two or more latent classes and the presence of within-class dependence. The latter may be subjected to structural equation modeling. In addition to presenting various normal-based Markov models, we demonstrate how these models, formulated as multinormal finite mixtures, may be fitted using the freely available program Mx (Neale, Boker, Xie, & Maes, 2002). To illustrate the application of some of the models, we report the analysis of data relating to the understanding of the conservation of continuous quantity (i.e., a Piagetian construct).

Comparison of Two Procedures for Analyzing Small Sets of Repeated Measures Data.

Abstract: This article compares two methods for analyzing small sets of repeated measures data under normal and non-normal heteroscedastic conditions: a mixed model approach with the Kenward-Roger correction and a multivariate extension of the modified Brown-Forsythe (BF) test. These procedures differ in their assumptions about the covariance structure of the data and in the method of estimation of the parameters defining the mean structure. Simulation results show that the BF test outperformed its competitor, in terms of Type I errors, particularly when the total sample size was small, and the data were normally distributed. Under non-normal distributions the BF test tended to err on the side of conservatism. Results also indicate that neither method was uniformly more powerful. With very few exceptions, the power differences between these two methods depended on the population covariance structure, the nature of the pairing of covariance matrices and group sizes, and the relationship between mean vectors and dispersion matrices.

Factor Analytic Procedures for Assessing Social Desirability in Binary Items.

Abstract: This article proposes and describes factor-analytic procedures for assessing and controlling socially desirable responding in binary personality items. The basic procedures are applications of the restricted (confirmatory) item factor analysis model for ordered-categorical variables. Orthogonal and oblique solutions based on marker variables are discussed. Next, the basic procedures are extended so that validity relations with external non-test variables can be assessed. Two empirical applications are given, and the substantive implications of the results are discussed.

Relating Factor Models for Longitudinal Data to Quasi-Simplex and NARMA Models.

Abstract: In this article we show the one-factor model can be rewritten as a quasi-simplex model. Using this result along with addition theorems from time series analysis, we describe a common general model, the nonstationary autoregressive moving average (NARMA) model, that includes as a special case, any latent variable model with continuous indicators and continuous latent variables. As an example, we show the NARMA representations of the linear growth curve model and the growth curve model with estimated basis vector coefficients. In certain instances rewriting competing models may help the investigator to compare different models. Here we compare the "hybrid" behavior genetics model of Eaves and Hewitt to the quasi-simplex model of Boomsma and Molenaar and show that both have equivalent NARMA representations which differ only in order.

Latent-Trait Latent-Class Analysis of Self-disclosure in the Work Environment

Abstract: Based on the literature about self-disclosure, it was hypothesized that different groups of subjects differ in their pattern of self-disclosure with respect to different areas of social interaction. An extended latent-trait latent-class model was proposed to describe these general patterns of self-disclosure. The model was used to analyze the data of 1,113 subjects, tested on extraversion and with respect to their degree of self-disclosure toward different categories of people in the work environment. A model with one latent trait and a latent class variable with three categories was identified. Subjects belonging to the different latent classes differ in their general tendency to self-disclose, in their choice to whom they will show self-disclosure and in the degree to which they are selective in their self-disclosure. The collateral variable extraversion was associated with both latent variables. The association of extraversion with selectivity in self-disclosure was not significant.

Maximum Likelihood Analysis of Nonlinear Structural Equation Models With Dichotomous Variables

Abstract: In this article, a maximum likelihood approach is developed to analyze structural equation models with dichotomous variables that are common in behavioral, psychological and social research. To assess nonlinear causal effects among the latent variables, the structural equation in the model is defined by a nonlinear function. The basic idea of the development is to augment the observed dichotomous data with the hypothetical missing data that involve the latent underlying continuous measurements and the latent variables in the model. An EM algorithm is implemented. The conditional expectation in the E-step is approximated via observations simulated from the appropriate conditional distributions by a Metropolis-Hastings algorithm within the Gibbs sampler, whilst the M-step is completed by conditional maximization. Convergence is monitored by bridge sampling. Standard errors are also obtained. Results from a simulation study and a real example are presented to illustrate the methodology.

Bias and Precision of Measures of Association for a Fixed-Effect Multivariate Analysis of Variance Model

Abstract: The sampling distributions of five popular measures of association with and without two bias adjusting methods were examined for the single factor fixed-effects multivariate analysis of variance model. The number of groups, sample sizes, number of outcomes, and the strength of association were manipulated. The results indicate that all five unadjusted measures of association can be extremely biased when sample sizes are small and the bias increases as the number of groups and outcome variables increase. The Tatsuoka (1973) adjustment procedure minimized the bias in a limited number of contexts, while the Serlin (1982) procedure provided an adequate adjustment for most contexts studied. The precision of both the adjusted and unadjusted effect-size measures was similar, with the Serlin approach having slightly better precision than the Tatsuoka procedure.

The Tabu Search Procedure: An Alternative to the Variable Selection Methods.

Abstract: The effectiveness of the Tabu variable selection algorithm, to identify predictor variables related to a criterion variable, is compared with the stepwise variable selection method and the all possible regression approach. Considering results obtained from previous research, Tabu is more successful in identifying relevant variables than the stepwise method, and the adjusted R⊃2 and the Mallow Cp criteria from all possible regression models for certain conditions. Although not completely effective in identifying all variables related to the criterion variable, Tabu is less likely to select variables unrelated to the criterion variable than the alternative methods. We encourage researchers to consider theory, previous research, and professional judgment when selecting the final set of predictors. Limitations of the study as well as the need for further research are also discussed.

Testing Linear Models for Ability Parameters in Item Response Models.

Abstract: Methods for testing hypotheses concerning the regression parameters in linear models for the latent person parameters in item response models are presented. Three tests are outlined: A likelihood ratio test, a Lagrange multiplier test and a Wald test. The tests are derived in a marginal maximum likelihood framework. They are explicitly formulated for the 3-parameter logistic model, but it is shown that the approach applies to a broad class of item response models. Since the distributions of the test statistics are derived asymptotically, simulation studies were performed to assess the Type I error rates of the tests for small realistic sample sizes. Overall, the Type I error rates for the null hypothesis that a regression coefficient equals zero, were close to the nominal significance level. A number of power studies were conducted. It is argued that on theoretical grounds the power of the Lagrange multiplier test might be less than the power of the other two tests, but this expectationwas not corroborated. The robustness of the tests to violation of the item response model was investigated with simulation studies of the power and Type I error rate. The results showed that the performance of the tests was acceptable in the cases where local independence and the constancy of the discrimination parameters over treatment groups were violated to the same extent for all treatment groups. The simulation studies also showed that the tests were biased if local independence was violated for one of the treatment groups.

A Latent-Change Scaling Model for Longitudinal Multiple Choice Data.

Abstract: A latent-change scaling model for the analysis of repeated-measures multiple-choice data is presented. The model extends previous work by combining latent class analysis and low dimensional scaling techniques in a longitudinal framework where subjects may change their preferences for the response categories over time. The latent structural component of the model characterizes both the cross-sectional heterogeneity of the population and an underlying change process over time; the measurement component of the model uses a scaling procedure to produce a joint representation of latent classes and response categories in a low dimensional space that represents individual differences in the utilities of the categories. An analysis of a national panel data set is used to illustrate both aspects of the model. A hypothetical example illustrates additional features of the model that can be tested when multiple indicators are collected at each time point.