Model modifications in covariance structure analysis: the problem of capitalization on chance.
01 May 1992-Psychological Bulletin (American Psychological Association)-Vol. 111, Iss: 3, pp 490-504
TL;DR: Results demonstrate that over repeated samples, model modifications may be very inconsistent and cross-validation results may behave erratically, leading to skepticism about generalizability of models resulting from data-driven modifications of an initial model.
Abstract: In applications of covariance structure modeling in which an initial model does not fit sample data well, it has become common practice to modify that model to improve its fit. Because this process is data driven, it is inherently susceptible to capitalization on chance characteristics of the data, thus raising the question of whether model modifications generalize to other samples or to the population. This issue is discussed in detail and is explored empirically through sampling studies using 2 large sets of data. Results demonstrate that over repeated samples, model modifications may be very inconsistent and cross-validation results may behave erratically. These findings lead to skepticism about generalizability of models resulting from data-driven modifications of an initial model. The use of alternative a priori models is recommended as a preferred strategy.
Citations
More filters
TL;DR: The factor structure of the combined BDI and BAI items was virtually identical to that reported by Beck for a sample of diagnosed depressed and anxious patients, supporting the view that these clinical states are more severe expressions of the same states that may be discerned in normals.
9,443 citations
TL;DR: In this article, a framework for hypothesis testing and power analysis in the assessment of fit of covariance structure models is presented, where the value of confidence intervals for fit indices is emphasized.
Abstract: A framework for hypothesis testing and power analysis in the assessment of fit of covariance structure models is presented. We emphasize the value of confidence intervals for fit indices, and we stress the relationship of confidence intervals to a framework for hypothesis testing. The approach allows for testing null hypotheses of not-good fit, reversing the role of the null hypothesis in conventional tests of model fit, so that a significant result provides strong support for good fit. The approach also allows for direct estimation of power, where effect size is defined in terms of a null and alternative value of the root-mean-square error of approximation fit index proposed by J. H. Steiger and J. M. Lind (1980). It is also feasible to determine minimum sample size required to achieve a given level of power for any test of fit in this framework. Computer programs and examples are provided for power analyses and calculation of minimum sample sizes.
8,401 citations
TL;DR: In this article, the authors examined the factorial structure of a new instrument to measure engagement, the hypothesized 'opposite' of burnout in a sample of university students (N=314) and employees (N = 619).
Abstract: This study examines the factorial structure of a new instrument to measure engagement, the hypothesized `opposite' of burnout in a sample of university students (N=314) and employees (N=619). In addition, the factorial structure of the Maslach-Burnout Inventory-General Survey (MBI-GS) is assessed and the relationship between engagement and burnout is examined. Simultaneous confirmatory factor analyses in both samples confirmed the original three-factor structure of the MBI-GS (exhaustion, cynicism, and professional efficacy) as well as the hypothesized three-factor structure of engagement (vigor, dedication, and absorption). Contrary to expectations, a model with two higher-order factors – ‘burnout’ and ‘engagement’ – did not show a superior fit to the data. Instead, our analyses revealed an alternative model with two latent factors including: (1) exhaustion and cynicism (‘core of burnout’); (2) all three engagement scales plus efficacy. Both latent factors are negatively related and share between 22% and 38% of their variances in both samples. Despite the fact that slightly different versions of the MBI-GS and the engagement questionnaire had to be used in both samples the results were remarkably similar across samples, which illustrates the robustness of our findings.
7,491 citations
Cites methods from "Model modifications in covariance s..."
...…three-factor burnout model can be slightly improved by allowing three pairs of errors to correlate.2 Despite the danger of chance capitalization (MacCallum et al., 1992) we decided to include these error terms since they were exactly similar to those that were included by Schutte et al. (2000)…...
[...]
TL;DR: In this paper, a model is tested in which burnout and engagement have different predictors and different possible consequences, showing that burnout is mainly predicted by job demands but also by lack of job resources, whereas engagement is exclusively predicted by available job resources.
Abstract: This study focuses on burnout and its positive antipode—engagement. A model is tested in which burnout and engagement have different predictors and different possible consequences. Structural equation modeling was used to simultaneously analyze data from four independent occupational samples (total N = 1698). Results confirm the hypothesized model indicating that: (1) burnout and engagement are negatively related, sharing between 10 per cent and 25 per cent of their variances; (2) burnout is mainly predicted by job demands but also by lack of job resources, whereas engagement is exclusively predicted by available job resources; (3) burnout is related to health problems as well as to turnover intention, whereas engagement is related only to the latter; (4) burnout mediates the relationship between job demands and health problems, whereas engagement mediates the relationship between job resources and turnover intention. The fact that burnout and engagement exhibit different patterns of possible causes and consequences implies that different intervention strategies should be used when burnout is to be reduced or engagement is to be enhanced. Copyright © 2004 John Wiley & Sons, Ltd.
7,068 citations
Cites background from "Model modifications in covariance s..."
...Despite the danger of chance capitalization ( MacCallum, Roznowski, & Necowitz, 1992 ) we decided to include these error terms in a re-specified model (M3) since both can be interpreted straightforwardly....
[...]
TL;DR: Research on experienced repeat online shoppers shows that consumer trust is as important to online commerce as the widely accepted TAM use-antecedents, perceived usefulness and perceived ease of use, and provides evidence that online trust is built through a belief that the vendor has nothing to gain by cheating.
Abstract: A separate and distinct interaction with both the actual e-vendor and with its IT Web site interface is at the heart of online shopping Previous research has established, accordingly, that online purchase intentions are the product of both consumer assessments of the IT itself-specifically its perceived usefulness and ease-of-use (TAM)-and trust in the e-vendor But these perspectives have been examined independently by IS researchers Integrating these two perspectives and examining the factors that build online trust in an environment that lacks the typical human interaction that often leads to trust in other circumstances advances our understanding of these constructs and their linkages to behavior
Our research on experienced repeat online shoppers shows that consumer trust is as important to online commerce as the widely accepted TAM use-antecedents, perceived usefulness and perceived ease of use Together these variable sets explain a considerable proportion of variance in intended behavior The study also provides evidence that online trust is built through (1) a belief that the vendor has nothing to gain by cheating, (2) a belief that there are safety mechanisms built into the Web site, and (3) by having a typical interface, (4) one that is, moreover, easy to use
6,853 citations
References
More filters
Book•
28 Apr 1989
TL;DR: The General Model, Part I: Latent Variable and Measurement Models Combined, Part II: Extensions, Part III: Extensions and Part IV: Confirmatory Factor Analysis as discussed by the authors.
Abstract: Model Notation, Covariances, and Path Analysis. Causality and Causal Models. Structural Equation Models with Observed Variables. The Consequences of Measurement Error. Measurement Models: The Relation Between Latent and Observed Variables. Confirmatory Factor Analysis. The General Model, Part I: Latent Variable and Measurement Models Combined. The General Model, Part II: Extensions. Appendices. Distribution Theory. References. Index.
19,019 citations
TL;DR: In this article, a general null model based on modified independence among variables is proposed to provide an additional reference point for the statistical and scientific evaluation of covariance structure models, and the importance of supplementing statistical evaluation with incremental fit indices associated with the comparison of hierarchical models.
Abstract: Factor analysis, path analysis, structural equation modeling, and related multivariate statistical methods are based on maximum likelihood or generalized least squares estimation developed for covariance structure models. Large-sample theory provides a chi-square goodness-of-fit test for comparing a model against a general alternative model based on correlated variables. This model comparison is insufficient for model evaluation: In large samples virtually any model tends to be rejected as inadequate, and in small samples various competing models, if evaluated, might be equally acceptable. A general null model based on modified independence among variables is proposed to provide an additional reference point for the statistical and scientific evaluation of covariance structure models. Use of the null model in the context of a procedure that sequentially evaluates the statistical necessity of various sets of parameters places statistical methods in covariance structure analysis into a more complete framework. The concepts of ideal models and pseudo chi-square tests are introduced, and their roles in hypothesis testing are developed. The importance of supplementing statistical evaluation with incremental fit indices associated with the comparison of hierarchical models is also emphasized. Normed and nonnormed fit indices are developed and illustrated.
16,420 citations
Book•
01 Jan 1989
9,143 citations
"Model modifications in covariance s..." refers methods in this paper
...In applications of CSM, researchers typically begin with one or more initial models and use a computer program such as LISREL (Joreskog & Sorbom, 1988) or EQS (Bentler, 1989) to fit those models to sample data and obtain estimates of model parameters....
[...]
TL;DR: An up-to-date handbook on conceptual and methodological issues relevant to the study of industrial and organizational behavior is presented in this paper, which covers substantive issues at both the individual and organizational level in both theoretical and practical terms.
Abstract: An up-to-date handbook on conceptual and methodological issues relevant to the study of industrial and organizational behavior. Chapters contributed by leading experts from the academic and business communities cover substantive issues at both the individual and organizational level, in both theoretical and practical terms.
7,809 citations
TL;DR: Dans les differentes procedures existantes pour l'evaluation and the modifications sequentielles des modeles structuraux, l'auteur s'attache a discuter celle connue sous le terme PMM.
Abstract: Dans les differentes procedures existantes pour l'evaluation et les modifications sequentielles des modeles structuraux, l'auteur s'attache a discuter celle connue sous le terme PMM. Plus generalement, les propositions de KAPLAN (1990) sont critiquees dans le detail
6,544 citations