In this article, we provide guidance for substantive researchers on the use of structural equation modeling in practice for theory testing and development. We present a comprehensive, two-step modeling approach that employs a series of nested models and sequential chi-square difference tests. We discuss the comparative advantages of this approach over a one-step approach. Considerations in specification, assessment of fit, and respecification of measurement models using confirmatory factor analysis are reviewed. As background to the two-step approach, the distinction between exploratory and confirmatory analysis, the distinction between complementary approaches for theory testing versus predictive application, and some developments in estimation methods also are discussed.

/pdf/structural-equation-modeling-in-practice-a-review-and-1ue2omj492.pdf

Structural equation modeling in practice: a review and recommended two-step approach

Content Memory (Learning Ability) As Comprehension 82 Vocabulary Cs .30 ( ) .23 .31 ( ) .31 .31 .35 ( ) .29 .48 .35 .38 ( ) .30 .40 .47 .58 .48 ( ) As judged against these latter values, comprehension (.48) and vocabulary (.47), but not memory (.31), show some specific validity. This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components. Some of the correlations in Chi's (1937) prodigious study of halo effect in ratings are appropriate to a multitrait-multimethod matrix in which each rater might be regarded as representing a different method. While the published report does not make these available in detail because it employs averaged values, it is apparent from a comparison of his Tables IV and VIII that the ratings generally failed to meet the requirement that ratings of the same trait by different raters should correlate higher than ratings of different traits by the same rater. Validity is shown to the extent that of the correlations in the heteromethod block, those in the validity diagonal are higher than the average heteromethod-heterotrait values. A conspicuously unsuccessful multitrait-multimethod matrix is provided by Campbell (1953, 1956) for rating of the leadership behavior of officers by themselves and by their subordinates. Only one of 11 variables (Recognition Behavior) met the requirement of providing a validity diagonal value higher than any of the heterotrait-heteromethod values, that validity being .29. For none of the variables were the validities higher than heterotrait-monomethod values. A study of attitudes toward authority and nonauthority figures by Burwen and Campbell (1957) contains a complex multitrait-multimethod matrix, one symmetrical excerpt from which is shown in Table 6. Method variance was strong for most of the procedures in this study. Where validity was found, it was primarily at the level of validity diagonal values higher than heterotrait-heteromethod values. As illustrated in Table 6, attitude toward father showed this kind of validity, as did attitude toward peers to a lesser degree. Attitude toward boss showed no validity. There was no evidence of a generalized attitude toward authority which would include father and boss, although such values as the VALIDATION BY THE MULTITRAIT-MULTIMETHOD MATRIX

Convergent and discriminant validation by the multitrait-multimethod matrix.

Validation of psychological tests has not yet been adequately conceptualized, as the APA Committee on Psychological Tests learned when it undertook (1950-54) to specify what qualities should be investigated before a test is published. In order to make coherent recommendations the Committee found it necessary to distinguish four types of validity, established by different types of research and requiring different interpretation. The chief innovation in the Committee's report was the term construct validity.[2] This idea was first formulated by a subcommittee (Meehl and R. C. Challman) studying how proposed recommendations would apply to projective techniques, and later modified and clarified by the entire Committee (Bordin, Challman, Conrad, Humphreys, Super, and the present writers). The statements agreed upon by the Committee (and by committees of two other associations) were published in the Technical Recommendations (59). The present interpretation of construct validity is not "official" and deals with some areas where the Committee would probably not be unanimous. The present writers are solely responsible for this attempt to explain the concept and elaborate its implications.

/pdf/construct-validity-in-psychological-tests-1tuszfx7to.pdf

Construct validity in psychological tests.

Mediation is said to occur when a causal effect of some variable X on an outcome Y is explained by some intervening variable M. The authors recommend that with small to moderate samples, bootstrap methods (B. Efron & R. Tibshirani, 1993) be used to assess mediation. Bootstrap tests are powerful because they detect that the sampling distribution of the mediated effect is skewed away from 0. They argue that R. M. Baron and D. A. Kenny's (1986) recommendation of first testing the X --> Y association for statistical significance should not be a requirement when there is a priori belief that the effect size is small or suppression is a possibility. Empirical examples and computer setups for bootstrap analyses are provided.

/pdf/mediation-in-experimental-and-nonexperimental-studies-new-36pi5vkg4y.pdf

Mediation in experimental and nonexperimental studies: New procedures and recommendations.

A Monte Carlo study compared 14 methods to test the statistical significance of the intervening variable effect. An intervening variable (mediator) transmits the effect of an independent variable to a dependent variable. The commonly used R. M. Baron and D. A. Kenny (1986) approach has low statistical power. Two methods based on the distribution of the product and 2 difference-in-coefficients methods have the most accurate Type I error rates and greatest statistical power except in 1 important case in which Type I error rates are too high. The best balance of Type I error and statistical power across all cases is the test of the joint significance of the two effects comprising the intervening variable effect.

/pdf/a-comparison-of-methods-to-test-mediation-and-other-21xho8v2xm.pdf

A comparison of methods to test mediation and other intervening variable effects.

Professionals are frequently consulted to diagnose and predict human behavior; optimal treatment and planning often hinge on the consultant's judgmental accuracy. The consultant may rely on one of two contrasting approaches to decision-making--the clinical and actuarial methods. Research comparing these two approaches shows the actuarial method to be superior. Factors underlying the greater accuracy of actuarial methods, sources of resistance to the scientific findings, and the benefits of increased reliance on actuarial approaches are discussed.

https://courses.washington.edu/pbafhall/514/514%20Readings/clinical%20versus%20actuarial.pdf

Clinical versus actuarial judgment

IN the course of the last decade, while spending several thousand hours in the practice of intensive psychotherapy, I have treated— sometimes unknowingly except in retrospect—a considerable number of schizoid and schizophrenic patients. Like all clinicians, I have formed some theoretical opinions as a result of these experiences. While I have not until recently begun any systematic research efforts on this baffling disorder, I felt that to share with you some of my thoughts, based though they are upon clinical impressions in the context of selected research by others, might be an acceptable use of this occasion. Let me begin by putting a question which I find is almost never answered correctly by our clinical students on PhD orals, and the answer to which they seem to dislike when it is offered. Suppose that you were required to write down a procedure for selecting an individual from the population who would be diagnosed as schizophrenic by a psychiatric staff; you have to wager $1,000 on being right; you may not include in your selection procedure any behavioral fact, such as a symptom or trait, manifested by the individual. What would you write down? So far as I have been able to ascertain, there is only one thing you could write down that would give you a better than even chance of winning such a bet—namely, \"Find an individual X who has a schizophrenic identical twin.\" Admittedly, there are many other facts which would raise your odds somewhat above the low base rate of schizophrenia. You might, for example, identify X by first finding mothers who have certain unhealthy child-rearing attitudes; you might enter a subpopulation defined jointly by such demographic variables as age, size of community, religion, ethnic background, or social class. But these would leave you with a pretty unfair wager, as would the rule, \"Find an X who has a fraternal twin, of the same sex, diagnosed as

Schizotaxia, Schizotypy, Schizophrenia

Theories in “soft” areas of psychology lack the cumulative character of scientific knowledge. They tend neither to be refuted nor corroborated, but instead merely fade away as people lose interest. Even though intrinsic subject matter difficulties (20 listed) contribute to this, the excessive reliance on significance testing is partly responsible, being a poor way of doing science. Karl Popper’s approach, with modifications, would be prophylactic. Since the null hypothesis is quasi-always false, tables summarizing research in terms of patterns of “significant differences” are little more than complex, causally uninterpretable outcomes of statistical power functions. Multiple paths to estimating numerical point values (“consistency tests”) are better, even if approximate with rough tolerances; and lacking this, ranges, orderings, second-order differences, curve peaks and valleys, and function forms should be used. Such methods are usual in developed sciences that seldom report statistical significance. Consistency tests of a conjectural taxometric model yielded 94% success with zero false negatives.

Paul E. Meehl

Papers

Construct validity in psychological tests.

Clinical versus actuarial judgment

Schizotaxia, Schizotypy, Schizophrenia

Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.

Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence