In this article, we provide guidance for substantive researchers on the use of structural equation modeling in practice for theory testing and development. We present a comprehensive, two-step modeling approach that employs a series of nested models and sequential chi-square difference tests. We discuss the comparative advantages of this approach over a one-step approach. Considerations in specification, assessment of fit, and respecification of measurement models using confirmatory factor analysis are reviewed. As background to the two-step approach, the distinction between exploratory and confirmatory analysis, the distinction between complementary approaches for theory testing versus predictive application, and some developments in estimation methods also are discussed.

/pdf/structural-equation-modeling-in-practice-a-review-and-1ue2omj492.pdf

Structural equation modeling in practice: a review and recommended two-step approach

Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users.

Since the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site (http://www.prisma-statement.org/) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

/pdf/the-prisma-statement-for-reporting-systematic-reviews-and-3j53e5oe33.pdf

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration

Part One: The Context Of Educational Research Part Two: Planning Educational Research Part Three: Styles Of Educational Research Part Four: Strategies And Instruments For Data Collection And Researching Part Five: Data Analysis

Research Methods in Education

During the past 30 years, meta-analysis has been an indispensable tool for revealing the hidden meaning of our research literatures. The four articles in this special section on meta-analysis illus...

Meta-Analysis: A Constantly Evolving Research Integration Tool

Systematic reviews and meta-analyses are essential to summarise evidence relating to efficacy and safety of healthcare interventions accurately and reliably. The clarity and transparency of these reports, however, are not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users. Since the development of the QUOROM (quality of reporting of meta-analysis) statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realising these issues, an international group that included experienced authors and methodologists developed PRISMA (preferred reporting items for systematic reviews and meta-analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions. The PRISMA statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this explanation and elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA statement, this document, and the associated website (www.prisma-statement.org/) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

/pdf/the-prisma-statement-for-reporting-systematic-reviews-and-48ibbzju0k.pdf

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration

The study of potential racial and gender bias in individual test items is a major research area today. The fact that research has established that total scores on ability and achievement tests are predictively unbiased raises the question of whether there is in fact any real bias at the item level. No theoretical rationale for expecting such bias has been advanced. It appears that findings of item bias (differential item functioning; DIP) can be explained by three factors: failure to control for measurement error in ability estimates, violations of the unidimensionality assumption required by DIP detection methods, and reliance on significance testing (causing tiny artifactual DIP effects to be statistically significant because sample sizes are very large). After taking into account these artifacts, there appears to be no evidence that items on currently used tests function differently in different racial and gender groups. For the past 30 years, civil rights lawyers, journalists, and others have alleged that cognitive ability and educational achievement tests are predictively biased against minorities. That is, they have argued that when test scores are equal, minorities have higher average levels of educational and work performance, meaning that test scores underestimate the real world performance of minorities. Thousands of test bias studies have been conducted, and these studies have disconfirmed that hypothesis. The National Academy of Sciences appointed two blue ribbon committees to study the data from these studies, and both committees concluded that professionally developed tests are not predictively biased (Hartigan & Wigdor, 1989; Wigdor & Garner, 1982). Thus, the issue of test bias is scientifically dead. But for the past 15 years, there have been repeated claims that while tests may not be biased, many individual test items are biased. These claims have spawned major efforts to identify and remove biased test items. These claims are logically inconsistent. Consider the issue of racial bias. If a large portion of items on a test were biased against Blacks, then the test as a whole would be biased against Blacks. Because the test as a whole is known to be unbiased, there must be something wrong with the claim that there are a large number of biased items on the test. The current proposed solution to this logical problem is this: Current claims do not postulate that items are biased only against minority groups. Rather, they say that there are items biased against Whites as well as items biased against

https://psycnet.apa.org/journals/law/6/1/151.pdf

Racial and gender bias in ability and achievement tests: Resolving the apparent paradox.

History, Development, Evolution, and Impact of Validity Generalization and Meta-Analysis Methods, 1975–2001

This article compares the usefulness of standard score results (such as correlations and standardized regression coefficients) to that of raw scores results (such as covariances and raw score regression coefficients). The 2 main advantages of presenting standardized results is then examined. First, reporting standardized coefficients allows recognition of causal relationships that replicate across studies, facilitating meta-analysis. Second, an analysis with standardized coefficients allows the researcher to determine which of the predictor variables has the largest relative impact on the criterion variable. The implications of these 2 standardized score advantages are demonstrated with examples.

The Advantages of Using Standardized Scores in Causal Analysis.

This study used Monte Carlo simulation to examine the increase in accuracy resulting from 2 statistical refinements of the interactive Schmidt-Hunter procedures for meta-analysis: the use of the mean correlation instead of individual correlations in the estimation of sampling error variance, and a procedure that takes into account the nonlinear nature of the range-restriction correction. In all of the cases examined, these refinements increased the accuracy of the interactive procedure in estimating the variance of population correlations and resulted in more accuracy than other procedures examined. The use of the mean correlation in the sampling error variance formula also increased the accuracy of variance estimates for the multiplicative and Taylor Series procedures. Meta-analysis has become a widely used research technique in behavioral science. Among the different available meta-analytic approaches, the artifact-distribution-based meta-analytic methods for correlation coefficients are used most often, with numerous applications of this technique in different areas of behavioral science (Hunter & Schmidt, 1990; Schmidt, 1992). Computer simulation studies have examined the accuracy of procedures of this type (Callender & Osburn, 1980; Kemery, Mossholder, & Roth, 1987; Mendosa & Reinhardt, 1991; Raju & Burke, 1983). Most applications of meta-analysis to large real-world databases have yielded positive nonzero estimates of the variance of population correlations (e.g., Pearlman, Schmidt, & Hunter, 1980), although the variance estimates have generally been very small. Simulations (e.g., see Raju & Burke, 1983) have shown that although all of the mean population correlation estimates were very accurate, all of the procedures except the Schmidt-Hunter noninteractive procedure overestimated the true variance of population correlations. As a result, there have been continuous efforts to improve the accuracy of the existing meta-analytic procedures and to explain why they overestimate the true variance of population correlations. This study focused on two major modifications of the Schmidt-Hunter noninteractive (Pearlman et al., 1980; Schmidt, Hunter, Pearlman, & Shane, 1979) and interactive (Schmidt, Gast-Rosenberg, & Hunter, 1980) procedures for meta-analysis and used computer simulations to test the estimates produced by these improved procedures. We hypothesized that with these improvements overestimation of true variance of population correlations across studies would be reduced and that more accurate estimates of the population parameters

https://www.biz.uiowa.edu/faculty/fschmidt/meta-analysis/Law_et_al_1994_b.pdf

A Test of Two Refinements in Procedures for Meta-Analysis

The authors evaluated an improved procedure for range-restriction corrections in meta-analysis . When population correlations were approximately normally distributed, the new nonlinear rangecorrection procedure improved the accuracy of the Schmidt-Hunter (S-H) interactive method in estimating both the mean (Me) and standard deviation (SDf) of population correlations, making it the most accurate of the procedures examined. In the homogeneous case (SDP = 0), the nonlinear range correction again improves accuracy of estimates ofSDf. In this important case, Taylor Series Approximation (TSA) 1 is considerably less accurate than the S-H interactive and the TSA2 procedures, and N. S. Raju, M. J. Burke, J. Normand, and G. M. Langlois's (1991) procedure yields the least accurate estimates of SDP. Finally, the authors found that the nonlinear range-correction procedure produces improvements in the accuracy of the interactive method even under extreme violations of the normality assumption.

https://www.biz.uiowa.edu/faculty/fschmidt/meta-analysis/Law_et_al_1994.pdf

John E. Hunter

Papers

Racial and gender bias in ability and achievement tests: Resolving the apparent paradox.

History, Development, Evolution, and Impact of Validity Generalization and Meta-Analysis Methods, 1975–2001

The Advantages of Using Standardized Scores in Causal Analysis.

A Test of Two Refinements in Procedures for Meta-Analysis

Nonlinearity of Range Corrections in Meta-Analysis: Test of an Improved Procedure