Despite the concern that has been expressed about potential method biases, and the pervasiveness of research settings with the potential to produce them, there is disagreement about whether they really are a problem for researchers in the behavioral sciences. Therefore, the purpose of this review is to explore the current state of knowledge about method biases. First, we explore the meaning of the terms “method” and “method bias” and then we examine whether method biases influence all measures equally. Next, we review the evidence of the effects that method biases have on individual measures and on the covariation between different constructs. Following this, we evaluate the procedural and statistical remedies that have been used to control method biases and provide recommendations for minimizing method bias.

Sources of Method Bias in Social Science Research and Recommendations on How to Control It

The job demands-resources (JD-R) model proposes that working conditions can be categorized into 2 broad categories, job demands and job resources. that are differentially related to specific outcomes. A series of LISREL analyses using self-reports as well as observer ratings of the working conditions provided strong evidence for the JD-R model: Job demands are primarily related to the exhaustion component of burnout, whereas (lack of) job resources are primarily related to disengagement. Highly similar patterns were observed in each of 3 occupational groups: human services, industry, and transport (total N = 374). In addition, results confirmed the 2-factor structure (exhaustion and disengagement) of a new burnout instrument--the Oldenburg Burnout Inventory--and suggested that this structure is essentially invariant across occupational groups.

The job demands-resources model of burnout

"With its emphasis on practical and conceptual aspects, rather than mathematics or formulas, this accessible book has established itself as the go-to resource on confirmatory factor analysis (CFA). Detailed, worked-through examples drawn from psychology, management, and sociology studies illustrate the procedures, pitfalls, and extensions of CFA methodology. The text shows how to formulate, program, and interpret CFA models using popular latent variable software packages (LISREL, Mplus, EQS, SAS/CALIS); understand the similarities and differences between CFA and exploratory factor analysis (EFA); and report results from a CFA study. It is filled with useful advice and tables that outline the procedures. The companion website offers data and program syntax files for most of the research examples, as well as links to CFA-related resources. New to This Edition *Updated throughout to incorporate important developments in latent variable modeling. *Chapter on Bayesian CFA and multilevel measurement models. *Addresses new topics (with examples): exploratory structural equation modeling, bifactor analysis, measurement invariance evaluation with categorical indicators, and a new method for scaling latent variables. *Utilizes the latest versions of major latent variable software packages"--

/pdf/confirmatory-factor-analysis-for-applied-research-4crbcxiv74.pdf

Confirmatory Factor Analysis for Applied Research

The establishment of measurement invariance across groups is a logical prerequisite to conducting substantive cross-group comparisons (e.g., tests of group mean differences, invariance of structura...

A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research

This meta-analysis reviewed the literature on socioeconomic status (SES) and academic achievement in journal articles published between 1990 and 2000. The sample included 101,157 students, 6,871 schools, and 128 school districts gathered from 74 independent samples. The results showed a medium to strong SES–achievement relation. This relation, however, is moderated by the unit, the source, the range of SES variable, and the type of SES–achievement measure. The relation is also contingent upon school level, minority status, and school location. The author conducted a replica of White’s (1982) meta-analysis to see whether the SES–achievement correlation had changed since White’s initial review was published. The results showed a slight decrease in the average correlation. Practical implications for future research and policy are discussed.

Socioeconomic Status and Academic Achievement: A Meta-Analytic Review of Research

Composition models specify the functional relationships among phenomena or constructs at different levels of analysis (e.g., individual level, team level, organizational level) that reference essentially the same content but that are qualitatively different at different levels (M. T. Hannan, 1971; K. H. Roberts, C. L. Hulin, & D. M. Rousseau, 1978; D. M. Rousseau, 1985). Specifying adequate composition models is a critical component of good multilevel research. A typology of composition models is proposed to provide a framework for organizing, evaluating, and developing constructs and theories in multilevel research. Five basic forms of composition are described and illustrated. Implications of the typology are discussed.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1223&context=soss_research

Functional Relations among Constructs in the Same Content Domain at Different Levels of Analysis: A Typology of Composition Models

The use of self-report data is widespread across diverse fields of 
empirical research such as organizational behavior, personality and 
individual differences, social psychology, and mental health. Despite 
the prevalent use of self-report data in empirical studies, there is a 
widespread belief among researchers that there are severe threats to 
its validity which serve to weaken the intended substantive inferences to be drawn from such data. Any researcher familiar with the 
journal review process could probably testify that one of the most 
common methodological criticisms of manuscripts under review 
tends to be associated with alleged problems concerning the use of 
self-report data. These criticisms are often evident in the comments 
by reviewers and editors. Even authors themselves tend to accept the 
alleged problems of self-report data, as indicated by the limitations 
they acknowledged in the Discussion section of their manuscripts.

So Why Ask Me? Are Self-Report Data Really That Bad?

The concept of change over time is fundamental to many phenomena investigated in organizational research. This didactically oriented article proposes an integrative approach incorporating longitudinal mean and covariance structures analysis and multiple indicator latent growth modeling to aid organizational researchers in directly addressing fundamental questions concerning the conceptualization and analysis of change over time. The approach is illustrated using a numerical example involving several organizationally relevant variables. Advantages, limitations, and extensions of the approach are discussed.

The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM)

On the basis of a distinction between test content and method of testing, the present study examined several conceptually and practically important effects relating race, reading comprehension, method of assessment, face validity perceptions, and performance on a situational judgement test using a sample of 241 psychology undergraduates (113 Blacks and 128 Whites). Results showed that the Black-White differences in situational judgment test performance and face validity reactions to the test were substantially smaller in the video-based method of testing than in the paper-and-pencil method. The Race x Method interaction effect on test performance was attributable to differences in reading comprehension and face validity reactions associated with race and method of testing. Implications of the findings were discussed in the context of research on adverse impact and examinee test reactions.

Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions.

Intraindividual change over time is the essence of the change phenomenon hypothesized to occur in the individual newcomer adaptation process. Many important adaptation questions cannot be answered without an adequate conceptualization and assessment of intraindividual change. Using a latent growth modeling approach to data collected from 146 doctoral program newcomers over 4 repeated measurements spaced at 1-month intervals, the authors explicitly modeled interindividual differences in intrain-dividual changes in newcomer proactivities (information seeking, relationship building) and proximal adaptation outcomes (task mastery, role clarity, social integration) during organizational entry. Results indicated that changes in proactivity may be related to newcomer characteristics and adaptation outcomes in interesting ways that have not been previously examined.

Interindividual differences in intraindividual changes in proactivity during organizational entry: a latent growth modeling approach to understanding newcomer adaptation.

David Chan

Papers

Functional Relations among Constructs in the Same Content Domain at Different Levels of Analysis: A Typology of Composition Models

So Why Ask Me? Are Self-Report Data Really That Bad?

The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM)

Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions.

Interindividual differences in intraindividual changes in proactivity during organizational entry: a latent growth modeling approach to understanding newcomer adaptation.