The use of interrater reliability (IRR) and interrater agreement (IRA) indices has increased dramatically during the past 20 years. This popularity is, at least in part, because of the increased role of multilevel modeling techniques (e.g., hierarchical linear modeling and multilevel structural equation modeling) in organizational research. IRR and IRA indices are often used to justify aggregating lower-level data used in composition models. The purpose of the current article is to expose researchers to the various issues surrounding the use of IRR and IRA indices often used in conjunction with multilevel models. To achieve this goal, the authors adopt a question-and-answer format and provide a tutorial in the appendices illustrating how these indices may be computed using the SPSS software.

Answers to 20 Questions About Interrater Reliability and Interrater Agreement

The role of personality traits in the decision to start a business and to maintain it successfully is discussed controversially in entrepreneurship research. Our meta-analysis builds upon and extends earlier meta-analyses by doing a full analysis of personality traits that includes a comparison of different traits from a theoretical perspective and by analysing a full set of personality predictors for both start-up activities as well as success. Theoretically, our article adds to the literature by matching traits to the tasks of entrepreneurs. The results indicate that traits matched to the task of running a business produced higher effect sizes with business creation than traits that were not matched to the task of running an enterprise, corrected r = .247, K = 47, N = 13,280, and corrected r = .124, K = 20, N = 3975, respectively. Moreover, traits matched to the task produced higher correlations with success, corrected r = .250, K = 42, N = 5607, than traits not matched to the task of running a business...

/pdf/let-s-put-the-person-back-into-entrepreneurship-research-a-2flx5zqiro.pdf

Let's put the person back into entrepreneurship research: A meta-analysis on the relationship between business owners' personality traits, business creation, and success

https://www.iranakhlagh.nipc.ir/uploads/science.L3pdf_6238.pdf

How low does ethical leadership flow? Test of a trickle-down model

Many researchers who use same-source data face concerns about common method variance (CMV). Although post hoc statistical detection and correction techniques for CMV have been proposed, there is a lack of empirical evidence regarding their efficacy. Because of disagreement among scholars regarding the likelihood and nature of CMV in self-report data, the current study evaluates three post hoc strategies and the strategy of doing nothing within three sets of assumptions about CMV: that CMV does not exist, that CMV exists and has equal effects across constructs, and that CMV exists and has unequal effects across constructs. The implications of using each strategy within each of the three assumptions are examined empirically using 691,200 simulated data sets varying factors such as the amount of true variance and the amount and nature of CMV modeled. Based on analyses of these data, potential benefits and likely risks of using the different techniques are detailed.

/pdf/a-tale-of-three-perspectives-examining-post-hoc-statistical-3teb42jgtf.pdf

A Tale of Three Perspectives: Examining Post Hoc Statistical Techniques for Detection and Correction of Common Method Variance

Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.

/pdf/the-viability-of-crowdsourcing-for-survey-research-20pd3afox8.pdf

The viability of crowdsourcing for survey research.

Everyone can recite methodological “urban legends” that were taught in graduate school, learned over the years through experience publishing, or perhaps just heard through the grapevine. In this article, the authors trace four widely cited and reported cutoff criteria to their (alleged) original sources to determine whether they really said what they are cited as having said about the cutoff criteria, and if not, what the original sources really said. The authors uncover partial truths in tracing the history of each cutoff criterion and in the end endorse a set of 12 specific guidelines for effective academic referencing provided by Harzing that, if adopted, should help prevent the further perpetuation of methodological urban legends.

The Sources of Four Commonly Reported Cutoff Criteria What Did They Really Say

Recently, the use of technology in assessment for personnel selection has increased dramatically. An important consideration is whether test scores obtained via Internet administration are psychometrically equivalent to those obtained by the more traditional paper-and-pencil format. Our results suggest that there is comparability of scores for many personality constructs, including conscientiousness. However, invariance was not found for some scales between persons allowed to choose formats and those not given a choice of formats. As testing-format preference may be related to membership in federally protected demographic groups, this latter finding was somewhat troubling. Additionally, we illustrate the use of an experimental laboratory design to investigate possible causes of a lack of measurement invariance in Internet and paper-and-pencil comparisons.

Lawrence C. Michels

Papers

The Sources of Four Commonly Reported Cutoff Criteria What Did They Really Say

Are Internet and Paper-and-Pencil Personality Tests Truly Comparable?: An Experimental Design Measurement Invariance Study