Are Internet and Paper-and-Pencil Personality Tests Truly Comparable?: An Experimental Design Measurement Invariance Study

TLDR: In this article, the authors investigated whether test scores obtained via Internet administration are psychometrically equivalent to those obtained by the more traditional paper-and-pencil format and found that there is comparability of scores for many personality constructs, including conscientiousness.

Abstract: Recently, the use of technology in assessment for personnel selection has increased dramatically. An important consideration is whether test scores obtained via Internet administration are psychometrically equivalent to those obtained by the more traditional paper-and-pencil format. Our results suggest that there is comparability of scores for many personality constructs, including conscientiousness. However, invariance was not found for some scales between persons allowed to choose formats and those not given a choice of formats. As testing-format preference may be related to membership in federally protected demographic groups, this latter finding was somewhat troubling. Additionally, we illustrate the use of an experimental laboratory design to investigate possible causes of a lack of measurement invariance in Internet and paper-and-pencil comparisons.