Normative scoring of multidimensional pairwise preference personality scales using IRT: Empirical comparisons with other formats

TLDR: In this article, an item response theory-based approach is presented to construct and model multidimensional pairwise preference responses directly, assessing information at the item and scale levels, and a way of computing standard errors for trait scores and estimating scale reliability.

Abstract: In this article, we offer some suggestions as to why tetrads and pentads have become the dominant formats for administering multidimensional forced choice (MFC) items but, in turn, raise questions regarding the underlying psychometric model and means of addressing item quality and scoring accuracy. We then focus our attention on multidimensional pairwise preference (MDPP) items and present an item response theory–based approach to constructing and modeling MDPP responses directly, assessing information at the item and scale levels, and a way of computing standard errors for trait scores and estimating scale reliability. To demonstrate the viability of this method for applied use, we show that the correspondence between MDPP scores derived from direct modeling with those obtained using single statement and unidimensional pairwise preference measures administered in a laboratory setting. Trait score correlations and criterion related validities are compared across testing formats and rating sources (i.e., s...