Multitrait-Multimethod Analysis: Distinguishing between Items and Traits

TLDR: In this article, the authors compare MTMM analyses performed on items and factor scores derived from the items, and show that the difference between discriminant validity and method/halo effect is ambiguous when the relationship between the observed measures and underlying traits is unclear.

Abstract: Multitrait-multimethod analysis is designed to assess convergent and discriminant validity, and to also detect method/halo effects. However, the distinction between discriminant validity and method/halo effect is ambiguous when the relationship between the observed measures and underlying traits is unclear. The purpose of this study is to compare MTMM analyses performed on items and factor scores derived from the items. MTMM analyses of the individual items indicated modest support for convergent and divergent validity, and a substantial method/halo effect. The same analyses performed on factor scores showed substantial increases in convergent and divergent validity, and a substantial decrease in the method/halo effect. Based upon this demonstration, researchers are encouraged to conduct a preliminary factor analysis before analyzing MTMM data whenever there is doubt about the underlying trait structure.