Lies, Damned Lies, and Survey Self-Reports? Identity as a Cause of Measurement Bias.

TLDR: This work offers an alternative explanation rooted in identity theory that focuses on measurement directiveness as a cause of bias in survey self-reports, finding that direct measures generated bias—overreporting in survey measures and reactivity in the directive text condition—but the nondirective text condition generated unbiased measures.

Abstract: Explanations of error in survey self-reports have focused on social desirability: that respondents answer questions about normative behavior to appear prosocial to interviewers. However, this paradigm fails to explain why bias occurs even in self-administered modes like mail and web surveys. We offer an alternative explanation rooted in identity theory that focuses on measurement directiveness as a cause of bias. After completing questions about physical exercise on a web survey, respondents completed a text message-based reporting procedure, sending updates on their major activities for five days. Random assignment was then made to one of two conditions: instructions mentioned the focus of the study, physical exercise, or not. Survey responses, text updates, and records from recreation facilities were compared. Direct measures generated bias-overreporting in survey measures and reactivity in the directive text condition-but the nondirective text condition generated unbiased measures. Findings are discussed in terms of identity.