National Surveys Via Rdd Telephone Interviewing Versus the Internet Comparing Sample Representativeness and Response Quality

TLDR: In this paper, the same questionnaires were administered simultaneously by RDD telephone interviewing, by the In- ternet with a probability sample, and by the Internet with a nonprobability sample of people who volunteered to do surveys for money.

Abstract: In a national field experiment, the same questionnaires were administered simultaneously by RDD telephone interviewing, by the In- ternet with a probability sample, and by the Internet with a nonprobability sample of people who volunteered to do surveys for money. The probabil- ity samples were more representative of the nation than the nonprobability sample in terms of demographics and electoral participation, even after weighting. The nonprobability sample was biased toward being highly engaged in and knowledgeable about the survey's topic (politics). The telephone data manifested more random measurement error, more survey satisficing, and more social desirability response bias than did the Inter- net data, and the probability Internet sample manifested more random error and satisficing than did the volunteer Internet sample. Practice at completing surveys increased reporting accuracy among the probability Internet sample, and deciding only to do surveys on topics of personal interest enhanced reporting accuracy in the nonprobability Internet sam- ple. Thus, the nonprobability Internet method yielded the most accurate self-reports from the most biased sample, while the probability Internet sample manifested the optimal combination of sample composition ac- curacy and self-report accuracy. These results suggest that Internet data collection from a probability sample yields more accurate results than do