Factors affecting response rates of the web survey: A systematic review

This study meta-analyzes thirty-nine study results published within last ten years that directly compared Web and mail survey modes. Although considerable variation exists across the studies, the authors’ findings show that mail surveys have higher response rates than Web surveys in general. Two study features (i.e., population types and follow-up reminders) are shown to contribute statistically to the variation of response rate differences between Web and paper surveys in the comparative studies. College respondents appear to be more responsive to Web surveys, while some other respondents (e.g., medical doctors, school teachers, and general consumers) appear to prefer traditional mail surveys. Follow-up reminders appear to be less effective for Web survey respondents than for mail survey respondents. Other study features (i.e., implementation of random assignment of survey respondents, incentives, and publication year) are not statistically useful in accounting for the variation of response rate differences between Web and mail surveys.

Comparing response rates from Web and mail surveys: A meta-analysis.

Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.

/pdf/the-viability-of-crowdsourcing-for-survey-research-20pd3afox8.pdf

The viability of crowdsourcing for survey research.

One question that arises when discussing the usefulness of web-based surveys is whether they gain the same response rates compared to other modes of collecting survey data. A common perception exists that, in general, web survey response rates are considerably lower. However, such unsystematic anecdotal evidence could be misleading and does not provide any useful quantitative estimate. Metaanalytic procedures synthesising controlled experimental mode comparisons could give accurate answers but, to the best of the authors' knowledge, such research syntheses have so far not been conducted. To overcome this gap, the authors have conducted a meta-analysis of 45 published and unpublished experimental comparisons between web and other survey modes. On average, web surveys yield an 11% lower response rate compared to other modes (the 95% confidence interval is confined by 15% and 6% to the disadvantage of the web mode). This response rate difference to the disadvantage of the web mode is systematically influenced by the sample recruitment base (a smaller difference for panel members as compared to one-time respondents), the solicitation mode chosen for web surveys (a greater difference for postal mail solicitation compared to email) and the number of contacts (the more contacts, the larger the difference in response rates between modes). No significant influence on response rate differences can be revealed for the type of mode web surveys are compared to, the type of target population, the type of sponsorship, whether or not incentives were offered, and the year the studies were conducted. Practical implications are discussed.

Web Surveys versus other Survey Modes: A Meta-Analysis Comparing Response Rates:

To Dwell Among Friends: Personal Networks in Town and City.

Two quite different reasons for using open-ended as opposed to closeended questions can be distinguished. One is to discover the responses that individuals give spontaneously; the other is to avoid the bias that may resultfrom suggesting responses to individuals. However, open-ended questions also have disadvantages in comparison to close-ended, such as the need for extensive coding and larger item non-response. While this issue has already been well researched for traditional survey questionnaires, not much research has been devoted to it in recently used Web questionnaires. We therefore examine the differences between the open-ended and the closeended question form in Web questionnaires by means of experiments within the large-scale RIS 2001 Web survey. The question "What is the most important, critical problem the Internet is facing today?" was asked in an open-ended and two close-ended question forms in a split-ballot experiment. The results show that there were differences between question forms in univariate distributions, though no significant differences were found in the ranking of values. Close-ended questions in general yield higher percentages than open-ended question for answers that are identical in both question forms. It seems that respondents restricted themselves with apparent ease to the alternatives offered on the close-ended forms, whereas on the open-ended question they produced a much more diverse set of answers. In addition, our results suggest that open-ended questions produce more missing data than close-ended. Moreover, there were more inadequate answers for open-ended question. This suggests that open-endedquestions should be more explicit in their wording (at least for Websurveys, as a self administered mode of data collection) than close-ended questions, which are more specified with given response alternatives.

https://begrijpelijkeformulieren.org/sites/begrijpelijkeformulieren/files/Reja_e.a._Open-ended_vs._Close-ended_Questions_in_Web.pdf

Open-ended vs. close-ended questions in Web questionnaires

Chapter 1: Survey research and web surveys Definition and typology Web survey process Evolution of web surveys, applications and related practices Chapter 2: Pre-fielding Mode elaboration Sampling Questionnaire preparation Technical preparations Nonresponse strategy General management Chapter 3: Fielding Recruiting Measurement Processing and monitoring Chapter 4: Post-fielding Data preparation Preliminary results Data exporting and documentation Chapter 5: Selected topics in web survey implementation Smartphones, tablets and other devices Online panels Web survey software Chapter 6: Broader context of web surveys Broader methodological context Web surveys within the project management framework The web survey profession Web survey bibliography Chapter 7: Future of web surveys General technological developments Web survey software Methodology Broader business and societal issues Chapter 8: Conclusions

Web Survey Methodology

Despite increased use of Web surveys, relatively little is known about standards for designing Web questionnaires. Since there is no help from an interviewer for the respondent taking a Web survey, the design of self-administered Web questionnaires is even more important in order to achieve high data quality. Question wording, form and graphic layout of the questionnaire are particularly important. This paper presents some basic experiments to address these issues: one vs. multiple-page design, use of logotypes, and survey topic. The research was performed within the national RIS – Research on Internet in Slovenia - project (http://www.ris.org) in extensive testing since 1996.

Katja Lozar Manfreda

Papers

Web Surveys versus other Survey Modes: A Meta-Analysis Comparing Response Rates:

Open-ended vs. close-ended questions in Web questionnaires

Web Survey Methodology

Design of Web Survey Questionnaires: Three Basic Experiments

Overview: Online Surveys