BACKGROUND: Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research. OBJECTIVES: To identify effective strategies to increase response to postal and electronic questionnaires. SEARCH STRATEGY: We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria. SELECTION CRITERIA: Randomised controlled trials of methods to increase response to postal or electronic questionnaires. DATA COLLECTION AND ANALYSIS: We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic. MAIN RESULTS: PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I(2) = 0%). AUTHORS' CONCLUSIONS: Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.

/pdf/methods-to-increase-response-to-postal-and-electronic-4sjz3r84m4.pdf

Methods to increase response to postal and electronic questionnaires

This article reports the results of a meta-analysis of experimental studies which have examined ten different factors felt to influence response rates to mail surveys. The form of meta- analysis used clearly defines the individual impact of each of the factors examined. Results indicate that prenotification and follow-ups increase the response rate, as does the type of outgo- ing postage used. Furthermore, studies sponsored by a university receive greater returns. Increases in the size of the monetary incentive used appear to have decreasing marginal gains. Finally, some evidence exists to suggest that the color of the question- naire influences response rate. The extensive use of mail surveys coupled with the low response rates typically encountered has made the issue of mail survey response rate improvement an intriguing topic. High response rates have the obvi- ous benefits of increased sample size, reduced costs associated with follow-up contacts, and reduced concern over nonresponse bias. Many experiments have been conducted to measure the effectiveness of one or more of the factors expected to influence response rates. The purpose of this paper is to apply a form of meta-analysis to reported experimental findings to isolate and estimate the individual impact of several typical response rate enhancement techniques. The meta-analysis technique used avoids the problems associated with past similar efforts but is more restrictive in the factors which can be exam-

https://www.jstor.org/stable/pdfplus/2749256.pdf

Mail survey response rate a meta-analysis of selected techniques for inducing response

A meta-analysis of prior studies of techniques designed to induce mail survey response rates was conducted. Research encompassing 184 effects (study outcomes) in 115 studies (articles) for 17 predictors of response rate was examined. The average effect size across all manipulations was r = .065, indicating an average increase of about 6.5 percent in response rates for manipulations. Effect sizes for specific predictors and two potential moderators of effects were examined. Results indicated that repeated contacts in the form of preliminary notification and follow-ups, appeals, inclusion of a return envelope, postage, and monetary incentives, were effective in increasing survey response rates. Significant effect sizes for the predictors ranged from an increase in response of 2 percent to 31 percent. Implications of the results for the conduct of mail surveys and future research on mail survey response behavior are discussed. Researchers have amassed myriad techniques to stimulate mail survey response rates, reduce item omission, speed up response, and reduce response bias. Examples include the use of preliminary notification, follow-up, sponsorship, appeals, postage, personalization, incentives, anonymity, prior commitment, and techniques affecting questionnaire Correspondence regarding this article should be addressed to FRANCIS J. YAMMARINO, School of Management and Center for Leadership Studies, State University of New York at Binghamton, P.O. Box 6000, Binghamton, NY 13902-6000. STEVEN J. SKINNER is at the College of Business and Economics, University of Kentucky, Lexington, KY. TERRY L. CHILDERS is at the Carlson School of Management, University of Minnesota, Minneapolis, MN. This research was supported in part by a New Faculty Development Award to F. J. Yammarino from the NYS/UUP Professional Development and Quality of Working Life Committee. The views expressed are ours. We gratefully acknowledge the helpful comments provided by the anonymous reviewers on an earlier version of this article. Public Opinion Quarterly Volume 55:613-639 © 1991 by the American Association for Public Opinion Research All rights nsscrved. 0033-362X/91/5504-0004$02.50 614 Yammarino, Skinner, and Childers appearance. These methods represent manipulations or predictors which, at least in some instances, have served to facilitate survey participation, A comprehensive list and review of studies related to the effectiveness of these methods can be found in the work of Duncan (1979), Houston and Ford (1976), Kanuk and Berenson (1975), and Linsky (1975). Linsky (1975), for example, found strong evidence that preliminary notification, follow-ups, "high powered" postage, sponsorship, small monetary incentives, and nonmonetary incentives were very effective for, or had positive effects on, increasing response rates. Other manipulations, according to Linsky (1975), are equivocal or interact with additional factors to affect response rates. Likewise, Kanuk and Berenson (1975) determined that monetary incentives (especially small ones), follow-up contacts, and perhaps preliminary notification, were the only techniques that had strong empirical evidence for effectiveness in increasing response rates. Other techniques showed no consistent effect. In a recent qualitative review, Harvey (1987) found that follow-ups, preliminary notification, stamped reply envelopes, and monetary incentives were important factors for increasing response rates. Harvey (1987) also determined that a number of other techniques such as personalization, length, sponsorship, and appeals had a more ambiguous effect on response rates. Although useful, traditional qualitative reviews are limited in that they do not explicitly consider differences in studies due to sample size, sample composition, experimental executions, and effect sizes (Fox, Crask, and Kim 1988). Recently, quantitative reviews such as meta-analysis, designed to address these limitations, have emerged. Several of these meta-analyses provide important background for the present study and are summarized in table 1. In an early comprehensive review of the mail survey literature, Heberlein and Baumgartner (1978) examined nine factors affecting initial and final survey response rates across 98 studies. They were able to explain 51 percent of the variance in final response through the number of respondent contacts (preliminary and follow-up) and the saliency of the survey topic. Additionally, Heberlein and Baumgartner (1978) found that sponsorship, length of the questionnaire, type of population surveyed, incentives, and postage (metered or franked) were also significant determinants of response (see table 1). Goyder (1982) replicated the Heberlein and Baumgartner (1978) study by conducting a meta-analysis on 145 citations. His findings were largely consistent with the predictive model of Heberlein and Baumgartner, finding that type of population, number of contacts, sponsorship, and A Meta-analysis of Mail Surveys 615 monetary incentives were most important in predicting final survey response rate. In another quantitative review of 93 studies for all types of surveys (personal, telephone, and mail), Yu and Cooper (1983) examined 15 factors reported potentially to affect survey participation rates. They found, across method of contact, that response rates were improved by incentives (prepaid and promised), nonmonetary premiums and rewards, and by increasing the amount of monetary rewards. Other techniques that increased response were preliminary notification, foot-inthe-door approaches, personalization, and follow-up contacts (table 1). In a recent meta-analysis. Fox, Crask, and Kim (1988) examined 82 studies with respect to eight factors that have been associated across studies with higher response rates to mail surveys. In their analysis, prenotification, follow-ups, and stamped return postage were found to be significant determinants of mail survey response rates (table 1). Additionally, sponsorship and small monetary incentives enhanced response. Two other more focused quantitative reviews examined the effects of personalization (Worthen and Valcarce 1985) and several types of return postage (Armstrong and Lusk 1987). Worthen and Valcarce (1985) reviewed 26 studies and concluded that the effect size for personalization from previous research was small, and in a follow-up experiment found no effects on response rate due to cover letter personalization. Armstrong and Lusk (1987) conducted their meta-analysis on 34 studies and concluded that the only effect of consequence on response rate was first-class postage as compared to business reply, and even that effect was weak (table 1). In synthesizing across the prior meta-analyses, a subset of the mail survey response determinants appears to be consistently predictive of response rates. Drawing from the above studies, repeated contacts, whether they be in the form of preliminary notification of a survey or a survey follow-up, appear to have a considerable positive impact on response rates (Fox, Crask, and Kim 1988; Goyder 1982; Heberlein and Baumgartner 1978; Yu and Cooper 1983). Another factor that also has been reported to have a consistent positive effect is the use of monetary incentives (Fox, Crask, and Kim 1988; Goyder 1982; Heberlein and Baumgartner 1978; Yu and Cooper 1983). In addition, the sponsor of a survey, although less under the control of the researcher, has been a key determinant of survey response (Fox, Crask, and Kim 1988; Goyder 1982; Heberlein and Baumgartner 1978). Finally, the inclusion of and type of outgoing and return postage has been observed consistently to affect response rates (Armstrong and Lusk 1987; Fox, Crask, and Kim 1988).

Understanding mail survey response behavior

Effects of source and paper color on response rates in mail surveys

This study examines the effects of the methodological characteristics of research source and anonymity of response upon mail survey response behavior. Unique features of the study were that the survey sponsor was identified, and the population sampled was one that should have a certain amount of commitment to both the sponsor and the topic because of membership in the sponsoring organization.

Paul L. Erdos

Papers

Visible vs. disguised keying on questionnaires.