Examination of the Equivalence of Self-Report Survey-Based Paper-and-Pencil and Internet Data Collection Methods.
TL;DR: Overall, the findings show that paper-and-pencil and Internet data collection methods are generally equivalent, particularly for quantitative and qualitative equivalence, with nonequivalence only for some aspects of auxiliary equivalence.
Abstract: Self-report survey-based data collection is increasingly carried out using the Internet, as opposed to the traditional paper-and-pencil method. However, previous research on the equivalence of these methods has yielded inconsistent findings. This may be due to methodological and statistical issues present in much of the literature, such as nonequivalent samples in different conditions due to recruitment, participant self-selection to conditions, and data collection procedures, as well as incomplete or inappropriate statistical procedures for examining equivalence. We conducted 2 studies examining the equivalence of paper-and-pencil and Internet data collection that accounted for these issues. In both studies, we used measures of personality, social desirability, and computer self-efficacy, and, in Study 2, we used personal growth initiative to assess quantitative equivalence (i.e., mean equivalence), qualitative equivalence (i.e., internal consistency and intercorrelations), and auxiliary equivalence (i.e., response rates, missing data, completion time, and comfort completing questionnaires using paper-and-pencil and the Internet). Study 1 investigated the effects of completing surveys via paper-and-pencil or the Internet in both traditional (i.e., lab) and natural (i.e., take-home) settings. Results indicated equivalence across conditions, except for auxiliary equivalence aspects of missing data and completion time. Study 2 examined mailed paper-and-pencil and Internet surveys without contact between experimenter and participants. Results indicated equivalence between conditions, except for auxiliary equivalence aspects of response rate for providing an address and completion time. Overall, the findings show that paper-and-pencil and Internet data collection methods are generally equivalent, particularly for quantitative and qualitative equivalence, with nonequivalence only for some aspects of auxiliary equivalence.
Citations
More filters
TL;DR: Results from 500 Australian employees indicate that knowledge of policy and procedures had a stronger influence on attitude towards policy and procedure than self-reported behaviour, suggesting that training and education will be more effective if it outlines not only what is expected but also provides an understanding of why this is important.
278 citations
Cites background or methods from "Examination of the Equivalence of S..."
...Finally, Weigold et al. (2013) compared self-report survey-based paper-and-pencil and Internet data collection methods and concluded that the methods are generally equivalent....
[...]
...Literature suggests there are potential problems associated with Internet data collection (Weigold et al., 2013; Reips, 2002)....
[...]
TL;DR: A unifying, comprehensive theoretical framework for understanding dark personality in terms of a general dispositional tendency of which dark traits arise as specific manifestations, which is called the Dark Factor of Personality (D).
Abstract: Many negatively connoted personality traits (often termed "dark traits") have been introduced to account for ethically, morally, and socially questionable behavior. Herein, we provide a unifying, comprehensive theoretical framework for understanding dark personality in terms of a general dispositional tendency of which dark traits arise as specific manifestations. That is, we theoretically specify the common core of dark traits, which we call the Dark Factor of Personality (D). The fluid concept of D captures individual differences in the tendency to maximize one's individual utility-disregarding, accepting, or malevolently provoking disutility for others-accompanied by beliefs that serve as justifications. To critically test D, we unify and extend prior work methodologically and empirically by considering a large number of dark traits simultaneously, using statistical approaches tailored to capture both the common core and the unique content of dark traits, and testing the predictive validity of both D and the unique content of dark traits with respect to diverse criteria including fully consequential and incentive-compatible behavior. In a series of four studies (N > 2,500), we provide evidence in support of the theoretical conceptualization of D, show that dark traits can be understood as specific manifestations of D, demonstrate that D predicts a multitude of criteria in the realm of ethically, morally, and socially questionable behavior, and illustrate that D does not depend on any particular indicator variable included. (PsycINFO Database Record (c) 2018 APA, all rights reserved).
233 citations
TL;DR: The totality of evidence indicates that there is no difference in social desirability between paper-and-pencil surveys and computer surveys.
227 citations
Cites background from "Examination of the Equivalence of S..."
...Weisband and Kiesler (1996) meta-analysed 30 studies that were published between 1967 and 1994 and found that computers evoked greater self-disclosure than did paper surveys (Cohen’s d = 0....
[...]
...Weisband and Kiesler (1996) meta-analysed 30 studies that were published between 1967 and 1994 and found that computers evoked greater self-disclosure than did paper surveys (Cohen’s d = 0.20), with the difference between administration modes reducing over time. Richman, Kiesler, Weisband, and Drasgow (1999) extended Weisband and Kiesler’s work by meta-analysing 42 studies published between 1967 and 1997, and found a near-zero effect (d = 0....
[...]
...Of the studies included in the meta-analysis, Hancock and Flowers (2001), Rossiter (2009), Uriell and Dudley (2009), and Weigold, Weigold, and Russell (2013) reported no significant differences between the response rates of Internet and paper respondents....
[...]
TL;DR: In this article, the authors examined the effect of biological sex on the willingness to use automated cars and found that these emotional processes vary as a function of respondent age in such a way that the differential effect of sex on anxiety was more pronounced among relatively young respondents and decreased with participants' age.
Abstract: Current research on willingness to use automated cars indicates differences between men and women, with the latter group showing lower usage intentions. This study aims at providing a first explanation of this effect. Research from other fields suggests that affective reactions might be able to explain behavioral intentions and responses towards technology, and that these affects vary depending on age levels. By examining a sample of 1603 participants representative for Germany (in terms of biological sex, age, and education) we found evidence that affective responses towards automotive cars (i.e., anxiety and pleasure) explain (i.e., mediate) the effect of biological sex on willingness to use them. Moreover, we found that these emotional processes vary as a function of respondent age in such a way that the differential effect of sex on anxiety (but not on pleasure) was more pronounced among relatively young respondents and decreased with participants’ age. Our results suggest that addressing anxiety-related responses towards automated cars (e.g., by providing safety-related information) and accentuating especially the pleasurable effects of automated cars (e.g., via advertising) reduce differences between men and women. Addressing the anxiety-related effects in order to reduce sex differences in usage intentions seems to be less relevant for older target groups, whereas promoting the pleasurable responses is equally important across age groups.
219 citations
TL;DR: Women were more accurate at identifying previously seen items than men in both Study 1 and Study 2, and Mechanical Turk participants were more likely to read instructions than undergraduate participants, regardless of whether they participated inside or outside of the lab.
154 citations
References
More filters
[...]
TL;DR: A convenient, although not comprehensive, presentation of required sample sizes is providedHere the sample sizes necessary for .80 power to detect effects at these levels are tabled for eight standard statistical tests.
Abstract: One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the inaccessibility of or difficulty with the standard material. A convenient, although not comprehensive, presentation of required sample sizes is provided here. Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects. The sample sizes necessary for .80 power to detect effects at these levels are tabled for eight standard statistical tests: (a) the difference between independent means, (b) the significance of a product-moment correlation, (c) the difference between independent rs, (d) the sign test, (e) the difference between independent proportions, (f) chi-square tests for goodness of fit and contingency tables, (g) one-way analysis of variance, and (h) the significance of a multiple or multiple partial correlation.
38,291 citations
TL;DR: Self-efficacy represents an important individual trait, which moderates organizational influences on an individual's decision to use computers, and is important to the successful implementation of systems in organizations.
Abstract: This paper discusses the role of individuals' beliefs about their abilities to competently use computers (computer self-efficacy) in the determination of computer use. A survey of Canadian managers and professionals was conducted to develop and validate a measure of computer self-efficacy and to assess both its impacts and antecedents. Computer self- efficacy was found to exert a significant influence on individuals' expectations of the outcomes of using computers, their emotional reactions to computers (affect and anxiety), as well as their actual computer use. An individual's self-efficacy and outcome expecta- tions were found to be positively influenced by the encouragement of others in their work group, as well as others' use of computers. Thus, self-efficacy represents an important individual trait, which moderates organizational influences (such as encouragement and support) on an individual's decision to use computers. Understanding self-efficacy, then, is important to the successful implementation of systems in organizations. The existence of a reliable and valid measure of self-efficacy makes assessment possible and should have implications for organizational support, training, and implementation.
5,717 citations
TL;DR: In a 6-group, random-assignment, placebo-controlled Internet study, the authors found that 3 of the interventions lastingly increased happiness and decreased depressive symptoms.
Abstract: Positive psychology has flourished in the last 5 years. The authors review recent developments in the field, including books, meetings, courses, and conferences. They also discuss the newly created classification of character strengths and virtues, a positive complement to the various editions of the Diagnostic and Statistical Manual of Mental Disorders (e. g., American Psychiatric Association, 1994), and present some cross-cultural findings that suggest a surprising ubiquity of strengths and virtues. Finally, the authors focus on psychological interventions that increase individual happiness. In a 6-group, random-assignment, placebo-controlled Internet study, the authors tested 5 purported happiness interventions and 1 plausible control exercise. They found that 3 of the interventions lastingly increased happiness and decreased depressive symptoms. Positive interventions can supplement traditional interventions that relieve suffering and may someday be the practical legacy of positive psychology.
4,813 citations
TL;DR: The authors reviewed the problems with null hypothesis significance testing, including near universal misinterpretation of p as the probability that H is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test.
Abstract: After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews the problems with this practice, including near universal misinterpretation of p as the probability that H₀ is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test. Exploratory data analysis and the use of graphic methods, a steady improvement in and a movement toward standardization in measurement, an emphasis on estimating effect sizes using confidence intervals, and the informed use of available statistical methods are suggested. For generalization, psychologists must finally rely, as has been done in all the older sciences, on replication. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
3,838 citations
01 Jan 1999
2,990 citations