Computational Language Assessments of Harmony in Life - Not Satisfaction With Life or Rating Scales - Correlate With Cooperative Behaviors.
TL;DR: In this paper, the authors examined whether harmony in life and satisfaction with life are related differently to cooperative behaviors depending on individuals' social value orientation and concluded that different types of well-being are likely to be associated with different kinds of behaviors.
Abstract: Different types of well-being are likely to be associated with different kinds of behaviors. The first objective of this study was, from a subjective well-being perspective, to examine whether harmony in life and satisfaction with life are related differently to cooperative behaviors depending on individuals' social value orientation. The second objective was, from a methodological perspective, to examine whether language-based assessments called computational language assessments (CLA), which enable respondents to answer with words that are analyzed using natural language processing, demonstrate stronger correlations with cooperation than traditional rating scales. Participants reported their harmony in life, satisfaction with life, and social value orientation before taking part in an online cooperative task. The results show that the CLA of overall harmony in life correlated with cooperation (all participants: r = 0.18, p 0.05). No significant correlations (measured by the CLA or traditional rating scales) were found between satisfaction with life and cooperation. In conclusion, our study reveals an important behavioral difference between different types of subjective well-being. To our knowledge, this is the first study supporting the validity of self-reported CLA over traditional rating scales in relation to actual behaviors.
Citations
More filters
TL;DR: The authors showed that using a recent break-through in artificial intelligence -transformers- psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales.
Abstract: We show that using a recent break-through in artificial intelligence -transformers-, psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales. Text-responses use people's primary form of communication -natural language- and have been suggested as a more ecologically-valid response format than closed-ended rating scales that dominate social science. However, previous language analysis techniques left a gap between how accurately they converged with standard rating scales and how well ratings scales converge with themselves - a theoretical upper-limit in accuracy. Most recently, AI-based language analysis has gone through a transformation as nearly all of its applications, from Web search to personalized assistants (e.g., Alexa and Siri), have shown unprecedented improvement by using transformers. We evaluate transformers for estimating psychological well-being from questionnaire text- and descriptive word-responses, and find accuracies converging with rating scales that approach the theoretical upper limits (Pearson r = 0.85, p < 0.001, N = 608; in line with most metrics of rating scale reliability). These findings suggest an avenue for modernizing the ubiquitous questionnaire and ultimately opening doors to a greater understanding of the human condition.
12 citations
TL;DR: In this paper, the authors investigated whether the semantic information that participants generate contain information of all, or some, of the criteria that define depression and anxiety in clinical practices and found that the semantic measures approach significantly predict all self-reported criteria and that items measuring cognitive aspect yielded higher predictability than behavioral items.
Abstract: Background. Self-reported language-based assessments, that are based on freely generated word responses and analyzed with artificial intelligence, is a potential complement to currently used methods in identifying mental health issues. In a previous study, this approach demonstrated higher, or competitive, validity and reliability as compared with the total score of the state-of-the-art rating scales. This study investigates to what extent this approach capture symptom-based items in rating scales targeting depression and anxiety. To add value to clinical practices the semantic measures approach needs to capture cognitive, behavioral and physiological symptoms associated with mental health aspects described in diagnostic criteria e.g., the Diagnostic and Statistical Manual of Mental Disorders (DSM- 5). We study this by investigating whether the semantic information that participants generates contain information of all, or some, of the criteria that defines depression and anxiety in clinical practices. Method. Participants (N=411) described their mental health with freely generated words and rating scales relating to depression and worry/anxiety. Word responses were quantified and analyzed using natural language processing and machine learning. Results. The semantic measures correlated significantly with the individual items connected to the DSM 5 diagnostic criteria of depression (Pearson’s r = .22 - .51, p<.001) and worry (anxiety rating scale: Pearson’s r = .29 - .44, p<.001; worry rating scale: Pearson’s r = .35 - .44, p<.001) for respective rating scales. Conclusion. The semantic measures correlated significantly with the individual items of depression and worry for respective rating scales. The results indicate that the semantic measures approach significantly predict all self-reported criteria and that items measuring cognitive aspect yielded higher predictability than behavioral items. The valence aspect of the semantic representation appears to carry a lot information, potentially capturing a more general negative feeling and a help seeking behavior common for depression and anxiety. Together these results support that semantic measures may be apt to complement other methods in measuring mental health in clinical settings.
5 citations
TL;DR: This article examined the relationship between self-reported everyday activities and subjective well-being, while allowing individuals to express their activities freely by allowing open-ended responses that were then analyzed with state-of-the-art NLP techniques.
Abstract: Activities and Subjective Well-Being (SWB) have been shown to be intricately related to each other. However, no research to date has shown whether individuals understand how their everyday activities relate to their SWB. Furthermore, the assessment of activities has been limited to predefined types of activities and/or closed-ended questions. In two studies, we examine the relationship between self-reported everyday activities and SWB, while allowing individuals to express their activities freely by allowing open-ended responses that were then analyzed with state-of-the-art (transformers-based) Natural Language Processing. In study 1 (N = 284), self-reports of Yesterday’s Activities did not significantly relate to SWB, whereas activities reported as having the most impact on SWB in the past four weeks had small but significant correlations to most of the SWB scales (r = .14 –.23, p < .05). In Study 2 (N = 295), individuals showed strong agreement with each other about activities that they considered to increase or decrease SWB (AUC = .995). Words describing activities that increased SWB related to physically and cognitively active activities and social activities (“football”, “meditation”, “friends”), whereas words describing activities that decreased SWB were mainly activity features related to imbalance (“too”, “much”, “enough”). Individuals reported both activities and descriptive words that reflect their SWB, where the activity words had generally small but significant correlations to SWB (r =. 17 –.33, p < .05) and the descriptive words had generally strong correlations to SWB (r = .39–63, p < .001). We call this correlational gap the well-being/activity description gap and discuss possible explanations for the phenomenon.
1 citations
TL;DR: In this paper , the authors assessed respondents' degree of depression using rating scales, descriptive words, selected words, and free text responses and probed the respondents for their preferences concerning the response formats across twelve dimensions related to the precision of communicating their mental states and the ease of responding.
Abstract: Background Closed-ended rating scales are the most used response format for researchers and clinicians to quantify mental states, whereas in natural contexts people communicate with natural language. The reason for using such scales is that they are typically argued to be more precise in measuring mental constructs; however, the respondents’ views as to what best communicates mental states are frequently ignored, which is important for making them comply with assessment. Methods We assessed respondents’ (N = 304) degree of depression using rating scales, descriptive words, selected words, and free text responses and probed the respondents for their preferences concerning the response formats across twelve dimensions related to the precision of communicating their mental states and the ease of responding. This was compared with the clinicians’ (N = 40) belief of the respondent’s view. Results Respondents found free text to be more precise (e.g., precision d’ = .88, elaboration d’ = 2.0) than rating scales, whereas rating scales were rated as easier to respond to (e.g., easier d’ = –.67, faster d’ = –1.13). Respondents preferred the free text responses to a greater degree than rating scales compared to clinicians’ belief of the respondents’ views. Conclusions These findings support previous studies concluding that future assessment of mental health can be aided by computational methods based on text data. Participants prefer an open response format as it allows them to elaborate, be precise, etc., with respect to their mental health issues, although rating scales are viewed as faster and easier.
TL;DR: This finding supports the idea that future assessment of mental health can be aided by computational method based on text data and preferred the free text responses to a greater degree than rating scales compared to clinicians.
Abstract: Background Closed-ended rating scales are the most used response format for researchers and clinicians to quantify mental states, whereas in natural contexts people communicate with natural language. The reason for using such scales is that they are typically argued to be more precise in measuring mental constructs, whereas the respondents views as to what best communicates mental states are frequently ignored. Methods We assessed respondents (N = 304) degree of depression using rating scales, descriptive words, selected words, and free text responses and probed the respondents and clinicians (N = 40) for their attitudes to the response formats across twelve dimensions related to the precision of communicating their mental states and the ease of responding. Results Respondents found free text to be more precise (e.g., precision d = .88, elaboration d = 2.0) than rating scales, whereas rating scales were rated as easier to respond to (e.g., easier d = -.67, faster d = -1.13). Respondents preferred the free text responses to a greater degree than rating scales compared to clinicians. Conclusions These finding supports the idea that future assessment of mental health can be aided by computational method based on text data.
References
More filters
TL;DR: The Satisfaction With Life Scale (SWLS) as mentioned in this paper is a scale to measure global life satisfaction, which does not tap related constructs such as positive affect or loneliness, and has favorable psychometric properties, including high internal consistency and high temporal reliability.
Abstract: This article reports the development and validation of a scale to measure global life satisfaction, the Satisfaction With Life Scale (SWLS). Among the various components of subjective well-being, the SWLS is narrowly focused to assess global life satisfaction and does not tap related constructs such as positive affect or loneliness. The SWLS is shown to have favorable psychometric properties, including high internal consistency and high temporal reliability. Scores on the SWLS correlate moderately to highly with other measures of subjective well-being, and correlate predictably with specific personality characteristics. It is noted that the SWLS is Suited for use with different age groups, and other potential uses of the scale are discussed.
20,751 citations
TL;DR: Findings indicate that MTurk can be used to obtain high-quality data inexpensively and rapidly and the data obtained are at least as reliable as those obtained via traditional methods.
Abstract: Amazon's Mechanical Turk (MTurk) is a relatively new website that contains the major elements required to conduct research: an integrated participant compensation system; a large participant pool; and a streamlined process of study design, participant recruitment, and data collection. In this article, we describe and evaluate the potential contributions of MTurk to psychology and other social sciences. Findings indicate that (a) MTurk participants are slightly more demographically diverse than are standard Internet samples and are significantly more diverse than typical American college samples; (b) participation is affected by compensation rate and task length, but participants can still be recruited rapidly and inexpensively; (c) realistic compensation rates do not affect data quality; and (d) the data obtained are at least as reliable as those obtained via traditional methods. Overall, MTurk can be used to obtain high-quality data inexpensively and rapidly.
9,562 citations
TL;DR: This review considers research from both perspectives concerning the nature of well-being, its antecedents, and its stability across time and culture.
Abstract: ▪ Abstract Well-being is a complex construct that concerns optimal experience and functioning. Current research on well-being has been derived from two general perspectives: the hedonic approach, which focuses on happiness and defines well-being in terms of pleasure attainment and pain avoidance; and the eudaimonic approach, which focuses on meaning and self-realization and defines well-being in terms of the degree to which a person is fully functioning. These two views have given rise to different research foci and a body of knowledge that is in some areas divergent and in others complementary. New methodological developments concerning multilevel modeling and construct comparisons are also allowing researchers to formulate new questions for the field. This review considers research from both perspectives concerning the nature of well-being, its antecedents, and its stability across time and culture.
8,243 citations
TL;DR: A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena.
Abstract: How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena. By inducing global knowledge indirectly from local co-occurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren. LSA uses no prior linguistic or perceptual similarity knowledge; it is based solely on a general mathematical learning method that achieves powerful inductive effects by extracting the right number of dimensions (e.g., 300) to represent objects and contexts. Relations to other theories, phenomena, and problems are sketched.
6,014 citations
Posted Content•
TL;DR: The authors presented new demographic data about the Mechanical Turk subject population, reviewed the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and compared the magnitude of effects obtained using Mechanical Turk and traditional subject pools.
Abstract: Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.
3,059 citations