There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Why most published research findings are false

National accounts of subjective well-being are being considered and adopted by nations. In order to be useful for policy deliberations, the measures of life satisfaction must be psychometrically sound. The reliability, validity, and sensitivity to change of life satisfaction measures are reviewed. The scales are stable under unchanging conditions, but are sensitive to changes in circumstances in people’s lives. Several types of data indicate that the scales validly reflect the quality of respondents’ lives: 1. Differences between nations in life satisfaction associated with differences in objective conditions, 2. Differences between groups who live in different circumstances, 3. Correlations with nonself-report measures of life satisfaction, 4. Genetic and physiological associations with life satisfaction, 5. Systematic patterns of change in the scales before, during, and after significant life events, and 6. Prediction by life satisfaction scores of future behaviors such as suicide. The life satisfaction scales can be influenced by factors such as question order, current mood, and mode of presentation, but in most cases these can be controlled. Our model of life satisfaction judgments points to the importance of attention, values, standards, and top-down effects. Although the scales are useful in research on individual well-being, there are policy questions that need more analysis and research, such as which types of subjective well-being measures are most relevant to which types of policies, how standards influence scores, and how best to associate the scores with current policy deliberations.

Theory and Validity of Life Satisfaction Scales

National accounts of subjective well-being are being considered and adopted by nations. In order to be useful for policy deliberations, the measures of life satisfaction must be psychometrically sound. The reliability, validity, and sensitivity to change of life satisfaction measures are reviewed. The scales are stable under unchanging conditions, but are sensitive to changes in circumstances in people’s lives. Several types of data indicate that the scales validly reflect the quality of respondents’ lives: (1) Differences between nations in life satisfaction associated with differences in objective conditions, (2) Differences between groups who live in different circumstances, (3) Correlations with nonself-report measures of life satisfaction, (4) Genetic and physiological associations with life satisfaction, (5) Systematic patterns of change in the scales before, during, and after significant life events, and (6) Prediction by life satisfaction scores of future behaviors such as suicide. The life satisfaction scales can be influenced by factors such as question order, current mood, and mode of presentation, but in most cases these can be controlled. Our model of life satisfaction judgments points to the importance of attention, values, standards, and top-down effects. Although the scales are useful in research on individual well-being, there are policy questions that need more analysis and research, such as which types of subjective well-being measures are most relevant to which types of policies, how standards influence scores, and how best to associate the scores with current policy deliberations.

We review research and theory on the HEXACO personality dimensions of Honesty-Humility (H), Agreeableness (A), and Emotionality (E), with particular attention to the following topics: (1) the origins of the HEXACO model in lexical studies of personality structure, and the content of the H, A, and E factors in those studies; (2) the operationalization of the H, A, and E factors in the HEXACO Personality Inventory–Revised; (3) the construct validity of self-reports on scales measuring the H factor; (4) the theoretical distinction between H and A; (5) similarity and assumed similarity between social partners in personality, with a focus on H and A; (6) the extent to which H (and A and E) variance is represented in instruments assessing the “Five-Factor Model” of personality; and (7) the relative validity of scales assessing the HEXACO and Five-Factor Model dimensions in predicting criteria conceptually relevant to H, A, and E.

The HEXACO Honesty-Humility, Agreeableness, and Emotionality Factors: A Review of Research and Theory

https://scottbarrykaufman.com/wp-content/uploads/2014/08/DeYoung-2014-CB5T-JRP.pdf

Cybernetic Big Five Theory.

In light of consistently observed correlations among Big Five ratings, the authors developed and tested a model that combined E. L. Thorndike's (1920) general evaluative bias (halo) model and J. M. Digman's (1997) higher order personality factors (alpha and beta) model. With 4 multitrait-multimethod analyses, Study 1 revealed moderate convergent validity for alpha and beta across raters, whereas halo was mainly a unique factor for each rater. In Study 2, the authors showed that the halo factor was highly correlated with a validated measure of evaluative biases in self-ratings. Study 3 showed that halo is more strongly correlated with self-ratings of self-esteem than self-ratings of the Big Five, which suggests that halo is not a mere rating bias but actually reflects overly positive self-evaluations. Finally, Study 4 demonstrated that the halo bias in Big Five ratings is stable over short retest intervals. Taken together, the results suggest that the halo-alpha-beta model integrates the main findings in structural analyses of Big Five correlations. Accordingly, halo bias in self-ratings is a reliable and stable bias in individuals' perceptions of their own attributes. Implications of the present findings for the assessment of Big Five personality traits in monomethod studies are discussed.

http://www.psych.utoronto.ca/users/lockwood/PDF/Anusic%202009.pdf

The nature and structure of correlations among Big Five ratings: the halo-alpha-beta model.

The stability of individual differences is a fundamental issue in personality psychology. Although accumulating evidence suggests that many psychological attributes are both stable and change over time, existing research rarely takes advantage of theoretical models that capture both stability and change. In this article, we present the Meta-Analytic Stability and Change model (MASC), a novel meta-analytic model for synthesizing data from longitudinal studies. MASC is based on trait-state models that can separate influences of stable and changing factors from unreliable variance (Kenny & Zautra, 1995). We used MASC to evaluate the extent to which personality traits, life satisfaction, affect, and self-esteem are influenced by these different factors. The results showed that the majority of reliable variance in personality traits is attributable to stable influences (83%). Changing factors had a greater influence on reliable variance in life satisfaction, self-esteem, and affect than in personality (42%-56% vs. 17%). In addition, changing influences on well-being were more stable than changing influences on personality traits, suggesting that different changing factors contribute to personality and well-being. Measures of affect were less reliable than measures of the other 3 constructs, reflecting influences of transient factors, such as mood on affective judgments. After accounting for differences in reliability, stability of affect did not differ from other well-being variables. Consistent with previous research, we found that stability of individual differences increases with age. (PsycINFO Database Record

Stability and change of personality traits, self-esteem, and well-being: Introducing the meta-analytic stability and change model of retest correlations.

http://news.msu.edu/media/documents/2012/05/3eee4c51-e850-4028-ad59-bff3e186b9d0.pdf

Does Personality Moderate Reaction and Adaptation to Major Life Events? Evidence from the British Household Panel Survey.

Set-point theory posits that individuals react to the experience of major life events, but quickly adapt back to pre-event baseline levels of subjective well-being in the years following the event. A large, nationally representative panel study of Swiss households was used to examine set-point theory by investigating the extent of adaptation following the experience of marriage, childbirth, widowhood, unemployment, and disability. Our results demonstrate that major life events are associated with marked change in life satisfaction and, for some events (e.g., marriage, disability), these changes are relatively long lasting even when accounting for normative, age related change.

Testing Set-Point Theory in a Swiss National Sample: Reaction and Adaptation to Major Life Events.

The Day Reconstruction Method is a useful tool for evaluating short-term changes in emotional experiences over a variety of daily situations. However, traditional method of collecting DRM data can be time-intensive for both researchers and participants. In this paper we provide evidence that a random-sampling approach to DRM assessment can provide useful data that are largely consistent with previous research that used the full version of the DRM. In a nationally representative sample of 2,303 people, we demonstrate that (1) there is variability in emotional ratings of episodes that replicates what has been found in prior studies, (2) correlations with global measures are typically small in magnitude (< .30), (3) correlations with personality are for the most part negligible, (4) correlations with global ratings of domain satisfaction are higher for domain-relevant situations, and (5) parents report more positive affect while providing care for their children when compared to other activities, and this effect can account for the observed differences in emotional experiences of parents and non-parents.

/pdf/the-validity-of-the-day-reconstruction-method-in-the-german-1fezazm5bx.pdf

Ivana Anusic

Papers

The nature and structure of correlations among Big Five ratings: the halo-alpha-beta model.

Stability and change of personality traits, self-esteem, and well-being: Introducing the meta-analytic stability and change model of retest correlations.

Does Personality Moderate Reaction and Adaptation to Major Life Events? Evidence from the British Household Panel Survey.

Testing Set-Point Theory in a Swiss National Sample: Reaction and Adaptation to Major Life Events.

The Validity of the Day Reconstruction Method in the German Socio-economic Panel Study