The measurement of meaning

We introduce the Berlin Numeracy Test, a new psychometrically sound instrument that quickly assesses statistical numeracy and risk literacy. We present 21 studies (n=5336) showing robust psychometric discriminability across 15 countries (e.g., Germany, Pakistan, Japan, USA) and diverse samples (e.g., medical professionals, general populations, Mechanical Turk web panels). Analyses demonstrate desirable patterns of convergent validity (e.g., numeracy, general cognitive abilities), discriminant validity (e.g., personality, motivation), and criterion validity (e.g., numerical and nonnumerical questions about risk). The Berlin Numeracy Test was found to be the strongest predictor of comprehension of everyday risks (e.g., evaluating claims about products and treatments; interpreting forecasts), doubling the predictive power of other numeracy instruments and accounting for unique variance beyond other cognitive tests (e.g., cognitive reflection, working memory, intelligence). The Berlin Numeracy Test typically takes about three minutes to complete and is available in multiple languages and formats, including a computer adaptive test that automatically scores and reports data to researchers (www.riskliteracy.org). The online forum also provides interactive content for public outreach and education, and offers a recommendation system for test format selection. Discussion centers on construct validity of numeracy for risk literacy, underlying cognitive mechanisms, and applications in adaptive decision support.

/pdf/measuring-risk-literacy-the-berlin-numeracy-test-f89dsieofl.pdf

Measuring risk literacy: The Berlin numeracy test

The Realm of Rights

Much research in the last two decades has demonstrated that human responses deviate from the performance deemed normative according to various models of decision making and rational judgment (e.g., the basic axioms of utility theory). This gap between the normative and the descriptive can be interpreted as indicating systematic irrationalities in human cognition. However, four alternative interpretations preserve the assumption that human behavior and cognition is largely rational. These posit that the gap is due to (1) performance errors, (2) computational limitations, (3) the wrong norm being applied by the experimenter, and (4) a different construal of the task by the subject. In the debates about the viability of these alternative explanations, attention has been focused too narrowly on the model response. In a series of experiments involving most of the classic tasks in the heuristics and biases literature, we have examined the implications of individual differences in performance for each of the four explanations of the normative/descriptive gap. Performance errors are a minor factor in the gap; computational limitations underlie non-normative responding on several tasks, particularly those that involve some type of cognitive decontextualization. Unexpected patterns of covariance can suggest when the wrong norm is being applied to a task or when an alternative construal of the task should be considered appropriate.

Individual differences in reasoning: Implications for the rationality debate?-Open Peer Commentary-Differences, games, and pluralism

As members of a democratic society, individuals face complex decisions about whether to support climate change mitigation, vaccinations, genetically modified food, nanotechnology, geoengineering, and so on. To inform people’s decisions and public debate, scientific experts at government agencies, nongovernmental organizations, and other organizations aim to provide understandable and scientifically accurate communication materials. Such communications aim to improve people’s understanding of the decision-relevant issues, and if needed, promote behavior change. Unfortunately, existing communications sometimes fail when scientific experts lack information about what people need to know to make more informed decisions or what wording people use to describe relevant concepts. We provide an introduction for scientific experts about how to use mental models research with intended audience members to inform their communication efforts. Specifically, we describe how to conduct interviews to characterize people’s decision-relevant beliefs or mental models of the topic under consideration, identify gaps and misconceptions in their knowledge, and reveal their preferred wording. We also describe methods for designing follow-up surveys with larger samples to examine the prevalence of beliefs as well as the relationships of beliefs with behaviors. Finally, we discuss how findings from these interviews and surveys can be used to design communications that effectively address gaps and misconceptions in people’s mental models in wording that they understand. We present applications to different scientific domains, showing that this approach leads to communications that improve recipients’ understanding and ability to make informed decisions.

https://www.pnas.org/content/pnas/110/Supplement_3/14062.full.pdf

Assessing what to address in science communication

Graph literacy is an often neglected skill that influences decision making performance. We conducted an experiment to investigate whether individual differences in graph literacy affect the extent to which people benefit from visual aids (icon arrays) designed to reduce a common judgment bias (i.e., denominator neglect—a focus on numerators in ratios while neglecting denominators). Results indicated that icon arrays more often increased risk comprehension accuracy and confidence among participants with high graph literacy as compared with those with low graph literacy. Results held regardless of how the health message was framed (chances of dying versus chances of surviving). Findings contribute to our understanding of the ways in which individual differences in cognitive abilities interact with the comprehension of different risk representation formats. Theoretical, methodological, and prescriptive implications of the results are discussed (e.g., the effective communication of quantitative medical data). Copyright © 2011 John Wiley & Sons, Ltd.

Individual Differences in Graph Literacy: Overcoming Denominator Neglect in Risk Comprehension

Recent research has shown that patients frequently experience difficulties understanding health-relevant numerical concepts. A prominent example is denominator neglect, or the tendency to pay too much attention to numerators in ratios (e.g., number of treated patients who died) with insufficient attention to denominators (e.g., overall number of treated patients). Denominator neglect can lead to inaccurate assessments of treatment risk reduction and thus can have important consequences for decisions about health. Here, we reviewed a series of studies investigating (1) different factors that can influence patients' susceptibility to denominator neglect in medical decision making--including numerical or language-related abilities; (2) the extent to which denominator neglect can be attenuated by using visual aids; and (3) a factor that moderates the effectiveness of such aids (i.e., graph literacy). The review spans probabilistic national U.S. and German samples, as well as immigrant (i.e., Polish people living in the United Kingdom) and undergraduate samples in Spain. Theoretical and prescriptive implications are discussed.

https://downloads.hindawi.com/journals/tswj/2012/562637.pdf

Using Visual Aids to Improve Communication of Risks about Health: A Review

Order Effects in Moral Judgment Alex Wiegmann (Alex.Wiegmann@psych.uni-goettingen.de) Jonas Nagel (jnagel1@uni-goettingen.com) Stefan Mangold (smangol@gwdg.de) Department of Psychology, University of Gottingen, Germany Yasmina Okan(yokan@ugr.es) Department of Experimental Psychology, University of Granada Campus Universitario de la Cartuja s/n ,18071, Granada, Spain Abstract Explaining moral intuitions is one of the hot topics of recent cognitive sciences. In the present article we focus on a factor that attracted surprisingly little attention so far, namely the temporal order in which moral scenarios are presented. We argue that previous research points to a systematic pattern of order effects that has been overlooked until now: Only judgments of actions that are normally regarded as morally acceptable are affected by the order of presentation. Additionally, this is only the case for dilemmas immediately preceded by a scenario where the proposed action was judged as morally unacceptable. We conducted an experiment that confirmed this pattern and allowed us to analyze the individual level responses it was generated by. We argue that investigating order effects is necessary for approaching a complete descriptive moral theory. Keywords: moral intuitions; trolley dilemmas; order effects Introduction In the past decades, trolley dilemmas have been used extensively for testing philosophical and psychological theories of moral judgments. In the standard description of the trolley dilemma introduced by Philippa Foot (1967), an out-of-control train threatens to kill five people standing on its track. The only way to prevent this is to pull a switch that redirects the train onto a different track where it will kill only one person. In a modification of this scenario (Thomson, 1976), the only possibility to prevent the five people from being killed is to push a heavy person from a footbridge onto the track. This would stop the train but kill the heavy person. Numerous studies (e.g., Hauser, Cushman, Young, Jin, & Mikhail, 2007) have shown that given the same number of people being killed vs. saved, participants approve of acting in the first but not in the second scenario. Several competing descriptive theories explicate psychological principles supposed to underlie this pattern of moral intuitions (e.g. Greene et al., 2001; Hauser, 2006). However, surprisingly little is known about potential effects of the order in which several consecutive scenarios are presented. It is plausible to assume that consecutive scenarios will not be judged independently of each other: A principle or mechanism that is activated when a particular scenario is represented or evaluated might later be applied to a series of subsequent scenarios. However, only few studies have dealt with this issue so far (Petrinovich & O’Neill, 1996; Lanteri, Chelini, & Rizzello, 2008). Their results suggest that under certain circumstances moral judgments can indeed be transferred from one situation to another. If such order effects could be replicated systematically, this would have important implications for psychological theories aiming to explain patterns of moral reasoning at a descriptive level. Furthermore, relevant practical implications would arise both for methodological considerations inside the research laboratory (in terms of controlling for order effects when designing experiments) and for everyday judgments outside the lab. The present work has three main goals. First, we will provide the first comprehensive review of previous empirical research on order effects in moral judgments, and we will demonstrate that a systematic pattern of results has been overlooked so far. Second, we will empirically test the existence, extent and direction of order effects in reasoning about moral dilemmas. Finally, we will discuss the theoretical and practical implications of our findings, focusing on psychological theories of moral reasoning. Order Effects in Previous Research Speaking of order effects in moral judgment, there are at least two possible interpretations that could be labelled “within-scenario order effect” and “between-scenarios order effect”, respectively. The first type of effect results if the order in which information concerning one particular situation is presented affects judgment. If, for example, the task is to judge the permissibility of an action, and the results solely differ as a function of the particular sequence in which positive and negative consequences are presented, a “within-scenario order effect” occurs. Second, a judgment regarding an action in a particular scenario might be influenced by a judgment that had previously been made about a different scenario. To illustrate, consider two conditions in which a given scenario C is preceded by one of two different scenarios (A vs. B). Differences in judgments of the action scenario C between the two conditions would – all other things being equal – instantiate a “between-scenarios order effect”. The present research will focus on this second category of order effects.

/pdf/order-effects-in-moral-judgment-20y6qg3wwe.pdf

Order Effects in Moral Judgment

Research on moral judgment has shown that the order in which dilemmas are presented to subjects often has a strong influence on their judgment. However, the psychological mechanisms underlying order effects are still opaque. In this paper we aimed to isolate the features that a scenario must exhibit in order to influence judgment of subsequent scenarios. For this enterprise, we identified several features from a scenario known to cause order effects, and tested which of these features are necessary to influence subsequent scenarios. Although we still do not have a full understanding of what causes order effects, we made significant progress towards this aim. In five experiments we ruled out some promising explanations such as order effects being driven by an emotional activation linked to the first scenario. Instead, we found order effects to depend on whether the scenario being influenced and its preceding scenario share rather subtle structural similarities.

/pdf/order-effects-in-moral-judgment-2nsg0av4mg.pdf

Yasmina Okan

Papers

Individual Differences in Graph Literacy: Overcoming Denominator Neglect in Risk Comprehension

Using Visual Aids to Improve Communication of Risks about Health: A Review

Order Effects in Moral Judgment

Order effects in moral judgment

Do the media provide transparent health information? A cross-cultural comparison of public information about the HPV vaccine.