A Meta-Analysis on the Correlation Between the Implicit Association Test and Explicit Self-Report Measures

doi:10.1177/0146167205275613

Home
/
Papers
/
A Meta-Analysis on the Correlation Between the Implicit Association Test and Explicit Self-Report Measures

Journal Article•DOI•

A Meta-Analysis on the Correlation Between the Implicit Association Test and Explicit Self-Report Measures

Wilhelm Hofmann, Bertram Gawronski¹, Tobias Gschwendner, Huy Le, Manfred Schmitt - Show less +1 more•Institutions (1)

University of Western Ontario¹

01 Oct 2005-Personality and Social Psychology Bulletin (SAGE Publications)-Vol. 31, Iss: 10, pp 1369-1385

TL;DR: The results suggest that implicit and explicit measures are generally related but that higher order inferences and lack of conceptual correspondence can reduce the influence of automatic associations on explicit self-reports.

read less

Abstract: Theoretically, low correlations between implicit and explicit measures can be due to (a) motivational biases in explicit self reports, (b) lack of introspective access to implicitly assessed representations, (c) factors influencing the retrieval of information from memory, (d) method-related characteristics of the two measures, or (e) complete independence of the underlying constructs. The present study addressed these questions from a meta-analytic perspective, investigating the correlation between the Implicit Association Test (IAT) and explicit self-report measures. Based on a sample of 126 studies, the mean effect size was .24, with approximately half of the variability across correlations attributable to moderator variables. Correlations systematically increased as a function of (a) increasing spontaneity of self-reports and (b) increasing conceptual correspondence between measures. These results suggest that implicit and explicit measures are generally related but that higher order inferences and lack of conceptual correspondence can reduce the influence of automatic associations on explicit self-reports.

...read moreread less

Figures (1)

Figure 1. Funnel-graph of reported uncorrected effect size and sample size.

Content maybe subject to copyright Report

Citations

PDF

Open Access

More filters

Book•

The WEIRDest People in the World

[...]

Joseph Henrich¹, Steven J. Heine¹, Ara Norenzayan¹•Institutions (1)

University of British Columbia¹

08 Sep 2020

TL;DR: A review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers.

...read moreread less

Abstract: Behavioral scientists routinely publish broad claims about human psychology and behavior in the world's top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers - often implicitly - assume that either there is little variation across human populations, or that these "standard subjects" are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species - frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior - hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges.

...read moreread less

6,370 citations

Journal Article•DOI•

Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity.

[...]

Anthony G. Greenwald¹, T. Andrew Poehlman², Eric Luis Uhlmann³, Mahzarin R. Banaji⁴•Institutions (4)

University of Washington¹, Southern Methodist University², Northwestern University³, Harvard University⁴

01 Jul 2009-Journal of Personality and Social Psychology

TL;DR: A review of 122 research reports (184 independent samples, 14,900 subjects) found average r =.274 for prediction of behavioral, judgment, and physiological measures by Implicit Association Test (IAT) measures as mentioned in this paper.

...read moreread less

Abstract: This review of 122 research reports (184 independent samples, 14,900 subjects) found average r = .274 for prediction of behavioral, judgment, and physiological measures by Implicit Association Test (IAT) measures. Parallel explicit (i.e., self-report) measures, available in 156 of these samples (13,068 subjects), also predicted effectively (average r = .361), but with much greater variability of effect size. Predictive validity of self-report was impaired for socially sensitive topics, for which impression management may distort self-report responses. For 32 samples with criterion measures involving Black-White interracial behavior, predictive validity of IAT measures significantly exceeded that of self-report measures. Both IAT and self-report measures displayed incremental validity, with each measure predicting criterion variance beyond that predicted by the other. The more highly IAT and self-report measures were intercorrelated, the greater was the predictive validity of each.

...read moreread less

2,690 citations

Journal Article•DOI•

Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change.

[...]

Bertram Gawronski¹, Galen V. Bodenhausen²•Institutions (2)

University of Western Ontario¹, Northwestern University²

01 Sep 2006-Psychological Bulletin

TL;DR: An integrative review of the available evidence on implicit and explicit attitude change that is guided by a distinction between associative and propositional processes is provided.

...read moreread less

Abstract: A central theme in recent research on attitudes is the distinction between deliberate, "explicit" attitudes and automatic, "implicit" attitudes. The present article provides an integrative review of the available evidence on implicit and explicit attitude change that is guided by a distinction between associative and propositional processes. Whereas associative processes are characterized by mere activation independent of subjective truth or falsity, propositional reasoning is concerned with the validation of evaluations and beliefs. The proposed associative-propositional evaluation (APE) model makes specific assumptions about the mutual interplay of the 2 processes, implying several mechanisms that lead to symmetric or asymmetric changes in implicit and explicit attitudes. The model integrates a broad range of empirical evidence and implies several new predictions for implicit and explicit attitude change.

...read moreread less

2,191 citations

Cites background from "A Meta-Analysis on the Correlation ..."

...…any of these additional propositions is inconsistent with the automatic evaluative response, the extra elaboration is likely to reduce the correlation between automatic affective reactions and evaluative judgments (e.g., Florack et al., 2001; Hofmann et al., 2005; see also Shiv & Nowlis, 2004)....
[...]

Journal Article•DOI•

An inkblot for attitudes: affect misattribution as implicit measurement.

[...]

B. Keith Payne¹, Clara Michelle Cheng¹, Olesya Govorun¹, Brandon D. Stewart¹•Institutions (1)

Ohio State University¹

01 Sep 2005-Journal of Personality and Social Psychology

TL;DR: The affect misattributions procedure (AMP) was sensitive to normatively favorable and unfavorable evaluations, and the misattribution effect was strong at both fast and slow presentation rates.

...read moreread less

Abstract: Misattributions people make about their own affective reactions can be used to measure attitudes implicitly. Combining the logic of projective tests with advances in priming research, the affect misattribution procedure (AMP) was sensitive to normatively favorable and unfavorable evaluations (Experiments 1-4), and the misattribution effect was strong at both fast and slow presentation rates (Experiments 3 and 4). Providing further evidence of validity, the AMP was strongly related to individual differences in self-reported political attitudes and voting intentions (Experiment 5). In the socially sensitive domain of racial attitudes, the AMP showed in-group bias for Black and White participants. AMP performance correlated with explicit racial attitudes, a relationship that was moderated by motivations to control prejudice (Experiment 6). Across studies, the task was unaffected by direct warnings to avoid bias. Advantages of the AMP include large effect sizes, high reliability, ease of use, and resistance to correction attempts.

...read moreread less

1,189 citations

The Implicit Association Test at Age 7: A Methodological and Conceptual Review

[...]

Brian A. Nosek¹, Anthony G. Greenwald², Mahzarin R. Banaji³•Institutions (3)

University of Virginia¹, University of Washington², Harvard University³

01 Jan 2007

TL;DR: In psychology, the ability to introspect comes with the palpable feeling of " knowing, " of being objective or certain, of being mentally in control of one's thoughts, aware of the causes of the thoughts, feelings, and actions, and of making decisions deliberately and rationally as discussed by the authors.

...read moreread less

Abstract: A mong earthly organisms, humans have a unique propensity to introspect or look inward into the contents of their own minds, and to share those observations with others. With the ability to introspect comes the palpable feeling of " knowing, " of being objective or certain, of being mentally in control of one's thoughts, aware of the causes of one's thoughts, feelings, and actions, and of making decisions deliberately and rationally. Among the noteworthy discoveries of 20th century psychology was a challenge posed to this assumption of rationality. From the groundbreaking theorizing of Herbert Simon (1955) and the mind-boggling problems posed by Kahneman, Slovik, and Tversky (1982) to striking demonstrations of illusions of control (Wegner, 2002), the paucity of introspection (Nisbett and Wilson, 1977), and the automaticity of everyday thought (Bargh, 1997), psychologists have shown the frailties of the minds of their species. As psychologists have come to grips with the limits of the mind, there has been an increased interest in measuring aspects of thinking and feeling that may not be easily accessed or available to consciousness. Innovations in measurement have been undertaken with the purpose of bringing under scrutiny new forms of cogni-tion and emotion that were previously undiscovered and especially by asking if traditional concepts such as attitude and preference, belief and stereotype, self-concept and self-esteem can be rethought based on what the new measures reveal. These newer measures do not require introspection on the part of the subject. For many constructs this is considered a valuable, if not essential, feature of measurement; for others, avoiding introspection is greeted with suspicion and skepticism. For example, one approach to measuring math ability would be to ask " how good are you at math? " whereas an alternative approach is to infer math ability via a performance on a math skills test. The former requires introspection to assess the relevant construct, the latter does not. And yet, the latter is accepted

...read moreread less

1,095 citations

Cites background from "A Meta-Analysis on the Correlation ..."

...As already reviewed, IAT scores are predictive of self-report attitudes and the strength of that relationship is moderated by multiple factors (Hofmann et al., 2005; Nosek, 2005). Poehlman, Uhlmann, Greenwald, and Banaji (2004) recently compiled 61 studies with 86 individual effect sizes to show the predictive validity of the IAT when perceptual, judgment, and action processes were examined as criterion variables....
[...]
...The difference between the Hofmann et al. (2005) meta-analysis and the Nosek (2005) finding can be understood by noting the difference in domains examined....
[...]
...More recent research has shown that, in some cases, the IAT and self-report can be strongly related (Greenwald et al., 2003; Hofmann et al., 2005; Nosek et al., 2002; Nosek, 2005)....
[...]
...As already reviewed, IAT scores are predictive of self-report attitudes and the strength of that relationship is moderated by multiple factors (Hofmann et al., 2005; Nosek, 2005)....
[...]
...research has shown that, in some cases, the IAT and self-report can be strongly related (Greenwald et al., 2003; Hofmann et al., 2005; Nosek et al., 2002; Nosek, 2005)....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Measuring individual differences in implicit cognition: The implicit association test.

[...]

Anthony G. Greenwald¹, Debbie E. McGhee¹, Jordan L. K. Schwartz¹•Institutions (1)

University of Washington¹

01 Jun 1998-Journal of Personality and Social Psychology

TL;DR: An implicit association test (IAT) measures differential association of 2 target concepts with an attribute when instructions oblige highly associated categories to share a response key, and performance is faster than when less associated categories share a key.

...read moreread less

Abstract: An implicit association test (IAT) measures differential association of 2 target concepts with an attribute. The 2 concepts appear in a 2-choice task (e.g., flower vs. insect names), and the attribute in a 2nd task (e.g., pleasant vs. unpleasant words for an evaluation attribute). When instructions oblige highly associated categories (e.g., flower + pleasant) to share a response key, performance is faster than when less associated categories (e.g., insect + pleasant) share a key. This performance difference implicitly measures differential association of the 2 concepts with the attribute. In 3 experiments, the IAT was sensitive to (a) near-universal evaluative differences (e.g., flower vs. insect), (b) expected individual differences in evaluative associations (Japanese + pleasant vs. Korean + pleasant for Japanese vs. Korean subjects), and (c) consciously disavowed evaluative differences (Black + pleasant vs. White + pleasant for self-described unprejudiced White subjects).

...read moreread less

9,731 citations

"A Meta-Analysis on the Correlation ..." refers background or methods in this paper

...Second, the present meta-analysis was concerned with a particular implicit measure: the Implicit Association Test (Greenwald et al., 1998)....
[...]
...In contrast to the independence account, we found a small but significant positive mean population correlation of .24 between selfreported representations and representations assessed with the IAT (Greenwald et al., 1998)....
[...]
...24 between selfreported representations and representations assessed with the IAT (Greenwald et al., 1998)....
[...]
...As Greenwald, McGhee, and Schwartz (1998) noted, means and standard deviations of the IAT effect are somewhat higher when the compatible block is administered first than when the incompatible block is administered first....
[...]
...THE IMPLICIT ASSOCIATION TEST In the present meta-analysis, we focused on correlations revealed by a particular implicit measure: the Implicit Association Test (IAT), developed by Greenwald et al. (1998)....
[...]

Journal Article•DOI•

The file drawer problem and tolerance for null results

[...]

Robert Rosenthal¹•Institutions (1)

Harvard University¹

01 May 1979-Psychological Bulletin

TL;DR: Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed.

...read moreread less

Abstract: For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the "file drawer problem" is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results. Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed. (15 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)

...read moreread less

7,159 citations

"A Meta-Analysis on the Correlation ..." refers background in this paper

...This so-called file-drawer problem (Rosenthal, 1979) usually leads to an overestimation of effect sizes....
[...]

Journal Article•DOI•

Attitude-behavior relations: A theoretical analysis and review of empirical research.

[...]

Icek Ajzen, Martin Fishbein

01 Sep 1977-Psychological Bulletin

TL;DR: In this article, a review of available empirical research supports the contention that strong attitude-behavior relations can be obtained only under high correspondence between at least the target and action elements of the attitudinal and behavioral entities.

...read moreread less

Abstract: Research on the relation between attitude and behavior is examined in light of the correspondence between attitudinal and behavioral entities. Such entities are defined by their target, action, context, and time elements. A review of available empirical research supports the contention that strong attitude-behavior relations dre obtained only under high correspondence between at least the target and action elements of the attitudinal and behavioral entities. This conclusion is compared with the rather pessimistic assessment of the utility of the attitude concept found in much contemporary social psychological literature.

...read moreread less

6,756 citations

"A Meta-Analysis on the Correlation ..." refers background or result in this paper

...Furthermore, the conceptual correspondence of indicators (cf. Ajzen & Fishbein, 1977) may influence the implicitexplicit relationship, such that correlations may be reduced when the self-report measure is only indirectly rather than directly related to the representation assessed by the implicit…...
[...]
...…is based on previous research on attitude-behavior consistency, showing that the relationship between attitudes and behavior is usually stronger when the respective measures correspond than when they do not correspond with regard to the particular attitude object (Ajzen & Fishbein, 1977)....
[...]
...With regard to characteristics of self-report measures, we were interested in whether lack of conceptual correspondence reduces correlations between implicitly and explicitly assessed representations (cf. Ajzen & Fishbein, 1977)....
[...]
...These results are consistent with previous research on attitude-behavior consistency, showing that the relation between attitudes and behavior is higher when the two measures correspond than when they do not correspond with regard to the particular attitude object (Ajzen & Fishbein, 1977)....
[...]

Journal Article•DOI•

Implicit Social Cognition: Attitudes, Self-Esteem, and Stereotypes.

[...]

Anthony G. Greenwald¹, Mahzarin R. Banaji²•Institutions (2)

University of Washington¹, Yale University²

01 Jan 1995-Psychological Review

TL;DR: The present conclusion--that attitudes, self-esteem, and stereotypes have important implicit modes of operation--extends both the construct validity and predictive usefulness of these major theoretical constructs of social psychology.

...read moreread less

Abstract: Social behavior is ordinarily treated as being under conscious (if not always thoughtful) control. However, considerable evidence now supports the view that social behavior often operates in an implicit or unconscious fashion. The identifying feature of implicit cognition is that past experience influences judgment in a fashion not introspectively known by the actor. The present conclusion--that attitudes, self-esteem, and stereotypes have important implicit modes of operation--extends both the construct validity and predictive usefulness of these major theoretical constructs of social psychology. Methodologically, this review calls for increased use of indirect measures--which are imperative in studies of implicit cognition. The theorized ordinariness of implicit stereotyping is consistent with recent findings of discrimination by people who explicitly disavow prejudice. The finding that implicit cognitive effects are often reduced by focusing judges' attention on their judgment task provides a basis for evaluating applications (such as affirmative action) aimed at reducing such unintended discrimination.

...read moreread less

5,682 citations

"A Meta-Analysis on the Correlation ..." refers background in this paper

...As a final consideration, it should be mentioned that—from the very start of implicit social cognition research— reporting nonsignificant relationships between implicit and explicit measures has never been at odds with the most prominent theorizing in the field (e.g., Greenwald & Banaji, 1995)....
[...]
...In a similar vein, implicit-explicit relations may depend on people’s awareness of implicitly assessed representations (cf. Greenwald & Banaji, 1995)....
[...]
...As a final consideration, it should be mentioned that—from the very start of implicit social cognition research— reporting nonsignificant relationships between implicit and explicit measures has never been at odds with the most prominent theorizing in the field (e.g., Greenwald & Banaji, 1995 )....
[...]

Journal Article•DOI•

Understanding and using the Implicit Association Test: I. An improved scoring algorithm.

[...]

Anthony G. Greenwald¹, Brian A. Nosek, Mahzarin R. Banaji²•Institutions (2)

University of Washington¹, Harvard University²

01 Aug 2003-Journal of Personality and Social Psychology

TL;DR: The best-performing measure incorporates data from the IAT's practice trials, uses a metric that is calibrated by each respondent's latency variability, and includes a latency penalty for errors, and strongly outperforms the earlier (conventional) procedure.

...read moreread less

Abstract: In reporting Implicit Association Test (IAT) results, researchers have most often used scoring conventions described in the first publication of the IAT (A.G. Greenwald, D.E. McGhee, & J.L.K. Schwartz, 1998). Demonstration IATs available on the Internet have produced large data sets that were used in the current article to evaluate alternative scoring procedures. Candidate new algorithms were examined in terms of their (a) correlations with parallel self-report measures, (b) resistance to an artifact associated with speed of responding, (c) internal consistency, (d) sensitivity to known influences on IAT measures, and (e) resistance to known procedural influences. The best-performing measure incorporates data from the IAT's practice trials, uses a metric that is calibrated by each respondent's latency variability, and includes a latency penalty for errors. This new algorithm strongly outperforms the earlier (conventional) procedure.

...read moreread less

5,049 citations

"A Meta-Analysis on the Correlation ..." refers background in this paper

...The difference between the mean response latencies for associationcompatible and association-incompatible assignments is usually interpreted as an indicator of the relative associative strength between the two pairs of concepts (for a discussion of different scoring algorithms, see Greenwald, Nosek, & Banaji, 2003 )....
[...]