1
Consensus on consensus: a synthesis
of consensus estimates on human-
caused global warming
John Cook
1,2,3
, Naomi Oreskes
4
, Peter T. Doran
5
, William R. L. Anderegg
6,7
, Bart Verheggen
8
,
Ed W. Maibach
9
, J. Stuart Carlton
10
, Stephan Lewandowsky
11,2
, Andrew G. Skuce
13
, Sarah A.
Green
12
, Dana Nuccitelli
3
, Peter Jacobs
9
, Mark Richardson
14
, Bärbel Winkler
3
, Rob Painting
3
,
Ken Rice
15
1 Global Change Institute, University of Queensland, Australia
2 School of Psychology, University of Western Australia, Australia
3 Skeptical Science, Brisbane, Queensland, Australia
4 Harvard University, USA
5 Geology and Geophysics, Louisiana State University, USA
6 Department of Biology, University of Utah, USA
7 Princeton Environmental Institute, Princeton University, USA
8 Amsterdam University College, The Netherlands
9 Department of Environmental Science and Policy, George Mason University, USA
10 Texas Sea Grant College Program, Texas A&M University, College Station, TX USA
11 University of Bristol, United Kingdom
12 Department of Chemistry, Michigan Technological University, USA
13 Salt Spring Consulting Ltd, Salt Spring Island, BC, Canada
14 Jet Propulsion Lab, California Institute of Technology, Pasadena, USA
15 Institute for Astronomy, University of Edinburgh, Edinburgh, UK
Abstract
The consensus that humans are causing most of recent global warming is shared by 90-98% of
publishing climate scientists according to six independent studies by co-authors of this paper.
Cook et al (2013) estimated a 97% consensus based on 11,944 abstracts of research papers, of
which 4,014 took a position on the cause of recent global warming. This 97% result has been
criticised for being both too high (Tol 2015) and too low (Powell 2015). In some cases, Tol
assumes that when the cause of global warming is not explicitly stated ("no position"), this
represents non-endorsement, while Powell assumes the opposite. Neither assumption is robust:
as argued by Powell, Tol’s approach would reject the consensus on well-established theories
such as plate tectonics. On the other hand, Cook et al surveyed authors of the studies
considered and some full papers rejected the consensus even when their abstracts were
classified as "no position", contradicting Powell's assumption. The author survey (N=2,412
papers) also resulted in a 97% consensus. Tol further asserts that Cook et al is “at the high end
in the consensus literature” by comparing its results with surveys of non-experts such as
Page 1 of 27 CONFIDENTIAL - AUTHOR SUBMITTED MANUSCRIPT ERL-101399.R1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
2
economic geologists and a self-selected group of those who reject the consensus with no
requirement for relevant expertise. Tol's selected surveys show that consensus increases with
relevant expertise: we conclude that the finding of 97% consensus in published climate research
is robust and consistent with other surveys of climate scientists and peer-reviewed studies.
1. Introduction
Scientists overwhelmingly agree that humans are causing global warming. The consensus
position is articulated by the Intergovernmental Panel on Climate Change (IPCC) statement that
“human influence has been the dominant cause of the observed warming since the mid-20th
century” (Qin et al 2014, p. 17). The National Academies of Science from 80 countries have
issued statements endorsing the consensus position (Table S2). Nevertheless, the existence of
the consensus continues to be questioned. Here we summarize studies that quantify expert
views and examine common flaws in criticisms of consensus estimates. In particular, we are
responding to comments by Tol (2015) and Powell (2015) on Cook et al (2013, referred to as
C13). We show that contrary to Tol's claim that the results of C13 differ from earlier studies, the
consensus of experts is robust across all the studies conducted by coauthors of this
correspondence.
Tol's erroneous conclusions stem from conflating the opinions of non-experts with those of
publishing climate scientists and assuming that lack of affirmation equals dissent. Powell, in
contrast, assumes that lack of dissent equals affirmation, so instead estimates a consensus of
99.9%. This assumption is not obviously invalid, but it is not supported by the data. C13
compared abstract ratings with the ratings of full papers by the papers’ authors and found that
some papers that reject the consensus have abstracts classified as taking no position, in
contradiction to Powell’s assumption. The self-rating data support a literature consensus of
97%, an important aspect of C13 that most criticisms fail to consider.
A detailed technical response to Tol is provided in (S1) and to Powell in (S2). The remainder of
this paper shows that a high level of scientific consensus, in agreement with our results, is a
robust finding in the scientific literature. This is used to illustrate and address the issues raised
by Tol and Powell that are relevant to our main conclusion.
2. Assessing Expert Consensus
Efforts to measure scientific consensus need to identify a relevant and representative population
of experts, assess their professional opinion in an appropriate manner, and avoid distortions
from ambiguous elements in the sample. Approaches that have been employed to assess
expert views on anthropogenic global warming (AGW) include analysing peer-reviewed climate
papers (Oreskes 2004; C13), surveying members of the relevant scientific community (Bray and
von Storch 2007; Doran and Zimmerman 2009; Bray, 2010;
Rosenberg et al 2010; Farnsworth
and Lichter 2012; Verheggen et al 2014; Stenhouse et al 2014; Carlton et al 2015), compiling
public statements by scientists (Anderegg et al 2010), and mathematical analyses of citation
Page 2 of 27CONFIDENTIAL - AUTHOR SUBMITTED MANUSCRIPT ERL-101399.R1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
3
patterns (Shwed and Bearman 2010). We define domain experts as scientists who have
published peer-reviewed climate research. Consensus estimates for these experts are listed in
Table 1, with the range of estimates resulting primarily from differences in selection of the expert
pool, the definition of what entails the consensus position and differences in treatment of no
position responses/papers.
Table 1: Estimates of Consensus on Human-Caused Global Warming among Climate Experts
Source
Year(s)
Total sample
(including non-publishing climatologists)
Sub-sample of publishing climatologists
Definition of consensus
Consensus
N
Description
Consensus
N
Description
Gallup 1991
1991
66%
400
AMS/AGU members
67%
97
Currently Performing
Research in Area
Global Warming
Human-induced greenhouse warming
is now occurring
Oreskes 2004
1993-
2003
100%
928
Peer-reviewed
papers on “global
climate change”
Earth’s climate is being affected by
human activities
[M]ost of the observed warming over
the last 50 years is likely to have
been due to the increase in
greenhouse gas concentrations
Bray and von
Storch
2007
1996
40%
539
1997: 5 countries
(US, Canada,
Germany, Denmark,
Italy)
Climate change is mostly the result of
anthropogenic causes
Bray and von
Storch 2007
2003
53%
530
2003: 30 countries
Climate change is mostly the result of
anthropogenic causes
Doran and
Zimmerman
2009
2009
82%
3146
Earth scientists
97%
77
Climatologists who
are active publishers
of climate research
Human activity is a significant
contributing factor in changing mean
global temperatures
Anderegg et al
2010
2010
66%
1372
Signatories of public
statements about
climate change
97%
200
Top 200 most
published authors (of
climate-related
papers)
Anthropogenic greenhouse gases
have been responsible for “most” of
the “unequivocal” warming of the
Earth’s average global temperature
over the second half of the 20th
century
Bray 2010
2008
83.5%
370
Authors of climate
journals, authors from
Oreskes 2004
sample, scientists
from relevant
institutes (NCAR,
AMS, etc)
How convinced are you that most of
recent or near future climate change
is, or will be, a result of
anthropogenic causes?
Rosenberg et
al 2010
2005
88.5%
433
U.S. climate
scientists authoring
articles in scientific
journals that highlight
climate change
research
Scientists can say with great certainty
that human activities are accelerating
global warming
Farnsworth
and Lichter
2012
2007
84%
489
AMS/AGU members
In your opinion, is human-induced
greenhouse warming now occurring?
Cook et al
2013
1991-
2011
97.1%
97.2%
4104
abstracts
1381 self-
rated
Published peer-
reviewed papers on
“global climate
change” or “global
warming” that state a
position on AGW
1. Explicitly states that humans are
the primary cause of recent global
warming
2. Explicitly states humans are
causing global warming
3. Implies humans are causing global
Page 3 of 27 CONFIDENTIAL - AUTHOR SUBMITTED MANUSCRIPT ERL-101399.R1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
4
papers
warming.
4a. Does not address or mention the
cause of global warming
4b. Expresses position that human’s
role on recent global warming is
uncertain/undefined
5. Implies humans have had a
minimal impact on global warming
without saying so explicitly
6. Explicitly minimizes or rejects that
humans are causing global warming
7. Explicitly states that humans are
causing less than half of global
warming
Stenhouse et
al 2014
2013
73%
1821
AMS members
93%
124
Self-reported
expertise is climate
science, publication
focus is mostly
climate
Humans are a contributing cause of
global warming over the past 150
years
Verheggen et
al 2014
2012
84%
86%
1461 (Q1)
1682 (Q3)
89% (Q1)
91% (Q3)
623 (Q1)
729 (Q3)
Published more than
10 climate-related
papers (self-reported)
Q1. Over half of global warming since
the mid-20th century can be
attributed to human-induced
increases in atmospheric GHG
concentrations
Q3. Greenhouse gases have made the
strongest or tied-strongest contribution
(out of different factors considered) to the
reported global warming of 0.8° C
since preindustrial times
Pew Research
Center 2015
2015
87%
3748
AAAS members
93%
132
Working Ph.D Earth
scientist
Climate change is mostly due to
human activity
Powell 2015
2013-
2014
99.9%
69,406
Published peer-
reviewed papers on
“global climate
change” or “global
warming”
Doesn’t explicitly reject AGW in
abstract
Carlton et al
2015
2014
91.9%
698
Survey of
biophysical scientists
across disciplines at
universities in the
Big 10 Conference
96.7%
306
Those who indicated
that “The majority of
my research
concerns climate
change or the
impacts of climate
change.”
Response to the following: (1) When
compared with pre-1800’s levels, do
you think that mean global
temperatures have generally risen,
fallen, or remained relatively
constant, and (2) Do you think human
activity is a signi cant contributing
factor in changing mean global
temperatures?
The studies in Table 1 have taken various approaches to selecting and querying pools of
experts. Oreskes (2004) identified expressions of views on AGW in the form of peer-reviewed
papers on “global climate change”. This analysis found no papers rejecting AGW in a sample of
928 papers published from 1993 to 2003, that is, 100% consensus among papers stating a
position on AGW.
Following a similar methodology, C13 analysed the abstracts of 11,944 peer-reviewed papers
published between 1991 and 2011 that matched the search terms “global climate change” or
“global warming” in the ISI Web of Science search engine. Among the 4,014 abstracts stating a
position on human-caused global warming, 97.1% were judged as having implicitly or explicitly
endorsed the consensus. In addition, the authors of the papers were invited to rate their own
papers (N=2,142), based on the full paper, not just the abstract.. Amongst 1,381 papers self-
Page 4 of 27CONFIDENTIAL - AUTHOR SUBMITTED MANUSCRIPT ERL-101399.R1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
5
rated by their authors as stating a position on human-caused global warming, 97.2% endorsed
the consensus.
Shwed and Bearman (2010) employed citation analysis of 9,432 papers on global warming and
climate published from 1975 to 2008. Unlike surveys or classifications of abstracts, this method
is entirely mathematical and blind to the content of the literature being examined. By
determining the modularity of citation networks, they concluded, “Our results reject the claim of
inconclusive science on climate change and identify the emergence of consensus earlier than
previously thought” (p. 831). Although this method does not produce a numerical consensus
value, it independently demonstrates the same level of scientific consensus on AGW as exists
for the fact that smoking causes cancer.
Anderegg et al (2010) identified climate experts as those who had authored at least 20 climate-
related publications and chose their sample from those who had signed public statements
regarding climate change. By combining published scientific papers and public statements,
Anderegg et al determined that 97 to 98% of the 200 most published climate scientists endorsed
the IPCC conclusions on AGW.
Other studies have directly queried scientists, typically choosing a sample of scientists and
identifying subsamples of those who self-identify as climate scientists or actively publish in the
field. Doran and Zimmerman (2009) surveyed 3,146 Earth scientists, asking whether “human
activity is a significant contributing factor in changing mean global temperatures,” and
subsampled those who were actively publishing climate scientists. Overall, they found that 82%
of Earth scientists indicated agreement, while among the subset with greatest expertise in
climate science, the agreement was 97.4%.
Bray and von Storch (2007) and Bray (2010) repeatedly surveyed different populations of
climate scientists in 1996, 2003 and 2008. The questions did not specify a time period for
climate change (indeed, in 2008, 36% of the participants defined the term "climate change" to
refer to “changes in climate at any time for whatever reason”). Therefore the reported
consensus estimates of 40% (1996) and 53% (2003) (which included participants not stating a
view on AGW) suffered from both poor control of expert selection and ambiguous questions.
Their 2008 study, finding 83% agreement, had a more robust sample selection and a more
specific definition of the consensus position on attribution.
Verheggen et al (2014) surveyed 1,868 scientists, drawn in part from a public repository of
climate scientists (the same source as was used by Anderegg et al), and from scientists listed in
C13, supplemented by authors of recent climate-related articles and with particular effort
expended to include signatories of public statements critical of mainstream climate science.
85% of all respondents (which included a likely overrepresentation of contrarian non-scientists)
who stated a position agreed that anthropogenic greenhouse gases (GHGs) are the dominant
driver of recent global warming. Among respondents who reported having authored more than
10 peer-reviewed climate-related publications, around 90% agreed that greenhouse gas
emissions were causing most of global warming.
Page 5 of 27 CONFIDENTIAL - AUTHOR SUBMITTED MANUSCRIPT ERL-101399.R1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60