The Gender Citation Gap in International Relations

TL;DR: This article investigated the extent to which citation and publication patterns differ between men and women in the international relations literature and found that women are systematically cited less than men after controlling for a large number of variables.

Abstract: This article investigates the extent to which citation and publication patterns differ between men and women in the international relations (IR) literature. Using data from the Teaching, Research, and International Policy project on peer-reviewed publications between 1980 and 2006, we show that women are systematically cited less than men after controlling for a large number of variables including year of publication, venue of publication, substantive focus, theoretical perspective, methodology, tenure status, and institutional affiliation. These results are robust to a variety of modeling choices. We then turn to network analysis to investigate the extent to which the gender of an article's author affects that article's relative centrality in the network of citations between papers in our sample. Articles authored by women are systematically less central than articles authored by men, all else equal. This is likely because (1) women tend to cite themselves less than men, and (2) men (who make up a disproportionate share of IR scholars) tend to cite men more than women. This is the first study in political science to reveal significant gender differences in citation patterns and is especially meaningful because citation counts are increasingly used as a key measure of research's quality and impact.

Figures

FIGURE 3. Coefficient on all female at different levels of article age

TABLE 6. Dyadic citations by gender, percentages represent the mean for all articles of each type

TABLE 3. Predicted vs. actual citation counts

TABLE 5. T-test comparing self-citations among author gender

FIGURE 2. Visualization of citation network

TABLE 1. Citations by gender and decade

FIGURE 1. Estimated citation counts with simulated confidence interval

Full text

The Gender Citation Gap in
International Relations
Daniel Maliniak, Ryan Powers, and
Barbara F+ Walter
Abstract This article investigates the extent to which citation and publication
patterns differ between men and women in the international relations ~IR! literature+
Using data from the Teaching, Research, and International Policy project on peer-
reviewed publications between 1980 and 2006, we show that women are systemati-
cally cited less than men after controlling for a large number of variables including
year of publication, venue of publication, substantive focus, theoretical perspective,
methodology, tenure status, and institutional affiliation+ These results are robust to a
variety of modeling choices+ We then turn to network analysis to investigate the extent
to which the gender of an article’s author affects that article’s relative centrality in
the network of citations between papers in our sample+ Articles authored by women
are systematically less central than articles authored by men, all else equal+ This is
likely because ~1! women tend to cite themselves less than men, and ~2! men ~who
make up a disproportionate share of IR scholars! tend to cite men more than women+
This is the first study in political science to reveal significant gender differences in
citation patterns and is especially meaningful because citation counts are increas-
ingly used as a key measure of research’s quality and impact+
To what extent—if any—are articles in international relations ~IR! cited differ-
ently depending on the gender of the publication’s author?
1
We address this ques-
tion by analyzing citation patterns in the IR literature, using data from the Teaching,
We thank Karen Alter, Tim Büthe, Peter Gourevitch, Zoltan Hajnal, Kelly Kadera, Bob Keohane,
David Lake, Lisa Martin, Rose McDermott, Andrew Moravcsik, Sara Mitchell, Maya Oren, Maggie
Peters, Jaime Settle, and two anonymous reviewers for invaluable comments+ We also thank the Insti-
tute for the Theory and Practice of International Relations at the College of William and Mary for
providing research assistance+ Powers acknowledges suppor t from the National Science Foundation
Graduate Fellowship under Grant No+ DGE-0718123+ Supplementary materials for this article are avail-
able at http:00dx+doi+org 010+10170S00208183000209+
1+ In this article we use the term gender rather than sex to refer to our male0female variable+ We
realize that the two terms are not synonymous, nor is gender dichotomous+ We prefer to use the term
gender because the coding of the author of a publication is based heavily on the pronouns an author
uses to identify him- or herself+ The result, however, is that we are unable to include a category for
transgendered scholars+ We regret this+ Still, because of the fact that transgendered individuals make
up such a small proportion of the total population of IR scholars, any analysis of citation patterns of
ar ticles authored by transgendered individuals would be unreliable at best+
International Organization 67, Fall 2013, pp+ 889–922
© 2013 by The IO Foundation+
doi:10+10170S0020818313000209
https://doi.org/10.1017/S0020818313000209 Published online by Cambridge University Press

Research, and International Policy ~TRIP! project+
2
We use the TRIP project’s
journal article database, which catalogues ar ticles published in the top twelve
peer-reviewed IR journals over the period 1980–2006+
3
The TRIP project has coded
approximately 3,000 articles on twenty-six different substantive and demo-
graphic variables+ Using these data, we demonstrate the existence of a persis-
tent gender gap in citation counts: articles written by women are consistently
cited less than articles written by men+ This is especially true of women who
have not yet earned tenure+ Observable differences between male and female IR
scholars—including productivity, institutional affiliation, publication venue, or
epistemology—cannot account for this gap + We explore the gender citation gap
further through an analysis of men and women in the network formed by cita-
tions between the articles in our sample+ Using a dyadic citation data set built
from the TRIP data and the Thomson Reuters Web of Knowledge ~WOK! data-
base, we show that women are also more concentrated on the periphery of the
IR network, where their work is cited less often by authors of the most heavily
cited work+
Taken together, these findings offer robust evidence for a gender gap in citation
counts in IR+ This is a cause for concern+ If women in IR are systematically cited
less than men in ways that do not appear to be associated with observable differ-
ences in their scholarship, and if citation counts continue to be used as a key mea-
sure of research impact, then women will be disadvantaged in tenure, promotion,
and salary decisions+ This article reveals this differential pattern of citation counts
and offers two potential explanations for it+ Our hope is that by identifying the
gender gap in citations and then identifying potential reasons for it, we can begin
to address and rectify it+
Gender and the IR Literature
The status of women in academia and political science has long been a topic of
discussion and concern at colleges and universities across the United States+
4
Women continue to be underrepresented on political science faculty, at confer-
ences, and in peer-reviewed publications+
5
Today, women are earning PhDs in polit-
2+ The TRIP Project is managed by the Institute for the Theory and Practice of International Rela-
tions at the College of William and Mary+ It gathers data to enable scholars to better understand
the development and current state of the discipline of IR and to what extent IR research informs
or is informed by the international policymaking process+ For more information, see ^http:00
ir theoryandpractice+wm+edu0projects0trip0&, accessed 8 May 2013+
3+ The TRIP project identified the “top” journals on the basis of their impact rating+ The database
itself contains bibliographic data on all articles up to 2010, but articles have been systematically coded
and arbitrated only through 2006+
4+ See Schuck 1969; Finifter 1973; Gruberg and Sapiro 1979; and Charles and Grusky 2005+
5+ See American Political Science Association ~ APSA! 1992; and Gruberg and Sapiro 1979+
890 International Organization
https://doi.org/10.1017/S0020818313000209 Published online by Cambridge University Press

ical science in record numbers but are failing to earn tenure in proportion to these
numbers+
6
The disproportionately low number of female faculty is potentially problematic
given that the majority of undergraduate and graduate students in the United States
are female+ In 2010, women represented 57 percent of all students in four-year
colleges and universities and 52 percent of all students in PhD programs+
7
It is
expected that by 2020, women will represent 60 percent of all students enrolled in
four-year degree-granting programs+
8
If colleges and universities are serious about
increasing the number of tenured female faculty, especially at large research uni-
versities, they will need to understand and address why women are failing to move
up the ladder at the same rate as their male counterparts+ Without this, college
faculty will continue to be dominated by men even as their student bodies are
increasingly dominated by women+
The number of women in academia, as well as their influence, will depend in
part on how often their research gets published and whether this research is then
cited by other scholars+ Decisions about tenure, promotion, and salary, especially
at research universities, take into account not just publications but impact, which
tends to be partly measured by citation counts+ Although hard data do not exist,
anecdotal evidence suggests that academia is weighting the influence of citation
counts more over time+ As one long-standing member of a top-ten political sci-
ence department observed:
In thirty years +++ citation counts used to be a non-issue in hiring and promo-
tion in my department+ Until about three years ago, the issue had come up
literally once+ I remember because it seemed so odd + Now, the social science
citation index and Google Scholar citation counts are regularly raised and
discussed except for starting assistant professors+
9
If departments are increasingly relying on citation counts to measure impact, then
at the very least they should know whether any systematic bias exists in such counts+
Work in a number of other scholarly fields shows a consistent trend in gender-
citation patterns between men and women+ In biology,
10
biochemistry,
11
ecolo-
gy,
12
library and information science,
13
and in general studies of the natural sciences,
men tend to be more productive in terms of quantity, while women tend to pro-
duce higher-quality work, at least as measured by citations+
14
6+ See Ginther 2004; APSA 2004 and 2007; and Hesli, Lee, and Mitchell 2012+
7+ US Department of Education 2011+
8+ Ibid+
9+ Büthe and Aggarwal 2013; not-for-attribution interview with a full professor+
10+ Sonnert 1995+
11+ Long 1992+
12+ Symonds et al+ 2006+
13+ Peñas and Willett 2006+
14+ Sonnert and Holton 1996+ However, Slyder et al+ ~2011! show no difference for ar ticles in the
field of forestry and geography for scholars from ten universities+ They argue that this may be the
The Gender Citation Gap in International Relations 891
https://doi.org/10.1017/S0020818313000209 Published online by Cambridge University Press

Do the same trends exist in IR? Copenheaver and colleagues argue that “gender
differences in citation rates appear to be discipline specific, so identifying whether
a difference exists within a discipline is an important factor for making fair and
equitable decisions regarding the evaluation and promotion of female and male
researchers+”
15
Mitchell and others provide limited evidence of a gender citation
gap in IR in their bibliographic analysis of articles published in International Stud-
ies Quarterly and International Studies Perspectives in 2005+
16
They find that arti-
cles published by men are less likely to cite work by women than are articles
published by women+ On the other hand, Østby and colleagues find that gender is
not a significant determinant of publication in their analysis of submission and
publication rates at the Journal of Peace Research between 1983 and 2010+
17
Given
the conflicting findings of some of these more limited data sets, more systematic
tests are needed to come to a general understanding of how author gender affects
the eventual influence of a given ar ticle+
To test if a citation gap exists in IR, we looked at more than 3,000 ar ticles
published between 1980 and 2006 in twelve influential peer-reviewed IR journals+
Our findings suggest that articles authored by women are cited less on average
than those authored by men+ They also suggest that this gap disappears as soon as
women coauthor with men+
We begin our investigation with a very basic analysis of the TRIP data set+ We
coded all articles in the TRIP database for the gender of the author~s!, grouping
them into three categories: those written by one or more male authors, those writ-
ten by one or more female authors, and those written by at least one author of
each gender+
18
A simple cross-tabulation suggests that author gender plays a sig-
nificant role in determining the number of citations a given article garners after
publication+ Table 1 displays the average number of citations a given article received
based on the gender of the author~s!+ Articles authored by men garnered an aver-
age of 4+8 more citations than those authored by women over the period 1980–
2006+ Given that the average number of citations per article during this time was
about twenty-five, this is quite a significant difference+
There are a number of possible explanations for why this gap exists+ First, men
and women tend to work at different institutions+ According to the 2006 TRIP
result of frequent coauthorship among men and women, a result our analysis supports+ In the field of
dendrochronology, Copenheaver, Goldbeck, and Cherubini ~2010! find no difference between the men
and women as first authors, but they also point to the role of coauthorship+
15+ Copenheaver, Goldbeck, and Cherubini 2010, 128+
16+ Mitchell, Lange, and Brus forthcoming+
17+ Østby et al+ forthcoming+
18+ This coding is based first on the pronouns that the individual authors use to refer to themselves
in articles or on their department website+ If no pronoun is used by the individual, we looked for
photographs of the individual on their department or personal website+ Finally, if no pronoun usage or
photo was available, we coded based on the most common gender associated with the individual+ In
cases where a name was not overwhelmingly associated with one gender or another, we left the gender
of the article as missing data+
892 International Organization
https://doi.org/10.1017/S0020818313000209 Published online by Cambridge University Press

survey of IR scholars, men are more likely than women to be employed by PhD–
granting institutions, while women are more likely than men to be employed by
liberal arts schools+
19
The focus on teaching at liberal arts colleges may lead fac-
ulty to produce less research or have fewer opportunities to publicize their work at
conferences and0or at seminars at other colleges or universities+ It may also be the
case that the research produced by faculty at liberal arts colleges could be differ-
entially valued by the profession because of their institutional affiliations+ Like-
wise, tenure requirements and a focus on research may generate different incentives
to engage in academic debates and produce research+ Because of the gender dis-
parity in placement at liberal arts colleges, we might expect institutional affiliation
to partially account for the gender divide in citation counts+
Second, women may publish less in the early years of their careers as a result
of their need to take parental leave+ This may not affect productivity over the long
term, but if citations depend in part on building name recognition, then fewer pub-
lications early in one’s career could translate into fewer citations over time+ Symonds
and colleagues find that discrepancies between men and women early in their careers
can lead to differences in citation rates throughout their time as scholars+
20
Taking
a temporary leave from research in the first part of one’s career, therefore, may
have lasting effects+
Third, the norms of coauthorship have changed over time and differentially across
genders+ Fisher and others show in their analysis of coauthorship in three political
science journals
21
that while coauthorship across these journals has increased, it
19+ The TRIP project has conducted more recent surveys, but we use the 2006 numbers in this
study because that survey is coincident with the last year of publications included in our sample+
20+ Symonds et al+ 2006+
21+ American Political Science Review, American Journal of Political Science, and Journal of
Politics.
TABLE 1
. Citations by gender and decade
1980s 1990s 2000s
Mean
Standard
deviation Median Mean
Standard
deviation Median Mean
Standard
deviation Median
all male 17+64 51+63 5 34+39 62+80 14 22+17 39+56 13
all female 10+47 11+64 6 28+99 41+43 14 19+84 21+46 15
mixed gender 18+29 25+36 10 32+44 40+26 22 30+86 37+91 16
Total 17+18 48+84 6 33+65 59+06 14 22+92 37+58 13
Observations 771 832 686
The Gender Citation Gap in International Relations 893
https://doi.org/10.1017/S0020818313000209 Published online by Cambridge University Press

Frequently asked questions

Q1. What have the authors contributed in "The gender citation gap in international relations" ?

This article investigates the extent to which citation and publication patterns differ between men and women in the international relations ~IR ! Using data from the Teaching, Research, and International Policy project on peerreviewed publications between 1980 and 2006, the authors show that women are systematically cited less than men after controlling for a large number of variables including year of publication, venue of publication, substantive focus, theoretical perspective, methodology, tenure status, and institutional affiliation+ The authors then turn to network analysis to investigate the extent to which the gender of an article ’ s author affects that article ’ s relative centrality in the network of citations between papers in their sample+ This is the first study in political science to reveal significant gender differences in citation patterns and is especially meaningful because citation counts are increasingly used as a key measure of research ’ s quality and impact+ Cited differently depending on the gender of the publication ’ s author ? The authors address this question by analyzing citation patterns in the IR literature, using data from the Teaching, they thank Karen Alter, Tim Büthe, Peter Gourevitch, Zoltan Hajnal, Kelly Kadera, Bob Keohane, David Lake, Lisa Martin, Rose McDermott, Andrew Moravcsik, Sara Mitchell, Maya Oren, Maggie Peters, Jaime Settle, and two anonymous reviewers for invaluable comments+ Supplementary materials for this article are available at http:00dx+doi+org010+10170S00208183000209+ In this article the authors use the term gender rather than sex to refer to their male0female variable+ The authors prefer to use the term gender because the coding of the author of a publication is based heavily on the pronouns an author uses to identify himor herself+ Still, because of the fact that transgendered individuals make up such a small proportion of the total population of IR scholars, any analysis of citation patterns of articles authored by transgendered individuals would be unreliable at best+ The authors use the TRIP project ’ s journal article database, which catalogues articles published in the top twelve peer-reviewed IR journals over the period 1980–2006+ The TRIP project has coded approximately 3,000 articles on twenty-six different substantive and demographic variables+ Using these data, the authors demonstrate the existence of a persistent gender gap in citation counts: articles written by women are consistently cited less than articles written by men+ The authors explore the gender citation gap further through an analysis of men and women in the network formed by citations between the articles in their sample+ Database, the authors show that women are also more concentrated on the periphery of the IR network, where their work is cited less often by authors of the most heavily cited work+ This article reveals this differential pattern of citation counts and offers two potential explanations for it+ The TRIP Project is managed by the Institute for the Theory and Practice of International Relations at the College of William and Mary+ 3+ The TRIP project identified the “ top ” journals on the basis of their impact rating+ The database itself contains bibliographic data on all articles up to 2010, but articles have been systematically coded and arbitrated only through 2006+ The number of women in academia, as well as their influence, will depend in part on how often their research gets published and whether this research is then cited by other scholars+ Given the conflicting findings of some of these more limited data sets, more systematic tests are needed to come to a general understanding of how author gender affects the eventual influence of a given article+ To test if a citation gap exists in IR, the authors looked at more than 3,000 articles published between 1980 and 2006 in twelve influential peer-reviewed IR journals+ The authors coded all articles in the TRIP database for the gender of the author~s !, grouping them into three categories: those written by one or more male authors, those written by one or more female authors, and those written by at least one author of each gender+ A simple cross-tabulation suggests that author gender plays a significant role in determining the number of citations a given article garners after publication+ Table 1 displays the average number of citations a given article received based on the gender of the author~s ! + Given that the average number of citations per article during this time was about twenty-five, this is quite a significant difference+ According to the 2006 TRIP result of frequent coauthorship among men and women, a result their analysis supports+ Find no difference between the men and women as first authors, but they also point to the role of coauthorship+ 18+ This coding is based first on the pronouns that the individual authors use to refer to themselves in articles or on their department website+ In cases where a name was not overwhelmingly associated with one gender or another, the authors left the gender of the article as missing data+ Third, the norms of coauthorship have changed over time and differentially across genders+ The TRIP project has conducted more recent surveys, but the authors use the 2006 numbers in this study because that survey is coincident with the last year of publications included in their sample+ Their hope is that by identifying the gender gap in citations and then identifying potential reasons for it, the authors can begin to address and rectify it+ The disproportionately low number of female faculty is potentially problematic given that the majority of undergraduate and graduate students in the United States are female+ Although hard data do not exist, anecdotal evidence suggests that academia is weighting the influence of citation counts more over time+ Their findings suggest that articles authored by women are cited less on average than those authored by men+ They also suggest that this gap disappears as soon as women coauthor with men+ 

Q2. What topics are more likely to be written by women?

Women are more likely to write articles on human rights, comparative foreign policy, health, international law, and the environment+ 

Q3. How many citations are predicted for each article in the TRIP data set?

Using the career model from their analysis, the authors calculate the predicted number of citations for each article in their data set authored by women only or having at least one author from each gender+ 

Q4. Why is IR a more “masculinized” subfield?

It is possible, for example, that IR is a more “masculinized” subfield because of its connections with the foreign policy world or because a greater proportion of IR scholars are male+ 

Q5. How many years of exposure to possible citations did the authors exclude?

the authors exclude articles published later than 2006 so that each article has at least five years of exposure to possible citations by the time the authors gathered the counts+ 

Q6. what is a dummy variable for a postpositivist article?

When the authors include a dummy variable for whether the article is postpositivist or nonpositivist, as seen in the epistemology model ~Model 4!, positivist articles receive relatively more cites than similar non- or postpositivist articles+ 

Q7. How many articles are cited by at least one other article in the largest cluster?

Using the network of citations produced by the twelve journals in the TRIP article-coding database, the authors calculate the “authority” score for all articles cited by at least one other article in the largest cluster of articles+ 

Q8. How many articles have been systematically coded and arbitrated?

The database itself contains bibliographic data on all articles up to 2010, but articles have been systematically coded and arbitrated only through 2006+4+ 

Q9. How many citations do the authors subtract from the total number of articles?

To address this, the authors not only subtract all self-citations from the total number of citations an article receives, but the authors also subtract an additional 3+65 citations for each self-cite+56 

Q10. How do you measure the quality of a scholar’s work?

Citations are one of the chief metrics used in academia to evaluate a scholar’s performance and influence, and to distribute resources, including salary+30