❉✉r❤❛♠ ❘❡s❡❛r❝❤ ❖♥❧✐♥❡
❉❡♣♦s✐t❡❞ ✐♥ ❉❘❖✿
✷✹ ❋❡❜r✉❛r② ✷✵✶✼
❱❡rs✐♦♥ ♦❢ ❛tt❛❝❤❡❞ ✜❧❡✿
❆❝❝❡♣t❡❞ ❱❡rs✐♦♥
P❡❡r✲r❡✈✐❡✇ st❛t✉s ♦❢ ❛tt❛❝❤❡❞ ✜❧❡✿
P❡❡r✲r❡✈✐❡✇❡❞
❈✐t❛t✐♦♥ ❢♦r ♣✉❜❧✐s❤❡❞ ✐t❡♠✿
▲❡❥❛❡❣❤❡r❡✱ ❑✉rt ❛♥❞ ❇✐❤❧♠❛②❡r✱ ●✉st❛✈ ❛♥❞ ❇❥♦❡r❦♠❛♥✱ ❚♦r❜❥♦❡r♥ ❛♥❞ ❇❧❛❤❛✱ P❡t❡r ❛♥❞ ❇❧✉❡❣❡❧✱ ❙t❡❢❛♥ ❛♥❞
❇❧✉♠✱ ❱♦❧❦❡r ❛♥❞ ❈❛❧✐st❡✱ ❉❛♠✐❡♥ ❛♥❞ ❈❛st❡❧❧✐✱ ■✈❛♥♦ ❊✳ ❛♥❞ ❈❧❛r❦✱ ❙t❡✇❛rt ❏✳ ❛♥❞ ❉❛❧ ❈♦rs♦✱ ❆♥❞r❡❛ ❛♥❞ ❞❡
●✐r♦♥❝♦❧✐✱ ❙t❡❢❛♥♦ ❛♥❞ ❉❡✉ts❝❤✱ ❚❤✐❡rr② ❛♥❞ ❉❡✇❤✉rst✱ ❏♦❤♥ ❑❛② ❛♥❞ ❉✐ ▼❛r❝♦✱ ■❣♦r ❛♥❞ ❉r❛①❧✱ ❈❧❛✉❞✐❛ ❛♥❞
❉✉❧❛❦✱ ▼❛r❝✐♥ ❛♥❞ ❊r✐❦ss♦♥✱ ❖❧❧❡ ❛♥❞ ❋❧♦r❡s✲▲✐✈❛s✱ ❏♦s❡ ❆✳ ❛♥❞ ●❛rr✐t②✱ ❑❡✈✐♥ ❋✳ ❛♥❞ ●❡♥♦✈❡s❡✱ ▲✉✐❣✐ ❛♥❞
●✐❛♥♥♦③③✐✱ P❛♦❧♦ ❛♥❞ ●✐❛♥t♦♠❛ss✐✱ ▼❛tt❡♦ ❛♥❞ ●♦❡❞❡❝❦❡r✱ ❙t❡❢❛♥ ❛♥❞ ●♦♥③❡✱ ❳❛✈✐❡r ❛♥❞ ●r❛♥❛❡s✱ ❖s❝❛r ❛♥❞
●r♦ss✱ ❊✳ ❑✳ ❯✳ ❛♥❞ ●✉❧❛♥s✱ ❆♥❞r✐s ❛♥❞ ●②❣✐✱ ❋r❛♥❝♦✐s ❛♥❞ ❍❛♠❛♥♥✱ ❉✳ ❘✳ ❛♥❞ ❍❛s♥✐♣✱ P❤✐❧ ❏✳ ❛♥❞
❍♦❧③✇❛rt❤✱ ◆✳ ❆✳ ❲✳ ❛♥❞ ■✉s❛♥✱ ❉✐❛♥❛ ❛♥❞ ❏♦❝❤②♠✱ ❉♦♠✐♥✐❦ ❇✳ ❛♥❞ ❏♦❧❧❡t✱ ❋r❛♥❝♦✐s ❛♥❞ ❏♦♥❡s✱ ❉❛♥✐❡❧ ❛♥❞
❑r❡ss❡✱ ●❡♦r❣ ❛♥❞ ❑♦❡♣❡r♥✐❦✱ ❑❧❛✉s ❛♥❞ ❑✉❡❝✉❡❦❜❡♥❧✐✱ ❊♠✐♥❡ ❛♥❞ ❑✈❛s❤♥✐♥✱ ❨❛r♦s❧❛✈ ❖✳ ❛♥❞ ▲♦❝❤t✱ ■♥❦❛ ▲✳
▼✳ ❛♥❞ ▲✉❜❡❝❦✱ ❙✈❡♥ ❛♥❞ ▼❛rs♠❛♥✱ ▼❛rt✐❥♥ ❛♥❞ ▼❛r③❛r✐✱ ◆✐❝♦❧❛ ❛♥❞ ◆✐t③s❝❤❡✱ ❯❧r✐❦❡ ❛♥❞ ◆♦r❞str♦♠✱ ▲❛rs
❛♥❞ ❖③❛❦✐✱ ❚❛✐s✉❦❡ ❛♥❞ P❛✉❧❛tt♦✱ ▲♦r❡♥③♦ ❛♥❞ P✐❝❦❛r❞✱ ❈❤r✐s ❏✳ ❛♥❞ P♦❡❧♠❛♥s✱ ❲❛r❞ ❛♥❞ Pr♦❜❡rt✱ ▼❛tt ■✳ ❏✳
❛♥❞ ❘❡❢s♦♥✱ ❑❡✐t❤ ❛♥❞ ❘✐❝❤t❡r✱ ▼❛♥✉❡❧ ❛♥❞ ❘✐❣♥❛♥❡s❡✱ ●✐❛♥✲▼❛r❝♦ ❛♥❞ ❙❛❤❛✱ ❙❛♥t❛♥✉ ❛♥❞ ❙❝❤❡✤❡r✱
▼❛tt❤✐❛s ❛♥❞ ❙❝❤❧✐♣❢✱ ▼❛rt✐♥ ❛♥❞ ❙❝❤✇❛r③✱ ❑❛r❧❤❡✐♥③ ❛♥❞ ❙❤❛r♠❛✱ ❙❛♥❣❡❡t❛ ❛♥❞ ❚❛✈❛③③❛✱ ❋r❛♥❝❡s❝❛ ❛♥❞
❚❤✉♥str♦❡♠✱ P❛tr✐❦ ❛♥❞ ❚❦❛t❝❤❡♥❦♦✱ ❆❧❡①❛♥❞r❡ ❛♥❞ ❚♦rr❡♥t✱ ▼❛r❝ ❛♥❞ ❱❛♥❞❡r❜✐❧t✱ ❉❛✈✐❞ ❛♥❞ ✈❛♥ ❙❡tt❡♥✱
▼✐❝❤✐❡❧ ❏✳ ❛♥❞ ❱❛♥ ❙♣❡②❜r♦❡❝❦✱ ❱❡r♦♥✐q✉❡ ❛♥❞ ❲✐❧❧s✱ ❏♦❤♥ ▼✳ ❛♥❞ ❨❛t❡s✱ ❏♦♥❛t❤❛♥ ❘✳ ❛♥❞ ❩❤❛♥❣✱ ●✉♦✲❳✉
❛♥❞ ❈♦tt❡♥✐❡r✱ ❙t❡❢❛❛♥ ✭✷✵✶✻✮ ✬❘❡♣r♦❞✉❝✐❜✐❧✐t② ✐♥ ❞❡♥s✐t② ❢✉♥❝t✐♦♥❛❧ t❤❡♦r② ❝❛❧❝✉❧❛t✐♦♥s ♦❢ s♦❧✐❞s✳✬✱ ❙❝✐❡♥❝❡✳✱
✸✺✶ ✭✻✷✽✵✮✳ ❛❛❞✸✵✵✵✳
❋✉rt❤❡r ✐♥❢♦r♠❛t✐♦♥ ♦♥ ♣✉❜❧✐s❤❡r✬s ✇❡❜s✐t❡✿
❤tt♣s✿✴✴❞♦✐✳♦r❣✴✶✵✳✶✶✷✻✴s❝✐❡♥❝❡✳❛❛❞✸✵✵✵
P✉❜❧✐s❤❡r✬s ❝♦♣②r✐❣❤t st❛t❡♠❡♥t✿
❚❤✐s ✐s t❤❡ ❛✉t❤♦r✬s ✈❡rs✐♦♥ ♦❢ t❤❡ ✇♦r❦✳ ■t ✐s ♣♦st❡❞ ❤❡r❡ ❜② ♣❡r♠✐ss✐♦♥ ♦❢ t❤❡ ❆❆❆❙ ❢♦r ♣❡rs♦♥❛❧ ✉s❡✱ ♥♦t ❢♦r
r❡❞✐str✐❜✉t✐♦♥✳ ❚❤❡ ❞❡✜♥✐t✐✈❡ ✈❡rs✐♦♥ ✇❛s ♣✉❜❧✐s❤❡❞ ✐♥ ❙❝✐❡♥❝❡ ♦♥ ✷✺ ▼❛r ✷✵✶✻✿ ❱♦❧✳ ✸✺✶✱ ■ss✉❡ ✻✷✽✵✱ ❉❖■✿
✶✵✳✶✶✷✻✴s❝✐❡♥❝❡✳❛❛❞✸✵✵✵
❆❞❞✐t✐♦♥❛❧ ✐♥❢♦r♠❛t✐♦♥✿
❯s❡ ♣♦❧✐❝②
❚❤❡ ❢✉❧❧✲t❡①t ♠❛② ❜❡ ✉s❡❞ ❛♥❞✴♦r r❡♣r♦❞✉❝❡❞✱ ❛♥❞ ❣✐✈❡♥ t♦ t❤✐r❞ ♣❛rt✐❡s ✐♥ ❛♥② ❢♦r♠❛t ♦r ♠❡❞✐✉♠✱ ✇✐t❤♦✉t ♣r✐♦r ♣❡r♠✐ss✐♦♥ ♦r ❝❤❛r❣❡✱ ❢♦r
♣❡rs♦♥❛❧ r❡s❡❛r❝❤ ♦r st✉❞②✱ ❡❞✉❝❛t✐♦♥❛❧✱ ♦r ♥♦t✲❢♦r✲♣r♦✜t ♣✉r♣♦s❡s ♣r♦✈✐❞❡❞ t❤❛t✿
•
❛ ❢✉❧❧ ❜✐❜❧✐♦❣r❛♣❤✐❝ r❡❢❡r❡♥❝❡ ✐s ♠❛❞❡ t♦ t❤❡ ♦r✐❣✐♥❛❧ s♦✉r❝❡
•
❛
❧✐♥❦ ✐s ♠❛❞❡ t♦ t❤❡ ♠❡t❛❞❛t❛ r❡❝♦r❞ ✐♥ ❉❘❖
•
t❤❡ ❢✉❧❧✲t❡①t ✐s ♥♦t ❝❤❛♥❣❡❞ ✐♥ ❛♥② ✇❛②
❚❤❡ ❢✉❧❧✲t❡①t ♠✉st ♥♦t ❜❡ s♦❧❞ ✐♥ ❛♥② ❢♦r♠❛t ♦r ♠❡❞✐✉♠ ✇✐t❤♦✉t t❤❡ ❢♦r♠❛❧ ♣❡r♠✐ss✐♦♥ ♦❢ t❤❡ ❝♦♣②r✐❣❤t ❤♦❧❞❡rs✳
P❧❡❛s❡ ❝♦♥s✉❧t t❤❡
❢✉❧❧ ❉❘❖ ♣♦❧✐❝② ❢♦r ❢✉rt❤❡r ❞❡t❛✐❧s✳
❉✉r❤❛♠ ❯♥✐✈❡rs✐t② ▲✐❜r❛r②✱ ❙t♦❝❦t♦♥ ❘♦❛❞✱ ❉✉r❤❛♠ ❉❍✶ ✸▲❨✱ ❯♥✐t❡❞ ❑✐♥❣❞♦♠
❚❡❧ ✿ ✰✹✹ ✭✵✮✶✾✶ ✸✸✹ ✸✵✹✷ ⑤ ❋❛① ✿ ✰✹✹ ✭✵✮✶✾✶ ✸✸✹ ✷✾✼✶
❤tt♣s✿✴✴❞r♦✳❞✉r✳❛❝✳✉❦
✷
Reproducibility in density-functional theory
calculations of solids
Kurt Lejaeghere,
1∗
Gustav Bihlmayer,
2
Torbj
¨
orn Bj
¨
orkman,
3,4
Peter Blaha,
5
Stefan Bl
¨
ugel,
2
Volker Blum,
6
Damien Caliste,
7,8
Ivano E. Castelli,
9
Stewart J. Clark,
10
Andrea Dal Corso,
11
Stefano de Gironcoli,
11
Thierry Deutsch,
7,8
John Kay Dewhurst,
12
Igor Di Marco,
13
Claudia Draxl,
14,15
Marcin Dułak,
16
Olle Eriksson,
13
Jos
´
e A. Flores-Livas,
12
Kevin F. Garrity,
17
Luigi Genovese,
7,8
Paolo Giannozzi,
18
Matteo Giantomassi,
19
Stefan Goedecker,
20
Xavier Gonze,
19
Oscar Gr
˚
an
¨
as,
13,21
E. K. U. Gross,
12
Andris Gulans,
14,15
Franc¸ois Gygi,
22
D. R. Hamann,
23,24
Phil J. Hasnip,
25
N. A. W. Holzwarth,
26
Diana Ius
,
an,
13
Dominik B. Jochym,
27
Franc¸ois Jollet,
28
Daniel Jones,
29
Georg Kresse,
30
Klaus Koepernik,
31,32
Emine K
¨
uc¸
¨
ukbenli,
11
Yaroslav O. Kvashnin,
13
Inka L. M. Locht,
13,33
Sven Lubeck,
14
Martijn Marsman,
30
Nicola Marzari,
9
Ulrike Nitzsche,
31
Lars Nordstr
¨
om,
13
Taisuke Ozaki,
34
Lorenzo Paulatto,
35
Chris J. Pickard,
36
Ward Poelmans,
1,37
Matt I. J. Probert,
25
Keith Refson,
38,39
Manuel Richter,
31,32
Gian-Marco Rignanese,
19
Santanu Saha,
20
Matthias Scheffler,
15,40
Martin Schlipf,
22
Karlheinz Schwarz,
5
Sangeeta Sharma,
12
Francesca Tavazza,
17
Patrik Thunstr
¨
om,
41
Alexandre Tkatchenko,
15
Marc Torrent,
28
David Vanderbilt,
23
Michiel J. van Setten,
19
Veronique Van Speybroeck,
1
John M. Wills,
42
Jonathan R. Yates,
29
Guo-Xu Zhang,
43
Stefaan Cottenier
1,44∗
1
Center for Molecular Modeling, Ghent University, Technologiepark 903, BE-9052
Zwijnaarde, Belgium,
2
Peter Gr
¨
unberg Institut and Institute for Advanced Simulation, Forschungszentrum J
¨
ulich and
JARA, D-52425 J
¨
ulich, Germany,
3
Department of Physics,
˚
Abo Akademi, FI-20500 Turku, Finland,
4
COMP/Department of Applied Physics, Aalto University School of Science, P.O. Box 11100,
FI-00076 Aalto, Finland,
1
5
Institute of Materials Chemistry, Vienna University of Technology, Getreidemarkt 9/165-TC,
A-1060 Vienna, Austria,
6
Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC
27708, USA,
7
Universit
´
e Grenoble Alpes, INAC-SP2M, L
Sim, F-38042 Grenoble, France,
8
CEA, INAC-SP2M, L
Sim, F-38054 Grenoble, France,
9
Theory and Simulation of Materials (THEOS), and National Centre for Computational
Design and Discovery of Novel Materials (MARVEL),
´
Ecole Polytechnique F
´
ed
´
erale de
Lausanne, CH-1015 Lausanne, Switzerland,
10
Department of Physics, University of Durham, Durham DH1 3LE, United Kingdom,
11
International School for Advanced Studies (SISSA) and DEMOCRITOS IOM-CNR Trieste,
Via Bonomea 265, I-34136 Trieste, Italy,
12
Max-Planck-Institut f
¨
ur Mikrostrukturphysik, Weinberg 2, D-06120 Halle, Germany,
13
Department of Physics and Astronomy, Division of Materials Theory, Uppsala University,
PO Box 516, SE-75120 Uppsala, Sweden,
14
Institut f
¨
ur Physik and IRIS Adlershof, Humboldt-Universit
¨
at zu Berlin, Zum Großen
Windkanal 6, D-12489 Berlin, Germany,
15
Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195 Berlin,
Germany,
16
Center for Atomic-scale Materials Design, Department of Physics, Technical University of
Denmark, DK-2800 Kgs. Lyngby, Denmark,
17
Material Measurement Laboratory, National Institute of Standards and Technology, 100
Bureau Drive, Stop 8553, Gaithersburg, MD 20899, USA,
18
Department of Chemistry, Physics, and Environment, University of Udine, via delle Scienze
208, I-33100 Udine, Italy,
19
Institute of Condensed Matter and Nanosciences - NAPS, Universit
´
e catholique de Louvain,
Chemin des
´
etoiles 8, B-1348 Louvain-la-Neuve, Belgium,
20
Institut f
¨
ur Physik, Universit
¨
at Basel, Klingelbergstr. 82, CH-4056 Basel, Switzerland,
21
School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138,
USA,
22
Department of Computer Science, University of California Davis, Davis, CA 95616, USA,
23
Department of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854-8019,
USA,
24
Mat-Sim Research LLC, P.O. Box 742, Murray Hill, NJ 07974, USA,
25
Department of Physics, University of York, Heslington, York YO10 5DD, United Kingdom,
26
Department of Physics, Wake Forest University, Winston-Salem, NC 27109, USA,
27
Scientific Computing Department, Science and Technology Facilities Council, Rutherford
Appleton Laboratory, Didcot OX11 0QX, United Kingdom,
28
CEA, DAM, DIF, F-91297 Arpajon, France,
29
Department of Materials, University of Oxford, 16 Parks Road, Oxford OX1 3PH, United
Kingdom,
2
30
University of Vienna, Faculty of Physics and Center for Computational Materials Science,
Sensengasse 8/12, A-1090 Vienna, Austria,
31
IFW Dresden e.V., P.O. Box 270 116, D-01171 Dresden, Germany,
32
Dresden Center for Computational Materials Science (DCMS), TU Dresden, D-01069
Dresden, Germany,
33
Institute for Molecules and Materials, Radboud University, Heyendaalseweg 135, 6525 AJ
Nijmegen, the Netherlands,
34
Institute for Solid State Physics, The University of Tokyo, Kashiwa 277-8581, Japan,
35
Institut de Min
´
eralogie, de Physique des Mat
´
eriaux, et de Cosmochimie (IMPMC), Sorbonne
Universit
´
es, UPMC University Paris 06, UMR CNRS 7590, Mus
´
eum National d’Histoire
Naturelle, IRD UMR 206, 4 Place Jussieu, F-75005 Paris, France,
36
Department of Materials Science & Metallurgy, University of Cambridge, 27 Charles
Babbage Road, Cambridge CB3 0FS, United Kingdom,
37
UGent HPC, Ghent University, Krijgslaan 281 S9, BE-9000 Ghent, Belgium,
38
Department of Physics, Royal Holloway, University of London, Egham TW20 0EX, United
Kingdom,
39
ISIS Facility, Science and Technology Facilities Council, Rutherford Appleton Laboratory,
Didcot OX11 0QX, United Kingdom,
40
Department of Chemistry and Biochemistry and Materials Department, University of
California, Santa Barbara, CA 93106-5050, USA,
41
Institute for Solid State Physics, Vienna University of Technology, A-1040 Vienna, Austria,
42
Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA,
43
Institute of Theoretical and Simulational Chemistry, Academy of Fundamental and
Interdisciplinary Sciences, Harbin Institute of Technology, Harbin 150080, People’s Republic
of China,
44
Department of Materials Science and Engineering, Ghent University, Technologiepark 903,
BE-9052 Zwijnaarde, Belgium
∗
To whom correspondence should be addressed;
E-mail: kurt.lejaeghere@ugent.be, stefaan.cottenier@ugent.be.
The widespread popularity of density-functional theory has given rise to a
vast range of dedicated codes to predict molecular and crystalline proper-
ties. However, each code implements the formalism in a different way, raising
questions on the reproducibility of such predictions. We report the results of
a community-wide effort that compares 15 solid-state codes using 40 differ-
ent potentials or basis set types, assessing the quality of the Perdew-Burke-
3