Recent developments in the general atomic and molecular electronic structure system.

TLDR: A discussion of many of the recently implemented features of GAMESS (General Atomic and Molecular Electronic Structure System) and LibCChem (the C++ CPU/GPU library associated with GAMESS) is presented, which include fragmentation methods, hybrid MPI/OpenMP approaches to Hartree-Fock, and resolution of the identity second order perturbation theory.

Abstract: A discussion of many of the recently implemented features of GAMESS (General Atomic and Molecular Electronic Structure System) and LibCChem (the C++ CPU/GPU library associated with GAMESS) is presented. These features include fragmentation methods such as the fragment molecular orbital, effective fragment potential and effective fragment molecular orbital methods, hybrid MPI/OpenMP approaches to Hartree-Fock, and resolution of the identity second order perturbation theory. Many new coupled cluster theory methods have been implemented in GAMESS, as have multiple levels of density functional/tight binding theory. The role of accelerators, especially graphical processing units, is discussed in the context of the new features of LibCChem, as it is the associated problem of power consumption as the power of computers increases dramatically. The process by which a complex program suite such as GAMESS is maintained and developed is considered. Future developments are briefly summarized.

Figures

TABLE IV. GAMESS Jenkins build-test configuration for integration into the development branch.

TABLE II. The highest analytic derivative of the energy with respect to nuclear coordinates for each QM method interfaced with FMO in GAMESS (0 = energy, 1 = gradient, and 2 = Hessian).a

FIG. 2. The power consumption of the wat-19 calculation using the feedback strategy when the power allocation is set to 315 W. The power shown is the sum of the instantaneous GPU, CPU, and DRAM power, which are obtained from nvidia-smi for GPU and from the Intel Running Average Power Limit (RAPL) interface for CPU and DRAM.

Full text

Old Dominion University Old Dominion University
ODU Digital Commons ODU Digital Commons
Computational Modeling and Simulation
Engineering Faculty Publications
Computational Modeling and Simulation
Engineering
2020
Recent Developments in the General Atomic and Molecular Recent Developments in the General Atomic and Molecular
Electronic Structure System Electronic Structure System
Guiseppe M.J. Barca
Colleen Bertoni
Laura Carrington
Dipayan Datta
Nuwan De Silva
See next page for additional authors
Follow this and additional works at: https://digitalcommons.odu.edu/msve_fac_pubs
Part of the Computer Engineering Commons, and the Electrical and Computer Engineering Commons
Original Publication Citation Original Publication Citation
Barca, G. M. J., Bertoni, C., Carrington, L., Datta, D., De Silva, N., Deustua, J. E., . . . Gordon, M. S. (2020).
Recent developments in the general atomic and molecular electronic structure system.
The Journal of
Chemical Physics, 152
(15), 1-26. doi:10.1063/5.0005188
This Article is brought to you for free and open access by the Computational Modeling and Simulation Engineering
at ODU Digital Commons. It has been accepted for inclusion in Computational Modeling and Simulation
Engineering Faculty Publications by an authorized administrator of ODU Digital Commons. For more information,
please contact digitalcommons@odu.edu.

Authors Authors
Guiseppe M.J. Barca, Colleen Bertoni, Laura Carrington, Dipayan Datta, Nuwan De Silva, J. Emillano
Deustua, Dmitri G. Fedorov, Jeffrey R. Cour, Anastasia O. Gunina, Emilie Guidez, Taylor Harville, Stephan
Irle, Joe Ivanic, Karol Kowalski, Sarom S. Leang, Wei Li, Jesse J. Lutz, Ilias Magoulas, Joani Mato, Vladimir
Mironov, Hiroya Nakata, Buu Q. Pham, Piotr Piecuch, David Poole, Spencer R. Pruitt, Alistair P. Rendell,
Luke B. Roskop, Klaus Ruedenberg, Tosaporn Sattasathuchana, Michael W. Schmidt, Jun Shen, Lyudmila
Slipchenko, Masha Sosonkina, Vaibhav Sundriyal, Ananta Tiwari, Jorge L. Galvez Vallejo, Bryce
Westheimer, Marta Włoch, Peng Xu, Federico Zahariev, and Mark S. Gordon
This article is available at ODU Digital Commons: https://digitalcommons.odu.edu/msve_fac_pubs/59

J. Chem. Phys. 152, 154102 (2020); https://doi.org/10.1063/5.0005188 152, 154102
© 2020 Author(s).
Recent developments in the general atomic
and molecular electronic structure system
Cite as: J. Chem. Phys. 152, 154102 (2020); https://doi.org/10.1063/5.0005188
Submitted: 20 February 2020 . Accepted: 19 March 2020 . Published Online: 16 April 2020
Giuseppe M. J. Barca , Colleen Bertoni, Laura Carrington, Dipayan Datta , Nuwan De Silva , J.
Emiliano Deustua , Dmitri G. Fedorov , Jeffrey R. Gour, Anastasia O. Gunina , Emilie Guidez ,
Taylor Harville, Stephan Irle , Joe Ivanic , Karol Kowalski , Sarom S. Leang, Hui Li, Wei Li, Jesse
J. Lutz, Ilias Magoulas , Joani Mato, Vladimir Mironov , Hiroya Nakata, Buu Q. Pham, Piotr Piecuch
, David Poole, Spencer R. Pruitt, Alistair P. Rendell, Luke B. Roskop, Klaus Ruedenberg, Tosaporn
Sattasathuchana, Michael W. Schmidt, Jun Shen , Lyudmila Slipchenko , Masha Sosonkina,
Vaibhav Sundriyal, Ananta Tiwari, Jorge L. Galvez Vallejo, Bryce Westheimer , Marta
Włoch
, Peng
Xu, Federico Zahariev , and Mark S. Gordon
COLLECTIONS
Paper published as part of the special topic on Electronic Structure Software
Note: This article is part of the JCP Special Topic on Electronic Structure Software.
ARTICLES YOU MAY BE INTERESTED IN
Quantum ESPRESSO toward the exascale
The Journal of Chemical Physics 152, 154105 (2020); https://doi.org/10.1063/5.0005082
The Molpro quantum chemistry package
The Journal of Chemical Physics 152, 144107 (2020); https://doi.org/10.1063/5.0005081
DFTB+, a software package for efficient approximate density functional theory based
atomistic simulations
The Journal of Chemical Physics 152, 124101 (2020); https://doi.org/10.1063/1.5143190

The Journal
of Chemical Physics
ARTICLE
scitation.org/journal/jcp
Recent developments in the general atomic
and molecular electronic structure system
Cite as: J. Chem. Phys. 152, 154102 (2020); doi: 10.1063/5.0005188
Submitted: 20 February 2020 • Accepted: 19 March 2020 •
Published Online: 16 April 2020 • Corrected: 24 April 2020
Giuseppe M. J. Barca,
1
Colleen Bertoni,
2
Laura Carrington,
3
Dipayan Datta,
4
Nuwan De Silva,
5
J. Emiliano Deustua,
6
Dmitri G. Fedorov,
7
Jeffrey R. Gour,
8
Anastasia O. Gunina,
4
Emilie Guidez,
9
Taylor Harville,
4
Stephan Irle,
10
Joe Ivanic,
11
Karol Kowalski,
12
Sarom S. Leang,
3
Hui Li,
13
Wei Li,
14
Jesse J. Lutz,
15
Ilias Magoulas,
6
Joani Mato,
4
Vladimir Mironov,
16
Hiroya Nakata,
17
Buu Q. Pham,
4
Piotr Piecuch,
6,18
David Poole,
4
Spencer R. Pruitt,
4
Alistair P. Rendell,
1
Luke B. Roskop,
19
Klaus Ruedenberg,
4
Tosaporn Sattasathuchana,
4
Michael W. Schmidt,
4
Jun Shen,
6
Lyudmila Slipchenko,
20
Masha Sosonkina,
21
Vaibhav Sundriyal,
21
Ananta Tiwari,
3
Jorge L. Galvez Vallejo,
4
Bryce Westheimer,
4
Marta Włoch,
22
Peng Xu,
4
Federico Zahariev,
4
and Mark S. Gordon
4,a)
AFFILIATIONS
1
Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
2
Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, USA
3
EP Analytics, 12121 Scripps Summit Dr. Ste. 130, San Diego, California 92131, USA
4
Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
5
Department of Physical and Biological Sciences, Western New England University, Springfield, Massachusetts 01119, USA
6
Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
7
Research Center for Computational Design of Advanced Functional Materials (CD-FMat), National Institute
of Advanced Industrial Science and Technology (AIST), Umezono 1-1-1, Tsukuba 305-8568, Japan
8
Microsoft, 15590 NE 31st St., Redmond, Washington 98052, USA
9
Department of Chemistry, University of Colorado Denver, Denver, Colorado 80217, USA
10
Computational Science and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37830, USA
11
Advanced Biomedical Computational Science, Frederick National Laboratory for Cancer Research, Frederick,
Maryland 21702, USA
12
Physical Sciences Division, Battelle, Pacific Northwest National Laboratory, K8-91, P.O. Box 999, Richland, Washington
99352, USA
13
Department of Chemistry, University of Nebraska, Lincoln, Nebraska 68588, USA
14
School of Chemistry and Chemical Engineering, Key Laboratory of Mesoscopic Chemistry of Ministry of Education,
Institute of Theoretical and Computational Chemistry, Nanjing University, Nanjing 210023, People’s Republic of China
15
Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87185, USA
16
Department of Chemistry, Lomonosov Moscow State University, Leninskie Gory 1/3, Moscow 119991, Russian Federation
17
Kyocera Corporation, Research Institute for Advanced Materials and Devices, 3-5-3 Hikaridai Seika-cho, Souraku-gun,
Kyoto 619-0237, Japan
18
Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, USA
19
Cray Inc., a Hewlett Packard Enterprise Company, 2131 Lindau Ln #1000, Bloomington, Minnesota 55425, USA
20
Department of Chemistry, Purdue University, West Lafayette, Indiana 47907, USA
21
Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, USA
22
530 Charlesina Dr., Rochester, Michigan 48306, USA
Note: This article is part of the JCP Special Topic on Electronic Structure Software.
a)
Author to whom correspondence should be addressed: mark@si.msg.chem.iastate.edu
J. Chem. Phys. 152, 154102 (2020); doi: 10.1063/5.0005188 152, 154102-1
Published under license by AIP Publishing

The Journal
of Chemical Physics
ARTICLE
scitation.org/journal/jcp
ABSTRACT
A discussion of many of the recently implemented features of GAMESS (General Atomic and Molecular Electronic Structure System) and
LibCChem (the C++ CPU/GPU library associated with GAMESS) is presented. These features include fragmentation methods such as the
fragment molecular orbital, effective fragment potential and effective fragment molecular orbital methods, hybrid MPI/OpenMP approaches
to Hartree–Fock, and resolution of the identity second order perturbation theory. Many new coupled cluster theory methods have been
implemented in GAMESS, as have multiple levels of density functional/tight binding theory. The role of accelerators, especially graphical
processing units, is discussed in the context of the new features of LibCChem, as it is the associated problem of power consumption as the
power of computers increases dramatically. The process by which a complex program suite such as GAMESS is maintained and developed is
considered. Future developments are briefly summarized.
Published under license by AIP Publishing. https://doi.org/10.1063/5.0005188
.,
s
I. OVERVIEW/BACKGROUND
GAMESS (General Atomic and Molecular Electronic Structure
System) was originally developed by Dupuis and co-workers in the
late 1970s under the auspices of the National Resource for Com-
putational Chemistry (NRCC), an organization that was sponsored
by the National Science Foundation. GAMESS is a multi-functional
electronic structure program with users in more than 100 coun-
tries and is run on nearly every available architecture, ranging from
MacOS and Windows to the pre-exascale system Summit at Oak
Ridge National Laboratory. GAMESS is a “cousin” of the HONDO
program, which continues to be developed by Dupuis. GAMESS is
distributed at no cost with a very simple license to prevent unau-
thorized redistribution. GAMESS itself is primarily written in For-
tran77, with an increasing number of functionalities written in For-
tran90. Associated with GAMESS is an object-oriented C++ library
called LibCChem, initiated in 2010, which contains an increasing
number of quantum chemistry functionalities and is written for both
central processing unit (CPU) and GPU (graphical processing unit)
architectures.
As discussed in two previous reviews in 1993
1
and 2005,
2
GAMESS has essentially all of the commonly used electronic struc-
ture methods, including Hartree–Fock (HF) self-consistent field
(SCF), density functional theory (DFT) with many of the pop-
ular functionals, second order perturbation theory (MP2), cou-
pled cluster (CC) theory, including CCSD(T), and novel methods
such as CR-CC(2,3) that are capable of correctly breaking single
bonds, equations-of-motion (EOM) coupled cluster theory, time-
dependent density functional theory (TDDFT), configuration inter-
action (CI) up to and including full CI, complete active space (CAS)
SCF, multi-reference (MR) MP2, and multi-reference CI (MRCI).
Also available in GAMESS is the effective fragment potential (EFP)
method, a sophisticated model potential with no fitted parameters,
which is applicable to any molecular system. Other functionalities
include fully analytic second energy derivatives (Hessians) for closed
shell HF and CASSCF, fully analytic energy first derivatives (gradi-
ents), and, therefore, semi-numeric Hessians for HF, DFT, MP2, CI,
and EFP, thereby enabling the prediction of vibrational frequencies
and IR and Raman spectra. Related to vibrational spectroscopy is the
vibrational SCF suite of methods developed by Gerber and cowork-
ers.
3
GAMESS also has several options for reaction path following
and for performing classical trajectories using any of the available
electronic structure methods. Solvent effects can be incorporated
explicitly using the EFP method or implicitly using the polariz-
able continuum model (PCM
4
), COSMO (conductor-like Screening
Model),
5
or the surface volume polarization (SVP) model.
6
Surface
science can be studied using the surface integrated molecular orbital
molecular mechanics (SIMOMM)
7
method.
If one desires very high accuracy in electronic structure calcu-
lations, there is a CEEIS (correlation energy extrapolation by intrin-
sic scaling)
8
method developed by Ruedenberg and Bytautas that
provides essentially the exact full CI energy at a fraction of the cost.
The ability of GAMESS to treat excited electronic states, pho-
tochemistry, and related phenomena such as surface crossings and
conical intersections has made significant advances with the intro-
duction of spin-flip (SF) methods
9
for the energy and the ana-
lytic gradient,
10
including the development of a general approach to
spin-correct spin-flip.
11
An exciting new feature of GAMESS is the quasiatomic orbital
(QUAO) analysis developed by West and colleagues.
12
This anal-
ysis, which continues to be developed, has been applied to several
interesting problems in chemistry.
Since the early 1990s, a major effort related to the development
of GAMESS has been to maximize the scalability (parallelism) of the
code. The ability of GAMESS to explore potential energy surfaces
accurately and efficiently is much improved with the development
of several GAMESS functionalities that can take advantage of com-
bining MPI (message passing interface) and OpenMP into a hybrid
approach that takes optimal advantage of both distributed comput-
ing (MPI) and shared memory computing (OpenMP). This combi-
nation has now been applied to HF, DFT, and the resolution of the
identity (RI) version of MP2.
In the past several years, this stride toward high performance
computational chemistry has increasingly taken center stage.
13–18
An
important component of this endeavor has been to make optimal use
of accelerators. In the remainder of this review, the primary focus
is on new features that have been implemented since 2005 and, in
particular, the advances in the development of highly scalable code,
with the aim of achieving the ability to make use of the anticipated
exascale computers, where exascale may be defined as 10
18
flops or a
gigagigaflop.
An important component of the development of highly scalable
electronic structure software is the innovation of reliable fragmen-
tation methods. In GAMESS, this specifically means the fragment
molecular orbital (FMO),
19
the effective fragment potential (EFP),
20
and the effective fragment molecular orbital (EFMO)
21
methods.
J. Chem. Phys. 152, 154102 (2020); doi: 10.1063/5.0005188 152, 154102-2
Published under license by AIP Publishing

Frequently asked questions

Q1. What are the contributions in "Recent developments in the general atomic and molecular electronic structure system" ?

A discussion of many of the recently implemented features of GAMESS ( General Atomic and Molecular Electronic Structure System ) and LibCChem ( the C++ CPU/GPU library associated with GAMESS ) is presented. Surface science can be studied using the surface integrated molecular orbital molecular mechanics ( SIMOMM ) method. In the remainder of this review, the primary focus is on new features that have been implemented since 2005 and, in particular, the advances in the development of highly scalable code, with the aim of achieving the ability to make use of the anticipated exascale computers, where exascale may be defined as 10 flops or a gigagigaflop. 1063/5. 0005188 152, 154102-2 Published under license by AIP Publishing The Journal of Chemical Physics ARTICLE scitation. In the following, several fragmentation methods that are available in GAMESS are discussed. These features include fragmentation methods such as the fragment molecular orbital, effective fragment potential and effective fragment molecular orbital methods, hybrid MPI/OpenMP approaches to Hartree–Fock, and resolution of the identity second order perturbation theory. Also available in GAMESS is the effective fragment potential ( EFP ) method, a sophisticated model potential with no fitted parameters, which is applicable to any molecular system. The ability of GAMESS to explore potential energy surfaces accurately and efficiently is much improved with the development of several GAMESS functionalities that can take advantage of combining MPI ( message passing interface ) and OpenMP into a hybrid approach that takes optimal advantage of both distributed computing ( MPI ) and shared memory computing ( OpenMP ). In GAMESS, this specifically means the fragment molecular orbital ( FMO ), the effective fragment potential ( EFP ), and the effective fragment molecular orbital ( EFMO ) methods. B. Effective fragment potential The effective fragment potential ( EFP ) method is an ab initio force field designed to model intermolecular interactions accurately and efficiently. 

Q2. What are the future works mentioned in the paper "Recent developments in the general atomic and molecular electronic structure system" ?

Many of the future developments of GAMESS and Libcchem have already been mentioned. Work is underway on enriching the existing CC routines with the double electron-attachment and double ionization potential EOMCC options, which are particularly useful in determining the electronic spectra of biradicals,154 and approximate coupled-pair approaches, which extend traditional CC truncations to a strongly correlated regime. 

Q3. What is the work on enhancing the existing CC routines?

Work is underway on enriching the existing CC routines with the double electron-attachment and double ionization potential EOMCC options, which are particularly useful in determining the electronic spectra of biradicals,154 and approximate coupled-pair approaches, which extend traditional CC truncations to a strongly correlated regime. 

Q4. How many integrals can be eliminated before the shell quartet is executed?

Before the shell quartet is executed, a fairly large number of small integrals can be eliminated90,93 using the Cauchy–Schwarz inequality. 

Q5. What is the purpose of the build-test?

Each build-test compiles GAMESS using the GNU compiler and performs validation testing using a small test set consisting of serial and parallel runs. 

Q6. What is the key component of the stride toward exascale computing?

A second key component of the stride toward exascale computing is the recognition that accelerators/co-processors, such as GPUs, will play an important role in the future of high performance computational chemistry. 

Q7. What are the recent additions to the ORMAS module?

More recently, single reference (SR) and multireference (MR) coupled electron pair approximation (CEPA) methodologies were added to the ORMAS module. 

Q8. What is the important reason why the GPU should be given the highest priority?

Previous experiments, however, showed that the highest priority should be given to DRAM if (part of) a calculation is memoryintensive, such as storing/reading the integrals, to avoid a huge performance penalty. 

Q9. What is the main idea behind the parallel CCSD(T) code?

Following the lead of the new RI-MP2 code discussed in Sec. III A, the parallel CCSD(T) code will make use of a hybrid DDI/OpenMP model by substituting the process-based parallelism on each node with thread-based parallelism. 

Q10. What is the approach to obtain virtual orbitals?

The recommended approach is to perform a preliminary CISD calculation, compute the corresponding oneparticle density matrix, and diagonalize the virtual–virtual block to obtain natural orbitals for the virtual space (VSDNOs). 

Q11. Who has developed the multiconfigurational pair density functional theory?

Truhlar, and co-workers have developed the multiconfigurational pair density functional theory (MCP-DFT) that introduces multi-configurational character into DFT. 

Q12. What is the important consideration for parallel computer coding?

In addition to the development of strategies for parallel computer coding, some of which have been discussed in previous sections, consideration must be given to the power consumption by massively parallel computers (i.e., Dennard’s law275), which can be as costly on an annual basis as the initial cost of the hardware. 

Q13. What is the way to reduce the time for the calculation of fragment energies and gradients?

it is necessary to use a computationally less expensive electronic structure method that reduces the time for the calculation of fragment energies and gradients by at least one order of magnitude. 

Q14. What are some methods that are common to the GAMESS, NWChem, PSI?

There are some methods, of course, such as HF, DFT, MP2, and coupled-cluster methods that are common to the GAMESS, NWChem, PSI4, and CFOUR programs. 

Q15. How many semi-empirical dispersion energy corrections have been implemented?

De Silva, Adreance, and Gordon implemented the Grimme–D3 semi-empirical dispersion energy correction (including the “E8 term”) for EFP1 and for QM–EFP1 systems.69 

Q16. What is the way to reduce the unfavorable scaling of the DFTB?

This step scales cubically with system size, similar to the parent DFT method, and hence fragmentation is ideally suited to reduce this unfavorable scaling. 

Q17. What is the difference between SF and conventional multi-determinant methods?

In contrast to conventional multi-determinant approaches, SF methods rely on a high-spin reference determinant (MS > 0), which, through a series of spin-flipping excitations (ΔMS < 0), generates a multi-determinant wave function of a lower multiplicity.