SchNet - A deep learning architecture for molecules and materials.

TLDR: SchNet as mentioned in this paper is a deep learning architecture specifically designed to model atomistic systems by making use of continuous-filter convolutional layers, where the model learns chemically plausible embeddings of atom types across the periodic table.

Abstract: Deep learning has led to a paradigm shift in artificial intelligence, including web, text, and image search, speech recognition, as well as bioinformatics, with growing impact in chemical physics. Machine learning, in general, and deep learning, in particular, are ideally suitable for representing quantum-mechanical interactions, enabling us to model nonlinear potential-energy surfaces or enhancing the exploration of chemical compound space. Here we present the deep learning architecture SchNet that is specifically designed to model atomistic systems by making use of continuous-filter convolutional layers. We demonstrate the capabilities of SchNet by accurately predicting a range of properties across chemical space for molecules and materials, where our model learns chemically plausible embeddings of atom types across the periodic table. Finally, we employ SchNet to predict potential-energy surfaces and energy-conserving force fields for molecular dynamics simulations of small molecules and perform an exemplary study on the quantum-mechanical properties of C20-fullerene that would have been infeasible with regular ab initio molecular dynamics.

Figures

TABLE II. Mean absolute errors for formation energy predictions in eV/atom on the Materials Project dataset. For SchNet, we give the average over three repetitions as well as standard errors of the mean of the repetitions. Best models in bold.

FIG. 4. The two leading principal components of the learned embeddings x0 of sp atoms learned by SchNet from the Materials Project dataset. We recognize a structure in the embedding space according to the groups of the periodic table (color-coded) as well as an ordering from lighter to heavier elements within the groups, e.g., in groups IA and IIA from light atoms (left) to heavier atoms (right).

FIG. 11. Histograms of absolute errors for all predicted properties of QM9. The histograms are plotted on a logarithmic scale to visualize the tails of the distribution.

FIG. 10. Histogram of absolute errors for the predictions of formation energies/atom for the Materials Project dataset. The histogram is plotted on a logarithmic scale to visualize the tails of the distribution.

FIG. 5. Local chemical potentials ΩC (r) of DTNN (top) and SchNet (bottom) using a carbon test charge on a

FIG. 6. Cuts through local chemical potentials ΩC (r) of SchNet using a carbon test charge are shown for graphite (left) and diamond (right).

Frequently asked questions

Q1. What is the effect of the atom embeddings on the properties of graphite?

Since the initial atom embeddings are obviously equivariant to the order of atoms, atom-wise layers are independently applied to each atom and continuous-filter convolutions sum over all neighboring atoms, indexing equivariance is retained in the atom-wise representations. 

Q2. What is the effect of the periodic filters on the properties of diamond and graphite?

Note that while the single-atom filters are circular due to the rotational invariance, the periodic filters become rotationally equivariant with respect to the orientation of the lattice, which still keeps the property prediction rotationally invariant. 

Q3. What have the authors contributed in "Schnet – a deep learning architecture for molecules and materials" ?

SchNet this paper is a variant of DTNNs that can learn a representation from first principles that adapts to the task and scale at hand from property prediction across chemical compound space to force fields in the configurational space of single molecules. 

Q4. What future works have the authors mentioned in the paper "Schnet – a deep learning architecture for molecules and materials" ?

This gives rise to the possibility to encode known quantumchemical constraints and symmetries within the model without losing the flexibility of a neural network. These encouraging results will guide future work such as studies of larger molecules and periodic systems as well as further developments toward interpretable deep learning architectures to assist chemistry research. This is crucial in order to be able to accurately represent, e. g., the full potential-energy surface and, in particular, its anharmonic behavior. The authors have presented the deep learning architecture SchNet which can be applied to a variety of applications ranging from the prediction of chemical properties for diverse datasets of molecules and materials to highly accurate predictions of potential energy surfaces and energy-conserving force fields. 

Q5. What could be used to understand the formation and distribution of defects?

2. In solids, such local chemical potentials could be used to understand the formation and distribution of defects, such as vacancies and interstitials. 

Q6. What is the function of the filter-generating network?

The filter-generating network determines how interactions between atoms are modeled and can be used to constrainthe model and include chemical knowledge. 

Q7. What is the cost of a training step?

Since there are a maximum number of atoms being located within a given cutoff, the computational cost of a training step scales linearly with the system size if the authors precompute the indices of nearby atoms. 

Q8. How many epochs does SchNet require to converge?

the model requires much less epochs to converge, e.g., using 110k training examples reduces the required number of epochs from 2400 with T = 2 to less than 750 epochs with T = 6. 

Q9. Why do the authors need to use the PBCs to describe the interactions of atoms?

Due to the linearity of the convolution, the authors are, therefore, able to apply the PBCs directly to the filter to accurately describe the atom interactions while keeping invariance to the choice of the unit cell. 

Q10. How many examples do you use for the training set?

The authors use 20k C20 reference calculations as the training set, 4.5k examples for early stopping, and report the test error on the remaining data. 

Q11. How many examples are used to predict bulk crystal formation energies?

The authors employ SchNet to predict formation energies for bulk crystals using 69 640 structures and reference calculations from the Materials Project (MP) repository. 

Q12. What is the effect of normalizing the filter response xl+1i?

The authors find that the training is more stable when normalizing the filter response xl+1i by the number of atoms within the cutoff range. 

Q13. How does SchNet perform on the small training set?

SchNet performs better while using the combined loss with energies and forces on 1000 reference calculations than training on energies of 50 000 examples. 

Q14. What is the effect of the filter-generating network on the properties of diamond and graphite?

While the authors have followed a data-driven approach where the authors only incorporate basic invariances in the filters, careful design of the filter-generating network provides the possibility to incorporate further chemical knowledge in the network. 

Q15. What is the distribution of the errors of all predicted properties?

The distributions of the errors of all predicted properties are shown in Appendix A. Extending SchNet with interpretable, property-specific output layers, e.g., for the dipole moment,57 is subject to future work. 

Q16. How many training examples are used to test the SchNet model?

the data are randomly split into 60 000 training examples, a validation set of 4500 examples and the remaining data as test set.