Including crystal structure attributes in machine learning models of formation energies via Voronoi tessellations

While high-throughput density functional theory (DFT) has become a prevalent tool for materials discovery, it is limited by the relatively large computational cost. In this paper, we explore using DFT data from high-throughput calculations to create faster, surrogate models with machine learning (ML) that can be used to guide new searches. Our method works by using decision tree models to map DFT-calculated formation enthalpies to a set of attributes consisting of two distinct types: (i) composition-dependent attributes of elemental properties (as have been used in previous ML models of DFT formation energies), combined with (ii) attributes derived from the Voronoi tessellation of the compound's crystal structure. The ML models created using this method have half the cross-validation error and similar training and evaluation speeds to models created with the Coulomb matrix and partial radial distribution function methods. For a dataset of 435 000 formation energies taken from the Open Quantum Materials Database (OQMD), our model achieves a mean absolute error of 80 meV/atom in cross validation, which is lower than the approximate error between DFT-computed and experimentally measured formation enthalpies and below 15% of the mean absolute deviation of the training set. We also demonstrate that our method can accurately estimate the formation energy of materials outside of the training set and be used to identify materials with especially large formation enthalpies. We propose that our models can be used to accelerate the discovery of new materials by identifying the most promising materials to study with DFT at little additional computational cost.

Figure 1

MAE measured using cross validation of models created using the PRDF [42], CM [30], and the method presented in this paper. Each model was trained on the DFT formation energies of a set of randomly selected compounds from the ICSD and was used to evaluate 1000 distinct compounds that were also selected at random. The black, dashed line indicates the expected error from guessing the mean formation energy of the training set for all structures.

Figure 2

Formation enthalpy ($\mathrm{\Delta }{H}_f$) computed using DFT and predicted using ML with models created using the (a) CM [30], (b) PRDF [42], and (c) the method proposed in this paper. Each model was trained on the same set of 30 000 entries from the OQMD and evaluated against the same validation set of 1000 compounds. The results from the validation set are shown in this figure along with several different performance metrics.

Figure 3

Performance of ML models for formation enthalpy created with the same ML algorithm but different representations. Each graph shows MAE for (a) KRR model and (b) RF algorithm in a cross-validation test where the model was trained on progressively larger training sets and validated against a separate test set of 1000 entries. For each algorithm, we compare the performance using the Voronoi-tessellation-based representation proposed in this paper against the CM [30] and the PRDF [42] matrix representations.

Figure 4

Comparison of model training and running time of three different techniques to predict the formation energy of inorganic compounds: Coulomb Matrix (CM) [30], Partial Radial Distribution Function (PRDF) [42], and our Voronoi-tessellation-based method. Training time is the sum of attribute generation and model construction with given data. Run time is the average time taken to compute the required attributes and to evaluate the ML model for a single compound.

Figure 5

(a) The DFT-computed formation enthalpy of a compound compared to the MAE between the DFT and machine-learning-predicted formation enthalpy of that compound during a cross-validation test. The red, dashed line indicates the 98th percentile of the MAE. (b) Comparison of the fraction of compounds that contain a certain element in our ICSD training set $P\left(\mathrm{ICSD}\right)$ to the ratio between the fraction of compounds in the 98th percentile of MAE and the fraction in the training set.

Figure 6

Performance of ML models trained on different representations in cross-validation tests using data from the (a) ICSD subset of the OQMD and (b) the entire OQMD. These include models trained using all the attributes in our proposed representation and, separately, models created using only the composition-dependent terms and only the structure-dependent terms. The results of a model created using the Coulomb matrix representation with the random forest algorithm is shown for comparison. Shaded regions represent the 90% confidence intervals.

Figure 7

Comparison of formation enthalpies ($\mathrm{\Delta }{H}_f$) predicted using ML and computed using DFT. The ML model was trained on the formation enthalpies of all 435 792 nonduplicate entries from the OQMD. Each material was selected from a list of 12 667 entries from the ICSD that have yet to be included in the OQMD using three different strategies: (green squares) random selection, (blue diamonds) predictions with the lowest $\mathrm{\Delta }{H}_f$, and (red circles) with the largest, negative difference between the predicted $\mathrm{\Delta }{H}_f$ and the OQMD convex hull.

Figure 8

Comparison of the ability of different ML methods to rank different types of compounds based on DFT formation energy, measured using two different metrics. (a) The Kendall τ ranking correlation coefficient, which is based on how well the model ranks the entire dataset. A correlation value of 1.0 corresponds to perfect ranking. (b) The number of the 100 compounds with the lowest DFT formation energy that were predicted by the model to be within the lowest 100 compounds. Each model was trained on the DFT-predicted formation energy of 32 111 inorganic compounds from the ICSD. The solid bar indicates the ranking performance using the input structure provided to DFT. Black outline around each bar indicates the ranking accuracy when provided with the fully relaxed output from DFT.