Efficient estimation of material property curves and surfaces via active learning

The relationship between material properties and independent variables such as temperature, external field, or time is usually represented by a curve or surface in a multidimensional space. Determining such a curve or surface requires a series of experiments or calculations which are often time and cost consuming. A general strategy uses an appropriate utility function to sample the space to recommend the next optimal experiment or calculation within an active learning loop. However, knowing what optimal sampling strategy to use to minimize the number of experiments is an outstanding problem. We compare a number of strategies based on directed exploration on several materials problems of varying complexity using a Kriging-based model. These include one-dimensional curves such as the fatigue life curve for 304L stainless steel and the Liquidus line of the Fe-C phase diagram, surfaces such as the Hartmann 3 function in three-dimensional space and the fitted intermolecular potential for Ar-SH, and a four-dimensional data set of experimental measurements for ${\mathrm{BaTiO}}_3$-based ceramics. We also consider the effects of experimental noise on the Hartmann 3 function. We find that directed exploration guided by maximum variance provides better performance overall, converging faster across several data sets. However, for certain problems, the tradeoff methods incorporating exploitation can perform at least as well, if not better than maximum variance. Thus, we discuss how the choice of the utility function depends on the distribution of the data, the model performance and uncertainties, additive noise, as well as the budget.

Figure 1

Flow chart of active learning strategy for efficient estimation of material property curves. (1) A small, labeled training data serves as the starting point for the targeted curve. (2) A surrogate model using machine learning methods provides an estimate of the fit with uncertainties. (3) Several criteria are used to check if the estimated fit is adequate and fulfills conditions for success. If not, (4) utility functions are evaluated and ranked to recommend the next data point to use from the pool of possible points for measurement or calculation to determine the label. (5) The recommended experiment or calculation is performed and the new labeled point augments the training data to obtain a revised estimate of the curve. The loop continues until the criteria for success are met.

Figure 2

Comparison of the performance of utility functions in optimizing the fatigue life curve for SS304L steel. The initial data size contains $n=5$ training points and the mean values and error bars showing the 95% confidence levels of the points are evaluated over 100 trials. Shown is the behavior of (a) mean absolute error and (b) maximum absolute error.

Figure 3

Comparison of the performance of utility functions in optimizing the Liquidus line of the Fe-C phase diagram. The initial data size contains $n=5$ training points and the mean values and error bars showing the 95% confidence intervals of the points are evaluated over 100 trials. Shown is the behavior of (a) mean absolute error and (b) maximum absolute error.

Figure 4

Comparison of the performance of utility functions in optimizing the targeted Hartmann 3 function. The initial data size contains $n=80$ training points and the mean values and error bars showing 95% confidence intervals of the points are evaluated over 100 trials. Shown is the behavior of (a) mean absolute error and (b) maximum absolute error.

Figure 5

The number of new measurements needed to achieve a given accuracy (3% and 1% of the $y$ range) for the curve as a function of the training data size [(a) and (b)]. Each optimization process is repeated 100 times. The points with the error bar are the mean values associated with 95% confidence levels over 100 trials. The probability density of the difference in the number of iterations using EGO and Max-v. Plotted along the $x$ axis is $N_{\text{EGO}}-N_{\text{Max-v}}$.

Figure 6

Comparison of the performance of utility functions in optimizing the targeted Hartmann 3 function subject to noise levels of 5%, 10%, and 15% corresponding to (a)–(c). The initial data size contains $n=80$ training points and the mean values and error bars showing the 95% confidence intervals of the points are evaluated over 100 trials. Shown is the behavior of mean absolute error (MAE).

Figure 7

Comparison of the performance of utility functions in optimizing the Ar-SH potential energy surface. Plotted is the deviation of the model prediction from that obtained using measurements fitted to the Schrödinger equation. The initial data size contains $n=52$ training points and the mean values and error bars showing 95% confidence intervals of the points are evaluated over 100 trials. Shown is the behavior of (a) mean absolute error and (b) maximum absolute error.

Figure 8

Comparison of the performance of utility functions in optimizing the targeted Curie temperature for ${\mathrm{BaTiO}}_3$-based ceramics. The initial data size contains $n=55$ training points and the mean values and error bars showing the 95% confidence intervals of the points are evaluated over 100 trials. Shown is the behavior of (a) mean absolute error and (b) maximum absolute error.

Figure 9

For the Liquidus line of the Fe-C phase diagram, the probability density functions (PDF) of the uncertainties and deviation from true result associated with the Kriging model prediction for Random, Max-v, B.EGO, and EGO are shown with successive iterations.

Figure 10

For the fatigue life curve for SS304L steel, the probability density functions (PDF) of the uncertainties and deviation from true result associated with the Kriging model prediction for Random, Max-v, B.EGO, and EGO are shown with successive iterations.

Figure 11

Typical solutions and the probability density functions (PDF) of the property values $y$ for the intermolecular potential energy surface (PES) and the FeC phase boundary data sets. The dark blue line represents the model prediction and the red line the true solution.