Predicting impurity spectral functions using machine learning

The Anderson Impurity Model (AIM) is a canonical model of quantum many-body physics. Here we investigate whether machine learning models, both neural networks (NN) and kernel ridge regression (KRR), can accurately predict the AIM spectral function in all of its regimes, from empty orbital, to mixed valence, to Kondo. To tackle this question, we construct two large spectral databases containing approximately 410 000 and 600 000 spectral functions of the single-channel impurity problem. We show that the NN models can accurately predict the AIM spectral function in all of its regimes, with pointwise mean absolute errors down to 0.003 in normalized units. We find that the trained NN models outperform models based on KRR and enjoy a speedup on the order of ${10}^5$ over traditional AIM solvers. The required size of the training set of our model can be significantly reduced using farthest point sampling in the AIM parameter space, which is important for generalizing our method to more complicated multichannel impurity problems of relevance to predicting the properties of real materials.

Figure 1

Cartoon of a typical Anderson impurity model with a set of physical input parameters (left). The physical properties are fed as training data into a neural network (center), which then learns to predict the spectral function (right).

Figure 2

Principal component analysis performed on the Kondo (top) and Anderson (bottom) spectral data sets. In each of the four columns the same PCA data are colored in terms of different parameters, left to right: temperature, Kondo temperature, magnetic field strength, and $E_0$.

Figure 3

Representative ground truth (black) and MLP-predicted (red) spectral functions from the testing set, ${\mathcal{T}}^{\mathrm{A}},$ from the model trained on ${\mathcal{R}}^{\mathrm{A}}.$ Data correspond to the best example of each of the five pentiles of the data top to bottom, respectively. The system parameters $x_p\equiv {(U,\mathrm{\Gamma },{\varepsilon }_{\mathrm{d}},B,T)}_p$ are specified within each panel. As the algorithms have no notion of the $\omega$-grid on which the spectra are defined, we present the “ML grid” (left) which represents the spectral functions, $y_p=\pi {\mathrm{\Gamma }}_p{A}_p$, on a uniformly spaced grid (emphasizing how the algorithms “see” the targets).

Figure 4

SPE histograms for the Kondo data set. The original on-grid data (blue) of 1.45 million trials was supplemented by an additional 329 000 off-grid trials (green) to ensure a more even $E_0$ distribution on a log-scale (left panel). Trials with SPE values outside our range of interest $E_0\in [{10}^{-4},{10}^{-1}]$ were discarded, which gave rise to a final SPE distribution (right panel). Bin widths are the same in both panels.

Figure 5

SPE histogram of 411 267 data point Kondo set on linear and log-scale (left and right panel respectively), with approximately equal distribution across trials at lower energies only where $E_0$ is dominated by $\left|B\right|$, $T$, or $T_K$. Regularly spaced spikes in right panel are due to the on-grid portion of the Kondo set.

Figure 6

SPE histogram of 599 578 data point Anderson set on linear and log-scale (left and right panel respectively), with approximately equal distribution across trials where $E_0$ is dominated by $\left|B\right|$, $T$, or $T_K$.

Figure 7

The input feature distribution for the full Anderson set is presented in black. Elements from the down-sampled subsets of the farthest-point and random-point algorithms each representing approximately 10% of the full set are superimposed in color at 10% opacity to exhibit trial density. The top row shows the farthest-point distribution in green while the bottom row shows the random points in red. See Secs. A.3 and A.4 for additional information regarding the symmetric logarithm $\left({\mathrm{symlog}}_{10}\right)$ procedure and farthest-point sampling algorithm, respectively.

Figure 8

The input feature distribution for the full Kondo set (black). The down-sampled subsets each representing about 10% of the full data set are superimposed in color at 10% opacity to demonstrate density. The top row shows the farthest-point algorithm's subset in green, while the bottom row has the random point subset in red. See Secs. A.3 and A.4 for additional information regarding the symmetric logarithm $\left({\mathrm{symlog}}_{10}\right)$ procedure and farthest-point sampling algorithm, respectively. The horizontal line at ${\mathrm{symlog}}_{10}\left(B\right)=-8$ in the rightmost panels is caused by setting our $B\equiv 0\to {10}^{-8}$ as explained in Sec. pp1-s3.

Figure 9

DC-KRR $R^2$ values for each subset within the Anderson (squares) and Kondo (circles) ${\mathcal{R}}_{\mathrm{F}}$ sets. Closed markers indicate the validation $R^2$ scores, and open markers indicate that of the test sets. Final DC-KRR values are the averages across all relevant subsets.

Figure 10

(a) MLP training and validation losses and learning rates plotted as a function of epochs for the Anderson set. (b) Histogram for MLP errors as measured by the average MAE values, $\overline{\delta y}$. Black represents the errors when trained with the full set ${\mathcal{R}}^{\mathrm{A}}$; red represents the errors when trained with the farthest-point-sampled subset ${\mathcal{R}}_{\mathrm{f}}^{\mathrm{A}}$; blue represents the errors when trained with the random-sampled subset ${\mathcal{R}}_{\mathrm{r}}^{\mathrm{A}}$.

Figure 11

(a) MLP training and validation losses and learning rates plotted as a function of epochs for the Kondo set. (b) Histogram for MLP errors as measured by the average MAE values, $\overline{\delta y}$. Black represents the errors when trained on the full Kondo set ${\mathcal{R}}^{\mathrm{K}}$; red represents the errors with the model trained with the farthest-point-sampled subset ${\mathcal{R}}_{\mathrm{f}}^{\mathrm{K}}$; blue represents the model errors when trained with the random-sampled subset ${\mathcal{R}}_{\mathrm{r}}^{\mathrm{K}}$.

Figure 12

Random samples from the Anderson testing set from the MLP models trained on different training sets: (a) the full Anderson set; (b) the randomized subset; (c) the farthest-point sampling subset.

Figure 13

Random samples from the Kondo testing set from the MLP models trained on different training sets: (a) the full Kondo set; (b) the randomized subset; (c) the farthest-point sampling subset.

Figure 14

(a) Anderson and (b) Kondo testing set histograms for the the KRR models trained with the random ${\mathcal{R}}_{\mathrm{r}}$ and farthest-point-ordered ${\mathcal{R}}_{\mathrm{f}}$ down-sampled subsets in red and blue, respectively. The DC-KRR model trained with the chunked farthest-point ordered set ${\mathcal{R}}_{\mathrm{F}}$ is shown in black.

Figure 15

Random samples from the Anderson testing set from the KRR models for (a) the random subset ${\mathcal{R}}_{\mathrm{r}}$, (b) the farthest-point subset ${\mathcal{R}}_{\mathrm{f}}$, and (c) the DC-KRR model with all trials ordered according to the farthest-point algorithm ${\mathcal{R}}_{\mathrm{F}}$.

Figure 16

Random samples from the Kondo testing set from the KRR models for (a) the random subset ${\mathcal{R}}_{\mathrm{r}}$, (b) the farthest point subset ${\mathcal{R}}_{\mathrm{f}}$, and (c) the DC-KRR model with all trials ordered according to the farthest point algorithm ${\mathcal{R}}_{\mathrm{F}}$.