Training biases in machine learning for the analytic continuation of quantum many-body Green's functions

We address the problem of analytic continuation of imaginary-frequency Green's functions, which is crucial in many-body physics, using machine learning based on a multilevel residual neural network. We specifically address potential biases that can be introduced due to the use of artificially created spectral functions that are employed to train the neural network. We also implement an uncertainty estimation of the predicted spectral function, based on Monte Carlo dropout, which allows us to identify frequency regions where the prediction might not be accurate, and we study the effect of noise, in particular also for situations where the noise level during training is different from that in the actual data. Our analysis demonstrates that this method can indeed achieve a high quality of prediction, comparable to or better than the widely used maximum entropy method, but that further improvement is currently limited by the lack of true data that can be used for training. We also benchmark our approach by applying it to the case of ${\mathrm{SrVO}}_3$, where an accurate spectral function has been obtained from dynamical mean-field theory using a solver that works directly on the real frequency axis.

Figure 1

The NN architecture. The complete structure on the left, a more detailed view of the “multilevel residual block” in the center, and the basic residual unit, labeled “100 selu,” on the right. This network architecture allows for accurate training while suppressing saturation effects with a manageable number of trainable parameters.

Figure 2

The covariance matrix $K_{l{l}^{\prime }}$ corresponding to the imaginary-time Green's function in the Legendre basis, $G_l$, for $l,l^{\prime }<30$, evaluated from 200 independent QMC runs on top of a preconverged $\mathrm{DFT}+\mathrm{DMFT}$ calculations for ${\mathrm{SrVO}}_3$.

Figure 3

Distribution of the MAE corresponding to the predicted spectral functions with different noise levels applied to the training and testing data, ${\sigma }_{\text{train}}$ and ${\sigma }_{\text{test}}$, respectively. The histogram in blue represents the distribution of the MAE, and the dashed line shows its average over all test spectral functions. The value of the average MAE is also indicated in the legends of the individual panels.

Figure 4

Comparison of the MAE for 400 spectral functions predicted by the NN (${\overline{\varepsilon }}_{\text{NN}}$) and by MaxEnt (${\overline{\varepsilon }}_{\text{MaxEnt}}$), respectively, for different noise levels, $\sigma$, applied to both testing (for the NN) and training data.

Figure 5

The pointwise absolute error $\varepsilon \left(\omega \right)$ plotted over the standard deviation from the MC dropout, ${\sigma }_{\text{Dropout}}$ for a noise level of $\sigma ={10}^{-3}$. The dashed line represents the case where $\varepsilon ={\sigma }_{\text{Dropout}}$, whereas the solid line, $\varepsilon =10{\sigma }_{\text{Dropout}}$, represents the value used for our empirical uncertainty estimation.

Figure 6

Examples of spectral functions constructed from data recipe 1. Each panel contains the ground truth, the NN prediction with the estimated uncertainty range, and the MaxEnt prediction. Predicted spectral functions are obtained with a noise level of $\sigma ={10}^{-3}$ on $G_l$.

Figure 7

Comparison between the NN and MaxEnt predictions on a special set of spectral functions not represented by Eq. (2), for a noise level of $\sigma ={10}^{-3}$. Cases (a)–(c) correspond to wider plateaus, cases (d)–(f) to asymmetric peaks, and (g)–(i) to Lorentzian peaks. The values in the top of each figure indicate the corresponding MAE for NN and MaxEnt, respectively.

Figure 8

Spectral functions obtained with the NN trained on (a) data recipe 1 (NN1) and (b) data recipe 2 (NN2), together with the MaxEnt prediction for the Green's function obtained from a $\mathrm{DFT}+\mathrm{DMFT}$ calculation for ${\mathrm{SrVO}}_3$ using the FTPS solver (with an artificial noise of ${10}^{-3}$ added to the forward-transformed FTPS data). The ground truth is the result obtained by the FTPS solver without analytic continuation.

Figure 9

Examples of spectral functions constructed from data recipe 2. Each panel contains the ground truth, the NN prediction with the estimated uncertainty range, and the MaxEnt prediction. Predicted spectral functions are obtained with a noise level of $\sigma ={10}^{-3}$ on $G_l$.

Figure 10

Cross-validation of the NNs trained on the two different data recipes. The top row shows the performance of NN2 on spectral functions generated according to data recipe 1. The bottom row shows the performance of NN1 on spectral functions generated according to data recipe 2. In both cases the MAE of the NN is compared to that achieved by MaxEnt on the same data.