Machine-learning approach to finite-size effects in systems with strongly interacting fermions

We investigate the applicability of machine learning techniques in studying the finite-size effects associated with many-body physics. These techniques have an emerging presence in many-body theory as they have been used for interpolations, extrapolations, and in modeling wave functions. We will resolve several issues associated with machine learning and many-body calculations such as small datasets, outliers, and discontinuities, for the purpose of extrapolating finite calculations to macroscopic scales. We carry out a systematic investigation of two related systems by developing metrics that aim to avoid spurious effects and capture desired features. This work uses neural networks to extrapolate the unitary gas to the thermodynamic limit at zero range, which is otherwise difficult to reach. The effective mass of strongly interacting neutron matter is also studied and makes use of the noninteracting problem to resolve discontinuous predictions. For this investigation, we also carried out new auxiliary field diffusion Monte Carlo (AFDMC) calculations for a variety of densities and particle numbers. Ultimately, we demonstrate an effective utility for neural networks in this context.

Figure 1

The unitary gas dataset plotted along with three types of linear fits given by Eq. (5), (6), and (7), which correspond to the “individual,” “all,” and “average” fits, respectively. The effective-range dependence is strongly linear while the particle-number dependence is more complex. At $N=70$ and $N=80$, the slopes of the different fits notably disagree almost exclusively at these points, which suggests that they may be spurious.

Figure 2

The dependence of the UG predictions on the dataset weightings is shown for three different datasets. These datasets consider all points included (left panel), an arbitrary pair removed (central panel), and the outliers removed (right panel). The TL energy estimates at 0 range are shown as a function of the dataset weighting $t$. In general, the estimates agree within a smaller margin when $t$ corresponds to the original dataset (emphasis on low $N$), but diverge when $t$ corresponds to the reflected dataset (emphasis on larger $N$ and consequently the skewed slopes associated with the outliers).

Figure 3

To verify that the pair of outliers identified in Fig. 2 has a unique effect we study the removal of different pairs from the dataset. Networks are trained on different datasets, each with an arbitrary pair removed. The spread is averaged over predictions for pairs containing $N$, which produces the values shown in the figure, according to Eq. (12). There is a significant reduction at both $N=70$ and $N=80$, which suggests that the outliers do uniquely skew the predictions.

Figure 4

The relative fivefold cross-validation error for various hyperparameters is shown, where darker color indicates poorer performance. There are three parameters under consideration: each subplot contains a different dataset size $F$, with the number of training epochs along the bottom, and the number of hidden units along the vertical. In all cases, the performance stops improving after about 50 hidden units. For dataset sizes that are sufficiently large, the performance plateaus after about 5000 epochs. The consistent improvements resulting from increasing the dataset size has a confounding effect, which is discussed in the main text.

Figure 5

A set of predictions from optimized neural networks (solid bands) trained on different dataset sizes, $F$, ordered in the caption the same way as in the plot itself at $N=100$. These are plotted alongside the original dataset (dotted) which is linearly extrapolated to 0 range. The outliers at $N=70$ and $N=80$ are hollow to denote that they were removed, as discussed in Sec. 3d. The kink at $N\approx 140$ in the black curve (solid) is evidence of overfitting; meanwhile the inability of the $F=1000$ curve (forward slash) to capture the peak at low $N$ is evidence of underfitting. The prediction from $F=2000$ (backward slash) is thus selected as the optimal network.

Figure 6

Interpolations predicted by the networks for the extrapolated quasiparticle energy $\mathrm{\Delta }{E}_{\text{TL}}^{\left(k_{\text{TL}}\right)}$ (top panel) and extrapolated momentum $k_{\text{TL}}^2$ (bottom panel) expressed in MeV, at a density of $0.05{\mathrm{fm}}^{-3}$. Both networks are trained on the same domain for consistency, although we show more points for the momentum since it is analytic. In both panels, the predictions (dashed) and calculated values (dots, solid) for the first four excited states are shown, increasing from bottom to top. Both extrapolated quantities have a similar dependence, which is generally captured by the networks. As expected, the discontinuities are not captured. However, since both models experience this same bias, the discontinuous predictions are resolved in the fit calculation for the effective mass.

Figure 7

The effective masses predicted by neural networks at a density of 0.05 ${\mathrm{fm}}^{-3}$ are contrasted for two fit methods. These predictions constitute an interpolation, since this density was not included during training. In the top panel, the energy predictions are fit to the analytic momentum, which results in discontinuities. In the bottom panel, the energy predictions are fit with predictions from a separate network that was trained on the momentum, which resolves the discontinuities. The smoothing effects shared by both network predictions eliminate the discontinuities in the fits. At low particle numbers, the effective-mass predictions are more sensitive to the large energy/momentum values (shown in Fig. 6) which results in more variability. At high $N$, the predictions begin struggling to extrapolate and must contend with the larger QMC error.

Figure 8

The effective-mass predictions $m^{*}/m$ as a function of the disagreement factor $D$ for a density of 0.04 ${\mathrm{fm}}^{-3}$ (left) and 0.18 ${\mathrm{fm}}^{-3}$ (right) for networks with various hyperparameter settings. This contrasts with the effect of increasing density; the networks at a lower density are more variable than those at higher densities. This is likely due to the reduced energy scale, which requires increased precision. Regardless of the density, there is a common trend that the network predictions limit towards some value as the disagreement factor decreases. At each density, the bottom 10% (the rightmost portion) are selected to calculate an effective-mass statistic, as done for Fig. 9. These networks averaged their predictions over the domain $N\in [0,130]$.

Figure 9

Network predictions for the effective mass $m^{*}/m$ as a function of number density $n$ along with raw AFDMC calculations for $N=66$. The blue dashed trend (which matches the raw calculation at the lowest density) shows predictions from networks with a single hidden layer, while the orange dotted trend uses networks with two equally sized hidden layers. These predictions were averaged over the domain $N\in [0,130]$. Overall, the agreement between the networks is quite good.