Direct Comparison between Bayesian and Frequentist Uncertainty Quantification for Nuclear Reactions

Until recently, uncertainty quantification in low energy nuclear theory was typically performed using frequentist approaches. However in the last few years, the field has shifted toward Bayesian statistics for evaluating confidence intervals. Although there are statistical arguments to prefer the Bayesian approach, no direct comparison is available. In this work, we compare, directly and systematically, the frequentist and Bayesian approaches to quantifying uncertainties in direct nuclear reactions. Starting from identical initial assumptions, we determine confidence intervals associated with the elastic and the transfer process for both methods, which are evaluated against data via a comparison of the empirical coverage probabilities. Expectedly, the frequentist approach is not as flexible as the Bayesian approach in exploring parameter space and often ends up in a different minimum. We also show that the two methods produce significantly different correlations. In the end, the frequentist approach produces significantly narrower uncertainties on the considered observables than the Bayesian. Our study demonstrates that the uncertainties on the reaction observables considered here within the Bayesian approach represent reality more accurately than the much narrower uncertainties obtained using the standard frequentist approach.

Figure 1

Elastic scattering for neutrons and protons on ${}^{48}\mathrm{Ca}$. Top row [panels (a),(b),(c)]: the predicted 95% confidence intervals from Bayesian (orange vertical hash) is compared with the frequentist approach (blue slanted hash), and to experimental data. Middle row [panels (d),(e),(f)]: percent uncertainty of the confidence intervals. Bottom row [panels (g),(h),(i)]: comparison of the percentage of data falling within the given confidence interval. First column, ${}^{48}\mathrm{Ca}(n,n)$ at 12 MeV; second column, ${}^{48}\mathrm{Ca}(p,p)$ at 14 MeV; third column, ${}^{48}\mathrm{Ca}(p,p)$ at 25 MeV.

Figure 2

Posterior distributions for the parameters (diagonal) and scatter plots for the correlations between parameters: Bayesian are shown in shades of orange and frequentist in shades of blue: ${}^{48}\mathrm{Ca}(n,n)$ at 12 MeV (a); ${}^{48}\mathrm{Ca}(p,p)$ at 14 MeV (b); and ${}^{48}\mathrm{Ca}(p,p)$ at 25 MeV (c). Depths (V, Ws, W) are in MeV and radii (r, rs, rw) and diffuseness (a, as, aw) are in fm.

Figure 3

Transfer cross sections for ${}^{48}\mathrm{Ca}(d,p)$: the predicted 95% confidence intervals from the Bayesian (orange vertical hash) and the frequentist approach (blue slashed hash) [panel (a)]; percent uncertainty of the confidence intervals [panel (b)]; comparison of the percentage of data falling within the given confidence interval [panel (c)].