Statistical evaluation of experimental determinations of neutrino mass hierarchy

Statistical methods of presenting experimental results in constraining the neutrino mass hierarchy (MH) are discussed. Two problems are considered and are related to each other: how to report the findings for observed experimental data and how to evaluate the ability of a future experiment to determine the neutrino mass hierarchy, namely, the sensitivity of the experiment. For the first problem where experimental data have already been observed, the classical statistical analysis involves constructing confidence intervals for the parameter $\Delta m_{32}^2$. These intervals are deduced from the parent distribution of the estimation of $\Delta m_{32}^2$ based on experimental data. Because of existing experimental constraints on $|\Delta m_{32}^2|$, the estimation of $\Delta m_{32}^2$ is better approximated by a Bernoulli distribution (a binomial distribution with one trial) rather than a Gaussian distribution. Therefore, the Feldman-Cousins approach needs to be used instead of the Gaussian approximation in constructing confidence intervals. Furthermore, as a result of the definition of confidence intervals, even if it is correctly constructed, its confidence level does not directly reflect how much one hypothesis of the MH is supported by the data rather than the other hypothesis. We thus describe a Bayesian approach that quantifies the evidence provided by the observed experimental data through the (posterior) probability that either hypothesis of MH is true. This Bayesian presentation of observed experimental results is then used to develop several metrics to assess the sensitivity of future experiments. Illustrations are made by using a simple example with a confined parameter space, which approximates the MH determination problem with experimental constraints on the $|\Delta m_{32}^2|$.

Figure 1

Distributions of $\Delta {\chi }_{\min }^2({\theta }_0)$ and ${\theta }_{\min }$ for case I and case II with 100 000 MC samples. The ${\theta }_{\min }$ distribution of case I (top right) and case II (bottom right) are a Gaussian and a Bernoulli distribution, respectively. The $\Delta {\chi }_{\min }^2({\theta }_0)$ distribution of case I (top left) is consistent with the chi-square distribution with degree of freedom 1. The commonly used $1\sigma$, (68.27% confidence level) and $2\sigma$, (95.45% confidence level) regions are labeled with red dashed and black dash-dotted lines, respectively, for case I. The $\Delta {\chi }_{\min }^2({\theta }_0)$ distribution of case II (bottom left) strongly deviates from the chi-square distribution. In case II, we also show the analytical approximation (derived in Appendix ) of the distribution of $\Delta {\chi }_{\min }^2$. We should emphasize that, while the chi-square distribution does not depend on any additional parameter (other than $\Delta {\chi }_{\min }^2$), the analytical approximation depends on $\overline{\Delta {\chi }^2}$.

Figure 2

The probability density functions $P(\Delta {\chi }^2|\mathrm{NH})$ and $P(\Delta {\chi }^2|\mathrm{IH})$ in the Bernoulli model are shown as the solid and dotted lines, respectively. The $|\overline{\Delta {\chi }^2}|$ is assumed to be 9.

Figure 3

The left panel shows the distribution of $P(\mathrm{NH}|x)=P(\mathrm{NH}|\Delta {\chi }^2)$ over the population of potential data $x$ that arises from an experiment with $\overline{\Delta {\chi }^2}=9$ where the truth is NH. The mean of this distribution is 90.14%. The lower bounds of the 68% and 90% probability intervals are plotted. That is, 68% (90%) of the data $x$ would yield a $P(\mathrm{NH}|x)$ that falls to the right of the dash-dotted (dashed) line. These two lines are also commonly referred as the 32nd and the 10th percentile, respectively. The right panel plots several sensitivity metrics (subtracted from 1 for clarity), against $\overline{\Delta {\chi }^2}$ that ranges from 1 to 50. Note that all the lines are decreasing because higher values of $\overline{\Delta {\chi }^2}$ correspond to more sensitive experiments. This is done for three different criteria: the Gaussian interpretation (derived from the one-sided $p$-value with 1 degree of freedom), $\overline{P}$, and $P_{T=\mathrm{NH}}^{90\%}$. The Gaussian interpretation is seen to be overoptimistic in describing the ability of the experiment to differentiate the two hypotheses.