Cosmological discordances: A new measure, marginalization effects, and application to geometry versus growth current data sets

The continuous progress toward more precise cosmological surveys and experiments has galvanized recent interest into consistency tests on cosmological parameters and models. At the heart of this effort is quantifying the degree of inconsistency between two or more cosmological data sets. We introduce an intuitive moment-based measure we call the index of inconsistency (IOI) and show that it is sensitive to the separation of the means, the size of the constraint ellipsoids, and their orientations in the parameter space. We find that it tracks accurately the inconsistencies when present. Next, we show that parameter marginalization can cause a loss of information on the inconsistency between two experiments, and we quantify such a loss using the drop in IOI. In order to zoom on a given parameter, we define the relative residual IOI and the relative drop in IOI. While these two quantities can provide insights on the parameters that are most responsible for inconsistencies, we find that the full IOI applied to the whole parameter spaces is what must be used to correctly reflect the degree of inconsistency between two experiments. We discuss various properties of IOI, provide its eigenmode decomposition, and compare it to other measures of discordance. Finally, we apply IOI to current geometry data sets (i.e., an improved Supernovae Type Ia compilation, baryon acoustic oscillations from 6dF, SDSS MGS and Lyman-$\alpha$ forest, and high-$\ell$ cosmic microwave background (CMB) temperature data from Planck-2015) versus growth data sets (i.e., Redshift Space Distortions from WiggleZ and SDSS, Weak Lensing from CFHTLenS, CMB Lensing, Sunyav-Zeldovich effect, and low-$\ell$ CMB temperature and polarization data from Planck-2015). We find that a persistent inconsistency is present between the two data sets. This could reflect the presence of systematics in the data or inconsistencies in the underlying model.

Figure 1

Two toy one-dimensional Gaussian probability density distributions (red and blue curves) and their joint distribution (black dotted curves).

Figure 2

Examples of IOIs. The centers of the ellipses are the likelihood maxima. The extra (blue) point is the likelihood maximum of the joint likelihood. The two in the middle have the same deviation of likelihood maxima, but they do not have the same IOI because of the different degenerate directions. The same is also true for the two on the right.

Figure 3

An example of a non-Gaussian case. $P_{(2)}(\lambda )$ given by experiment 2 is Gaussian, while $P_{(1)}(\lambda )$ given by experiment 1 is slightly different from Gaussian. The narrow peak of $P_{(1)}(\lambda )$ should be ignored when we compare the two experiments. Using $\mathrm{\Delta }{\chi }^2$ as a measure of inconsistency only considers the maximum that may have insignificant probability weight, but ignores the overall distribution. The moment-based IOI cares about the distribution as a whole.

Figure 4

Two toy experiments with different degenerate directions. As we explain in the text, inconsistency between experiments should be treated separately from the small overlapping area caused by degeneracy breaking.

Figure 5

Toy experiments to show marginalization artificially lowers inconsistency. The left panel shows the 2D contour plot of the posteriors of two toy experiments. The left panel shows the 1D probability distributions of the two experiments marginalized over the second parameter. Clearly the marginalized 1D plots have more overlapped region than the 2D plot, indicating an artificial decrease of inconsistency between the two experiments. The decrease of the 1D IOI also reflects such a fact but one needs to consider rather the full IOI on the left.

Figure 6

Toy experiments to show IOI can be smaller or larger for model extension. Model A is nested in model B, and model A has a fixed value of ${\lambda }_2={\lambda }_2^A=0$. In the left (right) panel model B has a larger (smaller) IOI than model A. See the text for a detailed explanation.

Figure 7

Demonstration of the problem of the tension in a non-Gaussian case. Upper panel: two original one-dimensional probability distributions. The one [$P_{(1)}(\lambda )$] given by experiment 1 is non-Gaussian, while the other [$P_{(2)}(\lambda )$] is Gaussian. Lower panel: the distribution $P_{(2)}(\lambda )$ is shifted to coincide its maximum with that of $P_{(1)}(\lambda )$. Because of the small overlap of the shifted distributions in the lower panel, the numerator in Eq. (36) is smaller than the denominator, which makes $\ln \mathcal{T}$ negative.

Figure 8

Two sets of toy experiments described by Eqs. (52) and (53) (left panel) and Eqs. (54) and (55) (right panel). Experiment 1 is described by contours in black, and experiment two in red. The two experiments in the right panel seems to have a larger inconsistency, which is correctly described by the increase of IOI. Using different argument orders in the surprise leads to opposite conclusions about the inconsistency in the right panel.

Figure 9

The posterior probability contours of the two experiments described by (56) (left panel) and their marginalized 1D distribution over ${\lambda }_2$ (right panel). The surprise before marginalization is $S({𝒫}^{(2)}||{𝒫}^{(1)})=0$, but after marginalization the surprise is $S_1=1$ with a significance 0.7. However, we argued in Sec. 5 that ${\lambda }_2$ should not be relevant to the inconsistency; a measure of the inconsistency should be preserved after marginalization over ${\lambda }_2$.

Figure 10

The marginalized 1D probability distributions and 2D likelihood contour plots for the $\mathrm{\Lambda }\mathrm{CDM}$ model constrained by the geometry versus the growth data sets. Black is for the geometry set and red for the growth set. The two relative measures of one-parameter inconsistency [${\mathrm{IOI}}_i^{\text{Rel}}$ and $\mathrm{\Delta }{\mathrm{IOI}}_i^{\text{Rel}}$; see Eq. (18) for their definitions] are shown on top of each 1D marginalized plot. While these relative IOI measures can indicate the parameter(s) most associated with the inconsistency, the important number here is the value of the full IOI shown at the upper right corner of the figure. Note that summation of all individual ${\mathrm{IOI}}_i^{\text{Rel}}$’s or $\mathrm{\Delta }{\mathrm{IOI}}_i^{\text{Rel}}$’s is generally not 1. The full IOI between the two data sets is 4.34 and indicates a moderate inconsistency on Jeffreys’ scales (see Table 3).

Figure 11

Marginalized 1D probability distributions and 2D likelihood contour plots in a triangular displacing diagram for the $w\mathrm{CDM}$ model constrained by the geometry and growth sets of experiments. The structure of this figure is the same as Fig. 10. Again, the important number here is the value of the full IOI shown at the upper right corner of the figure. Note that summation of all individual ${\mathrm{IOI}}_i^{\text{Rel}}$’s or $\mathrm{\Delta }{\mathrm{IOI}}_i^{\text{Rel}}$’s is generally not 1. The full IOI between the two data sets is 5.34 for $w\mathrm{CDM}$ and indicates a strong inconsistency on Jeffreys’ scales (see Table 3).

Figure 12

An example of situation which satisfies the equality condition (b8). Left: A two-parameter model constrained by two experiments shown in black and red. Middle: The constraint of ${\lambda }_1$ marginalized over ${\lambda }_2$. Right: The constraint of ${\lambda }_2$ marginalized over ${\lambda }_1$. In this particular example, the condition $\frac{C_{12}}{C_{11}}{\delta }_1={\delta }_2$. If we marginalize over ${\lambda }_2$, IOI remains the same, i.e., $\mathrm{IOI}={\mathrm{IOI}}_1$ even though ${\mathrm{IOI}}_2\ne 0$.