Catalog variance of testing general relativity with gravitational-wave data

Combining multiple gravitational-wave observations allows for stringent tests of general relativity, targeting effects that would otherwise be undetectable using single-event analyses. We highlight how the finite size of the observed catalog induces a significant source of variance. If not appropriately accounted for, general relativity can be excluded with arbitrarily large credibility even if it is the underlying theory of gravity. This effect is generic and holds for arbitrarily large catalogs. Moreover, we show that it cannot be suppressed by selecting “golden” observations with large signal-to-noise ratios. We present a mitigation strategy based on bootstrapping (i.e. resampling with repetition) that allows assigning uncertainties to one’s credibility on the targeted test. We demonstrate our findings using both toy models and real gravitational-wave data. In particular, we quantify the impact of the catalog variance on the ringdown properties of black holes using the latest LIGO/Virgo catalog.

Figure 1

Cumulative distribution function of $Q_0$ for three sets of catalog realizations. We consider Gaussian likelihoods with homoscedastic errors (solid), heteroscedastic errors (dashed), and heteroscedastic with an SNR cut (dash-dotted). Orange and blue curves are produced by either assuming the null hypothesis ${\mu }_{\mathrm{true}}=0={\sigma }_{\mathrm{true}}^2=0$ or drawing $({\mu }_{\mathrm{true}},{\sigma }_{\mathrm{true}}^2)$ from their prior, respectively.

Figure 2

Distribution of Bayes factors ${\log }_{10}{\mathcal{B}}_{\star }$ and GR quantiles $Q_0$ over 1000 bootstrapped realizations of the flagship pseobnr catalog test. Blue crosses and red hexagons indicate constraints on the ringdown frequency $\delta {\widehat{f}}_{220}$ and damping time $\delta {\widehat{\tau }}_{220}$, respectively. In the marginalized histograms, colored triangles indicate 90% confidence intervals. Black markers indicate the values of ${\log }_{10}{\mathcal{B}}_{\star }$ and $Q_0$ corresponding to the original catalog; this is just one possible realization among many.