Accuracy of binary black hole waveform models for aligned-spin binaries

Coalescing binary black holes are among the primary science targets for second generation ground-based gravitational wave detectors. Reliable gravitational waveform models are central to detection of such systems and subsequent parameter estimation. This paper performs a comprehensive analysis of the accuracy of recent waveform models for binary black holes with aligned spins, utilizing a new set of 84 high-accuracy numerical relativity simulations. Our analysis covers comparable mass binaries (mass-ratio $1\le q\le 3$), and samples independently both black hole spins up to a dimensionless spin magnitude of 0.9 for equal-mass binaries and 0.85 for unequal mass binaries. Furthermore, we focus on the high-mass regime (total mass $\gtrsim 50M_{\odot }$). The two most recent waveform models considered (PhenomD and SEOBNRv2) both perform very well for signal detection, losing less than 0.5% of the recoverable signal-to-noise ratio $\rho$, except that SEOBNRv2’s efficiency drops slightly for both black hole spins aligned at large magnitude. For parameter estimation, modeling inaccuracies of the SEOBNRv2 model are found to be smaller than systematic uncertainties for moderately strong GW events up to roughly $\rho \lesssim 15$. PhenomD’s modeling errors are found to be smaller than SEOBNRv2’s, and are generally irrelevant for $\rho \lesssim 20$. Both models’ accuracy deteriorates with increased mass ratio, and when at least one black hole spin is large and aligned. The SEOBNRv2 model shows a pronounced disagreement with the numerical relativity simulation in the merger phase, for unequal masses and simultaneously both black hole spins very large and aligned. Two older waveform models (PhenomC and SEOBNRv1) are found to be distinctly less accurate than the more recent PhenomD and SEOBNRv2 models. Finally, we quantify the bias expected from all four waveform models during parameter estimation for several recovered binary parameters: chirp mass, mass ratio, and effective spin.

Figure 1

Parameter space coverage of the simulations considered here. For mass ratio $q=\{1,2,3\}$ we indicate the spin components $({\chi }_1,{\chi }_2)$ projected onto the orbital angular momentum. Each point is color coded by the lowest total mass to which the waveform can se scaled, such that the initial GW frequency remains $\gtrsim 15\mathrm{Hz}$. In the $q=1$ panel, each simulation is plotted twice at $({\chi }_1,{\chi }_2)\to ({\chi }_2,{\chi }_1)$ to represent the symmetry under exchange of the two objects.

Figure 2

Parameters of numerical-relativity simulations used to calibrate the various inspiral-merger-ringdown models that we investigate in this paper, i.e. (left to right) SEOBNRv1, SEOBNRv2, IMRPhenomC and IMRPhenomD.

Figure 3

Unfaithfulness between SEOBNR and NR waveforms as a function of mass ratio $q=m_1/m_2$, component spins ${\chi }_1$, ${\chi }_2$, and total mass $M$. SEOBNRv1 (top panel) reproduces NR well when the spin on the bigger BH does not exceed $+0.5$, with inaccuracies increasing with mass ratio. SEOBNRv2 (bottom panel) significantly improves over SEOBNRv1 with overlaps against NR higher than 98% over most of the parameter space considered. However, when spins on both component BHs are large and positive aligned, SEOBNRv2 fails to produce accurate waveforms ($\mathcal{O}\simeq 0.80$). We note that both models are accurate within their respective calibration range, but become inaccurate outside this range. Therefore it is crucial to test waveform models before using them in aLIGO analyses.

Figure 4

SEOBNRv2 and NR waveforms for the problematic cases identified in the high spin corner of Fig. 3. Top: $q=2,{\chi }_1={\chi }_2=+0.85$. Bottom: $q=3,{\chi }_1={\chi }_2=+0.85$. Waveforms are aligned during their first few inspiral cycles.

Figure 5

This figure is similar to Fig. 3 with the difference that the models considered here are IMRPhenomC and IMRPhenomD (top and bottom panels, respectively). We note that both of the phenomenological models have been calibrated over most of the mass-ratio and spin range probed here. While IMRPhenomC shows significant deviation from NR as soon as we increase the mass ratio above $q=1$, and/or spin magnitudes above $\approx 0.5$, we find that IMRPhenomD reproduces NR remarkably well with overlaps above 99% everywhere (above 99.5% over most of the space).

Figure 6

We show the effective SNR level at which the SEOBNRv2 and IMRPhenomD models become distinguishable from NR waveforms with the Advanced LIGO instruments. Here we use the indistinguishability criterion proposed in Ref. [104].

Figure 7

Effectualness of the four waveform models considered. Plotted is the fractional loss in recovered SNR. Rows correspond to different models, and within each row, the data are plotted as a function of mass ratio $q$, BH spins ${\chi }_1$, ${\chi }_2$, and total mass $M$. The black crosses denote the values of component spins in the $x-y$ plane. We note that SEOBNRv1 does not model binaries with component spins higher than $+0.6$. We find that the more recent SEOBNRv2 and IMRPhenomD models supersede their earlier counterparts, SEOBNRv1 and IMRPhenomC, respectively, with FFs over 99.5% over most of the spin and mass parameter space probed. However, we do find that for binaries with high spins on both BHs, IMRPhenomD clearly outperforms all others with FFs $>99.5\%$, while SEOBNRv2’s FFs against NR deteriorate to 97%.

Figure 8

Systematic bias in the recovery of chirp mass ${\mathcal{M}}_c$ (left column) and total mass $M$ (right column) for different waveform models (rows). In each panel, the respective bias is shown as a function of the normalized effective spin of the NR waveforms. The plot markers show the bias for a binary with total mass fixed at $M=80M_{\odot }$. The “error bars” show the range of biases for total masses between the minimum allowed mass and $150M_{\odot }$.

Figure 9

Systematic bias in the recovery of the binary mass ratio $q≔m_1/m_2$, as a function of the normalized effective spin of the NR waveforms. Different mass ratios are shown with different color, with horizontal dashed lines of the same color drawn to guide the eye. The plot markers show the recovered $q$ for a binary with total mass fixed at $80M_{\odot }$, while the “error bars” show the range spanned by the recovered $q$ as the injected binary mass is varied between its lowest allowed value and $150M_{\odot }$.

Figure 10

Systematic bias in the recovery of the effective spin parameter ${\chi }_{\mathrm{eff}}$, as a function of the normalized effective spin of the NR waveforms. The plot markers show the recovered ${\chi }_{\mathrm{eff}}$ for a binary with total mass fixed at $80M_{\odot }$, while the “error bars” show the range spanned by the recovered $q$ as the injected binary mass is varied between its lowest allowed value and $150M_{\odot }$.

Figure 11

Systematic bias in the recovered chirp mass ${\mathcal{M}}_c$ for each waveform model; compare to Fig. 8. As in Fig. 7, the black crosses denote the values of component spins in the $x-y$ plane. Biases below 2% are shown nearly transparently, to emphasize regions with larger biases.

Figure 12

Systematic bias in the recovered total mass $M$ for each waveform model; compare to Fig. 8. As in Fig. 7, the black crosses denote the values of component spins in the $x-y$ plane. Biases below 2% are shown nearly transparently, to emphasize regions with larger biases.

Figure 13

Systematic bias in the recovered symmetric mass ratio $\eta$ for the considered waveform models. As in Fig. 7, the black crosses denote the values of component spins in the $x-y$ plane.

Figure 14

Systematic bias in the recovered values of the 1.5PN effective spin ${\chi }_{\mathrm{eff}}$, for the SEOBNRv1, SEOBNRv2, IMRPhenomC, and IMRPhenomD models (from top to bottom).

Figure 15

Systematic bias in the recovered values of the mass-weighted effective spin ${\chi }_{\mathrm{mw}}$, for the SEOBNRv1, SEOBNRv2, IMRPhenomC, and IMRPhenomD models (from top to bottom).