Multivariate classification with random forests for gravitational wave searches of black hole binary coalescence

Searches for gravitational waves produced by coalescing black hole binaries with total masses $\gtrsim 25{\mathrm{M}}_{\odot }$ use matched filtering with templates of short duration. Non-Gaussian noise bursts in gravitational wave detector data can mimic short signals and limit the sensitivity of these searches. Previous searches have relied on empirically designed statistics incorporating signal-to-noise ratio and signal-based vetoes to separate gravitational wave candidates from noise candidates. We report on sensitivity improvements achieved using a multivariate candidate ranking statistic derived from a supervised machine learning algorithm. We apply the random forest of bagged decision trees technique to two separate searches in the high mass ($\gtrsim 25{\mathrm{M}}_{\odot }$) parameter space. For a search which is sensitive to gravitational waves from the inspiral, merger, and ringdown of binary black holes with total mass between $25{\mathrm{M}}_{\odot }$ and $100{\mathrm{M}}_{\odot }$, we find sensitive volume improvements as high as ${70}_{\pm 13}\%–{109}_{\pm 11}\%$ when compared to the previously used ranking statistic. For a ringdown-only search which is sensitive to gravitational waves from the resultant perturbed intermediate mass black hole with mass roughly between $10{\mathrm{M}}_{\odot }$ and $600{\mathrm{M}}_{\odot }$, we find sensitive volume improvements as high as ${61}_{\pm 4}\%–{241}_{\pm 12}\%$ when compared to the previously used ranking statistic. We also report how sensitivity improvements can differ depending on mass regime, mass ratio, and available data quality information. Finally, we describe the techniques used to tune and train the random forest classifier that can be generalized to its use in other searches for gravitational waves.

Figure 1

A cartoon example of a random forest. There are five decision trees in this random forest. Each was trained on a training set of objects belonging to either the black set or the cyan set. Note that the training set of each decision tree is different from the others. At each numbered node, or split in the tree, a binary decision based on a threshold of a feature vector parameter value is imposed. The decisions imposed at each node will differ for the different trees. When no split on a node can reduce the entropy or it contains fewer events than a preset limit, it is no longer divided and becomes a leaf. Consider an object that we wish to classify as black or cyan. Suppose the object ends up in each circled leaf. Then the probability that the object is black is the fraction of black objects in all leaves, $p_{\text{forest}}=73\%$.

Figure 2

Cumulative distributions of coincident events found as a function of inverse combined false alarm rate. The data plotted here show results for a ringdown-only search over $\sim 9$ days of H1L1V1 network data at Veto Level 3. Blue triangles represent the coincident events found by the ringdown-only search [7]. Grey lines plot the coincident events in each of the 100 time-shift experiments. Yellow contours mark the $1\sigma$ and $2\sigma$ regions of the expected background from accidental coincidences. (a) The results of the search obtained with a RFBDT classifier trained on a contaminated injection set. (b) The results when the classifier is trained on a clean injection set. A RFBDT classifier trained on a clean injection set properly ranks H1L1V1 coincidences with low significance so that there is not a “bump” in the distribution at low combined FAR.

Figure 3

Investigation of the effect of using a different number of trees on the recovery of $q=1$ EOBNRv2 simulated waveforms at Veto Level 3. In general, we find that the use of more than 100 trees gives roughly the same sensitivity regardless of mass ratio or veto level. In this ROC, to adjust for the loss in analysis time in moving from Veto Level 2 to Veto Level 3, we scale the volume fraction in Eq. (8) by the ratio of analysis times $f=t_{\mathrm{VL}3}/t_{\mathrm{VL}2}$. From the analysis times reported in Sec. 3, we find $f=0.88$.

Figure 4

Percent improvements (over the use of the ${\rho }_{\text{high},\text{combined}}$ [6] ranking statistic at Veto Levels 2 and 3) in sensitive volume multiplied by analysis time $(VT)$ for the recovery of EOBNRv2 (a) simulated waveforms at Veto Levels 2 and 3 and the recovery of IMRPhenomB (b) waveforms at Veto Levels 2 and 3 by the IMR search. Results for IMRPhenomB are shown for signals with spin parameter ($|{\chi }_s|<0.85$) and no spin (${\chi }_s=0$). The quantity $VT$ gives us a measure of the true sensitivity of the search and allows us to compare performances across veto levels. Results are shown for total binary masses from $25\le M/{\mathrm{M}}_{\odot }\le 100$ in mass bins of width $12.5{\mathrm{M}}_{\odot }$.

Figure 5

Percent improvements (over the use of the ${\rho }_{\text{high},\text{combined}}$ [6] ranking statistic at Veto Levels 2 and 3) in sensitive volume multiplied by analysis time $(VT)$ for the recovery of EOBNRv2 simulated waveforms at Veto Levels 2 (a) and 3 (b) by the IMR search. Percent improvement results are shown as a function of binary component masses. Note that the color scales of (a) and (b) are not equivalent.

Figure 6

Percent improvements (over the use of the ${\rho }_{\mathrm{S}5/\mathrm{S}6}$ ranking statistic [71] at Veto Level 2) in sensitive volume multiplied by analysis time $(VT)$ for the recovery of $q=1$ (a) and $q=4$ (b) EOBNRv2 simulated waveforms at Veto Level 2. The quantity $VT$ gives us a measure of the true sensitivity of the search and allows us to compare performances across veto levels. Results are shown for total binary masses from $50\le M/{\mathrm{M}}_{\odot }\le 450$ in mass bins of width $50{\mathrm{M}}_{\odot }$.

Figure 7

Percent improvements (over the use of the ${\rho }_{\mathrm{S}5/\mathrm{S}6}$ ranking statistic [71] at Veto Level 3) in sensitive volume multiplied by analysis time $(VT)$ for the recovery of $q=1$ (a) and $q=4$ (b) EOBNRv2 simulated waveforms at Veto Level 3. The quantity $VT$ gives us a measure of the true sensitivity of the search and allows us to compare performances across veto levels. Results are shown for total binary masses from $50\le M/{\mathrm{M}}_{\odot }\le 450$ in mass bins of width $50{\mathrm{M}}_{\odot }$.

Figure 8

Signal and background density distributions for a selection of feature vector statistics for the IMR search.

Figure 9

Signal and background density distributions for a selection of feature vector statistics for the IMR search.

Figure 10

Signal and background density distributions for a selection of feature vector statistics for the ringdown-only search.

Figure 11

Signal and background density distributions for a selection of feature vector statistics for the ringdown-only search.

Figure 12

Signal and background density distributions for a selection of feature vector statistics for the ringdown-only search.

Figure 13

Signal and background density distributions for a selection of feature vector statistics for the ringdown-only search.