Detecting gravitational waves in data with non-stationary and non-Gaussian noise

Searches for gravitational waves crucially depend on exact signal processing of noisy strain data from gravitational wave detectors, which are known to exhibit significant nonstationary and non-Gaussian behavior. In this paper, we study two distinct effects in the LIGO/Virgo data that reduce the sensitivity of searches: first, variations in the noise power spectral density (PSD) on timescales of more than a few seconds; and second, loud and abrupt transient “glitches” of terrestrial or instrumental origin. We derive a simple procedure to correct, at first order, the effect of the variation in the PSD on the search background. Given the knowledge of the existence of localized glitches, in particular segments of data, we also develop a method to insulate statistical inference from these glitches, so as to cleanly excise them without affecting the search background in neighboring seconds. We show the importance of applying these methods on the publicly available LIGO data and estimate an increase in the detection volume of at least 15% from the PSD-drift correction alone, due to the improved background distribution.

Figure 1

Histogram of variance values of overlaps computed on all the valid O2 data. Measurement error on these values is 2%. Measured deviations from unity are much bigger (about 10%) and therefore produce a big difference in significance determination if unaccounted for.

Figure 2

It is necessary to track the drifting PSD on timescales of seconds. Solid lines show the empirically measured power spectrum $S_{z^2}$ of a time series composed of the measured variance of the overlaps in every second (note that the frequencies are much lower than the frequency content of the waveform itself). For reference, dashed lines show the same on artificially generated stationary Gaussian noise. The variance time series has a red-noise power spectrum.

Figure 3

Cumulative trigger rates in bank BBH (0,0) (binary black holes whose chirp mass falls in the range $3–5M_{\odot }$ [18]) using the entirety of the O2 bulk data. The initial trigger distribution (blue line) is the distribution of the maximum overlap obtained every second, computed after removal of bad data, as described in Ref. [13]. In green is shown the final cumulative trigger distribution, after correcting for PSD drift and applying signal consistency vetoes. In black we show the cumulative trigger distribution when PSD drift is unaccounted for. For reference we show in orange the trigger distribution when the signal consistency vetoes are not applied. While the PSD drift correction multiplying real events is on average one (see Fig. 1), the background distribution undergoes a dramatic change. This is because if left unaccounted for, the fluctuations in places with variance misestimation dominate the tail of the trigger distribution. This effect becomes more severe at higher trigger significance.

Figure 4

Cumulative trigger rates in bank BBH (3,0) (binary black holes whose chirp mass falls in the range $20–40M_{\odot }$ and whose effective spin does not have a very negative value [18]), using the entirety of the O2 bulk data. This bank contains most of the detected BBH events in O1 and O2. As can be seen, signal consistency checks for triggers are much more important in this domain as the waveforms in general have a very short duration in band. But by pushing back the background distribution, PSD drift correction still mitigates a substantial amount of sensitivity loss.

Figure 5

Effect of masking and inpainting glitches. Top panel: A segment (Data shown is from the Livingston observatory, GPS time: 1134155453.8789, first advanced LIGO observing run.) of whitened strain data (in units of the noise standard deviation) that contains a glitch. The orange curve tracks the standard deviation $\sigma$ calculated from a running window of 100 samples and is typically close to unity as expected for whitened data. Second panel: Gating the glitch with an upside-down Tukey window (green curve), and then whitening generates artifacts in the whitened data, even outside the Tukey window. For example, $\sigma$ stays above 1.1 for approximately 2 s to each side of the glitch (not shown). Third panel: The inpainted whitened data has unit variance outside the hole (shaded). Bottom panel: After inpainting, the “blued” strain is identically zero inside the hole, so overlaps with templates do not depend on the waveform information from inside the hole. The figure was previewed already in the pipeline description paper, Ref. [13].

Figure 6

Demonstration of inpainting in the segment of data containing GW170817. Upper left panel: Whitened spectrogram (Spectrogram was made with 4 Hz frequency resolution, 75% overlap between adjacent spectra.) of the original Livingston strain few seconds around the merger of GW170817, with a PSD measured using the Welch method, with 64 s chunks. Gating whitened data as in, e.g., Refs. [10, 11] would result in the shown power leakage in low frequencies $\sim 4\mathrm{s}$ either side, caused by the interaction of the glitch with the whitening filter. Upper right panel: Glitch gated away with an inverse Tukey window, with a timescale of 1.2 s and $\alpha =5/6$, as done in Ref. [6]. Note the leakage from the spectral lines (around 500 Hz) is strongly affecting the data 5 s to either side. Bottom left panel: Same as upper right, but with a 160 ms window, leaving a more prominent noise leakage. Bottom panel: 160 ms inpainted by the hole-filling method presented in this work. The resulting data are free from disturbances and allow accurate inference and significance assessment. Note the PSD resolution chosen in the making of this plot (reciprocal of the window size of 64 s) differs from the one in Refs. [10, 12] (4 s or 8 s) (but is the default choice in Ref. [9]). We show similar plots with lower PSD resolution in Appendix pp3.

Figure 7

Same as Fig. 6 but with a PSD with a lower frequency resolution of $1/4\mathrm{Hz}$. Prior to whitening, we had to manually notch a loud spectral line at 500 Hz, as it was unresolved and presented with a varying amplitude, which caused artifacts in the whitened data. It can be seen that at this frequency resolution, the gate that was used in the official analysis of GW170817 was enough to cover all the residuals from the glitch. It can also be seen that in order to make a narrower hole, inpainting is essential.