Bayesian estimation of correlation functions

We apply Bayesian statistics to the estimation of correlation functions. We give the probability distributions of auto- and cross-correlations as functions of the data. Our procedure uses the measured data optimally and informs about the certainty level of the estimation. Our results apply to general stationary processes and their essence is a nonparametric estimation of spectra. It allows one to better understand the statistical noise fluctuations, assess the correlations between two variables, and postulate parametric models of spectra that can be further tested. We also propose a method to numerically generate correlated noise with a given spectrum.

Figure 1

Left: Estimating distributions—Eq. (9)—for $\overline{\mathrm{\Lambda }}=1$ and different sizes $M$ of the data sample. The curves have been shifted and scaled in the vertical axis for the sake of visibility. Right: 90%-confidence intervals for the estimating distributions. The top figure shows how the maximum-likelihood estimator and confidence intervals are defined. The bottom figure plots the relation between confidence and sample size. The color code is consistent with the plot on the left.

Figure 2

Comparison of estimating distributions of the variable $A$ when considering the partial data $\widehat{\mathit{A}}$ (dashed lines) vs the whole data $\widehat{\mathit{X}}$ (colored-solid lines). The different colors indicate the values of ${\overline{s}}_k$, ranging from 0 to 0.9 in steps of 0.1. The batch size $M$ appears on the top right corners, ${\overline{\mathrm{\Lambda }}}_k^A=1$ and ${\overline{\alpha }}_0=0$. Top: Estimating distributions of ${\mathrm{\Lambda }}_k^A$, namely Eq. (21) versus Eq. (36). Bottom: Estimating distributions of ${\mu }_A$, namely Eq. (23) vs Eq. (46). The deviations from the dashed lines are significant for low values of $M$ and ${\overline{s}}_k$ close to 0 or 1.

Figure 3

Distributions of cross-correlations as a function of ${\overline{s}}_k, {\overline{\varphi }}_k$, and $M$. (a) Contours of $P\left(s_k{\varphi }_k\right|\widehat{\mathit{X}})$—Eq. (40)—enclosing the 75, 50, 25, 10, and 5%-probability regions. The plot is in the complex plane, with $s_k$ as the radial coordinate and ${\varphi }_k$ as the polar angle. The unit length is given by the radius of the circle. For red, yellow, blue, and green, ${\overline{s}}_k=\{0.2,0.5,0.7,0.9\}$ and ${\overline{\varphi }}_k=\{0,3\pi /4,5\pi /4,7\pi /4\}$, respectively. In all cases, $M=10$. (b) Dependence of $P\left(s_k\right|\widehat{\mathit{X}})$—Eq. (43)—on ${\overline{s}}_k$ for $M=10$. The curves have been shifted and scaled in the vertical axis. (c) Contours like in (a) as a function of ${\overline{s}}_k$ and $M$. The complex-plane axes have been omitted (but note the given length scale) and the plots arranged in a matrix for the sake of visibility and ease of comparison. We take ${\overline{\varphi }}_k=0$, but recall that the value of ${\overline{\varphi }}_k$ is irrelevant for the shape of $P\left(s_k{\varphi }_k\right|\widehat{\mathit{X}})$ up to polar rotations. The crosses mark the maximum-likelihood estimators.

Figure 4

Distributions of cross-correlations as a function of $M$ for ${\overline{s}}_k=0.7$ and ${\overline{\varphi }}_k=\pi$. The curves are shifted and scaled in the vertical axis for visibility.

Figure 5

Top: Time evolution of the generated variables $A, B$, and $C$ for batch $m=1$. Bottom: Estimation of the spectra ${\mathrm{\Lambda }}_k^A$ with Eq. (21); and of ${\mathrm{\Lambda }}_k^B, {\mathrm{\Lambda }}_k^C$ with Eq. (36). The maximum-likelihood estimators appear circled, and the estimating distributions for ${\mathrm{\Lambda }}_k^A, {\mathrm{\Lambda }}_k^B$, and ${\mathrm{\Lambda }}_k^C$ are overlaid as a linear color gradient. The dashed lines plot the spectral functions used to generate the data, see Eqs. (49). Note that adopting the log scale for the horizontal axis makes the distributions for high $k$ overlap. To declutter the plot and keep its information content, we locally average the points using the procedure described in Appendix pp6.

Figure 6

Left: Estimation of the correlation strength for all possible pairs in the set $\{A,B,C\}$, with $P\left(s_k\right|\widehat{\mathit{X}})$ as a linear color gradient. The legend appears in the upper right corner and the color code is consistent with Fig. 5. The maximum-likelihood estimators appear circled. The results reflect that only the variables $B$ and $C$ are correlated. The value $s_k=0.7$, used to generate the data, is marked with a faint gray line. Right: The same for the correlation phase with the estimating distribution $P\left({\varphi }_k\right|\widehat{\mathit{X}})$. The gray line marks ${\varphi }_k=\pi$. The points were averaged according to the procedure described in Appendix pp6 for better visibility.

Figure 7

The analog to Fig. 6 for fully or partially anticorrelated signals for a single batch, $M=1$. The left panels show the estimating distributions for the correlation strength $s_k$, the right ones for the correlation phase ${\varphi }_k$. We generate $B$ and $C$ with spectra as shown in Fig. 5, with $s_k=0.7$ and ${\varphi }_k=\pi$ for all $k$. In the first row we calculate correlation of $B$ with its inverse $-B$, while in the second row the correlation of $B$ and $C$. To declutter the plot the horizontal axis was divided into 100 bins equally spaced in the logarithmic scale and all distributions falling within the same bin were arithmetically averaged. To deliver the point of the figure, we intentionally do not use here a more efficient averaging described in Appendix pp6.

Figure 8

Distribution of the magnitude of unnormalized cross-correlations as a function of $M$ for ${\overline{s}}_k=0.7$. The curves are shifted and scaled in the vertical axis for visibility.

Figure 9

Central-limit (large-$M$) approximations to estimating distributions. In each panel, exact estimating distributions for the variable given on the x axis are plotted for several values of $m$ as given by the color bar. The black-dashed curves give the approximate (central-limit) estimating distributions for $m=200$.