Permutation entropy of finite-length white-noise time series

Permutation entropy (PE) is commonly used to discriminate complex structure from white noise in a time series. While the PE of white noise is well understood in the long time-series limit, analysis in the general case is currently lacking. Here the expectation value and variance of white-noise PE are derived as functions of the number of ordinal pattern trials, $N$, and the embedding dimension, $D$. It is demonstrated that the probability distribution of the white-noise PE converges to a ${\chi }^2$ distribution with $D!-1$ degrees of freedom as $N$ becomes large. It is further demonstrated that the PE variance for an arbitrary time series can be estimated as the variance of a related metric, the Kullback-Leibler entropy (KLE), allowing the qualitative $N\gg D!$ condition to be recast as a quantitative estimate of the $N$ required to achieve a desired PE calculation precision. Application of this theory to statistical inference is demonstrated in the case of an experimentally obtained noise series, where the probability of obtaining the observed PE value was calculated assuming a white-noise time series. Standard statistical inference can be used to draw conclusions whether the white-noise null hypothesis can be accepted or rejected. This methodology can be applied to other null hypotheses, such as discriminating whether two time series are generated from different complex system states.

Figure 1

Expectation value of $1-H$ [i.e., the residual term in Eq. (7)] as a function of time series length. Each data point was obtained by generating 1000 white-noise sets of the stated length and calculating the average value of $1-H$. Solid lines indicate the expectation value of $1-H$ calculated using Eq. (7). Dashed lines are plotted for $D=6,7$ to emphasize the divergence between numeric and theoretical values for shorter time series.

Figure 2

Variance of $H$ as a function of time series length. Each data point was obtained by generating 1000 white-noise sets of the stated length and calculating $\text{var}\left[H\right]$ over these noise realizations. Solid lines indicate $\text{var}\left[H\right]$ calculated using Eq. (14). Dashed lines are shown for $D=6,7$ to emphasize the divergence between numeric and theoretical values at shorter time series lengths.

Figure 3

Histograms of $H$, calculated for 10 000 white-noise sets for (a) $D=3, L=2000$, (b) $D=4, L=2000$, (c) $D=5, L=2000$, and (d) $D=5, L=20000$. The solid curve indicates the predicted distribution based on a ${\chi }^2$ distribution with $D!-1$ degrees of freedom calculated using Eq. (17). It can be seen that for $D=5$, the normalized PE becomes approximately normally distributed. Convergence toward the predicted ${\chi }^2$ distribution is observed as $L$ is increased.

Figure 4

Variance of $H$ as a function of time series length, calculated over 1000 fractional Gaussian noise sets with a Hurst index of 0.8 (circles), and estimates of KLE variance generated using Eq. (23) (lines). Note that the estimate of $\text{var}\left[H\right]$ is a single estimate over all 1000 fractional Gaussian noise sets, while estimates for $\text{var}\left[D_{\text{NKL}}\right]$ are for each noise set (1000 estimates in all). The black line indicates the average estimate, while the gray lines indicate the 2.5-th and 97.5-th estimate percentiles.

Figure 5

Estimated probability density functions for a hypothetical white-noise series (blue) and the experimental noise series (orange) as a function of time series length. Solid lines indicate the mean, while dotted lines mark the 95% confidence interval.

Figure 6

Probability of obtaining the observed PE for the experimental time series, assuming white noise, $P_{wn}$, as a function of time series length. Dotted lines indicate $5\%$ and $1\%$ confidence thresholds.