Specific transfer entropy and other state-dependent transfer entropies for continuous-state input-output systems

Since its original formulation in 2000, transfer entropy has become an invaluable tool in the toolbox of nonlinear dynamicists working with empirical data. Transfer entropy and its generalizations provide a precise definition of uncertainty and information transfer that are central to the coupled systems studied in nonlinear science. However, a canonical definition of state-dependent transfer entropy has yet to be introduced. We introduce a candidate measure, the specific transfer entropy, and compare its properties to both total and local transfer entropy. Specific transfer entropy makes possible both state- and time-resolved analysis of the predictive impact of a candidate input system on a candidate output system. We also present principled methods for estimating total, local, and specific transfer entropies from empirical data. We demonstrate the utility of specific transfer entropy and our proposed estimation procedures with two model systems, and find that specific transfer entropy provides more, and more easily interpretable, information about an input-output system compared to currently existing methods.

Figure 1

A demonstration of (a) specific transfer entropy and (b) input-blind $f\left(x_t\right|x_{t-1})$ and input-conditional predictive densities $f\left(x_t\right|y_{t-1},x_{t-1})$ for the model system developed in Sec. 4 at $x_{t-1}=2$. The colors of the input-conditioned predictive densities correspond to colors of the points on specific transfer entropy curve, and the red density corresponds to the input-blind predictive density.

Figure 2

An example realization of the input (top) and output (bottom) from the STARX input-output system with the parameters given in the text. The output $X_t$ is shaded according to $Y_{t-1}$ to indicate how the dynamics vary according to the input, with blue corresponding to negative $Y_{t-1}$ and red corresponding to positive $Y_{t-1}$.

Figure 3

A contour plot of the specific transfer entropy ${\text{t}}_{Y\to X}(y,x)$ from the input process to the output process for the STARX model.

Figure 4

The (a) local and (b) specific transfer entropies and their estimates in the input-to-output (top of panels) and output-to-input (bottom of panels) directions from a particular realization of the STARX system with $T_{\text{sub}}=1000$. The exact transfer entropies are indicated by black (solid) lines, while the estimates based on kernel density estimation with rule of thumb (RoT) and tuned bandwidths and $k\mathrm{th}$ nearest neighbor (kNN) are indicated by red (dot-dashed), blue (short dashed), and green (long dashed) lines, respectively.

Figure 5

A summary of the sampling distribution for the total transfer entropy estimates based on kernel density estimation with rule-of-thumb (RoT) and tuned bandwidths and $k\mathrm{th}$ nearest neighbor (kNN) in the (a) input-to-output and (b) output-to-input directions. Thick lines indicate the 0.25–0.75 quantiles of the 1000 realizations, and thin lines indicate the 0.025–0.975 quantiles. The dashed horizontal lines indicate ${\text{T}}_{Y\to X}^{\left(1\right)}\approx 0.0939$ and ${\text{T}}_{X\to Y}=0.$

Figure 6

A summary of the sampling distribution for the mean absolute error between the true local transfer entropy and the estimates based on kernel density estimation with rule-of-thumb (RoT) and tuned bandwidths and $k\mathrm{th}$ nearest neighbor (kNN) in the (a) input-to-output and (b) output-to-input directions. Thick lines indicate the 0.25–0.75 quantiles of the 1000 realizations, and thin lines indicate the 0.025–0.975 quantiles.

Figure 7

A summary of the sampling distribution for the mean absolute error between the true specific transfer entropy and the estimates based on kernel density estimation with rule-of-thumb (RoT) and tuned bandwidths and $k\mathrm{th}$ nearest neighbor (kNN) in the (a) input-to-output and (b) output-to-input directions. Thick lines indicate the 0.25–0.75 quantiles of the 1000 realizations, and thin lines indicate the 0.025–0.975 quantiles.

Figure 8

Results for the unidirectionally coupled stochastic Hénon maps. (a) Input and output time series. (b) Local transfer entropy. (c) Specific transfer entropy.

Figure 9

The specific transfer entropies as a function of the observed state variables for coupled stochastic Hénon maps with unidirectional coupling. (a) and (c) show the nominal output future as a function of the nominal input-output past in the input-to-output and output-to-input direction while (b) and (d) show the nominal output future as a function of two lags of the nominal output past. The projections onto the planes show the nominal output future as a function of the most recent nominal output past. Each point is shaded according to the estimated specific transfer entropy associated with that nominal input-output state in the input-to-output direction [(a) and (b)] and output-to-input direction [(c) and (d)], with black (dark) indicating low specific transfer entropy and yellow (light) indicating high specific transfer entropy.

Figure 10

Results for the stochastic Hénon maps with switched coupling. (a) Input and output time series. (b) Local transfer entropy. (c) Specific transfer entropy. The dashed orange line indicates when the coupling switches on.

Figure 11

The specific transfer entropies as a function of the observed state variables for coupled stochastic Hénon maps with switching coupling. (a) shows the nominal output future as a function of the nominal input-output past and (b) shows the nominal output future as a function of two lags of the nominal output past. The projections onto the planes show the nominal output future as a function of the most recent nominal output past. Each point is shaded according to the specific transfer entropy associated with that nominal input-output state, with black (dark) indicating low specific transfer entropy and yellow (light) indicating high specific transfer entropy.