Transfer entropy rate through Lempel-Ziv complexity

The transfer entropy and the transfer entropy rate are closely related concepts that measure information exchange between two dynamical systems. These measures allow us to study linear and nonlinear causality relations and can be estimated through the use of different methodologies. However, some of them assume a data model and/or are computationally expensive. This article depicts a methodology to estimate the transfer entropy rate between two systems through the Lempel-Ziv complexity. This methodology offers a set of advantages: It estimates the transfer entropy rate from two single discrete series of measures, it is not computationally expensive, and it does not assume any data model. The simulation results over three different unidirectional coupled dynamical systems suggest that this methodology can be used to assess the direction and strength of the information flow between systems. Moreover, it provides good estimations for short-length time series.

Figure 1

Diagram to obtain the sequence $z_n$ to calculate the entropy rate $h\left(Y_t^{\left(m\right)},X_t^{\left(m\right)},X_t\right)$. The matrix $\mathcal{V}$ is obtained by embedding ($m=3$ and $\tau =5$) the binarized version of $x_t$ and $y_t$. The median values of both time series are shown as horizontal dashed lines.

Figure 2

Henon-Henon coupled system. Boxplot of global transfer entropy rate as a function of the coupling parameter $\epsilon$. $\mathcal{T}$ was calculated with $m=4$, $\alpha =4$, and $\tau =1$ for different data lengths: (a) $N=500$, (b) $N=3000$, and (c) $N=5000$.

Figure 3

Henon-Henon coupled system. Boxplot of global transfer entropy rate as a function of the data length $N$ for $m=4$, $\alpha =4$, $\tau =1$, and $\epsilon =0.6$.

Figure 4

Rössler-Lorenz coupled system. Boxplot of global transfer entropy rate as a function of the coupling parameter $\epsilon$. $\mathcal{T}$ was calculated with $m=6$, $\alpha =6$, and $\tau =10$ for different data lengths: (a) $N=3000$, (b) $N=5000$, and (c) $N=10000$.

Figure 5

Comparison of transfer entropy rate estimation with three different methods. Lempel-Ziv complexity-based method (a) $N=200$, (d) $N=1000$, and (g) $N=10000$. Symbolic transfer entropy (b) $N=200$, (e) $N=1000$, and (f) $N=10000$. The kNN method (c) $N=200$, (f) $N=1000$, and (i) $N=10000$. Boxplot of the $\mathcal{T}$ as a function of the coupling parameter, calculated for the coupled Lorenz system with $m=6$, $\tau =5$, and $\alpha =4$.

Figure 6

Computation time as a function of the embedding dimension ($m$) for different transfer entropy rate estimation methods: (a) Lempel-Ziv complexity-based method, (b) symbolic transfer entropy and (c) kNN method. The simulation was made using the coupled Lorenz system with parameters $N=10000$, $\tau =5$, $\alpha =4$, $\epsilon =3$, and $K=30$.