Information transfer from causal history in complex system dynamics

In a multivariate evolutionary system, the present state of a variable is a resultant outcome of all interacting variables through the temporal history of the system. How can we quantify the information transfer from the history of all variables to the outcome of a specific variable at a specific time? We develop information theoretic metrics to quantify the information transfer from the entire history, called causal history. Further, we partition this causal history into immediate causal history, as a function of lag $\tau$ from the recent time, to capture the influence of recent dynamics, and the complementary distant causal history. Further, each of these influences are decomposed into self- and cross-feedbacks. By employing a Markov property for directed acyclic time-series graph, we reduce the dimensions of the proposed information-theoretic measure to facilitate an efficient estimation algorithm. This approach further reveals an information aggregation property, that is, the information from historical dynamics are accumulated at the preceding time directly influencing the present state of variable(s) of interest. These formulations allow us to analyze complex inter-dependencies in unprecedented ways. We illustrate our approach for: (1) characterizing memory dependency by analyzing a synthetic system with short memory; (2) distinguishing from traditional methods such as lagged mutual information using the Lorenz chaotic model; (3) comparing the memory dependencies of two long-memory processes with and without the strange attractor using the Lorenz model and a linear Ornstein-Uhlenbeck process; and (4) illustrating how dynamics in a complex system is sustained through the interactive contribution of self- and cross-dependencies in both immediate and distant causal histories, using the Lorenz model and observed stream chemistry data known to exhibit $1/f$ long memory.

Figure 1

Illustration of the causal history ${\vec{X}}_t^{-}$ of a target node $Z_t$. (a) The partition of ${\vec{X}}_t^{-}$ into an immediate causal history, $C_{\vec{V}\Rightarrow Z_t}$ (the dashed blue box), and the complementary distant causal history, ${\vec{X}}_t^{-}\setminus C_{\vec{V}\Rightarrow Z_t}$ (the dashed red box). The parents of the target $Z_t$ [Eq. (3)], $P_{Z_t}$, are identified by the cyan colored box. (b) The aggregation of contemporaneous momentary information from each set of contemporaneous nodes ${\vec{X}}_{t-i}$ (the dashed hollow box) at an early time step $t-i$ in the causal history [Eq. (10)].

Figure 2

Illustration of the self- and cross-dependencies in both simplified immediate and distant causal histories for a target $Z_t$ (the black node). The self-dependencies, ${\vec{Z}}_{\mathcal{J}}$, and the complementary part, ${\vec{Z}}_{\mathcal{J}}^{{}^{\prime }}$, in the simplified immediate causal history, $P_{Z_t}^{C_{\vec{V}\Rightarrow Z_t}}$, are identified in solid and dashed black boxes, respectively. The self-dependencies, ${\vec{Z}}_{\mathcal{D}}$, and the complementary part, ${\vec{Z}}_{\mathcal{D}}^{{}^{\prime }}$, in the simplified distant causal history, ${\vec{W}}_{\tau }$, are identified in solid and dashed gray boxes, respectively.

Figure 3

Illustration of the trivariate coupled logistic model. (a) The times-series graph of the system with the causal subgraph $C_{\{X_{1,t-\tau },X_{2,t-\tau },X_{3,t-\tau }\}\Rightarrow X_{3,t}}\mathrm{as}$ the immediate causal history [the representations of the nodes are the same as in Fig. 1]. (b) Plots of $\mathcal{J}, \mathcal{D}/\mathcal{T}, \overline{\mathcal{S}}, {\mathcal{I}}_{\mathcal{D}}$, and ${\mathcal{I}}_{\mathcal{J}}$ for $\tau$ ranging from 1 to 50 with $\epsilon \in [0.1,0.2,0.3,0.5,0.8]$ (blue and red crosses, connected through a vertical line, represent the convergence points of $\mathcal{J}, \mathcal{D}/\mathcal{T}$, and $\overline{\mathcal{S}}$ for $\epsilon =0.1$ and $\epsilon =0.2$, respectively; note that results for $\epsilon =0.8$ are not plotted (except $\mathcal{J}$) due to its high variability resulting from a near-zero total information $\mathcal{T}$).

Figure 4

Illustration of the Lorenz model with parameters $\sigma =10, \rho =28$, and $\beta =8/3$. (a) The times-series graph of the system with the causal subgraph $C_{\{X_{t-\tau },Y_{t-\tau },Z_{t-\tau }\}\Rightarrow U_t}$ ($U\in \{X,Y,Z\}$) as the immediate causal history. (b) The corresponding plots of the lagged mutual information, $\mathcal{J}$, and $\mathcal{D}$ for the time lag $\tau$ ranging from 1 to 1000. (c) The corresponding plots of ${\mathcal{I}}_{\mathcal{D}}, {\mathcal{I}}_{\mathcal{J}}$, and $\mathcal{J}-\mathcal{D}$ for the time lag $\tau$ ranging from 1 to 1000.

Figure 5

Illustration of the Ornstein-Uhlenbeck process in Eq. (18). (a) The trajectories of the process (left) and the time series of each variable (right). (b) The corresponding plots of the lagged mutual information, $\mathcal{J}$, and $\mathcal{D}$ for the time lag $\tau$ ranging from 1 to 1000. (c) The corresponding plots of ${\mathcal{I}}_{\mathcal{D}}, {\mathcal{I}}_{\mathcal{J}}$, and $\mathcal{J}-\mathcal{D}$ for the time lag $\tau$ ranging from 1 to 1000.

Figure 6

Time-series graph constructed by using the Tigramite algorithm from (a) observed logarithm of flow rate and six catchment chemistry time series data; and (b) the six catchment chemistry data with the variation of logarithmic flow rate corrected. The thickness of edges represents the coupling strength between two nodes computed by momentary information transfer shown in Fig. 9 (see the details of the graph construction in Appendix pp2).

Figure 7

Plots of the information transfers $\mathcal{D}$ (left) and the proportion $\mathcal{D}/\mathcal{T}$ (right) over the time lag $\tau$ for the raw data and the flow rate-corrected data taking the immediate causal history initiated from all the variables with a same lag $\tau$ based on the estimated time-series graph in Fig. 6.

Figure 8

Plots of the interaction information from distant causal history, ${\mathcal{I}}_{\mathcal{D}}$ in Eq. (12) (black line), and immediate causal history, ${\mathcal{I}}_{\mathcal{J}}$ in Eq. (13) (blue line), over the time lag $\tau$ for the raw data and the flow rate-corrected data taking the immediate causal history initiated from all the variables with a same lag $\tau$ based on the estimated time series graph in Fig. 6.

Figure 9

Illustration of the estimated lag functions ($y$ axis: the coupling strength [nats] computed based on momentary information transfer (MIT) [32]; $x$ axis: the time lag $\tau$) of the catchment chemistry data by using Tigramite algorithm for: (a) the logarithm of flow rate and six chemistry variable; and (b) the six chemistry variables with the variation of the logarithm of flow rate excluded.

Figure 10

Number of data points for computing $\mathcal{D}$ in Eq. (4) in terms of the time lag $\tau$ for each variable in the two time-series graphs constructed in Fig. 9.

Figure 11

The cardinality of the estimated $\mathcal{T}, \mathcal{D}$, and $\mathcal{J}$ in Eqs. (3), (4), and (6), respectively, in terms of the time lag $\tau$ for each variable in the two time-series graphs constructed in Fig. 9.

Figure 12

The estimated $\mathcal{D}$ in Eq. (4) from the two networks constructed in Fig. 9 as well as the corresponding threshold for shuffle test with significance level $\alpha =0.05$.