Information Thermodynamics on Causal Networks

We study nonequilibrium thermodynamics of complex information flows induced by interactions between multiple fluctuating systems. Characterizing nonequilibrium dynamics by causal networks (i.e., Bayesian networks), we obtain novel generalizations of the second law of thermodynamics and the fluctuation theorem, which include an informational quantity characterized by the topology of the causal network. Our result implies that the entropy production in a single system in the presence of multiple other systems is bounded by the information flow between these systems. We demonstrate our general result by a simple model of biochemical adaptation.

Figure 1

(a) Schematic of a BN. (b) Stochastic dynamics of system $X$ under the influence of other systems.

Figure 2

(a) Time evolution of a single-molecule gas without feedback control. (b) BN corresponding to (a). (c) The Szilard engine with feedback control by a memory device. (d) BN corresponding to (c).

Figure 3

(a) Schematic of $\mathcal{C}$ and $\mathrm{pa}(x_1)$. $\mathcal{C}$ is the history of other systems that can affect the final state $x_N$. $\mathrm{pa}(x_1)$ describes the variables correlated with the initial state $x_1$. (b) An example of a BN that describes three-body interactions.

Figure 4

(a) BN corresponding to a simple Markov chain. (b) BN corresponding to feedback control. (c) BN corresponding to two Brownian particles.

Figure 5

(a) Feedback loop of a time-delayed chemical reaction model. (b) BN that describes our model. (c) The free-energy levels and the interactions between output $O$ and memory $M$. For instance, $F_{\mu }^O(t)$ at time $\delta \le t\le 2\delta$ depends on $m_1$ and $m_2$.

Figure 6

Numerical illustration of the non-negativity of $⟨\sigma ⟩-⟨\Theta ⟩=⟨-{\beta }^O{Q}_O⟩+⟨\ln p(o_1,m_1,m_2)⟩-⟨\ln p(o_2,m_1,m_2)⟩$. We set the initial states to $p(o_1,m_1)=p(o_1)p(m_1)$. The amount of $⟨\sigma ⟩-⟨\Theta ⟩$ is close to 0 when the initial states are close to the stationary state of this system. The parameter set is noted in the Supplemental Material [55].