Information-Theoretical Bound of the Irreversibility in Thermal Relaxation Processes

We establish that entropy production, which is crucial to the characterization of thermodynamic irreversibility, is obtained through a variational principle involving the Kulback-Leibler divergence. A simple application of this representation leads to an information-theoretical bound on entropy production in thermal relaxation processes; this is a stronger inequality than the conventional second law of thermodynamics. This bound is also interpreted as a constraint on the possible path of a thermal relaxation process in terms of information geometry. Our results reveal a hidden universal law inherent to general thermal relaxation processes.

Figure 1

We draw a state space of probability distributions. Two thin curves represent trajectories of probability distributions in dynamics with the transition matrix $R$. Green lines represent the Kullback-Leibler divergence between two distributions. The variational principle (2) claims that the difference between two Kullback-Leibler divergences $D\mathbf{(}\mathit{p}(0)||\mathit{q}(0)\mathbf{)}-D\mathbf{(}\mathit{p}(\mathrm{\Delta }t)||\mathit{q}(-\mathrm{\Delta }t)\mathbf{)}$ (solid line minus dashed line) is maximized when $\mathit{q}(0)=\mathit{q}(-\mathrm{\Delta }t)={\mathit{p}}^{\mathrm{eq}}$.

Figure 2

Demonstration of the relation (3) in a three-state toy model. The dynamics shows a long-time trap dominated by the states 2 and 3. (a) We plot ${\sigma }_{[0,\tau ]}$ (orange solid line) and $D\mathbf{(}\mathit{p}(0)||\mathit{p}(\tau )\mathbf{)}$ (black dashed line). Although the entropy production shows anomalous two-step relaxation, $D\mathbf{(}\mathit{p}(0)||\mathit{p}(\tau )\mathbf{)}$ robustly bounds the entropy production. (b) We plot the difference between these two.

Figure 3

Schematic of the constraint (4) in terms of the information geometry. As a natural extension of the Pythagorean theorem in the information geometry, the inequality is interpreted as that the probability distribution at $\tau$ forms an obtuse angle and its path is inside the green “circle.” We emphasize that these relations are stronger than the conventional second law: ${\sigma }_{[0,\tau ]}=D\mathbf{(}\mathit{p}(0)||{\mathit{p}}^{\mathrm{eq}}\mathbf{)}-D\mathbf{(}\mathit{p}(\tau )||{\mathit{p}}^{\mathrm{eq}}\mathbf{)}\ge 0$.