Learning Entropy Production via Neural Networks

This Letter presents a neural estimator for entropy production (NEEP), that estimates entropy production (EP) from trajectories of relevant variables without detailed information on the system dynamics. For steady state, we rigorously prove that the estimator, which can be built up from different choices of deep neural networks, provides stochastic EP by optimizing the objective function proposed here. We verify the NEEP with the stochastic processes of the bead spring and discrete flashing ratchet models and also demonstrate that our method is applicable to high-dimensional data and can provide coarse-grained EP for Markov systems with unobservable states.

Figure 1

(a) Architecture of the NEEP. (b) Illustration of a MLP with three hidden layers for an $N=2$ bead-spring model where $s_t=(x_1^t,x_2^t)$.

Figure 2

(a) Entropy production rate as a function of $T_c/T_h$ for models with two and five beads. The solid lines (symbols) indicate the analytical EP rate $\dot{\sigma }$ (estimated EP rate ${\dot{\sigma }}_{\theta }$). (b) Cumulative EP over time $\tau$ along a single trajectory, which is randomly sampled from the test set. The inset shows the ensemble-averaged EP. (c) Local EP rate as a function of $x_1$ and $x_2$. Top, bottom: the NEEP and analytical results, respectively. (d),(e) ${\dot{\sigma }}_{\theta }$ with respect to training iteration for (d) two beads and (e) five beads. The left (right) inset corresponds to results before (after) training, showing scatter plots between $\mathrm{\Delta }S$ and $\mathrm{\Delta }{S}_{\theta }$ with a fitted linear regression line (solid red line). The results in (b)–(e) are performed at $T_c/T_h=0.1$. (f) Results of NEEP for high-dimensional bead-spring models. ${\dot{\sigma }}_{\theta }$ as a function of $N$ for each number of steps ($L$) are plotted with five different markers, as indicated in the legend. The $R^2$ values of the linear regression between $\mathrm{\Delta }S$ and $\mathrm{\Delta }{S}_{\theta }$ are shown in the inset. The red dashed line denotes $\dot{\sigma }$. Error bars and shaded areas represent the standard deviation of estimations from five independently trained estimators.

Figure 3

(a) NEEP architecture for discrete state Markov chains. (b) Schematic of a discrete flashing ratchet model. (c) Entropy production per step as a function of potential $V$. Error bars represent the standard deviation of ${\dot{\sigma }}_{\theta }$ from five independently trained estimators.

Figure 4

(a) RNEEP architecture for a hidden Markov model. (b) Results of RNEEP for a partial information problem. The estimations ${\dot{\mathrm{\Sigma }}}_{\theta }^n$ as a function of potential $V$ for each sequence length $n$ are plotted with six different markers as shown in the legend. The black $\mathsf{x}$’s are the semianalytical values of ${\dot{\mathrm{\Sigma }}}^{\infty }$. The inset shows a plot of the $y$ axis in log scale. The red (blue) dashed line denotes the analytic value of $\dot{\sigma }$ (${\dot{\mathrm{\Sigma }}}^2$). See Fig. S6 in the Supplemental Material [47] for a comparison with $\dot{\sigma }$ in linear scale. Error bars represent the standard deviation of estimations from five independently trained estimators.