Derivation of Markov processes that violate detailed balance

Time-reversal symmetry of the microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the condition of detailed balance. However, cyclic Markov processes that do not admit equilibrium distributions with detailed balance are often used to model systems driven out of equilibrium by external agents. I show that for a Markov model without detailed balance, an extended Markov model can be constructed, which explicitly includes the degrees of freedom for the driving agent and satisfies the detailed balance condition. The original cyclic Markov model for the driven system is then recovered as an approximation at early times by summing over the degrees of freedom for the driving agent. I also show that the widely accepted expression for the entropy production in a cyclic Markov model is actually a time derivative of an entropy component in the extended model. Further, I present an analytic expression for the entropy component that is hidden in the cyclic Markov model.

Figure 1

(a) Cyclic Markov model with three states. (b), (c) Examples of extended models where an additional degree of freedom $X$ is introduced.

Figure 2

Time evolution of three-state model with discrete time. Model 1: $a=b=\alpha =\beta =\gamma =0.25/\mathrm{\Delta }t$ and $c=0.5/\mathrm{\Delta }t$. Model 2: $a=b=\alpha =\beta =0.25/\mathrm{\Delta }t, c=1.0(X/N)\mathrm{\Delta }{t}^{-1}$, and $\gamma =0.5(1-X/N)\mathrm{\Delta }{t}^{-1}$. The probability distribution and currents are displayed as functions of time. (a) Probability distribution and currents of both models at early times. (b) Probability distribution and currents of model 1 at late times. For better visibility, $J$ was multiplied by 10. (c) Probability distribution and currents of model 2 at late times. For better visibility, $J$ was multiplied by 10.

Figure 3

Time evolution of three-state model with discrete time. Model 1: $a=b=\alpha =\beta =\gamma =0.25/\mathrm{\Delta }t$ and $c=0.5/\mathrm{\Delta }t$. Model 2: $a=b=\alpha =\beta =0.25/\mathrm{\Delta }t, c=1.0(X/N)\mathrm{\Delta }{t}^{-1}$, and $\gamma =0.5(1-X/N)\mathrm{\Delta }{t}^{-1}$. The entropy components are displayed as functions of time. (a) Entropy components of both models at early times. Separate constants were added to $S_{\mathrm{hk}}$ of the two models, so that their graphs are superposed. (b) Entropy components of model 1 at late times. (c) Entropy components of model 2 at late times. For each model, a constant was added to $S_{\mathrm{hk}}$ for easier comparison, and $S_{\mathrm{cyc}}$ was multiplied by 100 for better visibility.

Figure 4

Equilibrium distribution of models 1 and 2 as a function of $X$, shown in log scale. Parameters are set to the values used for generating Figs. 2 and 3.