Game-theoretical approach to minimum entropy productions in information thermodynamics

In a situation where each player has control over the transition probabilities of each subsystem, we game-theoretically analyze the optimization problem of minimizing both the partial entropy production of each subsystem and a penalty for failing to achieve a given state transition. In the regime of linear irreversible thermodynamics, we obtain the Nash equilibrium solution of the probability flow and calculate each partial entropy production for this solution. We find a trade-off such that a partial entropy production should be larger if we want the other partial entropy production to be smaller. The total entropy production can be minimized if each subsystem equally shares the penalty. We identify that this trade-off is due to the interacting contribution of the probability flow and discuss a possible biological validity for Escherichia coli chemotaxis.


I. INTRODUCTION
In physical systems such as living systems, there universally exist situations where many-body systems cooperatively perform a state transition to achieve a given task.Achievement of a given task generally takes a physical cost.When the state transition is performed by physical stochastic processes [1], the cost can be introduced as the entropy production in stochastic thermodynamics [2][3][4].Minimizing this entropy production might be crucial for the many-body systems to maintain their performance.
For many-body systems, the entropy production for the whole system can be partitioned into partial entropy productions of the subsystems [19,[35][36][37][38][39][40][41][42][43][44][45][46].This partitioning is originally introduced for non-autonomous system in information thermodynamics [47][48][49] and is applied to autonomous information processing in living systems [42,[50][51][52][53][54][55][56][57][58].Although the minimization problem of the partial entropy production is important to achieve an efficient information-to-free energy conversion [59][60][61][62][63], a situation of the two-body system is often seen that the one partial entropy production should be increased to reduce the other partial entropy production in a living system.The total entropy production for the whole system is not minimized in this situation, and this situation is not well treated in the conventional minimization problem of the total entropy production.Therefore, a framework has been required to deal with a certain kind of thermodynamic optimality when many-body systems cooperatively achieve a given task.Such optimality is often discussed in the field of game theory [64][65][66][67][68][69][70] or the mean-field game theory [71][72][73][74][75].For example, the prisoner's dilemma game is used to analyze a cooperative behavior of multi-agent systems in game theory [76], and the Nash equilibrium [77] gives the optimality for multi-agent systems.
In this study, we newly introduce a game-theoretic framework for a conflict between minimizations of partial entropy productions of two subsystems X and Y with a given task on the whole system.We treat this optimization problem with the Markov jump process for the bipartite system under the near-equilibrium condition and introduce the concept of Nash equilibrium into stochastic thermodynamics to explain this conflict.This Nash equilibrium solution shows a trade-off such that the partial entropy production of X should be increased when the partial entropy production of Y is reduced.Moreover, this conflict provides an inevitable dissipation in the Nash equilibrium solution, and this dissipation is expressed by the interacting contribution of the probability flow.We illustrate these facts by the numerical calculations and discuss a possible biological validity for Escherichia coli (E.coli) chemotaxis in terms of evolutionary processes.We also remark on our game-theoretic minimization of partial entropy productions in terms of optimal transport theory.

II. SETUP A. Markov jump process in bipartite model
We consider a system consisting of two subsystems X and Y , with binary states x ∈ {0, 1} and y ∈ {0, 1}.A probability distribution of state z = (x, y) ∈ {0, 1} 2 at time t is denoted as p (t) z .In this system, a Markov jump process from t = t 0 to FIG. 1. Schematics of settings.Blue bars indicate a probability distribution on space v, which changes with time from an initial state p (t 0 ) to a final state p (t 1 ) .Player X (Y) controls probability flows of e1, e3 (e2, e4), which generate its partial entropy production ΣX (ΣY ).In the final state, both the players take a penalty D generated by an error between the final state p (t 1 ) and the target state p f (black bars).t = t 1 is described as a master equation; z ′ →z and J z ′ →z are a transition rate and a probability flow in a state transition from z ′ = (x ′ , y ′ ) ∈ {0, 1} 2 to z at time t.By assuming a bipartite condition [38,39,[78][79][80] that the state transition is independently performed in the subsystems X and Y , the transition rate satisfies W (t) z ′ →z = 0 (x = x ′ , y = y ′ ), = 0 (otherwise).

(3a) (3b)
Because W (t) z ′ →z should not be zero if W (t) z→z ′ = 0, the absolute irreversible transitions are not assumed.

B. Game-theretical formulation
We further introduce players X and Y (see Fig. 2-(a)).Player X can control the set of the transition rates ŴX = {W This study proposes a problem of how the players cooperatively bring the final state p (t1) closer to the target p f from a given initial condition p (t0) = p i as efficiently as possible (see Fig. 1).To deal with this problem, we now introduce two types of costs.The first cost is relevant to the ongoing processes of state transition in each subsystem.Players X and Y independently cost amounts of Σ X and Σ Y , given by the partial entropy productions for a bipartite condition [38,39] Here, F z ′ →z is called a thermodynamic force.Σ X (Σ Y ) is always non-negative and gives 0 if and only if ).We can define the total entropy production as the sum of the partial entropy productions Σ tot = Σ X +Σ Y .This total entropy production quantifies the dissipation in the total system during the time interval from t = t 0 to t = t 1 [2,4] because the nonnegativity of the total entropy production Σ tot ≥ 0 can be regarded as the second law of thermodynamics.The entropy production is widely discussed in terms of the stability of the system [2,[81][82][83][84], the fluctuation of the observable [85,86], the speed of the time evolution [10,18,19,23], the accuracy of the information transmission [36-39, 47, 48] and so on.The partial entropy productions Σ X and Σ Y can also be interpreted as the dissipation in the subsystems X and Y , respectively.
The other cost is an error penalty of D to both players due to failing to achieve the target state at the end.We introduce this penalty as the Pearson's chi-square divergence [87], which is used in a statistical test as a degree of current state p (t1) achieving the target p f , Here, D is always non-negative and gives 0 if and only if the final and target states match, i.e., p (t1) = p f .We remark that this error penalty D is a kind of the standard f -divergence [34], which provides the Fisher information Thus, the result in this study is robust against the choice of the error penalty at least if the error penalty is given by the standard f -divergence and we only consider the small change dp f vi .For example, we can replace the error penalty D with the Kullback-Leibler divergence ), which is a kind of the standard f -divergence widely used in thermodynamics [2,83,84,[88][89][90].
For such costs Σ X , Σ Y , and D, we now consider a problem that each player X and Y aims to achieve the target state with as small partial entropy production as possible.Such a problem is given by two minimizations Here, we introduce the penalty parameter, λ X or λ Y , as the importance of achieving the target state for each subsystem.This study aims to find a Nash equilibrium solution [77] that satisfies minimization of costs C X and C Y simultaneously.We remark that −C X and −C Y correspond to the payoffs in game theory, and we consider the game that maximizes −C X and −C Y by changing the rate matrices ŴX and ŴY , and its optimal solution is obtained by a Nash equilibrium solution.

III. OVERVIEW OF MAIN RESULT
We now overview the main results of this study.In order to obtain the Nash equilibrium solution, we assume a nearequilibrium condition, where (the reciprocal of) the Onsager coefficient α ei is defined for the transition e i [2].We remark that the words "equilibrium" and "near-equilibrium condition" without the word "Nash" are used only for thermal equilibrium that is introduced by the detailed balance condition.To avoid confusion, we also use the term "Nash equilibrium" every time without abbreviation for distinction.We also define summations for each subsystem as α X = ei∈EX α ei and α Y = ei∈EY α ei .
Our goal is to compute the partial entropy productions of the subsystem X and Y in the Nash equilibrium, i.e., Σ N X and Σ N Y , especially when the given state transition is completely achieved p (t1) = p f in the limit λ X → ∞ with the fixed ratio of importance r := λ X /λ Y , or equivalently in the limit λ Y → ∞ with the fixed ratio r.We obtained the main results as follows, where Σ min X (Σ min Y ) is the minimum partial entropy productions of X (Y ), and Σ XY is the shared minimum entropy production, which is given by the minimum total entropy production The factor f X (r; γ) which is monotonically increasing (decreasing) with r.Thus, the minimum partial entropy production Σ min X (Σ min Y ) is achieved when r → 0 (r → ∞).f X (r; γ) and f Y (r; γ) are also functions of γ := α X /α Y .For r = 1, the factor is given by f X (1; γ) = 1/(γ + 1), f Y (1; γ) = γ/(γ + 1), and the minimum total entropy production is achieved Σ N tot = Σ min tot when r = 1.
The above result implies that there exists a trade-off relation between the partial entropy productions such that a partial entropy production should be larger if we want the other partial entropy production to be smaller.Moreover, the total entropy production can be minimized if each subsystem equally shares the penalty.That implies that an equivalent penalty in two subsystems reduces the dissipation in the total system, and the minimization of dissipation in only one subsystem increases the dissipation of the total system.

A. Physical interpretations based on information-energy conversion
Our results on the lower bound of the partial entropy productions can be immediately applied to the topic of the efficiency of the information-energy conversion in terms of Maxwell's demon.To discuss the information-energy conversion, we first decompose the thermodynamics force, i.e., Eq. ( 8), into the thermodynamic part (T) and information part (I) as The partial entropy production is also decomposed into two contributions [38,80] where the thermodynamic contribution means the entropy change of the heat bath, while the informational contribution means the sum of the entropy change of the system and information flow because F ei is given by the stochastic Shannon entropy change.For a bipartite system, the sum Σ I X + Σ I Y is regarded as the change of the Shannon entropy in the total system (X, Y ), which is given by the sum of the Shannon entropy in X and Y , and the mutual information between two subsystems X and Y .Because the partial entropy production Σ X ≥ 0 is nonnegative, we obtain the inequality which is called the second law of information thermodynamics.This result can explain the information-energy conversion by Maxwell's demon because this inequality explains a tradeoff between the thermodynamic contribution Σ T X and the informational contribution −Σ I X , and the thermodynamic contribution Σ T X can be negative due to the information contribution −Σ I X .The value of the partial entropy production Σ X is regarded as the dissipation in the information-energy conversion.For example, the equality Σ T X = −Σ I X (or Σ X = 0) holds for the Szilard engine, which is an optimal heat engine driven by Maxwell's demon to achieve the maximal efficiency for information-energy conversion [60].
Thus, a minimization problem of Σ X can be interpreted as a maximization problem of the efficiency for informationenergy conversion, and our study can be regarded as tackling a problem to maximize the efficiency for information-heat conversion under a restriction that the subsystems cooperatively achieve a given task within a finite time.Indeed, we obtained a positive bound for the partial entropy production which means that there is an inevitable dissipation Σ min X in the information-energy conversion caused by the finite-time task.

B. Example: E. coli chemotaxis and game-theoretic interpretation of E. coli evolution
This study might be applied to how E. coli evolves in its adaptation process in chemotaxis.To discuss the applicability, we first explain E. coli chemotaxis based on the bipartite model [91] (see Fig. 2 for visualization).The bipartite model of E. coli chemotaxis is organized by the activity of kinase (CheA), the methylation level of the receptor, and the ligand concentration in the environment.Let subsystem X denote whether the kinase is inactive (x = 0) or active (x = 1).On the other hand, let subsystem Y denote whether the receptor is demethylated (y = 0) methylated (y = 1).The activation of the kinase and the methylation of the receptor independently occurred stochastically.Thus, Eqs.(3a)-(3b) is satisfied, and the master equation is given by Eqs. ( 1)-( 2).In stochastic thermodynamics, heat dissipation can be introduced based on the following local detailed balance conditions, ln where β X and β Y are inverse temperatures in the kinase and the receptor, ∆Q is the heat dissipation of the kinase from state x ′ to state x under the condition of y, and ∆Q is the heat dissipation of the receptor from state y ′ to state y under the condition of x.The heat dissipation ∆Q (x ′ ,y)→(x,y) X can be given by the difference between the energy and chemical potential [80].Because the activation of the kinase is driven by the receptor-ligand binding, the heat dissipation ∆Q (x ′ ,y)→(x,y) X generally depends on the ligand concentration in the environment.
We here explain typical dynamics of adaptation in E. coli chemotaxis with this 4-state model.The adaptation is transient dynamics after the ligand concentration is changed.At time t = t 0 , we assume that the initial state stochastically tends to be (1, 0).This means that the kinase tends to be active, and the receptor tends to be demethylated.We here assume that the ligand concentration is changed from L = 0 to L = 1 at time t = t 0 , where L = 0 means that the ligand concentration is relatively low, and L = 1 means that the ligand concentration is relatively high, respectively.At time t > t 0 , the kinase activity can change from active state x = 1 to inactive state x = 0 rapidly, and this change activates other kinases CheY.The activity of CheY changes the mode of the flagellar motor from the tumbling mode to the run mode, and this mode change in the flagellar motor explains the behavior of chemotaxis.From time t = t 0 to t = t 1 , the kinase activity gradually returns to the active state x = 1, and the mode of the flagellar motor returns to the tumbling mode.This behavior is regarded as the adaption in E. coli chemotaxis because E. coli adapts to the environmental change from L = 0 to L = 1.In summary, the state stochastically tends to change in this adaptation dynamics as In terms of probability distribution, the adaptation dynamics is given by the initial condition p vi and final condition p v2 is relatively large rather than p v3 and p v2 and p (t1) v4 , respectively.We now consider evolutionary processes to minimize dissipation in the kinase and the receptor under the constraint of precise adaptation.Because the partial entropy production, especially the thermodynamic contributions Σ T X and Σ T Y , are related to the heat dissipation and ) can be interpreted as the local free energy difference under the condition of the other system Y (X) [48,59], the minimization of the partial entropy production is related to the savings of the local free energy as a thermodynamic resource.For E. coli, the thermodynamic resource like feed can be obtained as a consequence of chemotaxis.Thus, E. coli is motivated to achieve the task of chemotaxis precisely to gain the thermodynamic resource.This achievement of the task is measured by an error penalty D given by Eq. ( 9) for adaptation dynamics from t = t 0 to t = t 1 .To gain feed as a thermodynamic resource from the environment, E. coli would like to minimize this error penalty D, and its thermodynamic contributions of the penalty for the kinase (X) and the receptor (Y ) can be treated independently by multiplying the importance of achieving the target state in the adaptation dynamics such as λ X D and λ Y D. Thus, correspond to the net thermodynamic dissipations for the kinase and the receptor, respectively.The minimization problems of C X and C Y are needed for E. coli to efficiently survive an environment with low feed.In evolutionary processes, the phenotype of the kinase and the receptor can be changed accidentally, and the difference of the phenotype in adaptation dynamics can be quantified as the set of the transition rates ŴX for the kinase and ŴY for the receptor.In evolutionary processes, the phenotype can be changed to maximize the fitness of E. coli.To survive an environment with low feed, the fitness can be −C X and −C Y as the net free energy gain.Because mutations of the phenotype for the kinase and the receptor can happen independently, we can consider two minimization problems, which are equivalent to the maximization problems of the fitness If we assume that the current phenotype of the receptor and the kinase ŴX and ŴY in adaptation dynamics is obtained as a consequence of evolutionary processes, we can assume that the dissipation Σ X and Σ Y in the kinase and the methylation may be discussed in terms of the Nash equilibrium solutions Σ N X and Σ N Y for two minimization problems Eqs.( 30) and (31).
Based on the above evolutionary processes, we can discuss a possible validity of our game-theoretic framework in E. coli chemotaxis.Now, we may assume that the transition from the initial state to the final state for the methylation level of the receptor is more important than that for the kinase activity in the adaptation dynamics.That is because the methylation level tends to be changed from the demethylated state y = 0 at time t = t 0 to the methylated state y = 1 at time t = t 1 during the adaptation dynamics.On the other hand, the kinase activity is only instantaneously changed during the adaptation dynamics, and the initial state x = 1 at time t = t 0 tends to be the same as the final state x = 1 at time t = t 1 .Thus, achievement of the target state p (t1) vi may be more important for the receptor than for the kinase in adaptation dynamics.In our game-theoretic framework, the difference in the importance may be quantified as λ Y ≫ λ X .Experimentally, Σ Y ≫ Σ X can be seen in adaptation dynamics of E. coli because the relaxation time of the receptor is relatively slower than the relaxation time of the kinase [42,91].This fact is consistent with our main result that Σ N Y ≫ Σ N X for λ Y ≫ λ X .Thus, our main result may provide a possible explanation of the huge difference in thermodynamic dissipation of each subsystem in the signal transduction of E. coli chemotaxis, which is based on the game-theoretic evolution in an environment with low feed under the constraint of the precise adaptation dynamics.

V. NASH EQUILIBRIUM SOLUTION IN LINEAR IRREVERSIBLE THERMODYNAMICS
In general, the problem of Eqs.(10a) and (10b) does not give a non-trivial conclusion without any constraint because the entropy production can be zero without any constraint for the Markov jump process.In this study, we consider a constraint of the near-equilibrium condition in linear irreversible thermodynamics [2,81].In linear irreversible thermodynamics, we assume that the transition rate is given by W where δW is the small change of the transition rate and W eq z ′ →z satisfies the detailed balance condition W eq z ′ →z p eq z ′ = W eq z→z ′ p eq z with the equilibrium distribution p eq z for all pairs of z and z ′ .We also assume that the initial state satisfies the near-equilibrium condition.For example, p (t0) is the steady state distribution for W eq z ′ →z + O(δW ).In the above setup, we can confirm J (t) = O(δW ) and p (t) = p eq + O(δW ) during the transition process t 0 ≤ t ≤ t 1 := t 0 + τ .
We here explain the Nash equilibrium solution (see also Appendix A-C for detailed derivation).Let α z ′ →z := (W eq z ′ →z p eq z ′ ) −1 = (W eq z→z ′ p eq z ) −1 be (the reciprocal of) the Onsager coefficient.Σ X and Σ Y become quadratic functions by ignoring terms O(δW 3 ), and thus the lower bounds are given by Σ where Jei is time-averaged flow defined as Jei := ( t0+τ t0 dtJ (t) ei )/τ .The penalty D is also given by the quadratic function of Jei .Thus, we can describe the lower bounds of cost CX ( JX , JY )(≤ C X ) and CY ( JX , JY )(≤ C Y ) as functions of time-averaged flows Optimal time-averaged flows to minimize the cost are described by Thus, the Nash equilibrium solution for time-averaged flows as the fixed point of Eqs.(32a) and (32b).
The analytical calculation of the Nash equilibrium solution is as follows (see also Appendix A-C).From Eqs. (33a)-(32b), JN X and JN Y satisfy extreme value conditions for C X ( JX , JY ) and C Y ( JX , JY ), respectively.Because C X ( JX , JY ) and C Y ( JX , JY ) are quadratic functions of ( JN ) i := JN ei , the solution of the extreme value conditions JN are analytically obtained.Especially in the limit λ X →∞ with the fixed ratio r, JN are given by for e i ∈ E X , and for e i ∈ E Y , where the permutation σ 2 (i) and σ 3 (i) are defined as σ 2 (i) = σ(σ(i)) and σ 3 (i) = σ(σ(σ(i))), respectively.

VI. MINIMUM PARTIAL ENTROPY PRODUCTIONS FOR THE NASH EQUILIBRIUM SOLUTION
We discuss the minimum partial entropy productions for the Nash equilibrium solution, that is (Σ N X , Σ N Y ) := (Σ X , Σ Y )| J (t) = J N .In the limit λ X →∞ with the fixed ratio r, 11) and ( 13) are analytically given by where the flow J N X , J N Y and J N XY are the linear transformation of JN defined as Here, J N X ( J N Y ) implies the probability flow for the marginal distribution of X (Y ).J N XY represents the probability flow of the interaction between X and Y , which cannot be written by J N X and J N Y .The probability flow J N rot does not contribute to the time evolution dp (t) /dt = BT −1 J N because BT −1 (0, 0, 0, J N rot ) T = 0, and thus J N rot can be interpreted in terms of optimal transport theory [30], which explains the minimization of the entropy production in a finite time [23] (see Appendix D).We remark that Σ min X , Σ min Y , Σ XY are proportional to 1/τ , and this fact is consistent with the thermodynamic speed limit based on optimal transport theory [10,15,18,19,23].Indeed, Eq. ( 36) can be interpreted as a thermodynamic speed for the partial entropy production (see Appendix D).Moreover, there is one-to-one correspondence between J N rot and r (see Appendix B).Thus, the minimization of the partial entropy production Σ N X = Σ min In Fig. 3, we illustrate a trade-off relation between Σ N X and Σ N Y by a behavior of probability flows J N X and the minimum entropy productions Σ N X , Σ N Y , Σ N tot for the Nash equilibrium solution in the case of λ X →∞ with the fixed r := λ X /λ Y .As seen in Fig. 3(a), only J N rot monotonically changes with r, and J N X , J N Y and J N XY do not depend on r (see also Appendix B).As seen in Fig. 3(b), there is the trade-off relation between Σ N X and Σ N Y in terms of r.We can see that when r→∞.We briefly summarize results for cases of finite values of λ X and λ Y discussed in Appendix E. For finite λ X and λ Y , J N X , J N Y , J N XY generally depend on r.Nevertheless, we can see a trade-off relation between Σ N X and Σ N Y for finite λ X and λ Y because the dependence of J N rot , Σ N X , and Σ N Y on r is similar to the case of λ X →∞ with the fixed ratio r.

VII. CONCLUSION AND DISCUSSION
This study incorporated a game-theoretic approach into the minimization problems of the partial entropy productions.As a representative example, we consider bipartite systems where two minimization problems of the partial entropy productions can be conflicted.We consider a problem that the partial entropy productions should be minimized for each subsystem with a given state transition for whole systems.We formulated the solution to this problem as Nash equilibrium and analytically calculated it.We found that both subsystems inevitably take their necessary partial entropy productions for achieving the task in a finite time.Lower bounds of the partial entropy productions for each subsystem and the total system are analytically derived.Interestingly, there is a trade-off relation between the minimum partial entropy productions of the subsystems.Furthermore, the minimum values of these partial entropy productions and the total entropy production cannot be achieved at the same time.
Our findings may apply to several interesting topics in thermodynamics.For example, our result may explain the minimum thermodynamic cost and optimal protocol in the finitetime information-energy conversion.The analytical expressions of the minimum entropy productions Eqs. ( 36) and ( 37) can be interpreted as the thermodynamic speed limit of information thermodynamics based on optimal transport theory.The above findings may also explain how large the dissipation of each subsystem in the signal transduction of E. coli chemotaxis as a consequence of the evolutionary processes.
In general, such a game-theoretic conflict of the minimum partial entropy productions arises in a more complex system such as the living organism, whereas this study only considered a simple bipartite system.For example, thermodynamics of the membrane transport [92,93] and its information transmission [58] can be discussed by using a more complex system.To understand a game-theoretic conflict in other living systems such as the membrane transport, we may need to generalize our result.Especially, how the trade-off of the minimum partial entropy productions between several subsystems emerges for complex systems is still questioned.To solve this question, we may need to consider multi-body interactions and conflicts between several subsystems.We believe that such a game-theoretic minimization of the partial entropy productions is important to explain the thermodynamic structure of the complex living systems as a consequence of evolutionary optimization.
Appendix A: Calculation of Nash equilibrium solution In this section, we concretely calculate the Nash equilibrium solution Eqs. (33a) and (33b) under the near-equilibrium condition in linear irreversible thermodynamics [2].By introducing the near-equilibrium condition, the minimization problem is considered within the region of linear response.Outside of the region of linear response, the minimization problem for the transition rates ŴX and ŴY has a trivial solution that gives the zero entropy production without any constraints.Therefore, we only consider this problem within the non-trivial region of linear response where the non-zero entropy production can be obtained under the constraint that the Onsager coefficient is fixed.
In linear irreversible thermodynamics, (the reciprocal of) the Onsager coefficient defined as α z ′ →z := (W eq z ′ →z p eq z ′ ) −1 = (W eq z→z ′ p eq z ) −1 leads to the linear relation 80].By using this Onsager coefficient, the partial entropy production is approximated as by ignoring the term O(δW 3 ).
To introduce a time-averaged probability flow Jz ′ →z := ( z ′ →z )/τ , we solve the minimization problem of Eq. (10a).Because dt( Jei ) 2 , the lower bound on Σ X is given by where JX := { Jei |e i ∈ E X } and JY := { Jei |e i ∈ E X } are the sets of Jei .The penalty D is also given by the function of Jei ; where δp vi := p i vi − p f vi .Thus, the minimization problem of Eq. (10a) is rewritten as under the near-equilibrium condition.For the subsystem Y , we also rewrite the minimization problem Eq. (10b) as The above minimization problem for the player X (Y) is bounded by the minimization on the time-averaged probability flows JX ( JY ).The Nash equilibrium solution for the time-averaged probability flow ( JN X , JN Y ) is given by as the fixed point of the following two solutions in the minimization problems Eqs.(A5) and (A7), Eqs. (A11a)-(A11b) show that C X ( JX , JY ) and C Y ( JX , JY ) take extreme values under the Nash equilibrium condition in the directions of JX and JY , respectively.Thus, by using we calculate the extreme value conditions as (A13) By using the vector notation ( JN ) i := JN ei , these extreme value conditions are rewritten as where we used Therefore, the Nash equilibrium solution is analytically solved by using Cramer's rule ( JN ) i = det(M i )/det(M).Here, M i denotes a matrix M whose i-th column is replaced by δq.
To calculate this Cramer's rule, we use the following notations for simplicity, and the matrix M and M i are given by The analytical solutions of det M and det M i are given by and Thus, ( JN ) i = det(M i )/det(M) is analytically obtained.
In particular, we consider ( JN ) i in the limit λ X → ∞ for the fixed , the second-order, the third-order and the fourth-order of αi give O(λ −2 X ).Thus, we obtain and where we used pi + pσ(i) + pσ 2 (i) + pσ 3 (i) = j pj = 1 and δi + δσ(i) + δσ 2 (i) + δσ 3 (i) = j δj = j (p i vj − p f vj )/τ = 0.By using these equations and substituting the definitions of Eqs.(A17) into Eq.(A26), we compute the flow JN ei as which corresponds to Eqs. ( 34) and (35), for e i ∈ E X , and for e i ∈ E Y .
Appendix B: Analytical expressions of J N In this section, we calculate J N in the limit λ X → ∞ with the fixed ratio r.From the definition of J N in Eqs. ( 39)-( 41), we obtain where we used i δp vi = 0, /τ can be obtained from Eq. (A27) as follows, where we used /τ is consistent with δp = τ B JN where (δp) i = δp vi .We can see that these flows are determined only by the state transition δp and independent of r.
We next discuss J N rot .To calculate J N rot , we obtain JN ei r=1 and JN ei − JN ei r=1 from Eqs. (A28)-(A31) as follows, where we used α X = α e1 + α e3 , α Y = α e2 + α e4 and i δp vi = 0. From these equations, we obtain where we used i α e σ 2 (i) Here, we can see that J N rot depends on r and a strength of this dependence is determined by J N XY .This J N rot means a counterclockwise rotation of probability flows from the definition, and J N rot does not contribute to the time evolution of probability distribution δp.Because detT = 1, T has the inverse matrix and JN = T −1 J N .By using JN = T −1 J N , and δp = τ B JN , we can describe δp as the linear function of which implies that J N rot does not contribute to δp.Whereas J N rot does not contribute to δp, J N X and J N Y and J N XY contribute to the time evolution δp.In Eqs.(B1) and (B2), we obtain if we define total probability flows in the marginal distributions δP X := y δp (1,y) = δp v2 + δp v3 , and δP Y := x δp (x,1) = δp v3 + δp v4 .Thus, J N X and J N Y correspond to the probability flows in the marginal distributions of subsystem X and Y , respectively.Here, J N XY represents probability flows concerning the interaction between the subsystems, which cannot be written only by the contribution of J N X and J N Y .
Appendix C: Calculation of partial entropy productions in Nash equilibrium In this section, we calculate Σ N X and Σ N Y in the limits λ X → ∞ with the fixed ratio r = λ X /λ Y and derive Eqs. ( 11) and ( 13).To calculate Σ N X , we use the following notations By using these notations, we obtain where we used J N X = (δp v2 + δp v3 )/τ , α Y = α e2 + α e4 , and i δp vi = 0. We also obtain where we used Eq.(B3).By using these equations, J N X = (δp v2 + δp v3 )/τ , i δp vi = 0, α X = α e1 + α e3 , and Eqs.(A28) and (A30), we calculate Σ N X as follows, , we obtain Eq. ( 11), To calculate Σ N Y , we use the following notations By using these notations, we obtain where we used J N Y = (δp v3 + δp v4 )/τ , α X = α e1 + α e3 and i δp vi = 0. We also obtain where we used Eq.(B3).By using these equations, J N Y = (δp v3 + δp v4 )/τ , i δp vi = 0, α Y = α e2 + α e4 , and Eqs.(A29) and (A31), we calculate Σ N Y as follows, , we obtain Eq. ( 13), and its minimization for the fixed initial and final states P (t0) (x) and P (t1) (x) is given by where (ν * (t) ) t0≤t≤t1 provides the same time evolution for (ν (t) ) t0≤t≤t1 , Here, we only assume that (ν (t) ) t0≤t≤t1 satisfies the boundary conditions P (t0) (x) and P (t1) (x) with the continuity equation In optimal transport theory, this minimization is given by the L 2 -Wasserstein distance W(P (0) , P (τ ) ) [25] defined as thus the lower bound on the entropy production [10] is obtained as This result is also recently discussed in terms of the thermodynamic speed limit [19,20].The thermodynamic speed limit provides the lower bound on the entropy production which is proportional to 1/τ .In Ref. [19], the thermodynamic speed limits for the partial entropy production Σ X and Σ Y are also obtained.To generalize the above fact for dynamics of the discrete state described by the Markov jump processes, J. Maas proposed a correspondence of the L 2 -Wasserstein distance [30] for the Markov jump processes.We also discussed the minimum entropy production based on this correspondence of the L 2 -Wasserstein distance for the Markov jump process [23].For general Markov jump networks, for example, a bipartite model in the letter, we can use the incidence matrix as follows, where B is the incidence matrix and (J (t) ) ρ = J (t) is the vector of the flow on each edge, where J z for the directed edge ρ = z ′ → z.We only assume that dp (t) /dt = BJ (t) satisfies the boundary condition p (t0) = p i and p (t1) = p f as discussed in the letter.The entropy production is obtained as the product of the force (F where ρ is (the reciprocal of) the edgewise Onsager coefficient that satisfies ρ .For the fixed α ρ , the minimum entropy production cannot be zero for the given transition from p (t0) to p (t1) .This condition is satisfied automatically in the framework of linear irreversible thermodynamics.The minimum entropy production is given by where (J * (t) ρ ) t0≤t≤t1 provides the same time evolution for (J Here, we only assume that (J (t) ) t0≤t≤t1 satisfies the boundary conditions p (t0) = p i and p (t1) = p f with dp (t) /dt = BJ (t) .If we compare this result with the result for the continuous case, we can find that the incidence matrix B corresponds to t) corresponds to ν (t) (x)P (t) (x) and α ρ corresponds to 1/[µT P (t) (x)] respectively.In terms of the kernel, means J (t) − J * (t) ∈ KerB.We also can define a correspondence of the L 2 -Wasserstein distance by using W(P (0) , P (τ ) ) = min ) that provides the thermodynamic speed limit for the Markov jump networks [23], Σ ≥ [ W(P (0) , P (τ ) )] 2 τ . (D14) We here consider our result for the bipartite model in the letter from the viewpoint of optimal transport theory and the minimization of the entropy production.The concept of the rotation flow J N rot , which does not contribute to the time evolution of probability distribution, is related to the quantity J (t) − J * (t) in the above discussion.Indeed, the probability flow J N rot does not contribute the time evolution Thus, the contribution of the rotation flow T −1 (0, 0, 0, J N rot ) T ∈ KerB corresponds to J (t) −J * (t) in the optimal transport theory.This rotational flow J N rot can be identified with the cycle flow for the bipartite system and the minimum entropy production can be achieved when J (t) − J * (t) vanishes without any assumption in the theory [23].
In the letter, we consider a more complicated problem of the minimum entropy productions by focusing on the Nash equilibrium solution, which is not discussed in Ref. [23].Thus, the minimum entropy production is not necessarily achieved when J (t) − J * (t) = 0 or J N rot = 0.However, the minimum total entropy production Σ min tot , and the minimum partial entropy productions Σ min X and Σ min Y can be achieved when the mode of rotation flow J N rot is changed.We also remark on the thermodynamic speed limit for partial entropy production.In the main result, we showed that under the constraint on min ŴY C Y , and under the constraint on min ŴX C X when λ X →∞ and λ Y →∞ that means p (t0) = p i and p (t1) = p f .Here, the time evolution of the marginal distributions, defined as P Appendix E: Nash equilibrium solution on finite weights of error penalty In the letter, we considered the Nash equilibrium solution and partial entropy production on infinite weights of error penalty, i.e., λ X → ∞ with the fixed ratio r, where the final state completely matches the target state p (t1) = p f .In this section, we consider cases of finite values of λ X and λ Y .X , Σ N Y , and Σ N tot .We can also see the trade-off relation between partial entropy productions for the finite values of λ X and λ Y .The minimum partial entropy production in X is achieved when r → 0 and the minimum partial entropy production in Y is achieved when r → ∞.However, the minimum total entropy production is not always achieved when r = 1.Fig. E2 shows the dependence of Σ N X on λ X .From this figure, we see that Σ N X is almost zero in the limit λ X → 0. The partial entropy production Σ N X monotonically increases as λ X becomes larger.This monotonic behavior implies the effect of the penalty of a given state transition in the minimization problem of the partial entropy productions.If the penalty is smaller, the partial entropy production can be minimized much more.In the limit λ X → 0, a state transition in the subsystem X does not occur to maintain the partial entropy production to be zero.

FIG. E 2. Dependence of Σ N
X on λX .The various parameters other than λX are equal to those in Fig. 3.The horizontal axis is log r again.The color difference represents the difference in λX , and the closer it gets from blue to orange, the larger λX is.In final, the line for λX = 10 15 almost matches the limit value of λX → ∞ represented by the dashed line in black.

FIG. 2 .
FIG. 2.A physical example of a bipartite system, signal transduction of E. coli chemotaxis.Subsystem X takes an active (orange-colored dot) or inactive (uncolored dot) state of a kinase.The other subsystem Y takes a methylated (green-colored dot) or demethylated (uncolored dot) state of the receptor.During a state transition, heat dissipation emerges for each subsystem as the partial entropy production.

2 τ
( ei∈EY α −1 ei ) , (D22)which are regarded as thermodynamic speed limits for the partial entropy productions Σ X and Σ Y under the constraint on min ŴY C Y and min ŴX C X , respectively.
Fig. E1 considers the Nash equilibrium when λ X is fixed to a finite value, whereas λ Y varies.Fig. E1-(a) shows probability flows J N .As different to the case in λ X → ∞ with the fixed ratio r, J N X , J N Y , and J N XY are not invariant on r.Fig. E1-(b) shows the entropy productions of Σ N

FIG. E 1 .
FIG. E 1. (a) Probability flows and (b) entropy productions in λX = 10 and 10 −4 ≤ r ≤ 10 4 .The other various parameters are equal to those in Fig. 3.The horizontal axis indicates log r again.In panel (a), the orange, green, blue, and red lines indicate J N X , J N Y , J N XY , and J N rot , respectively.In panel (b), the orange, green, and black lines indicate Σ N X , Σ N Y , and Σ N tot , respectively.