The thermodynamics of computational copying in biochemical systems

Living cells use readout molecules to record the state of receptor proteins, similar to measurements or copies in typical computational devices. But is this analogy rigorous? Can cells be optimally efficient, and if not, why? We show that, as in computation, a canonical biochemical readout network generates correlations; extracting no work from these correlations sets a lower bound on dissipation. For general input, the biochemical network cannot reach this bound, even with arbitrarily slow reactions or weak thermodynamic driving. It faces an accuracy-dissipation trade-off that is qualitatively distinct from and worse than implied by the bound, and more complex steady-state copy processes cannot perform better. Nonetheless, the cost remains close to the thermodynamic bound unless accuracy is extremely high. Additionally, we show that biomolecular reactions could be used in thermodynamically optimal devices under exogenous manipulation of chemical fuels, suggesting an experimental system for testing computational thermodynamics.


I. INTRODUCTION
If it were possible to perform many measurements using a single bit of memory without putting in work, Maxwell's demon could use the information gained to violate the second law of thermodynamics and extract net work from an equilibrium system. Landauer's insight that computational processes require a physical instantiation and therefore have thermodynamic consequences [1][2][3] is key to exorcising the demon, and the survival of the second law has been demonstrated in a range of physical models [2][3][4]. If, unlike Maxwell's thought experiment, the correlations generated by a measurement or copy are not used to perform work, the cycle increases the entropy of the universe (by at least k ln 2 if the measurement is binary, perfectly accurate, and has a 50=50 outcome) [2,4,5]. Landauer and others have provided specific physical implementations of binary devices along with protocols that achieve the thermodynamic bound for measurement cycles [2,3] or memory erasure protocols [1,[6][7][8][9]. Examples include magnetic systems [1,2,9] or single particles in pistons [4,5,[10][11][12][13].
Do biomolecules perform measurement and copying within this computational paradigm? Many biological processes involve creating long-lived molecular copies of other molecules [2,14,15]. Perhaps the most tantalizing analogy is in the cellular sensing of external ligand concentrations. Following the seminal work of Berg and Purcell [16], it has been shown that cells can reduce their sensing error by averaging a noisy receptor signal over time [17][18][19][20][21][22][23][24]. Recent studies claim that cells implement time integration by dissipatively copying receptor states into the chemical modification states of readout molecules [24][25][26]. Other authors have highlighted the necessity of dissipation in adaption [27][28][29] and kinetic proofreading [30,31].
While it has been noted that there is a connection between the dissipation present in cellular copying and the thermodynamics of computation [24][25][26], the nature of the connection remains nebulous. How do cellular protocols compare to the canonical copy protocols typically considered in the computational literature? Can cellular systems reach the fundamental thermodynamic limit on the accuracy and energetic cost of a measurement? If not, what is the underlying reason for the additional dissipation? Is it due to the nature of the biomolecular reactions, or due to the design of the signaling network? And if cellular systems cannot reach the fundamental limit, how does the trade-off between energy and precision differ from the ideal case? To answer all of these questions, it is necessary to construct a rigorous mapping between cellular processes and computational copying. Understanding the connection between the thermodynamics of computation and the thermodynamics of biological processes [24,25,27,30,[32][33][34][35][36]] at a mechanistic level would enable translating quantitative, not just qualitative, results from the literature on computation.
We rigorously map a canonical push-pull signaling motif to a computational copy device at the level of the master equation. We thereby identify a lower bound on dissipation that arises from the failure to exploit correlations generated between receptors and readouts (rather than the widely discussed costs of "erasure" [1,[6][7][8][9]25]). Our mapping demonstrates that the push-pull network cannot converge on this fundamental limit for a general input signal, regardless of its parameters, and we prove that more complex biochemical networks involving multiple steps or parallel pathways cannot perform any better. Remarkably, however, cellular systems can operate close to the lower bound at moderate-to-high accuracy, even at a high rate of copying. Further, cellular networks are naturally adaptive, dissipating less when the sampling challenge is reduced. Finally, we show that an artificial copy device based on biochemical reactions can achieve the thermodynamic bound with exogenous manipulation of fuel concentrations, providing an alternative platform for investigating the thermodynamics of computation.

II. PUSH-PULL SYSTEM
To explore biochemical copying, we consider bifunctional kinase systems, which are common in bacteria [37]. The bifunctional kinase tends to either phosphorylate or dephosphorylate a readout x, depending on the ligandbinding state of the receptor to which the kinase is coupled (Fig. 1). If the receptor R is bound to ligand L, then the bifunctional kinase acts as a kinase, catalyzing phosphorylation and dephosphorylation reactions that are both coupled to hydrolysis of adenosine triphosphate (ATP). If the receptor is not bound to the ligand, then the bifunctional kinase acts as a phosphatase, catalyzing phosphorylation and dephosphorylation reactions that are uncoupled from ATP hydrolysis. Such a system can be described by the following reactions: where the kinase or phosphatase activity is coarse grained into the ligand-binding state of the receptor and ADP and P stand for adenosine diphosphate and inorganic phosphate, respectively. Here, x and x Ã represent unphosphorylated and phosphorylated readout states, respectively. For simplicity, we take the concentration [L] to be constant, and assume the system is maintained in a nonequilibrium steady state: [ADP], [ATP], and [P] are fixed. We treat phosphorylation and dephosphorylation as instantaneous second-order reactions. Thus, The master equation for this system, and the chemical kinetics approximation, are provided in Discussion 1 in the Supplemental Material [38]. We emphasize that the reactions within each equation are the microscopic reverses of each other, while the reactions of the second and third equations correspond to distinct reaction paths. This yields a thermodynamically consistent model. Qualitatively, this circuit performs copies (or measurements) in the following way. As spontaneous phosphorylation and dephosphorylation in the absence of the bifunctional kinase occur at a low rate which we take to be zero, the two chemical modification states of the readout are analogous to two stable states of a memory bit separated by a large barrier, as widely considered in the computational literature [2,3,14]. There are, however, two separate paths between the two wells, via exchange of phosphate with ATP and via exchange of phosphate with the cytosol. Moreover, the ATP-independent (de)phosphorylation reaction has a high yield of x in equilibrium, whereas the ATPcoupled reaction has an intrinsically high yield of x Ã . The resultant extended free-energy landscape for a single readout that explicitly considers ATP turnover is illustrated in Fig. 1(b). The presence of RL lowers the barrier between pairs of states connected by ATP turnover, whereas the presence of R lowers the barrier between states not connected by ATP turnover. In this way the receptor's binding state effectively restricts the free-energy landscape to favor either x or x Ã , which can then be thought of as copying the state of the receptor into the chemical modification state of the readout [24,25]. In the next section, we make this analogy concrete, allowing a quantitative analysis of biochemical copying. The signaling network utilizes a receptor that acts as a bifunctional kinase or phosphatase: when bound to a ligand, it catalyzes the activation of the downstream readout; when unbound, it catalyzes the deactivation of the downstream readout. Free-energy dissipation due to the use of fuel, coarse grained from this representation, drives the reactions. (b) Schematic free-energy landscape of a single readout molecule in the biochemical network. We plot the free energy G as a function of the number of ATP molecules that are converted into ADP molecules, for the two states x and x Ã . Thermodynamically favorable transitions are shown with solid arrows, and unfavorable transitions with dashed arrows. The presence of catalysts R and RL makes these transitions faster, and thereby push the system towards x or x Ã , but the overall thermodynamic drive is fixed for both reactions.
Since each readout molecule provides a stable memory of the receptor state, the readout molecules collectively provide information on the state of the receptor in the recent past. This enables time integration of the receptor signal and, hence, enhanced accuracy of concentration estimates [16,[24][25][26]. We are not concerned, however, with the precision of sensing a constant concentration [16,17,[24][25][26]39], nor with the "learning rate" between the ligand concentration and the network, which is important when changes in external concentrations are more rapid [40][41][42]. Rather, we are interested in whether the readout reaction can be rigorously described in terms of a copy process, and if so, how this cellular protocol compares to optimal quasistatic computational protocols involving the manipulation of energy landscapes that are common in the literature [2,3,14].

III. RESULTS
A. Mapping the biochemical network to a copy process Let us consider the dynamics necessary for the biochemical network to be described rigorously as a stochastic copying system, in which randomly selected data bits (receptors) are copied into randomly selected memory bits (readouts) at certain rates. To make such a mapping, each readout molecule should perform a copy of any one ligandbound receptor at a rate k copy RL , and any one unbound receptor at a rate k copy R . These copies are performed with accuracies s RL and s R , respectively, and the result of the copy should be independent of the prior state of the readout. Thus, a copy of RL (R) returns x Ã (x) with probability s RL (s R ). Note that the state of the readout can be identical both before and after the copy, just as bits overwritten with new data may have the same value as before; transitions and copies are not equivalent. Indeed, if the network is to be described as a copy process, then readouts in state x should be converted into readouts in state x Ã at a rate and x Ã should be converted into x at a rate Here, N R and N RL are numbers of receptors, and N x and N x Ã the numbers of readouts, in each state. By definition, copies are made at a rate ðk copy RL N RL þ k R copy N R ÞN x T , with N x T the total number of readouts.
Returning to our actual biochemical network, Eq. (2) specifies the dynamics of x and x Ã . The transitions occur with rates Comparing Eqs. (3)-(5), we see that the biochemical network and stochastic copy process are equivalent at the level of transition rates, which specify the master equation, if we perform the mapping in which V is the volume of the system and the copy accuracies are Thus, the average copy rate is in which square brackets indicate a concentration within the volume V. The biochemical network can, therefore, be directly mapped to a stochastic copying process.

B. Energetic cost per copy cycle
In the nonequilibrium steady state, the average dissipation rate of chemical free energy ( _ w chem ) is where _ n flux is the average current of readout molecules around the phosphorylation-dephospohorylation loop; Δμ 1 ¼ μ ADP − μ ATP and Δμ 2 ¼ μ P . The sum −ðΔμ 1 þ Δμ 2 Þ is the free energy of ATP hydrolysis, which is dissipated when a readout goes around this cycle once.
To proceed, we introduce the following averages: with N R T ¼ N RL þ N R the total number of receptors. In the mean-field limit, _ n flux follows from Eq. (2) as Furthermore, the fraction of phosphorylated readout f is For the full calculation, we refer to Discussion 1 in Supplemental Material [38]. The mean-field approach holds in the limit of many receptors, or when the readout phosphorylation dynamics is slower than the receptorligand dynamics, as required for the mechanism of time integration. If these conditions are not met, a given readout performs many copies of autocorrelated data. Detailed analysis of this regime is beyond the scope of this work, but a brief discussion is provided in Discussion 1 of Supplemental Material [38]. Given the rate of copying [Eq. (8)] and the rate at which chemical work is done [Eq. (13)], we are now in a position to calculate the chemical work done per copy cycle: This result can be simplified by noting that the probability a copy is made of RL is not p, but rather Indeed, if k 3 þ k 4 > k 5 þ k 6 , a given ligand-bound receptor is more frequently copied than a given unbound receptor molecule (see Fig. 1). Using the expression for p 0 , Eq. (15), the fractional yield of phosphorylated readout can be written in the intuitive form f ¼ p 0 s RL þ ð1 − p 0 Þs R , and Eq. (14) can be simplified to The quantity kT lnðk 3 k 6 =ðk 4 k 5 ÞÞ has dimensions of energy and is equal to the chemical work done during a single phosphorylation-dephosphorylation cycle. We can split it into an energy related to the accuracy of copying RL, E s RL ¼ kT lnðk 3 =k 4 Þ ¼ kT ln½s RL =ð1 − s RL Þ, and an energy related to the accuracy of copying R, E s R ¼ kT Note that, although E s R þ E s RL ¼ −Δμ 1 − Δμ 2 , E s R and E s RL incorporate differences in internal free energy between x and x Ã and, hence, E s RL ≠ −Δμ 1 and E s R ≠ −Δμ 2 , in general. Inverting the sign of E s R and E s RL corresponds to a mirror-image encoding, in which RL is copied to x and R to x Ã . Low accuracy corresponds to E s R , E s RL → 0, when A similar analysis, with an equivalent outcome, is performed in Discussion 2 of Supplemental Material for a related system in which receptors act only as kinases [38].

C. Optimal devices set a thermodynamic bound on the dissipation of the biochemical network
The biochemical network has the dynamics of a process in which readouts randomly perform copies of receptors. It is also fundamentally dissipative-does the fact that it is a copy process set a practically relevant thermodynamic bound on this dissipation?
We can calculate the minimal thermodynamic cost of the random computational copy process that is equivalent to our biochemical network motif. In each copy operation of this equivalent process, a memory bit M is exposed to an initially uncorrelated data bit D [in state 1 with probability pðd ¼ 1Þ ¼ p 0 as defined in Eq. (15)]-the final result is a memory bit in state 1 with probability Mutual information I is then generated between the data and the memory during each individual copy of the equivalent process, with We emphasize that the above expression is simply the mutual information between bits M and D at the end of a single discrete copy operation, calculated directly as IðM; DÞ ¼ P m;d pðm; dÞ ln½pðm; dÞ=pðmÞpðdÞ. Here, pðm; dÞ is determined straightforwardly from pðm; dÞ ¼ pðmjdÞpðdÞ, using the expressions above for pðdÞ and pðmjdÞ in terms of p 0 and the measurement accuracies s R and s RL .
Immediately after copying, the data (receptor) and memory (readout) are decoupled without loss of information. This means that they remain correlated, even though there is no direct physical interaction between the data and memory anymore. Generating correlations that persist after direct interactions cease implies pushing the combined system out of equilibrium: the free energy required is kTI [4,15,43]. Making computational copies thus amounts to storing free energy in mutual information, and if this information is not used to extract work (as done by an efficient Maxwell demon) but simply lost in an uncontrolled fashion, then the process is irreversible and the information lost sets a lower bound on dissipation. In the random computational copy process to which we map our biochemical network (see Sec. III A), the stored information is not used to extract work, and, hence, kTI sets a lower bound on entropy generation for the entire cyclic copy operation [4]. For completeness, a typical copying device and a set of quasistatic protocols that can achieve this bound for various input data are given in Fig. 1 of the Supplemental Material and analyzed in Discussion 3 therein [38].
The biochemical network is equivalent to the random copy process that generates information Iðs R ; s RL ; p 0 Þ given by Eq. (18) at each copy event, and work is not extracted from the information generated. Its entropy generation is thus constrained by the informational bound: This bound implies an efficiency η ¼ ðkTIÞ=ðw chem = n copy Þ ≤ 1.
It is important that the readout molecule states are persistent-the ligand-binding state of the receptor is remembered even after detaching, thus enabling time integration [24,26]. Systems that do not make persistent copies, including passive readouts [26], the receptors themselves, the formation of templated copolymers in which the copy remains bound to the template [44][45][46], and also some more complicated sensing networks [28], are not bound by equivalent limits. Previous work has linked energy dissipation in similar systems to the erasure of the memory [24,25], that is the resetting of the memory bit to a well-defined state. As shown by Landauer, this erasure does lead to the transfer of heat from the system to the surroundings. Resetting a bit, however, is not intrinsically thermodynamically irreversible [2,3,5,13,47]-indeed, Landauer's calculation of the minimal heat transfer applies to a thermodynamically reversible erasure step. It is therefore the failure to extract work from correlations [5], rather than erasure itself, that is the origin of the thermodynamic irreversibility of a copy cycle. It is also this failure that sets the fundamental thermodynamic lower bound on the work to perform copy cycles. In fact, erasure is not even necessary in an optimal copy cycle, which leaves open the possibility that the biochemical network, which actually contains no explicit erasure, could achieve the lower bound of Eq. (19).

D. Trade-off between dissipation and error
The bound in Eq. (19) holds for all choices of s RL , s R , and p 0 . But how close does the biochemical network come to this bound, and can it be reached in certain conditions? Our concrete mapping allows us to examine these questions. Initially, we consider the simplest case of equal accuracy (s R ¼ s RL ¼ s). The dissipation per copy cycle for the biochemical network [Eq. (17)] then reduces to , half of the free energy released by the breakdown of ATP into ADP and P. The chemical work per copy cycle, w chem =n copy , is plotted against p 0 in Figs. 2(a) and 2(b) for two values of s (red dashed lines). We also indicate behavior forbidden by the thermodynamic bound of Eq. (19) (gray region). In Fig. 2(c), we fix p 0 ¼ 0.5 and plot the cost per copy cycle against s for the biochemical network (red dashed line) and an optimal system that saturates the bound (solid blue line). Both the cost of the biochemcial network and the thermodynamic bound drop as the required accuracy is decreased. The irreversible loss of information bounds the work, and lower accuracy implies less information is lost. It is clear from Fig. 2(c), however, that the two systems have a very different trade-off between dissipation and accuracy. The required free-energy input for the biochemical case diverges as s→1 [w chem =n copy ≈ −2p 0 ð1 − p 0 ÞkT lnð1 − sÞ], whereas the dissipated work remains finite in an optimal system. It is clear that a cell will have to sacrifice some accuracy for the sake of efficiency. Even in the limit of low Trade-off between dissipation and accuracy in copying. The (chemical) work per copy cycle for different probabilities p 0 of attempting to copy RL is plotted at two different values of the measurement accuracy: (a) s ¼ 0.80 and (b) s ¼ 0.99. Note that p 0 ¼ p, the probability that a receptor is in state RL, if k 3 þ k 4 ¼ k 5 þ k 6 when the sampling rate for the two receptor states is the same. The biochemical implementation (red line) does not achieve the lower bound for a measuring device, which is the border of the shaded region. The blue lines correspond to quasistatic protocols for the device in Fig. 1 of the Supplemental Material (see Discussion 3 [38]) and are optimal for a specific value of p 0 , p 0Ã , at the given values of s. (c) Dissipation per copy cycle at p 0 ¼ 0.5 as a function of s for the biochemical network (dashed red line) and a system that saturates the bound (solid blue line). The solid blue line (p 0Ã ¼ 0.5) saturates at kT ln 2 for perfect accuracy (s ¼ 1), and the cost of the canonical biochemical motif diverges. accuracy (s → 1 2 ) and at p 0 ¼ 0.5, the biochemical network is twice as costly as the bound (η → 1 2 from above). Expanding Eqs. (19) and (20) for s → 1 2 gives w chem =n copy ≈ 4kTðs − 1 2 Þ 2 for the biochemical network and w chem =n copy ≈ 2kTðs − 1 2 Þ 2 for the optimal device and protocol. The fundamental difference between the biochemical network and an optimal protocol is illustrated in Fig. 3. An optimal protocol requires reversible quasistatic manipulation of (free-)energy levels over time [2,3]. In the biochemical network, however, the overall differences in free energy between levels are fixed over time, and not slowly varied during a measurement. Instead, the receptors selectively catalyze specific reactions dependent on the receptor state. Because the free-energy levels, and hence the energetic drive for the copy process, are constant, the cellular copy protocol is energetically more costly than the thermodynamically optimal one. This difference is particularly intuitive in the limits of low and high copy accuracy for p 0 ¼ 0.5. For the biochemical network, the difference between the fraction of correct copies s and the fraction of incorrect copies 1 − s is 2ðs − 1 2 Þ. For p 0 ¼ 0.5, in only half of these cases does a reaction occur, and so the net number of reactions in the intended direction is s − 1 2 per copy cycle. Each of these net reactions has a cost given by the driving free energy, E s ≈ 4ðs − 1 2 ÞkT, giving an overall cost per copy cycle of w chem =n copy ≈ 4kTðs − 1 2 Þ 2 . For an optimal quasistatic protocol, a similar argument can be constructed (see Discussion 3 in Supplemental Material [38]), but the average energy offset between the states at which a transition occurs is less than the final offset required by the accuracy E s . In the limit of E s → 0, we obtain an average cost of E s =2, rather than E s as for the biochemical network.
In the limit of high accuracy, s → 1 and E s → ∞, a similar analysis shows why the biochemical network is much less efficient than an optimal quasistatic protocol. The cost of each transition continues to grow linearly with E s in the biochemical network, explaining the divergence of chemical work with s. In an optimal protocol, however, the work saturates when E s ≫ kT. In this case, the energy difference between the two states is raised quasistatically to E s . Therefore, at each moment in time, the bit is in equilibrium. As E is raised, the probability that the bit is in the high-energy state decreases, until it rapidly becomes negligible when E ≫ kT: from hereon, the higher-energy state can be raised further without any additional cost.
As is evident from Figs. 2(a) and 2(b), the cost of both an optimal system and the biochemical network decreases as we move away from p 0 ¼ 0.5 at fixed s. A fixed accuracy measurement results in less information if the data themselves have low entropy, explaining the reduction in the bound. For the biochemical network, if the readout is exposed more often to one receptor state, it is more likely to be in the appropriate output state prior to a copy. Hence, fewer transitions are needed and less dissipation occurs. For an alternative network in which receptors only function as kinases, this automatic compensation occurs only at low p (see Discussion 2 in Supplemental Material [38]).
The adaption to low-entropy data is so effective that η is actually highest for p 0 → 0 or 1 (when η → 1 2 from below for all s), and lowest at p 0 ¼ 1 2 (shown explicitly in Fig. 2 of the Supplemental Material [38]). This intrinsic adaption is fundamentally different from the behavior of typical copying architectures [2,3]. For these devices the protocol must be parametrically adjusted for optimality as p 0 is varied. Figures 2(a) and 2(b) show, with blue lines, the work per copy of three fixed protocols that are optimal for p 0Ã ¼ 0.3, 0.5, and 0.7, respectively; each line is tangent to the forbidden region at the specific value of p 0 ¼ p 0Ã , but above it elsewhere, as derived in Discussion 3 of Supplemental Material [38].
While the different "optimal protocols" corresponding to the blue dashed lines in Fig. 2 are each optimal for a given p 0Ã , they do not perform better than the biochemical network for all values of p 0 . Moreover, the details of a  Fig. 1). In the absence of an enzyme, all transitions are slow (indicated by the dashed free-energy barrier connecting the states). In the presence of R, half of the transitions are catalyzed and can occur rapidly (solid lines); when exposed to RL, the alternative transitions are rapid. The heights of the rungs, however, are fixed, meaning that all transitions involve a fixed amount of chemical work equal to the offset. By contrast, for the optimal protocol in (b), switching between states is driven by slowly destabilizing one state with respect to the other, so that the majority of transitions have already occurred before the offset approaches its limiting value.
cycle depend on p 0Ã , and so implementing an efficient protocol with p 0Ã ≈ p 0 would require an estimate of p 0 . In our cellular context, however, p 0 is precisely the quantity that the system is trying to measure (rather than the state of individual receptors given a known p 0 ). The best that could be done, therefore, would be to pick a particular protocol and update it as more information became available. Clearly, however, implementing this behavior in an autonomous device would require substantial additional complexity. We show that the biochemical network cannot reach the fundamental limit of efficiency for s R ¼ s RL , regardless of the system parameters. Neither reducing the reaction rates nor the thermodynamic driving permits η > 1=2. It is sometimes assumed that systems under steady-state forcing are effectively quasistatic or reversible in the limit of this forcing being weak [25]-clearly, that is not the case here. For s R ≠ s RL , it is possible to obtain an efficiency η > 1=2, and η can even approach unity for extreme values of p 0 , s R , and s RL (see Fig. 3 and Discussion 4 in the Supplemental Material [38]). Nonetheless, it remains true that η cannot converge on unity for general p 0 , regardless of how s R and s RL are varied. The biochemical network is more dissipative than an optimal process, because operating at a constant thermodynamic driving force leads to more energetically expensive transitions than slowly manipulating energy levels.
Thus far, we have emphasized differences between the thermodynamic bound and the cost of the biochemical protocol. It is remarkable, however, that the biochemical network comes so close to the lower bound, even at fairly high accuracies. Reaching 99% accuracy for p 0 ¼ 0.5 requires less than 4 times the dissipation of the lower bound, and η is even higher for p 0 ≠ 0.5. Further, this efficiency can be achieved at an arbitrarily high rate of copying-the absolute rates do not enter the expression in Eq. (20). Through our quantitative mapping, we show that a physically reasonable model system operating autonomously at an arbitrary rate and with a high copying accuracy comes close to the fundamental thermodynamic limit on the cost of a copy process.

E. One-step copy processes are maximally efficient for the biochemical network
The push-pull network considered so far is a one-step copy process, with the conversion between the phosphorylation states occurring via a single instantaneous transition. We now show that more complex processes, involving multiple steps or pathways, cannot improve the trade-off between dissipation and precision in autonomous, steady-state systems driven directly by an out-ofequilibrium chemical fuel. We again consider the Markov process with discrete states, but we now allow for multiple states and multiple parallel pathways between x and x Ã (see Fig. 4). We explicitly consider the interconversion of x and x Ã by RL; an equivalent argument holds for reactions mediated by R. The autonomous requirement prohibits external control and implies that transition rates are fixed over time.
In the simple one-step process, the accuracy of copying RL is s RL ¼ k 3 =ðk 3 þ k 4 Þ. In a more general process, however, transitions between x and x Ã are not instantaneous and, hence, cannot be described with rate constants. The natural generalization is to consider the flux ϕ from x to x Ã and vice versa, which is defined as the rate at which trajectories leave x and subsequently reach x Ã instead of returning to x [48]. The fluxes determine the copy accuracy in the same way as the rate constants for the one-step process, since they represent the rates that receptors initiate correct and incorrect transitions between x and x Ã .
Consider the accuracy of copying RL, s RL ¼ ϕ 3 = ðϕ 3 þ ϕ 4 Þ ¼ 1=ð1 þ ϕ 4 =ϕ 3 Þ, which depends solely on ϕ 3 =ϕ 4 . We define the set of paths S that start by leaving x and reach x Ã via an interaction with RL without returning to x. The entropy change in the environment ΔS env ½zðtÞ due to each of these individual trajectories zðtÞ is related to the probability of observing the forward pathway, and its reversezðtÞ [49][50][51]: Here, ΔS int is an intrinsic entropy difference between macrostates x and x Ã that arises because discrete biochemical states contain multiple microstates. It is a property of the states x and x Ã , rather than the transitions, and modifying it will not reduce overall dissipation in any steady-state network since the antagonistic transitions via R must have an equal and opposite ΔS int . We can then divide S into subsets S i according to ΔS env ½zðtÞ; all paths within S i generate the same entropy in the environment, ΔS i env . Conceptually, different subsets might correspond to pathways that consume different amounts of ATP. A simple one-step pathway, as considered hitherto, is a special case of a system with a single S i . The probability of observing any pathway within S i given an initial state z ¼ x is Similarly, PðS −i jx Ã Þ ¼ P zðtÞ∈S i p½zðtÞjzð0Þ ¼ x Ã is the probability of observing a reverse of a pathway in S i given an initial state x Ã . By design, ΔS i env þ ΔS int ¼ k ln½PðS i jxÞ=PðS −i jx Ã Þ. Further, the ratio of forwards to backwards fluxes is given by An immediate consequence is that in systems with only one subset of S in which all transitions have the same ΔS env ≡ ΔS tight env , the ratio of forwards to backwards fluxes is fixed by Such a process is "tightly coupled." In our framework, being tightly coupled implies that each transition x to x Ã is associated with the breakdown of the same number of ATP molecules. In this case, regardless of the number of intermediate steps or possible pathways in a tightly coupled process between two given states x and x Ã , the copy accuracy is unambiguously determined by the entropy change in the environment. Thus, all tightly coupled processes with the same accuracy are associated with the same dissipation. Since the one-step process we have considered hitherto is tightly coupled, all tightly coupled processes have the same accuracy-efficiency trade-off as we have derived. It is also possible to consider processes that are not tightly coupled, with multiple subsets S i . In this case, the average entropy generated in the environment per x → x Ã transition is given by where we use the "m" superscript to denote probabilities for this specific multisubset system. The accuracy of this process is still determined by Eq. (23): ϕ multi We can compare this multisubset process to a tightly coupled process with the same accuracy: ϕ tight 3 =ϕ tight The key point is that since entropy generation and accuracy are unambiguously related for a tightly coupled process [Eq. (24)], any tightly coupled process between x and x Ã with the same accuracy as the multi-subset generates Combining Eqs. (25) and (26) thus yields where the subscript N indicates normalization: P m N ðS i jxÞ ¼ P m ðS i jxÞ= P j P m ðS j jxÞ, and P m N ðS −i jx Ã Þ is defined equivalently. Equation (27) is a Kullback-Leibler divergence between the families of paths taken by forwards and backwards transitions in the multi-subset process, and is therefore necessarily non-negative. Thus, Similarly, paths that begin and end in x or x Ã generate no entropy for a tightly coupled process, whereas the entropy generation of these paths is non-negative in general. For a system operating in steady state, increased entropy deposited into the environment implies less efficiency. Therefore, no process of a given copy accuracy is more efficient than a tightly coupled one. The limits derived for a one-step process, a special case of a tightly coupled process, are therefore general. Our derivation is related to the proof that the estimate of dissipation obtained from the irreversibility of a coarsegrained trajectory gives a lower bound on the true entropy generation [52,53]. The results are fundamentally distinct, however. We could consider coarse graining a complex copy process so that all states were now x or x Ã . However, the result would not be a simple one-step process with the same accuracy and lower dissipation; the dynamics would be non-Markovian and reflect the underlying complex process. To state that a true one-step process could reproduce the accuracy at the same cost as estimated from the coarse-grained description would be to assume the conjecture that is to be proven.

F. Biochemical implementation of an optimal device and protocol
We argue that no biochemical copying network, operating autonomously and directly powered by a nonequilibrium fuel supply, can reach the thermodynamic bound on efficiency for general input data. We now consider whether this is a fundamental property of biochemical reactions, or whether biomolecules could, in principle, act as thermodynamically optimal bits.
There are two principal differences between cellular biochemical networks and optimal protocols [2,3,14]. Firstly, cellular networks operate continuously, rather than taking a series of discrete measurements with external clocking. Secondly, as emphasized above, they involve no manipulation of (free-)energy levels over time, as illustrated in Fig. 3.
Concerning the first difference, in Discussion 5 of the Supplemental Material we show that a clocked analogue of the cellular push-pull motif (illustrated in Fig. 4 therein) gives a work per copy identical to the continuous case [38]. This is because, despite being operated in a clocked fashion, the device is still out of equilibrium and functions at the constant chemical potential of fuel molecules. The fact that cellular networks operate in a stochastic continuous manner rather than a clocklike fashion is not the fundamental reason why they cannot reach the bound on the energy cost of a copy operation.
Functioning out of equilibrium is necessary for a device operating at constant chemical potential of fuel; if the reactions were in equilibrium, the receptor (which is a catalyst) could not influence the yield of x=x Ã . We now show that a system driven by quasistaic manipulation of ATP, ADP, and P concentrations could reach the thermodynamic bound, confirming that the autonomous network is inefficient due to its static free-energy levels. To operate in the quasistatic limit, RL and R must be long-lived; in practice, constitutively active kinases and phosphatases could be used.
We consider a device and measurement cycle as illustrated schematically in Fig. 5. The key ingredients are the possibility of manipulating the concentrations of ATP, ADP, and P in the vicinity of the readout, and the ability to bring the readout into or out of close proximity with receptors. We consider the same receptor or readout reactions as in Eq. (1), and now define the free-energy changes of reaction ΔG p for the phosphorylation of x by RL and ΔG d for the dephosphorylation of x Ã by R. These quantities are given by in which ΔG x=x Ã quantifies the intrinsic stability of x and x Ã (assumed to be independent of the chemical potentials of ATP, ADP, and P [50]). The reactions can thus be manipulated by controlling μ ATP , μ ADP , and μ P , for example, by coupling of the system to a series of reservoirs [ Fig. 5(b)]. We outline an optimal cycle below, and calculate the chemical work done on the readout subsystem by the reservoirs (the average number of reactions of a given type, multiplied by the associated chemical work, integrated over the whole process) in Discussion 6 of the Supplemental Material [38]. Throughout, we assume that receptor states are stable, and that reactions cannot occur without a catalyst. The eight steps of the cycle are intended to be closely analogous to typical computational protocols [2,3,14]; for subtleties involved in this comparison, see Discussion 6 in Supplemental Material [38]. The system contains a readout (the memory bit); a receptor of known state R (a reference bit); and a receptor in either R or RL (the data bit). The readout begins coupled to a buffer with ΔG p , ΔG d ¼ −ΔG r (ΔG r is assumed to be large and positive). There is no receptor in close proximity, but the readout has been reset (equilibrated at the end of a previous measurement cycle by a receptor in the R state), and, therefore, is in state x with probability 1=½1 þ expð−ΔG r =kTÞ. In fact, the accuracy of this reset does not influence the cost of the cycle (see Discussion 6 in the Supplemental Material [38]).
(1) The readout is brought into close proximity with a receptor of known state R.
(2) ΔG p , ΔG d are slowly (quasistatically) raised from −ΔG r to 0. Steps 1 and 2 allow the state of the memory to be reversibly uncorrelated from the reference receptor, prior to the measurement.
(3) The readout is brought into close proximity with a receptor of unknown state (ligand-bound with probability p 0 ). ΔG p , ΔG d are then slowly lowered to −ΔG s . In this step, the state of the readout is set to match that of the receptor with accuracy s ¼ 1=½1 þ expð−ΔG s =kTÞ.
(4) The readout is moved away from the receptor. receptor is in state R, the readout is in x with probability s ¼ 1=½1 þ expð−ΔG s =kTÞ. (6) We now decorrelate the memory and data bits at a nonzero bias between the two readout states, ΔG p ; ΔG d ¼ ΔG off . To reach the fundamental thermodynamic bound, ΔG off must be chosen carefully, and depends on p 0 and s. To decorrelate, the readout is brought into close proximity with a known receptor of state R. The readout relaxes to a state reflective of ΔG off via the reaction R þ x Ã ⇌ R þ x þ P. (7) The readout molecule is reset by quasistatically lowering ΔG p , ΔG d to −ΔG r from ΔG off , returning it to a state dominated by x.
(8) The readout is separated from the reference bit, returning the system to the initial state.
The readout and receptor are in the same state at the start and finish of the cycle; thus, the net chemical work of the reservoirs equates to the free energy dissipated by the entire system. As shown in Discussion 6 of the Supplemental Material, minimizing dissipation with respect to ΔG off at fixed p 0 and accuracy s (fixed by ΔG s ) gives a chemical work per copy equal to the mutual information generated by a measurement [38]. In the special case p 0 ¼ 1=2, ΔG s → ∞, w chem → kT ln 2, as expected. A single readout could also be made to copy multiple receptors sequentially -using, for example, a series of receptors anchored to a polymer, as in Fig. 4 of the Supplemental Material [38]. We note that it is also possible to construct an optimal cycle if the receptor is only catalytically active in the RL state; see

IV. DISCUSSION
We describe a canonical cellular readout network rigorously in terms of computational copy operations, thus demonstrating that the system is indeed bound by the thermodynamics of computation. For a general distribution of input data, the cellular network cannot reach this fundamental limit of efficiency. Unlike optimal computational protocols, the thermodynamic driving force used to push the memory device between its states is not introduced quasistatically. Instead, a continuously operating autonomous network must have a constant thermodynamic discrimination between correct and incorrect copy outcomes over time. Even in the limit where the driving force and, hence, the accuracy become vanishingly small, the cellular system is not thermodynamically optimal. In the regime of high accuracy, the difference is larger: while optimal protocols can reach 100% accuracy for a finite cost of kT lnð2Þ, cellular networks can only achieve 100% accuracy for a cost that diverges. Nonetheless, achieving 99% accuracy for an unbiased distribution of input data requires less than 4kT ln 2 of dissipation per copy cycle, and the relative performance of the biochemical network for biased input data can be even better. Although kT sets an energy scale, it is not obvious a priori that the numerical factors should be so low. For example, the recently derived "thermodynamic uncertainty relation" [54]-in which the cost of achieving a relative uncertainty ϵ in the number of steps of a biomolecular process was shown to be at least 2kT=ϵ 2 -gives 20000kT for 99% accuracy.
Not only can this canonical cellular signaling system get remarkably close to the fundamental bound for efficiency of copying at relatively high accuracy, it can do so at an arbitrarily high absolute copy rate. Further, the system is autonomous, and so there is no need to consider the intrinsic costs of applying a time-varying yet stable control to a bit, as must be done in typical protocols [55,56]. The canonical biochemical network also naturally adapts to high or low levels of ligand-bound receptors, reducing its dissipation per copy cycle in a way that standard quasistatic protocols cannot achieve without feedback. The remarkable possibilities of this biochemical network not only show that the thermodynamic limits of computation are genuinely relevant to practical systems, but emphasize that biological systems are an excellent environment to rigorously investigate these limits in a concrete, autonomous setting.
Our mapping emphasizes the cause of the minimal thermodynamic dissipation for the readout network. Dissipation does not occur because the memory is "erased" [25]-erasure itself is not intrinsically irreversible, and no distinct erasure step is present. Rather, the stable correlations generated between noninteracting systems by copying are not used to extract work [5]. Other biochemical processes, such as ubiquitination, transcription, translation, and replication, result in correlations between degrees of freedom that are not maintained by direct interactions. Indeed, although we are motivated by the time integration of receptors by readouts, our analysis directly applies to other push-pull networks. Our work suggests that the thermodynamics of these persistent correlations is a central paradigm through which to understand this class of systems.
Our analysis reveals that biomolecules can, in principle, be used to implement protocols that achieve the thermodynamic bound, and we provide an example. The key difference from the canonical cellular network is the manipulation of concentrations of ATP, ADP, and P over time, during the course of the measurement. If these manipulations are performed slowly enough, reactions (except decorrelation of readout and receptor) can be performed reversibly as reactants or products are gradually stabilized or destabilized with respect to each other. We thus propose a new class of systems in which the fundamental thermodynamics of computation can be explored, to complement experiments done with optical or electrostatic feedback traps [7,8] and magnetic systems [9]. Our approach involves manipulating biomolecules by adjusting the chemical potential of fuel molecules. Our experimental system is particularly promising because the dissipation could, in principle, be measured directly for a large number of devices acting in parallel rather than inferred from positional trajectories, as it is done for the optical or feedback traps. A second advantage of the proposed setup is that the store of free energy used to perform work on the system-the chemical fuel-is explicit. It is clear exactly how free energy is transferred to and absorbed back from the bit under study. In other analyses, the explicit mechanism by which work is transferred between a bit and a store of free energy is implicit. It is usually assumed that the store of free energy can supply and absorb work efficiently, even if the operations on the bit itself are irreversible. In practice, however, work is typically supplied in a highly irreversible fashion such as via lasers [7,8], and any work done by the bit is lost rather than stored.
In an experimental realization, it would be natural to treat the reservoirs and memory together as an extended system thermally coupled to the outside world. In this case, the chemical free energy dissipated during measurement is not equal to the heat exchanged between the extended system and the outside world [50]-the increased entropy of the universe is instead manifest in a less uneven distribution of ATP, ADP, and P between reservoirs. It would, therefore, be most natural to measure dissipation through the changing concentrations of ATP, ADP, and P as the reservoirs exchange molecules-perhaps through radioactive labeling of phosphates. Further, it should be possible to perform full measurement cycles and probe the link between information loss and irreversibility. By measuring the state of the readout using, e.g., F̈orster resonance energy transfer (FRET), it would also be possible to test the generalized Jarzinksy equality, which shows that the state of the system can be changed more efficiently by exploiting knowledge of the state of the system [57][58][59].
The fact that cells employ thermodynamically inefficient out-of-equilibrium circuits, despite energy budgets being an important consideration in evolutionary fitness, highlights the constraints under which they function. Cells do not have infinite time to perform a measurement, meaning that quasistatic manipulations are infeasible. Further, our quasistatic protocol requires that a readout molecule exclusively encounters either ligand-bound or ligand-free receptors during each copy process. This requires that the ligand and ligand-free states of the receptors are stable on the time scale of the measurement cycle, and also that receptors in both states are not accessible to a single readout. In reality, the finite lifetime of receptor states places limits on the measurement time, and cells typically have multiple receptor molecules with which any one readout molecule can interact. The quasistatic cycle also necessitates coordinating the separation between readouts and receptors; although this is not inconceivable within a cell, it would require elaborate machinery. Perhaps most importantly, however, the quasistatic protocol requires manipulation of the concentration of chemical fuels; this manipulation must change chemical potentials by several kT to be effective. Given the relatively small number of chemical fuels available, and their extensive use in a range of systems, it would be very surprising if the cell manipulated chemical potentials purely for the sake of measurement efficiency.
The final observation may also explain why cells use such a strong chemical driving (the hydrolysis of ATP typically provides approximately 20kT, deep into the lowerror regime), rather than more efficiently taking measurements of only slightly lower accuracy [24]. Indeed, the most efficient strategy from the perspective of sampling is to make many low-accuracy copies [24]; this, however, requires time (for the measurements to be independent) and readout molecules (to store the measurements), resources that are not free for the cell. The design of more efficient copying architectures may be relevant in synthetic biology and biological engineering in which the constraints and goals are distinct from those of natural systems.
In this work, we focus on a single-step copy process. However, we also argue that more complex copy processes, including multiple bound receptor-readout states or even multiple pathways in which different amounts of ATP can be consumed, cannot be equally accurate at a lower cost. Indeed, we show that all "tightly coupled" processes in which all transitions consume the same amount of chemical fuel are equally efficient, and all others cannot perform better. At a physical level, it is intuitive that complex process can be less efficient-they naturally allow for dissipative cycles. Similarly, longer pathways are not helpful, because one cannot make an irreversible process less irreversible by breaking it into many small steps while keeping the overall driving force fixed. For processes such as kinetic proofreading, however, in which chemical fuel drives reactions out of equilibrium, complex or longer reaction pathways can allow the same outcome at a lower cost [31,36,60,61]. The crucial difference is that in a copy process the metric of accuracy is exactly the ratio of forwards to backwards transition probabilities, which is the quantity directly influenced by fuel consumption. More fuel consumption always improves the metric. For other tasks, fuel consumption is necessary, but other factors can also influence performance. In kinetic proofreading, the relevant metric is the relative occupancy of a binding site by two different ligands [36,61]. This quantity is influenced by the strength of chemical driving, but it is also limited by the number of intermediate states at which there is an opportunity to discriminate between ligands. Thus, it can be beneficial to consider multistage proofreading [62].
Our analysis of the cellular network uses the mean-field limit, which becomes accurate when the receptor correlation time τ c is shorter than the relaxation time τ r of the readout network. Interestingly, this is precisely the optimal regime for sensing [24], because the system can take multiple τ r =τ c > 1 concentration measurements per receptor molecule. In this regime, the readout molecules do not track the fluctuations in the receptor state, but, collectively, average it. As a result, the "learning rate" [40] between the readout and the receptor is actually zero in this limit (see Discussion 1 in the Supplemental Material [38]). While the opposite regime τ r =τ c < 1 is detrimental for the mechanism of time integration, we do note that the work per measurement is less. This is because in this regime the measurements become correlated, and taking correlated measurements requires less work for a given desired accuracy. A full analysis of this regime is the subject of further work. Similarly, we have not considered the consequences of spontaneous reactions not mediated by kinases and phosphatases. In the context of copying, these reactions equate to spontaneous thermalization of bits, which could be incorporated into our mapping.