Discriminatory proofreading regimes in non-equilibrium systems

We use ideas from kinetic proofreading, an error-correcting mechanism in biology, to identify new kinetic regimes in non-equilibrium systems. These regimes are defined by the sensitivity of the occupancy of a state of the system to a change in its energy. In biological contexts, higher sensitivity corresponds to stronger discrimination between molecular substrates with different energetics competing in the same reaction. We study this discriminatory ability in systems with discrete states that are connected by a general network of transitions. We find multiple regimes of different discriminatory ability when the energy of a given state of the network is varied. Interestingly, the occupancy of the state can even increase with its energy, corresponding to an"anti-proofreading"regime. The number and properties of such discriminatory regimes are limited by the topology of the network. Finally, we find that discriminatory regimes can be changed without modifying any"hard-wired"structural aspects of the system but rather by simply changing external chemical potentials.

Non-equilibrium systems often show varied behavior that depends on many parameters and details of the system. While it may be easy to investigate any particular point in the parameter space of a given system, it is harder to identify qualitative regimes relevant across systems. For example, the occupancy of states at thermal equilibrium is simply given by the Boltzman distribution ψ eq (E) ∼ e − E k B T which depends only on the energy E of a state and the temperature T . When driven out of equilibrium, the occupancy of states could in principle depend on all the details of the space of states of the system and the network of paths connecting them. There has been much effort invested [1][2][3][4] in understanding non-equilibrium steady states (their occupancy of states, fluxes and entropy production) in order to model the many systems in physics [5][6][7][8][9][10] and biology [11][12][13][14][15][16] that operate out of equilibrium.
In this paper, we identify steady-state kinetic regimes that can be exhibited by general systems that are driven out of equilibrium. These regimes are characterized by the change in the occupancy of a state in response to a change in its energy, with all driving forces held fixed.
We draw on intuition from generalized versions of a biological error correcting mechanism called kinetic proofreading. In biochemical contexts, two molecular substrates, one of them undesirable, might compete to participate in the same enzymatic reaction with different binding energies. Hopfield and Ninio [17,18] proposed a non-equilibrium "kinetic proofreading" mechanism which enhances the effect of the binding energy difference on reaction rates. As a result, reactions with the weakly binding substrate are suppressed to a much larger extent than one would expect at equilibrium. Kinetic proofreading has been evoked to explain very low error rates in a host of biochemical reactions despite the similarity of competing substrates; in DNA replication despite the similarity of the four nucleic acids, in protein syn-thesis despite the similarity of t-RNA molecules [17][18][19][20], in immune response that discriminates between native and foreign proteins [21] and in many other biological phenomena [13,15,22,23]. In all of these cases, proofreading enhances discrimination between competing substrates by making the occupancy of a biochemical state more sensitive to changes in its energy.
The central proposition of our paper is that nonequilibrium systems like proofreading mechanisms can be designed to have co-existing regimes of varying discrimination. In the context of proofreading, some undesirable substrates can be highly discriminated against while other substrates of even weaker binding energy can actually be promoted. The number and properties of such co-existing discriminatory regimes are constrained by the topology of the network of paths connecting the system's states. Within these constraints, we can design variable discrimination through an appropriate choice of kinetic parameters that might be "hardwired", for example, in the structural and dynamical properties of molecules.
On the other hand, non-equilibrium steady states are always powered by external sources of free energy, such as ATP hydrolysis in the case of proofreading. We find that merely changing the chemical potential of these external sources can change the discrimination characteristics. The ability to tune such characteristics on the fly, without having to change any hardwired properties, raises interesting ways in which natural and synthetic systems can respond to the environment.

I. ONE-LOOP NETWORK
We can illustrate many of our results using the simple 3-state system shown in Fig 1A, with stochastic transitions governed by kinetic rates k ij . We will then generalize these results to non-equilibrium networks with many more states and pathways (Fig. 2). The model shown in Fig. 1A was introduced by Hopfield and Ninio [17,18] as a non-equilibrium solution to the following discrimination problem in biochemistry: Enzyme E reacts with a substrate R, forms complexes ER, ER * , and thus processes R into a biologically active product (R-Product). However, a structurally similar competing substrate W might also be able to undergo the same set of reactions with E, with less favorable energetics, releasing an undesirable product W -Product.
The steady-state occupancies ψ ER , ψ ER * , ψ E are given by solving the master equation, where w R ij are the kinetic constants k ij shown on the network to the left in Fig 1A. Occupancies ψ EW , ψ EW * , ψ E of the W -network are given by solving the same equation with modified kinetic constants w W ij shown to the right in Fig 1A. The kinetics of W and R differ because the binding energy of EW is lower than ER by ∆; hence the off-rates w W 21 , w W 31 are higher than the corresponding rates for R by a factor of e ∆ . We do not consider the case of discrimination due to variable activation barriers studied in [24,25].
The error rate is the ratio of W -Product to R-Product formation, given (in the limit of slow formation rate f from the complexes EW , ER) by [17], If the system is at equilibrium and not driven, the rates w R ij , w W ij will both satisfy detailed balance: we can write where S = R or W and V S i is the energy of state i. (We have set β = 1/k B T = 1.) It is easy to see that η(∆) = e −∆ , the Boltzmann distribution.
Hopfield and Ninio proposed that discrimination between R and W can be much higher if the system is driven out of equilibrium by coupling some of the reactions to ATP hydrolysis. The rates w S ij , which take ATP coupling into account, will no longer obey detailed balance but can written as, Here S = R or W . µ ij is the external driving potential on the i ↔ j reaction due to ATP coupling and is conservatively assumed to not distinguish between W and R. Hopfield and Ninio identified a particular limit of kinetics w R ij , w W ij in which such driving would result in an error rate η ∼ e −2∆ that is typically significantly lower than the equilibrium error rate e −∆ . This mechanism has been invoked to explain low error rates in biochemical processes like protein synthesis [26].
Physically, the error rate η(∆) is a measure of how much the occupancy ψ ER of state ER changes when the The kinetic proofreading scheme of Hopfield and Ninio [17,18] shows variable discrimination between two substrates W and R that compete to participate in the same reactions with an enzyme E but with binding energies that differ by ∆. (B) The ratio η(∆) of W -Product to R-Product, with kinetic parameters in the limit proposed by [17,18], is shown in blue. While a substrate W is suppressed to the extent of e −2∆ for ∆ < 8kBT , proofreading is less effective for larger ∆. The orange curve in (B) corresponds an alternative limit of kinetics, distinct from that in [17,18]. Instead of correcting errors, the non-equilibrium network promotes the formation of the less stable W -Product for ∆ < 6kBT , with η(∆) ∼ 1. (C) The discriminatory index ν (i.e., slope of log(η)) shows distinctive discriminatory regimes for both limits of kinetics. The area between the graph of ν and the equilibrium value νeq = 1 is equal to the chemical potential µ used to drive the network. (The area of the orange curve is interpreted as a negative number.) energy of ER is raised by ∆, with external driving forces µ ij held fixed. A larger change in occupancy implies higher discriminatory power between substrates in the biochemical reaction. Hence we will find it useful to define a local measure of this discriminatory ability, the discriminatory index, At equilibrium, ν(∆) = 1 while ν(∆) = 2 in the kinetic limit uncovered by Hopfield and Ninio.
Discriminatory regimes The central observation of our paper is that η and ν are not constant but rather functions of ∆. The full plot of log η(∆) with kinetic parameters in Hopfield's [17] proposed limit is shown in Fig. 1B (blue curve). The classical result of η ∼ e −2∆ only holds for ∆ < 8k B T (in this example) after which log η(∆) quickly transitions to a different regime with slope −1 (the equilibrium value). Proofreading is less effective for substrates with ∆ in this regime. We say that the network exhibits two co-existing "discriminatory regimes", i.e, two distinct regions of ∆ characterized by ν ≈ 2 and ν ≈ 1 respectively.
Malleability of regimes: If we choose an alternative set of kinetic constants k ij , distinct from the limit in [17,27], we find a qualitatively different set of regimes with ν ≈ 0 and ν ≈ 1, shown in Fig 1B,C (orange). In this sense, discriminatory regimes are malleable and can be changed through an appropriate choice of kinetic constants. (See SI for the values of k ij for both curves and for a 2-loop network example.) Anti-proofreading: The kinetic limit corresponding to ν < 1 (Fig. 1C orange) uses non-equilibrium effects to actually lower the discrimination between W and R; the weakly binding substrate W with ∆ < 6k B T forms as much product as strongly binding R since ν ≈ 0 and η(∆) ∼ 1. This is an example of an "anti-proofreading" regime, in which higher energy states can have higher occupancy.
Size of regimes and chemical potentials: The size of the discriminatory regimes shown in Fig. 1C can be quantified by the area between the graph of ν(∆) and the equilibrium value ν eq = 1. This area is in fact equal to the net chemical potential around the loop µ = µ 12 +µ 23 +µ 31 that is used to drive the system. Thus higher chemical potentials are needed to enhance discrimination across a wider range of energies.
External shaping of regimes: The relationship between chemical potential and the shape of η(∆) and ν(∆) implies that we can modify discriminatory regimes without having to change any kinetic constants "hard-wired" in the structure of the enzyme. For example, we can switch from the kinetics corresponding to the blue curve in Fig. 1C to the orange curve's by flipping the sign of chemical potentials µ ij -i.e., by driving the system in the reverse direction around the reaction loop.

II. GENERAL NETWORKS
We will now show how the kinetics of a general driven network can give rise to the variable discriminatory regimes discussed above. We will find that topology of the network limits the number, size and index ν of discriminatory regimes that can be achieved.
A general driven network with multiple interconnected pathways is shown in Fig. 2. We have only shown the kinetics for substrate W , whose binding energy to E is lower than that for R by ∆. This binding energy difference can be reflected in the kinetics in many different ways. The only requirement is that the equilibrium constant computed along any path from start (E + S) to finish (ES) for S = R and S = W must differ by a factor of e ∆ . For now, we assume that the distinction in kinetics is localized in one reaction along each pathway as shown in Fig. 2. We connect such reactions by an imaginary line that we call the "discriminatory fence" (dotted line in Fig. 2), which divides the network in two. Reactions on either side of the fence do not discriminate between the two substrates; the binding energy difference ∆ is entirely accounted for by reactions that cross the discriminatory fence. (The fence is a surface for non-planar networks. See SI for a discussion.) Discrimination in several biological proofreading mechanisms does appear to be localized in select reactions [26,28,29]. We relax this simplifying assumption later and find that a de-localized fence lowers the discriminatory ability of the network.

A. Limit of highest discrimination
To gain intuition about general networks, we begin with the kinetic limit that leads to the highest possible discriminatory index ν > 0 (strongest proofreading) in any network. This kinetic limit generalizes several particular models of biological error correction [15,[28][29][30][31] and is defined by three properties: (i) The kinetics pick out a dominant path, shown in green in Fig 3A, from state E + S to the final state ES; the system can reach ES only by traversing this path.
(ii) The dominant path must parallel the discriminatory fence on the products (i.e., ES) side of the network before reaching ES. As a result, as the system travels down the dominant path towards ES, it is constantly exposed to pathways (shown in red) that cross the fence back to the reactants (E + S) side.

E+S ES
Limit of highest discrimination (proofreading)

Limit of highest negative discrimination (anti-proofreading)
FIG. 3. General limit of kinetics for highest (A) discrimination ν > 0 and (B) negative discrimination ν < 0 (antiproofreading). In both cases, the system can reach ES only by traveling down a dominant path (green) that parallels the discriminatory fence (dotted line). In this way, the system is exposed to multiple discard pathways (shown in red) that cross the discriminatory fence (dotted line) and abort the reaction by taking the system back to the initial state E + S through the bold black path (here, S = R or W ). (A) As shown in the inset, the probability p S f of not taking a red discard path across the fence is lower for S = W than S = R by a factor δi(∆). With c such discard paths along the dominant path, W is exponentially less likely to reach ES than R, resulting in η(∆) ∼ e −c∆ . In (B), the dominant path parallels the fence on the reactants E + S side of the network. The system can now take the (red) discard paths from the reactants side to the product side. Unlike in (A), we allow for the system to possibly bounce back to the dominant path after a discard. As shown in the SI, W is more likely to bounce back and proceed further along the dominant (green) path by a factor δi(∆) > 1. Hence the weakly binding S = W substrate is exponentially more likely to reach ES.
(iii) The kinetics need to ensure that if the system crosses the fence to the reactants side along a (red) discard pathway, the system moves back towards E + S along a path like that shown in bold. Hence we refer to the (red) pathways as discard pathways since they stop progress along the dominant path.
Hence a typical trajectory in Fig. 3A involves the system attempting to traverse the green path to the final state ES but being frequently discarded through the red fence-crossing reactions. The kinetics described thus far apply to both S = R and S = W substrates. The only distinction is that W is more likely take each discard pathway across the discriminatory fence. To see this, note that the probability of being discarded is determined by the kinetic constants d i for the discard pathway and f i for the forward direction on the dominant pathway (see inset in Fig. 3A). Since d i is higher for W than for R by a factor e ∆ , W is less likely to proceed forward along the dominant path by a factor δ i shown in Fig. 3A.
In the limit of high discards d i f i , the ratio of the forward probabilities δ i is e −∆ . If there are c such discard paths branching off the dominant path, the net probability of reaching ES is lower for W than for R by a factor η(∆) ∼ e −c∆ , giving the highest possible discriminatory index ν = c.
We note that much time is wasted in this limit since both R and W are frequently discarded along the dominant path and reaction completion is an exponentially unlikely event. Energy is also wasted since the system preferentially executes counterclockwise trajectories in the network shown in Fig. 3; such preferential cycling breaks detailed balance and consumes free energy from an external source (such as ATP hydrolysis) during each cycle. An alternative kinetic regime which trades a small increase in error for a large saving in time and energy was found in [32].
Anti-Proofreading: If the dominant path parallels the discriminatory fence on the reactants side (E + S) side instead (Fig. 3B), we find a counterintuitive "antiproofreading" regime with ν < 0 and η ∼ e +(c−2)∆ . The discard pathways now cross the fence from the reactants to product side. Unlike in the proofreading case, we assume that there is a high chance of reversing the discard reaction immediately. As a result, the system bounces back and forth repeatedly along a discard path before either proceeding forward along the dominant path or going back to the initial state along the bold black path in Fig. 3B. As shown in the SI, substrate W is more likely to bounce back across the fence and resume progress along the dominant path by a factor δ i (∆) > 1. Hence the weakly-binding substrate W reaching the final state EW more often than R reaches ER. In other words, the occupancy of the state ES increases as its energy is raised.

B. Sculpting multiple discriminatory regimes
The kinetic pathways for highest positive and negative discrimination suggest how variable discriminatory regimes arise for a general network. As shown in the Fig. 3 inset for the case of positive discrimination, each δ i (∆) = (d i + f i )/(d i e ∆ + f i ) provides varying discrimination that depends on ∆ and the discard d i and forward f i kinetics at that point. In the limits described in Fig. 3, the occupancy ratio η(∆) is (approximately) a product of δ i along the dominant path; η(∆) ∼ δ 1 (∆)δ 2 (∆) . . . δ c (∆).
As a result, we can design a large network to show mul-  Fig. 3. We use a ladder-like network shown in (B) and a dominant path that switches midway from paralleling the fence on the reactants side (like in Fig. 3B) to the products side (like in Fig. 3A). We position the regimes as shown (blue and orange plots) through two different choices of kinetic constants di, fi, ri, bi. These constants determine δi(∆) (see Fig. 3 and Eq. 5) along the dominant path. (See SI for numerical values of di, fi and more on δi(∆). We added a constant to the orange curve in (A) to fit both curves in the same plot.) tiple discriminatory regimes like that shown in Fig. 4A through a choice of dominant path and a choice of δ i s along the path. In Fig. 4B, the dominant path switches midway from paralleling the discriminatory fence on the reactants side (like in Fig. 3B) to the products side (like in Fig. 3A), allowing a more general product of δ i . (See SI for details on δ i on anti-proofreading dominant paths.) More generally, η is a ratio of polynomials p(e ∆ )/q(e ∆ ) in e ∆ . A discriminatory regime with index ν = j − i corresponds to a finite range of ∆ over which η(∆) can be approximated by a i e i∆ /b j e j∆ where a i e i∆ and b j e j∆ are particular monomials in p and q respectively. (See SI for a rigorous derivation of these results using Schnakenberg's network theory [5].)

C. Network topology constrains regimes
With η(e ∆ ) = p(e ∆ )/q(e ∆ ), we find that the degrees of the polynomials p, q are bound by c, the number of intersections between the discriminatory fence and the network. As a result, topological properties of the proofreading reaction network constrain the shape of η(∆) and index ν(∆): (i) The largest positive and negative values of ν are limited by c: −(c − 2) < ν < c.
(ii) The number of distinct discriminatory regimesdetermined by the number of monomials in p, q -is also bounded by c.
(iii) The extremal values attained by ν over all ∆ are related by ν max − ν min ≤ c.
(iv) The intersection number c cannot exceed the number of linearly independent pathways. Equivalently, in terms of the number of linearly independent loops n in the network, c ≤ n + 1.
(v) This sets an absolute limit on the discrimination between two substrates whose binding energies differ by ∆ by a proofreading network with n loops: If the discriminatory fence intersects only c < n + 1 reactions, then e (c−2)∆ ≥ ψ(∆) ≥ e −c∆ . For the network shown in Fig 2, c = 7 while n = 18.
(vi) We find that the minimal chemical potentials required for a discriminatory index ν(∆) are given by the areas of the n discriminatory regimes in the graph of ν − 1. When ν < 1, the area and chemical potential required are negative, indicating that the corresponding loop must be driven in the opposite direction. We leave a study of the precise mapping between the n linearly independent chemical potentials µ l in a network with n loops and the n resulting regimes to future work.

D. Localized discrimination fence and von
Neumann's error correcting scheme We have assumed that the binding energy difference ∆ is localized in a set of reactions, one along each pathway from E + S to ES, that define the sharp discriminatory fence. Consider the alternatives shown in Fig 5 where the energy ∆ is (B) spread over multiple reactions along each pathway or (C) where the energy difference ∆ is localized in a reaction common to both pathways. Proofreading can only combine independent discriminatory power in parallel pathways that form loops. Hence while the network in Fig. 5(A) achieves discrimination of e −2∆ , (B) and (C) can only achieve lower discrimination of e −(1−α)∆ × e −2α∆ = e −(1+α)∆ and e −∆ respectively. (In (B), we assume 0 ≤ α ≤ 1 so that no individual reaction has discriminatory power greater than e −∆ .) We can think of proofreading as a biochemical implementation of von Neumann's [33] reliable machine made of redundant unreliable components. von Neumann constructed logical machines that failed only when all of the individual error-prone components failed. Thus the machine itself has a lower error rate than the individual components, provided the components have independent probabilities of failure. In proofreading, each pathway can complete the entire reaction in isolation with an error rate of e −∆ but the network as a whole can have a lower error rate η(∆) e −∆ . In this context, the two pathways of Fig. 5(A) are like von Neumann components with fully independent error rates while the error rates of the pathways in (B) and (C) are not independent and cannot be combined as effectively. 5. (A) Discrimination is highest when the energy difference ∆ is localized to a discriminatory fence that cuts through as many reactions as possible. In (B) and (C), a fraction α or all of energy difference ∆ is in reactions common to both pathways (0 ≤ α ≤ 1). Proofreading acts by multiplying the effect of discrimination in parallel and independent reactions but cannot enhance discriminatory reactions common to both pathways.

III. DISCUSSION
By generalizing kinetic proofreading, we have uncovered new kinetic regimes in systems driven out of equilibrium, characterized by how much the occupancy of a state changes due to a change in its energy. The occupancy can fall with energy much faster than at equilibrium (conventional proofreading) but we also found anti-proofreading regimes where the occupancy of a state increases with increasing energy. We found bounds on the number, size and discriminatory ability of regimes in terms of topological properties of the network of transitions between states.
We were able to identify kinetic limits associated with discriminatory regimes in terms of a dominant path that parallels the discriminatory fence (the collection of reactions whose kinetics account for the change in energy of a state). We need such an understanding of kinetics that is not tied to a particular network in order to be able to incorporate non-equilibrium driving into synthetic systems to achieve particular goals. Such a general picture is also useful in studying proofreading properties of biochemical circuits, which are often large, complex and designed to perform several functions besides proofreading.
Biologically, having proofreading ability vary with binding energy can be useful when a biochemical reaction is presented with a family of competing substrates. The same proofreading scheme might be able to enhance discrimination between two given substrates while in fact suppressing differences between another pair, depending on their binding energies.
We emphasize that our results apply to networks that represent any driven system, including systems unrelated to biological kinetic proofreading. For example, the states of the network could represent different stages of assembly of microtubules [32] or synthetic self-assembled structures [34,35]. Our results then might point the way to incorporating driving forces into the assembly pathways in synthetic systems, reducing errors in the final structure.
An intriguing feature of the schemes discussed here is the possibility of changing the discriminatory regimes by changing only the chemical potentials µ i , without having to change any hard-wired structural kinetics. Such an ability to change discriminatory properties on the fly could be useful in both natural and synthetic systems; the network can switch between suppressing the reaction with a substrate and promoting it, depending on the environment or other considerations.