Statistical Physics of Adaptation

All living things exhibit adaptations that enable them to survive and reproduce in the natural environment that they inhabit. From a biological standpoint, it has long been understood that adaptation comes from natural selection, whereby maladapted individuals do not pass their traits effectively to future generations. However, we may also consider the phenomenon of adaptation from the standpoint of physics, and ask whether it is possible to delineate what the difference is in terms of physical properties between something that is well-adapted to its surrounding environment, and something that is not. In this work, we undertake to address this question from a theoretical standpoint. Building on past fundamental results in far-from-equilibrium statistical mechanics, we demonstrate a generalization of the Helmholtz free energy for the finite-time stochastic evolution of driven Newtonian matter. By analyzing this expression term by term, we are able to argue for a general tendency in driven many-particle systems towards self-organization into states formed through exceptionally reliable absorption and dissipation of work energy from the surrounding environment. Subsequently, we illustrate the mechanism of this general tendency towards physical adaptation by analyzing the process of random hopping in driven energy landscapes.


I. INTRODUCTION
It has long been understood that nonequilibrium driving can bring about novel patterns of complex organization in thermally fluctuating many-body systems. Whether in fluid dynamics [1,2] or so-called "active matter" mixtures [3,4], numerous examples have been characterized where qualitatively new types of collective behavior become possible because a large set of interacting degrees of freedom are allowed to act together as a conduit for chemical and heat dissipation in surrounding reservoirs.
Many have hoped that statistical thermodynamics might provide us with general predictive principles for the behavior of such systems, much in the same way as in the equilibrium case where the Boltzmann distribution applies. However, historically, efforts along these lines have encountered serious difficulty outside of the nearequilibrium, linear-response regime [5] because the number of time, length, and energy scales that are important in determining the dynamics can multiply confoundingly once external drives become arbitrarily strong.
Nonetheless, there is renewed reason to anticipate that a thermodynamic universality might yet be recognized in strongly driven systems that exhibit complex collective behavior. In the last two decades, many fundamental results in nonequilibrium statistical mechanics have been derived [6][7][8][9][10][11][12][13][14][15], for the first time establishing general relationships between far-from-equilibrium probability distributions over states and the thermodynamic fluxes that accompany their dynamical evolution. The common theme in these findings is that time-reversal symmetry and energy conservation together imply that the rates of stochastic transitions in driven open systems have quantitative relationships with amounts of energy and matter exchanged with surrounding baths. In particular, it has been shown that transitions that are more statistically irreversible must be accompanied by more positive amounts of total entropy production.
We aim here to suggest a new interpretation for one of these results that has potential relevance to our understanding of the organization of driven many-body systems. In particular, we point out that the likelihood of observing a given structure to emerge in nonequilibrium evolution is strongly influenced by the amount of absorption and dissipation of work during its history of formation. We examine the mechanism of this general relationship in simple analytical examples. Subsequently, taking inspiration from the way evolutionary adaptation is understood in a biological context, we argue that many structures formed far from equilibrium may appear to have been specially selected for physical properties connected to their ability to absorb work from the particular driving environment, and we discuss the relevance of this hypothesis to studying the physics of self-organization.

II. DISSIPATION AND DRIFT IN A TIME-VARYING FIELD
At thermal equilibrium, it is well known that the likelihood of observing a system in one of its microstates has a simple exponential relationship to the energy of that microstate. Once external drives push the system away from equilibrium, however, the relationship between probability and the flow of energy in the system, in general, becomes more complicated. In order to start to see how, let us first consider a simple model of a single particle stochastically hopping in a landscape of discrete energy levels. Suppose, for such a system, that state x i has a particular energy E i , and each pair of states x i and x j are separated by an activation barrier of energy B ij ≡ B ji > maxðE i ; E j Þ. To couple to a thermal bath of temperature T ≡ 1=β in a way that is consistent with our underlying assumptions about time-reversibility, we make the Arrhenius assumption that, when our particle is in state x i , it always has a constant probability rate r i→j of stochastically hopping to state x j , which is given by [16] Here, r 0 ij ¼ r 0 ji is a rate constant specific to the pair of states that remains unaffected when we change E i , E j , or B ij . By assuming transition probability rates to take this form, we are guaranteed that, so long as the energies of states and barriers do not change with time, the probability distribution for the location of our hopping particle must, at steady state, converge on a Boltzmann distribution p s ðx i Þ ∝ exp½−βE i that will obey the detailed balance condition p s ðx i Þ=p s ðx j Þ ¼ r j→i =r i→j required by timereversal symmetry.
To drive the system away from equilibrium, we choose the energies E i ðtÞ and B ij ðtÞ to be arbitrary functions of time. Work is done on the system whenever the energy E i changes while the particle is located at x i , and heat ΔQ ¼ E i ðtÞ − E j ðtÞ is exhausted into the surrounding thermal bath whenever the particle hops from x i to x j at time t. In particular, we consider an energy landscape composed of only three states x 1 , x 2 , x 3 arranged in a row so that r 0 12 ¼ r 0 23 ¼ r > 0 and r 0 13 ¼ 0 (Fig. 1). All three states are initially assumed to have the same energy E i ¼ 0 and to be interlaced with two barriers of equal height If the single particle is placed at x 2 at t ¼ 0, then, in the absence of any external drives, it is obviously equally likely to hop to x 1 on "the left" or to x 3 on "the right" in some short time τ ≪ 1=r since r 2→1 ¼ r 2→3 ¼ re −βΔE . Moreover, the probability of returning to x 2 in time τ after making that first jump is also the same for a particle at x 1 or x 3 . We may then choose to drive the system at high frequency so that E 1 ðtÞ ¼ −ΔE cosðωtÞ=2 and B 12 ðtÞ ¼ ΔE − ΔE cosðωtÞ=2, while E 2 , E 3 , and B 23 remain constant in time and ω ≫ 1=τ ≫ r.
The simple point here is that by contriving the external drive in the manner described, we have necessarily also introduced a strong correlation between dissipation and drift. To see this, we note that the rightward hopping rate r 2→3 ¼ re −βΔE does not change with time. In contrast, the leftward hopping rate gets an extra boost from the oscillation of the barrier energy B 12 ðtÞ, so that So long as βΔE ≫ 1, leftward hopping events are much more likely to happen during the part of the drive cycle when the hopping rate is maximal because of the exponential rate enhancement that comes from the downswing of B 12 ðtÞ. At this maximum, the ratio of the leftward and rightward rates is indicating a strong bias towards landing at x 1 rather than at x 3 after initially being placed at x 2 . One may confirm this bias by averaging each hopping rate over a whole drive period 2π=ω, At the same time, it is clear this bias towards one outcome over the other usually requires extra dissipation that accompanies motion in the likely direction. During moments in the drive cycle when transits from x 2 to x 1 are likely (such as at t ¼ 0), our chosen drive has guaranteed that E 1 will be at its minimum value, so a hop to the left must typically be accompanied by a positive amount of dissipative external entropy production βhΔQi 2→1 ¼ βΔE=2. However, hops to the right will not dissipate at all since E 3 ¼ E 2 ¼ 0 do not change with time, so βhΔQ 2→3 i ¼ 0. Thus, we can see that, at least in this case, consistently positive entropy production in the surroundings and concerted drift towards a more likely FIG. 1. Three separate microstates initially of equal energy E ¼ 0 are arranged on a line so that stochastic transitions may take place between adjacent locations separated by activation barriers of height ΔE. When state x 1 and the barrier separating it from x 2 are driven, such that their energies oscillate in time, a preference for x 1 over x 3 as a destination for finite time evolution from x 2 develops. This statistical drift is necessarily accompanied by the conversion of absorbed work into dissipated heat in the surroundings. The energies of the driven states are drawn at their extremal values in blue, with dashed lines and solid lines, respectively, in phase with each other. outcome of stochastic evolution are two sides of the same coin.
The notion that the flow of probability density in driven systems goes in directions that lead to more dissipation of heat has some intuitive appeal, but, not surprisingly, it turns out to be too simple to be true in general. In the next section, we present a relationship between probability flow and thermodynamic flux that does apply in a very broad class of driven systems, thus illustrating, more generally, what role dissipative history plays in determining the outcome of a nonequilibrium dynamical evolution.

III. ENTROPY PRODUCTION AND STOCHASTIC EVOLUTION
The general scenario of interest to us is one where some large number of interacting particles are confined to a volume V that is held in contact with a heat bath at constant inverse temperature β ≡ 1=T. In accordance with the approach pioneered in previous works [6], the classical Hamiltonian of the whole setup is written as where x ¼ fq N g accounts for all the coordinate degrees of freedom for the N particles in the system, y does the same for the bath, and the Hamiltonian functions H sys , H bath , and h int define conservative interactions among the various position coordinates of system and bath. The function λðtÞ plays the role of a time-varying external field that acts exclusively on the system and can do work on the coordinates x. Crucially, h int is assumed to be small, so the term merely plays the formal role of permitting the flow of energy between the bath and the system and may otherwise be ignored.
The standard approach [7] to modeling such a system is to posit that the coupling to the heat bath introduces stochasticity into the observed dynamics, so if one starts off at some particular point in the system's phase space xð0Þ at time t ¼ 0, then, for a given choice of the driving field λðtÞ, there is some probability density for microtrajectories of the system π τ ½xðtÞjxð0Þ; λðtÞ that expresses how likely one would be to observe the system progressing through a given series of subsequent arrangements xðtÞ over time τ. This density is governed by the microscopic reversibility relation put forward by Crooks [7]: The above equation relates the probability of observing a given microtrajectory to the probability of observing the time-reversed movie trajectory after flipping all momentum coordinates in the starting state [x † ðτÞ ¼ ðq Because of the underlying time-reversal symmetry of Newton's laws, the ratio of the forward and reverse probabilities must be exactly equal to the exponential of the heat βΔQ½xðtÞ evolved into the bath in the forward direction as the system traverses the microtrajectory xðtÞ. Since this heat is being transferred to a vast external reservoir at constant temperature T ¼ 1=β, we may accordingly identify it thermodynamically as entropy production and write βΔQ½xðtÞ ¼ ΔS bath ½xðtÞ. Thus, the Crooks result establishes a general, exact, microscopically detailed relationship between statistical irreversibility and entropy production in the surrounding bath, and for the remainder of this work, we treat heat evolution and external entropy production interchangeably.
Recently [14], we have demonstrated the consequences of the above relation for the stochastic dynamics of arbitrarily driven nonequilibrium macrostates. We may define a macrostate by labeling an arbitrary collection of microstates as sharing some macro-observable property I, and in that case, if the system is prepared according to some controlled experimental procedure to be in I, then it will implicitly possess some nonequilibrium density over microstates pðxjIÞdx ≡ p i ðxÞdx. If we furthermore designate some other disjoint set of microstates to have macroproperty II, then we can define π τ ½I → II; λðtÞ as the probability that the system is observed to have property II after stochastically evolving under the applied field λðtÞ for time τ. And if the system is indeed observed to be in II, then it will implicitly be distributed over the microstates available to it according to some new density pðxjII; I; λðtÞ; τÞ ≡ p f ðxÞdx. In this case, we can also define the probability π I τ ½II † → I † ; λðτ − tÞ of reverting back to I if particle momenta are reversed and the applied field is run backward (Fig. 2). With these definitions in hand, it may be shown that FIG. 2. An arbitrary nonequilibrium macrostate for the system of interest is constructed as the probability density p i ðxÞdx over the microstates x when the system is prepared by a controlled procedure to have some macroscopic property I. The system is externally driven for a period of duration τ by a time-varying field H sys ðx; λðtÞÞ while in contact with a thermal bath of inverse temperature β that absorbs heat ΔQ. Afterwards, one may ask whether the system now has a new macroscopic property II; if so, the new probability distribution over microstates is implicitly given by p f ðxÞdx.
where ΔS tot ½xðtÞ≡fln½p i ðxð0ÞÞ=½p f ðxðτÞÞgþβΔQ½xðtÞ. Put succinctly, this result says that statistical irreversibility measured on arbitrary macro-observables still has an exact quantitative relationship to entropy production, with two important differences: first, that one must account for the internal entropy difference between macrostates, and second, that now the entropy must be exponentially averaged in the appropriate way over the statistical weights of all the microtrajectories propagating from the starting macrostate to the ending macrostate. Of course, the fundamental physics behind this statement remains the same as that established by the microscopic relation in Eq. (5). The importance of layering a coarse-graining procedure on top derives only from the fact that experiments on many-body systems invariably involve observing or measuring a limited set of macroscopic criteria or quantities in the system, and so we need to do proper bookkeeping on the number of microscopic ways that the system could appear the same to an experimenter at the macro level. Previously, the above result was used to derive an extension of the second law of thermodynamics, which was applied to studying the thermodynamic constraints obeyed by self-replicators as they grow. Here, we instead use the equation to give a thermodynamic account of driven stochastic evolution. Thus, let us consider a many-particle system prepared in I in the manner described above, and let us suppose that we have defined two distinct possible macroscopic outcomes, II and III, that could lie a long way in the system's future (Fig. 3). A question of great interest to us would be which of the two outcomes should be more likely given H sys ðx; λðtÞÞ, that is, given the way the particles inside the system interact with each other and the way the system is being externally driven. Formally, we may write ln π τ ½I → II; λðtÞ Much can already be learned from the setup of this equation, but a few additional simplifying assumptions help make its structure even clearer: If we assume that the system is driven time symmetrically for a long time, we can dispense with using † operators, and it may also be reasonable to posit that there is no correlation between xð0Þ and xðτÞ other than the one introduced by constraining the starting and ending points to lie in certain macrostates. By conservation of energy, the work done by external fields is given by W½xðtÞ ¼ βΔQ½xðtÞ þ H sys ðxðτÞ; λðτÞÞ − H sys ðxð0Þ; λð0ÞÞ. Since W depends on the whole history of duration τ for the driven system and τ is assumed to be much longer than the time needed to lose all memory in the system of p i ðxÞ, we can treat any correlation between W and p f ðxÞ or p i ðxÞ as being negligible and thus write out the following separate terms: where p Ã bz ðxðtÞ; λðtÞÞdx ¼ I x∈Ã e −βH sys ðxðtÞ;λðtÞÞ dx=Z eq λðtÞ;Ã and Z eq λðtÞ;Ã ¼ R x∈Ã dxe −βH sys ðxðtÞ;λðtÞÞ . This result follows because, according to our assumptions and definitions, ; λð0ÞÞZ eq λð0Þ;I =½p Ã bz ðxðτÞ; λðtÞÞZ eq λðτÞ;Ã g. The first term in Eq. (8) has an appealingly intuitive meaning: The quantity − lnhp Ã f =p Ã bz i Ã ≤ 0 is a functional over the nonequilibrium distribution in macrostate Ã that is only equal to zero when p f ðxÞ is equal to the Boltzmann distribution p bz ðxÞ [which can be shown most easily for a discrete distribution over states i using Lagrange multipliers by solving ½∂=∂p f ðiÞ P i ðp f ðiÞ 2 =p bz ðiÞ − λp f ðiÞÞ ¼ 0]. The behavior of this functional for small variations from p bz has been examined previously in this context since one may then approximate ln P i p f ðiÞ 2 =p bz ðiÞ ¼ ln In this scenario, we consider a starting nonequilibrium macrostate I that is driven by an external field while it evolves stochastically in contact with a thermal bath. We may ask whether, after some finite time τ, it is more likely to be found in one of two other possible macrostates II and III. The relative likelihood of these two different possibilities will be determined in part by the statistical distributions for possible values of the external entropy production βΔQ I→Ã in each case.
the Kullback-Leibler divergence between the equilibrium and nonequilibrium distributions [17]. Meanwhile, F eq λðτÞ;Ã ¼ − 1 β lnZ eq λðτÞ;Ã clearly just measures the equilibrium free energy that would be obtained if the system were equilibrated at inverse temperature β while being constrained to inhabit the states belonging in macrostate Ã. Thus, we can recognize that the quite intuitive effect of these two terms together is to account for the dynamical tendency of any driven system to inhabit macrostates of low equilibrium free energy and to progress towards thermal equilibration within those macrostates. Here, we follow the example established in previous treatments of exponentially averaged work distributions [6] and, defining the so-called "dissipated work" for a trajectory ending in Ã as W d ¼ W − F eq λðτÞ;Ã þ F eq λð0Þ;I , we can combine the second and fourth terms in the above equation into one path ensemble average of he −βW d i I→Ã .
Thus, against the backdrop of an expected tendency to thermally equilibrate dictated by the first term, we are now in the position to weigh the effect of the latter two terms remaining in the equation. The first of these reflects the impact of kinetics. Succinctly, it says that, in a given amount of time, one is more likely to propagate to a macrostate from which one is more likely to revert back in the same amount of time. We can start to get a handle on this term by considering a simple transition-state theory model of a one-step chemical reaction between initial and final states [16]: When the activation free energy barrier is lowered, both the forward and reverse rates of the reaction are accelerated. Thus, one factor we would expect to affect the value of this term is the cumulative impact of all the kinetic barriers that need to be traversed in order to get from a given starting state to a given ending state in a fixed amount of time. Put another way, one more rapidly evolves to states that are closer to one's starting point and separated from it by fewer high barriers. Indeed, this remains the case even when we are talking about ensembles of trajectories that are likely in the forward direction; within this ensemble, the probabilities of trajectories are affected by the heights of barriers that must be traversed whether in the forward (likely) or reverse (unlikely) direction.
In light of this, we seem to be presented with two significant problems. The first of these is that we are purportedly interested in knowing how much more likely we are to see II happen in our driven system than III, and yet in order to say anything concrete, it would seem that we have to know the reversal probabilities π I τ , which requires complete knowledge of all the rates of stochastic transitions that are possible in our system. Put another way, we may as well run a fully detailed microscopic simulation and just see what happens. The second problem is the well-known difficulty of averaging exponentials of large quantities: There are many microtrajectories that run from I to II, and in general, it will be some subensemble of highly improbable ones that make the dominant contribution to hexpð−βW d Þi I→II if we average over all possible paths. This puts us in the untenable position of needing to know the dissipation associated with stochastic events in our system that are immeasurably unlikely to occur [18].
We can start at least by turning two problems into one if we note that everything we have said so far still applies if we choose not to average over all paths leading from one macrostate to another but rather only over a select subensemble of paths representative of most of the probability current. Our goal is to make an inference about how the probability of reaching a given macrostate is related to the typical thermodynamic fluxes that are associated with getting there. Thus, the practical meaning of this restriction is that one must choose a quantitative criterion that defines what we mean by "typical," which should generally be determined for the purposes of calculation by one's ability to sample the distribution of entropy production in the forward direction. Formally, we can propose a procedure whereby some finite number N ≫ 1 of paths are randomly drawn from the forward probability distribution for paths leading between the given starting and ending macrostates. Implicitly, such a procedure excludes "abnormal" types of events unlikely to occur once in N experiments. Thus, the associated work distribution may still be highly non-Gaussian and have substantial contributions to he −βW d i from higher-order cumulants, but it will now have a value that is determined by the statistics of observable events of measurable likelihood. To present a concrete example, if we observed the same seed in the same pot of soil over the course of several weeks in a million repetitions of the same experiment, we would never expect to see a fully grown plant spontaneously self-assemble in the last ten seconds of the interval of observation; rather, all the trajectories we observed (even those that were, in some sense, "one in a million") would look like a seed growing gradually into a plant.
If we restrict the average to these "normal" forward paths, the two things that change in tandem (since we are holding the starting and ending points of the ensemble of trajectories fixed) are the dissipated work averages and the reversal probabilities: We get to average dissipation over events of measurable likelihood, and the "price" we pay is that the reversal probabilities now correspond only to the likelihoods of trajectories that look like reverse movies of trajectories that look normal in the forward direction. This method works because excluding unlikely freak occurrences from our ensemble of trajectories, by definition, does nothing to measurably alter the value of the forward probabilities (i.e., π fwd I→II ≃ π I→II ). So, for example, in the case of a seed growing into a plant, averaging he −βW d i over all possible ways of going from seed to plant would give us a quantity related to the total probability π II→I of turning back from plant to seed. In contrast, averaging he −βW d i fwd only over typical forward paths will STATISTICAL PHYSICS OF ADAPTATION PHYS. REV. X 6, 021036 (2016) 021036-5 yield a quantity related to π rev II→I ≪ π II→I , which is the probability that we observe the plant turning back into a seed via a path that looks like a time-reversed movie of a seed growing in the normal way into a plant. We can thus denote the whole thermodynamic equation more compactly via ln π fwd I→II π fwd It should be noted in passing that in driven systems the value of the reverse probability term is not determined only by distance traveled or barriers traversed because external drives generally affect the probabilities of transition between macrostates in both forward and reverse directions. Moreover, since drives often do not impact forward and reverse transition probabilities between states in exactly the same way (since they push in particular directions at particular times) and because long paths connecting macrostates are made up of many partially independent microscopic moves, we expect π fwd I→Ã and π rev Ã→I between macrostates of equal equilibrium free energy to be positively, but imperfectly, correlated with each other over the range of possible choices of Ã, leading, in turn, to a range of possible values of he −βW d i I→Ã for different end states.
Taking one final heuristic step, we may also note that − lnhexpð−βW d Þi is a cumulant generating function [6] when we expand in β, and thus, it may be written as where Ψ ¼ βhW d i is the average dissipated work and Φ is accordingly defined to account for all of the fluctuations about this average via  [18]. For a simple example, one may consider the case where pðW d − hW d iÞdW d ¼ ð1=wÞ on the interval ½−w=2; w=2, with w > 0. In this instance, Φ ¼ ln ½ð2=βwÞ sinhðβw=2Þ, which grows without bound in the limit of large w ≫ k B T. We can also note that the behavior of Φ is particularly informative in the case of an ergodic nonequilibrium steady state since then hexp½−ΔS tot i ¼ 1, which implies that Ψ − Φ ¼ 0 for the dissipated work measured over any finite time τ. Clearly, the steady-state dissipation rate can take on any value, in principle, depending on the system in question. However, in such cases, the dispersion about the mean rate of dissipation is required by time reversibility to rise as well so that Ψ ¼ Φ. What this points to is that the appearance of Φ in our equation for relative probabilities, among other things, forces us to ignore contributions to the total average dissipated work that are associated with futile cyclical motion performed while the system dwells near a steady state. This factor is therefore closely related to the idea of "housekeeping heat" considered previously in studies of transitions between nonequilibrium steady states [8].
By incorporating this quantity into our thermodynamic expression, we may now write ln π fwd I→II π fwd Equation (11) has a rich structure and may be thought of as a generalization of the Helmholtz free energy for the stochastic evolution of arbitrarily driven nonequilibrium macrostates over finite time. Like many far-fromequilibrium thermodynamic relations, it essentially just summarizes the combined constraints on dynamics that come from time reversibility and conservation of energy and probability. What we would ultimately like to suggest here is that expressing these constraints in this form makes it possible to give them a new interpretation, in which a thermodynamic mechanism for nonequilibrium selforganization is revealed. In the next section, however, we first try to illustrate the relevance of the above expression in a few simple and instructive cases.

IV. DISSIPATIVE HISTORY AND OUTCOME-LIKELIHOOD: TWO EXAMPLES
The nonequilibrium systems described, in principle, by Eq. (11) are extremely diverse in their properties, and before discussing the general implications of the result, it will be helpful to consider how things play out in a few simple examples.

A. Driven hopping particle
First, it is appropriate to return to the hopping particle in a driven energy landscape considered in Fig. 1. Previously, we pointed out that there was a direct correlation in this system between the relative likelihood of two outcomes of the driven dynamics and the amount of heat dissipated on the way to each destination.
What we can now appreciate is that this correlation results from the weight of the ΔΨ term in Eq. (11). Indeed, the reason this effect is so complete in this case is that our choice of external drive in fact contrived the two outcomes of x 1 and x 3 to have equal return probability: By setting the energy E 1 ðtÞ to oscillate in sync with B 12 ðtÞ, the barrier height for the return jump remained the same as B 23 − E 3 at all times. Moreover, in the assumed limit of transitions dominated by the lowest instantaneous energy barrier, the amount of dissipation for a jump from x 2 to x 1 is always the same (Φ ¼ 0) and always positive because E 1 < E 2 at the moments when r 2→1 ðtÞ is maximized. Referring back to Eq. (11) and noting that ΔΦ ¼ 0 in this case, we thus can see we have recovered the predicted relationship Using similar reasoning, we may also consider the effect of the fluctuations Φ on drift. The above discussion has amounted to an argument for why scenarios involving drift in a particular direction must also exhibit elevated external dissipation (that is, more positive Ψ), yet it is clearly the case that not all heat evolution in driven systems is accompanied by such statistically irreversible flow of probability density. Indeed, using the Crooks relation for microstate-to-microstate transitions in such a discrete model, one may easily show that when exp½−βΔQ is averaged over all stochastic trajectories that both start and end in the same state x i , it is generally the case that Ψ i→i ¼ Φ i→i ≥ 0, reminding us that much of the dissipation generated in a driven system can result from futile cycles that go around in circles rather than drifting in some new direction in phase space [8]. Thus, some ways of making Ψ more positive also bring about an increase of Φ in lockstep that cancels any contribution to the flow of probability in Eq. (11). Put another way, high fluctuations in entropy production lessen the degree to which drift and dissipation are correlated.
To see this effect of fluctuations in the case of our hopping particle, we can specialize to a slightly different scenario (Fig. 4). Now considering a pair of states x 1 and x 2 , we can initialize the particle in state x 2 with constant energy E 2 ¼ 0 and suppose there are two distinct paths that connect the states. The first path has a barrier height B 12 ðtÞ ¼ ΔE þ ΔE cosðωtÞ=2, whereas the second path has a barrier of height B Ã 12 ðtÞ ¼ ΔE − ΔE cosðωtÞ=2, and both barriers are governed by the same time constant r. The energy of state x 1 , meanwhile, is assumed to vary with time as E 1 ðtÞ ¼ ΔE cosðωtÞ=2.
In a system thus described, we can straightforwardly compute all of the relevant quantities if we assume we are in a regime where βΔE ≫ 1 and r ≪ 1=τ ≪ ω so that transitions are rare events that take place during the moments in the drive cycle of their maximum likelihood. In this case, it is easy to see that the particle is equally likely to first jump to x 1 via either barrier since maxðr 2→1 ðtÞÞ ¼ maxðr Ã 2→1 ðtÞÞ ¼ r exp½−βΔE=2. Interestingly, however, the mean dissipated work averaged over the weights of these paths should be Ψ ¼ βðΔE=2 − ΔE=2Þ=2 ¼ 0, due to the fact that events on one path happen in phase with the rise of E 1 and events on the other path happen π radians out of phase with it. Moreover, in the limit of ΔE we have chosen, the total return probability rτ ¼ const is effectively insensitive to the exact value of the barrier height since the return journey will always be made via B Ã 21 and not B 21 . However, in this same regime, changes in ΔE strongly affect both the forward transition rate and the fluctuations in entropy production; specifically, the transition rate is ln r tot 2→1 ¼ ln½2re −βΔE I 0 ðβΔE=2Þ ≃ −βΔE=2, and the fluctuations are Φ¼0−ð−lnhe −ΔS iÞ¼lncosh½βΔE=2≃βΔE=2. Thus, for this particular case, we may write what we should have expected from Eq. (11): from which it is apparent that, as fluctuations in entropy production rise with return probability and average dissipation already held fixed, the forward probability rate must fall sharply. The origin of this effect lies in the diversity of paths connecting states, as well as the resulting possibility that the drive can sometimes help one return to the starting point via a different path than it pushed one along when the journey first began.

B. Competing self-replicators
In order to get a more phenomenological perspective on how dissipative history can be predictive of outcomes in FIG. 4. In the driven scenario depicted schematically here, two states x 1 and x 2 are separated by two different barriers representing two different possible paths for transiting between the states. Because the barriers are driven out of phase with each other, and since βΔE ≫ 1 and r ≪ 1=τ ≪ ω, the return probability and average dissipation for transitions from x 2 to x 1 are independent of ΔE, the choice of which should still affect the drift rate from x 2 to x 1 as well as the fluctuations Φ for the pair of paths. As in Fig. 1, the energies of the driven states are drawn at their extremal values in blue, with dashed lines and solid lines, respectively, in phase with each other. nonequilibrium dynamics, we can now consider two populations of self-replicating particles, A and B. The coarse-grained states of interest correspond to different ordered pairs ðN A ; N B Þ which count the numbers of individuals of each type in the system. In any infinitesimal interval of duration dt, an individual of type A or B can spontaneously and instantaneously divide into two individuals via a highly irreversible process X → X þ X with a probability of g A dt or g B dt, respectively. Without loss of generality, we will set g B > g A so that one of these exponentially growing populations is expected, on average, to become an ever-increasing fraction of the total population.
We further assume that both types of self-replicators are equally likely to undergo a reversal reaction X þ X → X with the same stochastic rate constant δ. Although the probability for this process to happen in reverse is assumed to be very small, the time reversibility of the underlying physics forbids it from being zero. Under these rules, the probability of finding N individuals of type X in a given population at time t obeys the master equation We consider the limit where δ → 0, but δ must remain nonzero in order for the entropy production associated with each replication event to remain finite [14] (Note: The form of the entropy production expression takes a slightly different form here than in Ref. [14] because the reverse path defined here occurs by pair annihilation, rather than single-particle decay). We therefore restrict our attention to the case where δ is nonzero but exponentially small compared with the growth rate: lnðg X =½δN X Þ ≫ 1.
Our goal here is to illustrate that external entropy production becomes predictive of the stochastic dynamics of this system once comparisons are made between possible outcome states that all have the same value (or greater) of spontaneous reversal probability. It must be emphasized that at no point is it assumed here that this reversal probability is constant across all possible choices of ordered pairs ðN A ; N B Þ. On the contrary, this quantity is expected to vary dramatically depending on how N A and N B are chosen. As we shall see, however, the examination of a large set of such ordered pairs that share the same value of this reversal term will allow us to make a correct inference about an emergent quality in the system's dynamical behavior as a whole from the way entropy production depends on N A and N B .
Since δ is negligible for the purposes of solving Eq. (14) in the regime we are considering, the probability for stochastic evolution forward in time for both species from a starting condition of ðN A ; N B Þ ¼ ð1; 1Þ over time τ is given by which may be shown by inspection to satisfy the master equation governing the whole population in the δ → 0 limit. Our goal, however, is to compute the reversal probabilities for returning from a given final state with population numbers ðN A ; N B Þ to an initial state with one individual of each species, in a fixed time interval of equal duration τ. The most straightforward way of doing this in this particular system is to leave all the different contributions to entropy together in one bundle calculated from Eq. (15) and exploit the Crooks-type relation in Eq. (7) that relates entropy production to forward and reverse probabilities. In this model, the total entropy production caused by a population as it goes from one individual of each species to N A and N B of them, respectively, will be assumed as which accounts both for the internal entropy of the mixture of indistinguishable self-replicators and for the heat dissipation in the surrounding bath. Combining this expression with the one for p½ð1; 1Þ → ðN A ; N B Þ; τ enables us to write the reversal probability in time τ as Although we have obtained this result by assuming thermodynamic consistency, one may confirm that it is correct by computing the quantity directly as a path integral over individual microtrajectories. All ordered pairs ðN A ≥ 0; N B ≥ 0Þ are, in principle, possible outcomes of this process after a time τ, and under the limits we have described, their associated return probabilities vary by many orders of magnitude (Fig. 5). There are, however, lines through this state space on which the reversal probabilities are constant, and we can use Eq. (18) to obtain the relation between N A and N B that defines these lines. We are interested in this relation in the limit of vanishingly small δ (lnðg X =½δN X Þ ≫ 1), N X ≫ 1 and comparable order-of-magnitude sizes for g A ∼ g B ∼ ð1=τÞ ln N X . In this case, by taking the logarithm of Eq. (18) and setting it equal to a constant, C, we find that the dominant contribution comes from the N X ln δ terms, and we can write N ¼ N A þ N B ≃ C= ln δ, so fixed-reversal probability lines nearly become lines of fixed total population number. In fact, this correspondence cannot be exact since the discreteness of integer lattice sites prevents most states from having identical values of reversal probability. We can, however, use the collection of lattice sites with fixed particle number N as a proxy for the fixed return probability ensemble in its role as a large and diverse set of outcomes. On such level sets, the entropy production takes the simple form where C 0 , C 00 are constants independent of N A and N B . Figure 5 shows how ΔS spans a wide range of values over each line of fixed reversal probability for a typical choice of parameters. Since g B > g A , the expression in parentheses is always positive, so entropy production increases with increasing N B along each line. It further follows that the relative likelihood of states on this level curve increases exponentially with N B , so we obtain here the result for this model that in any half-plane bounded from above by some value of reversal probability, the most likely outcomes (which are also those that have produced the most entropy) are those where N B ≫ N A .
It is worth restating the flow of the above argument. Of course, we already find it obvious that in a competition between two exponentially growing populations of individuals that essentially never die, the type of individual that grows at a faster rate is expected to become more numerous with the passage of time. This intuition is all we need to understand the dynamics of this particular system. However, our goal here is to provide an explanation for this outcome in a thermodynamic language that will remain applicable in settings where the stochastic dynamics are much less straightforward to predict by other means. Here, total entropy production was determined simply by the number of self-replication events of each type, whereas reversal probability essentially tracks the total population size. Once we bound this latter quantity from above, we find that the likeliest outcomes (which are those for which N B ≫ N A ) always turn out to be those for which ΔS is most positive, given the reversal constraint.
Conversely, we can say that if we had no a priori intuition for how the system should behave, we could still infer the likely takeover by species B from the fact that ΔS increases with N B along lines of fixed reversal probability. Here the "structural quality" that correlates strongly with an exceptional dissipative history on any fixed reversal probability surface is having N B ≫ N A , and it is this quality that indeed is expressed in the outcome of the stochastic dynamics. Moreover, if we instead looked at surfaces of fixed entropy production, we would uncover an inverse relationship between reversal probability and total particle number. Accordingly, although an infinite number of different particle numbers are possible, in principle, at any finite time, we expect (and find) that sufficiently large values of N are less likely to be observed than smaller ones.
The examples we considered here effectively illustrate why time reversibility in thermally fluctuating systems implies a general relationship between the concerted flow of probability density and the consistent conversion of work to positive amounts of dissipated heat. In the remainder of this article, we discuss how this relationship may be construed to suggest a thermodynamic mechanism for adaptive self-organization.

V. DISCUSSION
In this article, we have demonstrated a general expression for the time evolution of the probability distribution over macrostates for a driven classical many-body system. Moreover, we interpreted this equation to be a summary of the distinct pressures pushing the system towards particular outcomes. The first such pressure is the most obvious one: All things being equal, we expect our collection of matter in contact with a heat bath to tend towards thermal equilibrium. Next, we account for kinetic accessibility: Not all arrangements of a system are equally "adjacent" or "close" to a given starting point, and thus, only certain arrangements should be considered equally accessible on any finite time scale of evolution. Beyond these effects, the rest of what determines relative likelihood is a function of a state's implicit history of thermodynamic flux. Outcomes from absorption and dissipation of work that is consistently high (corresponding to exceptionally positive Ψ and small Φ) are expected to be favored once the kinetic and equilibration terms are held fixed in an all-things-equal comparison.
It was easiest to see how such a general thermodynamic statement plays out in the two simple examples we already considered, but now we may ask whether the same theoretical framework might reveal interesting consequences in more complex nonequilibrium settings. To answer this question, we pose a thought experiment that will allow us to see implications for systems that seem to undergo qualitative transformations in their pattern of organization as a result of strong external driving.
Imagine an experimental realization of a process of stochastic evolution for a many-particle system in which some particular outcome called II is reliably the one observed after the system is initialized in some nonoverlapping state I and driven for finite time τ with some particular pattern of time variation in the external field λðtÞ. For a concrete example, one might consider the phenomenon of shear thickening in a driven colloid, where particle clumping is caused by the application of external shear to a formerly dispersed mixture [19,20]. By requiring that λð0Þ ¼ λðτÞ and that F eq I ≤ F eq II , we avoid the trivial cases where the transition in question would be favored by undriven dynamics of equilibration. Moreover, we assume that, while the forward probability of evolution π fwd I→II is close to 1, the reverse probability π II→I ≪ 1 is very small (which, in turn, usefully implies that π rev II→I < π II→I ≪ 1 is also very small), so the effect of the drive is not simply to symmetrically accelerate the kinetics of exchange between I and II. Lastly, we need to posit that, with the driving field turned off, the forward transition probability π fwd;eq I→II should be much less than what results in the presence of the drive. Taken together, these stipulations mean that we are aiming specifically to consider a system where introducing the time-varying external driving field triggers a reliable and irreversible configurational shift into a state that would not normally be explored as a result of thermal fluctuations alone in the same amount of time.
In the scenario thus described, the assumptions we have already stated are sufficient to imply that II will have the optimally positive value of Ψ − Φ across a large and diverse set of possible macroscopic arrangements for the system.
Of course, not all arrangements of a many-body system will be equally kinetically trapped; thus, not all macrostates we construct will satisfy our assumption of π rev II→I < π II→I ≪ 1. However, in a system containing many different interacting degrees of freedom, the space of possible particle arrangements that do satisfy this inequality should be expected to remain combinatorially vast, and it is over these possible outcomes that the overwhelmingly likeliest one (II) must have an exceptional implicit history of consistently positive dissipative flux. (Note: Both the focus on dissipative history and account of the effect of fluctuations through the contribution of the Φ term serve to distinguish the result discussed here from maximum entropy production principles that have been explored elsewhere [21].) What does having an exceptionally dissipative past mean for the way the system is configured in the present? To provide an answer, it is essential to realize that not every microscopic arrangement x available to our system of interest will absorb the same amount of work from the same timevarying field; indeed, the instantaneous work rate of the field at any given moment is _ W ¼ ∂H sys ðx; λðtÞÞ=∂t, which generally must depend on the value of x [6,7]. This point can be made most intuitive by considering a system with a rugged energy landscape characterized by the existence of many degenerate local minima that might act as kinetic traps below a certain temperature. Small deviations from one such local minimum should behave harmonically and give rise to a spectrum of corresponding normal-mode frequencies specific to the local steepness of the potential. Consequently, when assessing how much work will be absorbed by the system when it is, for example, sinusoidally driven by an external force with frequency ω, a pertinent question will be the following: How many normal modes for the system have natural frequencies close to ω, such that they might exhibit a high-amplitude, high-dissipation response to the drive? This number of such modes could vary substantially across the vast space of many-particle arrangements. Thus, from such an example, we can take the more generic intuition that special subsets of particle configurations may be particularly well (or poorly) suited to absorb work from a given drive.
We expect, then, that in many cases where external driving brings about a strong shift in a system's state of organization, the outcome we observe at long times should reflect that the system had to be specially configured at some time in its past in order to absorb and dissipate large amounts of work from the drive. Of course, exactly how such a tendency toward "dissipative adaptation" to the drive plays out could lead in very different directions depending on the details of the system. For example, in a scenario where absorption and dissipation of work from a drive fuels a process of positive feedback (such as in self-replication), one expects the system to exhibit an improving ability to cause dissipation in its surroundings over time. On the other hand, in scenarios where the absorption and dissipation of work causes highly irreversible structural changes that cause a drop in the rate of work absorption (such as when resonant vibrations with a song cause the shattering of a wine glass into shards that no longer resonate well), one could observe an end state with a lessened ability to absorb work. In either event, however, we expect that the eventual configuration of the system should be correlated in an informative way with the moments in its history when it absorbed large amounts of work in order to undergo highly irreversible configurational changes.
Strikingly, this manner of thinking can be thought of as the natural generalization of the way people often reason about evolutionary adaptations in Darwinian examples in biology: When a self-replicator exhibits adaptive traits that match well with its present environment, we can seek an explanation in the hypothesis that the organism's ancestors possessed these traits and found them helpful in the struggle to survive and generate many offspring [22]. As we have already seen in the previous section, the link between this picture and the thermodynamic one comes from the fact that, as a result of its statistical irreversibility, self-replication is necessarily a dissipative process [14], so exponential population growth turns out to be one very reliable mechanism of driving external entropy production. In this more general physical frame, however, we can appreciate that even a system devoid of any recognizable self-replicator might still end up seeming "well adapted" to environmental drives in light of the marks on its current configuration left by its exceptional dissipative history.

VI. CONCLUSION
It has certainly long been known that an endless variety of far-from-equilibrium many-particle Newtonian systems are capable of exhibiting self-organization phenomena in which strikingly patterned structures emerge in the presence of dissipative external drives [5]. Whether in sand dunes or snowflakes, in hurricanes or in spiral bundles of protein filaments and motors [3,4], the nonequilibrium world offers many test cases for the general hypothesis that organized, kinetically stable structures emerge and persist because their formation is accompanied by extra work absorption and dissipation [1,2]. While the story in each of these cases must be different in many of its details, we may speculate that a thermodynamic commonality would be revealed to underlie all of them if the right physical observables were tracked and compared.
Guided by these theoretical considerations, we recently demonstrated the predicted phenomenon in a simple simulation framework that tracks the vibration spectrum of a sinusoidally driven toy chemical mixture over time.
The main result of the study (which will be published in a separate article, rather than shown here) is that we do indeed see emergent "adaptive" resonance in the system, such that our choice of frequency for an external driving field determines the location of the peak in the resonance spectrum for a particle mixture that evolves stochastically in the presence of such an environmental drive. This finding turns out to be highly suggestive of the results of recent experiments performed on silver nanorods assembling in the presence of light fields of different colors [23], which similarly "learned" to match their surface plasmon resonance to the frequency of the driving field. Thus, we are encouraged to explore further with related models, spurred on by the intriguing possibility that lifelike adaptive behavior in nonequilibrium systems may turn out to be surprisingly common, now that we have begun learning how to look for its physical signatures.