Nonequilibrium thermodynamics of uncertain stochastic processes

Stochastic thermodynamics is formulated under the assumption of perfect knowledge of all thermodynamic parameters. However, in any real-world experiment, there is non-zero uncertainty about the precise value of temperatures, chemical potentials, energy spectrum, etc. Here we investigate how this uncertainty modifies the theorems of stochastic thermodynamics. We consider two scenarios: in the (called \emph{effective}) scenario we fix the (unknown, randomly generated) experimental apparatus and then repeatedly observe (stochastic) trajectories of the system for that fixed apparatus. In contrast, in a (called \emph{phenomenological}) scenario the (unknown) apparatus is re-generated for each trajectory. We derive expressions for thermodynamic quantities in both scenarios. We also discuss the physical interpretation of effective (scenario) entropy production (EP), derive the effective mismatch cost, and provide a numerical analysis of the effective thermodynamics of a quantum dot implementing bit erasure with uncertain temperature. We then analyze the protocol for moving between two state distributions that maximize effective work extraction. Next, we investigate the effective thermodynamic value of information, focusing on the case where there is a delay between the initialization of the system and the start of the protocol. Finally, we derive the detailed and integrated fluctuation theorems (FTs) for the phenomenological EP. In particular, we show how the phenomenological FTs account for the fact that the longer a trajectory runs, the more information it provides concerning the precise experimental apparatus, and therefore the less EP it generates.


I. INTRODUCTION
The microscopic laws of classical and quantum physics are parameterized sets of equations that specify the evolution of a closed system starting from a specific state.To use those equations, we need to know that specific state, we need to be sure the system is closed, and we need to know the values of the parameters in the equations [1,2].
Unfortunately, in many real-world scenarios, we are uncertain about the precise state of the system, and very often, the system is open rather than closed, subject to uncertain interactions with the external environment.Statistical physics accounts for these two types of uncertainty by building on the microscopic laws of physics in two ways.First, to capture uncertainty about the state of the system, we replace the exact specification of the system's state with a probability distribution over states.Second, to capture uncertain interactions between the system and the external environment, we add randomness to the dynamics in a precisely parameterized form [3].
In particular, in the sub-field of classical stochastic thermodynamics [1, 2] we model the system as a probability distribution evolving under a continuous-time Markov chain (CTMC) with a precisely specified rate matrix.Often in this work, we require that the CTMC obeys local * jan.korbel@meduniwien.ac.at † david.h.wolpert@gmail.com;http://davidwolpert.weebly.comdetailed balance (LDB).This means that the rate matrix of the CTMC has to obey certain restrictions, which are parameterized by the energy spectrum of the system, the number of thermodynamic reservoirs in the external environment perturbing the system's dynamics, and the temperatures and chemical potentials of those reservoirs.
Often we also allow both the Hamiltonian of the system and the rate matrix of the associated CTMC to change in time in a deterministic manner, perhaps coupled by LDB.That joint trajectory is referred to as a "protocol".However, in addition to uncertainty about the state of the system and uncertainty about interactions with the external environment, there is an additional unavoidable type of uncertainty in all real-world systems: uncertainty about the parameters in the equations governing the dynamics.In the context of stochastic thermodynamics, this means that even if we impose LDB, we will never know the reservoir temperatures and chemical potentials to infinite precision (often even being unsure about the number of such reservoirs), we will never know the energy spectrum to infinite precision, and more generally, we will never know the rate matrix and its time dependence to infinite precision.
At present, almost nothing is known about the thermodynamic consequences of this third type of uncertainty despite its unavoidability [4].In this paper, we start to fill in this gap by considering how stochastic thermodynamics (and non-equilibrium statistical physics more generally) needs to be modified to account for this third type of uncertainty, in addition to the two types of uncertainty it already captures.

arXiv:2210.05249v2 [cond-mat.stat-mech] 22 May 2023
We define an apparatus α ∈ A to be any specific set of values of the thermodynamic parameters of an experiment, including the number of reservoirs, their temperatures and chemical potentials, the precise initial distribution over states (i.e., how the system was prepared) the (deterministic trajectories of the) rate matrices, the (deterministic trajectories of the) energy functions, etc.Here and throughout, we assume that these thermodynamic parameters are appropriately related by LDB for any specific α.For simplicity, we also assume that for all apparatuses, the system has the same state space, X.Also for simplicity, we assume that all non-protocol components of an apparatus (in particular the temperatures and chemical potentials) do not change in time.In addition, we assume that for all α, the process takes place in the same time interval, [t i , t f ].We write an element of X as x, and a trajectory of X values across [t i , t f ] as x x x.
We suppose that α is not precisely known and write its probability measure as dP α .Physically, it may be that we have an infinite set of apparatuses generated by IID sampling dP α .Alternatively, dP α could represent Bayesian uncertainty or a detailed model of the noise in the measuring instruments used to set the parameters in α. (Below, we will often abuse notation/terminology and refer to a "distribution" over apparatuses when properly speaking, we should be couching the discussion in terms of a probability measure.)Abusing notation, we will use A to denote both the random variable with values α, and the event space of that random variable [5].
Concretely, we consider two kinds of experimental scenarios.Both start by sampling dP α , but they differ after that: I) In the effective scenario, we generate an apparatus by sampling dP α .For that fixed apparatus we then generate many stochastic trajectories x x x.
After running all those trajectories for that fixed apparatus, we can, if we wish, rerun the scenario, generating another sample of the distribution over apparatuses, which we then use to generate a new set of stochastic trajectories.We call this the effective scenario.
Experimentally, in the effective scenario, one can generate and then observe frequency counts of the distribution p(x x x|α) for multiple random values of α, but without ever directly observing α.For example, the experimenter might construct a bit-eraser experimental apparatus involving a single thermal reservoir whose temperature is fixed throughout the experiment but only known to the finite precision of .1 • K.The experimenter then runs their experiment many times using this fixed apparatus and collected statistics concerning the trajectories across those experiments.They can then use those estimates to make (perhaps Bayesian) estimates of thermodynamic functions of a trajectory, like the associated entropy production.
II) In the phenomenological scenario, we again gener-ate an apparatus by sampling dP α , but the apparatus cannot be fixed while we generate multiple trajectories.Instead, in order to generate a new trajectory we must first generate a new apparatus by resampling dP α .We call this the phenomenological scenario.
To illustrate the phenomenological scenario we can return to the example where the experimenter constructs a bit-eraser experimental apparatus involving a single thermal reservoir whose temperature at the beginning of the experiment is only known to some finite precision of .001• K. Suppose though that the temperature is very slowly drifting randomly in time.Any given run of the experiment is very fast on the timescale of that drift, so we can treat the temperature as fixed throughout the run.However, after generating a trajectory by running the experiment, it takes a long time for the system to be reinitialized to rerun the experiment, and during that time the temperature has drifted to a new value that is statistically independent of the value during the preceding run.
As in the effective scenario, the experimenter runs their experiment many times and collected statistics concerning (functions of) the trajectories across those experiments.However, in the phenomenological scenario, one can only observe frequency counts of the α-averaged distribution over trajectories, p(x x x) := dP α p(x x x|α).
Illustrations of both scenarios are depicted in Fig. 1, for a simple three-state time-homogeneous system coupled to one of the three possible apparatuses.In each case, we measure the marginal probability distribution at the final time t f .Note that in neither scenario do we allow any direct measurement of α.However, there may be indirect information about α that arises from the precise trajectory of states that is generated once α is chosen.Crucially, the ensemble-level thermodynamic quantities generated in these two scenarios can differ, since the two types of average involved (once over α, once over x) do not necessarily commute.As an example, in the effective scenario, since we can form an estimate of p t (x|α) by running many iterations of a fixed experimental apparatus, we can experimentally estimate the entropy defined as using empirical frequency counts.This is not possible in the phenomenological scenario, in which we can only experimentally estimate a more "coarse-grained" version of entropy, (2) α 3 p(α 3 ) (a) 3-state system coupled to one of the apparatuses Comparison of the two uncertain apparatus scenarios considered in this paper.a) A simple, a three-state system that can be coupled to one of the three apparatuses, with respective probabilities.b) We will plot trajectories of the values of a quantity E that has three possible values across the time interval [ti, t f ], along with the associated empirical estimate of the relative probabilities that the system had each of those three values during [ti, t f ] c),d),e) Plots for the effective scenario, where we can estimate the marginal distribution at time t f pt f (E|α) for a fixed apparatus α, shown for three separate instances of the scenario corresponding to the three possible apparatuses.f) The phenomenological scenario, in which each trajectory is sampled with a different apparatus, and therefore only pt f (E) can be estimated.
using empirical frequency counts.Note also that if we are in the effective scenario and have the ability to force a new apparatus to be (randomly) generated whenever we want, then we can implement the phenomenological scenario just by forcing a new apparatus to be generated after every run.In this augmented version of the effective scenario, we could experimentally estimate the quantity in Eq. (2) using empirical frequency counts.However, if we are in the effective scenario and do not have this extra ability, then we cannot estimate the quantity in Eq. (2), only the quantity in Eq. ( 1).(In this paper, whenever we discuss the effective scenario, we will assume we do not have this extra ability.) The difference between the thermodynamics of the two scenarios will be a central focus of our analysis below.

Related research
It is important to distinguish between the focus of this paper and some of the issues that have been investigated in the recent literature.Some recent research has considered how to modify stochastic thermodynamics if the experimentalist is not able to view all state transitions in the system as it evolves [6,7].The uncertainty in these papers concerns what is observed as the system evolves, whereas we focus on uncertainty in the parameters governing that evolution.Similarly, some models consider either spatial [8] or temporal [9] variation of temperature and other parameters, but they assume that this evolution is known.In contrast, we assume that α is fixed throughout the interval, but to an unknown value.
Probably the closest research to what we consider in this paper is sometimes called superstatistics.It has long been known that an average over Gibbs distributions cannot be written as some single Gibbs distribution (Thm. 1 in [10]).This means that even equilibrium statistical physics must be modified when there is uncertainty in the temperature of a system.The analysis of these modifications was begun by Beck and Cohen [11], who developed an effective theory for thermodynamics with temperature fluctuating in time.They considered a system coupled to a bath, which is in a local equilibrium under the slow evolution of the temperature of the bath.The main assumption they exploit is scale-separation: while for short time scales, the distribution over states of the system is an equilibrium, canonical distribution with inverse temperature β, the long-scale behavior is determined by a superposition of canonical distributions with some distribution of temperatures f (β).The resulting superstatistical distribution p(E) = dβf (β) exp(−βE)/Z(β) was later identified with the distribution corresponding to generalized entropic functionals [12,13] due to the fact that particular generalized entropic functionals are maximized by the same distribution that can be obtained the superposition of the canonical distribution with given f (β) [14,15].
Later interpretations of superstatistics are not based on the notion of local equilibria but rather on the Bayesian approach to systems with uncertain temperature [16,17].These are conceptually closer to the focus of this paper, which focuses on off-equilibrium systems that are evolving quickly on the scale of the coupling with the thermal reservoirs, and so cannot be modeled in terms of time-scale separation.
Similar to the quasi-equilibrium scenarios considered in superstatistics, other research has focused on deriving an effective description of the system in local equilibrium averaged over uncertain thermodynamic parameters.In particular, this is the basis of a very rich and well-studied approach to analyzing spin glasses [18,19], in which the coupling constants J ij in the spin-glass Hamiltonian H = − (ij) J ij s i s j , are random variables drawn from a given distribution p(J ij ).Given such a distribution, the famous replica trick ln Z = lim n→0 Z n −1 n [20] can be used to calculate the Helmholtz free energy, averaged over all J ij .Let us note that in the terminology used in disordered systems, the annealed disorder corresponds to the effective scenario while quenched disorder corresponds to the phenomenological scenario.
Finally, several authors [21][22][23][24] investigated the case where the initial distribution differs from the one that would minimize EP.It describes the situation when the system is designed by a scientist who was mistaken in their assumption concerning the initial distribution.In this case, the choice of the non-optimal solution generates the extra entropy production that can be described by the so-called mismatch cost.These papers do not involve a distribution different initial distribution that is re-sampled each time the experiment is re-run.

Roadmap
One of the major themes of our investigation is that some of the details of how an experiment is conducted that experimenters currently do not consider in fact have major effects on the precise forms of various thermodynamic quantities.This is reflected in the difference between (the thermodynamics of) the effective and phenomenological scenarios, discussed above.Even within the effective scenario though, there are some important distinctions between different ways of running the experiment (and so different ways of defining thermodynamic quantities).In particular, there is a major effect on the thermodynamics itself that arises from whether the experimenter the protocol (time-dependent trajectory of Hamiltonians of the system) changes from one run of an experiment to the next, or instead is fixed in all runs.We call these the "unadapted" and "adapted" situations, respectively.We start in Section II with a simple illustrative example of these two situations, involving a moving optical tweezer with uncertain stiffness parameter.
We then begin our more general analysis.First, in Section III, we introduce the necessary notation and briefly recall the main results of traditional, full-certainty stochastic thermodynamics.In Section IV, we present the general form that stochastic thermodynamics takes in the effective scenario (recall the discussion of the effective and phenomenological scenarios in the introduction).We begin by noting that the evolution of the effective probability distribution is not Markovian.Then we derive the forms of the first and second laws of thermodynamics for effective thermodynamic quantities.Next we discuss the relation between effective EP and effective dissipated work.We illustrate this discussion with the numerical example of a fermionic bit erasure with uncertain temperature.We end this section by investigating the special case where the only uncertainty concerns the initial distribution, calculating the associated effective mismatch cost.
In Section VI, we focus on (feedback) control protocols for uncertain apparatuses.In contrast to the conventional case where the apparatus is precisely known, we assume we cannot tailor the protocol for each (uncertain) apparatus separately, but instead must use the same protocol for all apparatuses.We use this setting to investigate how apparatus uncertainty affects a foundational concern of stochastic thermodynamics: How much work can be extracted from a system during a process that takes it from a given initial distribution to a given target distribution.
First, we consider this issue when we are uncertain both about the initial distribution (though not the final one) and about the temperature of the system as it evolves.We focus on how that uncertainty changes the results of the standard analysis of this issue, in which we suppose a {quench; equilibrate; semi-statically-evolve} process is applied to the system immediately after the initial distribution is generated.
Next, we use this analysis to consider how uncertainty affects the "thermodynamic value of information" to a feedback controller [25,26].We restrict attention to the special case of the analysis where the temperature is known exactly, so the only uncertainty is in the initial distribution.We also suppose that there is a (perfectly known) delay between when the initial distribution is generated, t i , and the time τ when the {quench; equilibrate; semi-statically-evolve} process can begin, during which time the system evolves according to a (perfectly known) rate matrix.In particular, we derive expressions for how the thermodynamic value of information varies with the length of the delay.
In Section VII, we investigate the ensemble entropy production calculated from effective trajectory probabilities, i.e., from trajectory probabilities given by averaging over apparatuses.We call this the phenomenological (ensemble) EP.We begin by proving that phenomenological ensemble EP is a lower bound on the average over apparatuses of the effective ensemble EP.So fixing the apparatus and averaging over the trajectories -though without knowing what value the apparatus is fixed toand then averaging over apparatuses increases EP, compared to the case where we average apparatuses before averaging trajectories.
The difference between effective EP and phenomenological EP is called likelihood EP.It measures the difference between log-likelihood functions estimated from the forward and time-reversed trajectories.
Considering trajectory versions of all three EPs, we establish three detailed fluctuation theorems (DFT).In addition to the well-studied DFT in the literature which concerns a single, known apparatus, we establish the DFT for the phenomenological EP and for likelihood EP.The former represents the effective irreversibility of the system by coarse-graining all the apparatuses.The latter represents how irreversibility affects the estimation of the apparatus' parameters when estimated by observing the forward and time-reversed trajectories.These results are illustrated by a simple example of a two-state system coupled to one heat reservoir with uncertain temperature.
The paper ends with a discussion section in which we describe just a few of the myriad directions for future work.

II. ILLUSTRATIVE EXAMPLE
In this section we illustrate the importance of accounting for the uncertainty of the system parameters in an experiment, with a simple example of a colloidal particle in a moving laser trap.The dynamics of the particle is given by the overdamped Langevin equation where ξ is the white noise, and V is the potential.Let us consider that the particle is dragged by an optical tweezer with the harmonic potential where k is the stiffness parameter and λ(t) is the control protocol.The average work is given by the Sekimoto formula where .. is the ensemble average.Let us consider µ = 1.
Our aim is to move the trap from λ i = 0 at time t i = 0 to λ f at time t f such that the average work is minimal.Following [27], it is possible to express the optimal protocol starting that minimizes the average work as and the corresponding optimal work as The complete derivation is done in Appendix A. We focus on the realistic situation where the experimenter has to measure the stiffness parameter to be able to determine the optimal protocol.The estimation is typically done by repeated measurement of k, which leads to a histogram of k.In practice, often an experimenter will implicitly assume that the uncertainty in k is due to the measurement and takes the average value of stiffness k as the single possible value.However, often the uncertainty in the parameters can have a physical reason, e.g., imprecise calibration of the laser.In those kinds of scenarios the stiffness can change for each run of the experiment, and so the experimenter's implicit assumption is invalid.
Write the stiffness parameter that the experimenter uses to set up the control protocol as k, with the real stiffness parameter written as κ (which in general differs from k).In Appendix A we show that the work can then be expressed as using the definition in Eq. (3).
Let us now assume that the experimenter repeats the experiment many times.In each run, the stiffness parameter κ is drawn from a distribution p(κ).We want to compare two broad types of scenario.In the first type of scenario, the experimenter re-measures the stiffness parameter before each run of the experiment, so that they can adapt their protocol in that run to the new value of the stiffness parameter in order to minimizes the average work for that run.This is an example of what we call an adapted scenario below, in Section V.The average work in this scenario can be expressed as In the second scenario, the experimenter ignores the uncertainty in stiffness, and for each run, they assume that the system is described by the same stiffness parameter κ = dκp(κ)κ.This is an example of what we call an unadapted scenario below, in Section V.The minimal expected work in this unadapted scenario can be expressed as Recall that the "dissipated work" of a particular protocol is the difference between the work it expended and the minimal work that would have been expended by an optimal protocol.Dissipated work is one of the central concerns in non-equilibrium statistical physics.Because the "minimal work" differs between the adapted and unadapted scenarios, we would expect that the dissipated work does as well.We illustrate this for the current case of a particle in a trap in Fig. 2. To generate that figure we chose p(κ) ∼ LogN ormal(µ, σ), where µ and σ are the mean and the variance of the log-normal distribution [28].The parameters of the lognormal are λ f = 1, t f = 1, µ ≡ κ = 3 2 .Fig. 2 demonstrates the general fact that the adapted and unadapted scenarios can differ in their thermodynamics; with the increasing variance of the distribution, the difference is more pronounced (Note that both adapted and unadapted work decrease with increasing σ which is caused by the fact that the expected work is in the case a concave function of stiffness).
Note that choosing the protocol λ κ (t) in this scenario is not the optimal choice that minimizes the average work.(The issue of protocols that optimize the average work are discussed in Section VI.)

III. PRELIMINARIES
For simplicity, throughout this paper, we assume the system of interest has a countable state space X with elements generically written as x ∈ X.A trajectory of values of X across some non-infinitesimal time interval [0, t] will be written as x x x t , or just x x x for short.The trajectory's state at a specific time t is denoted as x x x(t).Throughout this paper, we will often leave the time t implicit.The Kronecker delta function with arguments x, x is written as δ x,x and D x (p||q) = x p(x) ln p(x) q(x) is the Kullback-Leibler (KL) divergence between distributions p and q, also known as the relative entropy [29].When the KL divergence is also averaged over apparatuses, we denote it as D x,α (p||q) = dP α x p(x|α) ln p(x|α) q(x|α) .Throughout this paper, capital letters indicate random variables that are averaged over, while lowercase letters indicate specific values of such random variables.

A. Instance-level, trajectory-level and ensemble-level quantities
A generic function y t (x) depending on state x (and possibly on time t) is called an instance-level value.Instance-level values are denoted by lowercase letters.Given a distribution over the states p t (x), we write for the expectation of the instance value y t (x) under a distribution p t (x).We refer to such expected values specified by a particular t as ensemble values, and denote them in uppercase letters.Also, we will often leave both the distribution and the time t implicit when they are obvious from context.We refer to quantities specified in terms of a sequence x x x of instance values over a time interval [t 0 , t] as trajectorylevel, typically denoted as y y y [t0,t] (x x x).When the initial time t 0 is fixed, we use only y y y t (x x x).Trajectories of values are denoted by bold lowercase letters, with an index t indicating the ending time of that trajectory.Given a distribution over trajectories P(x x x), we denote the expected value of an associated sequence as the ensemble(- Dx x x P(x x x)y y y t (x x x). (5) where Dx x x denotes the path integration over all trajectories.

B. Brief review of stochastic thermodynamics
We now quickly review conventional stochastic thermodynamics, in which there is no uncertainty about the thermodynamic parameters [1,2,30].A more extensive review can be found in Appendix B. Readers already comfortable with stochastic thermodynamics can skip to the next section.
Conventional stochastic thermodynamics considers a system with a specified Hamiltonian that is coupled to N independent infinite thermal reservoirs, which have associated inverse temperatures β ν , This coupling between the systems and the reservoirs results in the system evolving according to a CTMC, where as described below, the physical requirement of microreversibility puts constraints on how the rate matrix of the CTMC is related to the physical parameters of the process.Below we go through the main definitions of relevant quantities used in stochastic thermodynamics.Their explicit formulas as well as the extension to the case of particle reservoirs are shown in Appendix B.

Ensemble thermodynamics
The ensemble internal energy is written as The system exchanges energy with each of the reservoirs (e.g., via kinetic molecular collisions).Additionally, one can consider that the system exchanges particles with some of the reservoirs.This extension is discussed in Appendix B and used later in one of the examples.The first law of thermodynamics can be formulated as where is the total work on the system during that interval.The heat flow rate Q Q Q t can be decomposed into heat flows in from the separate reservoirs, i.e., direct energy flows in from the separate reservoirs: The entropy rate can be decomposed as where ∆S t = S t − S ti , Σ Σ Σ t is the entropy production (EP) and E E E t is the entropy flow (EF).The second law of thermodynamics is enforced by the fact that for any rate matrix, EP rate is non-negative, i.e., Σ Σ Σ t ≥ 0 and so Σ Σ Σ t ≥ 0. When LDB holds, the EF rate can be expressed in terms of thermodynamic entropy, i.e., In this case, Eq. ( 7) means that if there is only a single heat bath, with inverse temperature β By Eq. ( 6) this can be written as where ∆F t is the difference in the Helmholtz free energy between time 0 and time t.Suppose we wish to minimize the work expended over all possible protocols (u t , K t ) acting on such a system, so long as they take some specified initial pair of an energy function and distribution (u * t0 , p * t0 ) to some specified ending pair, (u * t f , p * t f ).For that specified pair, ∆U t f is fixed, independent of the protocol.So by Eq. ( 6) the minimal such work is Plugging in, the minimal work over all protocols is ∆U t f − ∆S t f /β.Therefore the quantity on the RHS of Eq. ( 10) is the difference between the actual work expended to go from (u * t0 , p * t0 ) to (u * t f , p * t f ) and the minimal possible.Accordingly, that quantity is called the dissipated work, and by the first law it can be written as a function of the initial distribution as follows: In this way Eq.( 10) connects a purely thermodynamic quantity to a purely dynamic quantity, defined in terms of rate matrices.Note that the energy function and distribution are treated specially in the definition of dissipated work -no other parameters of the thermodynamic process are allowed to vary when determining the "minimal" work.
Let us now define the prior initial distribution as the one that minimizes that function when Eq. (11).We can consider the directional derivative of that function, evaluated for any initial distribution, in the direction of any other initial distribution.In particular, that directional derivative of the dissipated work must equal 0 when evaluated at the prior initial distribution q ti (x) in the direction of any other initial distribution, p ti (x).This can be used to show that where the distributions p ti (x) and q ti (x) evolve into the distributions p t f (x) and q t f (x), respectively.Σ Σ Σ(q ti ) is called the residual EP [31].The thermodynamic process implemented by a given apparatus is thermodynamically reversible iff the residual EP of the process is zero, and the initial distribution happens to equal the prior of the process.
The drop in KL divergence in Eq. ( 12) is called the mismatch cost of running the process with initial distribution p ti .By the data-processing inequality for KL divergence, the mismatch cost is nonzero, and by inspection, it equals zero if the initial distribution equals the prior.So mismatch cost is the extra EP generated by running the process with initial distribution p ti rather than the prior q ti , in addition to the residual EP which would be generated if the process were run with the prior as the initial distribution.

Trajectory thermodynamics
The trajectory internal energy is written as u u u t (x x x).The first law of thermodynamics on the trajectory level for any time t is where q q q t (x x x) = ν q q q ν t (x x x) is the trajectory heat and ẇ w w t (x x x) = x δ x,x x x(t) ut (x) trajectory work.Trajectory entropy is defined as s t (x x x) := − ln p t (x x x(t)).The time derivative of entropy can be decomposed as where σ σ σ t (x x x) is the trajectory EP rate and ˙ t (x x x) is the trajectory EF rate.It is straightforward to verify that by averaging these trajectory-level quantities over all trajectories, we recover the ensemble-level versions, i.e., q Due to LDB, trajectory EP can be expressed as σ σ σ(x x x) = ln P P P (x x x) where P P P (x x x) is the probability of observing trajectory x x x and P P P † (x x x † ) is the probability of observing time-reversed trajectory x x x † (t) := x x x(t f −t) under time-reversed protocol.Defining P (σ) := Dx x xP(x x x)δ(σ − σ σ σ(x x x)), it is straightforward to show that the trajectory EP σ fulfills the detailed fluctuation theorem, where P † denotes the probability under a time-reversed protocol and σ is random variable [2].Finally, ensemble EP can be expressed as Kullback-Leibler divergence between probabilities of forward and reversed trajectories [32] Σ Σ Σ = D x x x (P P P (x x x)||P P P † (x x x † )) .
IV. EFFECTIVE THERMODYNAMICS As described in the introduction, we are interested in how the conventional laws of stochastic thermodynamics concerning the evolution of a system change when there is uncertainty about the parameters of that evolution, but we assume LDB holds, whatever those parameters are.
In Section IV A, we describe a general framework of effective quantities that are averaged over different apparatuses.In Section V B we continue the description of effective ensemble thermodynamics in the general case where the transition rates of the systems are not known with infinite precision.Such uncertainty must arise whenever we do not know the exact number of heat reservoirs, their temperatures/and or chemical potentials.However, uncertainty can even arise when we do know those quantities exactly, and even when we impose LDB.This is because even if we knew those quantities to infinite precision, LDB does not uniquely fix the rate matrix K, and so there can still be uncertainty concerning K.
We exemplify the effective thermodynamics approach in Section IV C on the example of a bit erasure for the case of fermionic bits.We start with the brief review of system when coupled to a heat bath with certain temperature.Then we continue with the generalized case when the system is coupled to a heat bath with uncertain temperature.

A. Effective quantities
Suppose we are given some generic quantity Y α , which depends on the apparatus α, and which may also depend on time and/or the random trajectory through the system's state space (where any of the dependencies may be implicit).The probability distribution at time t is denoted as p t (x|α).Trajectory probability is denoted as We define the effective value of Y as its expectation over α, and write it as where we require that the α-dependence of all arguments of Y α is made explicit.So for example, if Y α depends on the initial probability distribution over states, that distribution occurs in its argument list as p t0 (x|α).In particular, the effective state probability at time t is where we use the shorthand p α t (x) := p t (x|α).We also write the effective trajectory probability up to time t Note that in both scenarios I and II, since the apparatus is fixed throughout [t i , t f ] once it is sampled, the joint dynamics over X × A is given by the master equation, where . Averaging both sides of Eq. ( 22) over α and interchanging the derivative and average on the LHS, we get where So the dynamics over the system considered by itself is simply the dynamics of a coarse-graining of the joint system-apparatus.Since the transition rate matrix of that coarse-grained dynamics depends on the probability distribution, the effective dynamics is not described by a linear Markov master equation.

B. Effective ensemble stochastic thermodynamics
We write W W W α for the ensemble energetic work expended for a specific apparatus α as it sends p α ti to p α t f .By conservation of energy where the change in the internal energy for each apparatus α during the process is In addition, by the second law, for any specific apparatus α, These quantities are given by integrating over time and fixing a specific apparatus.If we instead average over apparatuses and fix a specific time, we get the formulas for the effective ensemble energy and effective ensemble entropy at time t: If we run over both time and apparatuses, we get the formulas for the effective work on the system and heat transferred to the reservoir(s) up to time t: In addition, the effective ensemble EF rate is and where the effective ensemble EP rate is Plugging in the expression for Σ Σ Σ α t in terms of rate matrices, we confirm that where Combining these definitions, we can write the first and second law of thermodynamics for effective ensemble quantities as respectively, simply by averaging the first and second laws over all α.Note in particular that since Σ Σ Σ t ≥ 0, the total effective ensemble EP generated in the interval is non-negative.This is why Eq. ( 38) can be identified as the effective scenario's second law.
On the other hand though, Eq. (32) shows that in general the effective ensemble EF rate cannot be expressed in terms of the expected effective heat flows for the separate reservoirs, Q t , despite our assumption that LDB holds for each apparatus separately.This discrepancy between the two rates would exist even if we knew with certainty that there was only one reservoir, i.e., N = 1; the discrepancy reflects the fact that there can be statistical coupling between β α (and K α,ν and therefore) Q Q Q α t , due to the uncertainty over α.This discrepancy means that the time-derivative of the effective ensemble entropy is not necessarily lower-bounded by the ensemble effective heat flow rate.So the version of the effective scenario's second law of thermodynamics, Eq. ( 38) is not as consequential as the standard version in which there is no uncertainty about α.
C. Example: Bit erasure of information stored in fermionic bits with uncertain temperature We now illustrate how environment uncertainty affects the design of an experimenter's protocol as well as the associated EP for the effective scenario, using the example of bit erasure of a fermionic bit in finite time [33].First, we review the analysis when there is no uncertainty about temperature.Then we extend that analysis by introducing uncertainty about the temperature of the single heat bath, and therefore (in order to enforce LDB) in the trajectories of the rate matrices and so of the energy function.
Bit erasure of information stored in fermionic bits Our system has two states, labeled 0 and 1.The probabilities of those two states are denoted as p t (1) ≡ p(t) and p t (0) ≡ 1 − p(t) respectively.In this subsection, we will denote the dependence of a generic quantity x on time t as x(t) (rather than x t ) to clarify the reasoning.In particular, the chemical potential of an electron in the quantum dot is written as µ(t).(Recall that in Appendix B we discuss the extension of stochastic thermodynamics to include chemical potentials.) For simplicity, we will only consider variations in the trajectory of energy functions across t ∈ [t i , t f ], with a fixed map specifying the rate matrix that goes with each possible energy function.Specifically, we set the rate matrix at all times t to where and we use the shorthand h(t) = u(t) − µ(t)n 1 for the difference between the energy and the chemical potential times n 1 = 1.Since there is no remaining freedom to vary the trajectory of rate matrices across t ∈ [t i , t f ] once we specify a trajectory of energy functions across t ∈ [t i , t f ], we will sometimes refer to that trajectory of energy functions as a "control protocol".Given Eq. ( 39), we can write and so The solution is From now on, for simplicity, we take t i = 0. Consider a special type of control protocol chosen so that for all t, arbitrary δ > 0. Note that the RHS is independent of β.Plugging in to Eq. ( 40), the associated control protocol is Similarly, plugging in to Eq. (43) shows that Note that despite the notation, this quantity does not depend on the temperature.Finally, the EP is Note that EP also does not depend on temperature, due to the choice of the control protocol h β (t).

Bit erasure with temperature uncertainty
We now consider the variant of this bit erasure scenario where the experimentalist does not know the temperature exactly but instead has some distribution dP β = p(β)dβ.We suppose that the experimenter is in full control of the control protocol, but that the same, single protocol will be used for all apparatuses.
Averaging both sides of Eq. ( 43) gives the formula for the evolution of the effective distribution: with where h * (t) is the protocol chosen by the experimentalist to be used for all apparatuses.Note that this is the same evolution one would get by first averaging both sides of Eq. ( 42), and then solving for p(t).This reflects the fact that in this particular situation, where Eq. ( 41) holds, we can write K(t)p(t) = K(t) • p(t), and so the dynamics of the effective distribution is Markovian (in contrast with the general case).
Similarly to the no-uncertainty analysis, we assume the transition rate matrix has the form To this end, we should find a control protocol (independent of β) such that Eq. ( 51) equals Eq. ( 46).Next, define the function Thus, the control protocol h (t) can be obtained by solving the equation for the transition rate k(t) = Ψ(h * (t)).
By solving the equation, we obtain Note that h * (t) is not the β-average of h β (t) which can be expressed as is the harmonic mean of beta (corresponding to the arithmetic mean of temperature).
In the actual experiment, however, the system is coupled to a bath with the certain temperature β and therefore the transition rate is Thus, the probability distribution of a system coupled to a heat reservoir with temperature β using the protocol h * , can be obtained from the equation As an illustration, consider the special case where there are two possible temperatures.The first temperature T 1 = 1/β 1 occurs with probability P (T 1 ) and the second temperature T 2 = 1/β 2 occurs with probability P (T 2 ) = 1 − P (T 1 ).Thus, the distribution P (T ) can be expressed as and therefore As usual, the control protocol can be obtained by solving Eq. ( 53).Moreover, since that protocol is independent of T , by definition the effective ensemble entropy production is In Fig. 3 we plot the control protocol h(t), probability distribution p 1 (t), and entropy production Σ Σ Σ(t) for for this case of bit erasure with uncertainty about which of two possible temperatures the bath has.We assume parameters T 1 = 1, T 2 = 10, t f = 10 and δ = 0.1, and take P (T 1 ) = P (T 2 ) = 1/2.
In the top-right panel of Fig. 3 we plot h * (t) along with the control protocols h T (t) for the two cases where there is no temperature uncertainty, for the two possible values of T = {T 1 , T 2 }.The expected distribution p(t) is displayed in the bottom-left panel, along with the two distributions p T h * which could actually occur in the experiment, when the experimenter fixes the protocol to h * (t) and the temperature of the bath is one of T = {T 1 , T 2 }.Finally, we compare the total EP for the case of certain temperature Σ Σ Σ T (t) (which does not vary with T ), with the case of total EP for the case of uncertain temperature given by Σ Σ Σ h * in the bottom-right panel of Fig. 3. Despite the fact that p(t) coincides with p(t), the effective ensemble EP increases to the case of EP for the case of a certain temperature.For comparison, we depict The main reason why Σ Σ Σ h * is higher than Σ Σ Σ is the fact that for the situation when T 1 is chosen, the protocol h * forces the distribution p T1 h (t) to decrease much below p(t), which on one hand makes the bit erasure more efficient (the final probability distribution is closer to (1, 0)), but one has to pay much more dissipated work and consequently entropy production Σ Σ Σ T1 h * to get the distribution closer to the ideal bit erasure.This work is not compensated in the other case when the temperature is equal to T 2 .We see that while the entropy production Σ Σ Σ T2 h * is lower than Σ Σ Σ, this does not compensate enough the entropy production Σ Σ Σ T1 h * and the average entropy production Σ Σ Σ h * is larger than Σ in the case when the temperature is certain.Let us finally mention that this is a similar situation hidden Markov pump [34] when the actual EP (here corresponding to Σ Σ Σ calculated from the complete microscopic structure of the hidden pump in lower than when the entropy is calculated for a coarsegrained pump.I.e., the experimenter's knowledge of the  3. Bit erasure of a fermionic bit with uncertain temperature.The design of the experiment is depicted in the top-left panel.For each site, the bit is coupled to one of the metallic leads with different temperatures T1 = 1 and T2 = 10 with equal probability.The control protocol for each temperature and for the case of uncertain temperature is depicted in the top-right panel.The actual probability distribution for a given temperature and control protocol is depicted on the bottom-right panel.The entropy production for the case of a certain temperature as well as an uncertain temperature is depicted in the bottom-right panel.
setup affects the obtained EP.While in the case of [34] the experimenter lacks the precise information about the system, there the experimenter lacks the precise knowledge of the heat bath and its temperature.

V. THE TWO TYPES OF EFFECTIVE DISSIPATED WORK
In much of the rest of this paper we focus on the case of a single bath.Specifically, we investigate how the properties of dissipated work change from those in Eq. ( 11) when we that number of baths but introduce uncertainty in the other parameters .
It turns out that there (at least) two natural ways to extend the reasoning that results in Eq. (11) to that case where the number of baths is fixed to one but there is uncertainty about the other parameters defining the apparatus.These two extensions reflect two different ways of defining "minimal possible effective work".

A. The adapted and unadapted scenarios
Write αt := (u α t , K α t ) to mean the components of α that specify the protocol, evaluated at time t.Similarly write α t to mean all components of α t other than those that specify the protocol, e.g., the initial probability distribution, the temperature, chemical potentials, and other thermodynamic forces, etc.Thus, α t can be decomposed as α t = ( αt , α t ).Recall though that for simplicity we are assuming that all non-protocol components of α are time-independent, so we can simplify this to α t = (α t , α).
In the sequel, when we are considering the entire trajectory over times t of the values (u α t , K α t ) specified by a particular α, we write (u α , K α ), in keeping with our convention that bold characters indicate trajectories.Similarly, we write α without a subscript to mean the entire trajectory of the protocol, (u α , K α ).
In the first approach to extending the definition of the minimal possible effective work, we define it as the least work that could be achieved if the energy component of the protocol, u, were fully fixed in an explicitly known manner by the experimentalist, before the other uncertain parameters, α, were (randomly) determined.To make sure that uncertainty in parameters like the temperature do not cause LDB to be violated even though the parameters other than u α are set independently of u α , we must allow the rate matrix trajectories K α to be statistically dependent on both u α and α.So in this approach, we restrict attention to measures over α of the form with no statistical coupling between the energy component of the the protocol and the non-protocol parameters defining the apparatus.(It is the term dP (K α |u α ,α) in Eq. ( 59) that allows us to ensure that LDB holds).
We use the term unadapted to refer to measures of this form.For unadapted measures, the minimal possible work is We also use the term "unadapted" to refer specifically to the kind of optimal protocol given in Eq. ( 60), and to the associated definitions of least possible (effective expected) work and of dissipated (effective expected) work.In particular, the unadapted dissipated work is the amount of work that could have been saved if the experimentalist has intervened to change the marginal distribution dP α, leaving all other aspects of the experimental setting the same.(An example of unadapted dissipated work is presented in Section IV C.) In the second approach, we do not compare the actual work to the best that could have been achieved if the experimentalist had changed dP α to be some delta function, leaving dP α unchanged.Rather we consider the best possible work that could have occurred "by luck ", if there were the best possible statistical coupling between the protocol and the other thermodynamic parameters, again leaving dP α unchanged.Formally, in this second approach we define the "least possible" work for a given dP α to be the minimum of the expected work as one varies over the conditional P α|α .
We refer to this measure over apparatuses as the adapted statistical coupling between the protocol and the other parameters specifying the apparatus that could have occurred, counterfactually, for the specified (fixed) marginal over those other parameters.We also refer to the associated minimal effective ensemble work as the adapted minimal (effective ensemble) work, and refer to the associated dissipated effective ensemble work as the adapted (effective ensemble) dissipated work.Note that we can self-consistently define the adapted dissipated work for a single apparatus α, in the sense that the adapted effective dissipated work is the α-average of the adapted dissipated work for a single α.The analogous property does not hold for the unadapted dissipated work in general, since the min and the integral in Eq. (60) will typically not commute.(Indeed, if they did commute, then the unadapted dissipated work would equal the adapted dissipated work.)

B. Physical interpretation of effective EP for a specific adapted scenario
We now describe a relationship between effective dissipated work and effective ensemble EP, to provide a thermodynamic interpretation of effective ensemble EP.In this subsection we focus on calculating the minimal adapted dissipated work.
For simplicity, we suppose that all the apparatuses α with nonzero probability take the same initial distribution p * ti (x) to the same ending distribution p * t f (x), i.e., for all α with nonzero probability, p α ti (x) = p * ti (x) and p α t f (x) = p * t f (x).This means that ∆S α is independent of α.
We similarly assume that the initial and final energy functions are independent of α.By Eq. ( 26), these two assumptions mean that ∆U α is independent of α, and so in particular it is independent of the protocol.So by Eq. ( 61), the adapted minimal effective ensemble work is Also for simplicity, from now on we assume that each reservoir for any specific apparatus α has the same temperature, so that we can write β ν,α = β α .Under this assumption the second law reduces to the inequality In general, the minimum of Eq. (63) will occur when the protocol α accompanying the rest of the apparatus α saturates the second law.As a result, the adapted minimal work is Plugging into Eq.( 37) and combining, the adapted dissipated work is On the other hand, multiplying both sides of Eq. (62) by β α before averaging over α, then again using the fact that ∆U α is independent of α, we get Plugging this into Eq.( 68) and rearranging gives So in general, in this setting where all apparatuses map p * ti (x) to p * t f (x) and ∆U α is independent of α, the effective ensemble EP is not proportional to the effective (adapted) dissipated work, in contrast to the nouncertainty case.Instead it equals the effective inversetemperature-weighted dissipated work (Eq.( 70)), or alternatively it is an affine function of the effective dissipated work (Eq.( 71)).Note though that if β is fixed, independent of α, then the relationship between adapted effective dissipated work and effective EP mirrors their relationship in the no-uncertainty case: (Compare to Eq. ( 70).)As a variant of this scenario, suppose instead that we still have a single reservoir, one which does not exchange particles with the system, and also still suppose that the other thermodynamic parameters can vary with α.However, now suppose that both p α ti and p α t f can also vary with α, unlike before.To have the definition of minimal adapted work still be meaningful in this situation, assume that the protocol cannot change either of those two distributions, although it can change intermediate distributions.Formally, this means that the support of P (α|α) is restricted so that for all α, no α is possible which affects the initial and final distributions.(If changing the protocol were allowed to change the initial and / or final distributions, then in general the minimal adapted work would be arbitrarily negative.)Under this assumption changes to α without any changes to α do not change the drop in entropy, ∆S α,α .Nor do they change ∆U α , which is still independent of α.However, now changes to α will change the drop in entropy.In this case Eq. ( 68) gets replaced by C. Mismatch cost and effective ensemble EP for a specific unadapted scenario A common setting in which we have uncertainty about thermodynamic parameters is where we know everything about the apparatus with complete certainty -the number of reservoirs and their parameters, the paths followed by the Hamiltonian and the rate matrix, etc. -except that we do not know the initial distribution that is run with those parameters.Concretely, this setting arises whenever we have a fixed physical apparatus that is run with a randomly generated initial distribution.(So we are concerned with unadapted dissipated work.)One common example is a computer that is used by different users; each user implicitly fixes a distribution over the inputs to the computer, i.e., fixes its the initial distribution over the states of the computer.Other common examples are a single cell floating in different environments, or a fixed digital gate that can be positioned at different locations in a digital circuit (and so have different distributions over the inputs it receives from the rest of the circuit).For simplicity, for the rest of this subsection take β = 1.
In the situation under consideration, where we are certain about the temperature of the process, effective dissipated work and effective EP are identical, given by Eq. ( 72).By applying the formula for the mismatch cost Eq.( 12), we get the formula for the effective mismatch cost In particular, the expected effective mismatch cost is where R α is the transition matrix of the apparatus α, with entries This effective mismatch cost is strictly positive in general since the integrand is always non-negative and only equals zero in degenerate cases.In particular, in the situation considered in this subsection, the rate matrix trajectory is independent of α, and therefore so is the prior.So the effective mismatch cost reduces to As an illustration, suppose that R takes all initial distributions to the same ending distribution (i.e., Rp α ti = Rq ti ).For example, this happens in bit erasure, or in complete relaxation of a system to its stationary state.In this case the ending KL divergence is zero, no matter what p α ti is.Since the KL divergence is a convex function of its arguments, this means that the prior that would minimize effective mismatch cost is just the effective initial distribution, q ti (x) = p ti (x).
If we plug this into Eq.( 76) we can evaluate the minimal value of the contribution to EP arising from uncertainty about the initial distribution.Since effective residual EP is non-negative, we see that where D JS ({p α ti }, P α ) denotes the Jensen-Shannon divergence among the set of distributions {p α ti } distributed according to P α .So minimal EP is strictly positive, as long as P α puts nonzero probability mass on at least one distribution p α = p.This provides a strengthened version of the second law, applicable whenever there is uncertainty about the initial distribution, and all initial distributions with nonzero probability get mapped to the same final distribution.

VI. CONTROL PROTOCOLS IN A SPECIFIC UNADAPTED SCENARIO
As discussed at the end of Section III B, along with Sections V and V B, in no-uncertainty stochastic thermodynamics, the "dissipated work" is defined as the difference between two amounts of work.The first is the actual work under a given protocol that maps an initial distribution p ti (x) to a final distribution p t f (x).The second is the least possible amount of work that would be required to implement that map under any (counterfactual) protocol, with all thermodynamic parameters other than the protocol, left the same.One way to modify this definition of dissipated work for the case of uncertain thermodynamic parameters was investigated at the beginning of Section V B. In that investigation, the initial and final distributions were both fixed, independent of the (non-protocol components of the) apparatus α.We also supposed that P (α|α) = P (u, K|α −(u,K) ) was optimized for extracting work, whether due to conscious intervention by the experimentalist or just by chance.That optimizing conditional distribution was called the adapted protocol.
In the next subsection we introduce a modification of that scenario considered in Section V B. In the following subsection, we analyze the dissipated work for that modified scenario.

A. Modifying the adapted scenario
First we list the modifications, and then discuss their formal subtleties: 1.One of the reasons for interest in dissipated work in no-uncertainty stochastic thermodynamics is because often the experimentalist can intervene in their experiment, in an (essentially) arbitrary way, even if they do not know α.In the language of Section V, this is an unadapted scenario.Formally, in such a scenario the distribution over the set of counterfactual protocols, P (α|α), is independent of α, since the experimentalist cannot choose the protocol they implement to match the other uncertain parameters specifying the apparatus (by definition of their being uncertain about those other parameters).Physically, this means that the experimentalist can set the protocol -but can only do so before the other thermodynamic parameters are generated by sampling the distribution over apparatuses, in a way that is independent of the protocol they have set.
2. Often the experimentalist does not in fact have complete freedom to vary the entire protocol arbitrarily.Often they will be able to set the trajectory of the energy function directly, but cannot directly set the trajectory of rate matrices.Instead, for each different apparatus, LDB will constrain the relationship between the rate matrix and the quantity that the experimentalist can control, the energy function.
3. Often, the experimentalist will want to only consider the counterfactual situations that result in some pre-specified final distribution, p * (x).However, they will be uncertain about the initial distribution, as well as other thermodynamic parameters that govern the dynamics, like the temperature.
Item 1 means that the protocol of the apparatus is statistically independent of the other thermodynamic parameters: To ensure Item 2 while requiring that LDB holds for each apparatus we make several simplifying assumptions.First, we assume that there is only a single reservoir in all of the apparatuses that have a nonzero probability of occurring and that the reservoir only exchanges energy with the system, not particles.So the only uncertainty parameter concerning the reservoirs that is relevant to ensuring LDB is the single number β α (which we will sometimes write as α β ).In addition, we suppose that there is a single-valued function M : (u t , β α ) → K α (t), a function which the experimentalist can set arbitrarily (subject to the constraint of LDB), and so will know with zero uncertainty.So by "varying the set of counterfactual protocols" we vary the marginal distribution P (α u ) and the function M , with In our analysis below, it will also be convenient to assume that the support of dP α β has a finite minimum (so that there is a maximal possible temperature), and to require that M be differentiable with bounded derivative.There are various formal challenges that arise in the analysis, depending on the precise definition of Item 3.Here we clarify Item 3 to mean that averaged over apparatuses, the final distribution is p * , rather than requiring it have that form for all α with nonzero probability.That then leaves the formal challenge of ensuring that we can meet this Item 3. If the protocol is chosen so that at least one of the apparatuses is non-infinitesimally off-equilibrium at t f , then if that apparatus happens to occur, in general the state distribution p t (x) would pass through p * (x) transitionally at the time t f , after which p t (x) would keep evolving to other distributions, no matter what the protocol happens to be then.Indeed, in general, it may be that two apparatuses α, α both cause the state distribution to pass through p * (x) at t f , but result in different state distributions at all other times, both before and after t f .This is a complication that doesn't occur in the conventional, no-uncertainty scenario for the case of a single reservoir, where the counterfactual protocol is completely arbitrary, and so the optimal counterfactual protocol results in the system being at thermal equilibrium at all times.
To see how to address this problem with ensuring that the condition in Item 3 is met, recall the conventional, two-step {quench; semi-static protocol} process often considered in analyses of the minimal thermodynamic cost, in which there is no uncertainty, and one needs to set a protocol to change (u ti , p ti ) → (u t f , p t f ) [35,36].
Here we restrict attention to protocols that expand that conventional two-step process, into a three-step {quench; equilibrate; semi-static protocol} process.In other words, we insert an intermediate step into the conventional process, a step in which we wait long enough for the system to relax to equilibrium, no matter what its temperature, before starting the step with the semi-static protocol.(We are assured of being able to do this due to our restrictions on the support of dP β α .)Due to our assumption that the function M (.) has a bounded derivative, we can choose the semi-static protocol in which u(x) varies in that subsequent third step slowly enough so that (the rate matrix evolves slowly enough so that) the system is always at equilibrium throughout that step, again no matter what α is.
As a formal point, we also assume that the very first energy function, u ti , cannot be chosen by the experimentalist, i.e., it is randomly distributed as specified by P (α).However, the experimentalist "takes over" the specification of the trajectory of the energy function starting with the energy function that the system is quenched to, the one immediately following u ti .
As in the conventional no-uncertainty version of the two-step protocol, we require that the distribution over states does not change during the quench step of this expanded, three-step protocol.This is true no matter what the temperature of the bath is.This means both that there is no heat exchange with the bath in that step, and that there is no change in Shannon entropy in that step, no matter what α is.So this step is thermodynamically reversible; if we inverse-quenched right away, we would return to the initial distribution and energy u, with zero net work expenditure.Moreover, since the system is at equilibrium throughout the semi-static evolution step, that step is also thermodynamically reversible; we could run it backward, returning to the energy and (equilibrium) distribution at the beginning of that step, and there would be zero net work.This is also true no matter what α is.
On the other hand, in contrast to the quench and semistatic steps, in general, the intermediate, equilibration step will result in dissipated work for each α considered by itself.More precisely, while no work is extracted during that step, in a counterfactual process work could have been extracted, if the protocol could be tailored for that α.
The first of the following subsections contain a preliminary analysis of this dissipated work.In the subsection after that, we investigate some related issues that arise when a feedback control protocol is used to define a "thermodynamic value of information about the actual apparatus".

B. Optimal work extraction when the temperature is uncertain
Write the Boltzmann distribution for arbitrary energy function u and inverse temperature β α as Also write u α ti for the initial, pre-quench energy function for the case where the apparatus is α, and write u q for the energy function that the experimentalist chooses for the system to quench to.Similarly, write u t (with no α superscript) for the energy function at times t during the semi-static evolution (times at which the experimentalist chooses the energy function).
With this notation the protocol of the process for each α (which is determined before the process starts) can be written as the following: where due to the semi-static nature of the evolution in the third leg, . As a shorthand, it will also be convenient to write the non-equilibrium internal energy and free energy for arbitrary energy u, state distribution p, and apparatus α as where in general, u and / or p may depend on α.Note that our requirement that dP α p α t f (x) = p * (x) means that This imposes a strong condition on the ending energy function that the experimentalist chooses.Indeed, in the no-uncertainty case, this requirement would uniquely fix that energy function, up to an overall additive constant.Since no work is expended or extracted during the equilibration process (p ti , u q ) → (π α , u q ), the total work is given by summing the work during the quench and semistatic evolution steps.Whatever the protocol is, for each α this total work is Adding and subtracting S(p α ti )/β α to Eq. ( 85) and rearranging, we can rewrite this total work as The expected work is the α-average of this expression.
The first term on the RHS of Eq. ( 86), the difference between the nonequilibrium free energies immediately preceding the quench and at the end of the process, is independent of u q .So the experimentalist cannot affect this term by appropriate choice of protocol.The second term is instead the change in nonequilibrium free energy that arises in going from just before to immediately following the equilibration step.
To evaluate that second term, first, note that since no work is done on the system in the equilibration step, the change of the expected energy is due to the heat flow, i.e., ∆U = Q Q Q.So the expected value of that second term is where abusing notation, Q α is defined to be the heat flow occurring in just the second, equilibration step, for apparatus α.Eq. (89) establishes that the (α-average of the) second term in Eq. ( 86) is just the effective dissipated work of the equilibration step, and therefore the effective dissipated work of the entire process.(See Section V B.) Because π α uq is the Boltzmann distribution for energy function u q and inverse temperature β α , as usual in equilibrium thermodynamics, In addition, S(p α ti ) is independent of u q .Combining, the u q that minimizes the expression in Eq. ( 87) is the one that minimizes To solve for the u q minimizing this expression, take ∂ ∂uq(x) for each x and set it to zero.This gives a set of coupled equations that u q (x) must satisfy: x e −β α uq(x ) = 0 i.e., x e −β α uq(x ) (92) where We denote this optimizing solution to Eq. (92) as u * q .Note that it is independent of u ti , u t f and p * .
As an example, suppose there is no uncertainty in the temperature, only in the initial distribution.Then we could immediately invert Eq. (92) to get up to an irrelevant additive constant.Note that this is also the energy function that would be optimal if with zero uncertainty we knew that the initial distribution were p ti (x).More generally, suppose that there is both a nonzero minimal value and a finite maximal value of supp(dP β α ), i.e., a nonzero minimal temperature and a finite maximal temperature that have nonzero probabilities under dP α .Then up to an overall additive constant, there is one and only one solution u q to Eq. (91) for any given p ti and dP α .(This is proven in Appendix C.) By plugging u * q (x) into Eq.( 86) and averaging over α, we get a formula for the expected work during the full protocol: The first line in Eq. ( 96) is the expected work that would be required if we were able to use a different (optimal) protocol that was set by the realized value of α, i.e., it is the minimal work, discussed in Section V B. So the second line in Eq. ( 96) is the extra work required due to our not being able to use such a protocol that depends on the realized value of α.In other words, it is the extra work due to our having limited information about the random apparatus.This kind of increase in the minimal work if the protocol is prevented from depending on the realized value of a random variable is sometimes called "thermodynamic value of information" in the literature [35,[37][38][39][40]. What differs between our analysis here and the previous analyses is that we are concerned with the thermodynamic value of information of the parameter α, whereas those previous analyses instead considered the thermodynamic value of information on the initial state of the system.

C. Dynamics of the thermodynamic value of information
In this subsection, we modify the scenario considered in Section VI B in two ways.First, we assume that the experimenter can only set the protocol starting at a time τ > t i , whereas dP α is sampled at t i .(We refer to the protocol during the interval starting at τ as the "experimenter's protocol".)Second, we assume that we know with certainty that β α = 1 for all α during that time of the experimenter's protocol, following τ .In particular, β α = 1 during the equilibration process.So the only uncertainty the experimenter faces in choosing their protocol is in what the distribution is when that protocol takes over the dynamics.
In general, the system will evolve between t i and τ , potentially according to an uncertain rate matrix.Our concern is with how the size of the gap τ − t i affects the minimal work that must be expended during the experimenter's protocol, in light of that evolution between t i and τ .In particular, we investigate how the thermodynamic value of knowing the precise α in that interval depends on the length of that interval, i.e., the derivative with respect to τ of that value of information.
Since there is no uncertainty in the temperature during the experimenter's protocol, we can plug into Eq.(95) with t i replaced by τ , to see that the optimal energy function for the experimenter to quench to at time τ is up to an overall additive constant [41].
If the experimenter had had no uncertainty when they perform the quench, the protocol starting at τ would have resulted in zero dissipated work.Accordingly, we define the thermodynamic value of information for this scenario as the effective dissipated work during the experimenter's protocol that does arise, since there in fact is uncertainty.
Recall that by the analysis in Section VI B, this dissipated work arises during the equilibration that starts immediately following the time of the quench, which in our current scenario is the time τ .This dissipated work is given by modifying Eq. (87) to reflect the fact that there is no temperature uncertainty during the experimenter's protocol: where u * q is given by the solution to Eq. (98).Expanding the two free energies and plugging in, we can write this effective dissipated work as (See the discussion at the end of Section VI B.) This is the Jensen-Shannon divergence of the set of distributions {p α (x)}, weighted according to dP α .Furthermore, since α is a random variable, we can re-express each distribution p α (x) as a conditional distribution p(x|α).This allows us to also identify I τ as the mutual information at time τ between x and α, mutual information that vanishes by the end of the equilibration step.So I τ is the usual expression for the thermodynamic value of information found in previous analyses in the literature [35,38,42].Here though it reflects the thermodynamic value in knowing what the initial distribution is, while in those previous analyses, it reflects the value in knowing what the precise initial state is [43].
We wish to evaluate the derivative of I τ with respect to τ .To begin, we consider the case where the rate matrix K xx (t) during the interval t ∈ [t i , τ ) has no uncertainty.We can see how the thermodynamic value of information of α depends on τ in this case, by taking the derivative of the RHS of Eq. ( 101) with respect to τ .In Appendix D we show that the value of information in this case, I τ , is monotonically decreasing in time, i.e., stopping to fall only when the system reaches equilibrium.(We can derive the same result from the dataprocessing inequality.)Physically, this means that the thermodynamic value of information of α decreases the longer we wait to use it.
In addition to the τ -dependence of the thermodynamic value of information of α, one might also be interested in how the total effective EP (i.e., effective dissipated work), generated during the entire interval [t i , t f ], changes with changes to τ ∈ [t i , t f ].There are two contributions to that EP: the thermodynamic value of information I τ , generated during the equilibration step, and the EP generated earlier, during the interval [t i , τ ).So to get the derivative of the total effective EP generated during the entire interval [t i , t f ], we must add two terms.The first is the derivative of the effective EP under K(t), i.e., the effective EP rate, evaluated at t = τ .The second is the time derivative of the value of information evaluated at that time.
From Eq. ( 34), that effective EP rate at τ is Adding this to Eq. (D3) of Appendix D gives the derivative of the total effective EP generated during [t i , t f ] as This is the EP rate generated by the effective dynamics of p, evaluated at τ .So the derivative of the effective EP generated during [t i , t f ] with respect to the time τ ∈ [t i , t f ] at which the experimentalist takes over the protocol is the phenomenological EP generated at τ .As a final comment, suppose that α indexes not just the distribution, but also the rate matrix, i.e., suppose that we are unsure about the dynamics as well as the distribution.This means that In this case, the value of information of α can increase with τ .As a simple example, suppose that the initial distribution is independent of α, i.e., p α ti (x) = p ti (x).Then I pt i = 0. Suppose as well that α has two possible values, as does x, and that dP α is uniform over α's two values.Finally, also suppose that for α = 0, the associated trajectory of rate matrices K 0 xx (t) sends p ti (t) to the ending distribution p 0 τ (x) = δ(x, 0) with arbitrarily high accuracy, while for α = 1, the associated trajectory of rate matrices K 1 xx (t) sends p ti (x) to the ending distribution p 1 τ (x) = δ(x, 1) with arbitrarily high accuracy.Then I pτ is arbitrarily close to ln 2, which establishes the claim.
To illustrate the richness of value of information due to uncertainty, we also consider the value of information for the case where there is uncertain temperature, and therefore (imposing LDB) uncertain rate matrices leading up to the time that the experimentalist's protocol takes over.It is unknown what the general necessary and sufficient conditions would be for the value of information due to uncertainty to be nondecreasing as τ increases.This is a potentially fruitful area of future research.To exemplify this fact, we calculate the value of information for the example from Section IV C, i.e., the bit erasure for the case of the uncertain temperature.We observe that the value of information increases with time, until it reaches its maximal value around t ≈ 8 and then starts to decrease.

VII. PHENOMENOLOGICAL EP AND FLUCTUATION THEOREMS
We now focus on the second, phenomenological scenario, where the apparatus changes after each stochastic trajectory is generated.In this scenario we are not able to measure P(x x x|α) for any (unknown) apparatus α, but only the average probabilities P(x x x).

A. EP in the phenomenological scenario
To begin, recall that the α-average of the effective EP can be expressed as Writing dP α = p(α)dα as shorthand, the joint trajectory probability is P(x x x, α) = P(x x x|α)p(α).So the α-averaged effective ensemble EP is Now by Bayes' theorem, where we used the fact that P(x x x) = dP α P(x x x|α).Combining this with the chain rule for KL divergence [44], we derive For later use, introduce shorthand for the first term on the RHS of Eq. ( 109), which we call phenomenological EP.Phenomenological EP measures the irreversibility of the dynamics quantified by the effective (i.e., apparatus-averaged) probabilities.
Similarly, define the second term on the RHS of Eq. (109) as which measures the difference between distributions of α estimated from forward and reverse trajectories.Like many forms of EP, Λ Λ Λ is a Kullback-Leibler divergence and is zero only if the forward probability is the same as the backward probability.For these reasons, we refer to Λ Λ Λ as likelihood EP, even though it need not have a straightforward relation to dissipated work.All three EPs in Eq. ( 109), Eq. ( 110), and Eq. ( 111) have associated trajectory-level versions: φ φ φ(x x x) := ln P(x x x) The first EP corresponds to the effective scenario, where we randomly fix the apparatus and generate an infinite set of trajectories for that apparatus.The second EP corresponds to the phenomenological scenario, where the apparatus randomly changes after generating each trajectory.
In general, there will both be α for which σ σ σ(x x x|α) > φ φ φ(x x x) and α for which σ σ σ(x x x|α) < φ φ φ(x x x).However, Eq. (109) means that So fixing α to the same value all stochastically generated trajectories, calculating the associated trajectoryaveraged EP, and then averaging over the unknown values of α increases the ensemble EP, compared to the case where we average over apparatuses to calculate EP, and only then average over apparatuses.The last of these trajectory-level EPs in Eq. ( 114), the likelihood EP, is the difference between the first two.While the likelihood EP is a log-likelihood ratio, in contrast to the common case in which log-likelihood ratios are based on the same data but different parametric models, this one is based on the same set of apparatuses, but on forward, resp.reversed trajectories.
The trajectory version of Eq. ( 109) can be written as For the first EP σ, we get to derive the ordinary fluctuation theorem for a given α.For the case of phenomenolog-ical EP and likelihood EP, we define the following probabilities: we can write down detailed fluctuation theorems: The consequence of the detailed fluctuation theorem is the integrated fluctuation theorem, i.e, e −φ = 1, resp.e −λx x x = 1.The first of those IFTs implies that φ ≡ Φ ≥ 0. The second of those IFTs means that We call this the second law of inference.It tells us that for each trajectory x x x, the α-averaged log-likelihood dP α ln P(α|x x x) is larger than the α-averaged likelihood obtained from the time-reversed trajectory.the reverse process.Of course, by averaging over all trajectories with probability P(x x x), we obtain Λ Λ Λ = Λ Λ Λ x x x ≥ 0.

B. Example: two-state system with uncertain temperature
To illustrate these results, consider a two-state system with states {0, 1}, as depicted in Fig. 5.The energy levels corresponding to the states are E 0 and E 1 .We consider the transition rate matrix The initial distribution is p 0 = {1/2, 1/2}.Suppose that the temperature is randomly drawn from the Gamma distribution P (T ) = T e −T (top-left panel of Fig. 5).
The trajectory entropy production is ) By observing a trajectory x x x (top-right panel of Fig. 5), we calculate an updated distribution P(T |x x x) (bottomright panel of Fig. 5).Finally, we can also calculate the likelihood EP (bottom-right panel of Fig. 5).In the inset histogram of the bottom-right panel, we see that the likelihood EP λ λ λ x x x can attain negative values, however λx x x = Λ x x x is greater than zero, which is in agreement with the detailed fluctuation theorem and the second law of inference.5. Illustration of the second law of inference on a two-state system with uncertain temperature.Before the experiment, the temperature is randomly chosen from the prior distribution p(T ) (top-left panel).The experimenter cannot measure the temperature, but can observe the trajectory x x x(t) (top-right panel).From observing the trajectory, it is possible to update the temperature distribution P (T ) → P (T |x x x) (bottom-left panel).Similarly, the temperature distribution obtained from observing the time-reversed trajectory under the time-reversed protocol is P (T |x x x † ).From their log ratio, it is possible to express the likelihood EP λ λ λ(T |x x x) (bottom-right panel).By calculating the histogram of λx x x, we observe that negative likelihood EP is observed much less common than positive likelihood EP, which corresponds to the detailed fluctuation theorem.Finally, the average value of likelihood EP remains positive, which is in agreement with the second law of inference.

VIII. DISCUSSION AND FUTURE WORK
In any real-world experimental test of a system there are three major types of uncertainty: a) uncertainty about the state of the system b) uncertainty about the state of the external environment that the system is interacting with, and c) uncertainty about the parameters of the equations governing the dynamics of the system and its interaction with its external environment.In (classical) stochastic thermodynamics, the first type of uncertainty is addressed by replacing the specification of the system's state (e.g., coarse-graining), and the second one is typically addressed by assuming the environment is infinite, at equilibrium, and evolving far faster than does the system ("separation of time scales").In essence, the entire field of stochastic thermodynamics concerns the consequences of those two types of uncertainty for the dynamics of energy and particle counts in the system.However, very little attention has been paid so far to the third type of uncertainty.Here we begin an investigation of the consequences of that third type of uncertainty, showing that it entails major modifications to the standard results previously derived in stochastic thermodynamics.
Our investigations have only scratched the surface of issues involved with this third type of uncertainty.In the light of recent studies [45] where the system is coupled to a reservoir with fluctuating temperature, it makes sense to extend the analysis from a static distribution p(α) to a distribution over dynamic trajectories of the parameters, i.e., P P P (α α α t ).Another possible future direction can motivated by extending the framework to the case of stochastic thermodynamics hidden Markov models [46] with the application to biophysical systems as flashing ratchets [47,48] when the experimenter cannot directly observe whether the ratchet is on or off, and therefore does not also know the switching rate.Some of the more immediate questions to be addressed in future work include the extension to the stochastic control protocols.The first possible question is whether the {quench; equilibrate; semi-statically-evolve} protocol is the optimal protocol which maps p x (t i ) → p x (t f ) or if there is an alternative protocol that generates less effective dissipated work.
Furthermore, it is reasonable to assume that in systems with limited possibility of measuring the system's parameters, it will be even more difficult to realize the desired control protocol with infinite precision.The related question is how the partial knowledge of the system obtained by the (imprecise) measurement from observing a trajectory, i.e., P (α) → P (α|x x x t ) can be utilized to adjust the control protocol (x) → (x|x x x t ).In more realistic situations, the measurement cannot be done continuously but at given time instants and might bear some costs.The question of optimal measurement and update of control protocol in uncertain environments is challenging but definitely a crucial question to be answered.
Let us now consider that κ is distributed by the distribution p(κ).We choose our k determining our protocol to be k = κ = dκ κ p(κ).Then the expected work (unadapted) W = dκW κ (λ κ(t)) is equal to On the other hand, if we were able to choose the optimal protocol for each κ, the expected work (adapted) would be Since κ 2+κt f is a concave function, we obtain from the Jensen inequality that W ad. ≤ W unad. (A17) Appendix B: Brief review of stochastic thermodynamics In this section we present some additional details of ordinary, no-uncertainty stochastic thermodynamics, and review some of the main associated results.We leave N and associated parameters like the inverse temperatures β ν and chemical potentials µ ν implicit.We write the rate matrix for going from state x to state x due to stochastic exchanges of heat and / or particles with reservoir ν as K ν x,x (t).From now on, we leave the time index implicit.The full rate matrix of the system is K x,x = N ν=1 K ν x,x , and so the master equation of the CTMC is ṗt (x) = x K x,x p t (x ).
Often in the literature, due to considerations involving time-symmetric microscale dynamics, we assume that at any (implicit) time t the separate matrices K ν x,x each satisfy local detailed balance (LDB) with respect to the energy level u(x) at that time, for the particle reservoir ν: x = e −β ν [(u(x)−u(x ))−µ ν (n ν (x)−n ν (x ))] .(B1) where u(x) is the energy level of the system in state x and n ν (x) is the number of particles of the type specified by ν when the system is in state x.(If the reservoir ν does not exchange particles with the system, then µ ν = n ν (x) = 0.) Note that in the absence of chemical reservoirs, the map from rate matrices to energy functions is singlevalued (up to an overall additive constant), but the inverse map is multi-valued.

Ensemble thermodynamics
The ensemble internal energy is written as U t := u t = x p t (x)u t (x).
The system exchanges particles with some of the reservoirs, as well exchanging energy with each of them directly (e.g., via kinetic molecular collisions).The first law of thermodynamics can be formulated as where ∆U t := U t − U 0 and Q Q Q t := t 0 dt Q Q Q t is total heat flow into the system during the interval [0, t], W W W t := t 0 dt W W W t is the total work on the system during that interval, and C C C t := t 0 dt C C C t is the total chemical work during that interval.The heat flow rate Q Q Q t can be decomposed into heat flows in from the separate reservoirs, i.e., direct energy flows in from the separate reservoirs: For simplicity, we ignore the possibility of more than one distinguishable type of particle.The mechanical work flow and chemical work flow are defined as Shannon entropy at time t is defined as S t := s t = − x p t (x) ln p t (x).
The second law of

*FIG. 2 .
FIG.2.The comparison of the average work for an unadapted and adapted scenario for the moving optical tweezer with uncertain stiffness.
FIG.3.Bit erasure of a fermionic bit with uncertain temperature.The design of the experiment is depicted in the top-left panel.For each site, the bit is coupled to one of the metallic leads with different temperatures T1 = 1 and T2 = 10 with equal probability.The control protocol for each temperature and for the case of uncertain temperature is depicted in the top-right panel.The actual probability distribution for a given temperature and control protocol is depicted on the bottom-right panel.The entropy production for the case of a certain temperature as well as an uncertain temperature is depicted in the bottom-right panel.
FIG. 4. Value of information for the case of bit erasure with uncertain temperature from Section IV C.
FIG.5.Illustration of the second law of inference on a two-state system with uncertain temperature.Before the experiment, the temperature is randomly chosen from the prior distribution p(T ) (top-left panel).The experimenter cannot measure the temperature, but can observe the trajectory x x x(t) (top-right panel).From observing the trajectory, it is possible to update the temperature distribution P (T ) → P (T |x x x) (bottom-left panel).Similarly, the temperature distribution obtained from observing the time-reversed trajectory under the time-reversed protocol is P (T |x x x † ).From their log ratio, it is possible to express the likelihood EP λ λ λ(T |x x x) (bottom-right panel).By calculating the histogram of λx x x, we observe that negative likelihood EP is observed much less common than positive likelihood EP, which corresponds to the detailed fluctuation theorem.Finally, the average value of likelihood EP remains positive, which is in agreement with the second law of inference.