Fluctuating Entropy Production on the Coarse-Grained Level: Inference and Localization of Irreversibility

Stochastic thermodynamics provides the framework to analyze thermodynamic laws and quantities along individual trajectories of small but fully observable systems. If the observable level fails to capture all relevant degrees of freedom, some form of effective, coarse-grained dynamics naturally emerges for which the principles of stochastic thermodynamics generally cease to be applicable straightforwardly. Our work unifies the notion of entropy production along an individual trajectory with that of a coarse-grained dynamics by establishing a framework based on snippets and Markovian events as fundamental building blocks. A key asset of a trajectory-based fluctuating entropy production is the ability to localize individual contributions to the total entropy production in time and space. As an illustration and potential application for inference we introduce a method for the detection of hidden driving. The framework applies equally to even and odd variables and, therefore, includes the peculiar case of entropy production in underdamped Langevin dynamics.


I. INTRODUCTION
How can we apply the principles of stochastic thermodynamics to effective descriptions?Originally, stochastic thermodynamics emerged as a framework to identify and formulate thermodynamic laws for small-scale systems coupled to the environment via, e.g., random thermal fluctuations and driving forces [1,2].As the dynamics itself and, consequently, associated thermodynamic quantities like heat, work or entropy become inherently stochastic, the notion of a 'system' only makes sense if there is a meaningful, 'clean' separation from the environment.More specifically, a meaningful notion of energy or entropy requires a notion of 'thermodynamic consistency' when implementing external effects into the stochastic dynamics of the system.
Thus, if such a system can be identified, e.g., in the form of a Langevin equation or Markovian dynamics on a discrete set of states, the framework of stochastic thermodynamics provides far-reaching, universal relations like, e.g., the Jarzynski equality and its generalizations [3][4][5], different formulations of fluctuation theorems [6][7][8][9][10], the Hatano-Sasa relation [11] and the Harada-Sasa relation [12], many of which have been realised experimentally, as reviewed in [13].
These results are formulated for a complete system in the sense that the stochastic dynamics gives rise to trajectories that include all relevant degrees of freedom.With complex real-world systems in mind, we can ask what remains if we are unable to observe the full system or, even worse, if a clear distinction between system and environment cannot be made based on the available data.Such questions have seen growing interest in recent research, leading to a point where the study of partially accessible information might be regarded as an emerging pillar of stochastic thermodynamics in its own right.* J.D. and J.v.d.M. contributed equally to this work.
Up to now, the study of stochastic thermodynamics of partial information contains two mostly disjoined aspects.Existing methods of thermodynamic inference [14] mainly focus on extracting or deducing particular averaged quantities based on thermodynamic relations and limited access to observables.However, a partially accessible system features its own effective dynamical laws, which emerge as a projection of the underlying dynamics.Thus, inference techniques formulated on such a dynamical level should be able to provide thermodynamic bounds on a fluctuating level beyond simple averages.
In contrast, thermodynamic inference employs a variety of methods primarily aiming at the estimation of mean entropy production.Most prominently, various formulations and generalizations of the thermodynamic uncertainty relation [15][16][17] provide bounds on the minimal thermodynamic cost to achieve a certain precision of, e.g., a current.Furthermore, lower bounds on entropy production can also be based on an identification as a Kullback-Leibler divergence [18][19][20][21][22][23][24], the speed at which the system evolves in time [25,26], stopping times [27], waiting times [28] and counting events [29].Novel recent approaches include dynamical correlations into entropy estimation techniques by relating entropy production to the asymmetry of cross-correlations of accessible observables [30][31][32], their power spectral density [33], correlation times [34], or, more technically, the spectrum associated with the dynamics [35].Beyond estimating the average value of entropy production, inference techniques for the topology of the underlying system [36,37] and its driving affinities [30,38] have also been proposed.
The dynamics of coarse-grained systems received much attention in earlier works on stochastic thermodynamics, usually in the context of state lumping [39,40].More recently, a gradual paradigm shift in the conception of coarse graining challenges established paradigms [41,42].To fully understand real-world scenarios it seems indispensable to find descriptions for dynamics on the coarsegrained level, with recent approaches including milestoning [43,44] or semi-Markovian dynamics [38].
The present work unifies the notion of entropy production along a single trajectory with that of coarse-grained dynamics.We establish a framework that allows us to identify entropy production along individual trajectories that retains meaning on the levels of both the underlying system and the coarse-grained, observable one, even if the coarse-grained dynamics does not obey simple rules like a master or Langevin equation.The concept of fluctuating entropy production for individual, coarse-grained paths comes along with the ability to localize individual contributions to the total entropy production in time and space within the coarse-grained description.This new aspect enables the inference of more detailed information far beyond the average total entropy production of a system.From a practical viewpoint, we demonstrate that the ability to attribute irreversibility to specific events or transitions between these provides access to complex settings like the localization of entropy production in the absence of any visible states as well as the qualitative and quantitative detection of irreversibility in hidden parts of the system.
The paper is structured as follows.In Section II, we start with an outline of our main results and the setup under consideration.We then present our theoretical framework by introducing the two crucial concepts through which the identification of the fluctuating entropy production becomes possible.This framework enables us to to derive a method to detect hidden driving and to estimate affinities of hidden cycles, as we describe in Section III.In Section IV, we discuss the intricacies of entropy production in the presence of odd variables.In particular, we show that the present identification of entropy production is compatible with odd variables that occur in underdamped Langevin dynamics and transition-based coarse graining.We conclude in Section V.

A. Illustration of the main concept
In real-world scenarios we commonly encounter systems for which a description that can be considered 'complete' or 'fundamental' in an appropriate sense would be highly complex.Thus, simplified descriptions for such systems that capture its key characteristics are indispensable.In this context, Markov networks have become one of the predominant paradigms for which the entire wellestablished framework of stochastic thermodynamics is available.A description based on Markov networks typically requires the presence of clear time-scale separations to separate system and environment.However, the nature of the underlying system or insufficient observational data may jeopardize such a clear time-scale separation.Our framework is designed for descriptions that lack this clear separation.
These aspects are qualitatively illustrated in Figure 1.The red lines show a section of an individual trajectory.We assign an entropy production to such sections that are both localized in time and space.This identification of entropy productions retains physical significance even on the coarsegrained level.
The energy landscape gives rise to five observable discrete states.The local minima at states 1, 3 and 4 are sufficiently deep to justify a description as Markov states, in contrast to the shallow minima of states 2 and 5.The red line shows an exemplary trajectory, which, on the observable level, reads 1 → 2 → 3. We will show how to identify a fluctuating entropy production for such trajectories that include non-Markovian observables like, in this case, state 2. This coarse-grained entropy production obeys several consistency conditions and provides an estimator for the entropy production on the fundamental level.
The ability to localize entropy production in this example translates into the ability to discern contributions to the total entropy production of, e.g., trajectory sections of the type 1 → 2 → 3 from the remaining ones as indicated in the lower part of Figure 1.Similarly, it is possible to localize the entropy production in time.Depending on the application, one might be interested in the contributions that arise from, e.g., particularly fast trajectories.

B. General setup
We consider a stochastic physical system whose microscopic trajectories γ are drawn according to some path weight P[γ].For example, if the time-evolution γ(t) obeys a Langevin or master equation, a corresponding path weight is known [1].We now assume that an observer can only measure particular observables, which give rise to a coarse-grained trajectory, rather than accessing this underlying level of description directly.
The probability to observe a particular coarse-grained trajectory Γ is encoded in its path weight P[Γ] and uniquely determined by the microscopic path weight P[γ] and the coarse graining γ → Γ.Similarly, the probability to observe the corresponding time-reversed trajectory Γ is determined by γ.Thus, calculating P[ Γ] either requires knowing the microscopic time-reversal operation γ → γ or knowing how the coarse-grained observables behave under time reversal.Reversing the coarse-grained trajectory Γ generally not only means to read Γ backwards, but also to modify its observables.This is the case when measuring, e.g., momenta or transitions.Put informally, a physical time-reversal operation not only requires 'playing the movie backwards' but also 'knowing what the movie shows', i.e., understanding the physical meaning of the model and the observables.This knowledge about the observables determines their time-reversal.

C. Markovian events
We define Markovian events as particular observables whose detection implies conditional independence between past and future time-evolution of γ.Such an instantaneous event determines the underlying state of the system from a dynamical point of view.We formalize the defining property for a Markovian event I as for any trajectory γ − → I → γ + , which contains I between its past and future time-evolution γ − and γ + , respectively.Importantly, these events are Markovian on the microscopic level.Generally, the state of the system is fully described by including both a suitable observation I and the absolute time τ at which it occurs.Therefore, a Markovian event takes the form of a tuple I ≡ (I, τ ). ( Suitable observations I are, for example, registering a transition or the current state of the system in the case of master equation dynamics on a Markov network.We denote Markovian events by script letters I, J , ... to emphasize an explicit time dependence.Note that in stationary systems the information about the absolute time τ is not required, which makes I itself the Markovian event. A coarse-grained trajectory may contain additional data besides Markovian events.We summarize such observables under the (potentially multidimensional) symbol O k , which includes any additional observables between the Markovian events I k−1 and I k .Thus, we write a coarse-grained trajectory as where the waiting times t k between the Markovian events I k−1 and I k are explicitly highlighted.

D. Snippets
Given a coarse-grained trajectory (3), we define a snippet [22] Γ s as a section of Γ, which starts with a Markovian event I and ends with the subsequent Markovian event J Any coarse-grained trajectory Γ comprises several of these snippets.In particular, the path weight associated with the trajectory (3) factorizes into contributions from the individual snippets, where Γ s i denotes the snippet between the Markovian events I i−1 and I i .
In the case of stationary systems, an alternative notation for the path weight of a snippet (4) is given by which concisely presents all relevant information, since the absolute time τ is not required so that the Markovian events are just the observations, i.e., I = I and J = J.Additionally, it highlights the waiting time characteristics of the path weights P[Γ s |I].Although useful for explicit calculations, this notation quickly appears overloaded if the system evolves in time.Therefore, we exclusively use it for systems in a stationary state.

E. Entropy production
Markovian events and snippets are the fundamental building blocks that allow us to extend the concept of a fluctuating entropy production to the coarse-grained level.For a system with constant driving but not necessarily in a stationary state, we identify as the entropy production of a snippet Γ s .It generally depends on the initial and final events I and J as well as the duration t and the remaining observables O.
For this identification to be physically meaningful, we assume that the entropy production on the microscopic level ∆s[γ] is of the form that is further detailed below in Section II G.This assumption is justified for virtually all classes of systems that are described in the general setup and for which a physical entropy production is of interest, such as Markov networks or Langevin dynamics.For equation ( 8), we do not need to discern between descriptions based on observables that are even under time reversal, such as state-based descriptions or overdamped Langevin dynamics and descriptions based on observables that are odd under time reversal, such as transition-based descriptions or underdamped Langevin dynamics.In Section IV we provide further details regarding the peculiar odd observables, which provide further support for the identification (8).
The following properties embed this notion of entropy production along the coarse-grained trajectory (8) into the framework of stochastic thermodynamics.First, it is additive in the sense that the entropy production of a coarse-grained trajectory (3) is the sum of the entropy production of its snippets, where we use equations ( 6) and ( 8).Entropy production has to be assigned in a way to avoid overcounting at the start-and endpoint, respectively.In Equation ( 8), the entropy production of a snippet includes its initial Markovian event but not the concluding one, which becomes more apparent when rewriting the entropy production (8) as where the path weights exclude J and J respectively.The superscript cf indicates that the path weight P cf [Γ s |J ] is conditioned on the final event of the trajectory snippet rather than the initial one.Second, the coarse-grained entropy production ∆S and the microscopic entropy production ∆s are linked by the exact relation as we show in Appendix A. The conditional mean ⟨•|Γ⟩ denotes an average over all microscopic trajectories that are mapped to Γ under coarse graining.In particular, this implies ∆S[Γ] = 0 if the underlying system is in equilibrium, where ∆s[γ] = 0 holds for all γ.Furthermore, equality (11) implies the thermodynamic consistency condition for the coarse-grained entropy production.While the equality (11) might typically be of a more theoretical rather than practical significance due to the statistically demanding nature of the mean [3], the inequality (12) will prove to be a useful and versatile tool for thermodynamic inference.In particular, it enables us to estimate entropy production localized in space and time.As a side note, the inequality (12) directly implies the estimator for the mean total entropy production of the system, which recovers the result in [22] for the mean total entropy production rate following a different reasoning.Third, the coarse-grained entropy production ∆S[Γ] extends the notion of a fluctuating entropy production to the coarse-grained level.One of the major achievements of stochastic thermodynamics has been to identify entropy production as a fluctuating quantity along individual trajectories [45], which equation ( 8) generalizes beyond microscopic trajectories.Conceptually, it is a key novelty to have a coarse-grained entropy production, which is endowed with physical meaning beyond its expectation value.

F. Fluctuating entropy production on the coarse-grained level
We illustrate key aspects of the fluctuating entropy production using the concrete example shown in Fig- b) The solid lines show the total entropy production rate as a function of time along an individual trajectory for a sample of 5 trajectories on the microscopic and the coarse-grained level.The black dashed lines show the corresponding expectation values.c) The markers show the localized entropy production both on the microscopic and the coarse-grained level as defined in equations ( 15) and ( 16), respectively.The statistics is obtained by considering a long trajectory in which occasionally snippets occur that start and end with the transition K = (21), see Figure 2a).As shown by the inset, the entropy production rates σ K→ K (N ) and σ K→ K (N ) converge towards their expectation values for a sufficiently high number of occurrences.ure 2a).This microscopic system is a six-state Markov network, in which only transitions K = ( 12) and L = (34) as well as their time-reversed counterparts K = ( 21) and L = (43) can be observed.This gives rise to the coarse-grained description shown in Figure 2b).For the following examples we generate the coarse-grained trajectories by simulating the microscopic ones, to which the coarse-graining is applied subsequently.This procedure is in accordance with data obtained in a real-world scenario.
The previously exclusively microscopic concept of a fluctuating entropy production can now be applied to a coarse-grained description.We define the total entropy production rate up to time t as σ(t) ≡ ∆s(t)/t and σ(t) ≡ ∆S(t)/t (14) on the microscopic and the coarse-grained level, respectively.For some trajectories of the system from Figure 2a), the total entropy production and the total entropy production rate along individual fluctuating trajectories on both levels is shown in Figures 3a) and b), respectively.In the coarse-grained description, the rate of jumps is lower due to the lower rate of events that contribute to the entropy production.Its expectation values fulfill inequality (12).The rate of coarse-grained entropy production converges towards its expectation value for each trajectory Γ individually in the long time limit, similarly to its microscopic counterpart.The localized entropy production exhibits a similar fluctuating behaviour.We define the entropy production rate after N realizations {γ 1 , ..., γ N } of trajectories that all start in I and end in J as and similarly its coarse-grained analogue that is defined through the corresponding snippets {Γ s 1 , ..., Γ s N } as with the accumulated duration T (N ) of these trajectories.This entropy production rate fluctuates with each realization as shown for K → K in Figure 3c).On the microscopic level, these fluctuations are due to different paths that all start with K and end with K.Although these paths cannot be resolved on the coarsegrained level, different paths typically lead to a different waiting time between the initial and final event.Therefore, the fluctuating duration of the snippets leads to a fluctuating entropy production.The entropy production rate converges towards its expectation value for a sufficient number of realizations as shown by the inset in Figure 3c).

G. Assumption on the microscopic level
The entropy production (8) for the coarse-grained trajectory Γ can be derived from a similar form on the microscopic level.We assume that the entropy production associated with a microscopic trajectory γ that starts with γ 0 and ends with γ 1 in a possibly time-dependent but not time-dependently driven system like, e.g., a steady state or relaxation into it, is given by Here, γ 0 = γ 1 , i.e., the time-reversed trajectory γ is initialized by the time-reversed final event of the original trajectory γ 1 .If the process is not stationary, the probabilities P (γ 0 ) and P (γ 1 ) depend on the initial and final time of the process, respectively.The identification ( 17) is well-known to be correct for Markovian and overdamped Langevin dynamics as well as underdamped Langevin dynamics, as further detailed in Section IV A. Beyond these major system classes, ( 8) cannot be derived without further knowledge about the specific system and its energetics.However, we argue that even for such systems, equation ( 8) is a reasonable starting point if appropriate Markovian events can be identified, as the identification is physically correct for all verifiable classes of systems.Note that in the case of even obserables we recover due to γ 0 = γ 1 = γ 1 .We discuss the suggested relationship between (8) and the detailed fluctuation theorem (18) later in Section IV C.

III. INFERENCE OF HIDDEN DRIVING
In a coarse-grained setup, the hidden part of a system is often not in equilibrium.Instead, it might even contain non-equilibrium processes, which do not drive any transition between two Markovian events and involve no further observables.We refer to such processes as hidden driving, which arises if, e.g., the hidden part of a Markov network contains cycles with non-zero affinity.Using our framework, we are generally able to detect these hidden driven cycles and to provide a lower bound on the corresponding affinities.
In this section, we assume an underlying Markov network in a stationary state, which is necessary to have a well-defined notion of a cycle affinity of the form where the sum runs over all transitions (ij) that point towards the same direction within a cycle C.The rate with which a transition (ij) occurs if the system is in state i is denoted by k ij .For now, we consider snippets of the form Figure 4. a) Toy model for the experimental inference of the affinity in a chemically driven system.We imagine a system in a solution of fuel carrying molecules, similar to systems driven by the hydrolysis of ATP.The original system has four states, states 2 to 5, which are interconnected by the following four steps: The binding and unbinding of the molecule carrying the fuel, ( 23) and ( 45) respectively, the consumption of the fuel (52) and an intermediate step (34).For all of these steps, the reverse is also possible in accordance with the condition of thermodynamic consistency, which makes the transitions bidirectional.The green triangles represent an additional chemical species, which may bind to the system if it is not occupied by a fuel-carrying molecule, i.e., if the system is in state 2 or 5. Its binding prevents any further steps along the cycle leading to the two additional states 1 and 6.
b) The Network corresponding to the system shown in a).We assume that only states 1 and 6 can be observed.The rates used for the simulations are listed in Appendix C.
and disregard any non-Markovian observables.The use of additional information O is discussed later.We introduce the quantity which allows us to bound the largest affinity of all cycles C hidden between I and J via max as proved in Appendix B. This affinity bound is related to similar bounds obtained independently in a recent preprint, which are formulated for logarithmic ratios of propagators rather than waiting time distributions [46].We demonstrate a potential biochemical application of the bound (22), which illustrates a method to infer cycle affinities in systems that are not directly observable.As an example, we consider the toy model shown in Fig- ure 4. We assume that we cannot measure any of the states that contribute to the cycle driving the system out of equilibrium.This remains true even after introducing an additional chemical species that is specifically designed to bind to the system in the fashion described in Figure 4.However, this procedure allows to infer the chemical affinity of the driving cycle while remaining agnostic about states or transitions of the original system.
In Figure 5a) we present the results of the affinity estimation for the example network shown in Figure 4. We introduce the quantity which is sufficient to determine ∆a I→J from equation (21) through since the probabilities P (I) and P (J) do not depend on the duration of the snippet t.In the present case with I = 1 and J = 6, the estimation yields ∆a 1→6 ≃ 3.3 compared to the real cycle affinity of A ≃ 4.8.Note that although this example relies on a unicyclic toy model, the same procedure can be applied to more complex systems, which then leads to a bound on the hidden cycle with the highest affinity.
It is also possible to make use of additional observables besides I, J and the time in between, but these have to satisfy an additional consistency condition.The proof of the bound (22) primarily relies on the construction of a suitable partial or mathematical time reversal R [38], which has to be indistinguishable from the true, physical time reversal for the given coarse graining.This condition has to be met also after introducing additional observables.In our case, the involution R acts like the true time-reversal operation on some parts of the trajectory while leaving other parts unchanged as further detailed in Appendix B. Thus, any additional observables need to be even under time reversal.In particular, we cannot use waiting times between events for data O in general, except for the duration of the snippet t.
An example for eligible further information is to include whether or not a compound state was visited during the snippet without including the time of the event.Including such additional information generally leads to an improvement of the bound (22) and allows us to further localize the hidden driving.We illustrate this procedure on the example network shown in Figure 2c) where we observe the states 1 and 4 as well as the lumped state H.If we discard the information concerning H, we obtain an estimate for the affinity in a similar way as for the model illustrated in Figure 4, as shown by the red line in Figure 5b).When additionally observing H, we are able to extract two bounds.The first one applies to all cycles between 1 and 4 that do contain H, shown by the green line, whereas the second one applies to all cycles between 1 and 4 that do not contain H, respectively.We are able to correctly identify that there is no hidden driving outside of H and its connecting edges while also improving on the affinity estimator that does not discern whether H has been visited or not.4b) leading to ∆a1→6 ≃ 3.3.b) Affinity estimation for the network shown in Figure 2c).The blue line shows a(t) ≃ −0.51 independent of t for trajectories 1 → 4 that do not pass H.The green curve shows a(t) for trajectories 1 → 4 that pass H leading to ∆a ≃ 3.4.For the red curve, the information about visiting H is discarded leading to ∆a ≃ 0.9.
Finally we mention a case in which equation ( 21) fails to detect hidden driving, i.e., ∆a = 0 despite A ̸ = 0.If the driven cycle is connected to the remaining network only through a single state as shown in the example of Figure 2d), then its affinity does not result in a timedependent a(t).Put more generally, if a cycle can be located inside a compound state with direction-time independence [47,48] for all transitions to and from it, we are not able to infer its irreversibility from observables located entirely outside the compound state.

IV. INTRICACIES OF ODD VARIABLES
The identification of entropy production is particularly subtle in the case of odd observables since the observations I and I denote genuinely different objects.In the following, we examine two model classes based on such odd observables in detail.
In the following, we show and illustrate with some simple and well-understood examples that our approach is consistent with requirements for a physically meaningful entropy production in underdamped Langevin systems.
Consider an underdamped Brownian particle of mass M on a one-dimensional ring subject to a potential V and additionally to a non-conservative external force Figure 6.Sketches of underdamped trajectories in phase space.a) A trajectory Γa that starts and ends in I and completes the circle exactly once (green line) and its time reverse (red line).b) A trajectory Γ b for which the velocity decreases linearly in time until the particle returns to its starting point (green curve).Its time reverse is identical to the forward trajectory (red curve) .
in a non-equilibrium steady state.It is described by the Langevin equation in which ξ models uncorrelated Gaussian white noise In this set-up γ and T denote the friction constant, which is not to be confused with the microscopic trajectories, and the temperature in units of energy, i.e., k B = 1, respectively.
Its entropy production has two contributions, one due to the stochastic entropy production ∆S st and one associated with dissipation into the medium ∆S m .First, we examine the entropy production along a trajectory Γ a that starts with I = (x 0 , v 0 ) at time T = 0 and ends with J = I after completing the cycle of length L at time T exactly once, as shown in Figure 6a) for x 0 = −L/2 = L/2.The stochastic contribution has to vanish since the initial and final states are identical.Therefore, we expect the total entropy production to be the cycle affinity A = f L. Indeed, this is consistent with equation (8), which here becomes ∆S = ln P (I) with ∆S st = 0 and ∆S m = T 0 f ẋt .= f L. In contrast, the tentative identification for which st would obey a detailed fluctuation theorem of the form (18), leads to ∆S (i) st ̸ = 0 without a clear physical interpretation.First, this identification ∆S (i) st would not warrant the property of the stochastic entropy to purely depend on the state of the system.Second, the entropy production would not be additive.The sum of the entropy productions of two trajectories of the form shown in Figure 6a) would not be the same as the entropy production of the corresponding joint trajectory that completes the cycle twice.
A second identification of ∆S st that reproduces the correct entropy production for the particular trajectory Γ a would be ∆S (ii) st = ln(P ( I)/P ( J)).
However, we can eliminate this possibility by considering a trajectory Γ b from I = (x 0 , v 0 ) to J = (x 0 , −v 0 ) where the velocity initially is parallel to the non-conservative driving force and then linearly decreases with time as shown in Figure 6b).This trajectory is identical to its time reverse and, therefore, satisfies P[Γ|I] = P[ Γ| J] by construction.Due to the negative velocity, the final state of this trajectory is less probable then the initial one, which implies that the stochastic entropy should increase, as is indeed the case for ∆S st = ln P (I) P (J) = ln P (I) while the alternative ∆S st gives the wrong sign.Finally, note that − ln(P ( I)/P ( J)) can be identified as the stochastic entropy production of the reverse trajectory in agreement with our formalism by applying equation (31) to Γ = J → I.

B. Observed transitions
Transitions on a Markov network form a second class of systems with a description based on odd observables.We assume a system in a stationary state with only a few observable edges similar to the settings described in [23,38].In this case, we can show that equation ( 8) is the correct identification of the entropy production through explicit calculations.We consider a coarse-grained trajectory Γ = I → J that starts with transition I and ends with transition J.The corresponding microscopic trajectories share the form where i, j and k, l are the Markov states associated with the transitions I and J, respectively.These occur with rates where p i denotes the probability to find the system in microstate i.The remaining path weights read and The term P[j → k|j] denotes the path weight for the trajectory starting immediately after entry into j until immediately before exiting k, given that the system starts in j.Inserting equations ( 33), ( 34) and ( 35) into the coarse-grained entropy production (8) yields where we identify ∆S[Γ] as the sum of the entropy production of the initial event I and the entropy production of the remaining part of the trajectory until immediately before the concluding event J.The final event is not included in the entropy production ∆S[Γ] in accordance with Section II E. By analogy to the cases of Markov networks and underdamped Langevin dynamics, equation ( 36) suggests an identification of stochastic and medium entropy production in the form ∆S = ln P (I) P (J) + ln This identification of ∆S st and ∆S m must be clearly distinguished from the identification ∆s = ∆s st + ∆s m that can be made on the microscopic level for the Markov network [45].We emphasize that the two identifications do not coincide even if the coarse graining retains the full entropy production, i.e., even if ⟨∆s⟩ = ⟨∆S⟩, which is the case for, e.g., unicyclic Markov networks with a single observed transition [38].In such a scenario, we have two physically sensible splittings into stochastic and medium entropy production.Thus, we interpret that the conception of 'system' and 'medium' can depend on the available information even if both descriptions apply to the same physical process on the microscopic level.For example, while stochastic entropy on the microscopic level of the Markov network may change whenever the actual state i of the system changes, a corresponding quantity on the coarse-grained level updates when an observed transition I is registered.

C. Discussion
The above analysis of odd variables shows that even when considering stationary situations only, entropy production, in general, does not obey the detailed fluctuation theorem that applies to a Markovian dynamics [45].Even if such an underlying Markovian description exists and equation ( 8) can be applied, we cannot expect a detailed fluctuation theorem of the same functional form as on the microscopic level, because the coarse-grained level involves different observables.However, validity is granted if all observables are even under time reversal, as seen by plugging P ( J ) = P (J ) into equation (8).
Thus, a refined understanding of the detailed fluctuation theorem may be as a symmmetry associated with the special case of a description based on even observables in a Markovian dynamics rather than a general property of entropy production itself.
For underdamped Langevin dynamics in particular, entropy production is often expressed using some 'modified' path weight for the reverse process [50,51,53], which leads to a formulation that appears similar to the detailed fluctuation theorem.However, this modified path weight and, therefore, the entire approach lack a clear physical interpretation.In light of the above insights regarding the detailed fluctuation theorem, it seems neither useful nor instructive to employ such a 'modified' path weight.
An immediate consequence is that an expression of the form ln (P[γ]/P[ γ]) or ln(P[Γ]/P[ Γ]) cannot be regarded as a guiding principle for the identification of entropy production in arbitrary systems.For any system that on the underlying level fulfills the conditions described in Section II G, one should resort to the identification in equation (8).Beyond these, the identification of an entropy production requires knowledge of the underlying energetics and, additionally, considerations similar to the ones in Section IV A. The analysis of scenarios for which we understand the corresponding entropy production physically, such as in equilibrium or for simple trajectories, gives rise to consistency conditions that entropy production has to fulfill.

V. CONCLUDING PERSPECTIVE
In this work, we have established requirements to identify a fluctuating entropy production on a coarse-grained level.This concept offers practical advantages in resolving sources of irreversibility.Depending on the information available, we can quantify the contribution of particular snippets of a long trajectory to the total entropy production and localize hidden driving with the aid of bounds on their affinity.As our identification is modelfree, we put particular emphasis on the subtle case of odd variables.
Our framework does not rely on distinguishing slow and fast degrees of freedom.Such knowledge typically is hard to come by, particularly in the realistic case where some degrees of freedom are hidden.Thus, it is not necessary to identify a 'correct fundamental layer', i.e., a system comprising all slow-moving degrees of freedom which is surrounded by an environment that equilibrates on a faster timescale.Instead, the hidden parts of the system and the surrounding system are treated in the same way without making particular distinctions beforehand.
Finally, we point out that, in the general case, one should not expect to identify a physically meaningful entropy production on the coarse-grained level solely based on our approach in the absence of Markovian events, since our formalism necessarily requires a coarse-grained tra-jectory to start and end with a Markovian event.Applying equation (8) to arbitrary sections of a coarse-grained trajectory would generally imply a violation of the properties mentioned in Section II E, which would undermine the corresponding physical interpretation.In particular, the correspondence between microscopic and coarsegrained entropy production relies on the defining property of Markovian events.
The results of this paper allow for future research beyond the insights already demonstrated.As knowledge about the dynamics of the hidden parts of the system is not required, the improved flexibility of our framework should prove useful for applications across different model classes.Nevertheless, incorporating additional knowledge remains possible, as demonstrated for the estimation of cycle affinities when supposing an underlying network of Markov states.In a realistic biochemical setup, cycle affinities might be constrained to integer multiples of, say, the free energy released in hydrolysis of one ATP.Thus, if some microscopic states or transitions can be observed directly, the established affinity bounds that are able to distinguish different pathways provide a qualitative tool to localize the ATP-driven cycles in the biochemical network.
We expect that the concept of spatially and temporally localized entropy estimation can be combined successfully with other related techniques.Inferring entropy production through waiting times in snippets provides a physical interpretation for the waiting time distributions that have received attention in stochastic thermodynamics in different contexts.For example, recently discovered thermodynamic bounds on waiting time distributions and correlation functions [30,31,[33][34][35]] might be combined with the notion of fluctuating entropy production to infer not only averages but full distributions of thermodynamic quantities.
In addition, our framework applies to systems in which the state of the system is known at some times, but the full dynamics remains inaccessible or unknown, which generalizes the paradigmatic cases of fully accessible Markov networks and overdamped Langevin equations.With this new perspective, we might speculate whether other concepts of stochastic thermodynamics beyond total entropy production like an identification of intrinsic and medium entropy production or stochastic energetics can also be extended to genuinely coarse-grained descriptions.
For a trajectory γ that starts with the Markovian event I and ends with the Markovian event J, we define Since the initial and final events are fixed, the probabilities P (I) and P (J) are identical for all these trajectories, so that the duration of the snippet becomes the only remaining variable, which finally results in To extract a physically meaningful bound from the inequality (B9), we need to explicitly construct R, where we follow an algorithm similar to [38].Consider the microscopic trajectory on the Markov network γ = I → k → • • • → l → J = Ik • • • lJ, where I and J are the Markovian events, in this case states or transitions.To contruct Rγ, we perform the following steps: 1) Separate the trajectory into closed loops and the remaining parts.Starting after I and stopping before J, if a state occurs more than once, we identify a closed loop as the intermediate section between the first and last appearance including the states at the beginning and end of the loop, It is not necessary to identify loops within loops.
2) Reverse the order of sections, i.e., 3) Reverse the order of states within the sections that do not form a closed loop.These are the sections with differing initial and final states, i.e.,

Figure 1 .
Figure1.Projection of a continuous energy landscape onto a set of 5 discrete states.States 2 and 5 cannot be modeled as Markov states due to their shallow local minima.For transitions involving these states, there is no clear time-scale separation as illustrated by the time series in the lower part.The red lines show a section of an individual trajectory.We assign an entropy production to such sections that are both localized in time and space.This identification of entropy productions retains physical significance even on the coarsegrained level.

Figure 2 .
Figure2.Markov networks and their effective descriptions.a) Markov Network with six states.We assume that only the transitions K and L and their reversed K and L can be observed.b) Coarse-grained description of the network shown in a).c) Markov network with nine states.We assume that only states 1 and 4 and the compound state H, consisting of states 8 and 9, can be observed.d) Markov Network with five states.We assume that only states 1 and 2 can be observed.The cycle 3 → 4 → 5 → 3 is driven in clockwise direction.The rates used for numerical results are listed in Appendix C.

Figure 3 .
Figure3.Entropy production as a fluctuating quantity on the coarse-grained level.All data is generated using the network shown in Figure2a).Blue lines and markers indicate quantities on the microscopic level, whereas the coarse-grained level is denoted with red lines and markers.a) The solid lines show the change in total entropy as a function of time along an individual trajectory both on the microscopic and the coarse-grained level.The dashed lines show the corresponding expectation values.b) The solid lines show the total entropy production rate as a function of time along an individual trajectory for a sample of 5 trajectories on the microscopic and the coarse-grained level.The black dashed lines show the corresponding expectation values.c) The markers show the localized entropy production both on the microscopic and the coarse-grained level as defined in equations (15) and (16), respectively.The statistics is obtained by considering a long trajectory in which occasionally snippets occur that start and end with the transition K = (21), see Figure2a).As shown by the inset, the entropy production rates σ K→ K (N ) and σ K→ K (N ) converge towards their expectation values for a sufficiently high number of occurrences.

4 Figure 5
Figure 5. a) Affinity estimation for the network shown in Figure4b) leading to ∆a1→6 ≃ 3.3.b) Affinity estimation for the network shown in Figure2c).The blue line shows a(t) ≃ −0.51 independent of t for trajectories 1 → 4 that do not pass H.The green curve shows a(t) for trajectories 1 → 4 that pass H leading to ∆a ≃ 3.4.For the red curve, the information about visiting H is discarded leading to ∆a ≃ 0.9.