Limits on inferring the past

Here we define and study the properties of retrodictive inference. We derive equations relating retrodiction entropy and thermodynamic entropy, and as a special case, show that under equilibrium conditions, the two are identical. We demonstrate relations involving the Kullback-Leibler divergence and retrodiction probability, and bound the time rate of change of retrodiction entropy. As a specific case, we invert various Langevin processes, inferring the initial condition of $N$ particles given their final positions at some later time. We evaluate the retrodiction entropy for Langevin dynamics exactly for special cases, and find that one's ability to infer the initial state of a system can exhibit two possible qualitative behaviors depending on the potential energy landscape, either decreasing indefinitely, or asymptotically approaching a fixed value. We also study how well we can retrodict points that evolve based on the logistic map. We find singular changes in the retrodictivity near bifurcations. Counterintuitively, the transition to chaos is accompanied by maximal retrodictability.

Figure 1

Retrodiction entropy generation. The lines for potential and the labels both go from top to bottom. Left: The retrodiction entropy $S_R$ of five particles in a convex, flat, and concave potential $U\left(\vec{x}\right)$ with a uniform prior. $S_R$ quantifies how poorly the initial state of the particles can be inferred, backwards in time. Free and trapped particles forget their origin monotonically, whereas particles dispersing in a concave potential remember their past no matter how much time passes. Right: An analogous plot, but with a Gaussian prior instead of a uniform prior. Free and trapped particles saturate to having maximum retrodiction entropy, whereas particles in a concave potential still remember their past, just as in the case of a uniform prior.

Figure 2

Role of convexity. The lines for potential and the labels both go from top to bottom, with the exception of the dashed line for $S_0$. Left: Prior entropy $S_0$, average thermodynamic entropy $\langle S_T\rangle$, and observational entropy $S_t$ for two particles in a harmonic trap, as derived in Appendix pp1. The retrodiction entropy is related to the other three, through $\langle S_R\rangle =\langle S_T\rangle -(S_t-S_0)$. The prior distribution is Gaussian with $\sigma =5$. Right: $S_R$ for the Ornstein-Uhlenbeck process at different times with a Gaussian prior, $\sigma =5$. For positive convexity, $S_R$ converges to $S_0$, for negative convexity, $S_R$ converges to some smaller value, meaning some information can still be recovered.

Figure 3

Retrodiction entropy, bifurcations, and chaos. The vertical dashed lines from left to right are (i) period doubling, (ii) period quadrupling, (iii) period $\times 8$, (iv) onset of chaos (red), and (v) the onset of one particular island of stability where chaos breaks off to periodic motion (green). Top: The bifurcation diagram for the logistic map, showing $X_t$ for multiple large $t$ values. Middle: Basins of attraction. The initial state $X_0$ determines the final state $X_t$, ($t=200$) within [0,1], which is mapped to a color gradient from dark red (0) to light yellow (1). The change in the number of basins can be clearly seen near the vertical lines. As the system transitions into chaos, nearby points start converging to distinct final points. The system becomes chaotic (at $r\simeq 3.58$) and then well mixed (at $r\simeq 3.68$. Bottom: The (normalized) retrodiction entropy vs logistic parameter $r$ is plotted at several different times, $t$. Lines from top to bottom are $t=500,t=50,t=25,t=15$. The $t=500$ line is an excellent approximation to the asymptotic limit of $\langle S_R\rangle$. Note that in the non-chaotic regime, the retrodiction entropy converges to flat steps, whereas in the chaotic regime, the retrodiction entropy converges to steps (with values equal to that in the nonchaotic regime) with occasional dips coinciding with islands of stability. Course graining was done with $b=500$ bins, with 10000 sample initial points per bin.