Time’s Barbed Arrow: Irreversibility, Crypticity, and Stored Information

We show why the amount of information communicated between the past and future—the excess entropy—is not in general the amount of information stored in the present—the statistical complexity. This is a puzzle, and a long-standing one, since the former describes observed behavior, while optimal prediction requires the latter. We present a closed-form expression for the excess entropy in terms of optimal causal predictors and retrodictors—both $ϵ$ machines of computational mechanics. This leads us to two new system invariants: causal irreversibility—the asymmetry between the causal representations—and crypticity—the degree to which a process hides its state information.

Figure 1

$ϵ$-machine information diagram for stationary stochastic processes. A schematic, the diagram only shows the set-theoretic relationships.

Figure 2

Forward and reverse $ϵ$ machines for the RIP: (a) $M^{+}$ and (b) $M^{-}$. Edge labels $t|x$ give the transition probabilities $t=T_{\mathcal{S}{\mathcal{S}}^{\prime }}^{(x)}$. (c) The bidirectional machine $M^{\pm }$ for $p=q=1/2$. Edge labels are prefixed with the scan direction $\{-,+\}$.