Thermodynamic Cost and Benefit of Memory

This Letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are allowed to use temperature differences. It is shown that the retention of irrelevant information limits efficiency. A data representation method is derived from optimizing a fundamental physical limit to information processing: minimizing the lower bound on dissipation leads to a compression method that maximally retains relevant, predictive, information. In that sense, predictive inference emerges as the strategy that least precludes energy efficiency.

Figure 1

Sketch of a partially observable Szilárd engine, run isothermally with a two-state memory (left panel), or with a three-state memory (right panel) using a temperature difference.