Information Processing and the Second Law of Thermodynamics: An Inclusive, Hamiltonian Approach

We obtain generalizations of the Kelvin-Planck, Clausius, and Carnot statements of the second law of thermodynamics for situations involving information processing. To this end, we consider an information reservoir (representing, e.g., a memory device) alongside the heat and work reservoirs that appear in traditional thermodynamic analyses. We derive our results within an inclusive framework in which all participating elements—the system or device of interest, together with the heat, work, and information reservoirs—are modeled explicitly by a time-independent, classical Hamiltonian. We place particular emphasis on the limits and assumptions under which cyclic motion of the device of interest emerges from its interactions with work, heat, and information reservoirs.

Popular Summary

That heat always flows from a hot place to a cold one is a manifestation of the second law of thermodynamics. It is such a part of our everyday experience that to propose the opposite would seem ridiculous. But, that was, in principle, what the Scottish physicist James Clerk Maxwell—one of the giants of physics—suggested in 1867 with his idea of Maxwell’s demon. Maxwell’s thesis was that the second law of thermodynamics has only a statistical certainty. In other words, with sufficiently detailed information about the microscopic motions of individual atoms and molecules, we might be able to separate the fast-moving (“hot”) ones from the slow-moving (“cold”) ones and induce the heat to flow from cold to hot, in apparent contradiction with the second law of thermodynamics. This apparent conundrum, which has caught the attention of many well-known physicists from Lord Kelvin onward, is typically interpreted as the result of an external observer tinkering with the actual microscopic dynamics of a thermodynamic system—the “demon” still alien to many.

In this paper, we take a different approach. To the traditional thermodynamic analyses that employ heat and work reservoirs, we add an information reservoir in place of an external observer. The information reservoir captures the physical consequences of Maxwell’s demon, or more precisely, its memory. We then use a single time-independent Hamiltonian to describe a collection of thermodynamic elements, including a device, one or more heat baths, a work reservoir, and an information reservoir (like, for instance, a stream of bits). As a main result, we generalize several classic formulations of the second law of thermodynamics, such as the Kelvin-Planck, Clausius, and Carnot statements, to situations involving information processing.

Our paper provides a general conceptual framework for further development and explorations in the cross-disciplinary field of thermodynamic theories of information processing.

Figure 1

Thermodynamic setup: A device exchanges heat with thermal reservoirs and work with a work source. The process is observed by a Maxwell demon.

Figure 2

Example of a work reservoir: The spring is attached to a piston with large mass $M$ and confining a rarefied gas in a cylinder. The gas is in thermal contact with a heat bath of temperature ${\beta }^{-1}$.

Figure 3

Illustration of the generalized Clausius statement [Eq. (49)]. Heat can flow from the cold to the hot reservoir if information is written to memory.