Relevance in the Renormalization Group and in Information Theory

Amit Gordon, Aditya Banerjee, Maciej Koch-Janusz, and Zohar Ringel
Phys. Rev. Lett. 126, 240601 – Published 17 June 2021
PDFHTMLExport Citation

Abstract

The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is interpretability. It is often unclear what relation, if any, the architecture- and training-dependent learned “relevant” features bear to standard objects of physical theory. Here we report on theoretical results which may help to systematically address this issue: we establish equivalence between the field-theoretic relevance of the renormalization group, and an information-theoretic notion of relevance we define using the information bottleneck (IB) formalism of compression theory. We show analytically that for statistical physical systems described by a field theory the relevant degrees of freedom found using IB compression indeed correspond to operators with the lowest scaling dimensions. We confirm our field theoretic predictions numerically. We study dependence of the IB solutions on the physical symmetries of the data. Our findings provide a dictionary connecting two distinct theoretical toolboxes, and an example of constructively incorporating physical interpretability in applications of deep learning in physics.

  • Figure
  • Figure
  • Figure
  • Received 14 December 2020
  • Revised 10 March 2021
  • Accepted 28 April 2021

DOI:https://doi.org/10.1103/PhysRevLett.126.240601

© 2021 American Physical Society

Physics Subject Headings (PhySH)

Statistical Physics & ThermodynamicsInterdisciplinary PhysicsCondensed Matter, Materials & Applied Physics

Authors & Affiliations

Amit Gordon1, Aditya Banerjee1, Maciej Koch-Janusz2,3, and Zohar Ringel1

  • 1Racah Institute of Physics, The Hebrew University of Jerusalem, Jerusalem 9190401, Israel
  • 2Department of Physics, University of Zurich, 8057 Zurich, Switzerland
  • 3James Franck Institute, The University of Chicago, Chicago, Illinois 60637, USA

Article Text (Subscription Required)

Click to Expand

Supplemental Material (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand
Issue

Vol. 126, Iss. 24 — 18 June 2021

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review Letters

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×