Scale-invariant feature extraction of neural network and renormalization group flow

Satoshi Iso, Shotaro Shiba, and Sumito Yokoo
Phys. Rev. E 97, 053304 – Published 8 May 2018

Abstract

Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value Tc=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards Tc and how the RBM learns to extract features of spin configurations.

  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
11 More
  • Received 13 February 2018

DOI:https://doi.org/10.1103/PhysRevE.97.053304

©2018 American Physical Society

Physics Subject Headings (PhySH)

Particles & FieldsStatistical Physics & ThermodynamicsNetworks

Authors & Affiliations

Satoshi Iso1,2, Shotaro Shiba1, and Sumito Yokoo1,2

  • 1Theory Center, High Energy Accelerator Research Organization (KEK), Tsukuba, Ibaraki 305-0801, Japan
  • 2Graduate University for Advanced Studies (SOKENDAI), Tsukuba, Ibaraki 305-0801, Japan

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand
Issue

Vol. 97, Iss. 5 — May 2018

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×