• Editors' Suggestion

Phase diagram of restricted Boltzmann machines and generalized Hopfield networks with arbitrary priors

Adriano Barra, Giuseppe Genovese, Peter Sollich, and Daniele Tantari
Phys. Rev. E 97, 022310 – Published 20 February 2018

Abstract

Restricted Boltzmann machines are described by the Gibbs measure of a bipartite spin glass, which in turn can be seen as a generalized Hopfield network. This equivalence allows us to characterize the state of these systems in terms of their retrieval capabilities, both at low and high load, of pure states. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e., weight) distribution and spin (i.e., unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region becomes larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case.

  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
2 More
  • Received 1 August 2017

DOI:https://doi.org/10.1103/PhysRevE.97.022310

©2018 American Physical Society

Physics Subject Headings (PhySH)

NetworksStatistical Physics & Thermodynamics

Authors & Affiliations

Adriano Barra1,*, Giuseppe Genovese2,†, Peter Sollich3,‡, and Daniele Tantari4,§

  • 1Dipartimento di Matematica e Fisica Ennio De Giorgi, Università del Salento, 73100 Lecce, Italy
  • 2Institut für Mathematik, Universität Zürich, CH-8057 Zürich, Switzerland
  • 3Department of Mathematics, King's College London, WC2R 2LS London, United Kingdom
  • 4Scuola Normale Superiore, Centro Ennio de Giorgi, Piazza dei Cavalieri 3, I-56100 Pisa, Italy

  • *adriano.barra@unisalento.it
  • giuseppe.genovese@math.uzh.ch
  • peter.sollich@kcl.ac.uk
  • §Present address: Scuola Normale Superiore, Piazza dei Cavalieri 7, 56126 Pisa, Italy; daniele.tantari@sns.it

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand
Issue

Vol. 97, Iss. 2 — February 2018

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×