Information storage in neural networks with low levels of activity

Daniel J. Amit, Hanoch Gutfreund, and H. Sompolinsky
Phys. Rev. A 35, 2293 – Published 1 March 1987
PDFExport Citation

Abstract

The Hopfield model of a neural network is extended to allow for the storage and retrieval of biased patterns, {ξiμ}, where N1JiNξiμ=a is arbitrary. Such patterns represent levels of activity (i.e., percentage of firing neurons) equal to (1/2)(1+a), -1<a<1. If the coupling constants (synaptic efficacies) are constructed as in the original Hopfield model, the system can retrieve at most a very small number of patterns (p<1+a2). This is due to the finite correlations (overlaps) between the patterns. The model is modified by subtracting the bias a from each pattern as it enters into the couplings. This modification restores the ability of the model to store a macroscopic number of patterns. Yet spurious states are found to plague the dynamics. It is then argued that the dynamics of the network should be consistent with the levels of activity of the stored patterns. This is implemented by adding a global constraint, which restricts the configuration space to states whose mean activity is in the neighborhood of (1/2)(1+a). The consequences of the restricted dynamics are analyzed in the replica symmetric mean-field theory. The global constraint suppresses spurious states and leads to the unexpected result that the storage capacity is higher than that of the unbiased network, up to very high values of the bias (‖a‖≃0.99). However, the information content of such networks is shown to be a monotonically decreasing function of a.

  • Received 16 July 1986

DOI:https://doi.org/10.1103/PhysRevA.35.2293

©1987 American Physical Society

Authors & Affiliations

Daniel J. Amit, Hanoch Gutfreund, and H. Sompolinsky

  • Racah Institute of Physics, Hebrew University, Jerusalem, Israel

References (Subscription Required)

Click to Expand
Issue

Vol. 35, Iss. 5 — March 1987

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review A

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×