Effect of similarity between patterns in associative memory

We study the stability of patterns in Hopfield networks in which a part of memorized patterns are similar. The similarity between patterns impacts the stability of these patterns, but the stability of other independent patterns is only changed slightly. We show that the stability of patterns is affected in different ways by similarity. For networks storing a number of patterns, the similarity between patterns enhances the pattern stability. However, the stability of patterns can be weakened by the similarity when networks store fewer patterns, and the relation between the stability of patterns and similarity is nonmonotonic. We present a theoretical explanation of the effect of similarity on stability using signal-to-noise-ratio analysis.

Figure 1

The overlap between stable states and patterns versus the number of memorized patterns. (a) The first memorized pattern is used as the initial state. (b) In the initial state of networks, 10% of nodes are flipped from the first pattern. These results are averaged over 500 realizations.

Figure 2

The overlap between stable states and patterns versus the number of memorized patterns. (a) One of independent pattern is used as the network's initial state. (b) A spotted pattern which has 10% of nodes flipped from an independent pattern is used as the network's initial state. These results are averaged over 500 realizations.

Figure 3

Distribution of the noise term of nodes whose degree is $k$. (a) The stability of similar patterns is considered. (b) The stability of independent patterns is considered. The unstable probability $U_1\left(k\right)$ and $U_3\left(k\right)$ is illustrated by the area of the region under distribution curves and on the left of the vertical lines in (a) in (b). Solid lines and dashed lines represent the networks of $\eta =1.0$ and $\eta =0.5$, respectively. The number of patterns is $n=10$ and the node degree is $k=10$.

Figure 4

Theoretical (lines) and simulation (points) results of overlap versus the similarity $\eta$ for different number of patterns $n$. Data points are averaged over 500 realizations.

Figure 5

Panels (a) and (b) illustrate the error functions for networks storing $n=3$ and $n=10$ patterns, respectively. The similarity is $\eta =0.8$. Panels (c) and (d) are the variation of unstable probability contributed by the second pattern in networks storing $n=3$ and $n=10$ patterns, respectively. The node degree is $k=10$.

Figure 6

The overlap versus the similarity for networks storing different number of patterns. The scatter plots are the simulation results of overlap; the curves are the theoretical results of overlap. Data points are averaged over 500 network realizations. The average degree of networks is $\langle k\rangle =10$ and 50 in panels (a) and (b), respectively.