Extended Temporal Association Memory by Modulations of Inhibitory Circuits

Hebbian learning of excitatory synapses plays a central role in storing activity patterns in associative memory models. Interstimulus Hebbian learning associates multiple items by converting temporal correlation to spatial correlation between attractors. Growing evidence suggests the importance of inhibitory plasticity in memory processing, but the consequence of such regulation in associative memory has not been understood. Noting that Hebbian learning of inhibitory synapses yields an anti-Hebbian effect, we show that the combination of Hebbian and anti-Hebbian learning can significantly increase the span of temporal association between correlated attractors as well as the sensitivity of these states to external input. Furthermore, these effects are regulated by changing the ratio of local and global recurrent inhibition after learning weights for excitation-inhibition balance. Our results suggest a nontrivial role of plasticity and modulation of inhibitory circuits in associative memory.

Figure 1

Extended interstimulus association in anti-Hebbian learning. Overlaps between a reference attractor (${\mu }_{\mathrm{init}}=11$) and memory patterns (a) and correlations between attractors (b) were calculated without the Monte Carlo approximation. Parameters are $P=21$, $p=0.5$. (c), (d) Overlaps and correlations were calculated with the Monte Carlo approximation for ${\mu }_{\mathrm{init}}=36$. Parameters are $P=71$, $p=0.5$.

Figure 2

Parameter dependence of the maximum overlaps (a) and $N_c$ (b) for unbiased stimulus patterns ($P=71$, $p=0.5$). Black lines and gray areas show the mean and standard deviation over five different samples, respectively. The gray dashed line in (b) indicates $N_c=5$ for comparison with the previous result [10]. (c), (d) Maximum overlaps and $N_c$ for biased patterns ($P=71$, $p=0.1$).

Figure 3

The parameter dependence of sensitivity to external inputs. (a) Overlaps between a reference attractor (${\mu }_{\mathrm{init}}=36$, ${\mu }_{\text{input}}=56$) and memory patterns. (b) The relationship between the value of $c$ and the threshold input strength for the attractor shift. The results are shown only for $b^{{\mu }_{\text{input}}}\le 0.3$. When the attractor did not shift in $b^{{\mu }_{\text{input}}}\le 0.3$, the threshold was plotted as $b^{{\mu }_{\text{input}}}=0.3$. We used $p=0.5$ for unbiased patterns, and $p=0.1$ for biased patterns, and the number of patterns is $P=71$. Black lines and gray areas show the mean and standard deviation, respectively, over five different samples.

Figure 4

Inhibitory regulations of correlated attractors. (a) A network model of excitatory neurons (orange) is regulated by local (L) and global (G) inhibitory neurons (blue). (b) The relaxation of synaptic weights of local and global inhibition during the initial adaptation period. We plotted the maximum weight changes every 5 ms $\{{\max }_{i,\mu }[w_{i\mu }^{\mathrm{L}}(t+5\mathrm{ms})-w_{i\mu }^{\mathrm{L}}(t)],{\max }_i[w_i^{\mathrm{G}}(t+5\mathrm{ms})-w_i^{\mathrm{G}}(t)]\}$, where the changes were normalized such that their peak is 1. The inhibition ratio was $a=0.9$. (c) The inhibition ratio was sinusoidally modulated during the test period (time $>100\mathrm{s}$). (d) Time evolution of overlaps between memory patterns and simulated neural activities is shown during the test period. Overlaps exceeding 1 or below 0 were truncated. We note that the overlap distributions are broadened and occasionally drifted at the peak times of the inhibition ratio (local-inhibition-dominant state). (e) Means (lines) and standard deviation (shaded areas) of overlap distributions were calculated for the epochs of low ($a<0.2$: green) and high ($a>0.8$: red) inhibition ratios. Memory patterns were renumbered such that memory pattern 0 takes the maximum value. (f) The same as (e), but after randomization of inhibitory synaptic weights.