Cusps enable line attractors for neural computation

Line attractors in neuronal networks have been suggested to be the basis of many brain functions, such as working memory, oculomotor control, head movement, locomotion, and sensory processing. In this paper, we make the connection between line attractors and pulse gating in feed-forward neuronal networks. In this context, because of their neutral stability along a one-dimensional manifold, line attractors are associated with a time-translational invariance that allows graded information to be propagated from one neuronal population to the next. To understand how pulse-gating manifests itself in a high-dimensional, nonlinear, feedforward integrate-and-fire network, we use a Fokker-Planck approach to analyze system dynamics. We make a connection between pulse-gated propagation in the Fokker-Planck and population-averaged mean-field (firing rate) models, and then identify an approximate line attractor in state space as the essential structure underlying graded information propagation. An analysis of the line attractor shows that it consists of three fixed points: a central saddle with an unstable manifold along the line and stable manifolds orthogonal to the line, which is surrounded on either side by stable fixed points. Along the manifold defined by the fixed points, slow dynamics give rise to a ghost. We show that this line attractor arises at a cusp catastrophe, where a fold bifurcation develops as a function of synaptic noise; and that the ghost dynamics near the fold of the cusp underly the robustness of the line attractor. Understanding the dynamical aspects of this cusp catastrophe allows us to show how line attractors can persist in biologically realistic neuronal networks and how the interplay of pulse gating, synaptic coupling, and neuronal stochasticity can be used to enable attracting one-dimensional manifolds and, thus, dynamically control the processing of graded information.

Figure 1

Graded, pulse-gated propagation in a neural Fokker-Planck model. (a) Graded population density transfer. Population densities in upstream, ${\rho }_u(V,t)$, and downstream, ${\rho }_d(V,t)$, layers. These are two layers from a 19-layer feedforward network with $T=5$ ms, $S=2.83$, $I_{\mathrm{gate}}=14.2$, and ${\sigma }_0=4.5$. Pulses gating the upstream and downstream populations are depicted at the bottom of this panel and similarly for subsequent panels. Color bar applies to both population density plots. (b) Synaptic current, ${\overline{I}}_j^{\mathrm{ff}}$ (blue), and firing rate, $m_j\left(t\right)$ (magenta), in upstream and downstream layers. Inset shows a comparison of a mean-field firing rate model (dashed red) with the population-averaged Fokker-Planck model (blue). Below the plot, we show the timing of the gating pulses. (c) Graded firing rates showing faithful, pulse-gated propagation across 19 layers. The sets of successive blue traces (six amplitudes from bottom to top) are currents from each population. Gradedness is shown by the fact that the pulses retain their height (roughly) and order as they propagate from population to population. Gating pulses for the solutions depicted are shown at the bottom of each panel.

Figure 2

Current transfer in the Fokker-Planck model. (a) Downstream firing rates plotted for given upstream synaptic input currents. (b) Downstream currents plotted for given upstream synaptic input currents. Note that the peak of the current at the end of the gating pulse, indicated by the vertical red line, is very close to the value of the synaptic input current. For (a) and (b), ${\overline{I}}^g=22$, $T=5$ ms, $S=2.98$, and $\sigma =0.6$. (c) Current transfer functions, $I_u\left(I_d\right)$, for three values of the gating current, ${\overline{I}}^g$. Here $S=2.98$. Note that changing the gating current moves the input-output function vertically upwards. (d) Current transfer functions for three values of the synaptic coupling, $S$. Here ${\overline{I}}^g=22$. Note that changing the synaptic coupling changes the slope of the input-output function. (e) An optimal current transfer function, with $S=2.98$ and ${\overline{I}}^g=22$. Note that this transfer function closely approaches the diagonal over a large range of inputs. (f) The distance of the current transfer function from the diagonal. The broad, roughly linear, range of gating currents, ${\overline{I}}^g$, and synaptic couplings, $S$, for which the distance is small indicates a large region of parameter space over which graded information may be faithfully propagated.

Figure 3

The slow manifold of graded propagation. (a) Trajectories $(m_j\left(jT\right),M_j^1\left(jT\right),M_j^2\left(jT\right))$ for successive values of $j$, showing the rapid approach of the initial distribution to a one-dimensional manifold and the slow evolution along the manifold representing the slow decay of the waveform. The trajectories are plotted as red lines with blue arrows at the tip. Small blue tips along the one-dimensional manifold show the slow evolution. (b) The trajectories in (a) projected onto the $M^1\text{−}m$ plane. In this panel, the black dots indicate the three fixed points on the one-dimensional manifold (outer fixed points stable, middle fixed point unstable). (c) The trajectories in (a) projected onto the $M^1\text{−}{M}^2$ plane. Parameters are as in Fig. 1.

Figure 4

Bifurcation diagram generated using numerically integrated Fokker-Planck equations (2), for $I=14.2$. The saddle-node bifurcation occurs at approximately $S=2.82$. Blue (dark gray) indicates unstable fixed points, and red (light gray) indicates stable fixed points. For values close to $S=2.82$ the trajectory evolves slowly along the one-dimensional manifold due to a fold bifurcation ghost, allowing graded propagation across many layers before significant change in the waveform.

Figure 5

$\sigma$-induced fold bifurcation ghost at cusp catastrophe. (a) The probability density as a function of time and membrane potential from a numerical simulation. (b) The probability density as a function of time from the transient pulse approximation. (c) A plot of the surface of zeros of ${\overline{I}}^{\mathrm{ff}}(S,{\overline{I}}^{\mathrm{g}},A,\sigma )-A$ plotted for $(S,{\overline{I}}^{\mathrm{g}},A)$ with $\sigma =0.2$. These are fixed points of the map of the synaptic feed-forward current generated from Eqs. (5). Note the fold bifurcation across many values of ${\overline{I}}^{\mathrm{g}}$, similar to that found numerically in Fig. 3. (d)) A plot of the surface of zeros of ${\overline{I}}^{\mathrm{ff}}(S,{\overline{I}}^{\mathrm{g}},A,\sigma )-A$ plotted for $(S,A,\sigma )$ with ${\overline{I}}^{\mathrm{g}}=6.5$. Note the obvious cusp catastrophe that occurs along the $\sigma$ axis. (e) Zeros of ${\overline{I}}^{\mathrm{ff}}(S,{\overline{I}}^{\mathrm{g}},A,\sigma )-A$ for ${\overline{I}}^{\mathrm{g}}=6.5$ and $\sigma =0.2$ plotted on a colored background, where the color indicates the speed of departure toward stable fixed points (either upward above the unstable fixed point or downward below the unstable fixed point). For small $\sigma$, solutions rapidly approach the fixed points, giving a propagated binary code for large $S$. (f) Zeros of ${\overline{I}}^{\mathrm{ff}}(S,{\overline{I}}^{\mathrm{g}},A,\sigma )-A$ for ${\overline{I}}^{\mathrm{g}}=6.5$ and $\sigma =1.0$ again indicating the speed of departure toward stable fixed points. For $\sigma$ near the cusp, there is a large region within which propagation away from the unstable point is slow, called a ghost. Ghosts occur near the cusp locus producing a bundle of models in parameter space that allow graded propagation across many layers.