Dynamics of random neural networks with bistable units

We construct and analyze a rate-based neural network model in which self-interacting units represent clusters of neurons with strong local connectivity and random interunit connections reflect long-range interactions. When sufficiently strong, the self-interactions make the individual units bistable. Simulation results, mean-field calculations, and stability analysis reveal the different dynamic regimes of this network and identify the locations in parameter space of its phase transitions. We identify an interesting dynamical regime exhibiting transient but long-lived chaotic activity that combines features of chaotic and multiple fixed-point attractors.

Figure 1

Examples of network activity as a function of $s$ and $g$. Each inset shows $x\left(t\right)$ for 6 out of 400 network units as a function of time, with its location indicating the values $g$ and $s$ used: for insets 1–12 (in order): $(g,s)=$ (0.5,2.5), (1.3,2.5), (2.5,2.5), (0.6,1.5), (1.5,1.5), (2.5,1.5), (0.4,0.4), (1.5,0.5), (2.5,0.5), (0.4,−0.4), (1.2,$-0.3$), (2.5,$-0.5$). The long-dashed line is the boundary between activity that decays to 0 (inserts 7, 10, and 11) and persistent chaotic activity (inserts 8, 9, and 12). The solid curve is the boundary between persistent chaos and what we will show to be transient chaotic activity that ultimately converges to one of many nonzero fixed points (inserts 1–6). For inserts 3–6, there is a break in the time axis, reflecting the long time required for convergence to a fixed point. The short-dashed line simply indicates $s=1$.

Figure 2

Graphical solution of the static mean-field equation (6). Solutions $x\left(\eta \right)$ are points on the curve $x-stanh\left(x\right)$ corresponding to a particular value of $\eta$. The dashed line shows one such $\eta$ value and indicates that there are three possible solutions in the region $-{\eta }_{\mathrm{m}}\le \eta \le {\eta }_{\mathrm{m}}$ (dots). Then open circle indicates that solutions along the portion of the curve with negative slope correspond to unstable fixed points (see text). The arrows show $\pm x_{\mathrm{m}}$, the two local extrema of the function $x-stanh\left(x\right)$, and $\mp {\eta }_{\mathrm{m}}$ are its values at these points as indicated by the dotted lines. Along the transition line between the regions with stable fixed point solutions and persistent chaos (Fig. 1), the stable solution is restricted to the highlighted portion of the curve where $x\left(\eta \right)$ has the same sign as $\eta$.

Figure 3

The entropy as a function of $s$ and $g$, which is $1/N$ times the average (over $J$) of the logarithm of the number of stable fixed points in the network model. In a and b, the entropy goes to 0 at the values of $s$ and $g$ corresponding to the phase transition between the nonzero fixed-point and persistent chaotic regions in Fig. 1. (a) The entropy as a function of $s$ for, from top to bottom curves, $g=3,3.5$, and 4. (b) The entropy over a range of $s$ and $g$ values represented by colors. The green line shows where the entropy reaches 0. The white circles indicate the results obtained from the zero intercepts of the curves in (a). (c) The weighting function $m\left(\eta \right)$ near the transition (solid curve; $s=1.133$ and $g=3$) and away from the transition (dashed curve; $s=1.135$ and $g=3$).

Figure 4

Autocorrelation functions for different $s$ values and $g=1.5$. (a) $C\left(\tau \right)$ for, from the top to the bottom curve, $s$ ranging from 1.6 to $-0.4$ in steps of 0.2. (b) Peak heights of the curves in (a). (c) Widths at 1/2 peak for the curves in (a).

Figure 5

Exponential dependence on the network size $N$ of the lifetime of the transient activity in the region with stable nonzero fixed points. Inset: Distribution of times to reach a fixed point for $N=100$, $s=2.3$, and $g=1.3$, shown with bars. The curve is a fit to a log-normal distribution. Main figure: The average of the logarithm of the time to reach a fixed point plotted as a function of $N$, for different $g$ values. In all these examples, $s=2.3$. For $g=0$ the units are decoupled and the lifetime is independent of $N$. For $g>0$, the lifetime is exponential in $N$.

Figure 6

Maximum Lyapunov exponents as a function of $s$ for three different $g$ values. The vertical lines indicate where the transition from persistent chaos (to the left of these lines) to stable fixed points (to the right) occurs. The maximal Lyapunov exponents vary continuously and smoothly through this transition.

Figure 7

Normalized histograms of unit $x$ values for different $s$, with $g=3.5$ and $N=1000$. Run time was $20000$, and $x$ values were sampled at intervals of 50 to avoid temporal correlations. The width of the distributions increases with $s$, and bimodality first becomes apparent between $s=0.2$ and $s=0.3$ (not shown). The transition to the region of transient chaos occurs at $s=1.15$ for this value of $g$.

Figure 8

(a) Typical activity of one unit in the region where stable fixed points exist ($s=1.6$, $g=0.7$). ${\tau }_{\mathrm{in}\mathrm{state}}$ is defined as the time between flips of the unit between states that fluctuate near $tanh\left(x\right)=\pm 1$. (b) Distribution of ${\tau }_{\mathrm{in}\mathrm{state}}$ values extracted as in (a) from all the units. (c) The average ${\tau }_{\mathrm{in}\mathrm{state}}$ for different values of $g$ and $s$, note the logarithmic scale of the vertical axis. From top to bottom, the curves correspond to $s$ ranging from 2 to 1 in steps of 0.2.