Dynamic Flux Tubes Form Reservoirs of Stability in Neuronal Circuits

Neurons in cerebral cortical circuits interact by sending and receiving electrical impulses called spikes. The ongoing spiking activity of cortical circuits is fundamental to many cognitive functions including sensory processing, working memory, and decision making. London et al. [Sensitivity to Perturbations In Vivo Implies High Noise and Suggests Rate Coding in Cortex, Nature (London) 466, 123 (2010).] recently argued that even a single additional spike can cause a cascade of extra spikes that rapidly decorrelate the microstate of the network. Here, we show theoretically in a minimal model of cortical neuronal circuits that single-spike perturbations trigger only a very weak rate response. Nevertheless, single-spike perturbations are found to rapidly decorrelate the microstate of the network, although the dynamics is stable with respect to small perturbations. The coexistence of stable and unstable dynamics results from a system of exponentially separating dynamic flux tubes around stable trajectories in the network’s phase space. The radius of these flux tubes appears to decrease algebraically with neuron number $N$ and connectivity $K$, which implies that the entropy of the circuit’s repertoire of state sequences scales as $N\ln (KN)$.

Popular Summary

Neurons in the brain communicate by exchanging tiny electrical impulses called spikes. Every second more than 100 billion spikes are generated in the human brain. It is often thought that each individual spike among this astronomical number plays only a negligible role in the greater picture of information processing. In this paper, however, we show, in an idealized model of cerebral circuits, that stopping a single spike from reaching its targets typically sets the entire neural network on a different dynamical path. We also present computational techniques to characterize this type of dynamical sensitivity and outline how it may enhance the computational capabilities of biological and artificial neural circuits.

In general, a conceptually clarifying way of describing the states and the temporal evolution of a system of nonlinear dynamics is through the phase-space perspective. Each state of the system is represented by a point in the phase space, and the temporal evolution of the system from any state onward draws out a trajectory in this space. How different trajectories are distributed in the phase space reveals telltale signs about the dynamics of the system, in particular, the dynamical sensitivity to small changes in the starting state. In the neuronal-circuit models we have investigated, we have uncovered that the high sensitivity to single spikes results from the occurrence of a previously unknown structure of organization of the phase space of the models, which we call exponentially separating dynamical flux tubes.

The nature of the high sensitivity that we have seen is novel and highly nontrivial. Classically, Lyapunov exponents, the rates of exponential state separation, measure the sensitive dependence on initial conditions. Positive Lyapunov exponents essentially define the notion of deterministic chaos, or high sensitivity to small perturbations. In the neuronal-circuit models studied in this paper, all of the Lyapunov exponents are negative, and the system is, in fact, formally stable and nonchaotic. The exponential state separation we have revealed originates from a fundamentally different mechanism: The phase space is organized into a complex landscape of stability basins of dynamical flux tubes, separated by exponentially separating borders. Single spike perturbations that can take the system across a border lead to the observed dynamical sensitivity. From this new understanding, we have also been able to obtain information on the neural-circuit models that were previously unavailable.

We are confident that not only will future studies building on this work lead to the discovery of ways to shape the landscape of flux tubes to enhance the power of network computations, but the new finding of dynamical flux tubes will also have considerable impact on nonlinear-dynamics research in general.

Figure 1

The balanced state in inhibitory LIF networks: (a) Asynchronous irregular spike pattern of 30 randomly chosen neurons. (b) Fluctuating voltage trace of one neuron (voltage increased to $V=2$ at spikes). (c) Broad distributions of individual neurons’ firing rates $\nu$ and coefficients of variation cv. (d) Network-averaged firing rate $\overline{\nu }$ and synchrony measure $\chi$ versus predicted rate ${\overline{\nu }}_{\mathrm{bal}}=I_0/(J_0{\tau }_m)$. The dotted line is a guide to the eye for $\overline{\nu }={\overline{\nu }}_{\mathrm{bal}}$, synchrony measure: $\chi =\frac{\mathrm{STD}([{\varphi }_i])}{[\mathrm{STD}({\varphi }_i)]}$, where $[·]$ denotes the population average and STD stands for standard deviation. The parameters are $N=10000$, $K=1000$, $\overline{\nu }=10\mathrm{Hz}$, $J_0=1$, ${\tau }_m=10\mathrm{ms}$.

Figure 2

Stable dynamics with respect to infinitesimal perturbations: (a) Spectrum of Lyapunov exponents $\{{\lambda }_i\}$ of networks with $N=10000$ neurons and different connectivities $K$. (b) Largest Lyapunov exponent ${\lambda }_{\max }={\lambda }_2$ and mean Lyapunov exponent ${\lambda }_{\mathrm{mean}}=\frac{1}{N}\sum _{i=1}^N{\lambda }_i$ versus connectivity $K$ and average firing rate $\overline{\nu }$ for $N=100000$. Dashed lines indicate results from random matrix theory (RMT) for ${\lambda }_{\mathrm{mean}}$ [49]. The parameters are as in Fig. 1. These are averages of ten initial conditions.

Figure 3

Weak firing-rate response after single-spike failures: (a) Sample spike pattern of 10 randomly chosen neurons (gray, reference trajectory; black, single spike skipped at $t=0$). (b) Network-averaged firing rate $\overline{\nu }$ of the reference trajectory and after skipped spike for different connectivities $K$ and network sizes $N$. (c) Number of extra spikes $S_{\mathrm{extra}}$ in the entire network versus time rescaled by input rate $K\overline{\nu }$. The parameters are as in Fig. 1. These are averages of 100 initial conditions with 10 000 perturbations each.

Figure 4

High sensitivity to single-spike failures: (a) Distance $D_{\varphi }(t)=\frac{1}{N}\sum _i|{\tilde{\varphi }}_i(t)-{\varphi }_i(t)|$ between trajectories after skipped spike and reference to log-lin plots for different connectivities $K$ and average firing rates $\overline{\nu }$. The small bars indicate the distances $D_{\varphi }^{\infty }=\iint |\varphi -\phi |P(\varphi )P(\phi )d\varphi d\phi$ of uncorrelated states obtained from self-consistent mean-field solutions $P(\varphi )$ [49, 50]. (b) Pseudo-Lyapunov-exponent ${\lambda }_p$ from exponential fits $D_{\varphi }\sim \exp ({\lambda }_{p}t)$ before reaching saturation. (c) Collapse to characteristic exponential state separation with a rate of ${\lambda }_p\sim K\overline{\nu }$. (Distance has been rescaled with approximate perturbation strength $K\frac{J_0}{\sqrt{K}}$; the time has been rescaled with input rate $K\overline{\nu }$. Inset: Different network sizes $N$ for $K=100$.) The parameters are as in Fig. 1 with $N=100000$. These are averages of ten initial conditions with 100 perturbations each.

Figure 5

Sensitivity to finite perturbations: (a) Distance $D_{\varphi }$ between perturbed and reference trajectories at spike times of the reference trajectory (projecting out possible time shifts) for perturbation strengths $\epsilon ={10}^{-5},{10}^{-3},{10}^{-1}$ in log-lin plots. The gray lines indicate ten random directions perpendicular to the trajectory; the color lines indicate averages of exponentially separating or converging cases. (b) Probability $P_s$ of the exponential state separation versus perturbation strength $\epsilon$ in the lin-log plot. The dotted line indicates characteristic perturbation size ${\epsilon }_{\mathrm{FT}}$ separating stable from unstable dynamics; the shaded areas indicate single-synapse, single-spike failures. (c) Characteristic perturbation size ${\epsilon }_{\mathrm{FT}}$ versus network size $N$, connectivity $K$, and average firing rate $\overline{\nu }$ in log-log plots. The straight lines indicate fits to ${\epsilon }_{\mathrm{FT}}\sim 1/(\sqrt{NK}\overline{\nu })$. (d) Symbolic picture of dynamic flux tubes with radius ${\epsilon }_{\mathrm{FT}}$, the boxes indicate the different spike sequences assigned to different flux tubes. (Inside the flux tubes are stable dynamics, but adjacent flux tubes separate exponentially.) The parameters are as in Fig. 1 with $N=100000$. These are averages of ten initial conditions with 100 calculations and 100 random directions each.

Figure 6

Rapid decorrelation through a cascade of new spikes replacing those in the original spike sequence. (a) Five examples of typical distance measurements $D_{\varphi }(t)$ for finite perturbations $\epsilon =0.01$. (b) Histogram of distances of all measurements. (c) Step sizes $d_n$ obtained from histograms as in panel (b) for different $\epsilon$. The dashed line indicates distance due to one replaced spike $d=2J_0\sqrt{K}/N$. (d) Step sizes $d_n$ versus connectivity $K$ for finite perturbations (solid lines) and single-spike perturbations (dotted lines). (e) Histograms of the individual rate of exponential state separation $\lambda =\ln (n)/t_n$ for all measurements (dots) and fits to log-normal distributions (solid lines). (f) Expectation values of fitted log-normal distributions $\overline{\lambda }$ versus connectivity $K$ for finite perturbations (solid lines) and single-spike perturbations (dotted lines). The ${\lambda }_p$ (dashed lines) from Fig. 4 is plotted for comparison. The parameters are as in Fig. 1 with $N=100000$. These are averages of ten initial conditions with 100 calculations and 1000 random directions each.

Figure 7

Visualization of the phase-space structure. (a) Phase-space cross sections for $N=200$ and $N=2000$, spanned by two random $N$-dimensional vectors perpendicular to the trajectory. Flux-tube sections are drawn in one color and separated by solid lines. (b) Exponential distribution of radii of flux-tube cross sections for different parameter sets. The inset shows a comparison of average radius $r_{\mathrm{FT}}$ with ${\epsilon }_{\mathrm{FT}}\sim 1/(\sqrt{KN}\overline{\nu })$ obtained in Fig. 5c. Other parameters used are $K=100$, $\overline{\nu }=10\mathrm{Hz}$, $J_0=1$, ${\tau }_m=10\mathrm{ms}$.