Population equations for degree-heterogenous neural networks

We develop a statistical framework for studying recurrent networks with broad distributions of the number of synaptic links per neuron. We treat each group of neurons with equal input degree as one population and derive a system of equations determining the population-averaged firing rates. The derivation rests on an assumption of a large number of neurons and, additionally, an assumption of a large number of synapses per neuron. For the case of binary neurons, analytical solutions can be constructed, which correspond to steps in the activity versus degree space. We apply this theory to networks with degree-correlated topology and show that complex, multi-stable regimes can result for increasing correlations. Our work is motivated by the recent finding of subnetworks of highly active neurons and the fact that these neurons tend to be connected to each other with higher probability.

Figure 1

The joint distribution function $N(k,k^{\prime })$ for $k,k^{\prime }\in [100,240]$ for the maximal possible correlation strength parameter $\gamma$ (red) and the joint distribution function $N{(k,k^{\prime })}^{\mathrm{unc}}$ (blue) for the same degree range.

Figure 2

(a) The function $F\left(\kappa \right)$ vs. $\kappa$ for a network with a flat degree distribution with $100\le k\le 240$. The blue dashed line indicates an exemplary threshold, in this case $\mathrm{\Theta }=108$. Intersections of $F\left(\kappa \right)$ and this threshold give the solutions of the self consistent equation $0=F\left(\kappa \right)-\mathrm{\Theta }$, which are shown in (b) for a number of different thresholds.

Figure 3

The number of active neurons versus the number of iteration steps. Different lines correspond to simulations with different initial conditions. This image captures the convergence behavior of an uncorrelated network of $N=80000$ binary neurons with a threshold of $\mathrm{\Theta }=108$, whose degree is flatly distributed in a range of $k\in [100,240]$.

Figure 4

The time evolution of the population activity for the three exemplary initial conditions discussed in Fig. 3. Every line corresponds to the population activity after one subsequent iteration step starting with the initial conditions indicated by the vertical lines of the same color. The expected stable and unstable stationary solutions shown in Fig. 3 are displayed as solid and dashed black lines. Starting from the initial step function, the shape of the curve rapidly changes to a sigmoidal “front,” which retains its form while traveling along the $\kappa$ axis.

Figure 5

The figure above shows the simulation results for an uncorrelated network of $N=80000$ neurons with flat degree distribution. The simulations were carried out for threshold values $\mathrm{\Theta }\in [100,111]$. (a) The population activity $u_k$ against the population degree $k$. Every line corresponds to the population activity, after the dynamics died out within the network, i.e., further iterations did not lead to a changed state of any neuron. Lines from left to right correspond to simulations with increased threshold value. (b) The calculated steady-state solution degree ${\kappa }_{\infty }$ as function of the population degree ${\kappa }_0$, above and including which all neurons were initially active in the simulation. Therein, vertical lines from left to right correspond to simulations with decreasing threshold value. (c) The stable (red dots on the lower branch) and unstable (blue hollow dots on the upper branch of the curve) fixed points calculated from simulations (see text for details) as functions of the threshold $\mathrm{\Theta }$. The dashed line corresponds to solutions of the equation $F\left(\kappa \right)-\mathrm{\Theta }=0$; cf. Fig. 2.

Figure 6

The figure shows $F\left(\kappa \right)$ as function of the population degree $\kappa$ for three different values of the correlation strength parameter $\gamma$. The green line corresponds to the minimal possible value $\gamma ={\gamma }_{\min }\approx -0.60\times {10}^{-6}$, the blue line to the uncorrelated case $(\gamma =0)$, and the red line to the maximal possible value $\gamma ={\gamma }_{\max }\approx 1.45\times {10}^{-6}$. The inset shows a zoom of the same graph, where the dashed black line shows an exemplary integer value for the threshold, here $\mathrm{\Theta }=99$, that intersects three times with $F\left(\kappa \right)$ for $\gamma ={\gamma }_{\max }$.

Figure 7

Simulation results for correlated networks of different sizes at a threshold of $\mathrm{\Theta }=99$. A steady-state solution is shown as function of the correlation strength parameter. For this simulation, the initially active population degree was chosen to be ${\kappa }_0=150$.

Figure 8

The figure shows the solutions $\kappa$ of the equation $0=F(\kappa ,\gamma )-\mathrm{\Theta }$ within the maximally possible range of $\gamma$ for networks with flat degree distribution in the range $100\le k\le 240$. The color code simply indicates the height of the surface.

Figure 9

The figure above shows the number of active neurons as function of iteration steps for an uncorrelated network with different initial conditions, similar to Fig 3, but at a threshold of $\mathrm{\Theta }=112$.

Figure 10

The figure above shows the propagation of a population activity “front” from one exemplary iteration to the next. It can be seen, that a growing of $u_{\kappa }$ in this domain leads to a shift of the front to the left (lower values of $\kappa$), as indicated by the red arrow. The red shaded lines show best fit error functions for each “front.” Black arrows indicate the change in $u_{\kappa }$ for exemplary values of $\kappa$.

Figure 11

The figure above shows a summary of the two possible cases for the stability discussion. (a) $F\left(\kappa \right)-\mathrm{\Theta }$ for an uncorrelated network with flat degree distribution as discussed in Sec. 4a, with $\mathrm{\Theta }=108$, corresponding to Fig. 2. (b) The same for the maximally correlated case as discussed in Sec. 4b and for $\mathrm{\Theta }=99$, corresponding to the red line in Fig. 6. In both figures, the fixed points are defined by the intersection, and their stability by Eq. (40), i.e., the slope of $F\left(\kappa \right)$ at the intersection. Hence, the uncorrelated network with flat degree distribution gives rise to one stable and one unstable nontrivial fixed point. Degree correlations in networks with flat degree distribution can lead to a third nontrivial fixed point, which is here unstable.

Figure 12

The figure above shows the joint distribution function, calculated from a realization of a random graph as described in the text (in blue), compared to the joint distribution function calculated from Eq. (8) (in red). In that case, the degree distribution is given by a binomial distribution. It can be seen that the two surfaces differ significantly toward the maximum and minimum in-degree $k$.

Figure 13

(a) The active neurons, sorted by degree and (b) the corresponding population activity. The system was initialized slightly above (blue dashed lines) and slightly below (red lines) the critical population degree and then iterated, until the steady state was reached (cf. Fig. 4). (c) The stable (red dots) and unstable (blue hollow dots) solutions of the system, compared to the expected result.

Figure 14

The figure above shows the temporal evolution of the relative activity of recurrent neuronal networks of different sizes (microscopic simulation, black lines), compared to the evolution of the relative activity as expected from the population equation, Eq. (4) (macroscopic simulation, red line). The black dotted, dashed, solid lines correspond to network sizes of $N=14100$, $N=28200$, and $N=84600$, respectively, $100,200,600$ neurons per population. The inset shows the population activity $u_k$, as function of the degree $k$ with the same color code.