Mean-field equations for neuronal networks with arbitrary degree distributions

The emergent dynamics in networks of recurrently coupled spiking neurons depends on the interplay between single-cell dynamics and network topology. Most theoretical studies on network dynamics have assumed simple topologies, such as connections that are made randomly and independently with a fixed probability (Erdös-Rényi network) (ER) or all-to-all connected networks. However, recent findings from slice experiments suggest that the actual patterns of connectivity between cortical neurons are more structured than in the ER random network. Here we explore how introducing additional higher-order statistical structure into the connectivity can affect the dynamics in neuronal networks. Specifically, we consider networks in which the number of presynaptic and postsynaptic contacts for each neuron, the degrees, are drawn from a joint degree distribution. We derive mean-field equations for a single population of homogeneous neurons and for a network of excitatory and inhibitory neurons, where the neurons can have arbitrary degree distributions. Through analysis of the mean-field equations and simulation of networks of integrate-and-fire neurons, we show that such networks have potentially much richer dynamics than an equivalent ER network. Finally, we relate the degree distributions to so-called cortical motifs.

Figure 1

The eight degree covariances in a network of excitatory and inhibitory neurons. (a) $\mathrm{cov}(y_{ee},x_{ee})$, (b) $\mathrm{cov}(y_{ee},x_{ei})$, (c) $\mathrm{cov}(y_{ei},x_{ii})$, (d) $\mathrm{cov}(y_{ei},x_{ei})$, (e) $\mathrm{cov}(y_{ie},x_{ee})$, (f) $\mathrm{cov}(y_{ie},x_{ei})$, (g) $\mathrm{cov}(y_{ii},x_{ii})$, (d) $\mathrm{cov}(y_{ii},x_{ie})$. In our analysis, we include the effect of the covariance $\mathrm{cov}(y_{ab},x_{bc})$ in the parameter ${\beta }_{abc}$; see Eq. (14).

Figure 2

A situation conducive to working memory states, even in the absence of recurrent excitation. (a) E cells tend to receive inhibition from neurons which are strongly inhibited, i.e., $\mathrm{cov}(y_{ei},x_{ii})$ is large. (b) E cells tend to receive inhibition from neurons which are weakly excited, i.e., $\mathrm{cov}(y_{ei},x_{ie})$ is small. (c) I cells tend to receive inhibition from neurons which are weakly inhibited, i.e., $\mathrm{cov}(y_{ii},x_{ii})$ is small. (d) I cells rend to receive inhibition from cells which are strongly excited, i.e., $\mathrm{cov}(y_{ii},x_{ie})$ is large. This situation is consistent with the condition ${\beta }_{iie}{\beta }_{eii}>{\beta }_{eie}{\beta }_{iii}$, which is necessary to satisfy Eq. (15).

Figure 3

An example of bistability without recurrent excitation. Shown is a bifurcation diagram of fixed-point solutions to Eqs. (16) for both a case without bistability (black lines) and a case with bistability (colored lines). Solid lines correspond to stable equilibria and dashed lines to unstable equilibria. (a) Black line $S_{ie}$ for ${\alpha }_{iie}=0$. Thin red line $S_{ie}$ for ${\alpha }_{iie}=1$. (b) Black line $S_{ei}$ and $S_{ii}$ for ${\alpha }_{iie}=0$. (These synaptic drives are indistinguishable in this case.) Thin red (thick green) lines: $S_{ei}\left(S_{ii}\right)$ for ${\alpha }_{iie}=1$. The dotted vertical line indicates the value of $I_i$ for which the simulations in Fig. 4 were done. The bistable region is shaded. Parameters: ${\alpha }_{abc}=0$ for all $a,b,c\in \{e,i\}$ except where noted, $I_e=1$, ${\tau }_e={\tau }_i=1$, $J_{ee}=0$, $J_{ei}=1$, $J_{ie}=J_{ii}=2$, ${\mathrm{\Phi }}_{ee}\left(x\right)={\mathrm{\Phi }}_{ie}\left(x\right)=0,x^2$ or $2\sqrt{x-3/4}$ if $x\le 0$, $0<x<1$ and $x\ge 1$, respectively. ${\mathrm{\Phi }}_{ei}\left(x\right)={\mathrm{\Phi }}_{ii}\left(x\right)=0$ if $x<0$ and $x$ otherwise.

Figure 4

A sample simulation in the bistable regime. The value of $I_i=1.9$ (see vertical dotted line in Fig. 3). (a) Excitatory synaptic output to I cells. (b) Inhibitory synaptic output to E cells. (c) Inhibitory synaptic output to I cells. (d) External input to E cells (black) and I cells (red).

Figure 5

The inhibitory population degree distribution of the network underlying bistability induced by degree correlations. (a) Correlation between inhibitory in-degree and out-degree to E cells. (b) Correlation between excitatory in-degree and out-degree to E cells. (c) Correlation between inhibitory in-degree and out-degree to I cells. (d) Correlation between excitatory in-degree and out-degree to I cells. Since ${\alpha }_{eie}^{\text{chain}}$ is small, the E out-degree and E in-degree of the inhibitory population are highly anticorrelated ((b); cf. Fig. 2). Since ${\alpha }_{iie}^{\text{chain}}$ is large, the I out-degree and E in-degree of the inhibitory population are highly correlated ((d); cf. Fig 2). The I in-degree is uncorrelated with the out-degrees of the inhibitory population (a, c).

Figure 6

Integrate-and-fire network without recurrent excitatory coupling demonstrating bistability due to degree correlations. (a) Raster plot of 50 neurons from the excitatory population. (b) Average firing rate of excitatory population. Bottom: input to excitatory population $R{I}_e$. Integrate-and-fire parameters: $E_r=-65$ mV, $E_e=0$ mV, $E_i=-70$ mV, $v_{\text{th}}=-50$ mv, $v_{\text{reset}}=-60$ mV, ${\tau }_e=20$ ms, ${\tau }_i=10$ ms, ${\tau }_a=2$ ms, ${\tau }_g=10$ ms, ${\tau }_{\text{ref}}=1$ ms, $J_{ei}=0.2$, $J_{ie}=0.1$, $J_{ii}=0.013$, and $R{I}_i=25$ mV. Network parameters: $p_{ee}=0$, $p_{ei}=p_{ie}=p_{ii}=0.1$, ${\alpha }_{ei}^{\text{recip}}={\alpha }_{ii}^{\text{recip}}=0.0$, ${\alpha }_{e,ii}^{\text{conv}}=0.3$, ${\alpha }_{ii,e}^{\text{div}}=0.3$, ${\alpha }_{eie}^{\text{chain}}=0$, ${\alpha }_{i,ee}^{\text{conv}}=1$, ${\alpha }_{i,ii}^{\text{conv}}=0.3$, ${\alpha }_{ee,i}^{\text{div}}=1$, ${\alpha }_{ii,i}^{\text{div}}=1$, ${\alpha }_{i,ei}^{\text{conv}}=0$, ${\alpha }_{ei,i}^{\text{div}}=-0.9{\alpha }_{eie}^{\text{chain}}=-0.9$, ${\alpha }_{iii}^{\text{chain}}=0$, ${\alpha }_{iie}^{\text{chain}}=0.9$, ${\alpha }_{eii}^{\text{chain}}=0$.

Figure 7

Average firing rates of inhibitory neurons illustrated as a function of their degree. (a) During the low-firing-rate mode (first or last second of simulation), the firing rates of individual inhibitory neurons are not strongly correlated with their degree. (b) During the high-firing-rate mode (middle period of simulation), only the inhibitory neurons with a high E in-degree and a low E out-degree are firing rapidly.

Figure 8

The probability of a chain motifs depends on the degrees. In this example there is a chain from excitatory neuron $i$ to $j$ to $k$. This particular motif depends on the four degrees $d_i^{\mathrm{out}}$, $d_j^{\mathrm{in}}$, $d_j^{\mathrm{out}}$, and $d_k^{\mathrm{in}}$.

Figure 9

Illustration of the effective mean-field network obtained from an EI network with arbitrary degree distributions. (a) For an Erdös-Rényi random network, one can obtain two-dimensional mean-field dynamics of the firing rates $R_e$ and $R_i$ of the excitatory and inhibitory population, coupled by an average connectivity $J_{ab}$. (b) In general, the presence of correlations between neurons' in- and out-degrees creates four-dimensional mean-field dynamics. One can represent these equations as a network with the four synaptic drives $S_{ee}$, $S_{ie}$, $S_{ei}$, and $S_{ii}$ as the nodes. The coupling among these nodes is determined not just by the average connectivity $J_{ab}$, but also the correlations ${\alpha }_{abc}$ among the in- and out-degrees, as captured in Eq. (16).

Figure 10

Illustration of degrees of excitatory and inhibitory neurons. To describe the degree of a single neuron requires four numbers: the number of connection to and from excitatory and inhibitory neurons. (a) The central excitatory neuron has three excitatory and two inhibitory incoming connections, giving it an E in-degree of $d_{ee}^{\text{in}}=3$ and an I in-degree of $d_{ei}^{\text{in}}=2$. The neuron has one connection onto an excitatory neuron and three connections onto inhibitory connections, giving it an E out-degree of $d_{ee}^{\text{out}}=1$ and an I out-degree of $d_{ie}^{\text{out}}=3$. (b) The central inhibitory neuron has an E in-degree of $d_{ie}^{\text{in}}=2$, an I in-degree of $d_{ii}^{\text{in}}=3$, an E out-degree of $d_{ei}^{\text{out}}=3$, and an I out-degree of $d_{ii}^{\text{out}}=2$.

Figure 11

The probability of a connection is determined by the out-degree of the presynaptic neuron and in-degree of the postsynaptic neuron. In this illustration, the presynaptic out-degree is ${\tilde{d}}^{\text{out}}=5$ and the postsynaptic in-degree is $d^{\text{in}}=4$, as represented by the arrows. The probability of a connection from the presynaptic neuron to the postsynaptic neuron is the probability that one of the right arrows is the same as one of left arrows, which is proportional to the product $d^{\text{in}}{\tilde{d}}^{\text{out}}$.

Figure 12

Illustration of the motifs underlying the SONET network model. (a) The first-order motif (a single edge represented by the parameter $p_{ab}$). (b) A convergent motif (${\alpha }_{a,bc}^{\text{conv}}$), (c) A reciprocal motif (${\alpha }_{ab}^{\text{recip}}$), (d) A divergent motif (${\alpha }_{ac,b}^{\text{div}}$), and (e) A chain motif (${\alpha }_{abc}^{\text{chain}}$). The relationship between the motif probabilities and the parameters is given in Eq. (A10). Gray shaded regions represent populations $a$, $b$, and $c$, some of which must indicate the same population, as in this model, we have two populations: $a,b,c\in \{e,i\}$.