Maximally informative pairwise interactions in networks

Several types of biological networks have recently been shown to be accurately described by a maximum entropy model with pairwise interactions, also known as the Ising model. Here we present an approach for finding the optimal mappings between input signals and network states that allow the network to convey the maximal information about input signals drawn from a given distribution. This mapping also produces a set of linear equations for calculating the optimal Ising-model coupling constants, as well as geometric properties that indicate the applicability of the pairwise Ising model. We show that the optimal pairwise interactions are on average zero for Gaussian and uniformly distributed inputs, whereas they are nonzero for inputs approximating those in natural environments. These nonzero network interactions are predicted to increase in strength as the noise in the response functions of each network node increases. This approach also suggests ways for how interactions with unmeasured parts of the network can be inferred from the parameters of response functions for the measured network nodes.

Figure 1

(Color online) Example analysis of firing rate for (a) a simple and (b) a complex cell in the cat primary visual cortex probed with natural stimuli from the data set [29]. Two relevant input dimensions were found for each neuron using the maximally informative dimensions method described in [30]. Color shows the firing rate as a function of input similarity to the first ($x$ axis) and second ($y$ axis) relevant input dimensions. The values on the $x$ and $y$ axes have been normalized to have zero mean and unit variance. Blue (dashed) lines show regions with signal-to-noise $\text{ratio}>2.0$. Red (solid) lines are drawn at half the maximum rate and represent estimates of the decision boundaries.

Figure 2

(Color online) Schematic of response probability and noise entropy. The response function in two dimensions (inset) is assumed to be deterministic everywhere except at the transition region which may curve in the input space. In a direction $x$, perpendicular to some point on the decision boundary, the response function is sigmoidal (blue, no shading) going from silent to spiking. The conditional response entropy (red, shading underneath) is $-P_{\text{spike}\mid x}{\text{log}}_2\left(P_{\text{spike}\mid x}\right)-\left(1-P_{\text{spike}\mid x}\right){\text{log}}_2\left(1-P_{\text{spike}\mid x}\right)$ and decays to zero at $x=\pm \infty$. The contribution to the total noise entropy due to this cross section, $\eta$, is the shaded area under the conditional response entropy curve.

Figure 3

(Color online) Network decision boundaries. Color in input space corresponds to the input distribution $P_{\mathcal{I}}$ and each line is a boundary for an individual neuron. (a) For a general intersection between neurons $i$ and $j$, the two boundaries divide the space into four regions corresponding to the possible network states (e.g., $G_{{\sigma }_i=\uparrow ,{\sigma }_j=\uparrow }$). Each segment has a width $\eta$ which determines the noise level and is described by a parameter ${\lambda }_{{\sigma }^{\prime }}^{\left(i\right)}$ [cf. Eq. (10)]. (b) In a uniform space, with two neurons (different colors/shading), the boundary segments satisfying the optimality condition are circular. Networks which have segments with different curvatures (dashed) are less informative than smooth circles (solid). (c) In a Gaussian space (in units of standard deviations), straight perpendicular lines (solid) provide more information about inputs than decision boundaries that intersect at any other angle (dashed). (d) For approximately natural inputs (plotted in units of standard deviations), the suboptimal balanced solutions (dashed) are two independent boundaries with the same $P$. The optimal solutions (solid) change their curvature at the intersection point and depend on the neuronal noise level $\eta$.

Figure 4

(Color online) Information from independent and optimally coupled network boundaries as a function of neuronal noise level. In both examples shown here, the independent boundaries (blue circles) lose information faster than the optimal boundaries (red triangles) as the noise level $\eta$ increases. Each curve represents two response probabilities (e.g., $P=0.45$ and 0.55) because information is invariant under switching the spiking or silent regions.

Figure 5

(Color online) Optimal coupling constants for naturalistic inputs. (a) The local fields show very little dependence on the noise level of the neurons, but (b) the magnitude of the interaction strength $J$ increases in the same fashion with the noise level regardless of the response probability $P$. (c) $h$ depends strongly on the response probability. (d) $J$ changes sign about $P=1/2$. Below this point, the coupling is excitatory and above, the coupling is inhibitory.