Maximum entropy principle analysis in network systems with short-time recordings

In many realistic systems, maximum entropy principle (MEP) analysis provides an effective characterization of the probability distribution of network states. However, to implement the MEP analysis, a sufficiently long-time data recording in general is often required, e.g., hours of spiking recordings of neurons in neuronal networks. The issue of whether the MEP analysis can be successfully applied to network systems with data from short-time recordings has yet to be fully addressed. In this work, we investigate relationships underlying the probability distributions, moments, and effective interactions in the MEP analysis and then show that, with short-time recordings of network dynamics, the MEP analysis can be applied to reconstructing probability distributions of network states that is much more accurate than the one directly measured from the short-time recording. Using spike trains obtained from both Hodgkin-Huxley neuronal networks and electrophysiological experiments, we verify our results and demonstrate that MEP analysis provides a tool to investigate the neuronal population coding properties for short-time recordings.

Figure 1

Raster plots for the HH neuronal network in three different dynamical regimes. Raster plots of 10 randomly selected neurons for each case are shown. For (a), (b), and (c), consider the statistics of the interspike interval for each neuron; the Fano factor is $14.8\pm 2.9, 10.3\pm 2.3$, and $4.8\pm 1$, respectively, and the coefficient of variation is $0.63\pm 0.05, 0.58\pm 0.05$, and $0.47\pm 0.04$, respectively ($\mathrm{mean}\pm \mathrm{std}$ across all neurons). A short bar indicates that the neuron with certain index fires at certain time. The coupling strength is selected at random from the uniform distribution of the interval $[0,s]$, where $s=0.071{\mathrm{ms}}^{-1}$ (the corresponding physiological excitatory postsynaptic potential is $\sim 1\mathrm{mV}$). The Poisson input parameters for in (a), (b), and (c) are $(\mu =0.6{\mathrm{ms}}^{-1},f=0.05{\mathrm{ms}}^{-1}), (\mu =1.1{\mathrm{ms}}^{-1},f=0.04{\mathrm{ms}}^{-1})$ and $(\mu =2.5{\mathrm{ms}}^{-1},f=0.03{\mathrm{ms}}^{-1})$, respectively.

Figure 2

Measured probability distributions for short-time recordings compared with long-time recordings. The frequency of each firing state measured from the short-time recording of network dynamics ($120\mathrm{s}$) is plotted against the frequency measured from the long-time recording of network dynamics ($1.2\times {10}^5\mathrm{s}$). Data for these three cases are from three dynamical regimes shown (the same 10 selected neurons in each case) in Fig. 1, respectively.

Figure 3

Hierarchy of effective interactions in the MEP analysis in three dynamical regimes. In the first row, mean absolute values of effective interactions from the first order to the fourth order for three different dynamical regimes (shown in Fig. 1) are plotted, with the standard deviation indicated by the error bar. In the second row, the absolute value of the summation of all same-order effective interactions from the first order to the fourth order is plotted. Data for these three cases are collected from the three experiments (the same 10 selected neurons in each case) in Fig. 1, respectively, with a long recording time of $1.2\times {10}^5\mathrm{s}$.

Figure 4

First-order and second-order MEP distribution in comparison with the probability distribution of network states. In the first row, the frequency of each firing state from the distribution of the MEP analysis, $P_2$ (blue) and $P_1$ (green), is plotted against the frequency measured from the long recording ($1.2\times {10}^5\mathrm{s}$). Data for three different dynamical regimes (shown in Fig. 1) are collected from the three experiments (the same 10 selected neurons in each case) in Fig. 1, respectively. Here the MEP analysis is performed with a short-time recording ($1.2\times {10}^2\mathrm{s}$). In the second row, for each dynamical regime, each red circle is one trial of $D(P_{\mathrm{true}},P_2)$ against the short-recording length. The blue line is the mean of 10 trials for each recording length. The green circles and the cyan line are for the distribution directly measured from the short-time recording, i.e., $D(P_{\mathrm{true}},P_{\mathrm{short}})$. Some circles are missing because the KL divergence cannot be computed due to some firing patterns (zero probability in the measurement) not present in the short-time recording.

Figure 5

Electrophysiological verification for the second-order MEP analysis with short-time recordings. Spike trains of randomly selected eight neurons are recorded from V1 in anesthetized macaque monkeys (see Appendix pp6). (a) A short bar indicates that the neuron with certain index fires at certain time. The Fano factor and the coefficient of variation for the interspike interval is $6.1\pm 2.8$ and $1.3\pm 0.2$, respectively ($\mathrm{mean}\pm \mathrm{std}$ across all neurons). (b) Mean absolute values of effective interactions from the first order to the fourth order are plotted, with the standard deviation indicated by the error bar, computed by the long recording of $3824\mathrm{s}$. (c) The absolute value of the summation of all same-order effective interactions from the first order to the fourth order is plotted, computed by the long recording of $3824\mathrm{s}$. (d) The frequency of each firing state from the distribution measured in the short-time recording of $191.2\mathrm{s}$ (magenta) and the distribution of the second-order MEP analysis, $P_2$ (blue), is plotted against the frequency measured from the long recording of $3824\mathrm{s}$. (e) Each red circle is one trial of $D(P_{\mathrm{true}},P_2)$ against the short-recording length. The blue line is the mean of 10 trials for each recording length. The green circles and the cyan line are KL divergence for the distribution directly measured from the short-time recording, i.e., $D(P_{\mathrm{true}},P_{\mathrm{short}})$. Some circles are missing because the KL divergence cannot be computed due to some firing patterns (zero probability in the measurement) not present in the short-time recording. The magenta circles and the black line are approximate KL divergence for the distribution directly measured from the short-time recording, i.e., $D_{\mathrm{approx}}(P_{\mathrm{true}},P_{\mathrm{short}})$. (f) The plot is similar to (e), except that the abscissa is the number of considered neurons. In each trial, neurons are randomly selected and the recording length is $100\mathrm{s}$.

Figure 6

The strength of summation of effective interactions vs. the number of neurons. The plot is the absolute value of the summation of all same-order effective interactions computed from the electrophysiological data (the same procedure as that in Fig. 5) for the first (red), second (green), third (blue), and fourth order (black). For each considered neuron number $M$, we analyze 20 different subnetworks of randomly selected $M$ neurons from the recording data. Each star is the result of one subnetwork consisting of $M$ neurons with different colors for different-order effective interactions. The solid line corresponds to the mean value of summation from all 20 different subnetworks consisting of $M$ neurons.