Entropy and information in neural spike trains: Progress on the sampling problem

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy-like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to synthetic data inspired by experiments, and to real experimental spike trains. The estimator performs admirably even very deep in the undersampled regime, where other techniques fail. This opens new possibilities for the information theoretic analysis of experiments, and may be of general interest as an example of learning from limited data.

Figure 1

Entropy estimation for a model problem. Notice that the estimator reaches the true value within the error bars as soon as $N^2\sim 2^S$, at which point coincidences start to occur with high probability. Slight overestimation for $N>{10}^3$ is expected (see text) since this distribution is atypical in ${\mathcal{P}}_{\mathrm{NSB}}$.

Figure 2

Data from a fly motion sensitive neuron in a natural stimulus setting. Top: a 500 ms section of a 10 s angular velocity trace that was repeated 196 times. Bottom: raster plot showing the response to 30 consecutive trials; each dot marks the occurrence of a spike.

Figure 3

Slice entropy vs sample size. Dashed line on both plots is drawn at the value of $S^{\mathrm{NSB}}{\mid }_{N=N_{\max }}$ to show that the estimator is stable within its error bars even for very low $N$. Triangle corresponds to the value of $S^{\mathrm{ML}}$ extrapolated to $N\to \infty$ from the four largest values of $N$. First and second panels show examples of word lengths for which $S_{\mathrm{ML}}$ can or cannot be reliably extrapolated. $S^{\mathrm{NSB}}$ is stable in both cases, shows no $N$ dependent drift, and agrees with $S^{\mathrm{ML}}$ where the latter is reliable.

Figure 4

Distribution of the normalized entropy error conditional on $S^{\mathrm{NSB}}\left(N_{\max }\right)$ for $N=75$ and $\tau =0.75$ ms. Darker patches correspond to higher probability. The band in the right part of the plot is the normal distribution around zero with the standard deviation of 1 (the standard deviation of plotted conditional distributions averaged over $S^{\mathrm{NSB}}$ is about 0.7, which indicates a non-Gaussian form of the posterior for small number of coincidences [29]). For values of $S^{\mathrm{NSB}}$ up to about 12 bits the estimator performs remarkably well. For yet larger entropies, where the number of coincidence is just a few, the discrete nature of the estimated values is evident, and this puts a bound on reliability of $S^{\mathrm{NSB}}$.