Theory of Spike Timing-Based Neural Classifiers

We study the computational capacity of a model neuron, the tempotron, which classifies sequences of spikes by linear-threshold operations. We use statistical mechanics and extreme value theory to derive the capacity of the system in random classification tasks. In contrast with its static analog, the perceptron, the tempotron’s solutions space consists of a large number of small clusters of weight vectors. The capacity of the system per synapse is finite in the large size limit and weakly diverges with the stimulus duration relative to the membrane and synaptic time constants.

Figure 1

(a) Example of voltage traces $U(t)$. (b) Probability density of the rescaled maximal potential $x$ as defined in Eq. (4) with a fitted scale factor $\beta$. (c) Probability of $N_{\mathrm{spikes}}$. The line in (b) is a standard Gumbel law. In (c) circles indicate the theoretical Poisson law. Data were measured with $K=400$, $\alpha =1.68$, $N=500$, and 34 samples.

Figure 2

(a) Capacity ${\alpha }_c$ of the tempotron vs $K$. Lines with symbols show results from the learning algorithm of 5. The solid line shows the large-$K$ theory (3), with an additive constant [${\alpha }_c=(\ln \ln K)/2\ln 2+{\alpha }_0$, with ${\alpha }_0=2.58$] fitted to the predictions of the replica method for the discrete tempotron for $K^{\mathrm{discrete}}=2$, 3, and 4 ($\times \mathrm{\text{symbols}}$, no line). To compare the theory of the discrete tempotron with the simulation results of the continuous time tempotron, we used $K^{\mathrm{discrete}}=K/8$. (b) Distribution of learning times for different ${\tau }_s$, and for fixed ${\tau }_m=T/25$, $\alpha =2.6$, and $N=1000$. As ${\tau }_s$ decreases, so does the mean learning time, indicating that $|\alpha -{\alpha }_c|$ has increased, as predicted by Eqs. (2, 3).

Figure 3

(a) Overlap between randomly chosen solutions, $q_0$, at $\alpha =2$ as a function of $K$. (b) Autocorrelation function $q_{\mathrm{AC}}(\Delta t)\equiv ⟨\widehat{\omega }(t)·\widehat{\omega }(t+\Delta t)⟩$ of a random walk inside a connected volume in solutions space for $K=150$ and $N=500$.

Figure 4

(a) Time traces of the potentials of a Tempotron with random weights, $U_1(t)$ (bold line), and of seven other Tempotrons, $U_2(t)$ (gray lines), having overlap $q=0.8$ with the first one. The pattern is the same for all Tempotrons, and is classified as $+1$ by the first Tempotron: $U_1(t)$ is maximal in $t_1$ and exceeds $U_{\mathrm{th}}$. The error bar is centered in $t=t_1$, ${\overline{U}}_2=q{U}_1(t_1)$, and has height $\sqrt{1-q^2}$. Parameters are $K=100$, $N=1000$. (b) Probability that two neurons will classify a random pattern in the same manner, $P_{\mathrm{equal}}$, vs overlap between their weight vectors, $q$, for the perceptron (theory and simulations in black), the tempotron (blue $\times$ and $+$ symbols correspond, respectively, to $K=100$ and $K^{\prime }=4K=400$) and the Hodgkin-Huxley (red squares and circles correspond to $T=1.5\sec$ and $T^{\prime }=4T=6\sec$, respectively [13]) models.