Discrete Langevin machine: Bridging the gap between thermodynamic and neuromorphic systems

A formulation of Langevin dynamics for discrete systems is derived as a class of generic stochastic processes. The dynamics simplify for a two-state system and suggest a network architecture which is implemented by the Langevin machine. The Langevin machine represents a promising approach to compute successfully quantitative exact results of Boltzmann distributed systems by LIF neurons. Besides a detailed introduction of the dynamics, different simplified models of a neuromorphic hardware system are studied with respect to a control of emerging sources of errors.

Figure 1

Comparison of the commonly used network structure (upper row) and the presented architecture with a self-interacting contribution (lower row). Both network structures can be considered as systems of two discrete states with an uncorrelated noise contribution, which corresponds to different implementations of the discrete Langevin machine. Their continuous counterpart is represented by an Ornstein-Uhlenbeck process with spiking character. The dynamics is based on the temporal evolution of a membrane potential $u:=u_{\mathrm{eff}}\left(t\right)$. The interaction of neurons relies on a projection of the potential onto two states and enables a comparison with the Langevin machine. The processes on the right-hand side are already very close to the fundamental dynamics of LIF sampling.

Figure 2

Comparison of the structure of a Boltzmann machine (a) and that of the sign-dependent discrete Langevin machine (b). The ${\mathrm{LM}}^2$ has a self-interacting term and rescaled weights and biases. Nevertheless the dynamics leads in equilibrium to a Boltzmann distribution.

Figure 3

Illustration of a step-by-step approach to map thermodynamic systems on a neuromorphic hardware system. The dashed line indicates the progress of the paper. The paper proposes dynamics which have the potential to exactly preserve the properties of the Boltzmann machine up to this line.

Figure 4

Example evolution of the free membrane potential (red) and the actual membrane potential (blue). After the membrane potential crosses the threshold $\vartheta$, a spike is emitted and the potential is set to a reset potential $\varrho$. If the effective membrane potential is still above $\vartheta$ after the refractory period ${\tau }_{\mathrm{ref}}$, the neuron spikes again. At the end of a “burst” of $n$ spikes the free and the actual membrane potential converge in negligible time (scheme taken from Ref. [37]).

Figure 5

Upper plots: Comparison of equilibrium distributions $P_{\mathrm{eq}}\left(u_{i,\mathrm{eff}}\right)$ for the continuous processes for the free case with bias $b=0.8$ without a refractory mechanism (a) and with a refractory mechanism with refractory time ${\tau }_{\mathrm{ref}}=8$ (b). Lower plots: Corresponding example trajectories of the ${\mathrm{OU}}^2$ processes with and without a refractory mechanism for $\epsilon =0.2$. The timescale of the lower plot is rescaled according to the transition probabilities to coincide.

Figure 6

Comparison of scaling and correction factors for the sign-dependent processes for a better convergence to the logistic distribution: (a) The numerical and the analytic scaling factor ${\lambda }_{\epsilon }=\lambda \left(\sqrt{\epsilon }\right)$ for the sign-dependent Langevin machine. (b) The numerical and analytic correction factors ${\alpha }_{\epsilon }$ for the sign-dependent Ornstein-Uhlenbeck process.

Figure 7

Illustration of the nonsymmetric deformation of the ${\mathrm{LM}}^1$ and the ${\mathrm{OU}}^1$ process for larger refractory times. The activation functions are shifted to comply: $P(z_i=1){|}_{b=0}=0.5$. The small plot contains the absolute deviation to the cumulative Gaussian distribution.

Figure 8

Comparison of the magnetization (a) and the specific heat (b) obtained by a standard Monte Carlo algorithm and by the ${\mathrm{LM}}^2$ for the four-state clock model on a $16\times 16$ lattice. The results of the ${\mathrm{LM}}^2$ converge for $\epsilon \to 0$ to the results of the standard Monte Carlo algorithm. Relative deviations of the inverse critical temperatures in dependence of $\epsilon$ are illustrated in Fig. 9.

Figure 9

Relative deviations of the obtained inverse critical temperatures for finite values of $\epsilon$ to the inverse critical temperature of a standard Monte Carlo algorithm.

Figure 10

Comparison of different properties of the activation function for the different processes with and without a refractory mechanism: (a) Activation function and absolute deviation to the logistic distribution without a refractory mechanism. (b) The same as in (a), but with a refractory mechanism with refractory time $\tau =8$. (c) Absolute deviation of the exact results of the activation for $b=1$ for the sign-dependent processes in dependence of $\epsilon$. The deviation converges in the limit $\epsilon \to 0$ for all processes to zero, with and without a refractory mechanism. (d) Absolute deviation of the results for the ${\mathrm{OU}}^2$ process without a refractory mechanism to results with a refractory mechanism and different values of ${\tau }_{\mathrm{ref}}$. The deviations are compared for $b=\{-1,0,1\}$.

Figure 11

Trajectories of the neuron state and the membrane potential in computer time for the different processes with a uniform timescale.

Figure 12

Scaling factors $a$ in dependence of $\epsilon$ for a mapping of the transition probabilities of the sign-dependent processes onto the transition probability of the Boltzmann machine and, hence, of the temporal evolution on the computer time. The scaling factors are computed for the free case with a bias $b=0$.

Figure 13

${\tau }_{\mathrm{ref}}^{\prime }$ in dependence of the refractory time ${\tau }_{\mathrm{ref}}$ for $\epsilon =0.2$ and $\epsilon =0.5$ (a) and of $\epsilon$ for ${\tau }_{\mathrm{ref}}=8$ (b) for the ${\mathrm{OU}}^2$ process. Both dependencies obey a power law.

Figure 14

Comparison of numerical results for the different models without a refractory mechanism (left column) and with a rectangular PSP shape: (a) Ising model without a refractory mechanism ($4\times 4$ lattice). (b) Ising model with a rectangular PSP ($\tau =8, 4\times 4$ lattice). (c) Kullback-Leibler divergence without a refractory mechanism. (d) Kullback-Leibler divergence with a rectangular PSP ($\tau =8$). (a, b) The absolute magnetization of the Ising model is plotted against the inverse temperature $\beta$. The deviations of the different models mirror the observed deviations of the activation function. The results confirm the presumption that small deviations in the activation function can have a large impact on the resulting observables. (c, d) Illustration of the evolution of the Kullback-Leibler divergence for a Boltzmann machine with three neurons based on their history. As an exception, the time is not rescaled with respect to the transition probabilities, i.e., the correlation times, in these plots. This causes a shift of the curves of the sign-dependent processes to larger times. The observed levels of convergence of the Kullback-Leibler divergence of the different models are in concordance to the results of the Ising model. The different levels of convergence for the fitted processes signifies the large dependency of the accuracy of resulting correlation functions on errors in the representation of the activation function and respectively on corresponding weights and biases within the network.

Figure 15

Absolute deviation of the exact results of the Ising model for the absolute magnetization $m$ at the inverse critical temperature for the sign-dependent processes in dependence of $\epsilon$. The curves converge for all methods and all dynamics for smaller values of $\epsilon$ to the results of the Metropolis algorithm. The rate of convergence differs and signifies dependencies on the properties of the model, the intrinsic parameters and the update dynamics. Minor differences of the ${\mathrm{OU}}^2$ process occur due to finite time steps in the simulation.

Figure 16

Illustrations regarding the limit ${lim}_{\epsilon \to 0}{n}_{\epsilon ,m}\left(x\right)$ of relations (B1) and (B3) for $m=0$ and $m=1$: (a) Comparison to the exponential function. (b) Dependency on $x$ for a fixed $\epsilon =0.7$. (c) Dependency on $\epsilon$ for a fixed value of $x=0.8$. (a)–(c): The vertical lines in (b) and (c) indicate the respective fixed value of $\epsilon$ and $x$. In general, the limit with the cumulative Gaussian distribution ($m=0$) has a lower deviation for equal values of $\epsilon$ then the limit with the Gaussian distribution ($m=1$).