Path integral approach to random neural networks

In this work we study of the dynamics of large-size random neural networks. Different methods have been developed to analyze their behavior, and most of them rely on heuristic methods based on Gaussian assumptions regarding the fluctuations in the limit of infinite sizes. These approaches, however, do not justify the underlying assumptions systematically. Furthermore, they are incapable of deriving in general the stability of the derived mean-field equations, and they are not amenable to analysis of finite-size corrections. Here we present a systematic method based on path integrals which overcomes these limitations. We apply the method to a large nonlinear rate-based neural network with random asymmetric connectivity matrix. We derive the dynamic mean field (DMF) equations for the system and the Lyapunov exponent of the system. Although the main results are well known, here we present the detailed calculation of the spectrum of fluctuations around the mean-field equations from which we derive the general stability conditions for the DMF states. The methods presented here can be applied to neural networks with more complex dynamics and architectures. In addition, the theory can be used to compute systematic finite-size corrections to the mean-field equations.

Figure 1

Qualitative behavior of $V(\mathrm{\Delta };{\mathrm{\Delta }}_0)$ for $-1+g^2{\left[{\varphi }^{\prime }\right]}_{{\mathrm{\Delta }}_0}<0$. Labels denotes the different possible behaviours of the solution. Label $c$: $E_{\mathrm{c}}>0$, $\mathrm{\Delta }\left(\tau \right)$ is periodic and changes sign. Label $e$: $E_{\mathrm{c}}<0$, $\mathrm{\Delta }\left(\tau \right)$ is periodic but remains positive. Label $d$: $E_{\mathrm{c}}=0$, $\mathrm{\Delta }\left(\tau \right)$ decays to zero as $\tau \to \infty$. Label $f$: minimum allowable value of $E_{\mathrm{c}}$, $\mathrm{\Delta }\left(\tau \right)={\mathrm{\Delta }}_0$, static solution.

Figure 2

Dynamical mean-field theory phase diagram. The curves delimit the regions of qualitative different behaviours. On the curve $f$ the energy $E_{\mathrm{c}}$ is equal to the minimum of $V(\mathrm{\Delta };{\mathrm{\Delta }}_0)$ and $\mathrm{\Delta }\left(\tau \right)$ is time-independent. Between the curve $f$ and the curve $d$ the energy $E_{\mathrm{c}}<0$ and $\mathrm{\Delta }\left(\tau \right)$ is time-periodic but positive. On the curve $d$ the energy $E_{\mathrm{c}}$ vanishes and $\mathrm{\Delta }\left(\tau \right)$ decays to 0 as $\tau \to \infty$. Below the curve $d$ the energy $E_{\mathrm{c}}>0$ and $\mathrm{\Delta }\left(\tau \right)$ is time-periodic with not definite sign. On the curve $b$ the potential $V(\mathrm{\Delta };{\mathrm{\Delta }}_0)$ changes from a double well shape to a single well shape. Above the curve $f$ there are no solution to the DMFT equations. For $g>1$ all curves collapse and only the static solution $\mathrm{\Delta }\left(\tau \right)={\mathrm{\Delta }}_0=0$ survives. Numerical values are for $\varphi \left(x\right)=tanh\left(x\right)$.

Figure 3

Qualitative construction of the quantum potential Eq. (91) from the potential Eq. (50).

Figure 4

Qualitative behavior of the quantum potential Eq. (91) for time-periodic solutions.

Figure 5

Qualitative behavior of the quantum potential Eq. (91) for the time-decaying solution.