Non-Hermitian quasilocalization and ring attractor neural networks

Eigenmodes of a broad class of “sparse” random matrices, with interactions concentrated near the diagonal, exponentially localize in space, as initially discovered in 1957 by Anderson for quantum systems. Anderson localization plays ubiquitous roles in varieties of problems from electrons in solids to mechanical and optical systems. However, its implications in neuroscience (where the connections can be strongly asymmetric) have been largely unexplored, mainly because synaptic connectivity matrices of neural systems are often “dense,” which makes the eigenmodes spatially extended. Here we explore roles that Anderson localization could be playing in neural networks by focusing on “spatially structured” disorder in synaptic connectivity matrices. Recently neuroscientists have experimentally confirmed that the local excitation and global inhibition (LEGI) ring attractor model can functionally represent head direction cells in Drosophila melanogaster central brain. We first study a non-Hermitian (i.e., asymmetric) tight-binding model with disorder and then establish a connection to the LEGI ring attractor model. We discover that (1) principal eigenvectors of the LEGI ring attractor networks with structured nearest-neighbor disorder are “quasilocalized,” even with fully dense inhibitory connections; and (2) the quasilocalized eigenvectors play dominant roles in the early time neural dynamics, and the location of the principal quasilocalized eigenvectors predicts an initial location of the “bump of activity” representing, for example, a head direction of an insect. Our investigations open up venues for explorations at the intersection between the theory of Anderson localization and neural networks with spatially structured disorder.

Figure 1

Heat maps of the effective inverse eigenfunction localization lengths ${\kappa }_{\lambda }^{\mathrm{eff}}=\frac{2{\kappa }_{\lambda }^{+}{\kappa }_{\lambda }^{-}}{{\kappa }_{\lambda }^{+}+{\kappa }_{\lambda }^{-}}$ of one-dimensional sparse random matrices for various values of the excitatory and inhibitory balance $f$ and clockwise bias $g$. (See Appendix pp1 for the details behind our definition of ${\kappa }_{\lambda }^{\mathrm{eff}}$.) The size of these sparse random matrices is $N=500$ and the strength of randomness of the matrix $\mathit{M}$ in Eq. (1) is fixed at $u=0.5$, while the excitatory and inhibitory balance $f$ and clockwise bias $g$ are varied in 0.25 and 0.1 increments, respectively. No states are possible in the regions where ${\kappa }_{\lambda }^{\mathrm{eff}}<0$; these white regions correspond to energy gaps in the complex plane. Eigenvalues obtained from numerical diagonalization of one particular realization are superimposed as black dots. As in Hermitian systems, the inverse localization length ${\kappa }_{\lambda }^{\mathrm{eff}}$ is the largest, i.e., the effect of localization is the strongest, at the outer boundaries of these eigenvalue spectra, including the modes with the largest real parts that (as discussed below) dominate the neural dynamics. See Fig. 2 for representative principal eigenvectors that are strongly localized. As discussed in Ref. [28], the localization length diverges near the origin without clockwise bias $g=0.0$, and as one approaches the rim of the hole (i.e., energy gap) in the spectra for finite clockwise bias $g>0$. These spectra are invariant under the transformation $g\to -g$ and $f\to (1-f)$.

Figure 2

Principal eigenvectors quasilocalize even with global inhibitory connections: (a) “Localized bump of activity” (the blue lines feeding through the blue dot at the center represent a one-to-many inhibitory connection from each neuron on the ring to all other neurons) within a ring attractor neural network designed to model Drosophila melanogaster's central brain that represents the head direction $\theta \in [0,2\pi ]$ [15]. (b) First, second, and third right principal eigenvectors (i.e., those whose eigenvectors have the largest real part) of the synaptic connectivity matrix $\mathit{M}$ [Eq. (1)] with excitatory and inhibitory balance $f=1.0$, clockwise bias $g=0$, and randomness $u=0.5$, for a ring with $N=200$ sites. Each neuron has excitatory connections with random strengths to its two nearest-neighbors [red lines in panel (b)]. The system is a strictly one-dimensional ring, and the principal eigenvectors are localized due to the random nearest-neighbor excitations with the same sign. (c) Same three principal eigenvectors of a ring attractor neural network $\mathit{J}$ without disorder, with a uniform set of inhibitory interactions (including self-inhibition) in blue. Without randomness, the synaptic connectivity $\mathit{J}$ is a circulant matrix, and the eigenvectors are spatially extended around the ring in the form of sines and cosines. (d) Principal eigenvectors of the ring attractor neural network ${\mathit{J}}^{\prime }$ with the same random nearest-neighbor connections as in panels (a) and (b). Each neuron has two nearest-neighbor excitatory connections (red) with the same random strength ($u=0.5$) as in panel (b), with the addition of flat global inhibitory connections (blue) without randomness ($w=0$). With the flat global inhibitory connections, the synaptic connectivity matrix is fully dense. However, the principal eigenvectors are nevertheless “quasilocalized” with only small deviations from the eigenvectors shown in panel (a). (e) The quasilocalization in panel (d) is destroyed when the randomness of the global inhibitory connections $w$ is as strong as the one of local excitatory connections $u$. The nearest-neighbor excitation parameter is $\alpha =1$, and the global inhibition is $\beta =0.5$.

Figure 3

Quasilocalized eigenvectors dominate the initial dynamics and bias the location of the stationary bump of activity: (a) Eigenvalue spectrum of the LEGI model with the connectivity matrix ${\mathit{J}}^{\prime }$ with nearest-neighbor randomness [see Eq. (5)]. All eigenvalues are real in this case. The vertical dashed line is at $Re\left[\lambda \right]=1$ and separates linearly stable eigenmodes from linearly unstable modes for the case of a nonlinear function given by $f\left[x\right]=(x+1)\mathrm{\Theta }(x+1)$ in Eq. (8). For our choice of parameters (see below) more than just three principal eigenvectors will grow exponentially at short times from a state of negligible excitation. (b) Time evolution of the activity of neurons $r_i\left(t\right)$ on the ring starting from an initial condition of homogeneous activity around the ring. Here the neural ring has the short-range random excitation and long-range flat inhibition as shown in Fig. 2. The dynamics at the intermediate time ($t=400\tau$) reflects the three principal eigenvectors plotted in Fig. 2. The long-range inhibitory interactions have suppressed the remaining modes corresponding to $\lambda \ge 1$ in panel (a). At later times, the system locks into a peak of activity dominated by the first principal localized mode (i.e., the eigenfunction with the largest real part). The parameters used for this simulation are total number of neurons $N=200$, the nearest-neighbor connections $\alpha =1$, global inhibition $\beta =0.5$, self-excitation $\gamma =0.3$, and the variance of the nearest-neighbor connections $u=0.5$.

Figure 4

Histogram of the final location of a bump of activity starting from a flat initial condition of activity around the ring. The final location is classified as “1st, 2nd, and 3rd” when the distance to the peak of the localized principal eigenvectors is less than three sites away. If the final location is far from any of these three localized positions, we classify this as “elsewhere.” When the variance of the nearest-neighbor connections is $u=0.5$ and the variance of the global inhibition is $w=0$ [in the case of Fig. 2], the quasilocalized first principal eigenvector location guides the final location of the localized bump of activity. In particular, $87\%$ of the final locations are at the position of the first principal eigenvector (see blue triangles). However, such localization is absent when the disorder is uniformly distributed over the local excitations and global inhibitions with comparable strengths $u=w=0.5$. In this case, only $5\%$ of the final locations of a local bump of activity are near a peak of the first principal eigenvector, and $89\%$ of the final positions are far from the peaks of any of three principal eigenvectors.

Figure 5

Selective excitation of the quasilocalized principal eigenmodes from inhomogeneous initial conditions: (a) The initial activity is set to $r_i\left(0\right)=0.1$ for $80\le i<120$ and otherwise $r_i\left(0\right)=0$. All other parameters are the same as these in Fig. 3. (b) (Pink triangles) Locations (with periodic boundary conditions) of the quasilocalized eigenmodes with the seven largest real parts of eigenvalues. (Blue line) The final position of the bump of activity as a function of a central location of a weak stimuli $c$. Each initial stimulus is applied with a 40-site-wide window ($c-20\le i\le c+20$) as in panel (a). The steplike structure of the line indicates that each of the discrete locations of the first seven localized eigenmodes can be selectively excited by broader initial external stimuli. Because of the underlying disorder, we expect that these final locations will be insensitive to time-dependent noise, which would otherwise cause the bump of activity to diffuse around the ring.

Figure 6

The principal eigenvectors with largest (real) eigenvalue, with the total number of neurons $N=200$, the excitatory and inhibitory balance $f=1$, the variance of the nearest-neighbor connections $u=0.5$ on a log scale, and the periodic boundary conditions appropriate to a ring. The blue dashed line is the conventional exponential localization without global inhibition ($\beta =0$), and the orange solid line corresponds to the quasilocalized eigenmode with global inhibition ($\beta =1$), with the same realization of the disordered nearest-neighbor interactions. Note that we plot the logarithm of the modulus of the dominant eigenfunction, and both eigenfunctions are strongly peaked near lattice site 150.

Figure 7

(a) (Left) Complex eigenvalue spectra with varying ratio $f$ of excitatory to inhibitory interactions and asymmetry parameter $g$. These spectra are colored based on the inverse participation ratio without global inhibition ($\beta =0$). (Right) The modulus of the three principal eigenvectors corresponding to each of the parameter set. The eigenvectors with the largest $\text{Re}\lambda$ are shown in green (these are the most strongly localized), and those with the next two largest $\text{Re}\lambda$ are shown in blue and red respectively. The eigenvectors are exponentially localized in space. (b) (Left) Similar to panel (a), we show complex eigenvalue spectra colored based on the inverse participation ratio with global inhibition ($\beta =1$) without randomness ($w=0$). (Right) Three principal eigenvectors corresponding to each of the parameter regime. The eigenvectors are still quasilocalized even with the global inhibition.

Figure 8

Complex eigenvalue spectrum of the discrete no-disorder model with total number of sites $N=500$, global inhibition $\beta =10/N$, and various clockwise bias $g=0,0.5,1.0$. The detached eigenvalues [given by $2cosh\left(g\right)-\beta N$] correspond to uniform states ($k=0$).

Figure 9

Graphical solution of Eq. (C20) for the shifted eigenvalue spectrum $E$ for a ring with $N=101$ and a single impurity for a particular value of $1/\delta m$ (horizontal dashed line). Blue lines represent analytically obtained eigenvalues as a function of ${\left(\delta m\right)}^{-1}$ with $\beta =0$ (no global inhibition), and red lines are with $\beta =1$ (strong global inhibition). The largest eigenvalue breaks off from what becomes the band of extended states when $N\to \infty$ and the eigenvalues close up ($E<2$) and corresponds to a localized eigenvector. We also confirmed that numerical solution agrees well with the analytical solution presented here.

Figure 10

Plot of the derived relation between $\xi$ and $V_0$, $V_0=-\frac{2\xi sin\left(\xi \pi \right)}{cos\left(\xi \pi \right)+\beta \frac{2sin\left(\xi \pi \right)}{\xi ({\xi }^2-2\pi \beta )}}\equiv f\left(\xi \right)$. The solid blue lines correspond to $V_0=f\left(\xi \right)$ for real $\xi$, and the dashed orange line corresponds to $V_0=f\left(\xi \right)$ for imaginary $\xi$. This dashed (orange) line corresponds to the trajectory of localized state and without impurity ($V_0=0$). (a) Without global inhibition ($\beta =0$), there is a uniform state at the origin ($\xi =0$) with the largest eigenvalue along $V_0=0$, and when $V_0$ becomes finite, $\xi$ immediately becomes imaginary to localize. (b) With global inhibition ($\beta =1$), the state with $\xi \pm 1$ has the largest eigenvalue reflecting the shift of eigenvalue of the uniform state $\xi =0$.

Figure 11

Localization of the principal eigenvalue of the ring model with a single imperfection at the origin. With a finite global inhibition such that $\beta =1\gg \frac{2{\pi }^2}{N^3}$, the uniform state $k=0$ is no longer at the top of the eigenvalue spectrum. Instead, a low-frequency eigenmode with finite wave number $k\ne 0$ will localize as we increase the single-site perturbation. Pink dashed lines correspond to the analytical solutions, and the blue solid lines are eigenvectors obtained with numerical diagonalization.