Pattern recognition with neuronal avalanche dynamics

Pattern recognition is a fundamental neuronal process which enables a cortical system to interpret visual stimuli. How the brain learns to recognize patterns is, however, an unsolved problem. The frequently employed method of back propagation excels at this task but has been found to be unbiological in many aspects. In this Rapid Communication we achieve pattern recognition tasks in a biologically, fully consistent framework. We consider a neuronal network exhibiting avalanche dynamics, as observed experimentally, and implement negative feedback signals. These are chemical signals, such as dopamine, which mediate synaptic plasticity and sculpt the network to achieve certain tasks. The system is able to distinguish horizontal and vertical lines with high accuracy, as well as to perform well at the more complicated task of handwritten digit recognition. Resulting from the learning mechanism, spatially separate activity regions emerge, as observed in the primary visual cortex using functional magnetic resonance imaging techniques. The results therefore suggest that negative feedback signals offer an explanation for the emergence of distinct activity areas in the visual cortex.

Figure 1

A network of 8000 neurons with ten output regions (green) and the input grid (black). The intermediate scale-free network has a color map indicating ${\langle r_i\rangle }_C-\langle r_i\rangle$, the localized activity for an input class $C$ in the MNIST task. ${\langle r_i\rangle }_C$ is the average activity for the class $C=3$ and $\langle r_i\rangle$ is the overall average activity of the network. The parameters used are $p_{\text{in}}=0.3$, $N_O=50$, $k_{\text{min}}=10$.

Figure 2

Performance vs the number of patterns presented to the network. The used pattern classes are shown in the bottom right corner. First, only the horizontal and vertical lines are presented ($N_C=2$). Other classes are then added to the task to increase the complexity. The parameters used are $N=8000$, $p_{\text{in}}=0.3$, $N_O=50$, $k_{\text{min}}=10$.

Figure 3

Performance on the MNIST data set vs the number of handwritten digits (patterns) presented to the network. As the performance increases, the activity vectors ${\langle r_i\rangle }_C$ become increasingly different from each other, leading to the decrease in the average cosine similarity $S_{\text{cos}}$. The parameters used are $N=8000$, $p_{\text{in}}=0.3$, $N_O=50$, $k_{\text{min}}=10$, $N_C=10$. The values presented are an average over 100 configurations. The performance approaches the saturation level $P_{\text{sat}}\approx 0.85$ as $P_{\text{sat}}-N_p^{-0.72}$. The cosine similarity decays exponentially to the saturation level $S_{\text{sat}}\approx 0.68$ as $S_{\text{sat}}+\gamma e^{-\delta N_p}$ with $\gamma =0.23$ and $\delta =5.5\times {10}^{-3}$. Some example patterns from the MNIST database are shown below.

Figure 4

(a) As the performance increases, the entropy $H=-{\sum }_{s}P\left(s\right)log\left[P\left(s\right)\right]$ decreases. (b) The normalized avalanche size distribution $P\left(s\right)$ narrows as the system starts to recognize the patterns.

Figure 5

The network after learning to recognize MNIST patterns. The colors of each neuron indicate for which digit $C$ it has the highest activity ${\langle r_i\rangle }_C$. Segregated activity regions emerge. The output neurons are shown in black. The parameters used are $N=8000$, $p_{\text{in}}=0.3$, $N_O=50$, $k_{\text{min}}=10$, $N_C=10$.