• Open Access

Effect of Heterogeneity on Decorrelation Mechanisms in Spiking Neural Networks: A Neuromorphic-Hardware Study

Thomas Pfeil, Jakob Jordan, Tom Tetzlaff, Andreas Grübl, Johannes Schemmel, Markus Diesmann, and Karlheinz Meier
Phys. Rev. X 6, 021023 – Published 18 May 2016
PDFHTMLExport Citation

Abstract

High-level brain function, such as memory, classification, or reasoning, can be realized by means of recurrent networks of simplified model neurons. Analog neuromorphic hardware constitutes a fast and energy-efficient substrate for the implementation of such neural computing architectures in technical applications and neuroscientific research. The functional performance of neural networks is often critically dependent on the level of correlations in the neural activity. In finite networks, correlations are typically inevitable due to shared presynaptic input. Recent theoretical studies have shown that inhibitory feedback, abundant in biological neural networks, can actively suppress these shared-input correlations and thereby enable neurons to fire nearly independently. For networks of spiking neurons, the decorrelating effect of inhibitory feedback has so far been explicitly demonstrated only for homogeneous networks of neurons with linear subthreshold dynamics. Theory, however, suggests that the effect is a general phenomenon, present in any system with sufficient inhibitory feedback, irrespective of the details of the network structure or the neuronal and synaptic properties. Here, we investigate the effect of network heterogeneity on correlations in sparse, random networks of inhibitory neurons with nonlinear, conductance-based synapses. Emulations of these networks on the analog neuromorphic-hardware system Spikey allow us to test the efficiency of decorrelation by inhibitory feedback in the presence of hardware-specific heterogeneities. The configurability of the hardware substrate enables us to modulate the extent of heterogeneity in a systematic manner. We selectively study the effects of shared input and recurrent connections on correlations in membrane potentials and spike trains. Our results confirm that shared-input correlations are actively suppressed by inhibitory feedback also in highly heterogeneous networks exhibiting broad, heavy-tailed firing-rate distributions. In line with former studies, cell heterogeneities reduce shared-input correlations. Overall, however, correlations in the recurrent system can increase with the level of heterogeneity as a consequence of diminished effective negative feedback.

  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
3 More
  • Received 28 November 2014

DOI:https://doi.org/10.1103/PhysRevX.6.021023

This article is available under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Physics Subject Headings (PhySH)

NetworksInterdisciplinary PhysicsPhysics of Living Systems

Authors & Affiliations

Thomas Pfeil1,*, Jakob Jordan2, Tom Tetzlaff2, Andreas Grübl1, Johannes Schemmel1, Markus Diesmann2,3,4, and Karlheinz Meier1

  • 1Kirchhoff-Institute for Physics, Heidelberg University, 69120 Heidelberg, Germany
  • 2Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA BRAIN Institute I, Jülich Research Centre, 52425 Jülich, Germany
  • 3Department of Psychiatry, Psychotherapy and Psychosomatics, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany
  • 4Department of Physics, Faculty 1, RWTH Aachen University, 52074 Aachen, Germany

  • *Corresponding author. thomas.pfeil@kip.uni-heidelberg.de

Popular Summary

Diversity in nature is often the result of an active decorrelation due to negative feedback. This effect can be exploited in a variety of technical applications such as data compression, crosstalk reduction, or random-number generation. Analog neuromorphic hardware constitutes an attractive substrate for brain-inspired implementations of such applications: It mimics biological neural systems by implementing physical models of neurons and synapses with analog microelectronics. Neural-network emulations on analog neuromorphic hardware are massively parallel, fast, and energy efficient. Similar to their biological counterparts, neural networks in analog neuromorphic hardware are characterized by a high degree of heterogeneity in neuron and synapse properties. Previous work has shown that this heterogeneity suppresses correlations in feedforward networks. For recurrent neural networks, the effect of heterogeneity on correlations is only poorly understood. Here, we study neural networks emulated on a highly configurable analog neuromorphic-hardware system known as Spikey.

We mimic roughly 200 neurons and nearly 3000 synapses; each neuron receives input from 15 different sources. We simulate over 100 days of biological time sped up by a factor of 10,000, and we show that the mechanism of decorrelation by negative feedback is effective also in highly heterogeneous networks. Counter to intuition, heterogeneity is, in general, not enhancing decorrelation in recurrent nonlinear systems and can even be detrimental. Automatic calibration procedures or homeostatic mechanisms may therefore facilitate active decorrelation and accordingly improve the performance of neuromorphic algorithms.

Our findings demonstrate that the Spikey neuromorphic system is ready for use as a research tool to address neuroscientific questions.

Key Image

Article Text

Click to Expand

Supplemental Material

Click to Expand

References

Click to Expand
Issue

Vol. 6, Iss. 2 — April - June 2016

Subject Areas
Reuse & Permissions

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 3.0 License. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

×

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×