Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Twentieth Annual Computational Neuroscience Meeting: CNS*2011

Open Access Poster presentation

Policy gradient rules for populations of spiking neurons

Johannes Friedrich*, Robert Urbanczik and Walter Senn

Author Affiliations

Department of Physiology, University Bern, Bern, Switzerland

For all author emails, please log on.

BMC Neuroscience 2011, 12(Suppl 1):P111  doi:10.1186/1471-2202-12-S1-P111

The electronic version of this article is the complete one and can be found online at:

Published:18 July 2011

© 2011 Friedrich et al; licensee BioMed Central Ltd.

This is an open access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Poster presentation

Population coding is widely regarded as a key mechanism for achieving reliable behavioral decisions given a high neuronal variability. Here, we present two general recipes to derive learning rules from a policy gradient approach for different neural codes and decision making networks, one based on partial integration across feature values, and one based on linear approximation around a target feature. The first technique leads to a tightly code-specific learning rule where details of the code-irrelevant spiking information are integrated away and the code-specificity enters at the synaptic level. The second technique yields modular learning rules which can be weakly code-specific, with a spike-timing dependent base synaptic plasticity rule which is modulated by a code specific population and decision signal. Decisions can be binary, multi-valued, or even continuous-valued. For illustration, we consider a spike count and a spike latency code. We test them on simple model problems and assess the superiority of tight over weak code-specificity with respect to the performance. While code-specific rules increase the performance only marginally when considering a single neuron [1], our tightly code-specific rule designed for population coding can strongly boost performance. Both code-specific learning rules improve in performance with increasing population size as opposed to standard reinforcement learning [2]. For mathematical clarity we presented the rules for an episodic learning scenario. But a biological plausible implementation of a fully online scheme is also possible [2,3].


  1. Sprekeler H, Hennequin G, Gerstner W: Code-specific policy gradient rules for spiking neurons.

    In Advances in Neural Information Processing Systems Edited by Bengio Y, Schuurmans D, Lafferty J, Williams CKI, and Culotta A. 2009, 22:1741-1749. OpenURL

  2. Urbanczik R, Senn W: Reinforcement learning in populations of spiking neurons.

    Nature Neuroscience 2009, 12(3):250-252. PubMed Abstract | Publisher Full Text OpenURL

  3. Friedrich J, Urbanczik R, Senn W: Learning spike-based population codes by reward and population feedback.

    Neural Computation 2010, 22(7):1698-1717. PubMed Abstract | Publisher Full Text OpenURL