Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Abstracts from the Twenty Second Annual Computational Neuroscience Meeting: CNS*2013

Open Access Poster presentation

Requirements for the robust operant conditioning of neural firing rates

Robert R Kerr123*, David B Grayden1234, Doreen A Thomas5, Matthieu Gilson6 and Anthony N Burkitt1234

Author Affiliations

1 NeuroEngineering Laboratory, Dept. of Electrical & Electronic Engineering, University of Melbourne, Australia

2 Centre for Neural Engineering, University of Melbourne, Australia

3 NICTA, Victoria Research Lab, University of Melbourne, Melbourne, Australia

4 Bionics Institute, Melbourne, Australia

5 Department of Mechanical Engineering, University of Melbourne, Melbourne, Australia

6 Laboratory for Neural Circuit Theory, RIKEN Brain Science Institute, Saitama, Japan

For all author emails, please log on.

BMC Neuroscience 2013, 14(Suppl 1):P48  doi:10.1186/1471-2202-14-S1-P48


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/14/S1/P48


Published:8 July 2013

© 2013 Kerr et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Poster presentation

Operant conditioning experiments have shown that changes in the firing rates of individual neurons in the motor cortex of monkeys can be elicited [1,2]. In these experiments, the firing rate of the neurons were measured using an implanted electrode, and the monkeys were presented with feedback based on these rates and rewarded for increasing them. Behavioral learning such as this is assumed to be due to plasticity at the synaptic level and reward-modulated spike-timing-dependent plasticity (RSTDP) has previously been proposed as such a model [3]. In this study, we propose a generalization of the existing RSTDP model (classical RSTDP) that can account for experiments where dopamine differentially modulates the amplitude of long-term potentiation and depression (LTP and LTD) [4]. Using analytical techniques and numerical simulations with leaky integrate-and-fire (LIF) neurons, we compare the classical RSTDP (see Figure 1A) with our generalized model (see Figure 1B). We consider the potential for these models to elicit the increased firing rates observed in operant conditioning experiments [1,2] and find two requirements. The first requirement is that, relative to their base level amplitudes, the strengthening of LTP by the reward signal must be greater than the strengthening of LTD. Classical RSTDP cannot exhibit this and, contrary to previous studies [3], we predict that it consequently cannot robustly elicit an increased firing rate. The second requirement is that the reinforced neuron must be able to exhibit short inter-spike intervals (ISIs) relative to its mean ISI. For the LIF neurons we consider, this corresponds to being in a fluctuation-driven regime, such as receiving a balance of excitatory and inhibitory inputs. The findings of this study are consistent with existing experimental studies and they also make testable predictions for possible future experiments.

thumbnailFigure 1. Effective STDP learning windows that show the synaptic change, ΔK, caused by a spike pair with the timing difference, Δt, between the pre- and post-synaptic spikes. The different windows are for different reward signal levels (y = 0, 1, 2, 3, 4, 5, and 6, shown in green, blue, purple, magenta, red, orange, and yellow lines, respectively). A: Classical RSTDP has no LTD or LTP when there is no reward and increasing amounts of both as reward increases. B: Generalized RSTDP allows separate modulation of LTP and LTD and non-zero LTP and LTD when there is no reward. Shown here is generalized RSTDP parameterized to match the effect dopamine has been observed to have on STDP [4]. Switching of LTD to LTP for high reward values is exhibited.

Acknowledgements

Funding is acknowledged from the Australian Research Council (ARC Discovery Project #DP1096699). The Bionics Institute acknowledges the support it receives from the Victorian Government through its Operational Infrastructure Support Program.

References

  1. Fetz EE, Baker MA: Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles.

    J Neurophysiol 1973, 36(2):179-204. PubMed Abstract | Publisher Full Text OpenURL

  2. Kobayashi S, Schultz W, Sakagami M: Operant conditioning of primate prefrontal neurons.

    J Neurophysiol 2010, 103(4):1843-1855. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  3. Legenstein R, Pecevski D, Maass W: A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback.

    PLoS Comput Biol 2008, 4(10):e1000180. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  4. Zhang J-C, Lau P-M, Bi G-Q: Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses.

    Proc Natl Acad Sci USA 2009, 106(31):13028-13033. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL