Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Eighteenth Annual Computational Neuroscience Meeting: CNS*2009

Open Access Poster presentation

Cortico-striatal plasticity for action-outcome learning using spike timing dependent eligibility

Kevin N Gurney*, Mark D Humphries and Peter Redgrave

Author affiliations

Adaptive Behaviour Research Group, Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK

For all author emails, please log on.

Citation and License

BMC Neuroscience 2009, 10(Suppl 1):P135  doi:10.1186/1471-2202-10-S1-P135


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/10/S1/P135


Published:13 July 2009

© 2009 Gurney et al; licensee BioMed Central Ltd.

Introduction

We recently proposed that short-latency, sensory-evoked dopamine release is critical for learning action-outcome causality [1]. If an action causes an unexpected outcome associated with a phasic visual event, there will be a phasic burst of dopamine in the striatum. Subsequent reinforcement of the striatal response to the cortical representation of the action then makes the selection of the action (and its outcome) more likely; i.e. there is "repetition biasing" of action selection. This, in turn, facilitates associative learning of the action-outcome pairing elsewhere in the brain. Here, we present a model of cortico-striatal plasticity in medium spiny neurons (MSNs) that could form the basis for a quantitative account of action-outcome learning in basal ganglia.

Methods

We used an Izhikevich-style spiking model MSN with 200 synapses. We constructed new cortico-striatal learning rules based on a recent in vitro study by Shen et al [2]. This study provided, for the first time, comprehensive data on MSN plasticity in the D1 and D2 receptor-dominated MSN subpopulations. For each population (D1/D2) we ascribed STDP-like kernel "templates" to very high and low levels of dopamine in a manner consistent with the data. At intermediate levels of dopamine, the kernels were formed from a linear superposition of these templates. These kernels then gave rise to an eligibility trace for learning that induced plasticity in the presence of subsequent delivery of dopamine [3]. We refer to this mechanism as spike-timing dependent eligibility: STDE. We then mimicked the cortical and dopaminergic signals that an MSN might see during action-outcome learning. Each cortical input comprised 50 highly active afferents with others at a background rate. The selection of the active afferents was fixed for the causal action, and chosen randomly for other actions at each trial (see Figure 1a).

thumbnailFigure 1. a. MSN spike count in response to causal, and other, actions over trials. The phasic DA dip occurs after assigning a noxious value to the outcome of the causal action. b. Synaptic conductances at key trials. Only the first 50 synapses contribute to the action representation.

Results

When phasic dopamine is elicited by the causal action, it induced a rapid increase in MSN response that would be the foundation for inducing repetition bias of action selection. Further, the MSN has become receptive to the action request through synaptic pattern matching (Fig 1b top panel). Subsequent trials induce more selective synaptic patterning (reduced response at trial 1200). Dopamine dips (caused by assigning a noxious value to the outcome) induce a reduction in response. We conclude that the recently discovered complex dopamine-receptor dependent forms of STDP [2] can lead to cortico-striatal plasticity that can support action-outcome learning.

Acknowledgements

This work was part funded by EPSRC grant EP/C516303/1

References

  1. Redgrave P, Gurney KN: The short-latency dopamine signal: a role in discovering novel actions?

    Nat Rev Neurosci 2006, 7:967-975. PubMed Abstract | Publisher Full Text OpenURL

  2. Shen W, Flajolet M, Greengard P, Surmeier DJ: Dichotomous dopaminergic control of striatal synaptic plasticity.

    Science 2008, 321:848-851. PubMed Abstract | Publisher Full Text OpenURL

  3. Izhikevich EM: Solving the distal reward problem through linkage of STDP and dopamine signaling.

    Cereb Cortex 2007, 17:2443-2452. PubMed Abstract | Publisher Full Text OpenURL