Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Eighteenth Annual Computational Neuroscience Meeting: CNS*2009

Open Access Poster presentation

The attention-gated reinforcement learning model can explain experimentally observed changes in tuning curves that follow category learning

Lawrence A Watling1*, Robbert Geerts1, Pieter R Roelfsema12 and Arjen van Ooyen1

Author Affiliations

1 Department of Integrative Neurophysiology, VU University Amsterdam, The Netherlands

2 The Netherlands Institute for Neurosciences, Vision & Cognition Group, Amsterdam, The Netherlands

For all author emails, please log on.

BMC Neuroscience 2009, 10(Suppl 1):P137  doi:10.1186/1471-2202-10-S1-P137


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/10/S1/P137


Published:13 July 2009

© 2009 Watling et al; licensee BioMed Central Ltd.

Poster presentation

A problem with many traditional neural network learning algorithms is that they lack a biologically plausible method of assigning credit to the nodes in the earlier levels of the network that played a decisive role in the stimulus response mapping. In the Attention-Gated Reinforcement Learning Model (AGREL) [1] the credit assignment problem is overcome by the focal influence of attention through the feedback connections in the network. In each trial, a global reward value (conveyed by neuromodulators) is calculated and serves to increase the likelihood that successful behavior be repeated. The activity in the output level of the network (which is limited so that only a single node at a time may be active/attended) is back propagated through the network via feedback connections. The feedback connections, which are reciprocal in strength to the feedforward connections, allow the network to selectively target the lower level nodes that drove the current output, which in combination with the global reward factor, allows the network to enhance connections with nodes that caused successful behaviors, see figure 1. This attentional gating of the reward signal offers a biologically realistic solution to the credit assignment problem and in so doing provides a coherent and unifying framework for learning, with attentional selection at its core. AGREL has previously been shown to closely replicate the changes in tuning curves observed in two physiological categorization tasks [1]. In the current study, we first replicate the findings of two further physiological tasks using AGREL, and second, examine the effect of removing the feedback connections, on our models performance in these and the previous tasks.

thumbnailFigure 1. AGREL network architecture, and learning rules for a 3 layered network. δ = reward signal.

One of the modeled tasks was conducted by Freedman and Assad [2], where monkeys were trained to group 12 directions of coherently moving dots into 2 arbitrary categories. They found that neurons in region LIP learned to respond selectively to a particular category following training (see figure 2). This finding was replicated in AGREL using a three layer network, in which the units in the hidden layer where shown to exhibit similar categorization dependent tuning properties, see figure 2. In general, when the feedback connections were removed, the performance of the network was severely impaired, and the physiologically observed changes in tuning curves were no longer seen, see figure 2. However, depending on the complexity of the task, the network was still able to converge to a solution, although convergence times were now far longer.

thumbnailFigure 2. Radial histogram plots, in which each neuron is assigned to the direction of its preferred boundary (i.e. max difference between responses on either side). 1st: taken from [2], 2nd: shows AGREL with feedback connections, and 3rd: shows AGREL without feedback connections.

References

  1. Roelfsema P, van Ooyen A: Attention-gated reinforcement learning of internal representations for classification.

    Neural Computation 2005, 17:2176-2214. PubMed Abstract | Publisher Full Text OpenURL

  2. Freedman D, Assad J: Experience-dependent representation of visual categories in parietal cortex.

    Nature 2006, 443:85-88. PubMed Abstract | Publisher Full Text OpenURL