Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Nineteenth Annual Computational Neuroscience Meeting: CNS*2010

Open Access Poster Presentation

Neural conduction delay forces the emergence of predictive function in simulated evolution

Timothy A Mann* and Yoonsuck Choe

Author Affiliations

Department of Computer Science and Engineering, Texas A&M University, College State, TX 77843, USA

For all author emails, please log on.

BMC Neuroscience 2010, 11(Suppl 1):P62  doi:10.1186/1471-2202-11-S1-P62

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/11/S1/P62


Published:20 July 2010

© 2010 Mann and Choe; licensee BioMed Central Ltd.

Poster Presentation

Evidence from biological studies suggests that humans are able to predict the sensory consequences of their own actions [1]. Computational studies also demonstrate the advantage of systems that predict sensory consequences of actions over those that predict the value of actions alone [2]. But how could the ability to predict sensory consequences of actions have evolved? One solution suggested by [3] is that prediction mechanisms first evolved to deal with natural sources of delay. Delay is commonly considered to be a purely negative feature of real world systems; however, we argue that delay can actually encourage evolution of the prediction of sensory consequences. We hypothesize that increasing sensory delay to an evolving population of sensory-motor agents will increase reliance on internal prediction of sensory consequences.

To test our hypothesis we evolved populations of artificial neural networks at a complex control task (i.e. pole balancing, see figure 1) with varied neural conduction delay (Δt) between sensory neurons and input to the control network (see figure 2), which estimates the long term cost of applying a specific action. For top fitness networks, hidden unit activations were recorded as well as the true consequent sensory state during several evaluation trials. Each sensory variable was associated with the hidden unit that the sensory variable was maximally correlated with. Taking the average of these correlation values provides a measure of how well an agent can predict the sensory consequences of actions. We expected to find that increasing sensory delay also increases the average correlation measure described above.

The result of the experiment (summarized in figure 3) show that with no delay successful agents use a range of strategies, however, as delay increases successful strategies are forced to rely more and more on prediction of the next state to compensate for sensory delay. This seems surprising when considering that under conditions of no delay it is considerably easier to predict the next state than conditions with increased delay.

Although the common conception of delay is negative, sensory delay can direct natural selection to favor individuals that are better able to predict the sensory consequences of their actions.

thumbnailFigure 1. Cart-Pole Balancing

thumbnailFigure 2. Control network structure

thumbnailFigure 3. Absolute correlation between hidden until activations and variables of the state at time t+Δt as delay increases.

References

  1. Bullock D, Grossberg S, Geunther F: A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm.

    Journal of Cognitive Neuroscience 1993, 54:408-435. Publisher Full Text OpenURL

  2. Kuvayev L, Sutton RS: Model-based reinforcement learning with an approximate, learned model.

    Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems 1996, 101-105. OpenURL

  3. Kwon J, Choe Y: Facilitating neural dynamics for delay compensation: A road to predictive neural dynamics?

    Neural Networks 2009, 22:267-276. PubMed Abstract | Publisher Full Text OpenURL