Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Twentieth Annual Computational Neuroscience Meeting: CNS*2011

Open Access Poster presentation

Self-organizing neural maps for multi-modal associations

Mathieu Lefort*, Yann Boniface and Bernard Girau

Author Affiliations

LORIA, Campus Scientifique, BP 239, 54506 Vandoeuvre-lès-Nancy Cedex, France

For all author emails, please log on.

BMC Neuroscience 2011, 12(Suppl 1):P125  doi:10.1186/1471-2202-12-S1-P125


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/12/S1/P125


Published:18 July 2011

© 2011 Lefort et al; licensee BioMed Central Ltd.

This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Poster presentation

To move in a complex and dynamic environment, according to the active perception theory, an agent needs to learn the multi modal correlations between its actions and the changes they induce in the environment [1]. Human beings interact with their environment through several distant organs, whose sensory flow processing are influencing each others as, for example, in the Mc Gurk effect [2]. In the functional view of the cortex, each sensory area processes a specific sensory flow and associative areas integrate these flows in a consistent representation of the world that influences in return uni-modal perceptions as for the ventriloquist effect [3]. At a mesoscopic level, the cortex shows a generic structure composed of multi-layer cortical columns. We propose a bio inspired model of perceptive map which, using a continuous and unsupervised learning, self-organizes to map a sensory data flow. Thus, using a spatial competition mechanism, a perceptive map provides an activity bump representing the current perception. In a multi modal architecture, by connecting multiple perceptive maps to an associative map with reciprocal spatially constrained connectivity, the spatial organization of all perceptions have to relax the multi modal constraints (see figure 1A). Thus, this multimodal architecture provides a way to learn multi modal correlations with generalization.

thumbnailFigure 1. A Example of use of our perceptive map model in a multi modal architecture with two sensors and one actuator. B Generic structure of one cortical column in a perceptive map.

More precisely, each perceptive map consists of generic cortical columns with computing and learning which are local and decentralized (see figure1 B). Sensitive layer receives the feed-forward flow coming from a sensor and uses the BCM learning rule [4]. This synaptic learning rule is based on the hebbian one and is able to autonomously raise a selectivity to one stimulus of the upcoming flow. From a computational point of view, the BCM rule uses a sliding threshold between long term potentiation and long term depression, which has been confirmed by biological evidences [5]. The cortical layer receives feedback influence coming from the associative map, whose activity represents the multi modal perception. The perceptive layer is based on the neural field theory [6] and it receives the sensitive activity modulated by the cortical one. Thanks to the competitive mechanism introduced by the lateral connectivity with a difference of Gaussian shape, an activity bump emerges at the map level, where the activity is the most spatially consistent, representing a consensus between the local sensation and the multi modal constraints.

To obtain a self-organization of the sensitive layer at the map level, the perceptive layer modulates the BCM activity, so that the spatial consistency of the activity bump is propagated to the selectivity organization. We have also introduced an unlearning term in the BCM learning rule in order to forget the current selectivity if it is not consistent with the received modulation. This unlearning mechanism provides plasticity to the self-organization in order to adapt to the multi modal constraints.

References

  1. O’Regan JK, Noë A: A sensorimotor account of vision and visual consciousness.

    Behavioral and brain sciences 2001, 24(05):939-973. Publisher Full Text OpenURL

  2. Mcgurk H, Macdonald J: Hearing lips and seeing voices.

    Nature 1976, 264(5588):746-748. PubMed Abstract | Publisher Full Text OpenURL

  3. Bonath B, Noesselt T, Martinez A, Mishra J, Schwiecker K, Heinze HJ, Hillyard SA: Neural basis of the ventriloquist illusion.

    Current Biology 2007, 17(19):1697-1703. PubMed Abstract | Publisher Full Text OpenURL

  4. Bienenstock EL, Cooper LN, Munro PW: Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex.

    Journal of Neuroscience 1982, 2(1):32. PubMed Abstract | Publisher Full Text OpenURL

  5. Bear MF: Mechanism for a sliding synaptic modification threshold.

    Neuron 1995, 15(1):1-4. PubMed Abstract | Publisher Full Text OpenURL

  6. Amari S: Dynamics of pattern formation in lateral-inhibition type neural fields.

    Biological Cybernetics 1977, 27(2):77-87. PubMed Abstract | Publisher Full Text OpenURL