Email updates

Keep up to date with the latest news and content from BMC Neuroscience and BioMed Central.

This article is part of the supplement: Twentieth Annual Computational Neuroscience Meeting: CNS*2011

Open Access Poster presentation

Architectural constraints on learning and memory function

Ann M Hermundstad*, Kevin S Brown, Danielle S Bassett and Jean M Carlson

Author Affiliations

Department of Physics, University of California, Santa Barbara, CA 93117, USA

For all author emails, please log on.

BMC Neuroscience 2011, 12(Suppl 1):P31  doi:10.1186/1471-2202-12-S1-P31


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2202/12/S1/P31


Published:18 July 2011

© 2011 Hermundstad et al; licensee BioMed Central Ltd.

This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Poster presentation

Recent experimental and computational studies have identified relationships between architecture and functional performance in information processing systems ranging from natural neuronal ensembles [1,2] to artificial neural networks [3,4]. While these systems can vary greatly in their size and complexity, they share certain structural features, such as parallel and layered motifs [5]. Quantifying how these features influence functionality is a first step toward understanding the behavior of both natural and artificial information processing systems. Of particular interest is the impact of structural architecture on the ability of the system to balance stability with flexibility, for example in memory versus learning.

In this study, we use neural networks as model information processing systems to examine tradeoffs in learning and memory processes arising from variations in structural organization. We compare the performance of parallel and layered structures during sequential function approximation, a task that requires networks to produce, retain, and dynamically adapt representations of external information. We measure network performance over a range of learning conditions by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time allowed for learning. By characterizing local error landscape curvature, we can directly relate the functional performance of the system to its underlying architecture.

Across a range of both parallel and layered system architectures, we find that variations in error landscape curvature give rise to tradeoffs between the ability of these networks to learn new versus retain old information, maximize success versus minimize failure, and produce specific versus generalizable representations of information. In particular, parallel networks generate smooth error landscapes with deep, narrow minima. Therefore, given sufficient time and through the adjustment of a large number of connection weights, parallel networks can find highly specific representations of the external information. Although accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations by adjusting a fewer number of weights. Although less accurate, these representations are more easily adaptable.

We have conducted a detailed analysis of network performance over a range of parallel and layered architectures, thereby isolating learning and memory tradeoffs that arise from underlying structural complexity. A thorough understanding of small network systems is crucial for predicting the behavior of larger systems in which statistical studies of performance would not be possible. In particular, these results may provide insight into the behavior of composite systems, such as cortical layers composed of structurally distinct columns [6] or modular divide-and-conquer networks [7], which share features of both parallel and layered architectures. Additionally, the existence of tradeoffs inherent to a range of network structures may help explain the variability of architectural motifs observed in large-scale biological [5] and technical [3] systems. Identifying the structural mechanisms that impact performance has implications for understanding a wide variety of both natural and artificial learning systems.

References

  1. Honey CJ, et al.: Predicting human resting-state functional connectivity from structural connectivity.

    Proc. Natl. Acad. Sci 2009, 106(6):2035-2040. Publisher Full Text OpenURL

  2. Scholz J, Klein MC, Behrens TEJ, Johansen-Berg H: Training induces changes in white-matter architecture.

    Nat. Neurosci 2009, 12:1370-1371. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  3. Backoglu HB: Circuits, Interconnections, and Packaging for VLSI. Addison-Wesley; 1990.

  4. Galushkin AI: Neural Networks Theory. Berlin: Springer; 2007.

  5. Chittka L, Niven J: Are bigger brains better?

    Curr. Biol 2009, 19(21):R995-R1008. PubMed Abstract | Publisher Full Text OpenURL

  6. Mountcastle VB: The columnar organization of the neocortex.

    Brain 1997, 120(4):701-722. PubMed Abstract | Publisher Full Text OpenURL

  7. Fu HC, Lee YP, Chang CC, Pao HT: Divide-and-conquer learning and modular perceptron networks.

    IEEE Tran. Neural Networks 2001, 12(2):250-263. Publisher Full Text OpenURL