Skip to main content

A cortical theory of super-efficient probabilistic inference based on sparse distributed representations

The remarkable structural homogeneity of isocortex strongly suggests a canonical cortical algorithm that performs the same essential function in all regions [1]. That function is widely construed/modeled as probabilistic inference, i.e., the ability, given an input, to retrieve the best-matching memory (or, most likely hypothesis) stored in memory. In [2], I described a cortical model for which both storage (learning) of new items into memory and probabilistic inference are constant time operations, which is a level of performance not present in any other published information processing system. This efficiency depends critically on: a) representing inputs with sparse distributed representations (SDRs), i.e., relatively small sets of binary units chosen from a large pool; and on b) choosing (learning) new SDRs so that more similar inputs are mapped to more highly intersecting SDRs. The macrocolumn (specifically, its pool of L2/3 pyramidals) was proposed as the large pool, with its minicolumns acting in winner-take-all fashion, ensuring that macrocolumnar codes consist of one winner per minicolumn. Here, I present results of large hierarchical model instances, having many levels and hundreds of macrocolumns, performing: a) single-trial learning of sets of sequences derived from natural video; and b) immediate (i.e., no search) retrieval of best-matching stored sequences. Figure 1 shows the major shift in going from the localist coding scheme present in most hierarchical cortical models, e.g., [3], to SDR coding. In Figure 1A, each feature in a coding module (red rectangle) is represented by a single unit, whereas in Figure 1B, each feature in a coding module (red hexagon) is represented by a set of co-active units, one per minicolumn. Yellow call-outs show a sample suggesting the potentially large number of other features stored in a macrocolumn. This change has a potentially large impact on explaining the storage capacity of cortex, but more importantly on explaining the speed and other characteristics of probabilistic/approximate reasoning possessed by biological brains.

Figure 1
figure 1

Comparison of localist (A) and SDR-based (B) versions of visual feature hierarchies.

References

  1. Douglas RJ, Martin KA, Witteridge D: A canonical microcircuit for neocortex. Neural Computation. 1989, 1 (4): 480-488. 10.1162/neco.1989.1.4.480.

    Article  Google Scholar 

  2. Rinkus GJ: A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality. Frontiers in Neuroanatomy. 2010, 4 (17): doi:10.3389/fnana.2010.00017

    Google Scholar 

  3. Giese MA, Poggio T: Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience. 2003, 4 (3): 179-192.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rod Rinkus.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Rinkus, R. A cortical theory of super-efficient probabilistic inference based on sparse distributed representations. BMC Neurosci 14 (Suppl 1), P324 (2013). https://doi.org/10.1186/1471-2202-14-S1-P324

Download citation

  • Published:

  • DOI: https://doi.org/10.1186/1471-2202-14-S1-P324

Keywords