In most areas of the brain, information is encoded in the correlated activity of large populations of neurons. We ask how neural responses should be coupled to best represent information about different ensembles of correlated stimuli. Three classical population coding strategies are independence, decorrelation and error correction. Here we demonstrate that balance between the intrinsic noise level and the statistics of the input ensemble induces smooth transitions between these three coding strategies in a network composed of pairwise-coupled neurons and tuned to maximize its information capacity.
We extend recent work [1,2] and theoretically explore small networks of neurons of the "Ising" form whose joint probability of firing is determined both by external inputs and by couplings between pairs of neurons. The neurons are taken to be binary, to represent spiking or silence. We then find the pairwise couplings to maximize information conveyed by neural states about different input ensembles in the presence of intrinsic noise. We consider two kinds of ensembles – binary patterns, and correlated Gaussian inputs – and vary the noise levels parametrically to scan the range of network behaviors.
For binary input ensembles at a high noise level, the optimal neural coupling reinforces the input correlations: a simple form of autoassociative error correction. As the noise level decreases, the coupling goes to zero: the neurons become independent. The Gaussian input ensemble leads to the same optimal network behavior at high noise as for the binary ensemble. This regime is characterized by the emergence of metastable states that serve as "memories" of the input patterns in the sense of a Hopfield network. The weights in a Hopfield net are often chosen by hand to store the desired patterns; here autoassociative memory emerges as an automatic consequence of maximizing information capacity. Furthermore, in this regime single neuron variability overestimates the variability of the code, suggesting that "noise" in single neurons is partly a misinterpretation of redundant population codes. At low noise, there is a new optimal network strategy: decorrelation of the stimulus. The absence of a decorrelating regime in the binary ensemble can be understood intuitively by the fact a correlated binary stimulus ensemble has comparable number of input and output states, while the Gaussian ensemble features an infinite number of possible input states, enabling decorrelation to fill the bandwidth efficiently.