Introduction
The first step toward discovering general principles of sensory processing is to determine the correspondence between neural activity patterns and sensory stimuli. We refer to this correspondence as a "neural code." The Information Bottleneck and the Information Distortion methods [1,3] approach the neural coding problem by finding an optimal clustering of paired stimulus/response observation (X; Y) by solving a constrained optimization problem, with both equality and inequality constraints, in hundreds to thousands of dimensions. The method of annealing has been used to solve this optimization problem: starting at an uninformative solution, one tracks this solution as an annealing parameter varies. The solutions undergo a series of rapid changes with the increase of the annealing parameter (Figure 1). We relate the changes to bifurcations or phase transitions in a dynamical system. The form of the bifurcations is dictated by the subgroup structure of S_{N }[2]. As a consequence of this symmetry, generically only pitchforklike and saddle node bifurcations are possible. The purpose of this contribution is to describe these bifurcations in detail, and to indicate some of the consequences of the bifurcation structure. The results are then applied to the neural coding problem.
We have been able to answer several questions about these bifurcations:.
1. There are N  1 symmetry breaking bifurcations observed when continuing from the initial solution because there are only N  1 subgroups in the chain S_{N }→ S_{N1 }→...→ S_{2 }→ S_{1}.
2. The annealing solutions in Figure 1 all have symmetry S_{M }for some M <N. There exist other suboptimal (and unstable) branches with symmetry S_{m }× S_{n }(m + n = N) that yield mutual information values below the envelope curve depicted in the figure.
3. Symmetry breaking bifurcations are generically pitchforklike and derivative calculations predict whether the bifurcating branches are subcritical or supercritical, as well as their stability. Symmetry preserving bifurcations are generically saddle nodes.
4. A local solution to the optimization problem does not always bifurcate through a symmetry breaking bifurcation.
5. The bifurcations of solutions dictate the convexity of the curve in Figure 1. In particular, a subcritical bifurcation of solutions at I_{0 }implies that the ratedistortion curve R(I) changes convexity in a neighborhood of I_{0}. This is in contrast to the rate distortion curve in information theory, R(D), for which D(q) is linear in q.
References

Gedeon T, Parker AE, Dimitrov AG: Information distortion and neural coding.

Parker AE, Gedeon T: Bifurcation structure of a class of S_{N}invariant constrained optimization problems.
J. Dynamics and Differential Equations 2004, 16:629678. Publisher Full Text

Tishby N, Pereira FC, Bialek W: The information bottleneck method.
The 37th annual Allerton Conference on Communication, Control, and Computing 1999.