Resolution:
standard / ## Figure 1.
Effective information, minimum information bipartition, and complexes. a. Effective information. Shown is a single subset S of 4 elements ({1,2,3,4}, blue circle), forming part
of a larger system X (black ellipse). This subset is bisected into A and B by a bipartition
({1,3}/{2,4}, indicated by the dotted grey line). Arrows indicate causally effective
connections linking A to B and B to A across the bipartition (other connections may
link both A and B to the rest of the system X). To measure EI(A→B), maximum entropy
H^{max }is injected into the outgoing connections from A (corresponding to independent noise
sources). The entropy of the states of B that is due to the input from A is then measured.
Note that A can affect B directly through connections linking the two subsets, as
well as indirectly via X. Applying maximum entropy to B allows one to measure EI(B→A).
The effective information for this bipartition is EI(A B) = EI(A→B) + EI(B→A). b. Minimum information bipartition. For subset S = {1,2,3,4}, the horizontal bipartition {1,3}/{2,4} yields a positive
value of EI. However, the bipartition {1,2}/{3,4} yields EI = 0 and is a minimum information
bipartition (MIB) for this subset. The other bipartitions of subset S = {1,2,3,4}
are {1,4}/{2,3}, {1}/{2,3,4}, {2}/{1,3,4}, {3}/{1,2,4}, {4}/{1,2,3}, all with EI>0.
c. Analysis of complexes. By considering all subsets of system X one can identify its complexes and rank them
by the respective values of Φ – the value of EI for their minimum information bipartition.
Assuming that other elements in X are disconnected, it is easy to see that Φ>0 for
subset {3,4} and {1,2}, but Φ = 0 for subsets {1,3}, {1,4}, {2,3}, {2,4}, {1,2,3},
{1,2,4}, {1,3,4}, {2,3,4}, and {1,2,3,4}. Subsets {3,4} and {1,2} are not part of
a larger subset having higher Φ, and therefore they constitute complexes. This is
indicated schematically by having them encircled by a grey oval (darker grey indicates
higher Φ). Methodological note. In order to identify complexes and their Φ(S) for systems with many different connection
patterns, each system X was implemented as a stationary multidimensional Gaussian
process such that values for effective information could be obtained analytically
(details in [8]). Briefly, in order to identify complexes and their Φ(S) for systems
with many different connection patterns, we implemented numerous model systems X composed
of n neural elements with connections CON_{ij }specified by a connection matrix CON(X) (no self-connections). In order to compare
different architectures, CON(X) was normalized so that the absolute value of the sum
of the afferent synaptic weights per element corresponded to a constant value w<1
(here w = 0.5). If the system's dynamics corresponds to a multivariate Gaussian random
process, its covariance matrix COV(X) can be derived analytically. As in previous
work, we consider the vector X of random variables that represents the activity of the elements of X, subject to
independent Gaussian noise R of magnitude c. We have that, when the elements settle under stationary conditions,
X = X * CON(X) + cR. By defining Q = (1-CON(X))^{-1 }and averaging over the states produced by successive values of R, we obtain the covariance matrix COV(X) = <X*X> = <Q^{t }* R^{t }* R * Q> = Q^{t }* Q, where the superscript t refers to the transpose. Under Gaussian assumptions,
all deviations from independence among the two complementary parts A and B of a subset
S of X are expressed by the covariances among the respective elements. Given these
covariances, values for the individual entropies H(A) and H(B), as well as for the
joint entropy of the subset H(S) = H(AB) can be obtained as, for example, H(A) = (1/2)ln
[(2π e)^{n}|COV(A)|], where |•| denotes the determinant. The mutual information between A and
B is then given by MI(A;B) = H(A) + H(B) - H(AB). Note that MI(A:B) is symmetric and
positive. To obtain the effective information between A and B within model systems,
independent noise sources in A are enforced by setting to zero strength the connections
within A and afferent to A. Then the covariance matrix for A is equal to the identity
matrix (given independent Gaussian noise), and any statistical dependence between
A and B must be due to the causal effects of A on B, mediated by the efferent connections of A. Moreover, all possible
outputs from A that could affect B are evaluated. Under these conditions, EI(A→B) = MI(A^{Hmax};B). The independent Gaussian noise R applied to A is multiplied by c_{p}, the perturbation coefficient, while the independent Gaussian noise applied to the
rest of the system is given by c_{i}, the intrinsic noise coefficient. Here c_{p }= 1 and c_{i }= 0.00001 in order to emphasize the role of the connectivity and minimize that of
noise. To identify complexes and obtain their capacity for information integration,
one considers every subset S of X composed of k elements, with k = 2,..., n. For each
subset S, we consider all bipartitions and calculate EI(A B) for each of them. We find the minimum information bipartition MIB(S), the bipartition for which the normalized effective information reaches a minimum,
and the corresponding value of Φ(S). We then find the complexes of X as those subsets S with Φ>0 that are not included within a subset having higher
Φ and rank them based on their Φ(S) value. The complex with the maximum value of Φ(S)
is the main complex. MATLAB functions used for calculating effective information and complexes are at
http://tononi.psychiatry.wisc.edu/informationintegration/toolbox.html webcite.
Tononi |