The MLE of the species tree under the likelihood function f(G|S). a) The species tree (shaded) has three ancestral populations. The ancestral population AP1 at the root of the tree is the common ancestral population of species A, B, C, and D. Two internal branches with length τ0 and τΔ in the species tree represent the ancestral populations AP0 and Δ. Population AP0 is the common ancestral population of species A and B, while Δ is the common ancestral population of species C and D. Each population has a population size θ and branch length τ. The purple lines represent a gene tree, the evolutionary history of sequences sampled from species A, B, C, and D. The red arrow indicates the coalescence of the two sequences sampled from species C and D. The corresponding coalescence time interval is t, which decreases to 0 as branch length τΔ decreases to 0 in the direction indicated by the two blue arrows. b) The likelihood of the species tree given three gene trees. The species tree and three gene trees are ultrametric trees. The number on each branch represents the branch length. In the species tree, W is the length of the internal branch and Y = X+W, θ1 is the population size for the root population, and θ2 is the population size for the ancestral population of species A and B.
Liu et al. BMC Evolutionary Biology 2010 10:302 doi:10.1186/1471-2148-10-302