We designed a new computational system, PSPE, specifically to perform simulations of regulatory sequence evolution, such as promoter sequences. Different from other programs for sequence evolution simulation, which frequently use different evolutionary models for functional and non-functional sites, PSPE imposes a variety of functional constraints and validates at discrete intervals that these constraints are maintained. Such functional constraints include GC content, presence and strength of functional sites, location and copy number restrictions on functional sites, and space constraints between different functional sites. Depending on the specification of these constraints, turnover events are thus possible, as functional sites are not generally tied to a specific location in the sequence.

PSPE reads a set of simulation parameters from a single configuration file (Figure 2). The root sequence for simulation can be provided by the user or generated by PSPE, according to user-specified length, a background Markov model, and functional constraints. PSPE can generate different random evolutionary trees by simulating evolution distances (branch length) with an exponential model, and the number of descendent sequences (number of branches from a parent node) by a Poisson process. While binary trees are commonly used in phylogenetic studies, PSPE can generate different tree structures with either a fixed or a random number of branches from the root or internal node. Given a phylogenetic tree and a sequence at its root, PSPE can use one of many commonly used DNA substitution models as well as different InDel models to simulate sequence evolution, subject to defined functional constraints, such as GC content, functional site locations and interactions of functional sites. By default, PSPE reports the alignment of the simulated sequences, as well as the sequences themselves and the locations of functional sites in each sequence. PSPE also has the capability to simulate replicates from the same tree and same root sequence, which is essential for quantitative evolution simulations.

In this study, a functional TFBS in a descendent sequence corresponds to the original TFBS if its sequence can be traced back to the TFBS sequence in the ancestor; otherwise, the TFBS is regarded as a new one. A TFBS replacement event is therefore defined as an event in which an original TFBS is replaced by a new TFBS of the same type through any two or more events (destruction of the old site and creation of the new one), including point mutations, insertions and deletions. The RTR is defined as the probability of a functional TFBS in an ancestral sequence to be replaced by a newly evolved one in the descendent sequence. We estimate TFBS RTR as the proportion of descendent sequences in which the TFBS is replaced at least once in the evolution process from an ancestral sequence. For example, assuming that we simulate M different descendent sequences from the same ancestral sequence, and we observe replacement turnover of the TFBS in m descendent sequences, then the estimate of RTR is m/M. In the following, we report the mean RTR averaged over different ancestor sequences, that is:

R T ^ R = 1 K ∑ i = 1 K m i M i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8GiVeY=Pipec8Eeeu0xXdbba9frFj0xb9Lqpepeea0xd9q8qiYRWxGi6xij=hbbc9s8aq0=yqpe0xbbG8A8frFve9Fve9Fj0dmeaabaqaciaacaGaaeqabaqabeGadaaakeaacaWGsbGabmivayaajaGaamOuaiabg2da9maalaaabaGaaGymaaqaaiaadUeaaaWaaabCaeaajuaGdaWcaaqaaiaad2gadaWgaaqaaiaadMgaaeqaaaqaaiaad2eadaWgaaqaaiaadMgaaeqaaaaaaSqaaiaadMgacqGH9aqpcaaIXaaabaGaam4saaqdcqGHris5aaaa@3FBC@

where K is the number of different ancestral sequences, M_i is the number of all descendent sequences of the i^th ancestral sequence, and m_i is the number of descendent sequences in which the TFBSs of interest have been subjected to replacement turnover. We also report the median values, as the distributions of RTRs are not necessarily approximate to the normal distribution.

Using PSPE for sequence evolution simulation, we are able to study the replacement turnover rate of functional conserved TFBSs in the evolution process of promoter sequences. In a complicated evolution process, many different events can occur at a TFBS, including point mutation, deletion, insertion, translocation, duplication and replacement. Our study here focuses only on TFBS replacement turnover in a simple 'status quo' scenario, assuming that all TFBSs in the sequences are essential to maintain proper gene expression levels and are thus functionally conserved in all descendent sequences. All functionally conserved TFBS are, however, allowed to be translocated to neighboring regions or replaced by newly evolved sites within a given restricted space. As ancestral sequences, we use either real or simulated human promoter sequences.

As the main transcription factor for this study, we used the well-known cell-cycle regulator E2F, and investigated two additional factors, Myc and NFκB, to validate our model for estimating TFBS replacement rates. Both E2F and Myc are important transcription regulators of cell cycle progression, DNA replication, and apoptosis 30313233. In some cases, E2F and Myc form a complex to regulate gene expression in a combinatorial fashion 3435. NFκB is a family of ubiquitously expressed transcription factors involved in both the onset and the resolution of inflammation. NFκB is also widely believed to govern the expression of many genes for stress response, intercellular communications, cellular proliferation and apoptosis 363738. To simulate ancestral sequences containing binding sites of these transcription factors, we used their positional weight matrix models in the JASPAR database 39. Binding sites in real human promoters known to be regulated by E2F were based on computational prediction (see Materials and methods). The simulated background promoter sequences were generated from a third order Markov model trained on 25,088 annotated human promoter sequences. We used the HKY85 model 40 to simulate nucleotide substitution, a geometric distribution for the size of sequence InDel events, and a gamma distribution and invariant rate (Γ+I) for modeling heterogeneity of substitution rates. The HKY85 model does not assume equal base frequencies and can account for the difference between transitions and transversions with one parameter. Sequence evolution was then additionally subject to diverse functional constraints related to the specific characteristics of transcriptional regulatory regions (Table 1). While many different factors may have significant impact on the RTR of a TFBS, we mainly focused on three important and interesting factors: evolution divergence distance, InDel rate, and restricted translocation distance.

We first studied the effect of divergence distance on the RTR of E2F sites (Figure 3). With increasing evolutionary divergence, we expect the RTR of a TFBS to increase, so the question is how fast and in what pattern the RTR increases along with the divergence distance. To answer this question, we estimated the RTR of a TFBS within a new descendent sequence, evolved from an ancestral sequence at 15 different divergent distances from 0.01 to 5.0, measured by the number of substitutions per site (see Materials and methods). At each of the different distances, we simulated 1,000 ancestor sequences and 1,000 descendent sequences from each ancestral sequence. In the simulation, E2F binding sites in ancestral and descendent sequences were subject to the same functional constraints (Figure 3), such that each simulated sequence had one and only one functional E2F site. As a consequence, E2F replacement could occur only at the time when the loss of the original functional site was accompanied by the creation of a new functional site. This requirement is likely to lead to conservative estimates of turnover rates.

Initial results showed that the RTR of E2F significantly increased as the divergence distance increased (Figure 4a). The change of RTR was faster at short divergence distances (number of substitutions per site <1) than at large divergence distances (number of substitutions per site >3). Based on the assumption that the number of E2F replacement events during any evolution time interval follows a Poisson distribution, we further analyzed the relationship between RTR and sequence divergence distance. Assuming that replacement turnover events occur at a Poisson rate λ, the probability of no replacement in a time interval t measured by number of substitutions per site is:

Pr ⁡ ( N = 0 ) = e − λ t ( λ t ) 0 0 ! = e − λ t MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8GiVeY=Pipec8Eeeu0xXdbba9frFj0xb9Lqpepeea0xd9q8qiYRWxGi6xij=hbbc9s8aq0=yqpe0xbbG8A8frFve9Fve9Fj0dmeaabaqaciaacaGaaeqabaqabeGadaaakeaaciGGqbGaaiOCaiaacIcacaWGobGaeyypa0JaaGimaiaacMcacqGH9aqpjuaGdaWcaaqaaiaadwgadaahaaqabeaacqGHsisliiGacqWF7oaBcaWG0baaaiaacIcacqWF7oaBcaWG0bGaaiykamaaCaaabeqaaiaaicdaaaaabaGaaGimaiaacgcaaaGaeyypa0JaamyzamaaCaaabeqaaiabgkHiTiab=T7aSjaadshaaaaaaa@482F@

Therefore, the probability of at least one replacement turnover, or expected RTR, of a TFBS in a time interval t is:

RTR = Pr(N ≥ 1) = 1 - Pr(N = 0) = 1 - e^-λt

which corresponds to the cumulative density function of an exponential distribution with mean 1/λ.

We fitted the observed E2F RTR data with this exponential model and estimated the model parameter λ. This simple exponential model fitted well with the RTR of E2F observed in our simulation (Figure 4a), where the model parameter λ was 0.0832 and 0.0724 for fitting the mean and median of the observed RTR, respectively. In other words, the average probability for a replacement turnover event of an E2F binding site was 8.3% at a divergence distance of one substitution per site, suggesting the potential of substantial E2F turnover.

To verify the RTR of E2F estimated on simulated promoter sequences, we repeated the experiment using real promoter sequences of human genes as ancestral sequences, known to be under E2F regulation from wet-lab experiments 4142. Among 127 E2F regulated genes confirmed by ChIP-chip experiments 42, we were able to select 11 genes, each having one and only one E2F binding site in the upstream region of 500 base pairs (bp) from its transcription start site (see Materials and methods; see Additional data file 1 for details of the 11 genes). Most of the 11 genes are well known to be under regulation of E2F, especially CDC6, for which the location of the E2F binding site and functional activity of E2F have been characterized 434445. Real promoter sequences would presumably give us a more realistic estimate of RTR of E2F sites than starting from simulated background sequences. One such potential difference is that real promoter sequences may contain remnants or 'ghosts' of previously functional binding sites accumulated during evolution, which could become functional again by a small number of sequence changes, which would thus result in higher turnover rates.

Starting with the real promoter sequences, we ran essentially the same simulation as the simulated promoter sequences above (Table 1), with the minor difference of using a different restricted location of E2F sites for each promoter, as the actual E2F locations were different. We kept, however, the same restricted distance for translocation of E2F sites as those in simulated promoter sequence (50 bp centered on the ancestral site). Since we had a limited number of real promoters, we simulated 10,000 descendent sequences from each ancestral promoter instead of 1,000 descendents as above. The RTRs of E2F sites estimated in this way were highly consistent with those using simulated ancestral sequences across different divergence distances. As a result, the exponential model given in equation 2 fitted well with the observed RTRs (Figure 4b), where the model parameter λ was 0.0833 and 0.0755 for fitting mean and median values, respectively. Both λ values were indeed slightly higher than the corresponding ones starting from simulated ancestral sequences (Table 2), but such small differences may easily be caused by other factors (for example, different locations of E2F sites).

To validate the good fit of estimated turnover rates with a simple exponential model, we performed similar independent simulation studies for the additional TFBSs of Myc and NFκB. Both Myc and NFκB have palindromic binding sites with a length of 11 and 10 bases, respectively. Myc sites have more conserved positions in the center region, consisting of mixed A/T and G/C nucleotides, whereas NFκB has highly conserved positions at the two sides, consisting of mostly G/C nucleotides (Figure 3). Overall, Myc sites are the most degenerate among the three TFBSs. These differences in information content and sequence composition may lead to different RTRs. It was instructive to see how these factors affected the RTR, and whether the exponential model provided as good a fit for these other TFBS as well. For each TFBS, we again simulated 1,000 ancestral promoter sequences, and for each ancestral promoter sequence, we simulated 1,000 descendent sequences at each of 15 divergence distances as above. We also used the same substitution and InDel models for the sequence evolution (Table 1). For the purpose of comparison, we imposed the same location and copy number constraints on both TFBSs as specified in Figure 3.

Our results indicated that the RTR of Myc was consistently more than two times higher than that of NFκB across all divergence distances (Figure 5 and Table 2) For example, the observed RTRs for Myc and NFκB were 0.219 and 0.083 at a divergence distance of 1.0, and 0.373 and 0.167 at a divergence distance of 2.0. These results suggested that differences in sequence composition had a significant impact on the RTRs of a TFBS. In this case, the sequence composition of the NFκB site, which is G/C rich at the two sides and A/T rich in the center, is more different from the background than that of Myc, for which A/T and G/C positions are almost uniformly distributed. Fitting the RTR data with our exponential model, we observed again a good fit for both TFBSs (see Table 2 for the estimated model parameters λ).

Both Myc and E2F are important transcription factors in coordinating cell-cycle regulation, and partner together to regulate some common target genes 3435. As a restricted space between two TFBSs, that is, to enable an effective interaction, can limit the replacement turnover of each individual TFBS, we were interested in assessing how two sites can evolve together as a regulatory module. We studied the RTR of the Myc-E2F pair in a simple scenario in which there was one and only one pair of Myc-E2F in a promoter sequence. For both E2F and Myc, we kept the location restriction relative to the TSS identical to the above studies on single sites, and studied their RTRs by simulations with and without a constraint of restricted space between them (Table 3). We performed simulations at different divergence distances as for individual sites above.

We calculated the observed RTRs of the Myc-E2F pair from the simulated sequences, and compared them to the expected ones assuming independent evolution of both sites. The expected RTR of both sites, defined as the probability of observing simultaneous replacement turnovers of both Myc and E2F, was estimated as the product of the individual RTRs from the simulation of single sites. The expected RTR of a single site, defined as the probability of observing a replacement turnover in only one site of the pair, was estimated from the above simulation of individual sites. Results showed that the expected RTRs were close to the observed ones in simulations without an additional space constraint between two TFBSs (Figure 6a,b), validating the independent evolution of both sites. For the simulation with additional space constraints between the pair, the observed RTRs of both sites showed significant deviation from the predicted ones assuming independent evolution, although the expected and observed RTRs of single sites were still close (Figure 6d). The significantly lower RTRs of both sites indicate that the space constraint between two sites made it less likely for them to turn over simultaneously (Figure 6c).

The small difference between the observed RTRs of the Myc-E2F pair and the expected ones assuming independence of individual TFBSs suggested that it was reasonable to describe the independent evolution of two sites within a simple predictive model. Based on this assumption, we thus described the RTR of a given TFBS pair by:

R T R p a i r = ( 1 − e − λ 1 t ) × ( 1 − e − λ 2 t ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8GiVeY=Pipec8Eeeu0xXdbba9frFj0xb9Lqpepeea0xd9q8qiYRWxGi6xij=hbbc9s8aq0=yqpe0xbbG8A8frFve9Fve9Fj0dmeaabaqaciaacaGaaeqabaqabeGadaaakeaacaWGsbGaamivaiaadkfadaWgaaWcbaGaamiCaiaadggacaWGPbGaamOCaaqabaGccqGH9aqpcaGGOaGaaGymaiabgkHiTiaadwgadaahaaWcbeqaaiabgkHiTGGaciab=T7aSnaaBaaameaacaaIXaaabeaaliaadshaaaGccaGGPaGaey41aqRaaiikaiaaigdacqGHsislcaWGLbWaaWbaaSqabeaacqGHsislcqWF7oaBdaWgaaadbaGaaGOmaaqabaWccaWG0baaaOGaaiykaaaa@4B3E@

where λ₁ and λ₂ are the expected Poisson rates of replacement turnover events for TFBS 1 (E2F) and TFBS 2 (Myc). Similarly, the probability of a replacement turnover of one and only one of two TFBSs can be modeled by:

R T R o n e _ i n _ p a i r = ( 1 − e − λ 1 t ) × e − λ 2 t + e − λ 1 t × ( 1 − e − λ 2 t ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8GiVeY=Pipec8Eeeu0xXdbba9frFj0xb9Lqpepeea0xd9q8qiYRWxGi6xij=hbbc9s8aq0=yqpe0xbbG8A8frFve9Fve9Fj0dmeaabaqaciaacaGaaeqabaqabeGadaaakeaacaWGsbGaamivaiaadkfadaWgaaWcbaGaam4Baiaad6gacaWGLbGaai4xaiaadMgacaWGUbGaai4xaiaadchacaWGHbGaamyAaiaadkhaaeqaaOGaeyypa0JaaiikaiaaigdacqGHsislcaWGLbWaaWbaaSqabeaacqGHsisliiGacqWF7oaBdaWgaaadbaGaaGymaaqabaWccaWG0baaaOGaaiykaiabgEna0kaadwgadaahaaWcbeqaaiabgkHiTiab=T7aSnaaBaaameaacaaIYaaabeaaliaadshaaaGccqGHRaWkcaWGLbWaaWbaaSqabeaacqGHsislcqWF7oaBdaWgaaadbaGaaGymaaqabaWccaWG0baaaOGaey41aqRaaiikaiaaigdacqGHsislcaWGLbWaaWbaaSqabeaacqGHsislcqWF7oaBdaWgaaadbaGaaGOmaaqabaWccaWG0baaaOGaaiykaaaa@6002@

We fitted the observed RTR data with both models 3 and 4. Both models fitted well with data as shown in Figure 6a,b,d, validating our assumption for the independent evolution of TFBSs. However, as the RTRs for the Myc-E2F pair in Figure 6c show, the simple models began to deviate from the simulations in more complex scenarios including dependencies between sites.

Because of the moderate divergence distance between mammalian genomes, such as those of human and mouse, there is a strong interest in comparative studies of their genomes as an important way to infer gene function and gene regulation as well as their evolutionary mechanisms. While it is relatively easy to compare the coding sequences of human and mouse orthologous genes, it remains a difficult task to compare their promoter sequences, largely because they are more divergent than coding sequences. One pioneering comparative genomics study estimated that a fraction as high as 32-40% of the human functional TFBSs may not be functional in rodents, suggesting a high turnover rate of TFBSs 6. A recent study estimated that the divergence distances of human and mouse from the last common ancestor are 0.1187 and 0.3987 substitutions per site, respectively 46. Another study estimated the total divergence distance of human and mouse at about 0.8 substitutions per site 47. Based on these two estimates, we here set the divergence distances of human and mouse from their last common ancestor to be 0.2 and 0.6, respectively, in terms of the number of substitutions per site in neutrally evolving regions. In this study, we simulated TFBS evolution of human and mouse from their last common ancestral species in the hope of shedding some light on the evolution of their TFBSs. Using the same three TFBSs as above, we estimated RTRs of individual TFBSs in human and mouse orthologous sequences at different InDel rates as well as at different restricted translocation distances.

In addition to the theoretical studies regarding turnover rates, the PSPE simulator can be used to assess the impact of the turnover phenomenon on practical applications in comparative genomics. In the following, we looked specifically at the problem of identifying functional binding sites in multiple sequence alignments. Most current alignment tools are based on the assumption that the functional sites in orthologous sequences are homologous in sequence space, that is, that they can be traced back to the same position in the ancestral genome. Replacement turnover events of functional sites in promoter sequences, however, make this assumption somewhat unrealistic, which could consequently limit the performance of a tool for aligning non-coding sequences. Our evaluation aimed to: compare different multiple sequence alignment tools for their robustness to violation of this assumption; and investigate the impact of increasing the number of species on tool performance.

We evaluated a set of representative MSA tools for their performance in detecting TFBSs in several sets of orthologous sequences, generated from an underlying phylogenetic tree of five mammalian genomes (Figure 9). The rationale for using the mammalian tree topology was to achieve a realistic assessment of TFBS detection accuracy and to allow for a fair comparison between different tools. First, in most comparative genomics studies, species in comparison often have different divergence distances from their last common ancestor. Second, it is also frequently assumed that an MSA tool should work better when aligning more closely related species at the beginning stage and adding more distantly related species in later stages, especially for those based on a progressive approach. We used evolutionary distances that were recently inferred from coding regions 46, but evaluated the tree at different scale factors as it is not generally known how well these distances reflect the actual substitution rates in non-coding regions. We extended the simulation to large divergence distances to test the notion that conserved sites should be readily picked up when the surrounding sequence has sufficiently diverged. To assess the validity of our observations, we consistently evaluated tool performance with additional benchmark datasets, generated from a phylogenetic tree with a star topology in which all descendent sequences had the same evolutionary distance from their last common ancestral sequence. The evaluation results are consistent with those reported below (see Additional data file 2 for details).

We scaled the mammalian phylogenetic tree at eight different levels from 0.25 to 5, relative to the actual distances, and generated a benchmark promoter dataset at each scale level (defined as divergence scale coefficient), where each dataset contained 1,000 replicates of orthologous promoter sequences of the five species. Sequences were simulated under the HKY85 nucleotide substitution model with gamma and invariant rate (Γ+I) for modeling substitution rate heterogeneity (Table 5). In the dataset, each sequence contained exactly one functional binding site for each of the six transcription factors: Pax6, TP53, IRF2, PPARG, ROAZ, and YY1E2F. YY1E2F is a composite TFBS consisting of YY1 and E2F binding sites that reportedly interact with each other in cell cycle gene regulation 48. Binding sites were subject to a set of functional constraints (Table 6) that were set to allow for turnover within a restricted distance, but keeping the overall order of the binding sites unchanged. Simulation allowed us to quantify the amount of turnover: how many non-aligned functional sites were due to turnover compared to 'simple' misalignments, and whether some tools would in fact be able to align functional sites despite turnover.

We used this dataset to assess the performance of five widely used MSA tools: CLUSTALW 49, DIALIGN 50, AVID/MAVID 1951, LAGAN/MLAGAN 27, and MUSCLE 20. Among the five tools, AVID/MAVID is the fastest alignment tool and uses exactly matching words as alignment seeds to speed up the alignment process, albeit at the expense of lower alignment accuracy. As an improvement, both DIALIGN and LAGAN/MLAGAN adopt non-exact word matching for finding alignment seeds, which can improve their ability to detect degenerate functional sites. DIALIGN identifies alignment seeds by finding consistent sequence segments of a fixed length between sequences, while LAGAN/MLAGAN locates alignment seeds by chaining together neighboring similar words. Both CLUSTALW and MUSCLE are primarily based on the dynamic programming algorithm. MUSCLE, however, has made significant improvements over CLUSTALW by employing anchoring techniques and a progressive refinement approach. The performance was measured as TFBS detection accuracy, defined as the proportion of nucleotides in functionally homologous TFBSs that were correctly aligned. The detection accuracy reported here is the average value over 1,000 replicates at each divergence scale level.

For the two species (human and baboon) alignment, all five tools showed high detection accuracies of TFBS with no significant difference between each other (Figure 10a(1)). When adding more distant species, such as mouse, to the alignment, we found that TFBS detection accuracies of all tools were dramatically decreased, especially those of MAVID and CLUSTALW (Figure 10b(1),c(1),d(1)). Again, we observed marked differences in performance between different tools for three or more species alignments. Overall, MUSCLE had the highest detection accuracy among all tools across all divergence scale coefficients; MAVID had a slightly worse performance than all other tools; and CLUSTALW, DIALIGN and MLAGAN showed similar performance, although their relative order in performance varied with the number of species or a change of the divergence scale coefficient. As expected, the TFBS detection accuracy decreased for all tools as the divergence scale coefficient increased. PSPE also allowed us to consider only the set of sites that had not turned over, and the relative performance of tools was unchanged (Figure 10a(2),b(2),c(2),d(2)). With increasing distance, a large fraction of sites has turned over, but many of those trace back to the same ancestral nucleotides in several descendants, due to turnover before a branch in the tree or convergent evolution. These sites should thus be aligned and are counted positive in at least some of the pairwise comparisons that our metric is based on, even if they are not in the location of the original TFBS (see Additional data file 2 for more evaluations on turnover sites).

The ability of a tool to detect the presence of a common TFBS varied among different TFBSs, depending on TFBS base composition, length, and restricted translocation distance, as well as the divergence scale coefficient of the phylogenetic tree. For example, Figure 11 shows that detection accuracies differed significantly among TFBSs in the alignments of the five species. In addition, the same figure shows that all tools had higher detection accuracies for TFBSs with low RTRs, such as YY1E2F and Pax6, than those with high RTRs, such as IRF2 and ROAZ. While MUSCLE showed a better performance than all other tools, CLUSTALW as the oldest tool performed slightly better than DIALIGN, MAVID, and MLAGAN in at least some cases (YY1E2F and ROAZ). Additionally, for YY1E2F, Pax6 and TP53, MUSCLE showed higher TFBS detection accuracies than the baseline of SimuALN, suggesting its capability of correctly aligning at least some TFBSs subject to turnover, that is, homologous only at the functional level. At large divergence scale coefficients, however, no tool seemed to perform well in detecting ROAZ.

When looking at the performance of each tool individually (Figure 12), we found that the TFBS detection accuracies of all tools decreased when adding one or more distant species to the human/baboon alignment. For alignments from three to five species, the TFBS detection accuracies of DIALIGN and MUSCLE showed little change, those of CLUSTALW and MLAGAN had a noticeable change and that of MAVID markedly decreased, especially at large divergence scale coefficients. We also compared tool performance again with respect to overall alignment sensitivity and TFBS sensitivity. We found that in terms of alignment sensitivity, MUSCLE and CLUSTALW had slightly better overall performance than the other three (data not shown). The ranks according to TFBS sensitivity were also in the same order as those according to detection accuracies, and this was also true if we considered non-turnover sites only (Figure 13).

Our simulator PSPE is designed specifically for studying the evolution of functional sites in regulatory sequences. PSPE is not only able to use one of many common models of nucleotide substitution, but it can also apply different InDel models important for regulatory sequence simulation. In contrast to other simulators, PSPE imposes a variety of functional constraints instead of sequence constraints. Such functional constraints include GC content, presence of functional sites, strength of the binding sites, location and copy number restrictions on functional sites, and space constraints between different functional sites. All these features enable PSPE to simulate evolution of promoter sequences more realistically than other simulation programs.

Consistent with previous simulation studies 514, our results show that TFBS turnover can occur rapidly in promoter evolution. For example, replacement turnover events can occur at a Poisson rate as high as 0.083 for the highly constrained E2F sites even if we only allow for a small translocation distance of 50 nucleotides, and is even higher for the less constrained sites of Myc (0.22) and NFκB (0.103). Furthermore, these parameters may be relatively conservative considering that we used stringent matrix score cutoffs to avoid false hits, highly restricted locations for functional sites, a relatively low rate for transversions, and the requirement of the presence of exactly one functional site throughout. However, a high turnover rate of TFBSs can frequently be detrimental to an organism, and highly increased turnover rates may not be observed in practice, even for degenerate sites. This is supported by an additional simulation study we carried out using a lower cutoff threshold of 0.85 for functional sites, in which promoters with Myc sites had a lower RTR despite the higher chance of creating a new site at the lower cutoff. This was mainly due to our restriction of allowing only one site to be present in the promoters (see Additional data file 1 for details). Therefore, TFBS replacement turnovers in real sequences may happen more frequently than we estimated, but there is an upper limit of turnover rate for each individual TFBS imposed by the resulting changes in fitness.

Altogether, our study suggests that the TFBS RTR of a functional site between different species does not depend only on the base composition of the site and the divergence distances between species, but also on location constraints, neighboring functional sites, the InDel rate, and the GC content. While not discussed in detail, a simulation using lower GC contents showed a consistently higher or lower RTR depending on the TFBS, suggesting that the high GC content in promoter regions near the TSS is affecting the turnover rates of important functional sites (Additional data file 1). Consequently, the RTR varies not only among different functional sites and different species, but also among different instances of the same functional site upstream of different genes.

While we attempted to choose realistic model parameters and biologically meaningful functional constraints in our simulations, our estimates are certainly biased by the assumptions behind the chosen constraints, and may be substantially different from the real ones. Furthermore, the TFBS and evolution models themselves represent simplified versions of the underlying biological processes, and other factors, such as the number of replicates used in the simulation, can add some additional variation as well. We realize that the weight matrices used here as models of functional sites may not be as adequate for modeling positional dependencies as other more advanced motif models 5253; however, PWMs are a valid model for many biological motifs, are available in open-access databases, and are computationally more efficient than other advanced models. Computational efficiency is an important factor in simulation studies that are as large as this one.

Simple evolutionary changes within regulatory regions, such as turnover events affecting individual sites only, can be modeled effectively by Poisson events. We could show good agreement of this for a variety of binding sites and conditions, such as different translocation distances. In theory, one could derive closed-form solutions for the probability of these events, based on the sequence composition of the region and the composition and degeneracy of a binding site. However, with an increasing number of restrictions and dependencies of sites in complex regulatory modules, this becomes increasingly cumbersome and not straightforward. Figure 6c showed that these simple models begin to deviate as soon as we address the conservation on the module level instead of individual sites only.

One can easily think of a large number of additional parameters and configurations of functional sites that we did not explore. A tool such as PSPE will allow researchers to explore empirically a wider range of restrictions and complex configurations of regulatory regions in an efficient manner. Enhancers come in many different flavors, from highly restricted 'enhanceosomes' corresponding to ultra-conserved elements, to highly flexible 'billboard' enhancers allowing for many drastic sequence changes without apparent functional consequence 54. PSPE is available to the public and we anticipate that it will be a beneficial tool for evolutionary biologists to explore the specific characteristics and evolutionary space of particular regulatory systems. Future extensions may include an adaptation for RNA regulatory regions, including specific modeling of compensatory mutations in RNA secondary structure, incorporating transposable elements, and neighbor-dependent substitution models.

During evolution, natural selection forces impose different functional constraints on protein coding and regulatory regions. The phenomenon of frequent TFBS turnovers in regulatory regions may partially explain why comparative genomics analysis, the most powerful approach so far, has met with only limited success in identifying functional sites despite the increasing availability of whole genome sequences. TFBS turnovers may also be responsible for the weak relationship between sequence conservation and functional conservation in promoter sequences, which makes the straightforward tracing of nucleotide evolution between divergent orthologous sequences meaningless with respect to their function. Our strategy of defining conservation on the level of functional constraints such as matrix score cutoffs is similar to a recent model, which defines conservation on the level of conserved binding energy 55. In this sense, functional homology maps of regulatory regions, where mapped elements correspond to functionally equivalent sites, can be more important than strict sequence homology.

While many alignment tools have been developed so far, it is difficult to systematically evaluate and compare these tools, especially regarding their performance in aligning non-coding sequences, for which we have a limited understanding of evolutionary constraints. Studies that rigorously assess alignment tools (for example, 28) can serve as useful reference for making more informed decisions about which tool to use for which task, and can also provide important insights or suggestions for improvement of existing algorithms. Most published evaluations of alignment algorithms were based on alignment sensitivity, specificity, and accuracy, and did not address replacement turnover of functional sites in evolution. The evaluation reported here is different: instead of trying to systematically assess all different performance aspects, we focus on one particular scenario, the capability of accurately aligning conserved TFBS in promoter sequences. Specifically, our evaluation was based on two aspects: the capability of aligning functionally homologous TFBS in promoter sequences in which TFBS replacement turnovers are allowed to occur; and the capability of increasing TFBS detection power with an increase in the number of homologous aligned species.

The five tools selected for our evaluation are representatives of many existing tools of different underlying algorithms. These differences were clearly reflected in the success of aligning TFBSs, which ranked MUSCLE at the top, AVID/MAVID at the bottom, and others in between. We purposefully chose transcription factors with long binding sites, and required strong conservation of orthologous sites (that is, a high matrix score threshold for each site). Furthermore, while our choice of constraints allowed for turnover events, it did not allow for a shuffling of sites, which none of the programs can take into account. Yet, our results suggest that the ability of existing tools to detect functionally homologous elements decreases with increasing replacement turnover rates of functional sites or, related, the sequence divergence distance. An increased divergence of the non-functional parts of the sequence does thus not necessarily help to locate individual functional binding sites, even if the sites are highly conserved and 15-20 bp long.

It is often reported that an increase in the number of species may significantly increase the power for functional site identification in comparative genomics analyses 56. On the contrary, our evaluation results show that we should be extremely cautious at this point to assume that this is a general property of many functional DNA regions and/or tools to analyze them. With the exception of DIALIGN, the TFBS detection accuracy of all tools was either decreased or relatively unchanged in most cases. This is in fact not surprising when ones take a closer look at the approaches used for multiple sequence alignment. CLUSTALW, MAVID, and MLAGAN all use the same progressive approach for aligning multiple sequences, in which intermediate alignments from the early stages are not allowed to change in later stages. That means that the mistakes that happen in an early stage of alignment will be propagated and cannot be corrected at a later stage. Since a tool based on the progressive approach can only accumulate more mistakes when aligning more sequences, it is conceivable that its performance decreases as the number of species increases. MUSCLE employs an improved progressive approach that allows changes in the alignment of sub-groups in a recursive refinement process, which explains why MUSCLE did not show a significant decrease in performance as the number of species increased. It is conceivable, however, that the particular choice of species, and the order in which they are presented to a phylogenetic aligner, may significantly change the accuracy of these approaches. DIALIGN is the only tool surveyed here that does not use the progressive approach. Instead, it assembles the whole alignment by greedily finding all consistent segments of significant similarity from all sequences 57, which allows DIALIGN to be able to take advantage of the information from additional species. While these features of DIALIGN are interesting, there is still much room for improvement as its overall performance is no better than MUSCLE.

We want to stress that the tools in this study were not specifically developed for the alignment of non-coding regions. In fact, some design principles may be counterproductive for this task: whole genome alignments are built to provide fast comparative maps and are certainly able to detect coding conservation. The progressive aligners in our evaluation are meant to provide the phylogenetic history, that is, to compute an accurate alignment of bases that are derived from the same nucleotide in the ancestral genome. Yet, there is no doubt that many researchers currently use these tools in studies concerning gene regulatory sequences, and we hope that this evaluation provides clues about what to expect if they are used in this way. We aimed to include a representative subset of tools fast enough to perform extensive comparisons. We do not expect this to have introduced a systematic bias, but of course some recently developed aligners (for example, TBA 58, Prank 59, or Probcons 60) may perform differently to our selected set.

The objective and systematic evaluation of alignment tools is a challenging task, in particular for an assessment on non-coding sequences whose actual functional and evolutionary mechanisms remain largely unknown. Since we simulated data under a set of specific conditions that are unlikely to represent all actual scenarios, one should carefully interpret our comparison results. For example, our study did not consider the ability of a tool to deal with very large insertions and deletions because of few large insertions/deletions in our simulated data. Furthermore, we were very conservative in our constraints, and, for example, allowed for turnover, but not for a shuffling of sites. Our results are therefore a rather optimistic estimate, and performance on real promoters with shorter sites that do not preserve their order can be expected to be significantly worse. We are also aware that the criteria for tool performance can be different in a different study depending on its objectives. Therefore, our results may not be applicable for some studies, such as the estimation of divergence distances between species. For such cases, the recent evaluation by Pollard et al. 29 may be a better reference.

To generate biologically relevant ancestral sequences, we used a 3rd order Markov model to generate background sequences of ancestral promoters. We trained the background Markov model on a large real dataset of regulatory sequences extracted from the NCBI human RefSeq database (build 35). The dataset consists of 25,088 human promoter sequences each spanning a region of 500 bp immediately upstream of the transcription start sites. The base frequencies of four nucleotides were also estimated on this dataset.

We obtained 127 experimentally confirmed E2F regulated genes from a previous publication 42. We removed the genes for which we were not able to extract their promoter sequences from NCBI, and extracted 500 bp long promoter sequences upstream of their annotated transcription start site. We then identified potential E2F sites in each promoter sequence using the PWM model. We removed those genes that had either zero or more than one E2F binding site based on the cutoff score of 0.92 given in Figure 3. The remaining 11 genes are given in Additional data file 1 and were used as ancestral sequences for our simulation study.

We used the PWM, a generic and widely used model for DNA motifs, to represent functional TFBSs. The PWM is generally given by a matrix with frequencies (or weights) of the four nucleotides at each position. While there are several different methods to calculate a motif score, we used a scoring function similar to the one proposed by 64 and defined by:

S c o r e = ∑ i = 1 w θ i × f i b ∑ i = 1 w θ i × max ⁡ b ∈ ( A , C , G , T ) f i b where  θ i = 1 + ln ⁡ 4 × ∑ b