REPEATMASKER annotation either in its original format or as a UCSC table is input to REANNOTATE. If this annotation reports similarity to N different reference repetitive elements, let R = {r¹, ... r^N} denote this library set of N reference elements. Here I call a hit to the reference element r a query sequence region homologous to r and with higher sequence similarity to r than to any other reference element in R, annotated by REPEATMASKER. As an example, a visual representation of the input annotation is given in Figure 1A.

REANNOTATE constructs repetitive element models assigned to the different families in the reference library R. A given reference element r defines a set of hits along the query sequence. REANNOTATE will search for subsets of hits to r that are to be defragmented into an element model if they satisfy the criteria:

(i) Colinearity with the reference element.

(ii) Maximum span.

Criterion (i) requires that the defragmented hits be in the same orientation on the query sequence, and that they match consecutive (though not necessarily contiguous) regions of the reference element r. (Note that along the query sequence these defragmented hits need not be consecutive, in the sense that there may be other hits to r nested between them.) An example is illustrated in Figure 1B. The reference element in question is PREM2_ZM_I (from REPBASE UPDATE), and hits to this reference element along the maize sequence are highlighted in magenta. REANNOTATE constructed a model defragmenting the three innermost hits to PREM2_ZM_I, shown in red, which respectively match nucleotide positions 1-787, 783-4958, and 4956-6864 along the reference PREM2_ZM_I sequence. Note that there is a small overlap between the matching coordinates along the reference sequence (5 nucleotide between the first two matches, and 3 nucleotides between the last two), which is an ostensible deviation from strict colinearity between the three hits on the maize chromosome and the reference sequence. This situation is common and is due to uncertainty about the ends of local alignments. REANNOTATE has a user-definable tolerance parameter ϵ in the requirement for colinearity between the element model and the reference element that allows for an overlap (o) between the matching coordinates of two defragmented hits along the reference sequence, if o ≤ ϵ. (Default ϵ = 40 nucleotides; if ϵ > L/10, where L is the length of a given reference sequence r, then the tolerance margin for that family is automatically set to L/10).

Criterion (ii) requires that the span (i.e. the query sequence length from the end of the first to the beginning of the last defragmented hit) of candidate repetitive element models do not exceed a (user-definable) length δ (default δ = 40 Kb).

For each reference family in the library R, REANNOTATE searches for candidate repetitive element models that satisfy (i) and (ii), but only constructs models that additionally satisfy:

(iii) Uniqueness.

(iv) Maximum defragmentation.

(v) Recursive model nesting.

Criterion (iii) requires that constructed models comprise mutually exclusive sets of hits.

If two candidate models are such that all the hits defragmented in one model are present in the other model that defragments a higher number of hits, then criterion (iv) requires that only the model defragmenting the maximum number of hits be constructed.

If two candidate models (assigned to the same family) comprise different sets of hits with at least one hit in common, then criterion (v) requires that the candidate model whose hits span the narrower region of the query sequence be constructed. If a hierarchy of such models exists then this pairwise criterion is repeated recursively. Additionally, if two candidate models (assigned to any families) are such that a) they comprise non-intersecting sets of hits and b) they span overlapping regions of the query sequence, then criterion (v) requires that they cannot both be constructed unless the span of one candidate model lies entirely within the span of the other.

If, upon human inspection of either the original REPEATMASKER annotation or the automated re-annotation, the occurrence of fragments of a given repetitive element matching multiple reference sequences is detected, a user may supply as input to REANNOTATE a text file containing lists of "related" reference elements, so that hits to different reference elements within one such list may be considered for defragmentation into a repetitive element model. The use of this option was essential for the re-annotation and analysis of human endogenous retroviruses in Results; the ERV names equivalence lists are provided in Additional file 1.

The defragmentation procedure described in step B above applies to any high-complexity repetitive element. Both the LTR and the internal region (i.e. the sequence between the two LTRs of a complete element) of a given LTR-element family may be found as dispersed repeats. Fossil remains of a given insertion may contain only LTR sequence or only internal region (IR) sequence. REANNOTATE performs additional analysis of LTR-elements (i.e. LTR-retrotransposons and retroviruses) if the reference library contains separate entries for the LTR and IR sequences. Names of reference LTR and IR elements of the same family should be identical (unless a name equivalence list is input to REANNOTATE) apart from a suffix. Reference LTR names do not need a suffix, but they may be suffixed with either the string '-LTR' or '_LTR' (case-insensitive). Reference IR names should be suffixed with either '-int', '_int', '-I', or '_I' (case-insensitive). This is the naming convention used in REPBASE UPDATE for LTR-elements in most genomes, though currently with the notable exception of human/primate endogenous retroviruses (for the solution to this problem adopted in this study see below and Additional files 1 and 2).

Models of LTRs and IRs are separately constructed with the Defragmentation algorithm in step B. REANNOTATE then constructs higher-order models combining models of defragmented LTRs and defragmented IRs. Higher-order model construction proceeds with requirements analogous to those in step B, but now the colinearity criterion refers to the LTR-IR-LTR structure of a canonical LTR-element. An example is illustrated in Figure 1C. Recursive model nesting (as in B) is used to defragment LTR-element structures within structures. Where nested structures corresponding to the same family are identified, these are resolved by mapping the coordinates of the component LTR or IR fragments onto their respective reference sequences. A nested element should interrupt the nesting element. (If there is ambiguity in the sense that a given LTR could be paired with two different LTRs and IRs, a model of a 'complete' element is constructed with the two LTRs whose sequences are most similar to each other).

If a candidate higher-order model contains only one LTR and one IR model, a higher-order model for a truncated LTR-element is constructed only if the LTR and IR are separated on the query by less than a (user-definable) distance σ (REANNOTATE default σ = 15 Kb). Furthermore, truncated higher-order models are only constructed when the constituent LTR and IR models cannot be accommodated in a model of a 'complete' LTR-element.

REANNOTATE classifies LTR-element models as either (i) 'complete', if they contain at least part of the sequence of both their original LTRs and of the IR; or (ii) 'truncated', if they are not 'complete' and not a 'solo' LTR; or (iii) a 'solo' LTR, if a model contains only sequence corresponding to a single LTR, and if this is separated from the nearest LTR or IR model – of the same family and inserted in the same orientation – by a distance greater than σ. (Note that the term 'solo LTR' is often used to mean an element that resulted from a deletion event that occurred via recombination between intra-element LTRs; such an event would preserve target site duplications (TSDs) flanking the original element, however REANNOTATE currently does not check for TSDs, and such a check would only be possible if the termini of the original element had not been truncated).

REANNOTATE will flag the possibility that LTR-element sequences have been involved in DNA rearrangements other than transposition of an entire element when an IR-LTR-IR structure is detected, i.e. two IR models flanking an LTR model of the same family, in the same orientation. Here "flanking" means that either i) the LTR model and the IR models are contiguous on the query sequence (within a tolerance margin ϵ), and that neighbouring ends have no missing sequence (within ϵ) – this excludes the possibility of a nested insertion of an IR or LTR of the same family being considered; or ii) the LTR and IR models are not contiguous on the query and they are not complete, but the length of the gaps between them equals the amount of sequence they are missing (within ϵ) – this includes structures in which the IR-LTR boundaries have been obliterated (possibly prior to the re-arrangement), or structures with segments that are not homologous to the closest library sequence; or (iii) the structure is highly symmetrical, i.e. the two IR models are equidistant from the LTR model between them (within ϵ), and provided that this separation on the query is less than σ.

After all hits to high-complexity repetitive elements have been defragmented into element models, any nested structures can be resolved by comparing the coordinate ranges of the models along the query sequence. If the span of a given model is contained within the span of another model, the former is classified as 'nested' in the latter (inferred to have inserted into the latter). An example is continued from step C and shown in Figure 1D.

REANNOTATE also provides an optional algorithm for the identification of 'truncated nesting': if one terminus of a given element model interrupts another model, the interrupting element may be classified as nested even if the interrupted element does not contain detectable sequence on both sides of the interrupting element. Truncated nesting is annotated only if there is no sequence missing from the the interrupting terminus of the interrupting element (so that a deletion spanning part of the truncated element and part of the interrupting element is not considered).

For each (structurally) complete LTR-element model constructed, REANNOTATE outputs the gapped sequence (see below) of each intra-element LTR separately. REANNOTATE then generates automated intra-element LTR alignments using the CLUSTALW(2) 39 alignment program. The number of nucleotide substitutions per site (K) between intra-element LTR sequences (and its variance) is then estimated (Figure 1E) using the Kimura 2-parameter model 40. If a rate of substitutions per site (s) is provided, the time elapsed since the insertion of a 'complete' LTR-element (t) is estimated as t=K2s MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiDaqNaeyypa0tcfa4aaSaaaeaacqWGlbWsaeaacqaIYaGmcqWGZbWCaaaaaa@326A@. In addition to the variance propagated from the estimation of K, the variance of the time estimate (σt2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0bqaaiabikdaYaaaaaa@3028@) accounts for the fact that the accumulation of nucleotide substitutions occurs stochastically over time, which can be modeled as a Poisson process; the variance in t is then estimated as σt2=(σKL)2+KL4(sL)2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0bqaaiabikdaYaaakiabg2da9KqbaoaalaaabaGaeiikaGIaeq4Wdm3aaSbaaeaacqWGlbWsaeqaaiabdYeamjabcMcaPmaaCaaabeqaaiabikdaYaaacqGHRaWkcqWGlbWscqWGmbataeaacqaI0aancqGGOaakcqWGZbWCcqWGmbatcqGGPaqkdaahaaqabeaacqaIYaGmaaaaaaaa@422E@, where σ_K is the standard deviation of the estimate of K, and L is the length of the intra-element LTR alignment.

Both the maize [GENBANK:AF123535.1] and wheat [GENBANK:AF459639.1] sequences were annotated with REPEATMASKER using a cutoff score of 200. Maize repeat sequences in the REPBASE UPDATE42 repeatmaskerlibrary version 20050112 were used as a reference library, supplemented with sequences from the TIGR ZEA REPEAT DATABASE v3.0 43. Library elements CINFUL1_ZM and CINFUL2_ZM are closely related, and hits to these elements were considered together for defragmentation into element models. In the diploid wheat analysis the monocot library from the REPBASE UPDATE repeatmaskerlibrary version 20050523 was used.

REPEATMASKER annotation was downloaded directly from the University of California at Santa Cruz (UCSC) Genome Browser web site http://www.genome.ucsc.edu.

Annotation of the human genome sequences was further processed with a custom script (Additional file 2) to suffix the names of reference LTR and IR elements for defragmentation. Additionally, a file with ERV reference name equivalence lists (Additional file 1) was input to REANNOTATE. This is because many HERV reference sequences in REPBASE UPDATE (which were used in the REPEATMASKER annotation) are closely related but may have disparate names, and for most ERV families the corresponding LTR and IR reference sequences have different names.

After integration, the sequence of a repetitive element may become fragmented by subsequent insertion/deletion events or other rearrangements. REANNOTATE generates models of repetitive element insertion events into an ancestral state of a query genomic sequence, in order to identify query sequence regions that originated in the same insertion event and that correspond to (possibly) multiple hits in the input REPEATMASKER annotation. Therefore for a given re-annotated query sequence the number of TE hits in the REPEATMASKER annotation is usually greater than the total number of repetitive element models obtained from the defragmentation of hits performed by REANNOTATE (the inferred number of insertion events). Figure 3A shows a visual representation of the REPEATMASKER annotation of multiple similarity hits to TEs along a DNA sequence. Figure 3B shows, as an example, the re-annotation of 6 similarity hits that were defragmented into the same TE model.

Fragmentation of repetitive element sequences in the REPEATMASKER annotation may also result from divergent or chimeric elements relative to the reference sequences used in the similarity searches, or from matches to closely related sequences in the reference library, rather than sequence evolution of a repetitive element since its integration into the genome. REANNOTATE provides a facility to defragment hits in this situation as well (see section "Defragmentation of chromosomal elements matching multiple reference sequences" in Implementation). Additionally, DNA re-arrangements other than transposition of an entire element (e.g. segmental duplication) may occur involving repetitive elements after their integration; in this situation a single insertion model is not adequate to describe all of a query sequence that is homologous to a given repetitive element in the re-arranged region. Currently for LTR-elements (only) REANNOTATE checks for the possibility of re-arrangements that result in multiplication of segments of elements, and generates models of such re-arrangements (see below and Implementation).

REPEATMASKER annotation implicitly contains the chromosomal distribution of sequence similarity hits to dispersed repeats along the query sequence. This first layer of re-annotation consequently contains the chromosomal distribution of repetitive element integration events.

The direct, long terminal repeats (LTRs) of any individual LTR-retrotransposon or retrovirus are created from the same parent template, and are therefore identical at the time the element integrates into the host molecule. Nucleotide substitutions in either of the two intra-element LTR sequences can accumulate as time passes, so that their sequence divergence works as a molecular timer, set to zero at the time of integration 45. REANNOTATE can generate automated intra-element LTR alignments and then estimate the number of nucleotide substitutions per site (K) that have occurred between the intra-element LTR of a given element since its insertion event. Estimates of K are a relative measure of time; but if the user can provide a rate of nucleotide substitution, estimates of K can be converted to estimates of absolute ages of (structurally) 'complete' LTR-elements. (Here by 'complete' I refer to elements for which at least a portion of each LTR is still detectable). Although direct dating of insertions is only available for 'complete' LTR-elements (and made possible by the previous Defragmentation layer of re-annotation), in nested structures bounds on the ages of other kinds of sequence elements can be estimated by using the age estimate for an overlapping LTR-element. If a 'complete' LTR-element is found in a nested cluster of repetitive elements, the previous Nesting Structure layer of re-annotation allows the placement of i) an upper bound on the age of any elements nested within, and ii) a lower bound on the age of any elements nesting, the LTR-element. Figure 3D shows examples of both direct dating of insertion events and indirectly using the nesting structure. Additionally, indirect dating of host molecules is possible in cases such as i) segmental duplications with a differential content of, and ii) haplotypes bearing insertional polymorphism for, 'complete' LTR-elements. Here a paleontological analogy is particularly apt, as the dating of both the molecular fossil and its sequence context are interrelated, just as real fossils can be dated by either absolute (e.g. radiocarbon) or relative (e.g. stratigraphic) methods. In eukaryotic genomes nesting of repetitive elements (see Background) and insertional polymorphisms 444647484950, for example] are common – in plants particularly for LTR-elements.

This layer of re-annotation implicitly contains the age distribution of 'complete' LTR-elements, and bounds on the ages of other kinds of sequences that may be found overlapping LTR-elements.

In addition to the main re-annotation output to tab-delimited text files, for visualisation and human analysis of the automated annotation REANNOTATE generates a General Feature Format (GFF) annotation file. The GFF annotation can be visualised (using a configuration file distributed with REANNOTATE) in the APOLLO genome browser 34. Figure 3C is taken from a screenshot of such a visualisation. The GFF output can be combined with any other kinds of annotation of the query sequences that are available in the GFF format.

Nesting patterns and counts of insertion events of repeats re-annotated on human chromosomes Y and 2 are summarised in Table 3. One striking result is the scarcity of TE insertions nested in satellite repeats, especially on chromosome Y. Even though, from these data, the possibility of functional constraint on satellite arrays cannot be ruled out, it is plausible that this result reflects a recent expansion of satellite repeats on human chromosome Y. Taking chromosome 2 as an example, not only the density of TE insertions into satellite sequence is almost four times that on the Y chromosome, but also there are ten times as many satellite repeats on the Y as in chromosome 2 (Table 3). The density of TE insertions into satellite arrays has been recently used to infer an age gradient for domains of such arrays around the primate X chromosome centromere 55.

Another noteworthy nesting pattern in the re-annotation is the difference between LINEs on the one hand, and SINEs and LTR-elements on the other. On chromosome 2, LINEs harbour on average over twice the density of TE insertions (of any kind) per unit of LINE sequence than do SINEs per unit of SINE sequence or LTR-elements per unit of LTR-element sequence. In addition, on chromosome 2 (and on the Y) the proportion of SINE and LTR-element insertions nested into other repetitive elements is higher than that of LINEs (Table 3). Multiple factors such as age, base composition, and insertional biases may contribute to these differences, although automated analysis of such factors is beyond what is currently implemented in REANNOTATE.

The age distribution of endogenous retroviruses (ERVs) on human chromosomes 1, 2, and Y was obtained from their automated re-annotation, and shown in Figure 6. ERVs on both chromosomes 1 and 2 have less divergent intra-element LTRs than those on chromosome Y (Wilcoxon rank sum tests, p < 10^-6), whilst there is no significant age difference between chromosomes 1 and 2 (p = 0.6). The main period(s) of retroviral activity over evolutionary time must have generated most ERV insertions on all chromosomes, therefore the older estimated ages are consistent with a faster rate of evolution on the Y than on chromosomes 1 and 2. Given that the "old" tail of the age distributions on those three chromosomes are similarly shaped, the age difference is not purely an effect due to longer persistence of ERVs on the Y. Evidence (from different methods) for a faster rate of evolution of the human Y chromosome relative to autosomes has been previously reported (e.g. 56).

In contrast to plant (cereal) genomes where LTR-element polymorphisms make the dating of these elements potentially useful for dating haplotypes from different lines within the same species, most primate ERV insertions are ancient. Using the rate of evolution for mammalian repeats estimated in 57, 2.1 × 10^-3 substitutions per site per million years, the age distributions of ERVs in the genome of the lineage leading to human peaks at around 40 Myr ago.

As REANNOTATE takes as input similarity-based (REPEATMASKER) annotation, only elements homologous to known families of repeats can be re-annotated. For genomes with poorly characterised repeat families, any similarity-based detection methodology needs to be complemented with de novo repeat discovery [62636465, for example]. Even though de novo discovery is useful for identifying uncharacterised families of repeats, it is not appropriate for genomic sequence annotation as it will fail to predict many repetitive elements that are are nested or fragmented. New families should be added to form an augmented reference library, and query sequences annotated with REPEATMASKER and REANNOTATE. Generally, sequence divergence between repetitive element lineages exposes the dependence of similarity-based annotation on the quality and comprehensiveness of the reference library. For defragmentation, the most difficult situation occurs when a chromosomal element is divergent from all available library elements. For example, if the chromosomal element is in parts homologous to two different library elements, then REannotate would construct two element models instead of one. Note that REannotate does provide a facility to correct defragmentation, provided that this situation is noticed through human inspection in the first place (see below).

Chimeric elements also present a challenging form of sequence divergence. They arise, for instance, when i) a transposable element nesting an unrelated element is mobilised, transducing the nested element; or ii) recombination between TE or ERV sequences leads to a new replication-competent element. If the progenitors of a chimeric element, but not the element itself, are both represented in the reference library, then REPEATMASKER will report separate hits to each progenitor reference, and REANNOTATE will construct separate element models for segments of the chimeric element. However, if upon human inspection of the automated annotation such chimerism is noticed, REANNOTATE does provide an option to combine hits to prescribed reference elements into element models, so that a new round of re-annotation will construct models capturing the full sequence of chimeric elements.

The optional REANNOTATE facility to defragment elements matching multiple reference sequences is also useful for solving another problem – overrepresentation of a lineage in the reference library, when the library redundantly contains very closely related sequences. Again this may cause REPEATMASKER to report matches to different reference elements that correspond to segments of the same chromosomal element. Finally, user-prescribed association of reference elements is essential for the construction of LTR-element models if the library entries for the LTR and IR representing the same family of elements have different names (as is the case for primate ERVs in REPBASE UPDATE).

However, the ideal approach for optimum similarity-based annotation is the construction of a reference library that non-redundantly represents repeat lineages as comprehensively as possible. In some cases, even when it is noticed (on human inspection) that chromosomal elements match multiple library sequences, the REannotate facility to defragment such elements will not help. This is the case when the rules for element model construction and defragmentation are violated. For example, the element labeled "3" in Table 1 (first column) and in Figure 4 was present in the original human annotation 51 with the name Sabrina_F2-2", an LTR-retrotransposon. The IR of this element was matched by three disntinct reference elements in the library used (Table 1: "SABRINA2_TM", "SABRINA3_TM", and "SABRINA_HV"). So in this case REannotate constructed three separate element models, which correspond to the element "Sabrina_F2-2" in the human annotation. The longest of these three models (labeled "3" in Figure 4) defragmented four hits to the reference element SABRINA2_TM (one IR and three LTR hits). The other two short element models are marked in Figure 4 by a horizontal bar and a "*". They correspond to the hits to SABRINA3_TM and SABRINA_HV. In the current defragmentation algorithm, these shorter hits could never be defragmented into the longer model containing the four SABRINA2_TM hits. This is because the SABRINA2_TM hits have in fact the opposite orientation on the chromosome relative to the SABRINA3_TM and SABRINA_HV hits.

REANNOTATE assembles fossil sequence fragments colinear with a given reference element into an element model, and the model assumes that such fragments are also colinear with the ancestral sequence of a repetitive element at the time of integration. DNA rearrangements involving a TE after its insertion into the genome may disrupt colinearity of its sequence with a reference element. The issue then arises as to whether to classify the re-arranged sequence as a single repetitive element. REANNOTATE will normally construct a separate model for any sequence segment violating colinearity with the reference. DNA rearrangements other than transposition (of an entire element) pose challenges for TE annotation; for example, if a segmental duplication has occurred within an element that remained replication-competent and that subsequently generated new copies, then for these new copies, colinearity with the reference element (which does not contain the duplication) is violated and two separate models are constructed. REANNOTATE does construct models of DNA rearrangements involving LTR-elements (which are based on re-arrangements of the common structure of these elements). One of the purposes of such annotation is cautioning against the validity of dating such an element, as the rearranged structure may be the result of inter-element recombination. The occurrence of post-integration, inter-element LTR recombination would invalidate the use of intra-element LTR sequence divergence as a molecular timer. It is also possible that inter-element LTR recombination may occur without altering the structure of the elements involved – in this case the re-arrangement will remain undetected by the algorithm currently implemented in REANNOTATE. Nevertheless, the aligned sequence data output by REANNOTATE can be used to reconstruct the phylogeny of all LTRs within a family: inter-element recombination would be detected if intra-element LTRs did not cluster on the phylogeny. There is scope for future implementation of algorithms for improving the detection and annotation of recombination, segmental duplication, and inversion events involving repetitive elements.