A commonly used database of experimentally verified transcription factor binding sites is Transfac 3. The release from May 2004 provides 694 position frequency matrices (PFMs) covering vertebrates, plants, insects and fungi. Recently, a publicly available Jaspar database 4 was compiled with 108 PFMs associated mainly to vertebrates. For our large-scale statistical analysis we discarded all matrices with inconsistencies, for example matrices, where the number of sites aligned to construct the matrix (sample size) could not be determined. Furthermore, we excluded rather poor matrices with a length below 6 bases or a sample size below 5. After these consistency checks and filtering steps we arrived at 637 different matrices for Transfac and 103 matrices for Jaspar. All the matrices can be characterized by their length, the sample size, and the information content 22 (Tab. 1).

For each possible overlap (of at least 6 bases) of two PFMs we count the number of corresponding columns which are statistically independent. This task can be addressed by the homogeneity test using the χ² measure with 3 degrees of freedom. The application of PFMs for the characterization of binding sites implies that the nucleotide positions are regarded as independent. Even though statistical dependencies between positions are known 232425 the assumption of independent positions is a rather good approximation 126. In the following we denote by f_b,i and g_b,i the entries of the overlapping parts of the two frequency matrices to be compared. The index i refers to the base position along the matrices and b enumerates the four nucleotides A, C, G and T. The χ² distance at the position i is then given by:

χ 2 = ∑ b = A , C , G , T ( N g , i f b , i − N f , i g b , i ) 2 N f , i N g , i ( f b , i + g b , i ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaHhpWydaahaaWcbeqaaiabikdaYaaakiabg2da9maaqafabaWaaSaaaeaacqGGOaakcqWGobGtdaWgaaWcbaGaem4zaCMaeiilaWIaemyAaKgabeaakiabdAgaMnaaBaaaleaacqWGIbGycqGGSaalcqWGPbqAaeqaaOGaeyOeI0IaemOta40aaSbaaSqaaiabdAgaMjabcYcaSiabdMgaPbqabaGccqWGNbWzdaWgaaWcbaGaemOyaiMaeiilaWIaemyAaKgabeaakiabcMcaPmaaCaaaleqabaGaeGOmaidaaaGcbaGaemOta40aaSbaaSqaaiabdAgaMjabcYcaSiabdMgaPbqabaGccqWGobGtdaWgaaWcbaGaem4zaCMaeiilaWIaemyAaKgabeaakiabcIcaOiabdAgaMnaaBaaaleaacqWGIbGycqGGSaalcqWGPbqAaeqaaOGaey4kaSIaem4zaC2aaSbaaSqaaiabdkgaIjabcYcaSiabdMgaPbqabaGccqGGPaqkaaaaleaacqWGIbGycqGH9aqpcqqGbbqqcqGGSaalcqqGdbWqcqGGSaalcqqGhbWrcqGGSaalcqqGubavaeqaniabggHiLdaaaa@6A6E@

where N_f,i = ∑_bf_b,i and N_g,i = ∑_bg_b,i are the sample sizes of the matrices columns at position i. If χ² exceeds the threshold of χth2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaHhpWydaqhaaWcbaGaeeiDaqNaeeiAaGgabaGaeGOmaidaaaaa@3248@ (p = 0.05) = 7.81 the null hypothesis that the base counts in both columns are from the same distribution is rejected with a p-value of 0.05. In order to simplify the analysis we simply count the number of significantly different positions. The example in Fig. 1 shows that for an appropriate alignment (with shift = 3) of the two matrices all χ²-values are below the χth2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaHhpWydaqhaaWcbaGaeeiDaqNaeeiAaGgabaGaeGOmaidaaaaa@3248@ threshold and hence no column appears to be different. Although the counts in some columns look quite different the limited sample size allows no statistically significant discrimination.

Obviously, the number of significantly different columns depends on the relative position of both matrices. In our algorithm we study all possible alignments with a minimum overlap of 6 bases and containing at least 75% of the information content of each matrix. We calculate the minimal number of different positions among these alignments. We call this number D and interpret it as the distance between the compared matrices. Fig. 1 illustrates that for a correct alignment of the ATF and CREB a distance D = 0 is obtained whereas other alignments lead to statistically significant different columns.

An advantage of the distance measure we use in comparison to earlier studies 15171920 is the emphasis on the limited sample size of many matrices. Only few binding sites, such as those recognized by the Sp1 factor, are characterized by hundreds of experimentally verified sites. The more common sample size is around 15–20 (see Tab. 1) and, thus, it is much more difficult to distinguish matrices. The χ² measure leading to the distance D takes into account the limited sample size in a statistically well defined manner. The proposed measure could be generalized by allowing gaps, using the sum of scores or by taking the number of possible shifts into account. Since we studied in this paper only rather strong similarities our simple discrete threshold D ≤ 1 was sufficient.

The information on experimentally verified binding sites stored in PFMs can be exploited to predict novel sites. For this purpose position weight matrices (PWMs) can be constructed from the counts f_b,i in the following manner 127. First, the probability p_b,i of a base b at a given position i is given by:

p b , i = f b , i + s b N i + ∑ b ′ = A , C , G , T s b ′ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGWbaCdaWgaaWcbaGaemOyaiMaeiilaWIaemyAaKgabeaakiabg2da9maalaaabaGaemOzay2aaSbaaSqaaiabdkgaIjabcYcaSiabdMgaPbqabaGccqGHRaWkcqWGZbWCdaWgaaWcbaGaemOyaigabeaaaOqaaiabd6eaonaaBaaaleaacqWGPbqAaeqaaOGaey4kaSYaaabeaeaacqWGZbWCdaWgaaWcbaGafmOyaiMbauaaaeqaaaqaaiqbdkgaIzaafaGaeyypa0JaeeyqaeKaeiilaWIaee4qamKaeiilaWIaee4raCKaeiilaWIaeeivaqfabeqdcqGHris5aaaaaaa@4D8D@

where N_i = ∑_b' f_b',i denotes the sample size at the position i leading to the relative frequency fb,iNi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWcaaqaaiabdAgaMnaaBaaaleaacqWGIbGycqGGSaalcqWGPbqAaeqaaaGcbaGaemOta40aaSbaaSqaaiabdMgaPbqabaaaaaaa@347B@. This estimator is modified using pseudo-counts s_b. As suggested earlier 28 we choose sb = Ni4 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWcaaqaamaakaaabaGaemOta40aaSbaaSqaaiabdMgaPbqabaaabeaaaOqaaiabisda0aaaaaa@3078@, i.e. the pseudo-count is proportional to the standard deviation of the counted frequencies. Such a choice of relatively large pseudo-counts has a pronounced effect on PWMs with a small sample size. Due to the pseudo-counts the estimated probabilities are strictly positive even if zeros appear in the PFM. From the estimated probabilities p_b,i we obtain the weights w_b,i as follows:

w b , i = log ⁡ 2 p b , i r b , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWG3bWDdaWgaaWcbaGaemOyaiMaeiilaWIaemyAaKgabeaakiabg2da9iGbcYgaSjabc+gaVjabcEgaNnaaBaaaleaacqaIYaGmaeqaaOWaaSaaaeaacqWGWbaCdaWgaaWcbaGaemOyaiMaeiilaWIaemyAaKgabeaaaOqaaiabdkhaYnaaBaaaleaacqWGIbGyaeqaaaaakiabcYcaSaaa@4134@

where r_b refers to the a priori probability to find a base b in the DNA sequence. Consequently, the weights w_b,i represent log-likelihood ratios to find a base b at a position i. Finally, the score S_k around the position k of a test DNA sequence is a sum of the weights corresponding to bases observed in the DNA sequence at the subsequent positions starting from the position k. The sum S_k is computed for each position k of the matrix along the DNA sequence. High positive scores S_k indicate locations in the test DNA sequence with strong binding affinities whereas zero or negative scores are found elsewhere (Fig. 2).

This widely used technique of score calculation leads immediately to the second similarity measure (similar in spirit to the method used in 18, but modified to take into account the sample sizes of compared matrices). For two given matrices f and g we can directly obtain the corresponding scores Skf MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGtbWudaqhaaWcbaGaem4AaSgabaGaemOzaygaaaaa@30BC@ and Skf MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGtbWudaqhaaWcbaGaem4AaSgabaGaemOzaygaaaaa@30BC@ along all positions k in a given test DNA sequence. If the weight matrices are highly similar we expect positive peaks at nearly the same positions, i.e. a prediction of nearly the same set of binding sites. In order to quantify the similarity of both matrices we calculate the Pearson correlation coefficient along a test sequence. Here we also consider all possible relative shifts between two PWMs (with a minimum overlap of 6 bases) and then take the maximum correlation coefficient as the similarity measure C of the two matrices. We have found, that the correlation coefficients do not depend strongly on the value of the pseudo-counts and reflect mainly the relevant rare peaks.

In this paper we take as the test DNA sequence a random sequence with equidistributed bases. For specific applications it might be appropriate to use other test sequences such as upstream regions of the genes of interest.

Sensitivity and specificity of different methods for measuring similarities of profiles recognized by transcription factors were assessed as follows: since large sets of experimentally verified similar matrix pairs are not available, artificial sets were prepared. A representative initial matrix (either ATF or CREB) was resampled to construct a set of matrices. On average we probed the initial matrix 18 times (which corresponds to the median sample size of Transfac matrices). In order to study varying sample sizes for each generated matrix the number of samples was randomly chosen out of the range from 13 to 21. All the matrices generated this way should be classified as similar to each other. A set with matrices dissimilar to each other was prepared by random shuffling of the contents of the initial matrix. The nucleotide counts at each position were randomly reordered as well as the order of the positions. Additionally, we take into account different lengths of the matrices. Both sets were extended with random columns and the number of added columns was chosen randomly from zero to half of the length of the initial matrix. In the analysis, sensitivity was defined as the fraction of resampled matrices which were correctly identified as similar matrices. Specificity was defined as the fraction of random matrices which were identified as dissimilar. Six methods quantifying similarity of profiles were compared. The D (chi2th) and C (corr) functions were calculated as introduced above. Another score was defined as a sum of χ² obtained for each compared columns (chi2sum). Three other methods (introduced in 151720) calculate a total sum over all compared columns of: Euclidian distance (ned), column-column correlation (ccc) and scalar product of columns (sp).

For the Transfac library we analyze whether the pairs of matrices with small distances D and high correlation coefficients C coincide, i.e. for what matrices the two measures give consistent results. Fig. 3 shows histograms of correlation coefficients C for matrices with distances D = 0, 1, 2. It turns out that there are many pairs of matrices with D = 0 and large values of C (see the right peak in the upper panel of Fig. 3). For such matrices the differences between their columns are negligible and predicted binding sites are essentially identical.

There are, however, also many pairs of matrices with D = 0 and relatively small correlation coefficients C (see the left peak in the upper panel of Fig. 3). These pairs refer mainly to matrices with a low information content and/or small sample size. In such cases the differences between columns are not statistically significant (many Ns in both consensus sequences) but their scores along a test DNA sequence correlate only weakly. For example, matrices V$STAT4_01 and V$MEF2_01 (see Transfac) are characterised by sample sizes N = 6, N = 5 respectively and have a distance D = 0 but a correlation C = 0.20.

There are also cases with a high correlation coefficient but with a distance D > 2. Such a situation appears for large matrices for which only a part is informative. For example matrices V$GR_01 and V$PR_01 (see Transfac) have a length of 27, but only six positions constitute the core sequence (TGTTCT). Among the others positions three are significantly different, leading to a distance D = 3 but these differences affect the correlation C only weakly (C = 0.92).

Several alternative measures have been proposed. We assessed the sensitivity and the specificity of these measures, as described in methods. The results of the comparison are presented in the supplemental Fig. 4. Both the our correlation measure and the column-to-column similarity give (for an appropriate threshold) a high specificity and sensitivity. However, in some cases, as illustrated above, adding a second criteria is useful to discard pairs involving large matrices for which only a part is informative. The D measure defined here can be used for this purpose. Both introduced measures quantify different properties and complement each other. Although alternative choices of measures might have been done, the advantage of using the correlation C is its implicit normalisation (the results do not depend much on the length and the sample size of the matrices) and the advantage of the distance D is its easy interpretation (number of different columns). Therefore, in the following, we focus on the most similar matrices based on the distance D and correlation C measures.

Here we study the matrices of both Jaspar and Transfac databases. We consider pairs of matrices for which D ≤ 1 and C ≥ 0.8 as highly similar. These stringent thresholds were chosen to identify the most obvious similarities and they imply that the matrices are almost indistinguishable from a statistical point of view and that their scores along DNA sequences are strongly correlated. We verified that for all these pairs of matrices both similarity measures select the same relative shift of the corresponding matrices.

Fig. 4 shows an overview of all such matrices. Even though details of these clusters are only readable in the supplementary material (Fig. 1) the graph reveals interesting properties: The connecting lines visualizing high similarity join Jaspar matrices (ellipses) with Transfac matrices (boxes) in many cases. Consequently, our technique allows an automatic "alignment" of these collections of matrices. This is not a trivial task since the naming conventions used in the databases is different, and thus finding matrices corresponding to each other requires expert knowledge. We find that 84 matrices from Jaspar have counterparts in Transfac with D ≤ 1 and C ≥ 0.8. Another 16 matrices have somewhat smaller similarities D ≤ 3 and C ≥ 0.6. Only the Jaspar matrices P_HMG-1, P_HMG-IY and V_Ghlf, have no obvious "partners" in Transfac. A complete list of Transfac-Jaspar matrix pairs with high similarities is provided in the supplementary material (Tab. 1). Lists for other thresholds or other sets of matrices can be calculated through our web interface 29.

In addition to the edges between Transfac and Jaspar matrices there are many clusters containing multiple Transfac or Jaspar matrices. These clusters reflect pronounced similarities in the matrix collections. There are for example, matrices of the same transcription factor with different degrees of stringency (see for instance AP1 matrices). Moreover, different transcription factors of certain families have almost identical binding motifs (see for example Myc-Max, USF and ARNT). A complete list of all clusters is provided in the supplementary material (Tab. S2). An interesting collection of structural classes of transcription factors has been compiled recently by Sandelin and Wasserman 17. Consistent with their results we find also clusters of the ETS family (see cluster 2 in Tab. S2, also enlarged in Fig. 5b), bHLH transcription factors (cluster 15), and REL family (cluster 5).

In Fig. 5 we present enlargements of two selected clusters representing the GATA (panel a) and ETS (panel b) transcription factors family. The high similarity of these matrices cannot be directly noticed by inspection of names or consensus sequences. Furthermore, subgroups might be detected using our statistical approach. For example, the GATA cluster reveals that the Jaspar matrix has particularly high similarity to the Transfac entries GATA1_02, GATA3_01 and GATA6_01, but less similarities to other members of the GATA class. The clusters visualized in Fig. 4 and Fig. 5 can be exploited to reduce the number of matrices. Highly similar matrices match a DNA sequence either both or not at all. Therefore, one could construct "consensus matrices" as in 17 or one might select representative matrices in each cluster. In this way the number of overlapping predictions in the search for transcription factor binding sites can be decreased 17.

A careful inspection of the clusters found automatically by our similarity analysis might reveal unexpected similarities pointing to possible cross-talks of different signaling cascades on the level of transcriptional regulation. As an example we discuss the regulation of circadian clock genes and cell cycle control 3031. In both processes bHLH transcription factors bind as dimers to E-boxes. The corresponding Myc-Max cluster appeared already in Fig. 4 (the largest cluster). In the mammalian circadian clock the CLOCK-BMAL1 dimer regulates clock genes such as Per1, Per2, Per3, Cry1 and Cry2. We found no matrix in Transfac or Jaspar describing explicitly the binding sites of CLOCK-BMAL1. Consequently, we constructed such matrices ourselves in two different ways. On one hand we collected 9 experimentally verified binding sites from 7 different clock genes 3233343536. On the other hand, we took from a SELEX experiment 10 sequences with high affinities to the CLOCK-BMAL1 dimer 37.

Both matrices are visualized in Fig. 6a. Details of the matrix construction are given in the supplementary material (Tab. S3). Both matrices contain the E-box consensus motif CACGTG but differ in the flanking regions.

Fig. 6b shows that these novel matrices have highly similar counterparts in Transfac (NMYC, MYC, USF). Consequently, cross-talk of the circadian clock with cell cycle regulation and tumor genesis can be expected at the level of transcriptional control. Indeed, the success of chronotherapies and recent detailed studies on cross-talk underline the dependence of circadian rhythms with tumor growth 38. Also in the process of liver regeneration a pronounced effect of the circadian clock on cell cycle control has been found 39. This example illustrates that a careful SELEX experiment combined with a mapping of the resulting matrix to known matrices can reveal possible functions of the corresponding transcription factor.