Comparison of two completely sequenced genomes sheds lights on the genetic basis of differences in biological traits between species. Of particular interest is the comparison of complete sets of genes and gene products encoded in two genomes. Manual comparison is important but time-consuming and labor-intensive at the whole-genome scale and thus must be aided by automated approaches.

Unambiguous automated comparison requires that both genomes be annotated with the same structured, controlled vocabulary. Currently, the most common choice for such a vocabulary is gene ontology (GO) 1. The November 15, 2005 version of GO contained 19,025 terms in three hierarchical structures—as Directed Acyclic Graphs (DAGs)—termed Biological Processes, Cellular Components, and Molecular Functions. Every branch in the graph represents a biological concept progressing from general to specialized with increasing graph depth. The depth of the branches in the graphs varies, with levels ranging from 2 to 15.

The GO web site currently lists 31 genomes that have been annotated with GO 2. The annotations that are of the highest quality and updated most frequently are usually carried out by researchers who sequence and study a particular species; these annotations are primarily stored in species-specific databases such as SGD 3 for Saccharomyces cerevisiae, FlyBase 4 for Drosophila melanogaster, WormBase 5 for Caenorhabditis elegans, MGI 6 for Mus musculus, and TAIR 7 for Arabidopsis thaliana. Since these species-specific databases are located in different sites on the web, there is need for integrated, searchable databases that contain annotations for multiple species. The GO Consortium has developed such a resource, called AMIGO 8, that allows users to search and browse GO annotations integrated from many species-specific databases. Additionally, the European Bioinformatics Institute (EBI) has developed the Gene Ontology Annotation (GOA) database 9 that provides GO annotations for non-redundant proteins from many species in UniProt 1011. We compare these two resources in the Methods section. In addition to sequences annotated with GO, 15,754 functional domains in the InterPro domain database 12 have been linked to 2,627 GO terms 13.

Using the above-mentioned resources, there are two main types of methods developed to automatically annotate new gene products with GO terms: sequence similarity-based methods such as GOFigure 14, Goblet 15, OntoBlast 16, GOtcha 17, and Blast2GO 18, and sequence domain-based methods such as InterProScan 19 and GOTrees 20. For genome-scale GO annotations the similarity-based, in particular BLAST-based methods have been the preferred choice 1721222324. BLAST is significantly faster than InterProScan and can annotate many more GO terms than InterProScan can. A recent evaluation showed that assigning GO terms of the top BLAST hit gave satisfactory results when compared with several more complex methods 25. Thus we chose the BLAST approach in our work.

After the sets of genes encoded in the two genomes are annotated with GO, they can then be compared. The goal is to find functional categories that differ between the two genomes, which may explain differences in biological traits or suggest interesting families for further detailed investigation. The most common practice is to use tools such as GOslim 2627 to tally the number of genes that fall within each functional category at the first level under Biological Processes, Cellular Components, and Molecular Functions, and then to compare between the two genomes. Because the two genomes usually differ in size, the absolute numbers of genes in each functional category need to be weighted before they are compared; they are often divided by the total number of genes in the respective genomes 282930. The results of the unweighted and weighted comparisons are usually presented as bar charts or fold changes.

The unweighted and weighted GO-based genome comparisons, although useful, have two drawbacks. First, focusing only on the high-level functional categories may miss differences that are detectable only at more refined levels. Second, bar charts or fold changes alone are not sufficient to separate true functional differences from those occurring by chance; thus, statistical testing of significance is necessary. Lessons can be learned from another, more extensively researched application of GO—the detection of significantly enriched GO categories in a set of co-expressed or differentially expressed genes in microarray experiments. Several tools have been developed to search complete GO trees (rather than just the high levels) and apply statistical testing of significance (e.g., Onto-Express 31; FatiGO 32; for an evaluation of these tools, see ref. 33).

Contrary to the situation in microarray analysis, there is a surprising lack of statistical approaches for GO-based comparison of two genomes. Here we propose such a statistical approach to find reliable differences between the complete sets of genes encoded in two genomes at all levels of GO. For each GO term the abundance of genes in the two genomes is compared using chi-squared test followed by false discovery rate (FDR) correction. Furthermore, to analyze the reliability of the differences detected, we studied two important issues. First, when new sequences are assigned GO terms by similarity (as determined by BLAST) to other sequences having known GO assignments, the choice of BLAST cutoff may affect the results. We therefore analyzed the effects of employing a wide range of BLAST cutoffs. Second, we studied how the results vary when only subsets of the genes were used. To our knowledge, our work is the first to address all the aforementioned issues.

We used this statistical approach to compare two cyanobacterial genomes, Synechocystis sp. PCC6803 and Anabaena sp. PCC7120. Cyanobacteria (also called blue-green bacteria, blue-green algae, cyanophyceae, or cyanophytes) are important model organisms for the study of photosynthesis, nitrogen fixation, evolution of plant plastids, and survival in diverse environments 3435363738394041. Two of the most widely studied cyanobateria species are Synechocystis sp. PCC6803 and Anabaena sp. PCC7120. PCC6803 is a fresh water unicellular cyanobacterium incapable of nitrogen fixation 42; PCC7120 is a filamentous, heterocyst-forming cyanobacterium that has long been used to study the genetics and physiology of cellular differentiation, pattern formation, and nitrogen fixation 43. These interesting biological differences as well as the appropriate evolutionary distance between PCC6803 and PCC7120 make them a popular pair of species to compare and contrast3444454647484950. We compared PCC6803 and PCC7120 genomes using our statistical method and evaluated the detected statistically significant differences against known biological differences. To analyze how results change when only subsets of the genes are used, a larger set of statistically significant differences is desirable and we used the comparison of human vs. mouse and that of Saccharomyces cerevisiae vs. Schizosaccharomyces pombe genomes.

BLAST and InterProScan are two most widely used automated GO annotation methods. BLAST is the preferred choice for genome-scale annotation because it runs much faster and, perhaps more importantly, can annotate many more GO terms than InterProScan can. We had used InterProScan to annotate and compare PCC6803 and PCC7120, and found that it missed some important differences including "nitrogen fixation, GO:0009399". However, BLAST has its own limitations. Accurate functional assignment is difficult in cases where the match is less well defined due to lower sequence similarity 57. In future research we will investigate how to combine results from BLAST and InterProScan to improve annotation quality and use grid computing to reduce computation time.

We used BLAST E-value cutoff as the criteria in assigning GO terms. Local sequence alignment programs such as BLAST may prefer short strong matches to long weak matches and may cause inaccurate GO assignment. The strict E-value cutoff we chose in our analysis ensured the relatively high quality of the results. It was reported that a match between two sequences is most likely reliable if the alignment is at least 70 residues in length with at least 40% sequence identity 58. We investigated the quality of the HSP (High scoring Segment Pair) in our BLAST results (detail provided in the Additional file 1). With E-value cutoff 1e-20, the minimum length of HSP was 64 and the minimum sequence identity was 68%. Thus the assignments in our results were reliable. It is possible that false negatives may occur with a strict cutoff. In our analysis we prefer accuracy to coverage. Others can use different criteria depending on their individual goals. The statistical testing we proposed in this paper is independent of the GO assignment method. We suggest doing the comparison and comparing the results using different E-value cutoffs and different subsets of the input gene sets to identify the most reliable differences between two genomes.

In any GO analysis, the quality of the original GO annotation is critical. The GO annotation data are continuously expanded; however, the present data are incomplete and noisy 59, and the annotation quality is uneven, with a mix of literature-supported annotations and those inferred automatically. We did not modify the GO annotation data for our present study, but further research will consider the quality of the original GO annotations when assessing the reliability of the results. One limitation of our approach is that it only compared the number of genes in each functional category. It cannot capture differences in the level of gene expression. Another inherent limitation of GO is that it does not map directly to pathways. As a result GO-based comparison cannot detect differences at the pathway level. We have recently used the KEGG Orthology (KO) as an alternative controlled vocabulary in a KO-Based Annotation System (KOBAS) and demonstrated that KOBAS is effective in automated annotation and pathway identification 60. In future research we will investigate KO-based comparison to compare two genomes at the pathway level.

Our goal is to achieve higher confidence in the differences detected between two genomes. Towards this end, we applied rigorous statistical testing followed by FDR correction instead of simply relying on fold changes. We also tested a wide range of BLAST cutoff values and different subsets of the input genes to provide additional measures of confidence in the results. If results beyond those having the highest confidence are required, then the cutoff values can be relaxed. The advantage of the statistical approach presented here is that, no matter what cutoff values are chosen, the resulting p-values, q-values, and sampling analysis can be used to assess the confidence in the results.

There are other procedures available to correct false positive rates resulting from multiple testing, including the Bonferroni correction, Sidak stepwise correction, Holm stepwise correction, Hochberg's stepwise correction, and others 6162. We chose the FDR correction because of its overall high quality and computational speed 6364. It is also the most common procedure used in GO-related and microarray analyses 626566.

Contrary to the situation in microarray analysis, there is a surprising lack of statistical approaches used in GO-based comparison of two complete genomes. Our work is the first to propose and test a statistical approach to comparing the complete sets of genes in two whole genomes at all levels of GO and study the effect of varying BLAST cutoffs and using subset of the input gene sets. We believe that such an approach can provide a measure of confidence in the identified differences and help ensure the reliability of the results.

Supplementary materials and related programs for the paper are provided on-line [See Additional file 1].

ZC and LW conceived of the study; ZC carried out most of the implementation and analysis; XM and LW participated in the analysis; SL participated in the design of statistical tests. All authors participated in preparation of the manuscript.