To conduct computational experiments we used an unloaded system of 36 4-way AMD 2.4 GHz Opteron processors with 8 GB of main memory per node which are interconnected by an Infiniband switch. Parafit and DistPCoA were compiled using g77 -ffixed-line-length-0 -ff90-intrinsics-delete -03. AxParafit and AxPcoords were compiled with -03 -fomit-frame-pointer -funroll-loops and linked with the AMD ACML library. We also assessed additional compiler optimizations (-fomit-frame-pointer, -funroll-loops, -m64, -march = k8) with g77 for Fortran, which actually lead to performance decrease of Parafit and DistPCoA (data not shown).

In order to assess performance of AxParafit we extracted subsets from a large empirical dataset with more than 30,000 host-associate links (collected from entries in the EMBL database 29), which we are currently analyzing with our tools. We sampled square association matrices A, i.e., n₁ = n₂ of dimensions 128, 256, 512, 1,024, and 2,048. The number nonZero(A) was 128, 256, 512, 1,024, and 2,048 respectively. The number of permutations p was set to 99, 99, 9, 2, and 2 respectively. A complete test on the dataset of size 4,096 was not conducted with Parafit due to the extremely long run-times on n₁ = n₂ = 2, 048 which already amounts to 19.9 days compared to 7.7 hours required by AxParafit.

To test AxPcoords we used the same compiler switches as indicated above and a subset of the square association matrices with nonZero(A) amounting to 512, 1,024, 2,048, and 4,096 respectively.

We collected a large sample of associations of smut fungi and their host plants. Smut fungi comprise more than 1,500 species of obligate phytoparasites and are arranged in the taxa Entorrhizomycetes, Microbotryales, and Ustilaginomycotina. These parasites cause syndromes such as dark, powdery appearance of the mature spore masses or may even lead to plant deformation in some cases 3031. The Ustilaginomycotina also comprise obligate plant parasites with distinct morphology 30.

With a few exceptions, hosts of smut fungi belong to the Angiosperms 30. For economically important hosts, such as barley and other cereals, smut fungi may cause considerable yield losses (see e.g., 32). Phylogeny and taxonomy of genera and higher ranks has been derived from sound molecular and ultrastructural data in recent years (see 30 and references therein). However, apart from the work presented in 14, co-phylogenetic analysis of smut fungi have so far been restricted to single genera with comparatively few species 3334.

In addition to the host plant index for European smut fungi 3135 that has been used in 14, information on smut fungus-host plant associations was extracted from the following publications: Bauer et al. 363738, Begerow et al. 3339, De Beer et al. 40, Hendrichs et al. 41, Nannfeldt 42, Piepenbring 43, Scholz and Scholz 44, an unpublished manuscript by K. Vanky (Smut fungi of the Indian subcontinent; Vanky, personal communication), and Vanky and McKenzie 45. Moreover, we included information contained in the "specific host" entries of the complete collection of core nucleotide sequences for Entorrhizomycetes, Microbotryales, and Ustilaginomycotina downloaded from GenBank 46 on September 01, 2007 (12,815 sequences). Parasite taxon names were corrected using Vanky's synonym-list 35. Synonyms for host taxon names were obtained from Palese and Moser 47.

Including synonyms, our data set contained 3,912 different fungus-plant associations. In order to retrieve taxon IDs and to construct taxonomy trees for hosts and parasites 14, we used the NCBI taxonomy release of September 01, 2007. For host and parasite species names that were not found in the NCBI taxonomy, the search was repeated after reducing the taxon name to the respective genus. In this way, a total of 2,362 different associations could be identified that covers 413 smut fungi and 1,400 host plants. Thus, the dataset assembled was more than three times larger than the one recently analyzed in 14, which contained 645 associations, corresponding to 140 smut fungi and 437 host plants. The Parafit analysis of this comparatively small dataset took already more than a week. For both hosts and parasites, two trees were constructed, one tree with branch lengths corresponding to the "true" (denoted as W for Weighted) taxonomical distance 14 and one with all branch lengths set to 1 (denoted as U for Un-weighted/Uniform). As outlined on page 4 the computational complexity of AxParafit is O(nonZero(A)n₃n₄n₁p) and thus the execution time requirements for this larger dataset increase significantly.

Production runs with Parafit and AxParafit on an initial version of our dataset were started on August 29, 2007. While the Parafit inferences with 99 permutations on this initial dataset were still running at the time of writing this manuscript(September 9, 2007), the parallel AxParafit run with 99 permutations terminated within less than 480 seconds on 128 CPUs of the Infiniband cluster. This made the results available immediately and allowed us to identify a bug in the data collection script. The buggy version of this script did not take the presence of non-unique scientific taxon names, (e.g.,Setaria (Magnoliophyta, Poales) and Setaria (Nematoda, Filarioidea)) into account to identify NCBI taxon IDs. Such errors are unfortunately typical and frequent in Bioinformatics analysis pipelines. As a typical example of such errors consider the retraction of "Measures of Clade Confidence Do Not Correlate with Accuracy of Phylogenetic Trees" by Barry G. Hall due to an error in a perl script 48.

In addition to the rapid detection of input data errors, the significant performance gains obtained by sequential optimization and parallelism allow for the assessment of different program parameters and analysis options, such as trees with different patristic distances (U and W trees) as well as the impact of the number of permutations on the results (AxParafit was run with 99/999/9,999 permutations on the U and W data), i.e., a significantly more thorough and detailed analysis.

The absolute execution times for AxParafit on 128 CPUs for 99/999/9,999 permutations are indicated in Table 1. Essentially, 99 permutations could be conducted within 7 minutes, 999 permutations in much less than 2 hours, and 9,999 permutations overnight in about 12 hours such that the whole study, including the detection of the script error and the analysis of the results could be completed in less than a week. As indicated in Table 2 there are a number of links (max. 48 out of 2,362 ≈ 2%) that are not uniformly significant or uniformly insignificant at low p-values between analyses with a distinct number of permutations. AxParafit therefore allows for rapid and much more thorough computation and analyses of large co-phylogenetic datasets. The results indicate that U-based analyses are in general more sensitive to the number of permutations than W-based runs. Note that the number of host/parasite eigenvectors for U (1,390/411) was higher than for W (1,200/372), which explains the longer execution times and potentially the larger differences in significance values.

Table 3 indicates the number of different significant links between the U- and W-based analyses for various p-values. The table indicates that there is no clear tendency for differences to decrease with increasing number of permutations.

In the following, we focus on the results obtained with 9,999 permutations and branch lengths scaled in terms of taxonomical distances (W-labeled results). The global test indicates a highly significant co-phylogenetic relationship (p = 0.0001). An overview of the results for individual host-parasite links based on the smut fungi genera is provided in Figure 6. Major taxonomic groups of host and parasites are indicated according to the NCBI taxonomy release used. Based on a significance threshold of p = 0.05 and the ParafitLink1 statistics 15, a total of 578 insignificant and 1,784 significant associations is obtained. As in our earlier study 14, genera of smut fungi are rather uniform with respect to their significance values, which facilitates the identification of a general distribution pattern with respect to significant and insignificant links, i.e., the "deep co-phylogeny" of smut fungi.

The single most important factor appears to be whether the hosts belong to the monocots (i.e.,Liliopsida) or not. Entorrhiza species, which are taxonomically isolated, mostly are linked with monocots (Poales) and do thus not contribute significantly to the overall fit between host and parasite phylogenies. In the case of Microbotryales, the majority of taxa are pathogenic of core eudicots, resulting in significant links. Fewer associations with monocots (mostly Poales) are present, which are considered insignificant. The same pattern can be observed in the class Exobasidiomycetes within Ustilaginomycotina: A minority of host-parasite links is within monocots (Poales, but also other orders), which are considered insignificant, whereas the associations with other hosts (Selaginellales, basal Magnoliophyta, magnoliids, and stem and core eudicots) are significant. Inverse relationships are present in the class Ustilaginomycetes within Ustilaginomycotina. Here, most species infect monocots, mainly Poales, significantly increasing the congruence between host and parasite taxonomy trees, whereas the associations with core eudicots appear to be insignificant.

Accordingly, the current analysis that is based on a considerably larger empirical sample (e.g., 66 instead of 25 included genera of smut fungi) confirms earlier results 14. Therefore, we can generalize the observation that the difference between Poales and non-Poales hosts is crucial for the distribution of significance values to the distinction between monocot and non-monocot hosts. We also observe a small number of exceptions from this general pattern. For instance, in Urocystis (Ustilaginomycetes), which occurs on a variety of host groups, the links with stem eudicots (species of Ranunculaceae) are significant, and a single link with monocots (PACCAD clade within Poaceae) is judged as insignificant. Thus, rather subtle details of the host-parasite relationships, such as the presence of Urocystis on several closely related Ranunculaceae hosts and its presence on distantly related hosts within Poaceae, are recognized by the AxParafit algorithm, and the uniform overall pattern does not merely reflect the relatively low topological resolution present in the taxonomy trees.

Some of the results obtained may also be due to flaws in the taxonomy of the species included, particularly in the nomenclature of the parasites. For instance, Entorrhiza isoetis is most likely conspecific with Ustilago isoetis 31. At present it is even doubtful whether this species belongs to smut fungi (R. Bauer, personal communication). Thus, the associations with Isoetes (Lycopodiophyta) mentioned in Scholz and Scholz 44, which show different significance values than the majority of hosts links in either Ustilago or Entorrhiza, are dubious. Likewise, the exceptional associations of Entyloma with monocots are probably due to species names that would need to be recombined into genera of the Georgefischeriales 37. Whereas these flaws have to be corrected by considering more comprehensive lists of species and synonyms in monographs and in future releases of the NCBI database, it is apparent that neither the highly significant overall co-phylogenetic relationship nor the general pattern regarding individual host-smut fungus links would be affected by the removal of the doubtful associations. Rather, their influence is overcome by the large total sample size; for each parasite genus dubious links are few relative to the total number of links or not present at all. Likewise, there are few differences in the significance between analyses with a distinct number of permutations (see Table 2). Discrepancies between U and W are also comparatively small (see Table 3). With 9,999 permutations, they are restricted to four genera of smut fungi and only affect hosts, such as Urocystis on monocots in Asparagales and Dioscoreales (details not shown), with an intermediate taxonomic position.

The analysis process presented here underlines the advantage of the large-scale approach to co-phylogenetic tests, that is enabled by AxPcoords/AxParafit. Furthermore, because many problems are more easily recognized after conducting preliminary runs, re-analysis after applying corrective measures may be necessary for many empirical datasets. Thus, efficient implementations and parallelism are of great practical importance for the analysis pipeline.