This article is part of the supplement: Selected articles from the 7th International Symposium on Bioinformatics Research and Applications (ISBRA'11)

Open Access Proceedings

A systematic comparison of genome-scale clustering algorithms

Jeremy J Jay1, John D Eblen2, Yun Zhang3, Mikael Benson4, Andy D Perkins5, Arnold M Saxton6, Brynn H Voy6, Elissa J Chesler1 and Michael A Langston6*

Author Affiliations

1 The Jackson Laboratory, Bar Harbor, ME 04609, USA

2 Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA

3 Pioneer Hi-Bred International Incorporated, Johnston, IA 50131, USA

4 Linköping University, SE-581 85, Linköping, Sweden

5 Mississippi State University, Mississippi State, MS 39762, USA

6 University of Tennessee, Knoxville, TN 37996, USA

For all author emails, please log on.

BMC Bioinformatics 2012, 13(Suppl 10):S7  doi:10.1186/1471-2105-13-S10-S7

Published: 25 June 2012



A wealth of clustering algorithms has been applied to gene co-expression experiments. These algorithms cover a broad range of approaches, from conventional techniques such as k-means and hierarchical clustering, to graphical approaches such as k-clique communities, weighted gene co-expression networks (WGCNA) and paraclique. Comparison of these methods to evaluate their relative effectiveness provides guidance to algorithm selection, development and implementation. Most prior work on comparative clustering evaluation has focused on parametric methods. Graph theoretical methods are recent additions to the tool set for the global analysis and decomposition of microarray co-expression matrices that have not generally been included in earlier methodological comparisons. In the present study, a variety of parametric and graph theoretical clustering algorithms are compared using well-characterized transcriptomic data at a genome scale from Saccharomyces cerevisiae.


For each clustering method under study, a variety of parameters were tested. Jaccard similarity was used to measure each cluster's agreement with every GO and KEGG annotation set, and the highest Jaccard score was assigned to the cluster. Clusters were grouped into small, medium, and large bins, and the Jaccard score of the top five scoring clusters in each bin were averaged and reported as the best average top 5 (BAT5) score for the particular method.


Clusters produced by each method were evaluated based upon the positive match to known pathways. This produces a readily interpretable ranking of the relative effectiveness of clustering on the genes. Methods were also tested to determine whether they were able to identify clusters consistent with those identified by other clustering methods.


Validation of clusters against known gene classifications demonstrate that for this data, graph-based techniques outperform conventional clustering approaches, suggesting that further development and application of combinatorial strategies is warranted.