This article is part of the supplement: Second Annual MidSouth Computational Biology and Bioinformatics Society Conference. Bioinformatics: a systems approach .

Proceedings

Reproducible Clusters from Microarray Research: Whither?

Nikhil R Garge^4* , Grier P Page¹ , Alan P Sprague² , Bernard S Gorman³ and David B Allison^1*

¹  Department of Biostatistics, Section of Statistical Genetics, Ryals Public Health Building Suite 327, University of Alabama at Birmingham, Birmingham, USA

²  Computer and Information Sciences department, University of Alabama at Birmingham, Birmingham, USA

³  Department of Psychology, Hauser Hall, Hofstra University, NY 11550, USA

⁴  Medical College of Georgia, 1120 15th Street, Ca4100, Augusta, GA 30912, USA. email: ngarge@mcg.edu

author email corresponding author email* Contributed equally

BMC Bioinformatics 2005, 6(Suppl 2):S10doi:10.1186/1471-2105-6-S2-S10

Published:	15 July 2005

Abstract

Motivation

In cluster analysis, the validity of specific solutions, algorithms, and procedures present significant challenges because there is no null hypothesis to test and no 'right answer'. It has been noted that a replicable classification is not necessarily a useful one, but a useful one that characterizes some aspect of the population must be replicable. By replicable we mean reproducible across multiple samplings from the same population. Methodologists have suggested that the validity of clustering methods should be based on classifications that yield reproducible findings beyond chance levels. We used this approach to determine the performance of commonly used clustering algorithms and the degree of replicability achieved using several microarray datasets.

Methods

We considered four commonly used iterative partitioning algorithms (Self Organizing Maps (SOM), K-means, Clutsering LARge Applications (CLARA), and Fuzzy C-means) and evaluated their performances on 37 microarray datasets, with sample sizes ranging from 12 to 172. We assessed reproducibility of the clustering algorithm by measuring the strength of relationship between clustering outputs of subsamples of 37 datasets. Cluster stability was quantified using Cramer's v² from a kXk table. Cramer's v² is equivalent to the squared canonical correlation coefficient between two sets of nominal variables. Potential scores range from 0 to 1, with 1 denoting perfect reproducibility.

Results

All four clustering routines show increased stability with larger sample sizes. K-means and SOM showed a gradual increase in stability with increasing sample size. CLARA and Fuzzy C-means, however, yielded low stability scores until sample sizes approached 30 and then gradually increased thereafter. Average stability never exceeded 0.55 for the four clustering routines, even at a sample size of 50. These findings suggest several plausible scenarios: (1) microarray datasets lack natural clustering structure thereby producing low stability scores on all four methods; (2) the algorithms studied do not produce reliable results and/or (3) sample sizes typically used in microarray research may be too small to support derivation of reliable clustering results. Further research should be directed towards evaluating stability performances of more clustering algorithms on more datasets specially having larger sample sizes with larger numbers of clusters considered.