Abstract
Background
Grouping genes into clusters on the basis of similarity between their expression profiles has been the main approach to predict functional modules, from which important inference or further investigation decision could be made. While the univocal determination of similarity metric is important, current practices are normally involved with Euclidean distance and Pearson correlation, of which assumptions are not likely the case for highthroughput microarray data.
Results
We advocate the use of a novel metric  BayesGen  to measure similarity between gene expression profiles, and demonstrate its performance on two important applications: constructing genomewide coexpression network, and clustering cancer human tissues into subtypes. BayesGen is formulated as the evidence ratio between two alternative hypotheses about the generating mechanism of a given pair of genes, and incorporates as prior knowledge the global characteristics of the whole dataset. Through the joint modelling of expected intensity levels and noise variances, it addresses the inherent nonlinearity and the association of noise levels across different microarray value ranges. The full Bayesian formulation also facilitates the possibility of metaanalysis.
Conclusion
BayesGen allows more effective extraction of similarity information between genes from microarray expression data, which has significant effect on various inference tasks. It also provides a robust choice for other objectfeature data, as illustrated through the results of the test on synthetic data.
Background
With the development of highthroughput experimental techniques, biological research have been transformed into a datarich discipline. DNA microarray, which allows user to measure the expression levels of thousands of gene simultaneously in a single experiment, emerged to be one of the most widely used technology. The analysis of microarray data is normally based on the reasoning that variations in gene expression patterns under different experimental conditions are the results of underlying cellular pathway changes [1]. During its inception, gene expression was analysed using semi qualitative considerations; for example, genes having the expression roughly two fold of their basal values were considered coexpressed. Nowadays we are using microarray data in a quantitative way, which has elicited the proposal of a large number of clustering algorithms tailored specifically for bioinformatics. By grouping genes with similar expression profiles into clusters, researchers have been able to make meaningful inference of regulatory modules and functional pathways [2].
Before a clustering procedure could be performed, it is naturally conceivable to ask if the metric of similarity between expression profiles has been univocally determined. While the normal practice has been largely involved with Euclidean distance and Pearson correlation, these metrics either assume a clean experimental space or the linearity between similar genes, which are not likely the case for highthroughput expression data. We expect a metric that could handle the dependency on the responsiveness/determination accuracy of different concentrations, and the nonlinearities that are likely to result from the production of mRNAs and deficiencies of measuring devices. Moreover, as commonly observed, the measurement average and dependency may be linked, or high intensity values are likely to be affected by larger error [3].
In this paper we apply Bayesian model selection to construct a principled framework for similarity/distance definition. One emergent feature about Bayesian approach is that it requires explicit statement of underlying assumptions, making it easier for users to evaluate the suitability of a given metric. We then propose a novel distance metric that addresses the nonlinearity and the variation of noise levels across different microarray value ranges, through the joint modelling of data points' intensity levels and noise variances. Another important aspect is that by deriving a full Bayesian model, it also facilitates the employment of metaanalysis through the estimation of the hyperparameters.
Bayesian model selection
Bayesian model selection uses the probability rules and Bayes theorem to choose among alternative hypotheses. To evaluate the plausibility of a given model H, one considers the probability of the data D given H (or the evidence of H) by marginalising over unknown parameters:
This quantity automatically encodes the Occam's factor or the preference for models with more constrained generating mechanism. In other words, since complex models have the capability of explaining over a wider range of data, their evidence distributions are more widely spread over the data space. Hence, if H_{2 }is a more complex model compared to H_{1}, and given a data D that could be explained by both H_{1 }and H_{2}, the evidence P (DH_{1}) will be larger than P (DH_{2}). More detailed discussion about Bayesian model selection could be found elsewhere [4].
Bayesian evidence serves as the basis for Bayes Factor (BF), which is defined as the evidence ratio P(DH_{1})/P (DH_{2}) times the hypotheses prior ratio P (H_{1})/P (H_{2}). In the case of no prior bias exists between the two hypotheses, the prior ratio could be safely ignored. Due to the Bayesian evidence's ability of automatically choosing the right model, BF was known to be more robust with sparse data in comparison to the popular likelihood ratio test (LRT). However, this advantage comes at the computational cost of integration over the parameter space, which normally employed Monte Carlo intergration and importance sampling [4]. Another option is to estimated BF through asymptotic approximation methods such as Bayesian information criterion (BIC) [5].
Bayes Factor and its approximated versions have recently attracted more interest as a tool for selecting alternative hypotheses in bioinformatics [68]. The task of grouping similar gene expression profiles into clusters was also recently formulated under the Bayesian framework [9]. Although considering a full Bayesian generative model, the authors assigned genes to clusters by estimating the single point at which the posterior distribution over latent variables was maximised. This MAP (maximum a posterior) approach inherently used Euclidean distance to measure expression profile similarities.
In this paper, we apply the model averaging principle of Bayesian evidence, which takes into account all possible models rather than relying on the best one. We start by constructing the general Bayesian formulation for pairwise similarity/distance measurement, from which the new distance metric BayesGen is described. We then compare BayesGen performance with Euclidean distance and Pearson correlation through three test sets. The first test on simulated data suggests that the full Bayesian approach is better in differentiating homologous and heterogeneous pairs. The second test on two genomewide S. cerevisiae datasets examines the capability of similarity/distance metrics in directly inferring the pairs of interacting proteins. The last test on four human cancer datasets quantifies the effect of metric selection on hierarchical clustering results, both in terms of cluster structure and partition accuracy. BayesGen delivers best or competitive performance in all cases.
Results and discussion
Bayesian pairwise distance
Suppose we are interested in a set of n objects {o_{1}, ..., o_{n}} of which observations of their behaviour are available as dataset D = {x_{1}, ..., x_{n}}. We assume an object behaviour is the result of an underlying generative process that takes into account its properties. We formalise such generative process as a probability distribution F(.θ) over the experimental space, where all observation vectors are generated from.
The similarity between two objects o_{i }and o_{j }would be best specified as the similarity between their inherent properties. Although such information is not directly available to us, it has been encoded into the generative processes that resulted in our observations. The similarity between o_{i }and o_{j }could then be defined to be proportional to the probability that two samples x_{i }and x_{j }were generated from the same process. Denoting H_{same }as the hypothesis that x_{i }and x_{j }are from a single process, and H_{diff }as its complement (two samples were generated from two different processes), we have:
where s(i, j) is the similarity function between two objects o_{i }and o_{j}, and p(H_{same}) and p(H_{diff}) are the prior beliefs. Since similarity/distance measurement are invariant to monotonic transformations, we could define a distance measurement between two objects o_{i }and o_{j }as the Bayes factor between the two hypotheses, employing the evidence expansion from (1):
BayesGen distance for gene expression data
Given a dataset D measuring the expression of n genes through d different experimental conditions, our objects of interest could be the set of n genes or the set of d conditions. Without violation to generality, we assume a default interest on genes.
As cellular processes are carried out through the coordination of gene modules, where the expression levels of coregulated genes within each module are similar under a given condition, we assume that each sample x_{i}, i = 1.. n is generated from a Gaussian distribution with parameter θ = {μ, Σ}. μ is a vector of length k that defines the expected expression level under k conditions, while Σ specifies the variance range accounted for measurement noise and cellular process inherent stochasticity. The formal generative process for each experimental condition is as follows (we remove parameters' sub and superscripts for presentation clarity):
where μ_{0}, Σ_{0}, λ_{0}, and ν_{0 }are hyperparameters indicating the prior mean, prior variance, and their belief levels respectively. Note that μ and Σ are not independent, reflecting the dependency between variance and intensity levels observed in expression data. The generative process has two stages: firstly, different processes are generated by mutating the global mean μ_{0 }with the expected variance of Σ_{0}; secondly, each process's samples are generated by adding fluctuations Σ to its expected expression level μ. Suppose that all observations of the given condition could be fitted to a Gaussian distribution (m, V), our model hyperparameters should be estimated such that:
Assuming no prior knowledge, the expected decomposition of processgenerated and samplegenerated variance is equiprobable and equals V/2. Plugging the model of (68) to (5), and assuming that Σ is a diagonal matrix, we obtain the closedform formula for BayesGen distance measurement for two given genes i and j as follows:
where
and m^{k}, v^{k}, , and are the kth component of the data global mean and variance, and the two sample local mean and variance respectively.
Experiment 1: Synthetic data
The first experiment was designed to compare the capability of the three metrics in differentiating between sample pairs that are generated from a single process, and those generated from two different processes. In order to explore the strengths and weaknesses of the metrics in a reasonably exhaustive way, we use synthetic data with different generating assumptions, which are not necessarily the valid assumptions for real microarray expression datasets.
We conducted the test over three cases, distinguished by the way samples within a process are linked: (1) Samples are independently generated from a Gaussian distribution, with different expected noise levels for different conditions; (2) Samples are independently generated from a Gaussian distribution, with fixed noise levels over all conditions; (3) Samples are generated as linear transformations from a common mean vector, with random noises added.
A dataset is the composition of 200 samples coming from two different processes (100 samples each). The distances between all pairs in the dataset were calculated, ranked, and scaled so that they are evenly distributed over the range [0, 1]. We then grouped distance values into two classes by the origin of their objects: within (the two samples were from the same process), and across (the two samples were from different processes). The results are averaged over 100 independent datasets for each case.
Figure 1 shows the distance distributions of the two classes (red to within, blue to across) obtained when BayesGen, Euclidean distance, and Pearson correlation were used as distance metrics over the three cases. The intersection between the two lines could be interpreted as the probability of error when using the distance as a tool for sample origin prediction. As expected, each metric was the best choice under its favoured assumptions. However, while Euclidean distance and Pearson correlation performance deviated when getting off their favouring case, BayesGen remained to be the best or competitive to the best over all three cases, suggesting its position as the safe choice for most application problems.
Figure 1. Distance distributions of the homologous and heterogeneous groups. Comparison of the three distance metric capability in differentiating between homologous and heterogeneous sample pairs over three generating cases. Red lines: densities of homologous distances (two samples are from the same process); blue lines: densities of heterogeneous distances (two samples are from two different processes). Case 1: Samples are independently generated from a Gaussian distribution with varying noises (favours BayesGen); Case 2: Samples are independently generated from a Gaussian distribution with fixed noise (favours Euclidean distance); Case 3: Samples are generated as noisy linear transformations from a common mean vector (favours Pearson correlation).
Experiment 2: Functional association discovery
In the second experiment, we examined the direct application of the proposed measurement approach in predicting protein pairs that participate in the same cellular processes from high throughput microarray expression data. Our application was based on the guiltbyassociation heuristic [1], which says that genes with similar expression profiles are likely to belong to the same functional module. Using this heuristics, coexpression gene networks were often constructed by Pearson correlations for all gene pairs [10].
Datasets
We used two public datasets measured genomewide gene expressions of Saccharomyces cerevisiae under different experimental conditions. Each row corresponds to a gene, which we treated as a sample, and each column corresponds to a sample feature.
The first dataset was extracted from the gene expressions of wildtype and Mec1 defective yeasts in response to two different DNAdamaging agents: methylmethane sulfonate and ionising radiation [11], making a total of 52 observed features for each gene. The experiments were performed on spotted microarrays.
The second dataset contains the gene expressions from triple replicates of 14 yeast samples differentiated by their sucrose gradients [12], making a total of 42 features for each gene. The experiments were also performed on spotted microarrays, with the focus on protein biosynthesis process.
Since the purpose of our experiment was to evaluate the proposed measurement directly, without intervention from any other algorithms, we did not apply any imputation method here. All the rows that contain missing values were ignored, leaving a total of 2,222 genes for [11] and 1,758 genes for [12]. To account for possible unfairness towards traditional approaches due to the inherent columnwise normalisation of BayesGen, we created a normalised version for each dataset, on which later we repeated the tests for Euclidean distance and Pearson correlation. Formally, for each column X^{k}, the following transformation was applied:
where and Σ^{k }are the mean and variance of feature k, leaving each feature column with the mean of 0 and variance of 1.
Experiment and results
For each dataset, 5 pairwise distance matrices were computed using: Euclidean distance on original data, Euclidean distance on normalised data, Pearson correlation on original data, Pearson correlation on normalised data, and BayesGen on original data (BayesGen has the inherent columnwise normalisation in its formula).
Given a distance matrix, the smallest t% were marked as positive pairs, which means protein pairs that belong to the same molecular process, where t is a userspecified threshold. For our experiment, we ranged t from 0.01 to 7.
To evaluate the quality of our prediction, we compared the predicted pairs against the positive pairs derived from the combination of Gene Ontology (GO) [13] and the associated annotations of S. cerevisiae [14]. Both the GO term and annotations files were downloaded from [15] on 16/02/2009. Since the GO structure consists of several thousands of terms, each of different levels of specificity, counting any protein pairs that were coannotated by a GO term as positive would be misleading. We selected a list of 140 qualified terms which got 5/6 votes in the survey performed by Myers et al. [16] on the validity of GO terms for concluding that coannotated proteins actually interact. We obtained 2,467,531 pairs for the 2,222 genes presented in Gasch et al. [11] data, and 1,544,403 pairs for the 1,758 genes of Avara et al. [12] data, which are equivalent to 6.6% and 7.2% of all possible pairs respectively, and agreed with estimated proportion of yeast (about 5%). The results are shown in Figure 2.
Figure 2. Protein functional association discovery. Comparison of the three distance metric capability in predicting interacting yeast protein pairs from genomewide microarray expression data. The standard positive pairs are derived from the annotations of GO terms that got 5/6 votes of expert survey. (A) Results from Gasch et al. [11] data; (B) Results from Avara et al. [12] data.
Experiment 3: Hierarchical clustering application
The aim of the third experiment was to quantify the advantage of the proposed approach in application to a distancebased clustering method. We chose agglomerative hierarchical clustering due to its popularity in the area of gene expression analysis. Starting from a set of N objects, considered as N clusters, the algorithm iteratively builds up a tree by linking the two closest clusters at each step. It goes through N  1 steps in total, resulting in a single tree for all the objects.
Datasets
We used four public datasets of gene expression profiles measured on cancer patients during the diagnosis stage [17]. Unlike the previous experiment, here we treated patients as the objects of interest, and genes as features. The classification of patients into distinct classes was known a priori, and only used for evaluation purpose.
The first dataset contained bone marrow samples obtained from acute leukemia patients, measured on the Human Genome HU6800 Affymetrix microarray [18]. Among the 38 patients, 11 were of acute myeloid leukemia (AML), and 27 were of acute lymphoblastic leukemia (ALL). The ALL group could be further divided into 2 subtypes: Tlineage (8 samples), and Blineage (19 samples), making a total of 3 known classes.
The second dataset consisted of leukemia bone marrow samples from ALLtype pediatric patients, measured on the Human Genome U95 Affymetrix microarray, with the focus on the patients' risk of relapse [19]. Among the 248 samples, 43 were of Tlineage, and 205 were of Blineage. The Blineage groups was further divided into 5 prognostically important subtypes: 15 containing t(9;22) [BCRABL], 27 containing t(1;19) [E2APBX1], 79 containing t(12;21) [TELAML1], 20 containing rearrangements in the MLL gene, and 64 containing hyperdiploid karyotype, making a total of 6 known classes.
The third dataset contained 103 cancer samples from 4 distinct tissues (26 breast, 26 prostate, 28 lung, and 23 colon), measured on the Human Genome U95 Affymetrix microarray [20].
The last dataset consisted of diagnostic samples from diffuse large Bcell lymphoma patients, measured on the Human Genome U133A and U133B Affymetrix microarrays [21]. Among the 141 subtypes, 3 discrete subtypes had been identified: oxidative phosphorylation (49 samples), Bcell receptor/proliferation (50 samples), and host response (42 samples).
Since it is possible that the datasets contained multiple signatures other than the known phenotypes, they had been preprocessed by applying a signaltonoise ratio test and selecting the most upregulated genes for each class [22], so that the observed phenotype would be the dominant signature in the data.
Experiment and results
For each of the described dataset, we calculated the distance matrix using the 5 approaches: Euclidean distance, Euclidean distance with zscore normalisation, Pearson correlation, Pearson correlation with zscore normalisation, and the newly proposed BayesGen. These distance matrices were then fed as inputs to the agglomerative hierarchical clustering to obtain one linkage tree for each metric. We used average linkage, which defines the distance between two clusters as the average of all betweencluster distances. Formally, given 2 clusters C_{1 }and C_{2 }of n_{1 }and n_{2 }objects respectively, the distance between C_{1 }and C_{2 }is:
Hierarchical clustering does not require users to specify the number of clusters beforehand. One could later decides on the number of partitions by looking at the tree structure. However, this process is normally bias and based on one's prior expectation about the data. In an attempt of achieving a reasonable fairness level for all approaches, we estimated the appropriate number of clusters for each tree using gap statistics [23]. The idea of gap statistics is to find the point at which the withincluster dispersion is minimised, by comparing it to a null reference distribution. More details about gap statistics is in the Method section. To evaluate the predicted clusters quality we used the adjusted Rand index [24] to compare between the known class labels and the cluster labels. The index ranges from 0 to 1, where 1 corresponds to perfect agreement, and 0 to the expected value of random cluster assignment. The computation detail of Rand index goes in the Method section.
Table 1 presents the adjusted Rand indices obtained using different distance matrices as the input for the hierarchical clustering and gap statistics procedure. While the Bayesian generative input is the clear winner for all 4 data sets, the position of the second best fluctuates among normalised Pearson correlation, Pearson correlation, and normalised Euclidean distance. Although zscore normalisation overally increased the performance of both Euclidean distance and Pearson correlation, in some cases it may lead to the loss of necessary information and decreased clustering accuracy.
Table 1. Clustering expression profiles into cancer subtypes
The numbers of clusters estimated from gap statistics are shown in table 2, and the cluster structure obtained from hierarchical clustering partitions are in figure 3. For clarity, we only show the structures of the best (which is always Bayesian generative), and the second best partitions (as of table 1) for each dataset. It can be seen that both in terms of cluster number estimation, and cluster structure, our approach outperformed the remaining four.
Table 2. Predicting number of clusters using gap statistics
Figure 3. Cluster structures resulted from the use of different metrics on hierarchical clustering. Comparison of the resulted cluster structures resulted from the use of different distance metrics on hierarchical clustering over 4 cancer datasets. Top row: the true structure derived from known phenotypes; Middle row: the structure resulted from BayesGen (offered highest Rand indices); Bottom row: the structure resulted from the metric that offered the second best Rand indices.
Conclusion
We suggested the use of BayesGen  a new metric for measuring similarity/distance between gene expression profiles. Based on the observation that both data points' intensity levels and their relative variance jointly contribute to the identification of the underlying cellular processes, the metric was derived using a full Bayesian approach, which incorporates as prior knowledge the global characteristics of the whole dataset.
In comparison to Euclidean distance and Pearson correlation, BayesGen was shown to be the superior in predicting the interacting protein pairs through the construction of pairwise relevance networks. The profound effect of metric selection on clustering results was confirmed in the last experiment, showing significant improvement brought by BayesGen to hierarchical clustering both in term of partition accuracy and cluster structure. Although encoding more information, BayesGen shares the calculation simplicity of the other two, and we expect its seamless integration capability to any downstream distancebased approach.
Despite the inspiration from gene expression data, BayesGen was designed with a general purpose in mind, and could be well applied to other objectfeature data. The test on synthetic data under different generating assumptions showed that BayesGen is robust enough to be considered as the safe choice in most cases. Work in progress is to extend the same Bayesian framework to other data types, including relational and structured data.
Methods
Euclidean distance
The Euclidean distance between two expression profiles x_{i }and x_{j }is defined as follows:
which measures the absolute distance between expression profiles in the ddimensional experimental space. From the statistical point of view, the unification between Bayesian distance and Euclidean distances occurs when generative processes are differentiated by only their expected intensity levels or θ = {μ}, and no restriction is put on the selection of possible means. Formally, each condition data points are generated as follows:
where is a constant and ℛ is the set of all real numbers. Plugging this model to (5), and assuming that Σ_{0 }is of the form λ I, the resulted distance between two given genes i and j is:
where
which is the squared version of Euclidean distance.
Pearson correlation
The Pearson correlation between two expression profiles x_{i }and x_{j }is defined as follows:
where and ) are the average expression levels over all d experimental conditions of gene i and gene j respectively. Pearson correlation takes lightly the magnitude of each data point, but pays attention to the shapes of vectors. It ranges from 1 to +1, with the absolute value specifies the level of correlation between two variables, and the sign indicates the direction of such correlation. The perfect correlation corr(i, j) = 1 is reached when x_{i }= ax_{j }where a is a random coefficient. Using Pearson correlation as a tool of similarity/distance measurement, one may choose to pay attention only to positive correlations, or utilise also the negative values by taking their absolute values.
Note that while Euclidean and Bayesian distance treat each expression profile x_{i }as a vector of d features, Pearson correlation treats it as a sequence of d independent observations of object i.
Gap statistic
Gap statistic [23] was developed to estimate the appropriate number of clusters for a dataset given the set of partitions resulted from a clustering technique with k = 1 .. K. Although its specific computation was designed for kmeans clustering, gap statistic is applicable to any method of data groupings. To estimate the number of clusters from a linkage tree, we applied gapuniform with empirical adjustment in the last step of deciding the appropriate .
Given a partition with k clusters C_{1}, ..., C_{k }of size n_{1}, ..., n_{k}, the dispersion index at k is defined as:
where D_{r }is the sum of withincluster distances of cluster r:
At each point of k, gap statistic compares this dispersion index with its expectation under the null hypothesis. The null hypothesis assumes that the whole dataset is of a single uniformly distributed cluster. The expectation is computed by averaging over B randomly generated datasets. The gap statistic is defined as follows:
where
Tibshirani et al. [23] suggested to choose the number of clusters as the smallest k such that
where sd_{k+1 }is the standard deviation of over B simulated datasets. However, in the case of hierarchical clustering, where the resulted trees may contain leafs or very small clusters, we suggest that k should be estimated as follows:
1. Choose candidate numbers {k^{c}}as all k that satisfies G_{k }≥ G_{k+1}.
2. Choose the number of clusters as the smallest k^{c }such that .
Adjusted Rand index
The Rand index [25] was developed to measure the agreement of two partitions of the same set of N objects. The two key assumptions underlying its derivation are: first, each object is assigned to exactly one cluster; and second, all objects are of equal importance in forming the clusters. Given two partitions P_{1 }and P_{2}, with k_{1 }and k_{2 }groups respectively (k_{1 }and k_{2 }are not necessarily equal). The matching of the two partitions is defined as confusion matrix C of size k_{1 }× k_{2}, where C_{ij }is the number of objects in group i of partition P_{1 }that are also in group j of partition P_{2}. Rand index computes the probability that any 2 out of N objects were grouped similarly. However, it is not well scaled and is bias towards increasing number of clusters.
The adjusted Rand index [24] addressed these issues, and is computed as follows:
where
and
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
VAN conceived and designed the study, carried out experiments, and drafted the manuscript. PL supervised the work, discussed the results and critically revised the paper. Both authors read and approved the final manuscript.
Note
Other papers from the meeting have been published as part of BMC Bioinformatics Volume 10 Supplement 15, 2009: Eighth International Conference on Bioinformatics (InCoB2009): Bioinformatics, available online at http://www.biomedcentral.com/14712105/10?issue=S15 webcite.
Acknowledgements
VAN was supported by the Computer Laboratory Premium Studentship, Cambridge Overseas Research Studentship, Cambridge Overseas Trust, and King's College Studentship. This work was supported by the WADA (World AntiDoping Association) grant to PL, and the Ferris Fund from King's College to VAN.
This article has been published as part of BMC Genomics Volume 10 Supplement 3, 2009: Eighth International Conference on Bioinformatics (InCoB2009): Computational Biology. The full contents of the supplement are available online at http://www.biomedcentral.com/14712164/10?issue=S3.
References

Brown PO, Botstein D: Exploring the new world of the genome with DNA microarrays.
Nature Genetics 1999, 21(1 Suppl):3337. PubMed Abstract  Publisher Full Text

Stuart JM, Segal E, Koller D, Kim SK: A GeneCoexpression Network for Global Discovery of Conserved Genetic Modules.
Science 2003, 302(5643):249255. PubMed Abstract  Publisher Full Text

Rocke DM, Durbin B: A Model for Measurement Error for Gene Expression Arrays.
Journal of Computational Biology 2001, 8:557569. PubMed Abstract  Publisher Full Text

Mackay DJC: Information Theory, Inference, and Learning Algorithms. Cambridge University Press; 2003.

Schwarz G: Estimating the Dimension of a Model.
The Annals of Statistics 1978, 6(2):461464. Publisher Full Text

Wakefield J: Bayes factors for genomewide association studies: comparison with Pvalues.
Genet Epidemiol 2008, 33:7986. Publisher Full Text

Yi N, Yandell BS, Churchill GA, Allison DB, Eisen EJ, Pomp D: Bayesian Model Selection for GenomeWide Epistatic Quantitative Trait Loci Analysis.
Genetics 2005, 170(3):13331344. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Zhao JX, Foulkes AS, George EI: Exploratory Bayesian model selection for serial genetics data.
Biometrics 2005, 61(2):591599. PubMed Abstract  Publisher Full Text

Hestilow TJ, Huang Y: Clustering of Gene Expression Data Based on Shape Similarity.
EURASIP Journal on Bioinformatics and Systems Biology 2009, 195712. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Wolfe C, Kohane I, Butte A: Systematic survey reveals general applicability of "guiltbyassociation" within gene coexpression networks.
BMC Bioinformatics 2005, 6:227. PubMed Abstract  BioMed Central Full Text  PubMed Central Full Text

Gasch AP, Huang M, Metzner S, Botstein D, Elledge SJ, Brown PO: Genomic Expression Responses to DNAdamaging Agents and the Regulatory Role of the Yeast ATR Homolog Mec1p.
Mol Biol Cell 2001, 12(10):29873003. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Arava Y, Wang Y, Storey JD, Liu CL, Brown PO, Herschlag D: Genomewide analysis of mRNA translation profiles in Saccharomycescerevisiae.
Proceedings of the National Academy of Sciences of the United States of America 2003, 100(7):38893894. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, Harris MA, Hill DP, IsselTarver L, Kasarskis A, Lewis S, Matese JC, Richardson JE, Ringwald M, Rubin GM, Sherlock G: Gene ontology: tool for the unification of biology. The Gene Ontology Consortium.
Nature Genetics 2000, 25:2529. PubMed Abstract  Publisher Full Text

Ball CA, Dolinski K, Dwight SS, Harris MA, IsselTarver L, Kasarskis A, Scafe CR, Sherlock G, Binkley G, Jin H, Kaloper M, Orr SD, Schroeder M, Weng S, Zhu Y, Botstein D, Cherry JM: Integrating functional genomic information into the Saccharomyces Genome Database.
Nucl Acids Res 2000, 28:7780. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Gene Ontology Downloads [http://www.geneontology.org/GO.downloads.shtml] webcite

Myers C, Barrett D, Hibbs M, Huttenhower C, Troyanskaya O: Finding function: evaluation methods for functional genomic data.
BMC Genomics 2006, 7:187. PubMed Abstract  BioMed Central Full Text  PubMed Central Full Text

Cancer Program Data Sets, Broad Institute [http://www.broad.mit.edu/cgibin/cancer/datasets.cgi] webcite

Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES: Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring.
Science 1999, 286(5439):531537. PubMed Abstract  Publisher Full Text

Yeoh EJ, Ross ME, Shurtleff SA, Williams W, Patel D, Mahfouz R, Behm FG, Raimondi SC, Relling MV, Patel A, Cheng C, Campana D, Wilkins D, Zhou X, Li J, Liu H, Pui CH, Evans WE, Naeve C, Wong L, Downing JR: Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling.
Cancer Cell 2002, 1(2):133143. PubMed Abstract  Publisher Full Text

Su AI, Cooke MP, Ching KA, Hakak Y, Walker JR, Wiltshire T, Orth AP, Vega RG, Sapinoso LM, Moqrich A, Patapoutian A, Hampton GM, Schultz PG, Hogenesch JB: Largescale analysis of the human and mouse transcriptomes.
Proceedings of the National Academy of Sciences of the United States of America 2002, 99(7):44654470. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Monti S, Savage KJ, Kutok JL, Feuerhake F, Kurtin P, Mihm M, Wu B, Pasqualucci L, Neuberg D, Aguiar RCT, Cin PD, Ladd C, Pinkus GS, Salles G, Harris NL, DallaFavera R, Habermann TM, Aster JC, Golub TR, Shipp MA: Molecular profiling of diffuse large Bcell lymphoma identifies robust subtypes including one characterized by host inflammatory response.
Blood 2005, 105(5):18511861. PubMed Abstract  Publisher Full Text

Monti S, Tamayo P, Mesirov J, Golub T: Consensus Clustering: A ResamplingBased Method for Class Discovery and Visualization of Gene Expression Microarray Data.
Machine Learning 2003, 52:91118. Publisher Full Text

Tibshirani R, Walther G, Hastie T: Estimating the number of clusters in a data set via the gap statistic.
Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2001, 63(2):411423. Publisher Full Text

Hubert L, Arabie P: Comparing Partitions.
Journal of Classification 1985, 2:193218. Publisher Full Text

Rand WM: Objective Criteria for the Evaluation of Clustering Methods.
Journal of the American Statistical Association 1971, 66(336):846850. Publisher Full Text