We integrate three independent microarray gene expression data sets to obtain an integrated training set of 358 samples and identify a set of features for predicting distant metastases. All the samples included in this study are from lymph-node-negative patients who have not received adjuvant systemic treatment. Each feature is based on an ordered pair of genes and assumes the value one if the first gene is expressed less than the second gene, and assumes the value zero otherwise. These genes may not all be highly differentially expressed, and one gene in the pair may serve as a "reference" for the other one. Since the features are rank-based, no data normalization is needed before data integration. A classical likelihood ratio test is used to classify patients as either poor-outcome, meaning they are likely to metastasize, or good-outcome, meaning that they are unlikely to develop distant metastases. The choice of features is motivated by achieving the highest possible specificity at an acceptable level of sensitivity, taken here to be 90% in accordance with the St. Gallen and NIH treatment guidelines. The number of features chosen in the prognostic signature, as well as the threshold in the likelihood ratio test (LRT), is optimized with k-fold cross-validation on the integrated training set. The optimal feature number is estimated to be 80, corresponding to 112 genes (since some genes appear in more than one feature). The prognostic test based on this signature is validated using an independent microarray data set. Upon further validation on large-scale independent data, the prognostic gene expression signature could support other breast cancer prognostic tests with high enough specificity to help avoid over-treatment of newly diagnosed patients.

Four breast cancer microarray data sets are included in this study. Each data set has been downloaded from publicly available gene expression repositories (e.g. Gene Expression Omnibus) or supporting web sites 7112526. All four data sets are generated from the same Affymetrix HG-U133A microarray platform. Here, the names of the first authors of individual studies are used as the names of the data sets. Three data sets, Miller (251 patients), Sotiriou (189 patients) and Wang (286 patients), are used as training data and the other one, Pawitan (159 patients), is used as independent test data. The reason for this division into training and test data is that detailed clinical information has been provided for the Miller, Sotiriou and Wang data sets and this information has been used to select specific patients for training, whereas little clinical information is provided for the Pawitan study. For the Miller, Sotiriou and Pawitan studies, because the gene expression data sets provided by them have undergone cross-sample normalization, we have downloaded the raw CEL files and calculated expression values using the Affymetrix GeneChip Operating Software version 1.4. There is an 85-patient overlap between Miller and Sotiriou data sets, so we have excluded the replicate samples from our study. Detailed patient information in each study has been described in the corresponding literature.

Motivated by a recent study 27, we employ the idea of restricting training data to extreme patient samples, which are more informative in identifying a prognostic signature. Extreme patients are either short-term survivors with poor-outcome within a short period or long-term survivors who maintain a good-outcome after a long follow-up time. Specifically, we select patients who developed distant metastases (relapse) within five years as poor-outcome samples and patients who were free of distant metastases (relapse) during the follow-up for a period of at least eight years as good-outcome samples. The sharp contrast between short-term and long-term survivors should identify more informative and reliable genes for a prognostic signature. Only early stage lymph-node-negative patients who had not received adjuvant systemic treatment are included in the training data because adjuvant treatment is likely to modify patient outcome. The selection is irrespective of age, tumor size and other clinical parameters. After applying the above selection criteria, a total of 358 patients are identified from the three training data sets and used to learn a prognostic signature and prognostic test. The numbers of selected patients from each training data set are listed in Table 1.

We directly merge the three microarray data sets in Table 1, using the 22283 probe sets on Affymetrix HG-U133A microarray, to form an integrated training data set. The integrated data set consists of 122 extreme poor-outcome samples (distant metastases within five years after surgery) and 236 extreme good-outcome samples (free of distant metastases during the follow-up for a period of at least eight years after surgery). Recall that each feature is based on a pair of genes. The integrated training set is used to estimate the relationship between the number m of features in a prognostic classifier and the specificity at 90% sensitivity level, evaluated by the 40-fold cross-validation, as described in 'Methods'. The result is plotted in Figure 1. As can be seen, the specificity is nearly constant after about 80 features are included. Our final prognostic signature then consists of the 80 top-ranked features (gene pairs) from the feature list generated from the original integrated training data, using the feature selection and transformation procedures described in 'Methods'. Because some genes appear in more than one feature, the 80 top-ranked gene pairs in our prognostic signature include 112 distinct genes (Table 2). To illustrate the behavior of the 80 features in the signature on the Wang data set (part of the integrated training data), we show the difference in expression between the two genes in each of the 80 gene pairs in the form of a heat map in Figure 2. Distinct patterns of expression differences can be observed for good- and poor-outcome samples.

In order to evaluate the reproducibility of the 112-gene signature, we repeat the same feature selection process with several re-samplings of 300 patients out of the 358 patients in the integrated data set. The average overlap is 39.0%. This is not surprising in view of the still modest sample size and the fact that most of the changes occur in the second half of the ranked list of gene pairs.

To validate the prognostic test, we compute its sensitivity and specificity on an independent set of samples, the Pawitan data set 26, which consists of 159 primary breast cancer patients. This test set includes both patients with lymph-node-negative tumors and patients with lymph-node-positive tumors, and who had or had not received adjuvant systemic therapy. Following the practice in most of the literature, our objective is to predict the development of distant metastases within five years. Of the 159 patients, 35 patients developed distant metastases (relapse) within five years ("poor-outcome"), and 119 patients were free of distant metastases (no relapse) during the follow-up for a period of at least five years ("good-outcome"). Note that the definition of good-outcome for patients in the validating data is different from the definition in the training data because we have used extreme samples to identify the prognostic signature.

Our prognostic test is the classical likelihood ratio test, determined by assuming that the features are conditionally independent under both classes, namely "poor outcome" (the null hypothesis) and "good outcome" (the alternative hypothesis); see 'Methods'. The LRT reduces to comparing a weighted average of the 80 features to a threshold. The weights depend on the statistics of the individual features under both classes and are estimated from the training data; the threshold is also estimated from the training set, using cross-validation. The LRT built from the prognostic signature achieves a sensitivity of 88.6% (31 out of the 35 poor-outcome samples) and a specificity of 54.6% (65 out of the 119 good-outcome samples) on the 154 samples included in the validating data set. The remaining five patients, who either developed distant metastases after five years or were free of distant metastases with a follow-up period less than five years, are not included in the validating data set. We compute the odds ratio of the prognostic test for developing metastases within five years between the patients in the poor-outcome group and in the good-outcome group as determined by the prognostic test. The prognostic test has a high odds ratio of 9.3 (95% confidence interval: 3.1 – 28.1) with a Fisher's exact test p-value < 0.00001. To make the results easier to understand, we have included in the additional files the heat maps of the two-group (good- and poor-outcomes) supervised clusters of the integrated training data and test data for the 112-signature genes (see Additional file 1 and file 2).

It is noteworthy that performance of the LRT on the validation data is actually somewhat better than the performance on the training set (which is estimated by cross-validation). Specifically, from Figure 1 (see also 'Methods'), the specificity of the LRT prognostic test is around 43% at approximately 90% sensitivity when estimated from the training data, whereas a specificity of approximately 55% at about the same sensitivity is achieved on the independent validation set.

To obtain another useful estimate of the clinical outcome, we apply the LRT built from the prognostic signature to all of the 159 samples in the Pawitan data set and calculate the probability of remaining free of distant metastases according to the prognostic signature by using Kaplan-Meier analysis. The Kaplan-Meier curve of the prognostic signature shows a significant difference (p-value < 0.001) in the probability of remaining free of distant metastases between the patients in the poor-outcome group and those in the good-outcome groups (Figure 3A). The p-value is computed by the use of log-rank test. The Mantel-Cox estimation of hazard ratio for distant metastases within five years in the poor-outcome group as compared to the good-outcome group is 9.3 (95% confidence interval: 2.9 – 29.9, p-value < 0.001).

To evaluate the potential statistical power gained by integrating multiple data sets to increase diversity and sample size, we compare the predictive power of our integrated prognostic signature with each of the three separate study-specific prognostic signatures identified from the three data sets in Table 1. We use exactly the same method we used for the integrated data and each of the resulting three prognostic tests is applied to the same independent test data, the Pawitan data. The results are reported in Table 3. In the case of the Sotiriou data, we do not achieve the targeted sensitivity of at least ninety percent due to the very small sample size; the estimate of the threshold in the LRT does not generalize to the Pawitan test set. For the Miller and Wang data sets, the desired sensitivity is achieved but the specificity is far lower than for the classifier learned from the integrated data set.

The Wang data set is the largest. Using 40-fold cross-validation, the optimal feature number of gene pairs for the prognostic signature is m_opt = 60. The 94.3% sensitivity on the test set (33 out of the 35 poor-outcome samples) is close to the target of 90%. The specificity of the classifier is 10.1% (12 out of the 119 poor-outcome samples), substantially lower than the classifier based on the integrated training set, albeit at somewhat higher sensitivity. (Indeed, the performance of the prognostic LRT test based on the Wang data alone is barely better than the completely randomized, data-independent procedure which chooses poor-outcome with probability 0.9 and good outcome with probability 0.1, independently from sample to sample.) The odds ratio of this test is 1.9 (95% confidence interval: 0.4 – 8.7, Fisher's exact test p-value = 0.74), and the Kaplan-Meier curve (Figure 3B) shows a less significant difference between the patients in the poor-outcome and good-outcome groups than that of the signature from the integrated data. Finally, the estimated hazard ratio of 1.6 (95% confidence interval: 0.4 – 6.8, 0.01 < p-value < 0.05) is much lower than that of the prognostic test from the integrated data.

These comparisons demonstrate that the prognostic test derived from the integrated data is superior to the prognostic test derived from any of the individual studies and highlight the value of data integration. By integrating several microarray data sets with our rank-based methods, study-specific effects are reduced and more features of breast cancer prognosis are captured.

Recently, our group has developed a family of statistical molecular classification methods based on relative expression reversals 2233, and applied one variant based on a two-gene classifier to microarray data integration 23. These methods only use the ranks of gene expression values within each profile and achieve impressive results in both molecular classification and microarray data integration. An important feature of rank-based methods is that they are invariant to monotonic transformations of the expression data within an array, such as those used in most array normalization and other pre-processing methods. This property makes these methods especially useful for combining data from separate studies since the nature of the primary features extracted from the data, namely comparisons of mRNA concentration between pairs of genes, eliminates the need to standardize the data before aggregation. Specifically, the ranks of gene expression values are invariant to monotonic data transformations within each microarray. Consequently, we directly merge gene expression data of the patients from three training data sets in Table 1, using the 22283 probe sets on Affymetrix HG-U133A microarray, to form an integrated training data set of 358 samples. After aggregation, we extract a list of pair-wise comparisons; each of these "features" corresponds to a pair of genes and is assigned the value zero or one depending on the observed ordering of expressions; see the following section. The number of features retained is much smaller than the number of genes on the array. The procedure is now described in more detail.

Consider G genes whose expression values X = {X₁, X₂, ..., X_G} are measured using a DNA microarray and regarded as random variables. The class label Y for each profile X is a discrete random variable taking on one of two possible values corresponding to the two prognostic states or hypotheses of interest, namely "poor-outcome," denoted Y = 1, and "good-outcome," denoted Y = 2. The integrated training microarray data represent the observed values of X and comprise a G × N matrix x = [x_gn], g = 1, 2, ..., G and n = 1, 2, ..., N, where G is the number of genes in each profile and N is the number of samples (profiles) in the integrated data set. Each column n represents a gene expression profile of G genes with a class label y_n = 1 (poor-outcome) or y_n = 2 (good-outcome) for the two-class problem in our study. Among the N samples, there are N₁ (respectively, N₂) samples labeled as class 1 (respectively, class 2) with N = N₁ + N₂.

For each pair of genes (i, j), where i, j = 1, 2, ..., G, i ≠ j, let P(X_i <X_j|Y = k), k = 1,2, denote the conditional probability of the event {X_i <X_j} given Y = k. We define a score by

Δ_j = |P(X_i <X_j|Y = 1) - P(X_i <X_j|Y = 2)|

and estimate the score of pair (i, j) based on the training set x by

Δ i j ≈ | N i j ( 1 ) N 1 − N i j ( 2 ) N 2 | MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaeuiLdq0aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGHijYUdaabdaqaaKqbaoaalaaabaGaemOta40aa0baaeaacqWGPbqAcqWGQbGAaeaacqGGOaakcqaIXaqmcqGGPaqkaaaabaGaemOta40aaSbaaeaacqaIXaqmaeqaaaaakiabgkHiTKqbaoaalaaabaGaemOta40aa0baaeaacqWGPbqAcqWGQbGAaeaacqGGOaakcqaIYaGmcqGGPaqkaaaabaGaemOta40aaSbaaeaacqaIYaGmaeqaaaaaaOGaay5bSlaawIa7aaaa@4937@

where

N i j ( k ) = | { n : 1 ≤ n ≤ N , x i n < x j n , y n = k } | , k = 1 , 2 . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabd6eaonaaDaaaleaacqWGPbqAcqWGQbGAaeaacqGGOaakcqWGRbWAcqGGPaqkaaGccqGH9aqpdaabdaqaaiabcUha7jabd6gaUjabcQda6iabigdaXiabgsMiJkabd6gaUjabgsMiJkabd6eaojabcYcaSiabdIha4naaBaaaleaacqWGPbqAcqWGUbGBaeqaaOGaeyipaWJaemiEaG3aaSbaaSqaaiabdQgaQjabd6gaUbqabaGccqGGSaalcqWG5bqEdaWgaaWcbaGaemOBa4gabeaakiabg2da9iabdUgaRjabc2ha9bGaay5bSlaawIa7aiabcYcaSaqaaiabdUgaRjabg2da9iabigdaXiabcYcaSiabikdaYaaacqGGUaGlaaa@5BCA@

In other words, the estimated score is simply the absolute difference between the fraction of poor-outcome patients for which gene i is expressed less than gene j and the same fraction in the good-outcome examples. The feature selection procedure consists of forming a list of gene pairs, sorted from the largest to the smallest according to their scores Δ_ij, and selecting the top M pairs. The M top-ranked gene pairs are then considered to be the most discriminating candidate gene pairs for breast cancer prognosis if only relative expressions are taken into account. During the process, we have transformed the original feature vector X = {X₁, X₂, ..., X_G}(G = 22283 in this study), each of which assumes scalar values, to a new ordered feature vector Z = {Z₁, Z₂, ..., Z_M}(usually, M <<G), each of which assumes only two values.

Suppose Z_m, m = 1, 2, ..., M, corresponds to the gene pair {i, j}. For convenience, the ordering (i, j) will signify which probability in Equation (1) is larger. The reason for this is to facilitate interpretation of the results, as will become apparent. If P(X_i <X_j|Y = 1) ≥ P(X_i <X_j|Y = 2), as estimated by the fractions in (2), we will write (i, j) and if P(X_i <X_j|Y = 1) <P(X_i <X_j|Y = 2) we will denote the pair by (j, i). The value assumed by Z_m is then set to be 1 if we observe X_i <X_j and set to 0 otherwise, i.e., if we observe X_i ≥ X_j. Of course the same definition is applied to each feature in the training set. In this way, observing Z_m = 1 (resp., Z_m = 0) represents an indicator of the poor outcome (resp., good outcome) class in the sense that p_m = P(Z_m = 1|Y = 1) ≥ q_m = P(Z_m = 1|Y = 2). For all the features selected in our signature we in fact have p_m > 1/2 > q_m.

After this procedure, the original G × N data matrix is reduced to an M × N data matrix. The number of distinct genes in a prognostic signature is obviously fewer than 2M. In our practice, there are always more than M distinct genes among the top M gene pairs. Given that the numbers of genes in published breast cancer prognostic signatures are mostly fewer than 100, we fix M = 200 in this study to maker sure we can identify a prognostic feature signature based on a reasonable number of genes.

The classical likelihood ratio test (LRT) is a statistical procedure for distinguishing between two hypotheses, each constraining the distribution of a random vector Z = {Z₁, Z₂, ..., Z_M}. In our case the variables Z_m are the simple, binary functions of the gene expression profile defined above.

The LRT is based on the likelihood ratio

L R ( z ) = P ( z | Y = 1 ) P ( z | Y = 2 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemitaWKaemOuaiLaeiikaGIaemOEaONaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGqbaucqGGOaakcqWG6bGEcqGG8baFcqWGzbqwcqGH9aqpcqaIXaqmcqGGPaqkaeaacqWGqbaucqGGOaakcqWG6bGEcqGG8baFcqWGzbqwcqGH9aqpcqaIYaGmcqGGPaqkaaaaaa@4556@

where z = {z₁, z₂, ..., z_M}are the observed values in a new sample. Notice that z assumes values in {0, 1}^M, the set of binary strings of length M. The LRT chooses hypothesis Y = 1 if LR(z) > t and chooses Y = 2 otherwise, i.e., if LR(z) ≤ t. The threshold t is adjusted to provide a desired tradeoff between type I error and type II error, i.e., between sensitivity and specificity. Choosing t small provides high sensitivity at the expense of specificity and choosing t large promotes the opposite effect.