All validation datasets (consisting of six GeneChips each) were produced by hybridizing triplicate RNA samples from a breast cancer cell line (MCF7) and an immortalised normal breast cell line (MCF10A) using a variety of different array types and sample preparation protocols. In all cases, the aim of the validation study was to assess the correspondence between the sets of differentially expressed probesets (or transcripts) identified when using different versions of the same underlying technology. In the hypothetical situation when all datasets yield identical results, the same set of differentially expressed probesets would be identified, irrespective of the array type or protocol used to process the data; we considered how close the data approaches this ideal. For example, a comparison of the fold changes between datasets (MCF7-amplified/MCF10A-amplified vs. MCF7-unamplified/MCF10A-unamplified) generated using Affymetrix's small sample protocol and Affymetrix's standard protocol yields good, but not perfect correspondence (Fig 1A, Table 1). However, when fold changes are calculated across datasets (MCF7-amplified/MCF10A-unamplified vs. MCF7-unamplified/MCF10A-amplified) correspondence falls dramatically (Fig. 1B grey dots, Table 1).

Batch mean-centering (see methods) of the amplified and unamplified datasets was found to dramatically increase the correspondence comparison across the datasets, with 100% of probesets having fold changes within two-fold between experimental branches (Fig. 1B black dots, Table 1). Similarly, when significance analysis of microarrays (SAM) was used to identify lists of probesets with statistically significant changes between the same replicate groupings used to generate Fig. 1, the intersection between sets was also found to be greater following mean batch correction (Table 1) and Pearson correlation of raw intensities was also found to increase (Figs. 1C, 1D and Additional File 1). It is notable that while correction improves fold-change correspondence across protocols, fold changes between datasets are preserved (Fig. 1E, Table 1).

The same approach was applied to a variety of other validation datasets that were designed to investigate the effect of using different generations of Affymetrix GeneChips, alternative protocols and scanning hardware (Table 1, Additional File 1). A systematic bias was found to be present in all datasets, and correspondence improved following mean-centering in all cases (median centering also performed similarly; data not shown). These results demonstrate that systematic, multiplicative bias is a widespread property within Affymetrix array data, and mean-centering was found to lead to improvements irrespective of the summarisation method used (RMA or MAS5 4) to process the data or when using alternative Chip Description Files (CDFs) to group probes according to Unigene cluster 5 (Additional File 2).

Breast tumours have been classified into five molecular subtypes; basal, luminal A, luminal B, ERBB2 and normal-like by identifying a set of genes with significantly greater variation in expression between different tumours than between paired samples from the same tumour 678. Since members of this set appear to define properties 'intrinsic' to each subtype, the authors referred to the genes as an 'intrinsic gene set'. 640 Affymetrix probesets representing the 534 'intrinsic gene set' from 13 were identified using NetAffx 9. These probesets were used to cluster MAS5 normalised expression data from two published Affymetrix gene expression studies 1011 with similar numbers and composition of tumours. Despite using similar starting material (primary tumours) and the same microarray platform, when combined the two datasets formed two distinct, independent clusters representing the two datasets (Fig. 2A), suggesting a dataset-specific systematic bias as observed with the validation datasets described above. Although clusters of known luminal and basal-specific genes show similar patterns of differential expression in each of the two datasets (Figure 2A iii, iv), the majority of the probesets representing the full 'intrinsic gene set' show greater differences in expression between the two studies than between the different classes of tumours. Following mean-centering as before, the 'basal-like tumours' from the Richardson et al. 10 dataset were found to cluster alongside the 'basal tumours' from Farmer et al. 11, and the 'non-basal tumours' with the 'luminal tumours' (Fig. 2B). A third cluster of tumours was identified with high expression of the ERBB2 cluster of probesets. This cluster contained all of the molecular apocrine tumours, plus a mixture of basal/basal-like and luminal/non-basal-like tumours. Using centroid prediction 12 as described previously, the tumours from both datasets were assigned to the five Norway/Stanford subtypes (basal, luminal A, luminal B, ERBB2 and normal-like 678), one of the tumours in this third cluster was assigned to the luminal B subtype, thirteen to the ERBB2 subtype and 9 could not clearly be assigned to a subtype (Fig 2B v).

The greatest single difference between molecular subtypes has repeatedly been demonstrated to be between estrogen receptor-positive (ER+) luminal tumours and ER-negative basal tumours 6781314. SAM analysis was used to identify probesets differentially expressed between the basal/basal-like and non basal-like/luminal subtypes using the combined data from both sets of samples. It was only following mean-centering that probesets were identified that represent genes that are known to characterize the differences between these subtypes (Additional File 3), including the fundamental estrogen receptor alpha and GATA binding protein 3, which maintains differentiation into the luminal cell fate in the mammary gland 15. In addition, following mean-centering, a greater number of statistically significant probesets were found to be differentially expressed between the tumour subtypes than were found between the two initial sets of samples (Additional File 4).

These results encouraged us to apply the approach to integrate six previously published datasets 161718192021 (Table 2) of primary breast cancer tumours processed on Affymetrix U133A, U133AA or plus 2.0 GeneChips. Multidimensional scaling of 1107 tumours based upon the expression of all common probesets between the three arrays (22,215) demonstrated that global gene expression is highly influenced by dataset, with tumours clustering by study (Fig. 3A), again suggesting a systematic, dataset-specific bias. However, following mean-centering, the tumours appear to cluster by breast cancer subtype (assigned using centroid prediction 12), regardless of the dataset from which they were generated (Fig. 3B).

A growing body of evidence has accumulated, supporting the notion that gene expression profiling of primary breast tumours can be used to stratify patients by subtype and the likelihood of disease progression (reviewed in 22). The approaches have included both unsupervised (intrinsic gene set 678) and supervised (genes associated with patient follow up 172123) methods, along with studies of cancer-associated pathways or tumor characteristics 24252627, in all cases these signatures appear to predict recurrence, despite the lack of overlap in their respective profiles or signatures 28. In order to establish whether integrating multiple datasets can improve prognostic prediction, the six published datasets (described above) were used individually or in combination as 'training sets' for supervised principal components analysis 2930. This method has been shown to produce more accurate predictions than several competing methods on both simulated and real microarray datasets 2930. Using the Superpc 30 package for R 31, a Cox proportional hazards model was fitted to each predictor (generated for all combinations of one to five datasets used as the 'training set') and cross validation curves were plotted to determine the optimum threshold for the predictor of survival as described previously 30. The 1^st supervised principal component was found to be the most significant in the vast majority of cases, which is consistent with the hypothesis that recurrence is an inherent property of primary tumour gene expression (examples of cross-validation and survival curves are shown in Additional File 5). The remaining dataset(s) were used as a test set for each predictor and an R² statistic was computed to assess the performance. Combining greater numbers of datasets or tumours significantly improves prediction of prognosis based upon gene expression data (Fig. 4). Mean-centering of the datasets significantly increased the correlation between the supervised principal components and clinical follow up, therefore improving prognostic performance. It is clear that the predictive power of some combinations of training and test sets is more reliable than others. Although only a limited number of patient and tumour characteristics were available (Table 2), it seems that the most accurate predictions are achieved for test datasets that have characteristics most similar to those of the individual or combined training dataset. R² statistic and p-values (log rank) for all possible combinations of training datasets and test datasets are given in Additional File 6.

We investigated the effect of altering the composition of luminal (ER+) and basal (ER-) tumours from the two published datasets of Farmer et al. 11 and Richardson et al. 10 compared above. Unbalancing the composition of the datasets from a 1:1 ratio of basal to luminal tumours to a 2:1 or 5:1 ratio of tumours reduced correspondence between datasets and caused a distortion across datasets (Additional File 7). Similar results were also observed with the MCF7/MCF10A datasets described above (Fig 1F, Table 3). Weighted-mean-centering for ER status removed the distortion but also reduced correspondence for the 2:1 ratio of luminal tumours, and increased correspondence in the 5:1 ratio luminal to basal comparison, at the expense of high false discovery rates (Table 3). An extreme example of the effect of dataset composition was seen when looking at the expression level of the estrogen receptor in homogeneous datasets from Loi et al. 32 and Minn et al. 33 composed wholly of ER+ or ER- tumours. Following mean-centering it appears that these datasets contain a mixture of ER+ and ER- tumours (Additional File 8A). Replacing any of the six heterogenous datasets above (containing 67–85% ER+ tumours) with homogeneous datasets (containing only ER+ or ER- tumours) showed a dramatic reduction in the correlation between dataset or tumour number and prediction of recurrence (Additional File 8B). Using weighted mean-centering to account for the differences in the composition of ER+ tumours in five out of the six datasets (individual ER status by immunohistochemistry for tumours in the Pawitan et al. dataset was not available) did not significantly improve prognostic performance over mean-centering alone (Additional File 9).

The mean-centering approach was compared with a previously described method for integrating breast cancer tumour microarray data generated on different platforms, using a distance weighted discrimination (DWD) method to adjust for systematic microarray data biases 34. For both the validation datasets and the published datasets, mean-centering out-performed DWD (Table 3 and Additional File 10).

An alternative approach to assess whether mean-centering improves comparisons across published datasets was to identify lists of the five hundred most highly differentially expressed probesets across each dataset (those with the highest variance) and compare these gene lists with the most differentially expressed probesets from other single or combined datasets. Bringing together greater numbers of datasets or tumours increased the overlap in differentially expressed probesets (Figure 5). The number of probesets in common was significantly greater with mean-centering (p = 3.6 × 10^-107) or weighted mean-centering (p = 9.2 × 10^-106) over uncorrected data, although there was no significant improvement between the two methods (p = 0.5). The mean number of genes in common was higher following weighted mean-centering than mean-centering when the dataset was made up of less ER-positive tumours (Chin et al., Farmer et al. and Richardson et al.) and lower when the dataset was made up of more ER-positive tumours (Ivshina et al., Sotoriou et al., Wang et al.).

Generation and processing of RNA comparing the two breast cell lines, MCF7 (cancer) and MCF10A (normal immortalised mammary epithelial) was described previously 4546. Briefly, cells were grown in Dulbecco's modified Eagle's medium (DMEM) with 10% fetal calf serum (MCF7) or DMEM/F12 with 5% horse serum, 2 ng/ml, 0.5 μg/ml hydrocortisone, 0.5 μg/ml cholera toxin, and 5 μg/ml insulin (MCF10A). Minimally passaged cells (< 20) were obtained from the American Type Culture Collection ATCC. RNA was isolated using Trizol^® (Ambion) according to manufacturer's instructions, purified using Qiagen RNeasy columns (Qiagen, Valencia, CA) and quantified using a Nanodrop spectrophotometer (Labtech). The quality and amount of starting RNA was confirmed with an Agilent Bioanalyzer 2100 (Agilent) prior to labelling and hybridisation to HG-U133A, HG-U133 plus 2.0, or Human Exon 1.0ST GeneChips (Affymetrix) using either the Affymetrix standard or small sample preparation protocols as previously described 45.

Each experiment was repeated in triplicate, with three samples per cell line for each amount of starting total RNA, protocol or GeneChip used (1 dataset = 3 × MCF7 and 3 × MCF10A = 6 GeneChips). The HGU133A Genechips for the standard protocol and amplification experiments were scanned using a GeneArray 2500 scanner and the HG-U133 plus 2.0, and Exon 1.0ST Genechips were scanned using a GeneChip Scanner 3000. All MCF7 and MCF10A microarray data is MIAME compliant and accessible via MIAME VICE 47. All protocols are described in full here, http://bioinformatics.picr.man.ac.uk/mbcf/downloads/. Raw spot readings were processed using R 31 and Bionconductor 48. Probeset summarisation was done using MAS 5.0 and RMA 49 as implemented in the Simpleaffy package 50 or plier algorithm from Affymetrix ExACT software. Mappings between the Exon and U133A plus 2.0 GeneChips were performed as described previously 46. Alternative cell description files, relating probesets to unigene sequences were implemented as described previously 5. Lists of common significantly differentially expressed genes before and after mean batch-centering were identified using SAM 51 analysis (siggenes BioConductor package) by adjust Δ value to find the top 1000 differentially expressed probesets using each protocol, as described previously 46.

The mean expression level per probeset for a given dataset is subtracted from the individual GeneChip expression level on the Log2 scale. This can simply be performed in R using the 'rowMeans' function. Whilst preparing the manuscript we also noticed that this can be achieved using the 'pamr.batchadjust' function within the pamr Bioconductor package.

Affymetrix data was downloaded from a total of ten datasets from published studies listed in Table 2 from the Gene Expression Omnibus 3 or Array Express 1 repositories. Raw .cel files were not available for the Wang et al. dataset, so all other datasets were normalized as in this study using the MAS 5.0 algorithm with a target intensity of 600 as implemented in the Simpleaffy package 50, using R 31 within BioConductor 48. NetAffx 9 was used to identify Affymetrix probesets representing the 'intrinsic gene set' previously used to classify human breast tumours 8. Centered average linkage clustering was performed using the Cluster 35 and TreeView programs as described previously 7. Supervised principal components analysis using the Superpc for R package was used as previously described 2930, in order to compare the predictive power of combining different published datasets. The follow up endpoints for the Loi et al., Pawitan et al. and Sotoriou et al. datasets were recurrence-free survival, for Desmedt et al. and Ivshina et al. datasets it was disease-free survival and for the Minn et al. and Wang et al. datasets it was distant metastasis-free survival.