Bayesian models for pooling microarray studies with multiple sources of replications1 Department of Mathematics and Statistics, University of Massachusetts, Amherst, Massachusetts, USA 2 Department of Statistics, Harvard University, Cambridge, Massachusetts, USA
BMC Bioinformatics 2006, 7:247doi:10.1186/1471-2105-7-247 The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2105/7/247
©
2006 Conlon et al; licensee BioMed Central Ltd. AbstractBackgroundBiologists often conduct multiple but different cDNA microarray studies that all target the same biological system or pathway. Within each study, replicate slides within repeated identical experiments are often produced. Pooling information across studies can help more accurately identify true target genes. Here, we introduce a method to integrate multiple independent studies efficiently. ResultsWe introduce a Bayesian hierarchical model to pool cDNA microarray data across multiple independent studies to identify highly expressed genes. Each study has multiple sources of variation, i.e. replicate slides within repeated identical experiments. Our model produces the gene-specific posterior probability of differential expression, which provides a direct method for ranking genes, and provides Bayesian estimates of false discovery rates (FDR). In simulations combining two and five independent studies, with fixed FDR levels, we observed large increases in the number of discovered genes in pooled versus individual analyses. When the number of output genes is fixed (e.g., top 100), the pooled model found appreciably more truly differentially expressed genes than the individual studies. We were also able to identify more differentially expressed genes from pooling two independent studies in Bacillus subtilis than from each individual data set. Finally, we observed that in our simulation studies our Bayesian FDR estimates tracked the true FDRs very well. ConclusionOur method provides a cohesive framework for combining multiple but not identical microarray studies with several sources of replication, with data produced from the same platform. We assume that each study contains only two conditions: an experimental and a control sample. We demonstrated our model's suitability for a small number of studies that have been either pre-scaled or have no outliers. BackgroundcDNA microarrays monitor gene expression for thousands of genes simultaneously. Two experimental conditions are compared by examining the ratio of expression between two samples, e.g. treatment versus control, wildtype versus mutant, or disease versus healthy. The primary goal of these experiments is to identify genes that are differentially expressed between the two conditions. The up-regulated and down-regulated genes shed light on biological mechanisms of the cell, such as functional pathways, response to treatments, and gene regulation. Bayesian models have been used extensively in single microarray studies to identify differentially expressed genes (e.g. [1-3]); the discrete mixture model approach in particular has had considerable use [4-13]. Additional Bayesian methods for identifying differentially expressed genes include Bayesian ANOVA for microarrays (BAM) of Ishwaran and Rao [14,15]. This approach redefines the search for differentially expressed genes as a Bayesian variable selection process and uses a hierarchical model that is tailored to adaptive shrinkage. Through the use of model averaging, BAM essentially shrinks only the effects of the non-differentially expressed genes relative to the least squares estimates. For a review of Bayesian approaches to microarray data analysis and their advantages over frequentist methods, see Yang et al. [16]. In addition to performing single microarray studies, biologists often conduct multiple but not identical studies to understand the same biological system. Pooling results of these studies can help identify truly differentially expressed genes. Meta-analyses for microarray studies have been used recently by many researchers in a non-Bayesian context [17-24]. Rhodes et al. [17] combined the results of four prostate cancer studies using a p-value approach. Genes were assigned a p-value in each study separately, and the results were combined to estimate a gene-specific p-value across all studies. This method avoids the necessity of integrating gene expression measures and thus can be used for data across multiple platforms. Choi et al. [18] presented a meta-analysis that integrated gene effect sizes, rather than p-values, into one mean effect. The effect sizes for each study were equal to the mean differences between affected and control groups, standardized by a pooled standard deviation. Due to this standardization, data was able to be integrated across platforms. A common parameter for inter-study variability was incorporated into the model, and statistical significance was determined by permutation tests. Parmigiani et al. [20] introduced an integrative correlation approach to combining data from multiple platforms. This procedure evaluated gene expression consistencies across platforms rather than pooling gene expression values. Using lung cancer data, this method identified genes with reproducible expression patterns across studies and improved correlation across studies. Additional theoretical approaches for combining data from different platforms include adding covariates to models to account for the differences among data types [24,25], although this has not been applied in a microarray setting. While the studies of Rhodes et al. [17], Choi et al. [18], Parmigiani et al. [20] and others provide methods for integrating data across platforms, other authors have shown the difficulties in such an approach ([26,27]; discussions in [25,28]). Working with cell lines, Kuo et al. [26] and Jarvinen et al. [27] both conclude that combining data across platforms is unreliable. Due to these difficulties, other meta-analysis methods focus on incorporating data from one platform only [23,24]. Here, we focus our approach to combine microarray data from the same platform, cDNA microarrays, and assume that the data has either been pre-normalized across studies or that there are no outlying studies. Choi et al. [18] also provided an alternative Bayesian meta-analysis method to their random effects approach. In this Bayesian model, uninformative prior distributions were assigned to the overall mean effects and the inter-study variation parameter. Within-study gene effects were modelled as t-distributions, and posterior estimates of the overall mean effect for each gene were produced by smoothing effects across studies. The authors demonstrated that Bayesian meta-analysis is more robust and flexible than traditional methods, confirming the findings of DuMouchel and Harris [29]. Bayesian models are also well-suited to data with many levels of replication, including replicate slides within repeated identical experiments. Due to these advantages, we introduce a Bayesian hierarchical model that provides a principled framework for incorporating data from multiple independent cDNA microarray studies with several sources of replication. Unlike the approach of Choi et al. [18], which smoothes the gene effects into one average, our method produces the posterior probability of differential expression based on gene expression levels across studies. Thus, inter-study variability does not need to be estimated by our model. The probability of differential expression provides a direct method for ranking genes, and also for estimating both integration-driven discovery rates and false discovery rates. In simulations, we illustrate that pooling studies increases the number of discovered genes for given thresholds of probabilities of differential expression and false discovery rates, compared to individual studies. In addition, for a fixed top number of genes, the pooled model identifies considerably more differentially expressed genes than separate studies. We also illustrate our method using experimental data from two independent studies in Bacillus (B.) subtilis. Background on cDNA microarray experimentscDNA microarrays measure the amount of messenger RNA (mRNA) contained in an experimental sample. They are produced by robotic arrayers, which place entire gene sequences complementary to mRNA onto glass slides. In an experiment, the mRNA in two samples, e.g. treated and control, are fluorescently labeled with two different dyes, typically red and green (Cy5 and Cy3), and mixed together. The combined sample is hybridized to the array, and complementary sequences bind to each other. The relative amounts of mRNA present in the two samples are measured by scanning the slide with two different wavelengths. The resulting fluorescent intensity values for the red and green-labeled mRNA are then compared by using the ratio of intensities. For further details, see [30-33]. We use log-ratios of intensities in each study since they even out highly skewed distributions and give a more realistic sense of variation [34]. Results and discussionBayesian model for pooling multiple studiesBiologists often conduct multiple but different studies that all target the same biological system or pathway. For example, when studying the effect of a key transcription factor σE in B. subtilis, Eichenberger et al. [35] conducted both the σE knockout and the σE over-expression experiments (i.e. the mutant and induction experiments, respectively; see Methods). Thus, those genes that are up-regulated in one experiment should be down-regulated in another. Pooling both experiments can help more accurately identify true target genes. More generally, we may imagine having available multiple independent studies of one specific biological system. We assume that each study contains only two conditions: an experimental and a control. It is desirable to combine information from these studies in a principled way. Our model to achieve this goal is as follows: for j = 1,..., J independent studies. Here, yjgse is the microarray data, i.e. the normalized log-expression ratios for gene g, experiment e, slide s, and μjge is the average over all slides Se within experiment e of study j. θjg is the log-expression ratio for each gene of study j. Conjugate inverse chi-squared prior distributions are assigned to Here, genes are divided into two groups, non-expressed (Ig = 0) and expressed (Ig = 1), with respective probabilities (1-p) and p. The model produces the posterior distribution for Dg = Prob(Ig = 1|data), which is the basis for inference. For prior distributions, when Ig = 0, we assume the θjg are distributed normally with mean zero and small variance Integration-driven discoveryChoi et al. [18] define the integration-driven discovery rate (IDR) as the number of genes discovered in a meta-analysis that were not discovered in any of the individual studies alone, divided by the total number of discoveries. IDR represents the gain in information from combining studies versus individual studies. For our model, we fix a threshold value, γ, and label genes differentially expressed if (Dg ≥ γ). The IDR is defined as the number of genes that are labelled differentially expressed in the pooled analysis and are not differentially expressed in any of the individual studies: False discovery rateBenjamini and Hochberg [37] introduced the false discovery rate (FDR), which is defined as the number of false discoveries divided by the number of discoveries. We refer to this as the true false discovery rate (tFDR), which can be exactly computed in our simulation studies since we know which genes are truly differentially expressed. Further applications of FDR to microarrays include [38-40]. Genovese and Wasserman [41] define the posterior expected FDR (peFDR) as: with δg an indicator for differentially expressed genes (see also Do et al. [13]). In the simulated data, we use tFDR and compare tFDR to peFDR; for the experimental data, we have no choice but to use peFDR. Simulation results for two studiesWe simulated data for two studies, Study 1 and Study 2, with similar format to the B. subtilis mutant and induction studies, using three different values for the percent of truly differentially expressed genes: p = 5%, 10% and 25% (see Methods). We implemented Model (1) for each of the two simulation studies separately and in a pooled analysis. The IDR ranged from 1.9% to 42.9% for all values of p for γ ≥ 50%, with maximum tFDR of 5% for the pooled analysis (Table 1). Note that tFDR is low for large values of γ due to the simulation procedure. The IDR increases as γ increases. IDR is also smaller for larger values of p (Figure 1a); this is due to the larger variability between studies. As a result, fewer genes have Dg less than γ in both studies separately, which reduces IDR. In addition to identifying highly expressed genes by choosing a γ threshold, researchers often choose a maximum tFDR and examine lists of differentially expressed genes with corresponding tFDR. In Figure 1b, we display all tFDR levels < 20% for p = 10% and show the number of discovered genes for the two individual studies and the pooled analysis. This plot shows the considerable increase in the number of differentially expressed genes found in the pooled analysis versus the separate analyses for the same level of tFDR. Table 1. Results for two-study simulations. Integration-driven discovery rate (IDR) and the number of discovered genes for various threshold values of the posterior probability of differential expression, γ, and three simulated levels of the percent of differentially expressed genes p = 5%, 10%, 25%. The true false discovery rate (tFDR) is controlled at 5% for all pooled studies.
In addition to choosing a threshold value of Dg or FDR, researchers are often interested in the top set of genes only, i.e. the top 300 genes. For this reason, we rank the genes based on Dg in both the pooled and individual analyses and compare the resulting numbers of differentially expressed genes that are included in the top genes. For each of the three simulation studies, p = 5%, 10%, 25%, we choose a threshold of the top p% of genes. We find that the pooled model always identifies a larger number of differentially expressed genes than individual studies (Table 2). Table 2. Number of differentially expressed (D.E.) genes for fixed top numbers of genes. The number of differentially expressed genes discovered by the pooled model and individual models for fixed threshold numbers of top genes, including the two-study simulation model, p = 5%, 10%, 25%, and the five-study simulation model, p = 10%. We also compared peFDR to tFDR for the simulation data to ensure that our peFDR is a reasonable approximation to the true values. As seen from Figure 2, which displays all values of peFDR versus tFDR for the simulation results, the peFDR was always larger than tFDR, so that peFDR is a conservative estimate of tFDR. The average differences between peFDR and tFDR were less than 2.3% for all pooled simulation results. The maximum difference between peFDR and tFDR decreased as the simulated percent of truly differentially expressed genes p increased. Specifically, for p = 5%, the average difference between peFDR and tFDR was 2.3%, with maximum difference of 12.6% at tFDR = 19.9%. For p = 10%, the average difference was 2.2%, with maximum difference of 7.8% at tFDR = 3.1%. For p = 25%, the average difference was 2.3%, with maximum difference of 5.1% at tFDR = 23.8%.
Simulation results for five studiesWe also assessed our model for a meta-analysis with five studies. For this, we used the same Study 1 and Study 2 as in the previous section, and p = 10%. We then simulated three further studies similar to Study 1, but with different model parameters for
We also show the number of differentially expressed genes identified by the pooled versus individual analyses for a fixed value of top expressed genes in Table 2. For the top 300 genes, the pooled model again identifies more differentially expressed genes than individual studies. We also compared peFDR to tFDR in Figure 2d. The average difference was 0.54%, with maximum difference of 2.7%at tFDR = 0.36%. These values are smaller than the results for the two-study simulations, showing improved accuracy of peFDR when pooling more data. Experimental data resultsWe implemented Model (1) to pool the mutant and induction B. subtilis studies, with 2,515 genes that had expression in both studies. We also implemented Model (1) for each study individually. For values of γ ≥ 50% and maximum peFDR of 11.5% for the pooled analysis, the IDR ranged from 8.2% to 53.3% (Table 3). We plot IDR versus γ in Figure 4a. Figure 4b presents the number of discovered genes for peFDR < 20% for both the separate and pooled analyses. The induction study had much lower log-ratios of expression than the mutant study; the average 97.5%-ile for the induction experiments was 0.65 versus 1.59 for the mutant experiments. As a result, the maximum Dg value for the induction study was 0.84, with minimum corresponding peFDR of 16%. In contrast, the mutant study had 33 genes with Dg of 1.0. Even though the values of Dg were lower for the induction than the mutant study, we found that combining the two data sets resulted in more discoveries of differentially expressed genes than either study alone for fixed levels of peFDR. Table 3. Results for Bacillus subtilis experimental data. Integration-driven discovery rate (IDR), posterior expected false discovery rate (peFDR) and the number of discovered genes for various threshold values of the posterior probability of differential expression, γ, for the pooled analysis of the B. subtilis mutant and induction experimental study data.
ConclusionWe demonstrated here the usefulness of a Bayesian hierarchical model for pooling data across independent microarray studies with several sources of variation. The pooled method provides a systematic analysis framework, producing probability estimates of differential expression for each gene. These estimates are used to rank genes, calculate IDR, and produce posterior expected FDR values. In the simulation of two and five studies, we found an appreciable increase in the IDR for various thresholds of the probability of differential expression, with corresponding low levels of tFDR. When fixing tFDR, we found more genes discovered in the pooled analysis than the separate analyses. When setting a threshold for the top genes of interest, the pooled model identified more truly differentially expressed genes than individual analyses. In the simulation of five studies, the IDR was somewhat lower than for two studies, but was still considerable. When comparing the peFDR to tFDR in simulations, we found reasonable agreement, with peFDR overestimating tFDR on average by less than 3%. The difference between peFDR and tFDR decreased for the simulation of five studies, indicating that pooling more data improves the posterior estimation of FDR. In our analysis of experimental data, the IDR was also large. One study had somewhat lower probabilities of differential expression, which resulted in more discoveries when the data was pooled. We conclude that combining information across studies strengthens the probabilities of differential expression, improves IDR, and increases the number of discovered genes for fixed tFDR, peFDR and fixed top percent of genes than individual study analyses. Our model is designed for studies from the same platform. In the B. subtilis experimental data, a common control sample was used for the mutant and induction studies. However, our model does not require a common reference sample across studies, and assumes the studies are independent. In addition, all studies do not need to have the same array-design layout. This is due to the studies being linked only through the common parameter of differential expression, p; no other parameters are shared between studies. For example, one study could have only replicate slides, and another study could have both replicate slides and replicate experiments. We also assume that there are either no outlying studies or that the data has been scaled across studies before analysis. Future work will address the issues of pooling studies from different platforms and sets of studies that may contain outliers. MethodsSimulation data for two studiesWe simulated data for two studies, with the same format as the B. subtilis mutant and induction experimental data (see Methods: Experimental data), with the percent of differentially expressed genes of p = 5%, 10% and 25%. Each study had 3,000 genes; the first study had 5 replicate slides within 3 replicate experiments, and the second study had 4 replicate slides within 3 replicate experiments. We simulated data from Model (1), with parameters similar to those found in the experimental data. For Study 1, we used Between study variance for the experimental data was 0.067 for all 2,515 genes, and 0.296 for the top 10% of genes. The simulated data had similar between study variance for all genes, and higher variability for the top genes than the experimental data. The between study variance was 0.053 for p = 5%, 0.105 for p = 10% and 0.23 for p = 25% for all 3,000 genes, with between study variance for the top genes of 0.714 for p = 5%, 0.887 for p = 10% and 0.86 for p = 25%. For each gene, log-expression ratios are simulated from normal distributions, independently of other genes. Although expression is expected to have some correlation among genes, this is difficult to model, and we thus assume independence for simulation purposes. The independence assumption was also used in simulation studies by other authors (see, for example, [8,9]). Simulation data for five studiesFor the simulation of five studies, we used the Study 1 and Study 2 data from the previous section, and simulated data for 3 additional studies, with p = 10% for all studies. For Studies 3, 4 and 5, we simulated 5 replicate slides within 3 replicate experiments. For the parameters Experimental dataThe B. subtilis experiments were designed to identify sporulation genes under the control of sigma factor E (σE). Two complementary experimental setups were used, the first was a deletion of σE (mutant study) and the second an overexpression of σE (induction study), described in the following (for additional details, see [35,42]). Mutant studyIn the mutant study, the treated sample contained sporulating cells with a null mutation in the gene for σE (i.e. the mutant sample), and the control sample contained sporulating cells that were wild type for σE. The wild-type/mutant ratios were examined; up-regulated genes were identified as belonging to the σE regulon. In total, five microarrays were produced from three independent identical experiments; the first experiment had three replicate arrays and the second and third experiments each had one array. The number of genes spotted on the five arrays ranged from 4,268 to 4,751; these values are larger than the B. subtilis genome size of 4,106 due to multiple spotting of selected genes on various arrays. The percent of low quality spots that were removed from analysis ranged from 18.6% to 64.5% of values across the five arrays. In total, there were 3,713 genes with measurable expression ratios in at least one microarray. Here, we analyze values after normalization using a rank-invariant method [2,43]. Induction studyIn the induction study, σE was overexpressed in response to an inducer, i.e. cells that had been treated with an inducer were compared to control cells. The induction/wild-type ratios were examined; up-regulated genes were identified as belonging to the σE regulon. In total, four microarrays were produced from three independent identical experiments. The first two experiments each had one array, and the third experiment had two replicate arrays. The number of genes spotted on the four arrays ranged from 4,608 to 4,751; the percentage of genes detected on the arrays ranged from 33.0% to 47.4%. In total, there were 2,552 genes with measurable expression ratios in at least one microarray. Here, we again analyze the post-normalized values. Markov chain Monte Carlo implementationIn the Markov chain Monte Carlo analysis, the full conditionals are simulated as follows. Joint posterior distributionFor the hierarchical model of (1), the joint distribution of the data and parameters is: where Ωj = ( Prior distributionsThe prior distributions are specified as follow. Here, where yjg.e is the average log-ratio of expression over the slides within an experiment: Similarly, the scale parameter for where yjg.. is the average log-ratio of expression over both slides and experiments. We use 3 degrees of freedom in each study for both We choose a, Full conditional posterior distributionsEach parameter is sampled from the full conditional posterior distributions by the following. Here, For Ig = 0, the following full conditionals are sampled: For Ig = 1, the following full conditionals are sampled: For all iterations, the following are sampled: Here G = total number of genes g, G' = set of genes with Ig = 1 in an iteration, and Yg is the data from all studies. Since the full conditional posterior distributions are all closed form when conditioned on the values of Ig, the Gibbs sampler [36] is used to generate samples from these distributions. We used 5,000 iterations for all analyses, except for the five study simulation, which required 8,000 iterations, which was more than adequate. The calculations are implemented using the WinBUGS software [44]. Availability and requirementsThe WinBUGS code for executing the models is freely available. Project name: BayesPoolMicro. Project home page: http://www.math.umass.edu/~conlon/research/BayesPoolMicro/ webcite Operating system: Windows 98 or later. Other requirements: WinBUGS software version 1.4 or later [44]. License: free. Authors' contributionsEMC and JJS contributed to writing the computer code. All authors contributed to the development of the methodology and to writing the manuscript. AcknowledgementsWe thank George Tseng, Jeffrey Townsend and John Staudenmayer for helpful discussion, and Patrick Eichenberger and the laboratory of Richard Losick for the B. subtilis microarray data and helpful advice. We also thank three anonymous referees for input that enhanced the manuscript. EMC was partially supported by a University of Massachusetts Healey Endowment Grant, and JSL was partially supported by the NIH Grant R01-HG02518-01, and the NSFChina Grant 10228102. References
Have something to say? Post a comment on this article! |




on Google Scholar







author email
corresponding author email











Figure 1.
Figure 2.
Figure 3.
Figure 4.












