We carried out three simulations, with similar settings and the noise level gradually increasing from simulation 1 to 3. In simulations, we assumed that the genes within an operon are co-transcribed. Since the true expression levels in a simulated data set were known, we could calculate the mean squared errors of the estimated expression levels (summarized in Table 1). Without incorporating the information from operons, the sample mean of the observed expression levels would be a natural estimator of a gene's expression level. The mean squared errors of the two estimates, the posterior mean from the proposed model and the sample mean, are shown in Table 1 for comparison. It can be easily seen that the incorporation of operon information leads to a better estimate, with a smaller mean squared error, of expression levels. When the noise level increased, the improvement from incorporating operon information also increased.

To evaluate the performance of the proposed method in detecting DE genes, the estimated expression level of each gene was used to rank the genes, and the highly ranked genes were identified as DE genes. Since the identities of DE genes in the simulation studies were known, we compared the performances of the proposed method, sample mean, and SAM t statistics in detecting DE genes using receiver operating-characteristic (ROC) curves (Figure 1). In a ROC curve, the sensitivity is plotted against 1 – specificity. The sensitivity is denned as a fraction of true DE genes being correctly detected and the specificity is a fraction of the true EE genes being correctly identified. The ROC curves in Figure 1 demonstrate that the performance of the sample mean and of the SAM t statistic were very close, and our hierarchical model, incorporating operon information, outperformed both of them. The difference in the performance became greater as the noise level increased. For example, as the specificity equals to 0.8, the sensitivities of the methods using the sample mean or SAM t were about 0.91, 0.80, and 0.68 for simulations 1,2, and 3, respectively, while the sensitivities of the proposed method were 0.95, 0.89 and 0.83, respectively.

To verify the assumption that the genes organized in operons are co-expressed, we pooled together data from 217 microarray experiments, obtained in 53 conditions 24. The distribution of pairwise correlations between expression profiles of genes in operons was greatly skewed towards positive values, with the mean correlation of 0.62 (Figure 2A). Unlike the profiles of genes organized in operons, expression profiles of randomly picked pairs of genes were not correlated; the corresponding distribution of correlation coefficients was almost symmetric around 0, with the mean correlation of 0.012 (Figure 2B). This result demonstrated the similarity of transcriptional activity of genes within operons and served as a motivation for borrowing information from other genes within the same operon.

The proposed method [see Additional file 1] was used to analyze differential transcriptional activity in an E. coli mutant lacking the flhDC gene, a master regulator of transcription of genes whose products mediate bacterial motility and chemotaxis [see Additional file 2]. The genes were ranked by their estimated expression levels, i.e. their posterior means of μ_i obtained from the proposed model. For the sake of comparison, the sample mean and SAM t statistics were also used to rank the genes [see Additional file 3]. Using the functional annotation from Macnab 25 as a standard, we obtained the number of false positives at different cut-off levels (total positives). The comparison revealed that ranking genes by the proposed method produced fewer false positives than the ranking based on the SAM t or sample mean statistics (Figure 3).

To find a reasonable cutoff value for differently expressed genes, we calculated the false discovery rate (FDR) 2627based on the posterior probability 7. In this experiment, we also estimated the FDR by using the functional annotation from Macnab 25 as a reference. Comparison of the estimated False Discovery Rates revealed that the estimated FDR from the posterior probability was a little lower than that derived from the annotation, which could be due to the partial incompleteness of the reference (Fig. 4). Overall, the estimated FDR from the posterior probability is close to the FDR using the reference list of genes, indicating that our method for estimating the FDR is adequate. We set the cutoff for the FDR to be 0.01, which identified the top 44 genes as DE genes. At such a cutoff, the estimated number of false negatives is 14 and the estimated false negative rate is about 0.003 (see the "Methods" section for details). The top 44 genes are listed in Table 2. Note that, in Table 2, the gene expression level is on the log scale and the FDR corresponds to a specific number of DE genes and not to each individual gene itself. According to Macnab's classification 25, 41 genes out of the 44 were expected to be differentially expressed in the flhDC- dependent manner, whereas the lists of 44 genes identified by using SAM t and sample mean contained only 36 and 38 expected genes, respectively.

We examined some genes and operons in more detail, to demonstrate the advantages of borrowing information from within an operon. For example, an operon argT-hisJQMP contains 5 genes (argT, hisJ, hisM, hisP, hisQ) and is not expected to be differentially expressed under the examined experimental condition, according to our biological knowledge 25. But from the microarray data, the mean expression level (an average log ratio) of the gene argT was -2.73, which ranked 15th among all the expression levels in the E. coli genome. This "high expression" of the argT could possibly be caused by random noise in microarray measurements and/or in biological samples, since the mean expression levels of other genes within the same operon were close to 0 (those for genes hisP, hisM, hisQ, hisJ were 0.00, -0.05, 0.03, and 0.05, respectively). However, accounting for expression levels of other genes within the same operon lowered the estimate of the expression level (posterior mean of μ_i of the argT to -0.02, rank of 1456, indicating that this gene was not differentially expressed. Analysis of another operon, fliDST, illustrates a complimentary case. Transcription of the fliDST operon (containing 3 genes, fliD, fliS, fliT) is known to be controlled by the FlhDC 25, and thus, under the experimental condition, differential expression of genes in that operon would be expected. While the expression level of the fliT, estimated by the sample mean at -0.90, ranked only 65th, the estimated expression level of the gene after borrowing information from two other genes in the operon was -3.25, which ranked 19th.

In general, through borrowing information, our Bayesian method worked in a way giving more consistent estimates of the expression levels for the genes of the same operon. For example, compared with using the sample mean to estimate expression levels, the Bayesian method tended to yield smaller standard deviations of the expression estimates for within-operon genes (see Figure 5).

RegulonDB 40 is a database containing information about known operons in E. coli. According to the RegulonDB annotations, 1486 genes (about one third of all genes predicted in the E. coli genome) are organized in 600 operons.

The E. coli data set contains results of 217 microarrays collected in 53 different experimental conditions. The fluorescent intensities of the test and control samples were measured, and the average log ratio of the intensities for each gene under the same condition was used here to represent an observed gene expression level under that condition 24.

An E. coli motility expression data set (41 series accession number: GPL2101) was obtained in a direct pair-wise comparison between a knock-out mutant of the flhDC, a master regulator of the motility/chemotaxis regulon 25, and its isogenic wild type strain. Total RNA samples of a mutant E. coli (test samples) and an isogenic wild type E. coli (control samples) were labeled with red (Cy5) and green (Cy3) fluorophors. The intensities from the red and the green channels were normalized by the lowess method 42. There were 4281 genes (G = 4281) with four replicates for each gene (n = 4). Let Y_i,j be defined as the log ratio of the intensities between the test and control samples for gene i on array j; that is,

We propose a hierarchical Bayesian model,

where, Y_i,j is the log ratio of gene i in replicate j, μ_i and σ_i are the true expression level and the standard deviation, respectively. In our method, the posterior mean of μ_i is used as the estimated expression level of gene i, while the sample mean, ., is referred to as the observed expression level.

As prior knowledge, we assume that if several genes belong to the same operon, in accordance with the RegulonDB annotation, then their expression levels are from a normal distribution, with the mean λ_p and the variance τ². Specifically,

where O_p denotes operon p. λ_p represents the expression level of the operon p, which is the average of the mean expression levels of all genes within the operon p. τ² is the within operon variation, and is assumed to be the same across all operons. A non-informative prior is assigned to λ_p, that is Pr(λ_p) α 1, to reflect the lack of prior information, and τ² have vague priors, which are inverse Gamma distributions with the shape and rate parameters equal to 0.01 and 0.01 respectively 43. If gene i is not in any operon, then

so the posterior mean of μ_i is just the the sample mean ; if gene i is in operon p, then the conditional distribution of μ_i can be derived as:

where

Equation (3) shows that, when borrowing information from the other genes within the same operon, the estimated expression level of the gene i becomes the weighted average of the observed expression level of gene i and the expression level of operon p, given that gene i belongs to operon p. The weights are inversely proportional to the variances. In this model, a key concept is to shrink the observed expression level , towards λ_p, the expression level of an operon, based on the knowledge of the operon structure. The degree of shrinkage is determined by the variability of . and λ_p. Without incorporating operon information, the estimated expression level would be close to the observed expression level, In the hierarchical model, λ_p represents the expression level of operon p, and

where, m_p is the number of genes in operon p, and i ∈ O _p denotes that gene t is in operon p. Although the posterior distribution was not available in a closed form, we could derive a closed form of the full conditional distribution, and used Markov chain Monte Carlo (MCMC) to simulate the parameters from the posterior distribution. With this closed form expression, the model could be easily coded in R [see Additional file 1] for MCMC simulation using Gibbs sampling 44. The expression level of gene i is estimated by the posterior mean of μ_i, and the genes are ranked by the absolute values of the posterior means of the μ_i's. Genes with high rankings were designated as differentially expressed (DE) genes.

To evaluate the performance of the proposed method in estimating gene expression levels and identifying DE genes, we compared the proposed method to the sample mean and SAM t statistics 13. Because of its good performance, the SAM t statistic 13 is widely used to rank genes and detect DE genes 45. We denote the SAM t statistic for the gene i as Z_i, then

where and S_i are the sample mean and sample standard deviation for the gene i, and So is the 90 th percentile of S_i's.

We conducted three simulation studies to assess the usefulness of our method. The operon structure of the E. coli genome from the RegulonDB database 40 was used in simulation studies. We randomly chose 100 operons (involving about 340 genes) and assumed that genes from those operons were differentially expressed DE genes. Then we randomly picked a subset of non-operon genes to be DE genes and adjusted the total number of DE genes to 400. Let μ_i be the expression level of gene i, for i = 1, 2,..., 4821. For DE genes, μ_i's were simulated from an equal mixture of N(1, 0.25²) and N(–1,0.25²) distributions, and genes within the same operon were from the same component of the mixture distribution. For EE genes, μ_i ~ N(0, 0.25²). We simulated 4 replicates from a normal distribution for each gene, Y_ij ~ N(μ_i,), where Y_i,j was the log ratio of transcript abundances for the gene i on the array j. To provide increasing noise levels for simulations 1, 2 and 3, the σ_i's were simulated from the uniform(0.25, 0.75), uniform(0.5,1.0), and uniform(0.75,1.25), respectively.

Using the posterior distributions, we can evaluate the FDR for specific number of DE genes. Using Pr(|μ_i| > δ'|Y_i,j) to estimate the probability of gene i to be a DE gene, we can estimate the number of false positives for a cut off value k 7:

Here, the genes are ranked based on the estimated mean expression level μ_i. In this study, we set δ = 1,

which corresponding to the commonly used 2-fold cutoff.

The false discovery rate (FDR) for the cut off k can be derived as:

Similarly, the number of false negatives for the cut off k can be calculated as:

The algorithm is implemented below:

Set initial values:

FOR t FROM 1 TO T, draws random samples: