Figure 1 displays an example of the dependence of gene variance on expression level, taken from the MEF Ahr^-/- dataset (see Methods section), similar to the observed dependency published previously 3. The fact that such a dependence exists is intuitive, in view of how the data are measured from the microarray images. Spots with low fluorescence level will likely have fewer pixels measured, and the resulting estimate of expression is an average or median of fewer or lower numbers. Furthermore, transcripts that are lowly expressed are changed by a greater proportion by the addition of a few labeled transcripts, and thus may actually vary more in biological tissue samples. This relationship between variance and expression level can be modeled as

s_0g²(α_g) = f(α_g) + ε_g     (1)

where the average log-expression level of gene g is denoted by α_g, f(α_g) is some function of α_g defined on the range of α_g, and s_0g² is the estimated prior variance. As explained below, we chose to model the function s_0g² (α_g) using local regression. The use of local regression differs from the window method of Cyber-T in that the window method pools the standard deviation estimates of all genes in the window, whereas local regression uses a weighted average of the log-variances, where the weight for each gene j depends on the difference between the intensity of gene j and the intensity of the gene g, of interest. This relationship on its own can significantly reduce the uncertainty in the true variance of gene expression variances. For example, the relationship shown in Figure 1 explains approximately 34% of variability in individual gene expression variances.

For our intensity-based method, we follow a hierarchical Bayesian set-up similar to SMT 12. Individual gene variances for genes with similar overall expression levels are assumed to have been generated by a single probability distribution. The parameters for the distribution of the variances, d₀ and s_0g², are termed the hyperparameters, and are estimated from the data using EB theory. In terms of the precision of the gene expression levels, which is defined as the reciprocal of the variance, 1/s_0g² is the mean, andthe hyperparameter d₀ is the prior degrees of freedom and determines the spread of the distribution for a given s_0g². Larger d₀ values result in smaller spread of the distribution for the precision and variance of gene expression levels. Similar to previous methods 311, by assuming a single hyperparameter for the prior degrees of freedom, we make the assumption that the spread of variance estimates about the background variance level is similar across the entire range of fluorescence levels.

Suppose that β^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFYoGygaqcaaaa@2E64@_g is the estimate of the contrast of interest obtained after fitting the appropriate linear model for gene expression data for gene g. In the simplest case when comparing expression levels between two samples, β^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFYoGygaqcaaaa@2E64@_g is just the difference in average log-expression levels for gene g under the two experimental conditions. We assume the β^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFYoGygaqcaaaa@2E64@_g measurements of log-fold change for each gene follow a normal distribution centered at β_g, the actual log-fold change:

β^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFYoGygaqcaaaa@2E64@_g ~ N (β_g, v_gσ_g²)

where σ_g² is the residual variance in the linear model for gene g and v_g is the coefficient of the variance required to calculate the standard error. For a two-sample t-test, v_g is 1/n₁ + 1/n₂ where n₁ and n₂ are the number of observations for each sample. Given the variance σ_g², the sample variance for each gene is assumed to follow a scaled Chi-square distribution with d_g degrees of freedom:

s g 2 | σ g 2 ~ σ g 2 d g χ d g 2 . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaem4zaCgabaGaeGOmaidaaOGaeiiFaWhcciGae83Wdm3aa0baaSqaaiabdEgaNbqaaiabikdaYaaakiabc6ha+naalaaabaGae83Wdm3aa0baaSqaaiabdEgaNbqaaiabikdaYaaaaOqaaiabdsgaKnaaBaaaleaacqWGNbWzaeqaaaaakiab=D8aJnaaDaaaleaacqWGKbazdaWgaaadbaGaem4zaCgabeaaaSqaaiabikdaYaaakiabc6caUaaa@45B4@

We adopt the conjugate prior distribution for σ_g²

1 σ g 2 ~ 1 d 0 s 0 g 2 χ d 0 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWcaaqaaiabigdaXaqaaGGaciab=n8aZnaaDaaaleaacqWGNbWzaeaacqaIYaGmaaaaaOGaeiOFa43aaSaaaeaacqaIXaqmaeaacqWGKbazdaWgaaWcbaGaeGimaadabeaakiabdohaZnaaDaaaleaacqaIWaamcqWGNbWzaeaacqaIYaGmaaaaaOGae83Xdm2aa0baaSqaaiabdsgaKnaaBaaameaacqaIWaamaeqaaaWcbaGaeGOmaidaaaaa@4114@

where d₀ and s_0g² are the hyperparameters for the degrees of freedom and variance, respectively. With this model, the closed-form solutions for the posterior mean of the variance and degrees of freedom given the hyperparameters are:

d f = d 0 + d g s ˜ g 2 = d 0 s 0 g 2 + d g s g 2 d 0 + d g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakqaabeqaaiabdsgaKjabdAgaMjabg2da9iabdsgaKnaaBaaaleaacqaIWaamaeqaaOGaey4kaSIaemizaq2aaSbaaSqaaiabdEgaNbqabaaakeaacuWGZbWCgaacamaaDaaaleaacqWGNbWzaeaacqaIYaGmaaGccqGH9aqpdaWcaaqaaiabdsgaKnaaBaaaleaacqaIWaamaeqaaOGaem4Cam3aa0baaSqaaiabicdaWiabdEgaNbqaaiabikdaYaaakiabgUcaRiabdsgaKnaaBaaaleaacqWGNbWzaeqaaOGaem4Cam3aa0baaSqaaiabdEgaNbqaaiabikdaYaaaaOqaaiabdsgaKnaaBaaaleaacqaIWaamaeqaaOGaey4kaSIaemizaq2aaSbaaSqaaiabdEgaNbqabaaaaaaaaa@50D4@

where df is the posterior degrees of freedom, d_g is likelihood degrees of freedom, and s˜g2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacuWGZbWCgaacamaaDaaaleaacqWGNbWzaeaacqaIYaGmaaaaaa@30A0@ is the posterior mean of the variance. Our goal is to calculate point estimates of hyperparameters so that we can calculate expected values for the posterior parameters, σ_g² and df.

We can now use the moderated t-statistic:

t g i = β ^ g i s ˜ g v g i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWG0baDdaWgaaWcbaGaem4zaCMaemyAaKgabeaakiabg2da9maalaaabaacciGaf8NSdiMbaKaadaWgaaWcbaGaem4zaCMaemyAaKgabeaaaOqaaiqbdohaZzaaiaWaaSbaaSqaaiabdEgaNbqabaGcdaGcaaqaaiabdAha2naaBaaaleaacqWGNbWzcqWGPbqAaeqaaaqabaaaaaaa@3E29@

to test the hypothesis H₀: β_g = 0 vs. H_A: β_g ≠ 0 with df degrees of freedom, where β^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFYoGygaqcaaaa@2E64@_gi is the estimate of log-fold change for gene g and contrast i, and s˜ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacuWGZbWCgaacaaaa@2E2A@_g is the posterior standard deviation.

As demonstrated by Smyth 12, under the null-hypothesis, the resulting moderated T-statistic in IBMT is distributed as Student's-t with df degrees of freedom. Thus, differentially expressed genes can be identified by calculating p-values and making appropriate multiple comparisons adjustments. However, if the data grossly deviate from the distributional assumptions, the moderated t-statistics can be used as a heuristic score for ranking genes based on the likelihood that they are differentially expressed, or an alternative empirical-based multiple comparison adjustment can be made, as in 21.

The formulas for posterior mean of the variance and degrees of freedom assume known hyperparameters d₀ and s_0g. We follow the empirical Bayes approach and estimate hyperparameters from the data. Gene-specific prior variances are estimated from f(α_g) as given in (1), where f(·) is a fitted local regression model of adjusted individual genes' log-variances (see equation 4) on the average log-expression levels. In this way, we avoid having to pre-specify a functional form for this dependency, and obtain predicted variances for each gene given their spot intensities.

To estimate the prior variance and prior degrees of freedom, we use the common empirical Bayesian method of equating the empirical to expected values for the first and second moments of log-variance. According to the hierarchical model, the sampling variance for each gene, marginally, has the following scaled-F distribution 12:

sg2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaem4zaCgabaGaeGOmaidaaaaa@3091@ ~ s0g2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaeGimaaJaem4zaCgabaGaeGOmaidaaaaa@317F@F_dg,d0

Consequently, the log-sample variance is distributed as the sum of a constant and Fisher's Z distribution and has the following expected value and variance:

E (log sg2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaem4zaCgabaGaeGOmaidaaaaa@3091@) = log s0g2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaeGimaaJaem4zaCgabaGaeGOmaidaaaaa@317F@ + ψ(d_g/2) - ψ(d₀/2) + log(d₀/d_g)     (2)

var(log sg2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaem4zaCgabaGaeGOmaidaaaaa@3091@) = ψ'(d_g/2) + ψ'(d₀/2)     (3)

where ψ() is the digamma function and ψ'() is the trigamma function 1222. We denote with e_g the non-constant part of (2) for each gene after solving for log(s_0g²)

e_g = log sg2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaem4zaCgabaGaeGOmaidaaaaa@3091@ - ψ(d_g/2) + log(d_g/2),     (4)

with

E(e_g) = log s0g2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGZbWCdaqhaaWcbaGaeGimaaJaem4zaCgabaGaeGOmaidaaaaa@317F@ - ψ(d₀/2) + log(d₀/2).     (5)

Next, we determine the predicted values for e_g, pred(e_g), as a function of average log-intensities by local regression. We define the prior variance for each gene, s_0g², to be the exponential of pred(e_g) + ψ(d₀/2) - log(d₀/2), by substituting pred(e_g) for E(e_g) in (5) and solving for log(s_0g²). To calculate the prior degrees of freedom we equate the empirical variance of the log-sample variances with the marginal variance in (3) and solve for d₀. As indicated before, we assume a priori that σ_g² varies with g, but its variance is constant for all g. Thus, if d_g's were all the same and ψ'(d_g/2) = c, say, then the marginal variance as given in (3) would be a constant, with a consistent estimator given by

m e a n [ e g − p r e d ( e g ) ] 2 = 1 n ∑ [ e g − p r e d ( e g ) ] 2 . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGTbqBcqWGLbqzcqWGHbqycqWGUbGBcqGGBbWwcqWGLbqzdaWgaaWcbaGaem4zaCgabeaakiabgkHiTiabdchaWjabdkhaYjabdwgaLjabdsgaKjabcIcaOiabdwgaLnaaBaaaleaacqWGNbWzaeqaaOGaeiykaKIaeiyxa01aaWbaaSqabeaacqaIYaGmaaGccqGH9aqpdaWcaaqaaiabigdaXaqaaiabd6gaUbaadaaeabqaaiabcUfaBjabdwgaLnaaBaaaleaacqWGNbWzaeqaaaqabeqaniabggHiLdGccqGHsislcqWGWbaCcqWGYbGCcqWGLbqzcqWGKbazcqGGOaakcqWGLbqzdaWgaaWcbaGaem4zaCgabeaakiabcMcaPiabc2faDnaaCaaaleqabaGaeGOmaidaaOGaeiOla4caaa@5B6E@

This would yield an estimator for ψ'(d₀/2), given by

mean[e_g - pred(e_g)]² - c.     (6)

When d_g's are different, the marginal variances in (3) differ for different g, but by known values ψ'(d_g/2). Thus if we assume that d_g does not vary drastically, in the sense that mean[ψ'(d_g/2)] = (1/n)∑ψ'(d_g/2) approaches a constant c as n gets large, then (6) is a consistent estimate of ψ'(d₀/2). Typically, d_g does not vary substantially with good quality data, and with Affymetrix data d_g is usually constant. Thus d₀ can be estimated consistently by solving

ψ'(d₀/2) = mean[e_g - pred(e_g)]² - mean[ψ'(d_g/2)]

for d₀. Note that if d_g is constant for all genes, then using log s_g² in placement of e_g results in the same solution for d₀.

Simulations were designed to imitate a six slide, single-channel microarray experiment with three treatments and three controls. The simulations were performed to compare the performance of five methods (t-test, fold change, SMT, IBMT, and Fox) with respect to: a) the strength of relationship between variance and signal intensity, b) estimation of the correct prior degrees of freedom, and c) unbiased estimation of the true false positive rate. Average expression intensities were generated assuming a log-normal distribution with a scale parameter of 1.1, shape parameter equal to 0.34, and threshold parameter 5.1. The parameters for this distribution were chosen to closely fit the actual distribution of average expression intensities seen from real experiments (Figure 2a). Simulations were run assuming prior degrees of freedom d₀ ∈ [1, 4, 16, 100]. For each prior degrees of freedom, actual and sample standard deviations were simulated for three different strengths of dependency on average log-intensities (Figure 2b), referred to as low, medium, and high. The specific functional form used for this was

g(x) = p₁e^-0.8(x-5) + p₂

with the following values used for p₁ and p₂: low: p₁ = p₂ = 0.875, medium: p₁ = 1.25 and p₂ = 0.5, and high: p₁ = 1.5, p₂ = 0.25. To determine differences among the methods due to sample size, additional simulations were run for a 4-slide experiment (two treatment, two control) and a 10-slide experiment (five treatment, five control), with the high strength dependency, and an additional simulation was also run for the 6-slide experiment with no dependency of variance on average intensities. In the case of no dependence, IBMT performed nearly identical to SMT. All simulations were performed with 15000 "genes", 300 (2%) of which were designed to be "differentially expressed". Log-ratios for all genes were simulated as described in 12. Actual mean log-ratios for the 300 differentially expressed genes were simulated from the normal distribution N(0, 3•σ_g²), and simulated measured mean log-ratios for all genes were assumed to follow the normal distribution N(μ, σ_g²/3), where μ = 0 if the gene is not differentially expressed, and the simulated log-ratio for the 300 (2%) differentially expressed genes.

The simulation process is summarized here:

For all 15000 genes:

1. Simulate α_g as random draws from a log-normal distribution,

2. Define function, f(α_g), for dependence of variance on α_g,

3. Simulate σ_g² as random draws from d₀*f(α_g)/(chi-square with d₀ degrees of freedom),

4. Simulate s_g² as random draws from σ_g²/d_g*chi-square with d_g = 4 degrees of freedom,

5. W.L.O.G., assume the first 300 genes are differentially expressed,

Simulate their mean log-ratios μ_g as random draws from N(0, 3σ_g²),

6. For the remaining 14700 non-differentially expressed genes

Set μ_g = 0,

7. Simulate estimated log-ratios as random draws from N(μ_g, σ_g²/3).

Results from the simulations indicate that the added complexity of the model is outweighed by the additional gain in information. Four methods were compared in their ability to correctly estimate the false positive rate, using estimated False Discovery Rates (FDR) 23: the simple T-statistic (T), Smyth's moderated T-statistic (SMT), our intensity-based moderated-T (IBMT) method, and Fox's method (Fox). All methods except Fox accurately estimate the percent of false positives, as demonstrated by Figure 3. When the prior degrees of freedom is low, Fox's method underestimates the percent of false positives (Figure 3a and 3b), suggesting the possibility of a real risk of Fox's method to give overly-optimistic results with real data. Control of the true false positive rate under additional parameter sets gave the same results, and may be viewed as Supplemental Figure S2 [see Additional file 1].

We compared the ability of the methods to identify differentially expressed genes by creating false positive rate curves for each parameter set. These were created by ranking the genes by significance level, and then calculating the number of accumulated false positives with rank less than or equal to x. Example false positive rate curves for the five methods are shown in Figure 4. Figure 5 summarizes the results for all parameter sets by presenting normalized areas under the false positive curves described above. All results shown are the average of 100 simulation runs. All methods performed poorly when the data was simulated with only one prior degree of freedom. As the number of prior degrees of freedom increased, the performance of all methods except the simple t-test improved with IBMT overall outperforming the other methods. Fox's method closely followed the performance of the fold change method, with a substantial advantage over fold with high dependence of variance on signal intensity. However, it had poor performance when gene's variances were approximately independent (small prior degrees of freedom). Both these results are probably due to this method's assumption that genes with similar intensities have identical variances. For the simulation with no dependence of variance on expression level, the areas under the false positive curves were the same for both SMT and IBMT. The poor performance of the simple T-statistic in these simulations is most likely related to the low number of experimental replicates. We used four sample degrees of freedom, which was insufficient to accurately measure the variance of each gene separately. In additional simulations performed with higher sample degrees of freedom (8, 12, and 16), the simple t-test showed marked improvement over results based on fewer degrees of freedom, while the other methods did not show as much improvement as the degrees of freedom increased (supplemental Figure S3).

Finally we compared the ability of IBMT to SMT to accurately estimate the prior degrees of freedom (Table 1). Since Fox's prior degrees of freedom is dependent only on the free parameter and sample size rather than estimated from the data (default d₀ = 16 for all 4-slide simulations), Fox was not included in this comparison. As expected, the empirical Bayes method that does not account for the relationship between the variance and the magnitude of expression measurements tends to underestimate the prior degrees of freedom, especially for larger d₀ values. As the dependency of variance on average intensities increases, this bias grows stronger. For the simulation with no dependence of variance on intensity level, using d₀ = 16, both methods accurately estimated the prior degrees of freedom, with estimates of d₀/(d₀+d_g) equal to 0.802 and 0.803 for SMT and IBMT respectively.

Two publicly-available, and completely controlled, "spike-in" Affymetrix datasets were used to compare the performance of the same methods, plus Cyber-T, on real-world microarray data. The analysis of these experiments is a natural extension of the simulation studies as the "correct" results are known. The first experiment consisted of three technical replicates each of control RNA samples and samples with known amounts of spiked-in RNA, and consisting of 3,860 individual cRNAs. We used the average of the top 10 expression datasets, as reported by Choe et al. 14 and available for download at 24. The description of all pre-processing steps used for these expression datasets, as well as further detail of the experimental methods are given in the original publication 3. In the original publication, Cyber-T was determined to be the preferred method for identifying differentially expressed genes, with SAM 9 and the simple t-test being the other methods tested. For all six methods (t-test, fold, SMT, IBMT, Cyber-T, and Fox), we ranked the genes by significance level, and then the number of false positives was calculated as a function of the number of genes deemed to be significant. The order of performance in accumulating the least number of false positives, from best to worst, is IBMT, Fox, Cyber-T, SMT, the simple t-test, and finally fold change (Figure 6a).

The ability of the different methods to correctly establish the statistical significance of differential expression was assessed by comparing estimated and empirically established False Discovery Rates (FDR) 23. The simple t-test performed best in correctly estimating the FDR (Figure 6b). Of the four other methods, IBMT and SMT resulted in estimated False Discovery rates closest to their true proportion of false positive rates (Figure 6b). All five methods underestimate the number of false positives, which under normal circumstances may result in an unacceptable amount of over-confidence in the significance of results. However, we stress that in this experiment even the simple t-test underestimated the true number of false positives, as has been previously noted 25. The prior degrees of freedom estimated for this study ranged from 4.0 – 5.4 for IBMT and 1.6 – 1.9 for SMT, and using the defaults for the other methods, Cyber-T used 10 and Fox used 16.

The second spike-in dataset used was the Affymetrix HG-U133 latin-square data set available at 26, and consisting of 22,300 probe sets. This dataset consists of 14 sets of 3 chips, each having 42 probe sets (0.19%) spiked-in. After preprocessing with RMA, each consecutive pair of triplicates was analyzed separately, to identify the 2-fold changes in expression. In addition, IBMT was used to analyze each set of three consecutive triplicates. Figure 7a and 7b compare the average accumulation of false positives by gene rank and estimation of the true proportion of false positives respectively. Note the slight improvement in using three sets at a time compared to pairs. Possibly due to the low number of spiked-in genes for this experiment, the ability of IBMT, Cyber-T, and Fox to rank the differentially expressed genes on top could not be differentiated, as the curves for these three experiments cross. However, these methods did outperform SMT, fold change, and the t-test, again indicating the importance of accounting for the dependence of variance on gene signal intensity. Similar to the previous spike-in experiment, Figure 7b shows that the t-test performed best in estimating the true proportion of false positives, and Cyber-T and Fox resulted in the greatest underestimation of false positives. Prior degrees of freedom for this data set ranged from 7.6 – 19.3 for IBMT and 5.2 – 8.2 for SMT, while Cyber-T and Fox used the same defaults as the previous data set. The relationship between variance and intensity for this study can be seen in Supplemental Figure S4.

Two dual-channel microarray experiments were performed. The first was a comparison of wildtype mouse embryo fibroblast (MEF) cells to aryl-hydrocarbon receptor (Ahr) knockout MEF cells. Four biological replicate cell cultures each of wildtype and knockout cells were compared, each with dye labelling switched for the second technical replicate of each biological pair.

The second dataset has been published 19 and the methods are summarized here. 129S1/SvImJ strain mice (females, age 7–10 weeks) were purchased from The Jackson Laboratory (Bar Harbor, ME). All mice were housed in our animal facilities ≥ 1 week prior to exposure. Nickel aerosol was generated from 50 mM NiSO₄•6H₂O (Sigma, St. Louis, MO) and monitored as described previously 39. Mice were exposed to 150 ± 15 μg Ni²⁺/m³ in a 0.32-m³ stainless steel inhalation chamber. All experimental protocols were reviewed and approved by the Institutional Animal Care and Use Committee at the University of Cincinnati Medical Center.

Mice were exposed to aerosolized nickel for 3, 8, 24, 48, and 72 h. Following exposure, mice were killed with pentobarbital (followed by exsanguination), and the lungs were removed, placed in liquid nitrogen, and stored at -80°C. Total cellular RNA was isolated from frozen lung tissue with TRIzol (Invitrogen), and quantity was assessed by A260/A280 spectrophotometric absorbance (SmartSpec 3000, Bio-Rad, Hercules, CA). RNA quality was assessed by separation with a denaturing formaldehyde/agarose/ethidium bromide gel, and quantified by analysis with an Agilent Bioanalyzer (Quantum Analytics, Foster City, CA) 19.

The two real datasets were performed using Qiagen-Operon's mus musculus version 1.1 70-mer oligonucleotide library, representing 13,664 annotated transcripts. The first dataset is a simple comparison of wildtype mouse embryo fibroblast (MEF) cells to Ahr^-/- MEF cells. A similar microarray comparison performed with mouse smooth muscle cells has previously been published 161718. The second dataset has been published 19, but we summarize the methods below. RNA quality for both experiments was assessed by separation with a denaturing formaldehyde/agarose/ethidium bromide gel, and quantified by analysis with an Agilent Bioanalyzer (Quantum Analytics, Inc., Foster City, CA). To examine differential gene expression, a 70-mer oligonucleotide library, representing 13,443 mouse genes (Operon Biotechnologies, Inc., Huntsville, AL), was used by the Genomic and Microarray Laboratory, Center for Environmental Genetics, University of Cincinnati, http://microarray.uc.edu/ was used to fabricate microarrays. The microarray hybridisations were carried out as described 1618. For the AHR experiment, each biological replicate consisted of one mouse cell culture, and for the Ni-treatment experiment, each exposure group consisted of nine mice. RNA from three mice was pooled for each microarray, and three separate microarrays per exposure group were compared to non-exposed controls. Both experiments were performed using 20 μg total RNA per array. Each sample of mRNA was reverse transcribed and tagged with either fluorescent Cyanine 3 (Cy3) or Cyanine 5 (Cy5) (e.g., Cy3 forcontrol and Cy5 for72-h exposure). Cy3 and Cy5 samples were co-hybridized with the printed 70-mers. Following hybridization, slides were washed and scanned at 635 (Cy5) and 532 (Cy3) nm (GenePix 4000B, Axon Instruments, Inc., Union City, CA).

Microarray protocols and analyses were performed as described in 16171856. Briefly, microarray hybridization data representing raw spot intensities generated by the GenePix^® Pro v5.0 software and data normalization was performed for each microarray separately. First, channel specific local background intensities were subtracted from the median intensity of each channel (Cy3 and Cy5). Second, background adjusted intensities were log-transformed and the differences (R) and averages (A) of log-transformed values were calculated as R = log₂(X1) - log₂(X2) and A = [log₂(X1) + log₂(X2)]/2, where X1 and X2 denote the Cy5 and Cy3 intensities after subtracting local backgrounds, respectively. Third, data centering was performed by fitting the array-specific local regression model of R as a function of A 57. Normalized log-intensities for the two channels were then calculated, and statistical analysis was performed for each gene separately by fitting a mixed effects linear model 58. For the MEF Ahr^-/- experiment the model used was: Y_ijkl = μ + A_i + S_j + M(S)_kj + C_l + 'Ω_ijkl, where Y_ijkl corresponds to the normalized log-intensity on the i^th array (i = 1,..., 8), with the j^th treatment (j = 1, 2), for the k^th mouse, and labeled with the l^th dye (l = 1 for Cy5, and 2 for Cy3). μ is the overall mean log-intensity, A_i is the effect of the i^th array, S_j is the effect of the j^th treatment, M(S)_kj is the effect of the k^th mouse with treatment j, and C_k is the effect of the k^th dye. Assumptions about the model parameters were the same as described elsewhere 58, with array and mouse effects assumed to be random, and treatment, and dye effects assumed to be fixed. The model for the second dataset was as described above, with the exception of no mouse-within-treatment effect, and a higher number of arrays (5·3 = 15) and treatment conditions (6) 19. Ordinary T-statistics and estimates of fold change were calculated for each gene using this model. The SMT 12 and IBMT significance levels were then calculated as described above.