1471-2105-9-142 1471-2105 Methodology article <p>Ranking analysis of <it>F</it>-statistics for microarray data</p> Tan Yuan-De tanyuande@hotmail.com Fornage Myriam myriam.fornage@uth.tmc.edu Xu Hongyan hxu@mcg.edu

College of Life Sciences, Hunan Normal University, Changsha, 410081, China

Institute of Molecular Medicine, the University of Texas-Houston, Houston, Texas 77030, USA

Department of Biostatistics, Medical College of Georgia, Augusta, Georgia, 30912, USA

 BMC Bioinformatics 1471-2105 2008 9 1 142 http://www.biomedcentral.com/1471-2105/9/142 18325100 10.1186/1471-2105-9-142 23 11 2007 06 3 2008 06 3 2008 2008 Tan et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Microarray technology provides an efficient means for globally exploring physiological processes governed by the coordinated expression of multiple genes. However, identification of genes differentially expressed in microarray experiments is challenging because of their potentially high type I error rate. Methods for large-scale statistical analyses have been developed but most of them are applicable to two-sample or two-condition data.

Results

We developed a large-scale multiple-group F-test based method, named ranking analysis of F-statistics (RAF), which is an extension of ranking analysis of microarray data (RAM) for two-sample t-test. In this method, we proposed a novel random splitting approach to generate the null distribution instead of using permutation, which may not be appropriate for microarray data. We also implemented a two-simulation strategy to estimate the false discovery rate. Simulation results suggested that it has higher efficiency in finding differentially expressed genes among multiple classes at a lower false discovery rate than some commonly used methods. By applying our method to the experimental data, we found 107 genes having significantly differential expressions among 4 treatments at <0.7% FDR, of which 31 belong to the expressed sequence tags (ESTs), 76 are unique genes who have known functions in the brain or central nervous system and belong to six major functional groups.

Conclusion

Our method is suitable to identify differentially expressed genes among multiple groups, in particular, when sample size is small.

Background

Microarray gene expression technology, which profiles the expression of multiple genes in parallel 12, affords the means for globally exploring physiological and pathological processes 3 regulated by the coordinated expression of thousands of genes 4. However, identification of genes that are differentially expressed in large-scale gene expression experiments requires global statistical methods rather than traditional statistical methods based on single hypothesis testing. A variety of multiple-testing procedures, such as the Bonferroni procedure, Holm procedure 5, Hochberg procedure 6, Benjamini-Hochberg (BH) procedure 7, and Benjamini-Liu (BL) procedures 8 have already been developed for testing a large family of null hypotheses. The first three methods bound the family-wise-error rate (FWER) that is the probability of at least one false positive over all tests and hence remain too stringent and have lower power for finding genes from the real data sets. The last two methods have an upper bound for the false discovery rate (FDR) with both strong and weak controls 9 and require a large sample size for valid p-values. Tusher et al. 9 has proposed a ranking statistic approach based on permutation for resampling. However, permutation is not a desirable approach to estimating null distribution in microarray data 101112 because in general a microarray dataset has a large number of genes but small sample sizes 13 due to cost. Permutation fails to remove treatment effect and due to small sample sizes the difference of treatment effects between permutated groups may become a main component in differences between group means so that the estimated null distribution is not well approximate to the true null distribution (13 and also see Appendix in Tan et al. 14). For example, Xie et al. 12 found that the estimated null F-distribution based on permutation has a larger variance and a heavier tail compared to the true null F-distribution, which leads to a potential loss of power. Similar phenomenon was also observed in comparison of the estimated null t-distribution to the true null t-distribution 14. To remove the group or treatment effects on the estimated null distribution, Tan et al. 14 developed a random splitting (RS) approach. Since treatment effects are completely eliminated, the estimated null distribution obtained by the RS method is smooth, stable and approximate true null distribution well.

For the multi-class microarray data, the analysis of variance (ANOVA) is useful to identify differentially expressed genes 4. In ANOVA, the F-test is a generalization of the t-test that allows for comparison of more than two samples 15. However, due to small sample sizes, the classical F-test is also subject to the same problems as the t-test: bias and unstable estimates of gene-specific variances. To tackle this issue, many authors 1516171819 proposed modified F-statistics. However, like the classical F-test, these modified F-tests still suffer from high false-positive rates because (i) the sample size is often so limited that the asymptotic F distribution is not accurate enough to obtain a valid p-value and (ii) they appeal to multiple-testing procedures such as the Bonferroni procedure or the BH-procedure. As mentioned above, these multiple-testing procedures have a basic requirement that sample sizes are large enough for valid p-values. In microarray data, the requirement is not realistic. Based on consideration of these problems, we here propose a novel statistical method for the analysis of multi-class gene-expression data called Ranking Analysis of F-statistics (RAF). RAF is a natural extension of our previous work, i.e., the ranking analysis of microarrary (RAM) for two-class t-tests 14. It works on finding genes that are differentially expressed among multiple treatment groups by comparing the ordered real F-statistics with the ordered estimated null F-statistics and implementing a two-simulation strategy to estimate the false discovery rate (FDR).

Methods

Animal model and design

Studies were performed on male stroke-resistant SHR/N (CRiv) (SHRSR) and stroke-prone SHR/A3 (Heid) (SHRSP) rats from a breeding colony maintained by the investigators as previously described 20. Age-matched male rats from each strain (N = 12 SHRSP and 12 SHRSR) were fed a standard rat chow and water ad libitum until age 8 weeks. Subsequently, animals from each strain were randomized to one of 2 dietary regimens (N = 6 in each strain × diet group): a "stroke-permissive diet" high in sodium (HS) (0.63% potassium, 0.37% sodium) and 1% NaCl drinking solution; a "stroke-protective diet" low in sodium and high in potassium (LS) (1.3% potassium, 0.35% sodium) and regular drinking water. All animals were housed at 23°C on a 12-hour light-dark cycle. Rats were sacrificed at 12 weeks of age, and brain tissue was collected for RNA extraction and subsequent microarray analysis. The study protocols were approved by the Animal Care Committee of the University of Texas – Houston. Since strain and dietary factor each have only two levels, we here treated them as one-way in statistical analysis instead of two-way, that is, we are neither interested in strain effects alone nor in dietary effects alone but focus on their combined contributions to gene expression. Thus, HS-SHRSPs, LS-SHRSPs, HS-SHRSRs, and LS-SHRSRs are viewed as four treatment groups for the purpose of the analyses.

Microarray experiment

Microarray analysis was performed as described by Lockhart et al. 21. Briefly, 10 μg total RNA extracted from each of the 24 rats was used to synthesize cDNA, which was then used as a template to generate biotinylated cRNA. cRNA was fragmented and hybridized to a Test2 chip to verify quality and quantity of the samples. Each sample was then hybridized to a RGU34A array (Affymetrix, Santa Clara, CA). After hybridization, each array was washed and scanned, and fluorescence values were measured and normalized using the Affymetrix Microarray Suite v. 5.0 software.

Ranking F-Test

Let xgij be the expression value for replicate j of gene g in group i where g = 1,..., N (number of genes), j = 1,..., rgi (number of replicate observed values of gene g in group i) and i = 1,..., n (number of groups). The traditional F-statistic in one-way ANOVA may be expressed as

F g = σ 2 ( G g ) σ 2 ( e g ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aaSbaaSqaaiabdEgaNbqabaGccqGH9aqpjuaGdaWcaaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqWGhbWrdaWgaaqaaiabdEgaNbqabaGaeiykaKcabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiabdwgaLnaaBaaabaGaem4zaCgabeaacqGGPaqkaaaaaa@3ED5@

where σ2 (Gg) and σ2 (eg) are inter- and intra-group variances of the expression values of gene g, respectively. In the conventional F-tests, for example, significance of p = 0.01 in the context of the standard F distribution is for a single hypothesis to be tested; therefore, it is unsuitable to microarray data because in a microarray experiment for 10,000 genes we would expect to identify 100 genes by chance 9. To address this problem, an alternative approach is to rank genes by magnitude of their F values so that F1 is the largest value, F2 is the second largest value, etc., and Fg* is the g*th largest value where g* is a rank position of gene g. Thus, we have a ranking F-test where

Fg* - fg* > Δ

indicates that the expression variation of gene g among multiple groups (or multiple conditions) is significant. In Eq. (2), fg* is the expectation of Fg* under the null hypothesis and Δ is a given threshold.

Estimation of fg*

In the ANOVA framework, we have

σ 2 ( G g ) = ∑ i = 1 n r g i ( x ¯ g i − x ¯ g ) 2 n − 1 = ∑ i = 1 n r g i [ ( τ g i + e ¯ g i ) − ( μ g + e ¯ g ) ] 2 n − 1 = ∑ i = 1 n r g i ( τ g i − μ g ) + ∑ i = 1 n r g i ( e ¯ g i − e ¯ g ) 2 n − 1 = σ 2 ( τ g ) + σ 2 ( e ¯ g ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeWabmqaaaqaaiabeo8aZnaaCaaaleqabaGaeGOmaidaaOGaeiikaGIaem4raC0aaSbaaSqaaiabdEgaNbqabaGccqGGPaqkcqGH9aqpjuaGdaWcaaqaamaaqadabaGaemOCai3aaSbaaeaacqWGNbWzcqWGPbqAaeqaaiabcIcaOiqbdIha4zaaraWaaSbaaeaacqWGNbWzcqWGPbqAaeqaaiabgkHiTiqbdIha4zaaraWaaSbaaeaacqWGNbWzaeqaaiabcMcaPmaaCaaabeqaaiabikdaYaaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGUbGBaiabggHiLdaabaGaemOBa4MaeyOeI0IaeGymaedaaOGaeyypa0tcfa4aaSaaaeaadaaeWaqaaiabdkhaYnaaBaaabaGaem4zaCMaemyAaKgabeaacqGGBbWwcqGGOaakcqaHepaDdaWgaaqaaiabdEgaNjabdMgaPbqabaGaey4kaSIafmyzauMbaebadaWgaaqaaiabdEgaNjabdMgaPbqabaGaeiykaKIaeyOeI0IaeiikaGIaeqiVd02aaSbaaeaacqWGNbWzaeqaaiabgUcaRiqbdwgaLzaaraWaaSbaaeaacqWGNbWzaeqaaiabcMcaPiabc2faDnaaCaaabeqaaiabikdaYaaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGUbGBaiabggHiLdaabaGaemOBa4MaeyOeI0IaeGymaedaaaGcbaGaeyypa0tcfa4aaSaaaeaadaaeWaqaaiabdkhaYnaaBaaabaGaem4zaCMaemyAaKgabeaacqGGOaakcqaHepaDdaWgaaqaaiabdEgaNjabdMgaPbqabaGaeyOeI0IaeqiVd02aaSbaaeaacqWGNbWzaeqaaiabcMcaPiabgUcaRmaaqadabaGaemOCai3aaSbaaeaacqWGNbWzcqWGPbqAaeqaaiabcIcaOiqbdwgaLzaaraWaaSbaaeaacqWGNbWzcqWGPbqAaeqaaiabgkHiTiqbdwgaLzaaraWaaSbaaeaacqWGNbWzaeqaaiabcMcaPmaaCaaabeqaaiabikdaYaaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGUbGBaiabggHiLdaabaGaemyAaKMaeyypa0JaeGymaedabaGaemOBa4gacqGHris5aaqaaiabd6gaUjabgkHiTiabigdaXaaaaOqaaiabg2da9iabeo8aZnaaCaaaleqabaGaeGOmaidaaOGaeiikaGIaeqiXdq3aaSbaaSqaaiabdEgaNbqabaGccqGGPaqkcqGHRaWkcqaHdpWCdaahaaWcbeqaaiabikdaYaaakiabcIcaOiqbdwgaLzaaraWaaSbaaSqaaiabdEgaNbqabaGccqGGPaqkaaaaaa@B8FF@

where x¯gi=τgi+e¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaakiabg2da9iabes8a0naaBaaaleaacqWGNbWzcqWGPbqAaeqaaOGaey4kaSIafmyzauMbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaaaaa@3B2C@ and x¯g=μgi+e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaWgaaWcbaGaem4zaCgabeaakiabg2da9iabeY7aTnaaBaaaleaacqWGNbWzcqWGPbqAaeqaaOGaey4kaSIafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@3867@; τgi and e¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaaaaa@301E@ are treatment effect and average random error in group i, respectively; x¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EE9@ and e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@ are the overall observed mean and the overall average error for gene g, respectively, and μg is the overall expected mean for gene g and rgi is the number of replicate observed values of gene g in group i. Therefore, the inter-group variance σ2 (Gg) consists of two parts: variance of treatment effects on expression of gene g, σ2 (τg), and average error variance, σ2 (e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@). Thus, F-statistic can be rewritten as

F g = σ 2 ( G g ) σ 2 ( e g ) = σ 2 ( τ g ) + σ 2 ( e ¯ g ) σ 2 ( e g ) = σ 2 ( τ g ) σ 2 ( e g ) + σ 2 ( e ¯ g ) σ 2 ( e g ) = σ 2 ( τ g ) σ 2 ( e g ) + f g . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aaSbaaSqaaiabdEgaNbqabaGccqGH9aqpjuaGdaWcaaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqWGhbWrdaWgaaqaaiabdEgaNbqabaGaeiykaKcabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiabdwgaLnaaBaaabaGaem4zaCgabeaacqGGPaqkaaGccqGH9aqpjuaGdaWcaaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqaHepaDdaWgaaqaaiabdEgaNbqabaGaeiykaKIaey4kaSIaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiqbdwgaLzaaraWaaSbaaeaacqWGNbWzaeqaaiabcMcaPaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqWGLbqzdaWgaaqaaiabdEgaNbqabaGaeiykaKcaaOGaeyypa0tcfa4aaSaaaeaacqaHdpWCdaahaaqabeaacqaIYaGmaaGaeiikaGIaeqiXdq3aaSbaaeaacqWGNbWzaeqaaiabcMcaPaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqWGLbqzdaWgaaqaaiabdEgaNbqabaGaeiykaKcaaOGaey4kaSscfa4aaSaaaeaacqaHdpWCdaahaaqabeaacqaIYaGmaaGaeiikaGIafmyzauMbaebadaWgaaqaaiabdEgaNbqabaGaeiykaKcabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiabdwgaLnaaBaaabaGaem4zaCgabeaacqGGPaqkaaGccqGH9aqpjuaGdaWcaaqaaiabeo8aZnaaCaaabeqaaiabikdaYaaacqGGOaakcqaHepaDdaWgaaqaaiabdEgaNbqabaGaeiykaKcabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiabdwgaLnaaBaaabaGaem4zaCgabeaacqGGPaqkaaGccqGHRaWkcqWGMbGzdaWgaaWcbaGaem4zaCgabeaakiabc6caUaaa@8E77@

Therefore, the null hypothesis is equivalent to Fg = fg because σ2 (τg) = 0 under the null hypothesis. Note that σ2 (τg) = 0 means the treatment effects τgi = ... = τgn = μg. In order to do a ranking F-test, it is necessary to obtain a good estimate of fg* . In the two-group scenario, Tusher et al. 9 employed a permutation approach to estimate the expected t-statistics. The permutation process cannot completely clear the treatment effect in the ranked d-statistics so that the estimated ranked d-statistics distribution is biased against its null distribution and unstable, in particular, when sample sizes are small (see Appendix A in Tan et al., 14). Tan et al. 14 developed a "Randomly Splitting" (RS) approach to estimate the null distribution of t-statistics. In this study, we extended the RS approach to estimating the null distribution of F-statistics.

In the RS approach, one sample consisting of rgi replicates is drawn from group i. Since only one sample is drawn from a group, sample i represents group i. Within a sample all the observed expression values of gene g come from the same group. These values have the same overall expected mean μg and the same treatment effect τgi on expression of gene g except for expression noises. A sample of rgi replicate values for gene g is randomly split into two sub-samples denoted by s = 1 and s = 2. If let x¯gisJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaqhaaWcbaGaem4zaCMaemyAaKMaem4CamhabaGaemOsaOeaaaaa@32D1@ be the mean of sub-sample s of sample i for gene g at split J(J = 1,..., M), then x¯gisJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaqhaaWcbaGaem4zaCMaemyAaKMaem4CamhabaGaemOsaOeaaaaa@32D1@ can be expressed as

X ¯ g i s J = μ g + τ g i + e ¯ g i s J MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiwaGLbaebadaqhaaWcbaGaem4zaCMaemyAaKMaem4CamhabaGaemOsaOeaaOGaeyypa0JaeqiVd02aaSbaaSqaaiabdEgaNbqabaGccqGHRaWkcqaHepaDdaWgaaWcbaGaem4zaCMaemyAaKgabeaakiabgUcaRiqbdwgaLzaaraWaa0baaSqaaiabdEgaNjabdMgaPjabdohaZbqaaiabdQeakbaaaaa@4479@

where e¯gisJ=∑j=1mgisJegisjJ/mgisJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaqhaaWcbaGaem4zaCMaemyAaKMaem4CamhabaGaemOsaOeaaOGaeyypa0ZaaabmaeaacqWGLbqzdaqhaaWcbaGaem4zaCMaemyAaKMaem4CamNaemOAaOgabaGaemOsaOeaaOGaei4la8IaemyBa02aa0baaSqaaiabdEgaNjabdMgaPjabdohaZbqaaiabdQeakbaaaeaacqWGQbGAcqGH9aqpcqaIXaqmaeaacqWGTbqBdaqhaaadbaGaem4zaCMaemyAaKMaem4CamhabaGaemOsaOeaaaqdcqGHris5aaaa@4FAD@ and mgisJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyBa02aa0baaSqaaiabdEgaNjabdMgaPjabdohaZbqaaiabdQeakbaaaaa@32A3@ and egisjJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyzau2aa0baaSqaaiabdEgaNjabdMgaPjabdohaZjabdQgaQbqaaiabdQeakbaaaaa@33F0@ are replicate number and noise in the observed expression value j in sub-sample s in group i for gene g at split J, respectively. e¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaaaaa@301E@ is estimated by the difference between two sub-sample means in sample i for gene g at split J,

e ¯ g i J = 1 2 ( x ¯ g i 1 J − x ¯ g i 2 J ) = 1 2 [ ( μ g + τ g + e ¯ g i 1 J ) − ( μ g + τ g + e ¯ g i 2 J ) ] . = 1 2 ( e ¯ g i 1 J − e ¯ g i 2 J ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabmqaaaqaaiqbdwgaLzaaraWaa0baaSqaaiabdEgaNjabdMgaPbqaaiabdQeakbaakiabg2da9KqbaoaalaaabaGaeGymaedabaGaeGOmaidaaOWaaeWaaeaacuWG4baEgaqeamaaDaaaleaacqWGNbWzcqWGPbqAcqaIXaqmaeaacqWGkbGsaaGccqGHsislcuWG4baEgaqeamaaDaaaleaacqWGNbWzcqWGPbqAcqaIYaGmaeaacqWGkbGsaaaakiaawIcacaGLPaaaaeaacqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabikdaYaaakmaadmaabaWaaeWaaeaacqaH8oqBdaWgaaWcbaGaem4zaCgabeaakiabgUcaRiabes8a0naaBaaaleaacqWGNbWzaeqaaOGaey4kaSIafmyzauMbaebadaqhaaWcbaGaem4zaCMaemyAaKMaeGymaedabaGaemOsaOeaaaGccaGLOaGaayzkaaGaeyOeI0YaaeWaaeaacqaH8oqBdaWgaaWcbaGaem4zaCgabeaakiabgUcaRiabes8a0naaBaaaleaacqWGNbWzaeqaaOGaey4kaSIafmyzauMbaebadaqhaaWcbaGaem4zaCMaemyAaKMaeGOmaidabaGaemOsaOeaaaGccaGLOaGaayzkaaaacaGLBbGaayzxaaGaeiOla4cabaGaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqaIYaGmaaGcdaqadaqaaiqbdwgaLzaaraWaa0baaSqaaiabdEgaNjabdMgaPjabigdaXaqaaiabdQeakbaakiabgkHiTiqbdwgaLzaaraWaa0baaSqaaiabdEgaNjabdMgaPjabikdaYaqaaiabdQeakbaaaOGaayjkaiaawMcaaaaaaaa@7F5D@

It can be seen from Eq. (6) that μg and τgi are cleared in difference between two sub-sample means, which is unrelated to sample size. Thus, the average random error variance σ2 (e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@) in Eq.s (3) and (4) can be estimated by

σ 2 ( e ¯ g J ) = ∑ i = 1 n r g i ( e ¯ g i J − e ¯ g J ) 2 n − 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaWbaaSqabeaacqaIYaGmaaGccqGGOaakcuWGLbqzgaqeamaaDaaaleaacqWGNbWzaeaacqWGkbGsaaGccqGGPaqkcqGH9aqpjuaGdaWcaaqaamaaqadabaGaemOCai3aaSbaaeaacqWGNbWzcqWGPbqAaeqaaiabcIcaOiqbdwgaLzaaraWaa0baaeaacqWGNbWzcqWGPbqAaeaacqWGkbGsaaGaeyOeI0IafmyzauMbaebadaqhaaqaaiabdEgaNbqaaiabdQeakbaacqGGPaqkdaahaaqabeaacqaIYaGmaaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemOBa4gacqGHris5aaqaaiabd6gaUjabgkHiTiabigdaXaaaaaa@51AD@

where e¯gJ=∑i=1ne¯giJ/n MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaqhaaWcbaGaem4zaCgabaGaemOsaOeaaOGaeyypa0ZaaabmaeaacuWGLbqzgaqeamaaDaaaleaacqWGNbWzcqWGPbqAaeaacqWGkbGsaaGccqGGVaWlcqWGUbGBaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabd6gaUbqdcqGHris5aaaa@3F65@ is estimate of mean (e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@) of expression noise of gene g among groups at split J. Variance σ2(e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@) is an estimate of expectation (σ2(e¯g MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyzauMbaebadaWgaaWcbaGaem4zaCgabeaaaaa@2EC3@)) of inter-group variance (σ2 (Gg)) under the null hypothesis at split J. We therefore have

f g J = σ 2 ( e ¯ g J ) σ 2 ( e g ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdEgaNbqaaiabdQeakbaakiabg2da9KqbaoaalaaabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiqbdwgaLzaaraWaa0baaeaacqWGNbWzaeaacqWGkbGsaaGaeiykaKcabaGaeq4Wdm3aaWbaaeqabaGaeGOmaidaaiabcIcaOiabdwgaLnaaBaaabaGaem4zaCgabeaacqGGPaqkaaGccqGGUaGlaaa@4293@

Note that since treatment effect is completely removed from the difference between two sub-sample means, the difference is pure noise. We rank fgJ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdEgaNbqaaiabdQeakbaaaaa@2FCB@ across all g and let fg∗J MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdEgaNjabgEHiQaqaaiabdQeakbaaaaa@30BA@ denote the value in ordered position g* at split J. After running M splits, we have M values of fg∗J MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdEgaNjabgEHiQaqaaiabdQeakbaaaaa@30BA@ for position g*. Thus fg* in Eq. (2) can be estimated by the average of fg∗J MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdEgaNjabgEHiQaqaaiabdQeakbaaaaa@30BA@ over all M splits, i.e., f¯g∗=∑J=1Mfg∗J/M MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOzayMbaebadaWgaaWcbaGaem4zaCMaey4fIOcabeaakiabg2da9maaqadabaGaemOzay2aa0baaSqaaiabdEgaNjabgEHiQaqaaiabdQeakbaakiabc+caViabd2eanbWcbaGaemOsaOKaeyypa0JaeGymaedabaGaemyta0eaniabggHiLdaaaa@3DF4@.

Estimation of FDR

To identify genes whose expression is significantly changed among multiple conditions, it is necessary to estimate the FDR for a given threshold 722. Here we propose a two-simulation approach for FDR estimation 14. Consider a series of threshold values Δk(k = 1,...L) and let Nk be the number of genes that are claimed as significant by RAF at threshold Δk. Nk comprises two parts: the number Nk(t) of the true positives and the number Nk(f) of the false positives, i.e., Nk = Nk(t) + Nk(f). Thus, given a threshold Δk, FDR is defined as λk = Nk(f)/Nk. Nk(f) is unknown, hence λk must be estimated. Many approaches such as BH procedure 722, BL procedure 8, Storey's procedure 2324, and Pounds and Cheng's procedure 25 have been proposed to estimate the FDR. These approaches, however, are based on the assumption that the tests are independent. As mentioned previously, this assumption may not be met in practice. Therefore, these methods may not be suitable to our ranking test. Based on the fact that sampling distribution fluctuates around the expected distribution via permutation, Tusher et al. 9 developed a permutation-based estimator to estimate FDR in the ranking tests. It has been proved, however, in theory and in simulation that when the sample sizes are small, the number of permutations is very limited so that the treatment effects cannot be removed in the permutated data 14. As a result, the estimator is biased for a given threshold. Here we extend the interval approach by Tan et al. 14 to the ranking analysis of F-statistics. In this approach, we first construct an estimated interval of the true FDR, and then we find a reasonable estimate of FDR. This interval is based on the complete and partial null distributions given by two simulations.

In simulation 1, for each gene, n samples (groups) each having r replicates are generated from normal distributions with a set of sample means (y¯g1,...,y¯gn MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyEaKNbaebadaWgaaWcbaGaem4zaCMaeGymaedabeaakiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiqbdMha5zaaraWaaSbaaSqaaiabdEgaNjabd6gaUbqabaaaaa@38CC@) and a set of sample error variances [s2 (eg1),..., s2 (egn)]. Here we set y¯g1=....=y¯gn=x¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyEaKNbaebadaWgaaWcbaGaem4zaCMaeGymaedabeaakiabg2da9iabc6caUiabc6caUiabc6caUiabc6caUiabg2da9iqbdMha5zaaraWaaSbaaSqaaiabdEgaNjabd6gaUbqabaGccqGH9aqpcuWG4baEgaqeamaaBaaaleaacqWGNbWzcqWGPbqAaeqaaaaa@3F7B@ and i is a randomly chosen group from the observed data, for each of a half of the genes with the null effect that the group variance is zero, i.e., σ2 (Gg) = 0 and y¯gi=x¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyEaKNbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaakiabg2da9iqbdIha4zaaraWaaSbaaSqaaiabdEgaNjabdMgaPbqabaaaaa@35C5@ for each of the other half with unknown effect that the group variance is larger than or equal to zero, i.e., σ2 (Gg) ≥ 0. s2 (egi) is set to be equal to σ2 (egi) where x¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiEaGNbaebadaWgaaWcbaGaem4zaCMaemyAaKgabeaaaaa@3044@ and σ2 (egi) are the observed values from the real microarray data set.

B sets of simulation data are obtained from this procedure. Each is subject to the ranking analysis described in the previous section. For simulation data set b, every ranked position has thus its corresponding F value that is denoted by Fg∗1b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aa0baaSqaaiabdEgaNjabgEHiQiabigdaXaqaaiabdYeambaaaaa@316E@(b = 1,..., B). Here those that are called significant by comparing Fg∗1b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aa0baaSqaaiabdEgaNjabgEHiQiabigdaXaqaaiabdkgaIbaaaaa@319A@ to f¯g∗ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOzayMbaebadaWgaaWcbaGaem4zaCMaey4fIOcabeaaaaa@2FB4@ at a given threshold Δk are counted as N1kb MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOta40aa0baaSqaaiabigdaXiabdUgaRbqaaiabdkgaIbaaaaa@30C3@ across all ranking positions. Let N1k=max⁡b=1BN1kb MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOta40aaSbaaSqaaiabigdaXiabdUgaRbqabaGccqGH9aqpcyGGTbqBcqGGHbqycqGG4baEdaqhaaWcbaGaemOsaOKaeyypa0JaeGymaedabaGaemOqaieaaOGaemOta40aa0baaSqaaiabigdaXiabdUgaRbqaaiabdkgaIbaaaaa@3DF0@ Given the fact that a small part of genes have unequal means in the samples, the simulation data set produces a partially null F-distribution. In other words, it may produce N1kb>Nk(f) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOta40aa0baaSqaaiabigdaXiabdUgaRbqaaiabdkgaIbaakiabg6da+iabd6eaonaaBaaaleaacqWGRbWAaeqaaOGaeiikaGIaemOzayMaeiykaKcaaa@3796@, possibly leading to λk = N1k/Nk > 1. To avoid this situation, suppose N1k(k = 1,..., L) takes the maximum value N(m) at Δk = m, we define

λ 1 k = 2 N 1 k N ( m ) + N 1 k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4UdW2aaSbaaSqaaiabigdaXiabdUgaRbqabaGccqGH9aqpjuaGdaWcaaqaaiabikdaYiabd6eaonaaBaaabaGaeGymaeJaem4AaSgabeaaaeaacqWGobGtcqGGOaakcqWGTbqBcqGGPaqkcqGHRaWkcqWGobGtdaWgaaqaaiabigdaXiabdUgaRbqabaaaaaaa@3F38@

as a function of threshold Δk for estimating FDR where we set N1k = N(m) for all Δk < Δm(k = 1,..., L). Obviously, λ1k is a decreasing function of the threshold and bounded between 0 and 1. For example, λ1k = 1 when N1k = N(m), and λ1k = 2/3 when N1k = N(m)/2. Results from the simulation study in Figure 1 indicate that λ1k ≥ λk (true value of FDR at threshold Δk) when threshold Δk is smaller than a value Δ* but λ1k ≤ λk when Δk > Δ* (see Figure 1).

<p>Figure 1</p>

Profile of estimates of FDRs for a series of thresholds

Profile of estimates of FDRs for a series of thresholds. λ1k and λ2k are two threshold functions from simulations 1 and 2 and were used to construct an estimation interval for estimate of FDR at threshold Δk. λk and λˆk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaee4AaSgabeaaaaa@2F22@ are true and estimated FDRs at threshold Δk, respectively, where k = 1, 2,...,L.

The second simulation for estimating FDR is carried out in the following fashion. n samples (groups) each having r replicates for each gene are generated from normal distributions with a set of sample means, y¯g1=....=y¯gn=x¯gi MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyEaKNbaebadaWgaaWcbaGaem4zaCMaeGymaedabeaakiabg2da9iabc6caUiabc6caUiabc6caUiabc6caUiabg2da9iqbdMha5zaaraWaaSbaaSqaaiabdEgaNjabd6gaUbqabaGccqGH9aqpcuWG4baEgaqeamaaBaaaleaacqWGNbWzcqWGPbqAaeqaaaaa@3F7B@ and a set of sample variances s2 (eg1) = σ2 (eg1),..., s2 (egn) = σ2 (egn).

We also produce B sets of data from simulation 2. As in simulation 1, for each simulation data set, every ranked position also has its corresponding F-value denoted as Fg∗2b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aa0baaSqaaiabdEgaNjabgEHiQiabikdaYaqaaiabdkgaIbaaaaa@319C@ (b = 1,..., B). Let Fg∗2=min⁡b=1BFg∗2b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aaSbaaSqaaiabdEgaNjabgEHiQiabikdaYaqabaGccqGH9aqpcyGGTbqBcqGGPbqAcqGGUbGBdaqhaaWcbaGaemOyaiMaeyypa0JaeGymaedabaGaemOqaieaaOGaemOray0aa0baaSqaaiabdEgaNjabgEHiQiabikdaYaqaaiabdkgaIbaaaaa@3FCE@. The positives found by comparing Fg∗2b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOray0aa0baaSqaaiabdEgaNjabgEHiQiabikdaYaqaaiabdkgaIbaaaaa@319C@ to Fg*2 at a given threshold Δk are counted as N2kb MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOta40aa0baaSqaaiabikdaYiabdUgaRbqaaiabdkgaIbaaaaa@30C5@ across all ranking positions. Here let N2k=∑b=1BN2kb/B MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOta40aaSbaaSqaaiabikdaYiabdUgaRbqabaGccqGH9aqpdaaeWaqaaiabd6eaonaaDaaaleaacqaIYaGmcqWGRbWAaeaacqWGIbGyaaGccqGGVaWlcqWGcbGqaSqaaiabdkgaIjabg2da9iabigdaXaqaaiabdkeacbqdcqGHris5aaaa@3DC6@. Unlike the first simulation, here the simulation data sets produce B null F-distributions, so N2k should be approximate to the true number of false positives Nk(f). However, when threshold Δk is large, it is possible to have Nk = 0 so that N2k/Nk is undefined. To avoid this situation, we define

λ 2 k = N 2 k N k + N 2 k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4UdW2aaSbaaSqaaiabikdaYiabdUgaRbqabaGccqGH9aqpjuaGdaWcaaqaaiabd6eaonaaBaaabaGaeGOmaiJaem4AaSgabeaaaeaacqWGobGtdaWgaaqaaiabdUgaRbqabaGaey4kaSIaemOta40aaSbaaeaacqaIYaGmcqWGRbWAaeqaaaaaaaa@3CB7@

as the second function of threshold (see Figure 1). In particular, we let λ2k = 1 if Nk = N2k = 0 because λ2k = 1 when Nk = 0 and N2k > 0.

Thus, an interval for FDR estimation at threshold Δk can be constructed between λ1k and λ2k . The third function of threshold for FDR estimation is given as

                         λ3k = αkλ1k + βkλ2k

where αk = min(λ1k, λ2k)/(λ1k + λ2k) and βk = 1 - ak. λ3k plays the role of weight in balancing λ1k and λ2k . Therefore, at threshold Δk, a putative probability that a false discovery is found in the genes called significant by RAF is

λ ˆ k = 1 3 ( λ 1 k + λ 2 k + λ 3 k ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaakiabg2da9KqbaoaalaaabaGaeGymaedabaGaeG4mamdaaOGaeiikaGIaeq4UdW2aaSbaaSqaaiabigdaXiabdUgaRbqabaGccqGHRaWkcqaH7oaBdaWgaaWcbaGaeGOmaiJaem4AaSgabeaakiabgUcaRiabeU7aSnaaBaaaleaacqaIZaWmcqWGRbWAaeqaaOGaeiykaKIaeiOla4caaa@4419@

Note that as shown in the simulation result section, λ2k is an underestimate of λk and λ1k is an overestimate of FDR when the threshold Δk < Δ*. However, the situation is reversed when threshold Δk > Δ*. This is because N1k becomes very small when Δk > Δ* so that λ1k becomes very small whereas, from Eq. (10), λ2k slowly decreases if Nk > N2k or increases if Nk <N2k as threshold increases. In addition, when the microarray data have no treatment effects for all the genes detected, then λ1k = λ2k = λ3k = 1, leading to λˆk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaaaaa@2F24@ = 1

In order to smooth λˆk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaaaaa@2F24@ between thresholds Δk and Δk+1, we define a recursive formula modifying the probability λˆk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaaaaa@2F24@ as

λ ˆ k = λ k p k + λ k + 1 q k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaakiabg2da9iabeU7aSnaaBaaaleaacqWGRbWAaeqaaOGaemiCaa3aaSbaaSqaaiabdUgaRbqabaGccqGHRaWkcqaH7oaBdaWgaaWcbaGaem4AaSMaey4kaSIaeGymaedabeaakiabdghaXnaaBaaaleaacqWGRbWAaeqaaaaa@3FBC@

where pk = (Nk - Nk+1)/(1 + Nk - Nk+1) and qk = 1 - pk. Eq. (13) suggests that λk+1 = λk if Nk = Nk+1. The number of false discoveries among those found to be significant at threshold Δk in the observed data is estimated by Nˆk(f)=λˆkNk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOta4KbaKaadaWgaaWcbaGaem4AaSgabeaakiabcIcaOiabdAgaMjabcMcaPiabg2da9iqbeU7aSzaajaWaaSbaaSqaaiabdUgaRbqabaGccqWGobGtdaWgaaWcbaGaem4AaSgabeaaaaa@38B5@. Figure 1 shows that the curve of λˆk MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafq4UdWMbaKaadaWgaaWcbaGaem4AaSgabeaaaaa@2F24@ agrees well with that of λk.

Results

Estimation of the null distribution of F-statistics

To examine if the empirical distributions obtained by the RS approach are appropriate for the analysis of the expression data, we simulated a microarray data set consisting of 3770 genes and four groups each having 6 replicates using one group mean and error variance for each gene. Thus, the simulation without treatment effect generated a set of pure noise data.

A set of 3770 Fk values was computed from the simulated data set. We applied our RS approach to this simulated data set to generate f¯k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOzayMbaebadaWgaaWcbaGaem4AaSgabeaaaaa@2ECD@ over 50 splits. This set of 3770 Fk values formed null distribution of F-statistics, whi