Figure 1 shows two typical sets of mRNA data. Based on these data, we have constructed a basic statistical model and next prepared some hypotheses. Hereafter, the log10 value of signal intensity is called the 'log-intensity' for simplicity.

Let x_ijk be the log-intensity in the kth replicate on the jth probe for the ith strain, where i = 1, 2 stand for two strains. Let μ_ij be the mean log-intensity on the jth probe for the ith strain. Here we assume that the difference between the log-intensity and the mean does not depend on the position of the probe. This tendency was seen for many probe sets as well as in Figure 1 and similarly adopted in 81112. To express this characteristic difference, we use the parameter ν_ik in the kth replicate for the ith strain. Consequently, the basic statistical model can be expressed as

where , ε_ijk's are the noise terms, J is the number of probes and K is the number of replicates. We assume that ε_ijk has a normal distribution with mean zero and variance σ².

If no SFP is present in a probe set, then the log-intensity differences between two strains are expected to be almost the same on all the probes in the probe set, in other words, the signal intensity ratios for two strains are almost the same on all the probes in the probe set. This tendency was seen for many probe sets as well as in Figure 1 with the exception of some SFP probes. Let the difference between two means be denoted by λ_j = μ_1j - μ_2j. The above expectation implies the hypothesis that

In this paper, the difference between λ_j and the common λ is called the 'probe effect'.

In Figure 1, some λ_j's are clearly larger than the common λ. This would be caused by SFPs, because the nucleotide polymorphism can reduce hybridization signal intensity. The alternative hypothesis can be expressed as a one-sided one, given by

which means that λ_j is larger (or smaller) than the common λ and the λ_j' 's except for the jth probe are the same as the common λ. If we reject the null hypothesis H and accept the alternative hypothesis K_j, then we will judge the jth probe to be an SFP for the second (or first) strain. For the 9th probe in Figure 1(a) and the 10th and 11th probes in Figure 1(b), we expect to accept the alternative hypothesis K_j: λ_j > λ.

Consider the case where the first strain is the platform one, in other words, the hybridization can be disturbed only for the second strain. Then we use only one alternative hypothesis, such as K_j: λ_j > λ, because μ_2j can become much smaller than expected.

Let us prepare the hypothesis H₀: λ = 0. If the null hypothesis H₀ is rejected, then the corresponding gene is judged to be differently expressed. For Figures 1(a) and 1(b), we expect to accept and reject the null hypothesis H₀: λ = 0, respectively.

We also incorporate random effects into the model to address various types of dispersion, including the noise dispersion. Such a device is often adopted to reduce the number of parameters when the number of replicates is small. This device also makes the robust procedure easily applicable, as described later. We assume that the difference in the means, λ_j = μ_1j - μ_2j, has a normal distribution with mean λ and variance τ² because λ_j's can be regarded to be dispersed around the common difference λ.

The following is a simple review about the adverse effects of an outlier. Let y₁,..., y₁₀ be the observations. Let y₁ = ⋯ = y₉ = 0 and y₁₀ = 50. Consider the estimation of the mean parameter μ = E [y]. The sample mean, a standard estimate of the mean parameter, is 5. This estimate may be inappropriate because we generally regard y₁₀ = 50 as an outlier and expect μ = 0 from the other observations. To carefully treat outliers, we often adopt the robust parameter estimation.

The signal intensities from the SFP probes can be regarded as outliers when they show different behaviors from other signal intensities, as seen in Figure 1. Thus we need to carefully examine the RNA data and then adopt a robust procedure against the outliers, as described later.

The parameter estimation of the probe effect (λ_j - λ) is illustrated in Figure 2. The probe effect is expected to be close to zero if the probe is not an SFP. Except for the 10th and 11th SFP probes in Figure 1(b), the robust estimates were balanced on both sides of zero, whereas most of the standard estimates (maximum likelihood estimates) were greater than zero.

Let and . Let . Here we assume that the hypothesis H holds. Then, , where κ² = τ² + 2σ²/K. Let us consider the parameter estimation based on z_j's.

Let ɸ(z; θ) be the normal density function with the parameter θ = (λ, κ²). The robust parameter estimate can be obtained by the minimizer of

This type of robust parameter estimation was investigated by Windham 20, Basu et al. 21, Jones et al. 22, and Fujisawa and Eguchi 23.

The positive tuning parameter γ controls the trade-off between efficiency and robustness. As γ goes to zero, the limits to the maximum likelihood estimator, which is efficient but not robust against outliers. When γ = 1, the is similar to the L₂ estimator, which is known as a strong robust estimator 24. As the tuning parameter γ is smaller or larger, the robustness will become weaker or stronger, respectively, whereas the efficiency will increase or decrease, respectively. We used γ = 0.5 for the analysis of the mRNA data from various experiences. We will also discuss the choice of γ later.

The normal density function ɸ(z; θ) belongs to an exponential family. For this reason, we can construct a convenient and iterative algorithm to obtain the robust parameter estimate (Appendix A1 of the additional file 1). By virtue of the standard theory of M-estimation, the distribution of the robust parameter estimator can be approximated to a normal distribution (Appendix A2 of additional file 1). The above robust parameter estimation shows strong robustness even when the ratio of the outlier is not small. This is suitable for the analysis of the mRNA data because the probe set may contain a number of SFP probes. For the detailed properties of the above robust parameter estimation, see Fujisawa and Eguchi 23.

It should be noted that the variance parameter κ² = τ² + 2σ²/K includes two types of variance parameter, τ² and σ². The estimate is sometimes underestimated for the analysis of the mRNA data when σ² is relatively large and the number of replicates is small. To overcome this difficulty, we modified the estimate as follows. We first estimated the variance parameter σ² by a standard unbiased estimate and then replaced the estimate by if , because κ² ≥ 2σ²/K.

Consider the testing problem for the null hypothesis H: λ₁ = ⋯ = λ_J = λ against the alternative hypothesis K_j: λ_j > λ. If we can make an appropriate estimate of λ_j, then we can propose the Wald-type test statistic:

where is an appropriate estimate of the standard deviation of (Appendix A3 of additional file 1). We simply estimated the parameter λ_j by . The distribution of the test statistic T_j can be approximated to the standard normal distribution. Let z_α be the upper 100α % point of the standard normal distribution. If T_j > z_α, then we will accept the alternative hypothesis K_j at significance level α and judge the jth probe as an SFP. By a similar way, we can also treat the alternative hypothesis K_j: λ_j <λ and two-sided alternatives.

Consider the testing problem for the null hypothesis H₀: λ = 0. We can propose the Wald-type test statistic:

where is an appropriate estimate of the standard deviation of (Appendix A4 of the additional file 1). If |T₀| > z_α/2, then we will reject the null hypothesis H₀ at significance level α and judge the corresponding gene to be differently expressed.

The Affymetrix GeneChip Rice Genome Array consisted of 57,381 probe sets containing 631,066 probes. Signal intensities of two fully sequenced rice cultivars, japonica rice "Nipponbare" 25 and indica one "93-11" 26, were observed by hybridizing their mRNA to the rice array. mRNA data were obtained for five biological replicates from 2 cm young panicles of both Nipponbare and 93-11.

The Affymetrix GeneChip Mouse Genome 430 2.0 Array consisted of 45,101 probe sets containing 496,468 probes. Signal intensities of two inbred strains, C57BL/6J (referred to below as B6) and MSM/Ms (Mus musculus molossinus), were observed by hybridizing their mRNA to the mouse array. mRNA data were obtained for two biological replicates from the liver of both B6 and MSM/Ms.

For a more detailed experimental environment and sequence analysis, see the additional file 1. All microarray data from this study are available from the Center for Information Biology gene EXpression (CIBEX) database http://cibex.nig.ac.jp/index.jsp under accession numbers CBX50 and CBX54.

The barley data were analyzed by Rostoks et al. 10. To detect SFP probes, they adopted a standard testing procedure based on a standard interaction model with SAM 19. There were 2,601 probes whose target sequences were confirmed in the two analyzed varieties: Morex and Golden Promise. They consisted of 2,200 non-polymorphic probes and 401 polymorphic probes among which 178 and 223 probes were polymorphic to Morex and Golden Promise sequences, respectively. There were six types of tissue and all except one were analyzed using three replicates (http://naturalvariation.org/barley).

We considered both alternative hypotheses, K_j: λ_j > λ and K_j: λ_j <λ, because a probe could be an SFP for both strains. The sensitivity was calculated by the ratio of the number of probes correctly judged as SFPs to the number of SFP probes. The false positive rate (FPR) was calculated by the ratio of the number of probes incorrectly judged as SFPs to the number of probes judged as SFPs. The sensitivities and FPRs of various methods are given in Table 1.

SNEP was applied to the barley data with three replicates (K = 3) without normalization. The significance levels were set at 10^-3/2 and 10^-2/2 for the first four and RAD samples, respectively, which allowed us to easily compare SNEP with the method employed by Rostoks et al. 10. SNEP markedly outperformed their method. For example, for the CRO samples, the sensitivity and FPR of SNEP were approximately 9% and 17% superior to those obtained using their method, respectively.

We also applied the likelihood ratio test (LRT) based on a standard interaction model without normalization. LRT is a standard testing procedure that is similar to the basic test statistic used by Rostoks et al. 10. The significance levels for LRT were set at 10^-5/2 and 10^-3/2 for the first four and RAD samples, respectively, which allowed us to easily compare LRT with the other methods. SNEP markedly outperformed LRT, whereas LRT almost outperformed the methods of Rostoks et al. 10.

We also analyzed the same barley data with the method employed by Greenhall et al. 14. Both of the one-sided alternative hypotheses were considered and the significance level was set at 10^-4/2 by using the standard normal approximation. Both SNEP and LRT markedly outperformed their method.

The method of Rostoks et al. 10 and LRT were based on a similar standard testing procedure, but the former was inferior to the latter. The major difference between two methods was that the former adopted the normalization and SAM. It might seem that the normalization markedly affected the performance of the method. A prerequisite for most normalization is that the mRNA affinities to the microarray are the same for all of the replicates. Normalization enables the total amount of hybridized mRNA to be equalized for all of the replicates. However, in different strains (or species), nucleotide polymorphisms affect the mRNA affinities to the microarray. For cross-strain (or cross-species) microarrays, normalization is not sufficient to equalize the affinities. Thus, we did not include normalization in SNEP.

Receiver operating characteristic (ROC) curves are shown in Figure 3. One method is said to outperform another when the associated ROC curve lies above the ROC curve of the comparator method. SNEP markedly outperformed LRT. In particular, when the FPR was approximately 0.2, the difference between the results obtained by SNEP and LRT tended to become larger as the FPR became smaller. We also tried to use γ = 0.2 and γ = 0.8 instead of γ = 0.5, which was the default value for SNEP. The results obtained with γ = 0.2 was worse than those obtained with the other values. It was also pointed out in Fujisawa and Eguchi 23 that the case γ = 0.2 would not suffice when the ratio of outlier was not small. We could not clearly determine which was better, γ = 0.5 or γ = 0.8. Because the objective function for robust parameter estimation tends to be flat for a large value of γ, the iterative algorithm for robust parameter estimation did not work well for synthetic data sets with a large value of γ (data not shown). Therefore, we used γ = 0.5 as the default value for SNEP.

SNEP was also applied to the rice data. Two points made this analysis different from that performed with the barley data. For barley, the genome sequence was only partly known, whereas we obtained the whole genome sequences of both Nipponbare and 93-11. We thus could study the performance of the methods in more detail. The rice arrays were designed mostly with japonica transcripts and therefore Nipponbare was regarded as the platform strain.

We prepared 'canonical rice data' and then we applied SNEP and LRT to the canonical rice data. The canonical rice data consisted of signal intensities for the probe sets in which all 11 probe sequences were perfectly matched as a single copy in the Nipponbare genome and were matched as a single copy in the 93-11 genome. Note that the term 'single copy' means there is only one similar sequence in a genome. For the canonical rice data, the SFP probes only interacted with the 93-11 sequences. For this reason, we used only one alternative hypothesis to detect SFP probes for 93-11.

We first examined the effects of the degree of signal intensity by the median of the log-intensities in a probe set, called the 'median-intensity', because we thought that low signal intensity level might not represent enough hybridization. SNEP was applied to the canonical rice data at significance level 10^-3. We constructed four classes (<2, 2–2.5, 2.5–3, and 3≤) of the median-intensity and categorized the probe sets into each class. The sensitivity and FPR of SNEP were calculated for each class when the number of sequence-verified SFP probes in a probe set, called the 'SFP number', was one (Table 2). It was clear that the sensitivity and FPR were much worse when the median-intensity was low. Moreover, when the median-intensities for both strains were more than 2.5, the sensitivity and FPR were stable at high and low levels, respectively. Thus, we say that the gene is sufficiently expressed when the median-intensity is more than 2.5.

SNEP and LRT were applied to the canonical rice data in which the genes were sufficiently expressed. The significance levels for SNEP and LRT were set at 10^-3 and 10^-4, respectively, which allowed us to easily compare the two methods. The sensitivity and FPR are given in Table 3. We omitted the extreme case in which all 11 probes were SFPs, because there were much more extreme cases compared with the other cases. The sensitivity and FPR became smaller as the SFP number became larger, as expected. In contrast to the analysis of the barley data, SNEP only slightly outperformed LRT. This would be because the disadvantages associated with LRT are not so remarkable in general when using only one of the two one-sided alternative hypotheses.

We also examined an alternative way of selecting genes. We used Affymetrix GeneChip Operating Software (GCOS), which has been often used to determine whether or not a probe is sufficiently expressed. We calculated the sensitivity and FPR of SNEP for detecting SFP probes in the probe sets in which all 11 probes were judged to be 'Present' at significance level 0.01(default) by GCOS. When the SFP number was one, the sensitivity and FPR became approximately 13% and 3% worse than those shown in Table 3.

SNEP was also applied to the mouse data. The mouse genome might be more complex than the rice genome, because the mouse genome is roughly six times larger, but mice contain less than half the number of genes identified in rice 252627. Because the mouse array was designed for B6 transcripts, we used only one alternative hypothesis to detect SFP probes for MSM/Ms. The overall nucleotide substitution rate between these two strains was as high as 0.0096 28. The genome sequence of MSM/Ms has been extensively studied and 187,560 probe target sequences from MSM/Ms were available at http://molossinus.lab.nig.ac.jp/msmdb/. We used 17,043 probe sets in which all 11 probe target sequences were known in both B6 and MSM/Ms.

SNEP was applied to the mouse data in which the median-intensity was more than 2.5 and the SFP number was one. There were 710 objective probe sets. The significance level was set at 10^-3. The sensitivity and FPR of SNEP were 0.524 and 0.316, respectively, which were inferior to the values obtained in a similar analysis of rice data (0.772 and 0.189). However, because the mouse genome is more complicated, it seemed that the performance of SNEP was still good.

In the analysis of mouse data, we increased the threshold value from 2.5 to 3 in order to avoid the effects of cross-hybridization. There were 165 objective probe sets. The sensitivity and FPR were improved to 0.733 and 0.243, respectively. We also used the threshold value 2.5 to examine the barley data. We first focus on the analysis of the COL samples. The number of objective probes was reduced from 2,601 to 1,937. The sensitivity and FPR did not markedly change in comparison to those obtained in the analysis of the rice data. We further increased the threshold value from 2.5 to 3. The number of objective probes was reduced to 838. The sensitivity and FPR were much improved from 0.579 and 0.259 to 0.810 and 0.156, respectively. For other types of tissue, similar improvements were observed. Thus, the way of selecting genes will be an important issue to stabilize the SFP detection.

In contrast to SFP detection, it is difficult to clearly investigate the performance of the method for detecting differently expressed genes, due to the paucity of data regarding which genes are differently expressed. Instead, we examined whether SNEP was robust against the adverse effects of an SFP probe for detecting differently expressed genes. We compared the robust test statistic T₀ with the standard t-statistic based on the Tukey's biweight estimate. We adopted the Tukey's biweight estimate instead of directly using the raw data, because this estimate has been commonly used for detecting differently expressed genes.

We first illustrate the robustness of the two test statistics against the adverse effects of an SFP probe by analyzing the data in Figure 1(a), which suggests the hypothesis that the gene is not differently expressed. The T₀-value was 1.53 and the p-value was 0.126. This result was consistent with our hypothesis. However, the t-value was 3.64 and the p-value was less than 10^-3. This result was not consistent with our hypothesis. The Tukey's biweight method tends to weaken the adverse effects of an outlier, but it is not always designed to weaken the adverse effects of an SFP probe because it is based on only one strain. In fact, the signal intensities on the 9th probe may not be outliers when we focus only on each replicate for 93-11 and neglect the other replicates. In such a case, the Tukey's biweight method produces a smaller estimate of gene expression level for 93-11, which results in a larger t-statistic. These would be the reason why the t-value was larger than expected. SNEP can weaken the adverse effects of an SFP probe because it examines two strains simultaneously, as described already.

Figure 4 shows the global robustness of the two test statistics by comparing two cases. One case was based on the canonical rice data in which the genes were sufficiently expressed and the SFP number ranged from 1 to 5. The other case was based on the modified data in which the signal intensities from the SFP probes were deleted. The former case might be affected by the SFP probes, because the data included the signal intensities from the SFP probes. If an SFP probe produced an adverse effect, the test statistic would tend to be larger in the former case than in the latter case, as illustrated above. As seen in Figure 4, the t-statistic tended to be larger in the former case, whereas the T₀-statistic did not. We showed that the robust test statistic T₀ was much more robust against the adverse effects of an SFP probe than the t-statistic based on Tukey's biweight estimate. We also found that the absolute value of the T₀-statistic tended to be slightly smaller in the former case than in the latter case. This may occur because the signal intensities from the SFP probes are not always outliers. In such a case, the sample size of meaningful probes tends to become large and then the denominator of T₀ tends to become small.