If the transcript-specific parameter θ t = ( μ t , θ t ∗ ) , where μ _t and θ t ∗ are the location and scale parameters respectively, then the conditional likelihood of the tth transcript’s expression measurement y _t = (y _t1,y _t2,…,y _tn ) can be expressed as ∏ i = 1 n f obs y ti | θ t (t=1,2,…,T). The location parameter μ _t follows the prior distribution, Π(μ _t | θ ), where θ is the hyper-parameter specifying the prior distribution. The predictive likelihood of y _t (unconditional on the location parameter μ _t ) is obtained by integrating over the location parameter, μ _t , as follows:

f 0 ( y t | θ , θ t ∗ ) = ∫ ∏ i = 1 n f obs y ti | μ t , θ t ∗ Π ( μ t | θ ) d μ t .

When expression measurements between two groups (for example, different cell types) are compared for transcript t, the measurements are partitioned into two user defined groups G ₁ and G ₂ of sizes n ₁ and n ₂ respectively, where n ₁ + n ₂ =n. If there is no significant difference between the means of the two groups, the gene is assumed to be equivalently expressed (EE); otherwise, it is assumed to be a DE gene. If the tth transcript is DE, the two groups will have different mean expression levels, μ t ( j ) , j = 1 , 2 . Given the values of μ t ( j ) , j = 1 , 2 and θ t ∗ , the conditional likelihood of y t = y t ( 1 ) : y t ( 2 ) is written as follows:

f 1 ( y t | μ t ( 1 ) , μ t ( 2 ) , θ t ∗ ) = ∏ i = 1 n 1 f obs y ti | μ t ( 1 ) , θ t ∗ × ∏ i ″ = 1 n 2 f obs y t i ″ | μ t ( 2 ) , θ t ∗ ,

because components of y _t are independent of each other. Assuming that the group means μ t ( j ) , j = 1 , 2 (such that μ t ( 1 ) ≠ μ t ( 2 ) ) independently originate from Π(μ _t | θ ), then the predictive likelihood of y _t (unconditional on the location parameters μ t ( j ) , j = 1 , 2 ) is obtained as a mean of the conditional likelihood of y _t (2) over the prior distribution of μ t ( 1 ) and μ t ( 2 ) as follows:

f 1 ( y t | θ , θ t ∗ ) = ∫ ∫ f 1 ( y t | μ t ( 1 ) , μ t ( 2 ) , θ t ∗ ) Π ( μ t ( 1 ) | θ ) Π ( μ t ( 2 ) | θ ) × d μ t ( 1 ) d μ t ( 2 ) = ∫ ∏ i = 1 n 1 f obs y ti | μ t ( 1 ) , θ t ∗ Π μ t ( 1 ) | θ d μ t ( 1 ) × ∫ ∏ i ″ = 1 n 2 f obs y t i ″ | μ t ( 2 ) , θ t ∗ Π μ t ( 2 ) | θ d μ t ( 2 ) = f 0 ( y t ( 1 ) | θ , θ t ∗ ) f 0 ( y t ( 2 ) | θ , θ t ∗ ) .

Because it is unknown whether the tth gene is EE or DE between the two groups, the final likelihood of y _t (unconditional on the location parameters) becomes a mixture of two distributions (1) and (3) as follows:

f ( y t | θ , θ t ∗ , p 0 ) = p 0 f 0 ( y t | θ , θ t ∗ ) + p 1 f 1 ( y t | θ , θ t ∗ ) .

Here, p ₀ and p ₁ are the mixing proportions of the EE and DE transcripts in the two user defined groups respectively, such that p ₀ + p ₁ = 1. The posterior probability of differential expression (PPDE) is calculated by Bayes rule using the estimates of p ₀, f ₀ and f ₁ as follows:

p 1 f 1 ( y t | θ , θ t ∗ ) p 0 f 0 ( y t | θ , θ t ∗ ) + p 1 f 1 ( y t | θ , θ t ∗ ) .

It should be noted here that θ and θ t ∗ in equations (1)-(5) are assumed to be exactly the same.

Box and Cox 38 proposed a family of power transformations of the dependent variable in regression analysis to robustify the normality assumption. By choosing an appropriate value of λ in the transformation,

g λ ( y ) = y λ − 1 λ ( λ > 0 ) log y ( λ = 0 ) ,

the standard linear regression model with the normality assumption fits well to a wide range of data. Inspired by this idea, Basu et al 36 and Minami and Eguchi 37 proposed a robust and efficient method for estimating model parameter θ by minimizing a density power divergence in a general framework of statistical modeling and inference. They 36 37 have also shown that minimizer of density power divergence is equivalent to the maximizer of β-likelihood function. According to the current problem in this paper, the β-likelihood function for θ given the values of the mixing parameter p ₀ = 1 − p ₁ and the gene specific scale parameter θ t ∗ for all t can be written as

L β ( θ | y ) = 1 Tβ ∑ t = 1 T f β ( y t | θ , θ t ∗ , p 0 ) − l β ( θ ) ,

where f(.) is the mixture of distributions as defined in (4) and l β ( θ ) = 1 1 + β ∫ f β + 1 ( y | θ , θ t ∗ , p 0 ) d y − β − 1 β which is independent of observations. Because the gradient of (6) can be converted as follows,

∂ ∂ θ L β ( θ | y ) = 1 T ∑ t = 1 T f β ( y t | θ , θ t ∗ , p 0 ) ∂ ∂ θ log f ( y t | θ , θ t ∗ , p 0 ) − ∂ ∂ θ l β ( θ ) ,

the maximum β-likelihood estimator (β-MLE) of θ can be regarded as a weighted (quasi-) likelihood estimator. Then the weight of gene t is described as a power function of its likelihood, f β ( y t | θ , θ t ∗ , p 0 ) , where f(.) is defined by equation (4). Thus, the genes with low likelihoods have unexpected expression patterns and have low weights because the normal density function produces smaller outputs for larger inputs. By assigning low weights to outliers, the inference becomes robust. It is obvious from (7) that β-MLE reduces to the classical MLE for β = 0. Because the expression pattern (EE or DE) of each gene is unknown, it is difficult to optimize both the classical log-likelihood function and the proposed β-likelihood function for directly estimating θ . To overcome this problem, we consider the EM-like algorithm to obtain β-MLE of θ treating the mixture distribution (4) as an incomplete-data density. The hyper-parameters θ and the mixing proportion p ₀ are estimated by EM algorithm as follows:

The hyperparameters, θ ,p ₀ are estimated by the EM algorithm in two steps. E-step: Compute the Q-function which is defined by the conditional expectation of the complete-data β-likelihood with respect to the conditional distribution of missing data ( Z ) given the observed data ( Y ) and the current estimated parameter value θ β ( j ) as follows:

Q β θ | θ β ( j ) = 1 Tβ ∑ t = 1 T ∑ k = 0 1 p k f k ( y t | θ , θ ̂ t ∗ ) β × π tk ( j ) − λ β ( θ )

where k = 0 for y _t belongs to EE pattern and k = 1 for y _t belongs to DE pattern. Here

λ β ( θ ) = 1 1 + β ∫ ∑ k = 0 1 p k f k ( y | θ , θ ̂ ∗ ) 1 + β dy − β − 1 β

which does not depend on observations,

π tk ( j ) = p k ( j ) f k ( y t | θ β ( j ) , θ ̂ t ∗ ) ∑ k ″ = 0 1 p k ″ ( j ) f k ″ ( y t | θ β ( j ) , θ ̂ t ∗ ) , ( k = 0 , 1 )

is the posterior probability of kth pattern for gene t and the value of p ₁ = 1 − p ₀ is updated by a separate EM formulation as follows:

p 1 ( j + 1 ) = ∑ t = 1 T f 1 β ( y t | θ β ( j ) , θ ̂ t ∗ ) π t 1 ( j ) ∑ t = 1 T f 0 β ( y t | θ β ( j ) , θ ̂ t ∗ ) π t 0 ( j ) 1 β − 1 + 1 − 1 , for β > 0 = 1 T ∑ t = 1 T π t 1 ( j ) , for β = 0 .

For β → 0 , the proposed Q-function Q β ( θ | θ ( j ) ) reduces to the standard Q-function Q( θ | θ ^(j)) of the standard empirical Bayes approaches 8 30 .

M-step: Find θ ^{(j + 1)} by maximizing the proposed Q-function as defined in (8). Continue EM iterations up to the convergence of successive estimates of θ . The estimate of θ after convergence is taken to be the β-MLE of θ according to the EM properties.

The tuning parameter, β, controls the balance between the robustness and efficiency of the estimators. By setting a tentative value for β ₀, the optimal value is estimated by maximizing the predictive β ₀-likelihood via a five-fold cross validation. The dataset is divided into five subsets by transcripts. For each value of β, the predictive β ₀-likelihood of each subset is calculated based on the maximum β-likelihood estimates of the parameters based on the rest of the data. Finally, the β value that maximizes the average predictive β ₀-likelihood is selected as the optimal value of β. For more information about β-selection, please see 39 40 .

Then, based on the estimate values of the model parameters, we can compute the PPDE between two groups of y _t using equation (5) for all t. However, PPDE of contaminated gene using equation (5) might be produced misleading result, since PPDE of y _t depends on the estimate values of parameters and measurements of y _t . To overcome this problem, we detect contaminated genes using β-weight function and replace the contaminated measurements in y _t by its group means. Then we compute the PPDE of contaminated y _t using equation (5) also. The PPDE based on β-MLE, we call β-PPDE in this paper. The detail discussion for computation of β-PPDE under LNN model is discussed below in the LNN model.

In this paper, we use the LNN (log-normal-normal) hierarchical model for computing the posterior probability of differential expressions. In the LNN model, log-transformed gene expression measurements are assumed to follow normal distribution for each gene with the transcript-specific parameter θ t = ( μ t , θ t ∗ ) , where μ _t is the transcript-specific mean and θ t ∗ = σ t 2 is the transcript-specific variance for gene t 8 30 . A conjugate prior for μ _t is assumed to follow the normal with some underlying mean μ ₀and variance τ 0 2 ; that is, Π μ t | θ ∼ N ( μ 0 , τ 0 2 ) , where θ = ( μ 0 , τ 0 2 ) . By integrating as in (1), the density f ₀(·) for an n-dimensional input becomes Gaussian with the mean vector μ ₀ = ( μ 0 , μ 0 , … , μ 0 ) t and an exchangeable covariance matrix as follows:

Σ tn = ( σ t 2 ) I n + ( τ 0 2 ) M n ,

where I _n is an n × n identity matrix and M _n is a matrix of ones.

The gene specific variance σ t 2 is computed separately assuming prior distribution for σ t 2 as scale-inverse χ 2 ( ν ∗ , σ ∗ 2 ) , where ν _∗ is the degrees of freedom and σ ∗ 2 is the scaled parameter. Yang et al. 30 proposed that σ t 2 could be estimated by a Bayes estimator defined as,

σ ̂ t 2 = ν ̂ ∗ σ ̂ ∗ 2 + ( n 1 + n 2 − 2 ) σ ~ t 2 n 1 + n 2 + ν ̂ ∗ − 2

where

σ ~ t 2 = ( n 1 − 1 ) σ ~ t 1 2 + ( n 2 − 1 ) σ ~ t 2 2 n 1 + n 2 − 2

is the pooled sample variances with

σ ~ tg 2 = ∑ i = 1 n g ( y ti ( g ) − y ̄ t ( g ) ) 2 / ( n g − 1 )

as the sample variance in group g = 1,2. By viewing the pooled sample variances σ ~ t 2 as a random sample from the prior distribution of σ t 2 , the estimates ( ν ̂ ∗ , σ ̂ ∗ 2 ) of ( ν ∗ , σ ∗ 2 ) are obtained using the method of moments. However, it is obvious that (12) will be very sensitive to outliers. Therefore, we have used a maximum β-likelihood estimation of σ tg 2 which is highly robust against outliers 39 and can be obtained iteratively as follows:

μ tg ( j + 1 ) = ∑ i = 1 n g ψ β ( y ti ( g ) | μ tg ( j ) , σ tg 2 ( j ) ) y ti ( g ) ∑ i = 1 n g ψ β ( y ti ( g ) | μ tg ( j ) , σ tg 2 ( j ) ) σ tg 2 ( j + 1 ) = ∑ i = 1 n g ψ β ( y ti ( g ) | μ tg ( j ) , σ tg 2 ( j ) ) ( y ti ( g ) − μ tg ( j ) ) 2 ∑ i = 1 n g ψ β ( y ti ( g ) | μ tg ( j ) , σ tg 2 ( j ) )

where

ψ β ( y ti ( g ) | μ tg , σ tg 2 ) = exp − β 2 y ti ( g ) − μ tg σ tg 2

is the β-weight function for estimating robust mean and variance which produces an almost zero or very small weight for y _ti if it is an outlying/extreme observation.

To estimate the hyper-parameters θ = ( μ 0 , τ 0 2 ) by maximizing of the proposed Q-function (8) in the M-step, we compute the gradient of Q β ( θ | θ ( j ) ) with respect to θ which is given by

∂ ∂θ Q β ( θ | θ ( j ) ) = 1 T ∑ t = 1 T ∑ k = 0 1 p k f k ( y t | θ , σ ̂ t 2 ) β × ∂ ∂ θ log p k f k ( y t | θ , σ ̂ t 2 ) × π tk ( j ) − ∂ ∂ θ λ β ( θ ) .

It reduces to the gradient of the standard Q-function denoted by ∂ ∂θ Q ( θ | θ ( j ) ) based on the log-likelihood function for β = 0. The second term on the right-hand side of equation (15) is independent of observations; the first term is the weighted gradient of Q( θ | θ ^(j)) with the weight function p k f k ( y t | θ , σ ̂ t 2 ) β . This weight function produces a smaller weight if the tth gene is contaminated by outliers; otherwise, it produces a comparatively larger weight for the tth gene independent of whether it is EE (k=0) or DE (k=1). Therefore contaminated genes cannot influence the estimates and robust estimates of the parameters can be obtained. For convenience of choosing the threshold weight to identify contaminated genes statistically, we define the β-weight function for the gene t as follows

ϕ β ( y t | θ ̂ , σ ̂ t 2 , k ) ∝ [ p k f k ( y t | θ ̂ , σ ̂ t 2 ) ] β ,

where the circumflex above a parameter indicates the proposed estimate of the parameters. Excluding the normalization constant, the β-weight function corresponding to an EE gene becomes,

ϕ β ( y t | θ ̂ , σ ̂ t 2 , k = 0 ) = exp { − β 2 ( y t − μ ̂ 0 ) ″ Σ ̂ tn − 1 ( y t − μ ̂ 0 ) } ,

which measures the deviation of each gene expression data vector from the grand mean vector for the expression of all the genes in the dataset. The β-weight function corresponding to a DE gene becomes

ϕ β y t | θ ̂ , σ ̂ t 2 , k = 1 = exp − β 2 y t ( 1 ) − μ ̂ 0 ( 1 ) ″ × Σ ̂ t n 1 − 1 y t ( 1 ) − μ ̂ 0 ( 1 ) + y t ( 2 ) − μ ̂ 0 ( 2 ) ″ × Σ ̂ t n 2 − 1 y t ( 2 ) − μ ̂ 0 ( 2 ) ,

where μ ̂ 0 ( 1 ) = ( μ ̂ 0 , μ ̂ 0 , … , μ ̂ 0 ) t and μ ̂ 0 ( 2 ) = ( μ ̂ 0 , μ ̂ 0 , … , μ ̂ 0 ) t are the grand mean vectors, and Σ ̂ t n 1 = ( σ ̂ t 2 ) I n 1 + ( τ 0 2 ) M n 1 and Σ ̂ t n 2 = ( σ ̂ t 2 ) I n 2 + ( τ 0 2 ) M n 2 are the exchangeable covariance matrices in two user defined groups. Both the β-weight functions defined by equations (17) and (18) for genes t = 1,2,…,Tproduce weights that are between 0 and 1 for any data vector y _t .

Because, both weight functions are the negative exponential function of the squared Mahalanobis Distance (MD) defined by MD t = ( y t − μ ̂ 0 ) ″ Σ ̂ − 1 ( y t − μ ̂ 0 ) ≥ 0 between the data vector y _t and and the mean vector μ ̂ 0 . From equations (17) and (18), the β-weight for gene t decreases when MD _t increases and increases when MD _t decreases. That is, the β-weight for a gene t becomes smaller (≥ 0) when y _t is contaminated by outliers, and larger (≤ 1) when it is not contaminated.

The large number of transcripts in microarray data enables a statistical investigation of the observed distribution of the β-weights compared to the predicted distribution under the assumption that the model is correct and the data is free from outliers. To investigate this further, we start with the case where the predicted distribution can be obtained theoretically. When the normality assumptions hold and there are no outliers, and when the gene-specific variance is known for EE genes, the cumulative distribution of the β-weight w t = ϕ β ( y t | θ , σ t 2 , k = 0 ) for gene t with known gene specific variance ( σ t 2 ) becomes,

G t ( w 0 ) = Pr { w t ≤ w 0 } = Pr exp − β 2 y t − μ 0 ″ Σ tn − 1 y t − μ 0 ≤ w 0 = 1 − P χ n 2 ( − 2 β log w 0 ) ,

which implies that w _t follows 2 β × w 0 p χ ( n ) 2 ( − 2 β log w 0 ) , where χ ( n ) 2 denotes the chi-square variable which assumes values − 2 β log w 0 for 0 < w ₀ ≤ 1, with n degrees of freedom. Similarly, for DE genes (18) the β-weight w t = ϕ β ( y t | θ , σ t 2 , k = 1 ) also follows 2 β × w 0 p χ ( n = n 1 + n 2 ) 2 ( − 2 β log w 0 ) , for 0 < w ₀ ≤ 1 using the additive property of χ ² distributions.

In many cases, however, the variance is unknown. For such cases, the distribution of the β-weights is obtained by parametric bootstrapping. Thus statistically, we can examine whether or not a gene is contaminated by outliers using either one of the two β-weight functions because both weight functions follow the same distribution and show similar trends for the observed weights of both gene expression patterns (DE and EE). However, the tth gene is defined as contaminated by outliers if

w t = ϕ β ( y t | θ ̂ , σ t 2 , k = 1 ) < w 0 = ξ p

where ξ _p is the p-quantile of the β-weights defined by

Pr ϕ β ( y t | θ ̂ , σ t 2 , k = 1 ) < ξ p ≤ p.

Heuristically, we choose p = 10⁻⁵ for the detection of contaminating genes. Then we compute the β-PPDE using equation (5) updating the measurements in the contaminated genes. To compute the β-PPDE with respect to a contaminating gene expression, say, for example, y t = y t ( 1 ) : y t ( 2 ) by equation (5), we modify the contaminated measurements in y t ( g ) using the robust mean μ ̂ tg obtained iteratively using equation (13). Here y ti ( g ) is taken to be the ith contaminated measurement of y t ( g ) in group g=1, 2 if

ψ β ( y ti ( g ) | μ ̂ tg , σ ̂ tg 2 ) < α p ,

where α _p is the p-quantile of the β-weights defined by

Pr ψ β ( y ti ( g ) | μ ̂ tg , σ ̂ tg 2 ) < α p ≤ p.

Here ψ β ( y ti ( g ) | μ tg , σ tg 2 ) is the β-weight function that is used to compute the robust mean and variance (14), which follows 2 β × w 0 p χ ( 1 ) 2 ( − 2 β log w 0 ) , where χ ( 1 ) 2 denotes the chi-square variable which assumes values of − 2 β log w 0 for 0 < w ₀ ≤ 1, with 1 degree of freedom. However, we can set an arbitrary threshold (α ₀ = 0.2 ) to detect contaminated measurements with weights that are below the threshold, because weights are close to zero for outlying/extreme observations.

The β-EB approach that we developed detected a large proportion of outliers with p-values less than 10⁻⁵. In the microarray data of head and neck cancer, 1.75% of the genes were outliers; in the lung cancer data, 13.75% were outliers; and in Arabidopsis thaliana, 16.59% were outliers in the empirical data analysis. A detailed inspection of the outliers detected in the lung cancer data reflected misspecification of the model. To investigate the effect of outliers and model misspecification, we conducted a numerical simulation in which we compared the performance of the proposed β-EB approach with the t-test, linear models for microarray data (Limma) 22 , SAM 17 , and other EB approaches (EB-LNN, eGG 29 , eLNN 29 , GaGa 21 ). The t-test, Limma, and SAM detect DE genes based on p-values while, the EB procedures and the β−EB approach detect DE genes based on posterior probabilities. Therefore, we calculated the AUC (area under the curve) and pAUC (partial area under the curve) of the ROC curves. We also compared the estimated proportion of DE genes obtained using the β−EB and EB approaches. This characteristic plays an important role, especially when the aim of the study is to identify the major regulatory elements that influence the expressions of a large number of genes. The EB approaches estimate the proportion of DE genes by the mean posterior probability. The β−EB approach estimates it by using equation (11). No reasonable procedure to calculate the proportion of DE genes for the t-test, Limma and SAM methods could be found, because, in these methods, the estimation depends on the threshold value of the p-values.

Assuming the LNN model, we used the β-EB approach to analyze the head and neck cancer data 41 . By cross-validation, the tuning parameter βwas estimated to be 0.016 [see Additional file 1: Figure S2(a)]. The distribution of β-weights was qualitatively similar to the previously reported parametric bootstrap-based predictive distribution for all but 261 outliers (2.2% of the total genes) that have small β-weights for which p < 10⁻⁵ (Figure 2). Because the sample size was large, the EB and β-EB approaches both generated consistently decisive results for the proportion of DE/EE for most of the genes. Of the 12,625 genes, 9,538 were estimated to be EE with posterior probabilities > 0.95 (posterior probabilities of DE were < 0.05). Both methods estimated the same 525 genes to be DE with posterior probabilities > 0.95 (Figure 3(a)). The mixing proportion of the DE genes p ₁ for the classical EB-LNN and β-EB approaches was estimated to be 0.095 and 0.084 respectively. The classical EB-LNN approach may have overestimated the proportion of DE genes (see Table 1).

The β-EB approach detected six contaminating genes (LRP8, S100A8, S100A9, TRIM29, CSTA, ACP5) as outliers with the posterior probability of DE > 0.95; the posterior probability for these genes by the classical EB-LNN approach was < 0.5. For the most part, even after log transformation, these genes were over-expressed or under-expressed in only one or two of the samples (Figure 3(b)). There is strong evidence that links all of these genes with cancer.

Aberrations of the short arm of chromosome 1 (1p) are common events in lung and many other types of cancer. The low-density lipoprotein receptor-related protein 8 (LRP8) which is associated with the Wnt developmental pathway is coded by a gene on chromosome 1p; this gene has been shown to be over-expressed in lung cancer 44 . Wnt ligands bind to LRPs, and interfere with the multi-protein APC/β-catenin destruction complex. The complex role of β-catenin in cell proliferation and cell adhesion has been the main focus of many mechanistic studies.

S100 proteins, belonging to the superfamily of EF-hand calcium-binding proteins, are involved in cellular processes translating changes in Ca²+ levels into specific cellular responses by binding to target proteins. At least 16 genes of the multigenic S100 family, including the genes coding for S100A8 (MRP8 or calgranulin A) and S100A9 (MRP14 or calgranulin B), are clustered on human chromosome 1q21, a region that is a frequent target for the chromosomal rearrangements that occur during tumor development. The complex of S100A8 and S100A9 (also called calprotectin) is actively secreted during the stress response of phagocytes 45 . The complex activates the signaling pathways that promote tumor growth and metastasis by inducing the expression of multiple downstream protumorigenic effector proteins 46 . The classical EB-LNN approach strongly identified S100A8 and S100A9 as EE genes with posterior probabilities of DE being 0.027 and 0.030 respectively.

The TRIM29 protein (tripartite motif-containing protein 29) was reported to bind p53 and antagonize p53-mediated functions 47 . CSTA (stefin-A) inhibits the cysteine proteinases that participate in the dissolution and remodeling of connective tissue and basement membranes in the processes of tumor growth, invasion, and metastasis 48 . Tartrate-resistant acid phosphatase 5 (ACP5 or TRAP) may act as a growth factor to promote proliferation and differentiation of osteoblastic cells and adipocytes. The intensity of histochemical activity in several human breast cancer cell lines and tissues that express TRAP was found to correlate with the degree of tumorigenicity 49 .

The classical EB-LNN approach attached lower posterior probabilities to these genes, probably because the extraordinary expression of these genes in a few samples led to an over-estimation of the variances within the groups.

The value of β was estimated to be 0.018 (Additional file 1: Figure S2(b)). The β-weight distribution of the two types of lung cancer data 42 showed a large deviation from the predicted distribution (Figure 4). The β-weight distribution had heavy tails on both sides, suggesting that some of the assumptions behind the LNN model were violated. We inspected the distribution of the mean expression levels of the genes and found that the distribution of the mean log-transformed expression levels is bi-modal and not uni-modal (Figure 5(a)). Most of the genes that had unexpectedly low and unexpectedly high weights had low mean-expression levels. To further investigate the properties of the outliers, we plotted the standard deviations against the means of the log-transformed expression levels of the genes (Figure 5(b)). We found that the genes with extremely low weights tended to have large standard deviations, implying their irregular expression in some samples. Genes with extremely high weights had low standard deviations and low means.

The β-weight is a monotone decreasing function of the squared Mahalanobis Distance between the log transformed expression profile and the transcript specific log transformed mean (equations 17 and 18). When the transcripts with little variation (standard deviation < 0.05) were excluded, the upper heavy tail disappeared (Figure 5(c)).

Assuming the LNN model, we applied the proposed β-EB approach to the combined microarray data and marker genotypes information from A. thaliana. To identify transcripts that are significantly linked to genomic locations, at each marker we tested for significant linkage across transcripts instead of testing each transcript for significant linkage across markers. This procedure amounted to identifying DE transcripts at each marker, with groups determined by marker genotypes “A” and “B”. For simplicity, we considered a backcross population from two inbred parental populations, P1 and P2, genotyped as either A or B at the M markers. The β-EB approach predicted a large number of DE genes compared with the classical EB-LNN approach, because of some gene expressions breakdown the normality assumptions or contaminated by outliers (Figure 6(c)). Through cross-validation, the tuning parameter β was estimated to be 0.016 for chromosomes 1-5. Here, we focus on a telomeric region of chromosome 4, where β-EB detected potential hotspots and the classical EB-LNN did not (Figure 6(a)). The parametric predicted distribution and observed distribution of the weights of the data from A. thaliana were measured for marker 73 on chromosome 4. The β-weight distribution showed a large deviation from the predicted distribution (Figure 6(b)). The expression levels of the 18 transcript with weights less than 0.003 (i.e., w < .003) are shown in Figure 6(c). The log-transformed expressions at marker genotype B are plotted below the lines while those at marker genotype A are plotted above the lines. Outliers with low weights are in red. According to information obtained from the Arabidopsis gene regulatory information server (AGRIS) 50 , this region inclu des three transcription factors one of which is CYC1 (cyclin-dependent protein kinase regulator) 51 .