We start this section with a brief review of the existing MMM based techniques. Consider Y_ij as the expression level of gene in array i (i = 1, ..., n; j = 1, ..., j₁, j₁ + 1, ..., j₁ + j₂), where the first j₁ and last j₂ arrays are obtained under two conditions. A general statistical model for the resulting data is:

Y_ij = a_i + b_ix_j + ε_ij     (1)

Where x_j = 1 for 1 ≤ j ≤ j₁ and x_j = 0 for j₁ + 1 ≤ j ≤ j₁ + j₂. In addition, ε_ij is a random error with mean 0. From the above formulation, it can be seen that a_i + b_i is the mean expression level of the first condition, and a_i is the mean expression level of gene i in the second condition. The method requires that both j₁ and j₂, the number of data sets for each experiment condition, be even.

The t-test statistic type scores (2) and (3) are calculated on the pre-processed data. Here, a_i is a random permutation of a column vector that contains j₁/2 1's and j₁/2 -1's and b_i contains j₂/2 1's and j₂/2 -1's.

Since the data are not assumed to be normally distributed, the distribution functions f₀ and f are estimated as in (4) and (5), respectively. The null distributions, f₀ and f, are estimated directly in a nonparametric model for gene expression data.

Where φ(z; μ_i, V_i) symbolizes the normal density function with mean μ_i, variance V_i, and the mixing proportions π_i define the linear combination of the normal basis function. We use Φ_g0 to represent all unknown parameters {(π_i, μ_i, V_i): i = 1, ..., g₀ } in a g₀-component mixture model. The number of normal basis functions, i.e. g₀ can be estimated by the EM algorithm, which maximizes the log-likelihood function of (6) to obtain the maximum likelihood estimation of .

Within K iterations, the EM algorithm is expected to find the local maxima for all unknown parameters. It is recommended to run the EM algorithm several times with various random starting parameters and choose the final estimate as the one resulting the largest log-likelihood 6. As mentioned above, using random starting points causes the result of the MMM instable and avoids reproducibility of the results. More specifically, in each run the MMM algorithm may give different number of expressed genes, which is not desirable in biological studies. This issue will be addressed in our proposed method.

After finding the optimized for different g₀ 's, the algorithm selects the sub-optimal g₀ corresponding to the first local minimum of BIC or AIC 16.

where v_g0 is the number of independent parameters in Φ_g0. Then, the algorithm uses the resulting g₀ as the number of normal functions to fit f₀. The same procedure is performed to estimate the sub-optimal number of normal functions to estimate f. As mentioned above, with the fixed number of normal functions, the parameters of functions f and f₀ are iteratively updated for a number of iterations. When the iterations are terminated, the likelihood ratio is estimated based on the final estimations of f₀ and f:

LR(Z) = f₀(Z) / f(Z)     (9)

A bisection method 17 with a Bonferroni adjustment is used to determine the cut-off points 18 for decision-making. This means that for a threshold value s, if LR(Z) <s, then the gene is identified to have significantly altered expression in two experiments. It is possible to determine the rejection region numerically, i.e. for any false positive rate α, the threshold value s = s(α) can be calculated from the following integral:

In literature of microarray processing, α = 0.01 is often used as the genome wide significant level, so the gene-specific significance level is: α* = α/(2n) Recently a new modification of the MMM algorithm, Mod2MMM hereafter, was introduced 6. This method points out a problem in constructing the test and null statistics and indicates that the true distribution of z may be different from the null distribution of Z, which can lead to invalid inference. The modified algorithm starts with the assumption that j₁ ≥ 2 j₂ 6, and constructs the new z and Z as you can follow in appendix1.

For the cases where j₁ ≥ j₂ but j₁ < 2 j₂, j₁ observations drawn under condition one are split into two equally-sized parts to calculate , v_i(1a) and , v_i(1b) respectively. To calculate and v_i(2) about j₁/2 observations are drawn under condition two. While this modification can address the differences in the distributions of f and f₀, the stability of the parameter estimation step still remains a major problem.

The main difference between the conventional MMM and its recent extensions are that the conventional MMM disregards the fact that the true distribution of z (the statistical variable under study) may be different from the null distribution of the statistics Z (as defined below). This assumption can potentially lead to invalid inference. A modified version of the MMM (Mod2MMM hereafter), introduced in 6, assumes that the denominator and the numerator of one of t-statistic-type score _zi may not be independent. This method addresses the issue by constructing new z_i and Z_i variables as will be discussed later.

A concern over all existing MMM based methods (including Mod2MMM) that greatly affects the results is associated with the way mixed distributions are estimated. In the MMM, Expectation Maximization (EM) algorithm 19 is often used to optimize the parameters of fitted mixture distribution functions of two t-statistic-type scores related with genes expression level. Starting the EM algorithm with random values as the parameters of the normal basis functions to estimate distributions makes the results depend highly on the exact initialization, and always makes variations in the results. On the other hand, if all parameters of the normal functions in the mixture model distributions are set without iterative optimization, the set values may never result to an accurate model of the data set in hand. We propose a modified version of MMM to address this problem. Our modified MMM (K5M hereafter) combines K-mean clustering and the EM estimation to not only optimize most of the parameters with the EM iteratively but also apply K-means to optimize other sensitive parameters to ensure complete reproducibility of the algorithm. The experimental results indicate superior robustness of the proposed algorithm compared to the conventional MMM and other recently introduced extensions of the MMM 6.

In order to address the stability and reproducibility of the MMM, we propose a new modified approach for the MMM that estimates the distribution function of z by using mixture of normal distributions in a stable and reliable way. The following observations made in the experimental study of the MMM for gene expression analysis were the main motivations for the proposed changes to the MMM:

1 The observed variations in the parameter estimation process in some versions of the MMM can be attributed to the algorithm's attempt to iteratively update the means and variances of the normal distributions using often noisy data. In experimental studies, often the direct observation of the data reveals specific points where centers (means) can be positioned and the scattering patterns that can give reliable estimates on the variance of each cluster. However, the iterative updating of model parameters with noisy data and based on some random starting points often misses the true optimal points and even creates variations and fluctuations in parameter estimation in many runs.

2 Even when variations do not occur, two runs of the algorithm can result to significantly different estimations of f and f₀. This in turns results to lists of differentially expressed genes in different runs. More specifically, a set of two typical runs of the algorithm on the same data set can result to two lists that are very different both in number of the genes as well as the exact genes picked up by the algorithm. In our study of the conventional MMM and Mod2MMM, two runs with the same algorithm (on the same data) resulted to lists whose size vary between 50 and 200.

3 The literature of other areas of research utilizing normal basis function for estimation including neural networks indicates that in order to have more robustness in different runs and have reproducible results, the means and variances of the basis functions must be estimated and fixed during the iteration on the coefficients 20. This is due to the fact that updating means and variances makes the estimation process a nonlinear one that is highly sensitive and very likely to become unstable. However, when updating the values of coefficients only, the problem is reduced to a reliable linear estimation that is much more robust and stable.

4 Based on the observations mentioned above, in our proposed method, finding the distribution of z is regarded partially as a clustering problem, i.e. the means and variances of the normal distributions are estimated as the prototypes of a clustering step. Specifically, if z is distributed in a one-dimensional space, wherever there is a mass of z, there is a cluster with mean μ_i and variance V_i, which are identified by the members of that cluster.

Hence applying a clustering method is capable of estimating the means and variances of each normal distribution. The key is to use a simple clustering technique such as K-mean to estimate the mixture distributions f₀ and f based on K normal distributions. While the algorithm can use K-means to find the optimal values of means and variances, the coefficients π_i 's need to be optimized using an optimization process such as the EM.

Based on the above discussion, the proposed algorithm can be described in the following two steps:

Step 1: Using BIC, find the sub-optimal number of normal distributions for both f and f₀ (as described above). These optimal numbers are then used as the number of clusters in K-means technique.

Step 2: Using K-means clustering technique, for both f and f₀ find the best mean μ_i and variance V_i for all clusters.

Step 3: With the obtained values of μ_i, V_i and using the EM algorithm, iteratively update the values of the optimized π_i for all clusters (both f and f₀), i.e.

A reasonable number of clusters is expected to be obtained from the first step of the algorithm, and the estimation results of the two bellow data sets in Tables 1 and 4 show that the used K (calculated based on AIC) is satisfactory. Table 3 shows the results of the MMM and K5M methods for the run with an unequal variance and four normal distributions for both f and f₀. The MMM creates the likelihood ratio (LR) statistics plotted in Figure 1, the K5M with K = 4 forms the LR statistics plotted in Figure 2, and the K5M with K = 2 results to the LR plot of Figure 3.

It is worth mentioning that due to the random initialization in K-means and the random initialization of the coefficients π_i 's, in each run, it is expected that the number of identified differentially expressed genes fluctuate slightly. However, as indicated above, since the K- means clustering algorithm is known to a robust method, and considering the fact that in the EM estimation process, only a linear estimation is performed, it is expected that the robustness of the proposed algorithm be much more than the other version of the MMM based algorithms. This observation, as have been shown before, is supported by our experimental results. In addition, our experimental indicate that the most expressed genes are identified in all runs or the algorithm and in each run one or two new genes with less expression ratio are added to this set.

In this section, we apply the nonparametric MMM method with and without the proposed modifications to the Leukaemia data presented in 7. The objective of this application is to identify the most important genes involved in development of different types of Leukaemia. The dataset used for this analysis includes 27 acute lymphoblastic leukaemia (ALL) samples and 11 acute myeloid leukaemia (AML) samples for 7129 genes. The main goal is to find genes with differential expression between ALL and AML cases. A second goal is to compare the result of MMM and Mod2MMM (as introduced in MMM & its recent extensions Section) with K5M and test the robustness of K5M. The genome-wide significance level is chosen α = 0.01 (according to Benferroni adjustment used in the MMM based methods). Each sample in the dataset is pre-processed as in 21, by subtracting its median and dividing the resulting variable by its quartile range (i.e. the difference between the first and the third quartile).

Thomas et al 22 used known biological information to identify the most important genes in Leukaemia and provided biological justifications for these identified genes. They introduced 50 genes out of the identified genes as the most expressed and related genes to the disease, including 25 most expressed genes for AML and 25 for ALL. We treat Thomas et al's list as the biology knowledge base and compare the capabilities of the computational techniques to correctly identify the genes discussed in 22 by processing the dataset.

The comparison of the result obtained by the K5M with those of the MMM and the Mod2MMM is summarized in Table 1. As can be seen in Table 1, The MMM has identified 187 differentially expressed genes 21, among which the total of 39 genes are in the list of genes obtained by Thomas et al 22. The Mod2MMM method found 30 genes of the Thomas's list. The K5M algorithm, determines 45 genes that are identified in the Thomas's list, i.e. the proposed algorithm successfully identifies 90% of biological result. This means that K5M improved the detection of expressed genes 12% compare to the MMM and 30% compare to the Mod2MMM for the Leukaemia data, i.e. our method identified more genes from the list of the 50 truly expressed genes identified by Thomas et al 22.

As the BIC suggested the optimum number of clusters K = 4 for the MMM, the K5M is applied with K = 4 also. Running K5M with different number of clusters leads to the different but reasonably similar results. As the number of the clusters increase, the number of expressed genes decreases. Table 1 shows that the K5M with K = 3 identifies the total of 185 differentially expressed genes, while with K = 4 the total of 58 genes are identified, however; the 58 genes found with K = 3 are the most expressed genes among 185 genes found by K = 4. This result shows the consistency of the K5M method.

In order to further compare the performance of the MMM and K5M on the leukaemia data, The ROC curve is plotted based on False Positive rate and True Positive rate of the data set calculated as in 5. The area under each curve is the measure of test accuracy. As can be seen in Figure 5, the area under the K5M curve is more than the area under the MMM curve, therefore the K5M is providing a more accurate classification than the MMM.

The following study is the main application for which the proposed method was specialized and therefore is described in more details. Hypophosphatemia is a common X-linked metabolic bone disorder in human. Hypophosphatemia results from phosphate wasting in the renal tubules. Phosphate that is normally reabsorbed from the urine is excreted. It appears that elevated levels of FGF-23 activate the excretion of phosphorous by the kidneys. Previous studies have demonstrated an impairment of the high- affinity, low capacity Na+ dependent phosphate co-transport system 2324. The main animal model used to study this disease is the Hyp mouse. Hyp mice have a mutation of the Phex gene 259. The disease is characterized by low reabsorption of phosphate, bone disease, and bone abnormalities in the lower extremities. The genes active in the regulation of phosphate re-absorption in the kidney are not well understood. It is also not clear whether mutations of the Phex gene block renal adaptation to low phosphate diet. Hyp mice have a primary osteoblast defect and defects in vitamin D metabolism. Parabiosis experiments on normal and Hyp mice have revealed that there is an intrinsic osteoblast defect in Hyp mice rather than an intrinsic renal abnormality. Hyp kidneys transplanted into normal mice reabsorbed phosphorus at normal levels. Kidneys transplanted from normal mice into Hyp mice began phosphate wasting in the Hyp mice.

The mechanism that leads to the excessive excretion of phosphorous is unknown. On a low phosphate diet a normal mouse will activate systems to conserve phosphate by increasing re-absorption. The genes activated in the normal mouse on the low phosphate diet, and the genes with differential expression between normal and Hyp mice should indicate the systems involved in the phosphorus homeostasis. In an attempt to identify these genes, nutritional experiments were performed on normal and Hyp mice [910118121314 and 15]. Normal and Hyp mice were placed on low phosphate diets for 3 – 5 days. Tissue samples from the kidneys of test and control mice were collected. 16 samples were analyzed using Affymetrix GeneChip mouse U74A arrays- 4 samples for each experiment state. The mRNA of 12,488 genes was analyzed. Two GeneChip microarrays were done for each diet for normal mice and three microarrays for each diet for the Hyp mice for a total of 10 arrays.

To investigate this, 5-week-old normal and Hyp were fed a control (1.0% P) or low phosphate (0.03% P) diet for five days. The four group experiments are shown in Table 2.

In this study, we consider the gene expression signal less than 100 as noise caused by the microarray machine, and in the pre-processing step we ignored the genes whose expression signals in both conditions are less than 100. The following two specific goals are considered in this study:

1. To identify the genes in whose mRNA expressions are altered by low phosphate diet in normal mice.

2. To determine the effect of Hyp mutation on this response, i.e. identifying the genes in Hyp condition that are differentially expressed across the normal and low phosphate diet experiments.

The Hyp dataset includes five samples for each group. In order to make the number of data samples even, we used four samples of each group. For this data set, since j1 = j2, the Mod2MMM cannot be applied. In MMM method, five mixture models are used to estimate f₀ and f (distributions under two experimental different conditions) with number of normal basis functions ranging from 1 to 5, i.e. The MMM algorithm was run several times and the run with maximum log-likelihood was chosen as the final model. Bayesian Information Criterion (BIC) 26 was used to determine the number of components. To find the rejection region for a given model, the bisection method is used. In this paper we assume α = 0.01, and therefore the gene-specific significance level used here is calculated as:

α* = 0.01/(95.44 * 2) = 5 * 10^-7

Using bisection method 17, as discussed in Section 4, the value of s is obtained as s = 3 × 10_^-6.. Both the MMM and K5M were run 100 times. Figure 4 presents the number of genes expressed in each run of the MMM. The difference between the number of identified differentially expressed genes in two runs with the minimum and the maximum number of genes amounts to 150 genes. This clearly indicates the high degree of inconsistency and irreproducibility of the results obtained by the MMM. The number of genes expressed in each run of the K5M indicates that all genes are the same in all runs and therefore indicates 100% repeatability and robustness of the proposed method.

The ten most significant genes expressed by the low phosphate diet in the normal mouse identified by the MMM, and the ten most significant genes provided by K5M are represented in Table 4. As can be seen in Table 5, the most differentially expressed genes are same for the MMM and K5M. Out of these 10 genes, six are directly related to the kidney's functions. For this data set, the main advantage of the K5M is its consistency and robustness as discussed above. A similar procedure is conducted to accomplish the second goal of this study, i.e. identifying the role of Hyp condition on the most definitely expressed gene in normal and low phosphate diet microarrays. The ten most significant genes that are differentially expressed across the two experimental conditions, i.e. Normal Low Phosphate and Hyp Low Phosphate, are listed in table 6. As shown in the table 6, again eight genes are related directly to the kidney's function. These further witnesses to the capability of the proposed technique to discover the genes that are truly involved in the biological study.