As a target network that we attempt to infer, we used a small-scale S-system model consisting of 5 genes (N = 5). The S-system model is often used to describe biochemical networks 81415181920242526. The model is structured as a set of non-linear differential equations of the form

d X n d t = α n ∏ m = 1 N X m g n , m − β n ∏ m = 1 N X m h n , m , ( n = 1 , ⋯ , N ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeGabiqaaaqaaKqbaoaalaaabaGaemizaqMaemiwaG1aaSbaaeaacqWGUbGBaeqaaaqaaiabdsgaKjabdsha0baakiabg2da9GGaciab=f7aHnaaBaaaleaacqWGUbGBaeqaaOWaaebCaeaacqWGybawdaqhaaWcbaGaemyBa0gabaGaem4zaC2aaSbaaWqaaiabd6gaUjabcYcaSiabd2gaTbqabaaaaaWcbaGaemyBa0Maeyypa0JaeGymaedabaGaemOta4eaniabg+GivdGccqGHsislcqWFYoGydaWgaaWcbaGaemOBa4gabeaakmaarahabaGaemiwaG1aa0baaSqaaiabd2gaTbqaaiabdIgaOnaaBaaameaacqWGUbGBcqGGSaalcqWGTbqBaeqaaaaaaSqaaiabd2gaTjabg2da9iabigdaXaqaaiabd6eaobqdcqGHpis1aOGaeiilaWcabaGaeiikaGIaemOBa4Maeyypa0JaeGymaeJaeiilaWIaeS47IWKaeiilaWIaemOta4KaeiykaKIaeiilaWcaaaaa@6512@

where X_n is the expression level of the n-th gene, α_n and β_n are multiplicative parameters called rate constants, and g_n,m and h_n,m are exponential parameters called kinetic orders. The S-system parameters of the target genetic network are shown in Table 11627. This study assumes that the m-th gene positively regulates the n-th gene when X_m promotes the synthesis or suppresses the degradation of X_n. Similarly, when X_m suppresses the synthesis or promotes the degradation of X_n, we assume that the n-th gene is negatively regulated by the m-th gene. When the m-th gene positively regulates the n-th gene, g_n,m is positive and/or h_n,m is negative in the S-system model. When the m-th gene negatively regulates the n-th gene, g_n,m is negative and/or h_n,m is positive. When the m-th gene has no influence on the n-th gene, the parameters g_n,m and h_n,m are zero. Thus, we can illustrate the structure of the target network, as shown in Figure 1.

As the observed gene expression patterns, fifteen sets of noise-free time-series data, each covering all five genes, were given. The sets began from randomly generated initial values in [0.0, 2.0] and were obtained by solving the differential equations (2) on the target model. In a practical application, these sets of time-series data could be obtained by actual biological experiments. Eleven sampling points for the time-series data were assigned to each gene in each set. Thus, the observed time-series data for each gene consisted of 15 × 11 = 165 sampling points. We inferred the reduced NGnet models solely from these time-series data, and then, extracted interactions between genes from the obtained models. As one reduced NGnet model corresponds to one gene, we must infer 5 models to solve the artificial problem defined here. We carried out 10 trials by changing the seed for the pseudo random number to obtain each model. According to the preliminary experiments (see Additional file 1), we used the following parameters; the weight parameter used in the prior probability distribution γ was 20, the maximum indegree I was 5, and the number of the units of the reduced NGnet model M was 3. We can change the function approximation capability of the reduced NGnet model using the number of units M. However, we cannot use the model with an unduly large M, since the large M makes the estimation problem of the model parameters difficult. An unduly small M, on the other hand, should make the model insufficient to represent complicated interactions between genes.

We extracted interactions between genes from the reduced NGnet models obtained using the method based on the sensitivity analysis 10 (see also the Method section). Figure 2 shows a typical genetic network inferred from the obtained models. As the figure illustrates, most of the regulations were correctly inferred by the proposed method. Our method inferred an average of 1.3 ± 1.7 unnecessary regulations that were absent in the target network, i.e., false-positive regulations, and failed to infer an average of 1.9 ± 0.5 regulations that were present in the target network, i.e., false-negative regulations. The sensitivity and the specificity of the proposed method were therefore 0.854 ± 0.041 and 0.967 ± 0.045. respectively. These measures are defined as

( sensitivity ) = T P T P + F N , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaeiikaGIaee4CamNaeeyzauMaeeOBa4Maee4CamNaeeyAaKMaeeiDaqNaeeyAaKMaeeODayNaeeyAaKMaeeiDaqNaeeyEaKNaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavcqWGqbauaeaacqWGubavcqWGqbaucqGHRaWkcqWGgbGrcqWGobGtaaGccqGGSaalaaa@4794@

( specificity ) = T N F P + T N , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaeiikaGIaee4CamNaeeiCaaNaeeyzauMaee4yamMaeeyAaKMaeeOzayMaeeyAaKMaee4yamMaeeyAaKMaeeiDaqNaeeyEaKNaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavcqWGobGtaeaacqWGgbGrcqWGqbaucqGHRaWkcqWGubavcqWGobGtaaGccqGGSaalaaa@4732@

where TP, FN, TN and FP are the numbers of true-positive, false-negative, true-negative and false-positive regulations, respectively. The sensitivity increases from 0 to 1 with decreasing the number of false-negative regulations, and the specificity increases from 0 to 1 with decreasing the number of false-positive regulations. The sum of the squared error between the time-course produced by the obtained model and the given time-series data, i.e., the value of the objective function (16) defined in the Methods section, was 1.57 × 10^-1 ± 1.23 × 10^-1 on average.

Although our method has an ability to infer the positive and negative regulations of the n-th gene from the m-th gene simultaneously, it failed in inferring these regulations of the 3rd gene from the 2nd gene in most of the trials. Given that X₂ works to suppress both the synthesis and the degradation of X₃ with the same kinetic order in the target model, we know that the 2nd gene has only a weak impact on the 3rd gene. Therefore, it should be difficult for our method to infer these regulations. In order to infer these difficult regulations correctly, we should give more sets of the gene expression data 1528.

While the proposed method was unable to infer the target network with perfect precision as mentioned above, its computational time was sufficiently short. The computational time to obtain one reduced NGnet model averaged about 60.8 sec on a single-CPU personal computer (Pentium IV 2.8 GHz). The proposed method therefore required about 60.8 sec × 5 ≃ 5.1 min to solve this genetic network inference problem. In order to infer the same network, on the other hand, PEACE1 16, the coevolutionary method 18, the method with a collocation approximation 21 and the decoupling method 15 reportedly took about 10 h on a PC cluster (Pentium III 933 MHz × 1040 CPUs), 89.0 min on a PC cluster (Pentium III 933 MHz × 8 CPUs), 2.84 h on a single-CPU personal computer (Pentium IV 2.4 GHz) and 6.38 min on a single-CPU personal computer, respectively. The comparison results present that the proposed method was faster than the other inference methods. As a high performance computer, such as a PC cluster, is not always available at every laboratory, the shorter computational time of the proposed method should be a preferable feature.

As mentioned before, the proposed method performs the inference of the reduced NGnet model and the estimation of the differential coefficients of the gene expression level simultaneously. When we can estimate the differential coefficients of the gene expression level correctly before inferring the genetic network, however, the proposed method can omit the simultaneous estimation of the differential coefficients of the gene expression level. When the simultaneous estimation of the differential coefficients is omitted, our method is almost equivalent to the inference method proposed in the paper 22. As the data used in this section contained no noise, the inference ability of our method was not degraded even when the simultaneous estimation of the differential coefficients was omitted. The sensitivity and the specificity of the proposed method without the simultaneous estimation of the differential coefficients of the gene expression level were 0.854 ± 0.041 and 0.961 ± 0.044, respectively. Moreover, the omission of the simultaneous estimation of the differential coefficients made the computational time much shorter. The method without the simultaneous estimation of the differential coefficients required about 2.0 × 5 ≃ 10.0 sec to solve this problem. However, the simultaneous estimation of the differential coefficients improved the sum of the squared error between the time-courses produced by the obtained model and the given time-series data. Thus, the averaged objective values (16) of the method with and without the simultaneous estimation of the differential coefficients were 1.57 × 10^-1 ± 1.23 × 10^-1 and 3.37 × 10^-1 ± 1.93 × 10^-1, respectively. These results indicate that, although the simultaneous estimation of the differential coefficients of the gene expression level, proposed in this study, makes the computational cost higher, it produces the models that are suitable for the computational simulation.

In this experiment, we used the following set of differential equations to describe target networks 710.

d X n d t = − λ n X n + α n + ∑ m ∈ A n X m γ n , m 1 + ∑ m ∈ A n X m γ n , m + ∑ l ∈ ℛ n X l β n , l , ( n = 1 , ⋯ , N ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeGabiqaaaqaaKqbaoaalaaabaGaemizaqMaemiwaG1aaSbaaeaacqWGUbGBaeqaaaqaaiabdsgaKjabdsha0baakiabg2da9iabgkHiTGGaciab=T7aSnaaBaaaleaacqWGUbGBaeqaaOGaemiwaG1aaSbaaSqaaiabd6gaUbqabaGccqGHRaWkjuaGdaWcaaqaaiab=f7aHnaaBaaabaGaemOBa4gabeaacqGHRaWkdaaeqaqaaiabdIfaynaaDaaabaGaemyBa0gabaGae83SdC2aaSbaaeaacqWGUbGBcqGGSaalcqWGTbqBaeqaaaaaaeaacqWGTbqBcqGHiiIZt0uy0HwzTfgDPnwy1egaryqtHrhAL1wy0L2yHvdaiqaacqGFaeFqdaWgaaqaaiabd6gaUbqabaaabeGaeyyeIuoaaeaacqaIXaqmcqGHRaWkdaaeqaqaaiabdIfaynaaDaaabaGaemyBa0gabaGae83SdC2aaSbaaeaacqWGUbGBcqGGSaalcqWGTbqBaeqaaaaaaeaacqWGTbqBcqGHiiIZcqGFaeFqdaWgaaqaaiabd6gaUbqabaaabeGaeyyeIuoacqGHRaWkdaaeqaqaaiabdIfaynaaDaaabaGaemiBaWgabaGae8NSdi2aaSbaaeaacqWGUbGBcqGGSaalcqWGSbaBaeqaaaaaaeaacqWGSbaBcqGHiiIZcqGFBeIudaWgaaqaaiabd6gaUbqabaaabeGaeyyeIuoaaaGccqGGSaalaeaacqGGOaakcqWGUbGBcqGH9aqpcqaIXaqmcqGGSaalcqWIVlctcqGGSaalcqWGobGtcqGGPaqkcqGGSaalaaaaaa@88F8@

where λ_n and α_n are the degradation rate and the synthesis rate, respectively, of the n-th mRNA, and γ_n,m and β_n,l are the activation cooperativity and the repression cooperativity, respectively, of the m-th gene on the n-th gene. The sets An MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8haXh0aaSbaaSqaaiabd6gaUbqabaaaaa@38D8@ and ℛn MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83gHi1aaSbaaSqaaiabd6gaUbqabaaaaa@3834@ specify the genes that activate and repress the n-th gene, respectively. As the target networks, we randomly constructed the systems of 10, 20 and 30 genes (N = 10, 20 and 30). Since the inference ability of the proposed method may depend on the structure of the target network, we changed the network structure on every trial. We generated the target networks of different structures by changing the model parameters described above. In order to-construct the sets An MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8haXh0aaSbaaSqaaiabd6gaUbqabaaaaa@38D8@ and ℛn MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83gHi1aaSbaaSqaaiabd6gaUbqabaaaaa@3834@, we randomly chose an integer k from a power-law distribution with a cutoff 5. Then, k genes were randomly selected from all of the genes contained in the network. Finally, the indices corresponding to the selected genes were added to the set An MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8haXh0aaSbaaSqaaiabd6gaUbqabaaaaa@38D8@ or the set ℛn MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83gHi1aaSbaaSqaaiabd6gaUbqabaaaaa@3834@ with the probability 0.5. The model parameters (λ_n, α_n, γ_n,m and β_n,l) are randomly selected from a uniform distribution.

Since the performances of inference methods generally depend on the amount of given time-series data, we performed the experiments with different numbers of sets of time-series data. We obtained the sets of time-series data by solving the set of differential equations (5). Each set consisted of the expression levels at the eleven time points. The measurement noise was simulated by adding 10% Gaussian noise to the computed time-series data. All of the other experimental conditions were the same as those used in the previous experiment.

Figure 3(a), (b) and 3(c) show the sensitivity and the specificity of the proposed method on the experiments of the target networks consisting of 10, 20 and 30 genes, respectively. The figures show that the sensitivity of our method is improved as the amount of given time-series data increases. Therefore, to infer genetic networks correctly, we should give the sufficient amount of observed time-series data. As shown in the figures, when we try to achieve a satisfactory level of the sensitivity in a larger-scale problem, we should give a larger amount of observed data. The number of required time-series sets, however, seems not to be proportional to the number of genes contained in the network. This disproportion may be caused by the sparseness of the target network. Therefore, the proposed method should not always require an immense number of time-series sets even when we try to infer large-scale genetic networks.

The specificity, on the other hand, seems to be independent from the amount of given time-series data. When we try to improve it, we must reduce the number of false-positive regulations contained in the inferred model. It is however difficult to reduce these regulations because the maximum indegree I used in the prior probability distribution (15) forgives our method for finding several false-positive regulations. Therefore, we should not expect the improvement of the specificity even when we give a larger amount of the observed data.

In this experiment, we used the proposed method to analyze the SOS DNA repair system in E. coli (Figure 4) 29. About 30 genes are known to be involved in this system. In a normal state, a master repressor, LexA, is bound to the interaction sites in the promoter regions of these genes. When DNA damages occur, one of the SOS proteins, RecA, becomes activated and, then, mediates LexA autocleavage. The drop in LexA level causes the de-repression of the SOS genes. Once damage has been repaired, the level of activated RecA drops, LexA accumulates and represses the SOS genes, and the cells return to their original state. We applied the proposed method to the expression data of this system, that were collected by Ronen and his colleagues 3031. Then, Cho and his colleagues chose 6 genes from these data and successfully inferred the regulatory network of the selected genes 14. Therefore, this study also selected the same genes, i.e., uvrD, lexA, umuD, recA, uvrA and polB. Although the data contain 4 sets of time-series data, we used only 2 sets (the third and fourth sets) that were measured under the same experimental condition (see Additional file 1). Each set of the time-series data consists of 50 measurements including the initial concentrations which are all zeros. We, however, removed the initial concentrations from both of the sets, since our model cannot produce different time-courses from the same initial conditions. As it is difficult to calculate an output of the reduced NGnet model against large input values, data corresponding to each gene were normalized against their maximum value. Since the target network contained a small number of the genes, we set the maximum indegree I to 3. All of the other experimental conditions were the same as those used in the previous experiments.

We succeeded in finding models that simulate the gene expression of the target system well. A sample of the gene expression calculated from the obtained models is shown in Figure 5. Although the network structures inferred by the proposed method were slightly different from each other in 10 trials, most of the regulations were commonly inferred. Figure 6 shows the core network structure where the regulations were inferred more than 7 times within 10 trials. While the inferred networks contained an average of 26.7 ± 3.3 regulations, the core network contained 21 regulations.

The core network contained some reasonable regulations. As Figure 6 shows, although the proposed method failed to infer the regulation from lexA to uvrA. the negative regulations from lexA to the other genes were successfully inferred. As described before, RecA is known to regulate LexA. Though this is a regulatory interaction between proteins, it should be represented by the regulation of lexA from recA in our network. A number of genes take part in repairing DNA damages, and the accomplishment of DNA repair makes RecA stop the system. The inferred regulations of recA from all of the genes might explain this mechanism.

The number of the regulations inferred by the proposed method was larger than that of the S-tree based method proposed by Cho and his colleagues 14. Although some of our inferred regulations that have not been experimentally observed might be new findings, the rest should be false-positive. In order to infer a more reliable network, we must give more sets of the expression data obtained from additional biological experiments or a priori knowledge about the target system. The computational time of the proposed method was, on the other hand, much shorter. While the S-tree based method running on the computer system (Athlon MP2800+) reportedly took about 35 h to infer the network of this system, the proposed method required about 47.1 sec × 6 ≃ 4.7 min on a single-CPU personal computer (Pentium IV 2.8 GHz). In this study, we focused only on whether or not the m-th gene regulates the n-th gene. However, the method based on the sensitivity analysis used in this study also provides us with the strength of the inferred regulation. This information would help biologists find the important regulations that are worth doing further investigation.

In the genetic network inference problem, we must find a good approximation of the function G_n (n = 1, ..., N), given in the equations (1). We define this problem as a function approximation problem in this study 7810. When trying to obtain an approximation of the function G_n on the basis of the function approximation problem, we must give observations at T points

( X | t 1 , d X n d t | t 1 ) , ⋯ , ( X | t T , d X n d t | t T ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaeWaaeaadaabcaqaaGqabiab=HfaybGaayjcSdWaaSbaaSqaaiabdsha0naaBaaameaacqaIXaqmaeqaaaWcbeaakiabcYcaSmaaeiaajuaGbaWaaSaaaeaacqWGKbazcqWGybawdaWgaaqaaiabd6gaUbqabaaabaGaemizaqMaemiDaqhaaaGccaGLiWoadaWgaaWcbaGaemiDaq3aaSbaaWqaaiabigdaXaqabaaaleqaaaGccaGLOaGaayzkaaGaeiilaWIaeS47IWKaeiilaWYaaeWaaeaadaabcaqaaiab=HfaybGaayjcSdWaaSbaaSqaaiabdsha0naaBaaameaacqWGubavaeqaaaWcbeaakiabcYcaSmaaeiaajuaGbaWaaSaaaeaacqWGKbazcqWGybawdaWgaaqaaiabd6gaUbqabaaabaGaemizaqMaemiDaqhaaaGccaGLiWoadaWgaaWcbaGaemiDaq3aaSbaaWqaaiabdsfaubqabaaaleqaaaGccaGLOaGaayzkaaGaeiilaWcaaa@58FF@

where X|tk=(X1|tk,⋯,XN|tk) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaeaaieqacqWFybawaiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaGccqGH9aqpdaqadaqaamaaeiaabaGaemiwaG1aaSbaaSqaaiabigdaXaqabaaakiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaGccqGGSaalcqWIVlctcqGGSaaldaabcaqaaiabdIfaynaaBaaaleaacqWGobGtaeqaaaGccaGLiWoadaWgaaWcbaGaemiDaq3aaSbaaWqaaiabdUgaRbqabaaaleqaaaGccaGLOaGaayzkaaaaaa@470C@ is the expression levels of all of the genes at time t_k, and dXndt|tk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaKqbagaadaWcaaqaaiabdsgaKjabdIfaynaaBaaabaGaemOBa4gabeaaaeaacqWGKbazcqWG0baDaaaakiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaaaaa@3817@ is the differential coefficient of the expression levels (rate of transcription) of the n-th gene at time t_k. The purpose of this problem is then to estimate the parameters of the function approximator that outputs dXndt|tk(k=1,⋯,T) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaKqbagaadaWcaaqaaiabdsgaKjabdIfaynaaBaaabaGaemOBa4gabeaaaeaacqWGKbazcqWG0baDaaaakiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaGccqGGOaakcqWGRbWAcqGH9aqpcqaIXaqmcqGGSaalcqWIVlctcqGGSaalcqWGubavcqGGPaqkaaa@4207@ against a corresponding input X|tk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaeaaieqacqWFybawaiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaaaaa@31DC@.

Though DNA microarray technologies allow us to measure the gene expression levels, we have yet to find a biological technique capable of measuring the differential coefficient of the gene expression level. As an alternative, the data we obtain by measuring the time-series of the gene expression levels allow us to estimate the differential coefficients using interpolation techniques, such as the spline interpolation 32, the local linear regression 33, the neural network 8, or the Whittaker's smoother 23. This study estimates them using the method proposed in the Estimation of differential coefficients section. When both the gene expression levels and their differential coefficients are given, the genetic network inference problem described here becomes solvable.

Any type of function approximator is available for approximating the function G_n. This study however uses a reduced Normalized Gaussian network (NGnet) model 22, that was proposed by modifying an NGnet model 3435, since we can easily estimate the model parameters using the EM algorithm. The original NGnet model, which transforms an N-dimensional input vector x to an output y, is defined as

y = ∑ i = 1 M [ α i N ( x | m i , Σ i ) ∑ j = 1 M α j N ( x | m j , Σ j ) ]   ( w i ⋅ x + b i ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemyEaKNaeyypa0ZaaabCaeaadaWadaqcfayaamaalaaabaacciGae8xSde2aaSbaaeaacqWGPbqAaeqaaiabd6eaojabcIcaOGqabiab+Hha4jabcYha8jab+1gaTnaaBaaabaGaemyAaKgabeaacqGGSaalcqqHJoWudaWgaaqaaiabdMgaPbqabaGaeiykaKcabaWaaabmaeaacqWFXoqydaWgaaqaaiabdQgaQbqabaGaemOta4KaeiikaGIae4hEaGNaeiiFaWNae4xBa02aaSbaaeaacqWGQbGAaeqaaiabcYcaSiabfo6atnaaBaaabaGaemOAaOgabeaacqGGPaqkaeaacqWGQbGAcqGH9aqpcqaIXaqmaeaacqWGnbqtaiabggHiLdaaaaGccaGLBbGaayzxaaaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGnbqta0GaeyyeIuoakiabbccaGmaabmaabaGae43DaC3aaSbaaSqaaiabdMgaPbqabaGccqGHflY1cqGF4baEcqGHRaWkcqWGIbGydaWgaaWcbaGaemyAaKgabeaaaOGaayjkaiaawMcaaiabcYcaSaaa@6B5F@

where the dot (·) denotes an operator of the inner product, and N(x|m_i, Σ_i) is an N-dimensional Gaussian function; its center is an N-dimensional vector m_i and its covariance matrix is an (N × N)-dimensional matrix Σ_i. The N-dimensional vector w_i and the scalar b_i are the linear regression parameters, α_i (∑i=1Mαi=1 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaabmaeaaiiGacqWFXoqydaWgaaWcbaGaemyAaKgabeaakiabg2da9iabigdaXaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaemyta0eaniabggHiLdaaaa@3776@ and α_i > 0) is the weight parameter, and M is the number of units. The NGnet model softly partitions the input space into M regions using M Gaussian functions. The i-th unit linearly approximates its output by w_i·x + b_i within the corresponding region. The weighted sum of these outputs is the final output of the NGnet model. In the genetic network inference problem, the input vector x represents the expression levels of all of the genes, i.e., X = (X₁, ..., X_N), and the output, y represents the differential coefficient of the expression level of the n-th gene, i.e., dXndt MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqWGKbazcqWGybawdaWgaaqaaiabd6gaUbqabaaabaGaemizaqMaemiDaqhaaaaa@3343@.

As the original NGnet, model has a large number of the model parameters, we must give a large number of observations to obtain a good function approximation. This nature is not preferable for the inference of genetic networks, since the measurement of the gene expression patterns is expensive. In order to decrease the amount of data we must observe, we limited a covariance matrix of the NGnet model Σ_i to being diagonal 22. This restricted model was referred to as the reduced NGnet model. While the number of the parameters of the original NGnet model is O(N²), that of the reduced model is O(N).

This study approximates the function G_n using the reduced NGnet model, as mentioned above. Therefore, our object in this study is to find the parameters of the reduced NGnet model that outputs dXndt|tk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaKqbagaadaWcaaqaaiabdsgaKjabdIfaynaaBaaabaGaemOBa4gabeaaaeaacqWGKbazcqWG0baDaaaakiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaaaaa@3817@ (k = 1, ⋯, T) against a corresponding input X|tk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaeaaieqacqWFybawaiaawIa7amaaBaaaleaacqWG0baDdaWgaaadbaGaem4AaSgabeaaaSqabaaaaa@31DC@. Our algorithm to estimate these parameters is described below.

We cannot make the Gaussian function N(x|m, Σ) independent from any components of the input vector x, since its covariance. matrix Σ must be non-singular. As the reduced NGnet model contains several Gaussian functions, its output y also depends on all of the components of the input vector x. This fact indicates that, even when the n-th gene is unaffected by the m-th gene in the target network, the reduced NGnet model cannot capture the disconnection.

In order to overcome this drawback of the reduced NGnet model, we use the gradual reduction strategy 22. When trying to obtain an approximation of the function G_n, this strategy decreases the input dimension of the model by removing the genes that are assumed to unaffect the n-th gene. As the model obtained by this strategy has no input from the removed genes, its output is independent from these genes. In order to determine the reasonable number of the input dimension of the model, this study uses the Bayesian Information Criterion (BIC) 36, a measure for evaluating statistical models. The followings are the algorithm of the gradual reduction strategy used in this study:

1. Let a set C be {1, ..., N}, where N is the number of the genes in the target network. We call the genes whose indices are contained in set C the candidate genes. Let N_C be the number of the elements of set C.

2. Extract the expression data of the candidate genes from the whole observed data. Then, obtain the reduced NGnet model by applying the DAEM algorithm, described below, to the constructed data. Note that, as the constructed data contain the expression levels of the candidate genes only, the input dimension of the model is N_C.

3. Compute the BIC of the model obtained in step 2. The BIC of the reduced NGnet model is defined as

BIC = M(3N_C + 2)log(T) - 2L,

where

L = − T 2 log ⁡ { 1 T ∑ t = 1 T [ y t − y ( x t ) ] 2 } , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemitaWKaeyypa0JaeyOeI0scfa4aaSaaaeaacqWGubavaeaacqaIYaGmaaGccyGGSbaBcqGGVbWBcqGGNbWzdaGadaqaaKqbaoaalaaabaGaeGymaedabaGaemivaqfaaOWaaabCaeaacqGGBbWwcqWG5bqEdaWgaaWcbaGaemiDaqhabeaakiabgkHiTiabdMha5jabcIcaOGqabiab=Hha4naaBaaaleaacqWG0baDaeqaaOGaeiykaKIaeiyxa01aaWbaaSqabeaacqaIYaGmaaaabaGaemiDaqNaeyypa0JaeGymaedabaGaemivaqfaniabggHiLdaakiaawUhacaGL9baacqGGSaalaaa@50D2@

x_t is the expression levels of the candidate genes at time t, y_t is the differential coefficient of the expression level of the n-th gene at time t (i.e., dXndt|t MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaqGaaKqbagaadaWcaaqaaiabdsgaKjabdIfaynaaBaaabaGaemOBa4gabeaaaeaacqWGKbazcqWG0baDaaaakiaawIa7amaaBaaaleaacqWG0baDaeqaaaaa@3680@) and y(x_t) is the output of the obtained model against the input x_t.

4. If the BIC calculated in step 3 is larger than of the previous iteration, output the model of the previous step and, then, stop.

5. Using the method based on the sensitivity analysis (see the Model interpretation section) 10, choose the genes that are assumed to unaffect the n-th gene. Then, remove the indices corresponding to the selected genes from set C. When no gene is selected, remove the index of the gene that has the weakest regulation to the n-th gene.

6. Return to step 2.

We can use the EM algorithm to obtain the reduced NGnet model that outputs y_t (t = 1, ..., T) against a corresponding input x_t, since it can be interpreted as a stochastic model 92237. In our genetic network inference problem, the input x_t and the output y_t represent the given expression levels of the candidate genes at time t and the given differential coefficient of the expression level of the n-th gene at time t, respectively.

The EM algorithm however often fails to estimate reasonable model parameters because it is based on a local search. In order to enhance the probability to obtain a reasonable model, this study therefore uses the Deterministic Annealing EM (DAEM) algorithm 38, a variant of the EM algorithm. The DAEM algorithm used in this study estimates the model parameters θ = {μ_i, Σ_i, w_i, b_i, α_i, σi2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaacciGae83Wdm3aa0baaSqaaiabdMgaPbqaaiabikdaYaaaaaa@3017@|i = 1, ..., M} according to the following procedure 922.

1. Let θ⁽⁰⁾, which is generated randomly, be the initial estimate of the model parameters θ. Set the counter k and the parameter corresponding to the temperature β to 0 and β_min, respectively.

2. For each pair of the input and the output (x_t, y_t), compute

f i ( k ) ( y t | x t ) = P ( x t , y t , i | θ ( k ) ) β ∑ j = 1 M P ( x t , y t , j | θ ( k ) ) β , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aa0baaSqaaiabdMgaPbqaaiabcIcaOiabdUgaRjabcMcaPaaakiabcIcaOiabdMha5naaBaaaleaacqWG0baDaeqaaOGaeiiFaWhcbeGae8hEaG3aaSbaaSqaaiabdsha0bqabaGccqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabdcfaqjabcIcaOiab=Hha4naaBaaabaGaemiDaqhabeaacqGGSaalcqWG5bqEdaWgaaqaaiabdsha0bqabaGaeiilaWIaemyAaKMaeiiFaWhcciGae4hUde3aaWbaaeqabaGaeiikaGIaem4AaSMaeiykaKcaaiabcMcaPmaaCaaabeqaaiab+j7aIbaaaeaadaaeWaqaaiabdcfaqjabcIcaOiab=Hha4naaBaaabaGaemiDaqhabeaacqGGSaalcqWG5bqEdaWgaaqaaiabdsha0bqabaGaeiilaWIaemOAaOMaeiiFaWNae4hUde3aaWbaaeqabaGaeiikaGIaem4AaSMaeiykaKcaaiabcMcaPmaaCaaabeqaaiab+j7aIbaaaeaacqWGQbGAcqGH9aqpcqaIXaqmaeaacqWGnbqtaiabggHiLdaaaOGaeiilaWcaaa@6CE3@

where

P(x, y, i|θ) = P(i|θ) P(x|i, θ) P(y|x, i, θ),

P((i|θ) = α_i,

P((x|i|, θ) = N (x|m_i, Σ_i),

P ( y | x , i , θ ) = 1 2 π σ i 2 exp ⁡ [ − ( y − w i ⋅ x − b i ) 2 2 σ i 2 ] . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeiikaGIaemyEaKNaeiiFaWhcbeGae8hEaGNaeiilaWIaemyAaKMaeiilaWccciGae4hUdeNaeiykaKIaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaadaGcaaqaaiabikdaYiab+b8aWjab+n8aZnaaDaaabaGaemyAaKgabaGaeGOmaidaaaqabaaaaOGagiyzauMaeiiEaGNaeiiCaa3aamWaaeaacqGHsisljuaGdaWcaaqaaiabcIcaOiabdMha5jabgkHiTiab=Dha3naaBaaabaGaemyAaKgabeaacqGHflY1cqWF4baEcqGHsislcqWGIbGydaWgaaqaaiabdMgaPbqabaGaeiykaKYaaWbaaeqabaGaeGOmaidaaaqaaiabikdaYiab+n8aZnaaDaaabaGaemyAaKgabaGaeGOmaidaaaaaaOGaay5waiaaw2faaiabc6caUaaa@5F1F@

Then, form a function

Q β ( θ | θ ( k ) ) = ∑ t = 1 T ∑ i = 1 M f i ( k ) ( y t | x t ) log ⁡ P ( x t , y t , i | θ ) + log ⁡ P β ( θ ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeaabmqaaaqaaiabdgfarnaaBaaaleaaiiGacqWFYoGyaeqaaOGaeiikaGIae8hUdeNaeiiFaWNae8hUde3aaWbaaSqabeaacqGGOaakcqWGRbWAcqGGPaqkaaGccqGGPaqkaeaacqGH9aqpdaaeWbqaamaaqahabaGaemOzay2aa0baaSqaaiabdMgaPbqaaiabcIcaOiabdUgaRjabcMcaPaaakiabcIcaOiabdMha5naaBaaaleaacqWG0baDaeqaaOGaeiiFaWhcbeGae4hEaG3aaSbaaSqaaiabdsha0bqabaGccqGGPaqkcyGGSbaBcqGGVbWBcqGGNbWzcqWGqbaucqGGOaakcqGF4baEdaWgaaWcbaGaemiDaqhabeaakiabcYcaSiabdMha5naaBaaaleaacqWG0baDaeqaaOGaeiilaWIaemyAaKMaeiiFaWNae8hUdeNaeiykaKcaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGnbqta0GaeyyeIuoaaSqaaiabdsha0jabg2da9iabigdaXaqaaiabdsfaubqdcqGHris5aaGcbaGaey4kaSIagiiBaWMaei4Ba8Maei4zaCMaemiuaa1aaSbaaSqaaiab=j7aIbqabaGccqGGOaakcqWF4oqCcqGGPaqkcqGGSaalaaaaaa@76E5@

where P_β (θ) is a prior probability distribution. We utilize a priori knowledge about genetic networks using P_β (θ), as described below.

3. Find a new estimate θ^{(k + 1)} that maximizes the function Q_β.

4. k ← k + 1.

5. Repeat steps 2, 3 and 4 until convergence.

6. β ← β + β_add.

7. Stop if β > 1. Otherwise, return to step 2.

The parameters of the DAEM algorithm, β_min and β_add, were both set to 0.1 in this study.

Note that this algorithm estimates the parameter σi2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaacciGae83Wdm3aa0baaSqaaiabdMgaPbqaaiabikdaYaaaaaa@3017@ along with the other model parameters. Although the reduced NGnet model does not contain the parameter, σi2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaacciGae83Wdm3aa0baaSqaaiabdMgaPbqaaiabikdaYaaaaaa@3017@ we cannot derive the learning algorithm without estimating it.

Our method may infer multiple candidate networks due to the high degree-of-freedom of the reduced NGnet model and the pollution of the observed data by the noise. In order to increase the probability to obtain a reasonable network, we therefore utilize a priori knowledge about the genetic network in this study 9. Genetic networks are known to be sparsely connected 39. In order to utilize this knowledge, we use the following function as P_β (θ) given in the equation (14).

P β ( θ ) = 1 Z β [ exp ⁡ ( − γ T ∑ j ∈ A ∑ i = 1 M w i , j 2 ) ] β , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aaSbaaSqaaGGaciab=j7aIbqabaGccqGGOaakcqWF4oqCcqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabdQfaAnaaBaaabaGae8NSdigabeaaaaGcdaWadaqaaiGbcwgaLjabcIha4jabcchaWnaabmaabaGaeyOeI0Iae83SdCgcbiGae4hvaq1aaabuaeaadaaeWbqaaiabdEha3naaDaaaleaacqWGPbqAcqGGSaalcqWGQbGAaeaacqaIYaGmaaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemyta0eaniabggHiLdaaleaacqWGQbGAcqGHiiIZcqWGbbqqaeqaniabggHiLdaakiaawIcacaGLPaaaaiaawUfacaGLDbaadaahaaWcbeqaaiab=j7aIbaakiabcYcaSaaa@593D@

where Z_β is a normalization factor, w_i,j is the j-th component of the vector w_i, β is the parameter of the DAEM algorithm, T is the number of the observations, γ is a constant parameter, and A is a set of indices corresponding to genes that are assumed to unaffect the n-th gene. The set A is constructed as follows.

1. Let the set A be {1, ..., N}.

2. Choose I genes in ascending order of the value of ωm=∑i=1Mwi,m2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaacciGae8xYdC3aaSbaaSqaaiabd2gaTbqabaGccqGH9aqpdaaeWaqaaiabdEha3naaDaaaleaacqWGPbqAcqGGSaalcqWGTbqBaeaacqaIYaGmaaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemyta0eaniabggHiLdaaaa@3CE5@ where I is a parameter named the maximum indegree. The maximum indegree determines the maximum number of genes that directly affect the n-th gene.

3. From the set A, remove the indices corresponding to the genes selected in the previous step.

When the n-th gene is assumed to be unaffected by the m-th gene, this probability distribution forces the corresponding regression parameters, i.e., w_1,m, ..., w_M,m, down to zero. As mentioned in the Gradual reduction strategy section, even when these parameters are zero, the weak regulation of the n-th gene from the m-th gene remains in the model. However, the gradual reduction strategy should remove these unnecessary regulations from the model.