Abstract
Background
Embryonic stem cells (ESC) have the capacity to selfrenew and remain pluripotent, while continuously providing a source of a variety of differentiated cell types. Understanding what governs these properties at the molecular level is crucial for stem cell biology and its application to regenerative medicine. Of particular relevance is to elucidate those molecular interactions which govern the reprogramming of somatic cells into ESC. A computational approach can be used as a framework to explore the dynamics of a simplified network of the ESC with the aim to understand how stem cells differentiate and also how they can be reprogrammed from somatic cells.
Results
We propose a computational model of the embryonic stem cell network, in which a core set of transcription factors (TFs) interact with each other and are induced by external factors. A stochastic treatment of the network dynamics suggests that NANOG heterogeneity is the deciding factor for the stem cell fate. In particular, our results show that the decision of staying in the ground state or commitment to a differentiated state is fundamentally stochastic, and can be modulated by the addition of external factors (2i/3i media), which have the effect of reducing fluctuations in NANOG expression. Our model also hosts reprogramming of a committed cell into an ESC by overexpressing OCT4. In this context, we recapitulate the important experimental result that reprogramming efficiency peaks when OCT4 is overexpressed within a specific range of values.
Conclusions
We have demonstrated how a stochastic computational model based upon a simplified network of TFs in ESCs can elucidate several key observed dynamical features. It accounts for (i) the observed heterogeneity of key regulators, (ii) characterizes the ESC under certain external stimuli conditions and (iii) describes the occurrence of transitions from the ESC to the differentiated state. Furthermore, the model (iv) provides a framework for reprogramming from somatic cells and conveys an understanding of reprogramming efficiency as a function of OCT4 overexpression.
Keywords:
Stem cells; Heterogeneity; Stochasticity; Computational model; Differentiation; ReprogrammingBackground
Understanding the molecular networks which give rise to pluripotency in embryonic stem (ES) cells is crucial for among other things developing reprogramming strategies. Recent work has shed light on several key aspects of the underlying network and its interaction with external factors, in particular the chemical media which maintain the cells [1]. The current understanding is that ESCs occupy a multiplicity of substates, with stochastic transitions between them. One aim is to understand the molecular interactions that maintain cells in a pluripotent state, destabilize this state leading to commitment, as well as allow a return to the pluripotent state from a committed state. Given the substantial experimental efforts currently underway to understand these mechanisms, a computational systems biology approach seems like a way forward within which such questions could be formulated [26]. As in many other biomedical problem areas, a computational approach would here allow diverse experimental results to be absorbed into the formulation of the model, but more importantly, could serve as a hypothesis generator to test mechanisms through further experimentation. The recognition that states of a ES cell are read out by the gene expression of key regulators, has lead to a simple hypothesis regarding the pluripotent nature of the ESC [7]. An ES cell can be in a “ground state”, in which it is neutral to any developmental specification. However, it is possible for the cell to transition to a differentiated state. Here we explore the dynamics of a simplified network model representing key elements of ESC transcription factor and signaling regulators to suggest mechanisms for such a transition state picture.
At the heart of the pluripotency network lies the triad OCT4, SOX2 and NANOG [810], where OCT4 and SOX2 act together as a heterodimer regulating several genes including NANOG, OCT4 and SOX2 [11]. There are additional TFs that also impact pluripotency. The exact regulatory mechanisms in the network with impact on pluripotency remain to be fully understood. However, it appears that selfreinforcing mechanisms through feedback of these key regulators upon themselves seem to be instrumental. Interacting with these key components in mice are external factors like Leukemia Inhibiting Factor (LIF), which can substitute for feeders by activating the transcription factor STAT3 that inhibits ES differentiation [12,13]. Another factor, Bone Morphogenetic Protein (BMP4), has been shown to inhibit the differentiation proteins and thus can be used as a replacement for serum [14]. There are corresponding factors active in humans. The common media for maintaining stem cells in cultures is LIF plus serum or BMP4. It has been shown that serum/BMP4 can be replaced by small molecules which inhibit FGF4 receptor tyrosine kinases and the ERK cascade (2i/3i medium) [15]. The 2i/3i (two or three types of differentiation inhibiting molecules) medium is used successfully to maintain stem cells in vitro in combination with or without LIF.
Biochemical systems naturally exhibit stochastic fluctuations due to random interaction processes, gene transcription and translation as well as degradation. Recent studies have explored the role of stochastic fluctuations in a variety of organisms ranging from bacteria to mammalian cells [16,17]. In ESCs, it was shown that the expression of some transcription factors important for pluripotency are heterogeneous when cells are maintained in the “classical” environment i.e. LIF plus BMP4 or serum. Stochasticity or heterogeneity has been observed in key stem cell TFs such as NANOG [1820], REX1 [21], STELLA [22]. Based upon these observations, it appears that stem cells exist in a multitude of substates, where each substate represents a certain multidistribution of TF concentrations. In particular, NANOG shows more heterogeneity than OCT4 and SOX2 [18,20]. Cells expressing lower levels of NANOG are more prone to differentiate [18,23], thereby conferring a stochastic component to the ability of the cell to selfrenew. Hence, the state space of ESCs is intricately woven into the heterogeneous gene expression of some of the key regulators of the network.
Underlying the ability of NANOG to act as a “gatekeeper” of pluripotency [24], is the fact that OCT4SOX2 also induces FGF4, a differentiation promoting growth factor [7]. The ES cell requires OCT4 and SOX2 to maintain it in a pluripotent state, while at the same time pushing it towards differentiation. NANOG is thought to prevent differentiation, and hence when it reaches low levels, the probability to commit increases. How FGF4 fits into this network has so far not been computationally explored. Mouse ESCs can be maintained in a pluripotent state, through introduction of small molecule inhibitors. Ying et al. [15] discovered two different sets of small molecule inhibitors; 3i – FGF receptor inhibitor, Mitigen activated protein (MAP) kinase/ERK kinase  MEK inhibitor and GSK3 inhibitor, 2i – MEK inhibitor and a GSK3 inhibitor. Wray et al. [25] established that the expressions of NANOG and REX1 within the mouse ES cultures under 2i conditions were not heterogeneous i.e. only NANOG high or REX high are present, suggesting the existence of cells in a state that is intrinsically less fluctuating. This could be denoted a true “ground” state, which they suggested is an inherent stable pluripotency network governed by OCT4, SOX2 and NANOG, but, which is perturbed by Erk signaling acting through the FGF receptors.
It follows that a quantitative analysis of network dynamics could improve our understanding of the multiple states of the ESC. Previous purely deterministic studies have explored the dynamics of the OCT4SOX2NANOG regulatory network, as well as its role in determining the cell fate, i.e the final lineage: epiblast, trophectoderm and endoderm [26,27]. However, neither of these computational studies analyzed heterogeneity in NANOG expression. Kalmar et al. [20] suggested by stochastic modeling of a simplified stem cell network based upon observations, how NANOG fluctuations could make the stem cell state transition between multiple states. Their model involved feedbacks, both positive and negative between OCT4 and NANOG which lead to NANOG levels cycling between high and low levels as an excitable system. Subsequently Glauche et al. [28] further studied the nature of such stochastic transitions with two different model scenarios. In one model NANOG, which is induced by OCT4SOX2 can act as a bistable switch, and can transition between high and low levels. In the other model, which is based upon an activatorrepressor mechanism, NANOG can oscillate on a fixed limit cycle, and can recapitulate the observed heterogeneity in NANOG levels. Hence, several types of mechanisms could lead to NANOG heterogeneity. It is also suggested how NANOG can act as a gatekeeper by suppressing any differentiation signals which would ultimately make the cell transition into a differentiated cell. However, in [28], the signal to differentiate is external, and cells therefore cannot differentiate spontaneously as observed.
In this work we build upon these ideas by further analyzing how fluctuations in NANOG play a role in both allowing cells to transition between ES substates and then to finally exit irreversibly into a differentiated state. However, this occurs in a spontaneous fashion. Key to our approach, which is different from that of refs. [20,28], is the development of a selforganized network, in which the pluripotent network governed primarily by OCT4SOX2NANOG interacts with a differentiation pathway gene denoted by “G”. Candidates for G are for example GATA6 and SOX17. It is the stochastic dynamics of this network in which several types of feedbacks give rise to the observed stochastic stem cell fate. The noise therefore is internal to the network, with external stimuli controlling the strength of the fluctuations. Hence, stem cells can spontaneously change fate in accordance with observations. This also allows us to answer the second question as to how reprogramming can be simulated in our model. In [29] it has been shown that overexpression of OCT4 can lead to reprogramming a somatic cell to an ESC. However, the efficiency is maximal for the levels of OCT4 within a certain window [30]. Our model can reproduce this result, and we show how the interaction between OCT4, NANOG and the differentiation pathway gene G lead to this result.
Results and discussion
A simplified computational model of the ESC
Our simplified ESC network model considers a combination of positive and negative feedbacks between OCT4SOX2 and NANOG and G. With stochastic simulations we demonstrate the permissive nature of this selfcontained network – most cells retain pluripotency except for a fraction that get pushed towards differentiation. This model, is based upon an epigenetic effect by which OCT4 regulates NANOG, is also employed for reprogramming somatic cells into ESC.
The heterodimer OCT4SOX2 is known to serve as an activator of OCT4, SOX2 and NANOG [11]. As in [28], we simplify the interaction of OCT4 and SOX2 with NANOG as shown in Figure 1. The feedback between NANOG and OCT4SOX2 must be weak. Otherwise it would be inconsistent with low levels of NANOG and high levels of OCT4 and SOX2 as pointed out in [20,28]. Hence, we do not explicitly have NANOG inducing OCT4 and SOX2 in contrast to refs. [26,27]. To describe both the embryonic as well as the differentiated state, we include G in the circuit. One candidate for G is Sox17, which was shown to play a role in the control of differentiation of ESCs into extra embryonic endoderm [31]. SOX17 interferes with the selfrenewal program by inhibiting SOX2, OCT4 and NANOG [31]. Another candidate for G is GATA6, which is responsible for endoderm formation and also mutually antagonizes NANOG [27]. In [28], the authors assumed an external signal promoting differentiation. However, in our approach the gene G is regulated by the ESC circuit itself, and hence is part of the network which determines the cell fate. Our circuit also includes the differentiation promoting autocrine growth factor FGF4, which is shown schematically in Figure 1 to repress NANOG [3234]. It has been suggested that FGF4 acts upstream in the induction of differentiation [32,35]. The Fgf4 gene is expressed in mouse embryonic stem cells and only OCTSOX complexes are able to promote its transcriptional activation [36]. Inhibition of FGF4 along with GSK3 consolidates the ESC selfrenewal and pluripotency [15,25]. As in [27], we have assume mutual antagonism between NANOG and the differentiation gene G, as well as activation of G by OCT4SOX2. As shown in [27], overexpression of OCT4 could either lead to the establishment or loss of the stem cell fate, depending on the level of OCT4. We have also considered epigenetic regulation of NANOG as it has been suggested that OCT4 activates the histone demethylases Jmjd2c, which in turn exposes key pluripotent genes to regulation, among them NANOG [37]. Including this mechanism in the model results in sharpening the nonlinearity of the OCT4NANOG interaction.
Figure 1. The transcription factor interaction circuit along with external factors influences. The core network for the mutual and selfregulatory interactions between NANOG, OCT4SOX2 heterodimer, FGF4 and differentiation gene G. The dashed lines indicate the effect of external factors when cells are maintained in two different media, LIF+BMP4 and the 2i/3i respectively. The dynamical model is based upon the following (see Methods): OCT4SOX2 induces NANOG. NANOG dimerizes and regulates itself positively. NANOG represses G, which regulates itself positively. OCT4SOX2 induces G and the latter suppresses both NANOG and OCT4SOX2. LIF induces NANOG through Klf4. OCT4SOX2 induces FGF4, which suppresses NANOG. The 2i/3i medium suppresses FGF4.
Exploring the ground state of the ESC
Commitment – transition from the stem cell state to a differentiated state
We first compute the steady states of the system for different values of LIF using the deterministic rate equations (Eq. 1) for the circuit in Figure 1 with the parameters given in Table 1.
With dynamics resulting from the interactions between G, NANOG and OCT4SOX2, there are basically two states of the system: (i) the stem cell state, when OCT4SOX2 and NANOG are ON and G is OFF, and (ii) vice versa for the somatic state.
In the somatic state G is high and both OCT4SOX2 and NANOG are suppressed and hence OFF. This state remains even when increasing LIF since the model for the NANOG gene regulatory function is based upon a simplified epigenetic mechanism. For Nanog to be activated, the Nanog promoter must be bound by OCT4 along with any of its activators OCT4, NANOG (autoregulation), LIF. To be repressed, Nanog must be bound by OCT4 along with its repressors FGF4 and G.
Adding LIF, has no effect on NANOG if OCT4 is OFF, since LIF cannot access NANOG. However, if initially the cell is in a stem cell state with high OCT4SOX2, then OCT4SOX2 exposes NANOG, which allows LIF to induce NANOG. This in turn leads to suppression of G, which finally relieves the suppression on OCT4SOX2. These sequential negative interactions implement a positive feedback loop between NANOG and OCT4SOX2 (analogous to a different mechanism suggested in [28]). Additional file 1: Figure S1A displays the two states of the cell. The regulation of NANOG occurs through a feedforward loop [38], in which OCT4 directly activates NANOG and indirectly represses NANOG through FGF4. Additional file 1: Figure S1B shows that adding 2i/3i to the media leads to suppression of FGF4 (magenta), and hence relieves NANOG from repression.
Additional file 1. Figure S1. Steady state analysis of the stem cell circuit as functions of LIF and _{I3}concentrations. The dashed lines indicate unstable states. (A) The steady state values of NANOG, OCT4SOX2 and G as functions of LIF concentration. The plots display two states of the cell. (i) the stem cell state – high OCT4SOX2/NANOG and low G and (ii) the differentiated state with low OCT4SOX2/NANOG and high G (B) Similar graphs for small molecule differentiation inhibitors _{I3} concentration (here for 100). Although there are still two states, NANOG is still higher here than in the previous case. Shown also is the level of FGF4 (magenta), which is at ≃20, which is much lower than in the previous case.
Format: EPS Size: 678KB Download file
So far we have described a deterministic approach. However, chemical reactions are necessarily stochastic, and hence protein levels fluctuate in time [3941]. We assume that all of the stochasticity originates from within the network, i.e internal noise, as it is entirely due to random events of protein production and degradation for each of the molecular components with no external noise [42]. Since this noise is generated by the network itself, it could be regarded as “permissive”, which has been conjectured to be the source of hematopoietic commitment [43,44]. To study the effects of stochasticity, we used a Gillespie approach where the deterministic equations (see Methods) provide transition rates for a master equation. The latter is simulated by a Monte Carlo procedure to provide the time evolution of the concentration levels [45].
Stochastic dynamics under LIF conditions
In Figure 2A, we show the time series of OCT4SOX2 and NANOG concentrations for a stochastic simulation of Equation 1 with LIF=85 for the parameters in Table 1. Not shown are the G and FGF4 time series; G fluctuates around extremely low levels and FGF4 is similar to OCT4SOX2. Although OCT4SOX2 remains at a fairly high level, NANOG displays a larger fluctuation. The corresponding distributions obtained from several MonteCarlo runs (Figure 2C), show a tail for low NANOG levels with a peak at higher levels. OCT4SOX2 displays less heterogeneity. This recapitulates the observed NANOG heterogeneity [1820]. NANOG regulation occurs due to the competition between OCT4SOX2, which directly induces NANOG, and suppression by FGF4, which itself is induced by OCT4SOX2. This type of regulation implements an incoherent feedforward loop [38]. It is the delay between the noisy OCT4SOX2 induction of NANOG and its subsequent suppression through induction of FGF4, which itself is fluctuating, that creates the excess fluctuations observed for NANOG.
Figure 2. Time series and distributions of [OS], [N] and [G] concentrations for the stochastic dynamics of the gene regulatory network with inputs from LIFBMP4 and 2i/3i media with concentrationsLIF=85and I_{3=6}respectively. Variation of residence time in the ES state. (A) Time series of NANOG (red) and OCT4SOX2 (blue) when in LIFBMP4 medium show significant fluctuations of NANOG expression between high and low levels. (B) Time series of NANOG and OCT4SOX2 when in 2i/3i medium. (C) NANOG and OCT4SOX2 distributions when in LIFBMP4 medium. NANOG exhibits a wide distribution. (D) NANOG and OCT4SOX2 distributions when in 2i/3i medium. (E) Time series showing the differentiation process occurring when the cells are maintained in LIFBMP4 medium; NANOG (red), OCT4SOX2 (blue) and G (green). The upregulation of the differentiation gene G leads to an irreversible downregulation of OCT4SOX2 and NANOG. (F) The mean time that a stem cell remains in the ESC state increases with LIF concentration in the LIFBMP4 stem cell medium.
It has been shown that NANOG expression fluctuations reaching very low levels lead to irreversible commitment [18,24]. Hence we have built into our model the possibility of leaving the stem cell state by NANOG interactions with the differentiation gene G. Figure 2E shows NANOG fluctuations from a typical simulation. Should the NANOG expression hit a low level, G is relieved from the suppressive effects of NANOG, and is turned ON. Then G shuts OFF OCT4SOX2 and hence the pluripotent state is transformed into differentiated one. Before this transition occurs, OCT4SOX2 is at high levels but NANOG could be either high or low. It is only when NANOG reaches a very low level, through several consecutive degrading events in NANOG or OCT4SOX2, and/or coupled with increase in FGF4 or G, that the switch to a differentiated state occurs. The above results which suggest the role of increased heterogeneity in NANOG as responsible for the fate of the stem cell, were obtained for the parameter set displayed in Table 1. To show that these results are robust to changes in parameter values we computed the fluctuations in NANOG and compared them with the fluctuations in OCT4, using the Linear Noise Approximation (LNA) [4648] for a wide range of parameter sets (see Methods). In Figure 3, in each panel, we see the distribution of NANOG and OCT4 fluctuations for random parameter sets, for changes in parameters in increasing order (5%,15% and 50% respectively). For each distribution in parameter space, in the majority of the cases, we see that the highest fluctuations occur in NANOG expression. However, there are cases marked by the oval A, in the middle and last subplots, where NANOG and OCT4 fluctuations are extremely low. These represent those cases where the state of the cell is in the differentiated state, and hence the fluctuations in G would be highest. In the last subplot, the oval B represents those cases where the parameter sets corresponded to: weak NANOG autoregulation, strong suppression of NANOG by FGF4, G and weak suppression of OCT4 by G. This therefore resulted in higher noise in OCT4 than NANOG, since the latter was strongly suppressed, at the same time G was unable to fully turn OCT4 off. However, the above results indicate that NANOG in most cases experiences the highest fluctuations for a wide range of parameters, thereby supporting its role as the “gatekeeper” of the stem cell state.
Figure 3. NANOG and OCT4 standard deviations for multiple parameter sets. Each parameter set was generated by randomly sampling ±ߙ5%, 15% and 50% around each parameter in Table 1 for LIF=100. In each case the NANOG standard deviation (SD) was greater than the OCT4SOX2 SD, thereby suggesting that NANOG displays more heterogeneity. The oval A correspond to those parameter sets representing the differentiated state in which G is high and all other components are suppressed (hence low values of NANOG and OCT4). The points enclosed by the oval B represent parameter sets in which: NANOG is weakly regulated by itself; strongly suppressed by FGF4 and G. In addition OCT4 is weakly suppressed by G, which allows NANOG and G both to be expressed.
The pluripotent state has high levels of OCT4SOX2 which are less heterogeneous than those of NANOG. The continuum of NANOG values spans both, high and low NANOG values. In [28] the NANOG distribution resulted in a bimodal one, due to a bistable switchlike mechanism. A critical point is that NANOG functions as a gatekeeper regardless of its exact distribution – at low values it is unable to repress G and hence causes a transition. Our simulations are consistent with the experimental observation that although LIFBMP4 maintains ESCs, a low amount of differentiated cells are nevertheless produced. Simulations also show that increasing LIF improves the maintenance of stem cell cultures, i.e. the mean time that a cell, which is initialized as a stem cell, remains a stem cell increases with LIF value (see Figure 2F).
Stochastic dynamics under 2i/3i conditions
Recently, it was shown that ESCs can be maintained in 2i/3i [25] media, with the interesting result that heterogeneity in NANOG is lost. Our model assumes that the effect of small molecules in the 2i/3i medium is to suppress FGF4. This would relieve the suppression on NANOG. As it is shown in Additional file 1: Figure S1B, the system now exhibits a higher level of NANOG.
Figure 2B and Figure 2D show time series and distributions of OCT4SOX2 and NANOG concentrations under 2i/3i conditions. They both fluctuate at high levels, with lower NANOG heterogeneity. Although LIF is not present, we assume that the stem cell state was initialized with G low and NANOG and OCT4SOX2 high. Suppression of FGF4 leads to higher induction of NANOG and hence increased positive feedback between OCT4SOX2 and NANOG through G, which ensures that their levels remain high. Hence, our simulations agree with the experimental observation of loss of NANOG heterogeneity with cells cultured in 2i/3i media. In Additional file 2: Figure S2 we display the mean and standard deviation of NANOG fluctuations using the LNA (see Methods). The figure shows that increasing 2i/3i while increasing the mean levels of NANOG decrease its fluctuations, while still being higher than fluctuations in OCT4. As mentioned earlier, although OCT4SOX2 maintains pluripotency, it also induces FGF4, which pushes cells to differentiate. However, since FGF4 receptor signaling and GSK3 are inhibited in 2i/3i media, NANOG is not repressed (see Figure 1) and hence the NANOG high state is observed. This would be the true “ground state”, which requires no feeders or serum, except the small molecule inhibitors, which prevent differentiation as well as provide improved biosynthetic environment for cell growth.
Additional file 2. Figure S2. NANOG mean and standard deviation as a function of the concentration of 2i/3i (LIF=0) computed using the LNA. In the upper panel, the mean value of NANOG , and the lower panel shows the standard deviation of NANOG. Also indicated as a red dashed line is the standard deviation of OCT4 which does not vary with 2i/3i.
Format: EPS Size: 13KB Download file
Reprogramming – transition from somatic to iPS cells
Ectopic expression of the pluripotency transcription factors OCT4, SOX2, KLF4 enables the transition from somatic cells to iPS cells (ES like cells) [49]. One notes that NANOG is not required for reprogramming despite its hub role in the architecture. Reprogramming with only these three factors is inefficient. Hence, substantial efforts are made to overcome this drawback. Initially, overexpression of cMYC was used but the addition of this factor increased not only the efficiency of reprogramming but also the tumorigenicity of the cells [49]. Recently, it has been shown that cMYC can be replaced by GLIS1, which does not have the same tumorigenic effect [29]. However, the reprogramming process still remains inefficient and more understanding of the process on the molecular level is needed.
Our minimal dynamical model elucidates the reprogramming process when only OCT4, SOX2 and KLF4 are overexpressed and identifies the obstacles to overcome: The differentiation gene G antagonizes OCT4 and NANOG, and since it feeds back positively upon itself, once ON, it ensures that OCT4SOX2 and NANOG are OFF. When OCT4SOX2 is OFF, NANOG cannot be induced since OCT4 is unable to fulfill its epigenetic role of exposing the NANOG promoter for transcription. Hence, NANOG stays OFF. NANOG is also repressed by FGF4, which in this case would be low, since its inducer OCT4SOX2 is OFF. Hence, overexpression of OCT4 is the key.
Deterministic analysis
The parameter αin Equation 1 governs the OCT4 overexpression. In Figure 4A, the bifurcation diagram shows how the network components change with increasing αkeeping the other parameters fixed. Three regions can be identified, approximately given by:
Figure 4. Steady state analysis when OCT4SOX2 overexpression is varied. Time series concentrations of NANOG (red), OCT4SOX2 (blue) and G (green) obtained from stochastic simulation when reprogramming occurs. Reprogramming efficiency when overexpression of OCT4SOX2 is varied. (A)The steady state values of NANOG, OCT4SOX2 and G as functions of overexpression (α) of OCT4SOX2 for LIF=100 and I_{3}=10 respectively. The gene regulatory circuitry is initially in the somatic state with G high and NANOG and OCT4SOX2 low. The reprogramming occurs when G turns OFF while NANOG and OCT4SOX2 turn ON (at α≃0.001). At higher values of α(α≃0.2), there now exist two states of the system: the existing ES state and the differentiated state which occurs since G turns ON while OCT4SOX2 and NANOG switch to lower levels. (B) Time series of NANOG, OCT4SOX2 and G for one case when the reprogramming was successful, G turns OFF while NANOG and OCT4SOX2 turn ON for LIF=100 and I_{3} = 0.05 respectively. (C) The reprogramming efficiency for values of α varying from 0 to 0.4 and I_{3} taking 0 and 0.4 values, LIF=100
(i) α< 0.001, (ii) 0.001 <α> 0.2, (iii) α> 0.2. When going from (i) to (ii) OCT4 exposes NANOG and the high value of LIF (KLF4 overexpression [50,51]) induces NANOG, raising its expression to a sufficiently high level. The antagonism between NANOG and G, reduces G to low levels. This removes the G repression of OCT4SOX2, which further induces NANOG. This implements the positive feedback loop, which makes the transition from the somatic state to the stem cell state. We note that with OCT4SOX2 at high levels, FGF4 is induced, which does repress NANOG. However, this suppression is not strong enough to counter the induction from LIF, to switch it OFF. In addition, with 2i/3i media present, this effect can be significantly reduced.
As OCT4SOX2 overexpression is further increased and reaches α≃0.2 (iii) another bifurcation occurs. Now, a new state emerges – a differentiated state in which NANOG can be at low levels while G is high. This new state is possible because overexpression of OCT4 leads to a relatively slow increase in G, as it can also be induced by OCT4 in the model (it is slow because it is also suppressed by NANOG which itself is strongly induced by OCT4). Once OCT4 crosses a certain threshold, G can overcome NANOG, leading to a differentiated state.
Our deterministic model analysis indicates that the stem cell circuit activation must be conditioned by OCT4SOX2 overexpression. It is known that overexpression of OCT4 is mandatory for obtaining iPS cells in the laboratory [29]. However, too large an overexpression leads to a differentiated state [52,53]. Hence OCT4 has to be within a set of thresholds to achieve reprogramming [30]. It should be pointed out though that the latter result was obtained for human ESC.
Stochastic simulations
In Figure 4B we present the results of a corresponding stochastic simulation when the reprogramming was successful. Initially, the differentiation gene G has high expression whereas NANOG and OCT4SOX2 have low expression values – the circuit is in the somatic state. Overexpression of OCT4SOX2 with an optimal dosage α≃0.05 ensures that NANOG is strongly activated by OCT4 and hence the stem cell circuit switches to a stem cell state.
A major challenge is to increase the efficiency of the reprogramming process. We used our model to study the variation of reprogramming efficiency when the value of OCT4SOX2 overexpression (α) is varied. Figure 4C shows the success rates obtained from multiple Monte Carlo simulations where the value of α was varied in the [0,0.4] range. Our results suggest that the dosage of reprogramming transcription factors has an important impact on reprogramming efficiency. This has been experimentally shown in a study on optimization of iPS cells generation [30]. We identified an optimal range for OCT4 added expression value (α) to be [0.001,0.1]. Such a success interval should be considered when reprogramming is conducted under LIF and BMP4 medium conditions. If the values of reprogramming factor expression is lower than the values in the reprogramming success range then reprogramming does not take place as OCT4 is not significantly expressed and it does not strongly induce NANOG. If the values of OCT4 are too high then they would correspond to values that lead to endoderm differentiation and reprogramming is not successful. Figure 4C shows a 10fold decrease of the percentage of obtained iPS cells for values of αoutside success intervals.
Our results show that the stem cell medium where the somatic cells are maintained after transduction also plays an important role in reprogramming efficiency. When reprogramming is successful, the differentiation gene G is OFF while NANOG and OCT4SOX2 are at high values. The latter induces FGF4 which represses NANOG. The NANOG suppression by FGF4 influences negatively the reprogramming outcome. Thus, repression of FGF4 should have a positive impact on reprogramming efficiency. Indeed, when increasing 2i/3i concentration, an increase in iPS cell generation efficiency is observed (see Figure 4C). The percentage of iPS cells in this study represents the outcome from a minimalistic model and there must be additional factors not considered here which might modify the percentages. However, such factors would equally influence every scenario under consideration.
Nevertheless, our results demonstrate that setting the degrees of overexpression and choosing the iPS cell medium should be considered for optimizing reprogramming efficiency.
For completeness we performed similar analyses for a modified network topology without the differentiation gene G (see Additional file 3: Figure S3 and Additional file 4).
Additional file 3. Figure S3. Time series of [N] and [OS] for the stochastic dynamics of the alternative architecture with no differentiation gene G for LIF=100. A differentiation event; NANOG (red) and OCT4SOX2 (blue). The OCT4SOX2 expression is not lost in a differentiated cell in the model without G.
Format: EPS Size: 106KB Download file
Additional file 4. An architecture with no differentiation gene G.
Format: PDF Size: 23KB Download file
This file can be viewed with: Adobe Acrobat Reader
Conclusions
Our computational model of the transcriptional dynamics of the embryonic stem cell suggests mechanisms in the simplified network feedback structure which allow cells to make a stochastic decision to exit from a stem cell state to a differentiated one. Such an event is random and occurs due to the internal noise of network components. In particular, we explicitly showed how NANOG heterogeneity enables such transitions. NANOG integrates several noisy signals. OCT4 both directly activates NANOG, as well as suppresses it through FGF4. When NANOG falls below a certain threshold, G gets activated, leading to shutdown of NANOG and OCT4. FGF4 can be suppressed by the 2i/3i media which leads to reduction of NANOG heterogeneity (specifically at higher levels) and hence to stability of the stem cell state, i.e the “ground state”. Our model could explain how the absence of the 2i/3i media, can result in the experimentally observed “leakage” to differentiated cells even under ideal culture conditions, since stochastic transitions of NANOG to relatively low levels can occur, in this case. The spontaneous commitment picture emerging from our model studies is consistent with the “permissive” scenarios suggested in the context of hematopoiesis [43,44]. One might speculate that this ESC property allows cells to form tissues in the natural environment of the embryo through a low rate of regulated differentiation events. Finally we studied the reprogramming scenario of somatic cells due to OCT4 overexpression. Our model was able to explain why reprogramming efficiency is biphasic with respect to OCT4 levels. Once reprogramming occurs, the external stimuli provide optimal conditions to maintain for the stem cell state.
Our simplified model could be expanded as more links in this network are explored. For example, recent work [54,55] suggests that NANOG is epigenetically modified by Ezh2, and as discussed in [56], this could have interesting consequences for a model seeking to describe NANOG fluctuations. It is expected that future experiments will discover additional network componentsand external media implicated to govern stem cell fate and reprogramming, which could be included into our current model. We have not explored the consequences of external noise due to the environmental signals: LIF and 2i/3i, which will be explored in another work. One area of immediate interest is to immerse our single cell stochastic dynamics in a spatial context of growing and dividing cells [57] with the aim to understand how noise in gene expression couples with mechanics and cell fate in the living embryo.
Methods
Model ingredients
NANOG is induced by OCT4SOX2.
NANOG dimerizes and regulates itself positively [58].
The differentiation gene G positively autoregulates itself and is repressed by NANOG.
LIF induces NANOG, presumably through Klf4 via the Stat3 pathway.
G is induced by OCT4SOX2 and suppresses both NANOG and OCT4SOX2.
Even though FGF4 is a growth factor, it is induced by OCT4SOX2 [36]. Furthermore, FGF4 signaling was shown to suppress NANOG.
The 2i/3i medium has the effect of suppressing FGF4.
Network dynamics
For the circuit in Figure 1, we obtain the following set of differential equations from a thermodynamic approach [5963]. The equations describe the behavior of NANOG, OCT4SOX2, FGF4 and differentiation gene G (SOX17), with concentration levels denoted by [N], [OS], [FGF] and [G]. The concentrations of LIF and small molecules in the 2i/3i medium are denoted as LIF and I_{3}respectively.
In [37], an epigenetic effect was implicated by which OCT4 regulates the NANOG region by regulating the histone demethylases Jmjd2c. Here we implement such an effect for NANOG by assuming that that all TF’s which can bind to the Nanog promoter, do so only when the OCT4SOX2 heterodimer [OS] is first bound to it. This functional form is motivated by the need to have OCT4SOX2 make NANOG available for transcription. The parameter values used for the simulations are displayed in Table 1. Using the above deterministic equations we can obtain their steady state values as a function of the parameters. We also use the reaction rates from Equation 1 to write down a master equation, which has been simulated using the Gillespie algorithm to obtain the results in Figures 2 and 4.
We have performed Linear Noise Approximation analysis to prove the robustness of our results, as described below.
Robustness analysis for NANOG fluctuations using the LNA
A second order expansion of the master equation, obtained from the transition rates in Equation 1, is called as the linear noise approximation (LNA) [4648]. The assumption is that at steady state each network component fluctuates about its mean level, given by solving Equation 1, and is described by a Gaussian distribution. The fluctuations are described by a covariance matrix C. The diagonal components of C, describe the variances in each component, and the off diagonal components describe the cross correlations between the various species fluctuations. C is obtained at steady state by solving the Lyapunov equation given by,
where J is the Jacobian matrix, and D the effective diffusion matrix, which is obtained from Equation 1. We compute C, for a given parameter set and obtain the standard deviations for NANOG, OCT4 etc. This is then repeated for 500 randomly generated parameter sets. Each randomly generated parameter set is obtained by varying each of the parameters within a uniform distribution around the fiducial parameter set in Table 1 by ±5%, 15% and 50%. We did not vary the common degradation parameter γ, since the major effect would only lead to rescaling of time.
The bifurcation analysis was performed using JDesigner [64], Oscill8 [65]. Matlab (The Mathworks) was used to solve the differential equations, stochastic Gillespie simulations and the covariances of the network components using the LNA.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
VC, VO, CP conceived and developed the project. VC, VO developed the simulation and analysis programs. VC, VO, CP analyzed the data and wrote the manuscript. All authors read and approved the final manuscript.
Acknowledgements
This work was funded by the Swedish Foundation for Strategic Research Senior Individual Grant (A3 06:215) and the Swedish Research Council (Vr 62120083074). VC would like to acknowledge Prof Elliot M Meyerowitz for support. We thank an anonymous reviewer for constructive comments.
References

Nichols J, Smith A: The origin and identity of embryonic stem cells.
Development 2011, 138(1):38. PubMed Abstract  Publisher Full Text

Peltier J, Schaffer DV: Systems biology approaches to understanding stem cell fate choice.
Iet Syst Biol 2010, 4(1):111. PubMed Abstract  Publisher Full Text

MacArthur BD, Ma’ayan A, Lemischka IR: Systems biology of stem cell fate and cellular reprogramming.
Nat Rev Mol Cell Biol 2009, 10(10):672681. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Enver T, Pera M, Peterson C, Andrews PW: Stem Cell States, Fates, and the Rules of Attraction.
Cell Stem Cell 2009, 4(5):387397. PubMed Abstract  Publisher Full Text

Huang S: Systems biology of stem cells: three useful perspectives to help overcome the paradigm of linear pathways.
Philos Trans R Soc Lond B Biol Sci 2011, 366(1575):224759. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Artyomov MN Meissner A Chakraborty AK: A Model for Genetic and Epigenetic Regulatory Networks Identifies Rare Pathways for Transcription Factor Induced Pluripotency.
PLoS Comput Biol 2010, 6(5):e1000785. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Silva J, Smith A: Capturing pluripotency.
Cell 2008, 132(4):5326. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Mitsui K, Tokuzawa Y, Itoh H, Segawa K, Murakami M, Takahashi K, Maruyama M, Maeda M, Yamanaka S: The homeoprotein Nanog is required for maintenance of pluripotency in mouse epiblast and ES cells.
Cell 2003, 113(5):63142. PubMed Abstract  Publisher Full Text

Boyer L, Lee T, Cole M, Johnstone S, Levine S, Zucker J, Guenther M, Kumar R, Murray H, Jenner R, Gifford D, Melton D, Jaenisch R, Young R: Core transcriptional regulatory circuitry in human embryonic stem cells.
Cell 2005, 122(6):94756. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Niwa H: How is pluripotency determined and maintained?
Development 2007, 134(4):63546. PubMed Abstract  Publisher Full Text

Chew J, Loh Y, Zhang W, Chen X, Tam W, Yeap L, Li P, Ang Y, Lim B, Robson P, Ng H: Reciprocal transcriptional regulation of Pou5f1 and Sox2 via the Oct4/Sox2 complex in embryonic stem cells.
Mol Cell Biol 2005, 25(14):603146. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Smith A, Heath J, Donaldson D, Wong G, Moreau J, Stahl M, Rogers D: Inhibition of pluripotential embryonic stem cell differentiation by purified polypeptides.
Nature 1988, 336:688690. PubMed Abstract  Publisher Full Text

Smith A: Embryoderived stem cells: Of mice and men.
Annu Rev Cell Dev Biol 2001, 17:435462. PubMed Abstract  Publisher Full Text

Ying Q, Nichols J, Chambers I, Smith A: BMP induction of Id proteins suppresses differentiation and sustains embryonic stem cell selfrenewal in collaboration with STAT3.
Cell 2003, 115(3):28192. PubMed Abstract  Publisher Full Text

Ying Q, Wray J, Nichols J, BatlleMorera L, Doble B, Woodgett J, Cohen P, Smith A: The ground state of embryonic stem cell selfrenewal.

Balazsi G, van Oudenaarden A, Collins JJ: Cellular Decision Making and Biological Noise: From Microbes to Mammals.
Cell 2011, 144(6):910925. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Eldar A, Elowitz MB: Functional roles for noise in genetic circuits.
Nature 2010, 467(7312):16773. PubMed Abstract  Publisher Full Text

Chambers I, Silva J, Colby D, Nichols J, Nijmeijer B, Robertson M, Vrana J, Jones K, Grotewold L, Smith A: Nanog safeguards pluripotency and mediates germline development.
Nature 2007, 450:12301234. PubMed Abstract  Publisher Full Text

Singh AM, Hamazaki T, Hankowski KE, Terada N: A Heterogeneous Expression Pattern for Nanog in Embryonic Stem Cells.

Kalmar T, Lim C, Hayward P, MuozDescalzo S, Nichols J, GarciaOjalvo J, Martinez Arias A: Regulated Fluctuations in Nanog Expression Mediate Cell Fate Decisions in Embryonic Stem Cells.
PLoS Biol 2009, 7(7):e1000149. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Toyooka Y, Shimosato D, Murakami K, Takahashi K, Niwa H: Identification and characterization of subpopulations in undifferentiated ES cell culture.
Development 2008, 135:909918. PubMed Abstract  Publisher Full Text

Hayashi K, Chuva de SousaLopesS Tang F: Dynamic Equilibrium and Heterogeneity of Mouse Pluripotent Stem Cells with Distinct Functional and Epigenetic States.
Cell Stem Cell 2008, 3(4):391401. PubMed Abstract  Publisher Full Text

Osorno R, Chambers I: Transcription factor heterogeneity and epiblast pluripotency.
Phil Trans R Soc B 2011, 366:22302237. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Silva J, Nichols J, Theunissen T, Guo G, van Oosten A W, Barrandon O, Wray SJ, amd Yamanaka Chambers I: Nanog Is the Gateway to the Pluripotent Ground State.
Cell 2009, 138(4):722737. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Wray J, Kalkan T, Smith AG: The ground state of pluripotency.
Biochem Soc Trans 2010, 38:10271032. PubMed Abstract  Publisher Full Text

Chickarmane V, Troein C, Nuber UA, Sauro H, Peterson C: Transcriptional Dynamics of the Embryonic Stem Cell Switch.
PLoS Comput Biol 2006, 2(9):e123. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Chickarmane V, Peterson C: A computational model for understanding stem cell, trophectoderm and endoderm lineage determination.
PLoS One 2008, 3(10):e3478. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Glauche I, Herberg M, Roeder I: Nanog Variability and Pluripotency Regulation of Embryonic Stem Cells  Insights from a Mathematical Model Analysis.
PLoS One 2010, 5(6):e11238. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Maekawa M, Yamaguchi K, Nakamura T, Shibukawa R, Kodanaka I, Ichisaka T, Kawamura Y, Mochizuki H, Goshima N, Yamanaka S: Direct reprogramming of somatic cells is promoted by maternal transcription factor Glis1.
Nature 2011, 474:225229. PubMed Abstract  Publisher Full Text

Papapetrou EP, Tomishimac MJ, Chambers SM, Micae Y, Reed E, Menona J, Tabara V, Mog Q, Studer L, Sadelain M: Stoichiometric and temporal requirements of Oct4,Sox2, Klf4, and cMyc expression for efficient human iPSC induction and differentiation.

Niakan K, Ji H, Maehr R, Vokes S, Rodolfa K, Sherwood R, Yamaki M, Dimos J, Chen A, Melton D, McMahon A, Eggan K: Sox17 promotes differentiation in mouse embryonic stem cells by directly regulating extraembryonic gene expression and indirectly antagonizing selfrenewal.
Genes Dev 2010, 24:312326. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Kunath T, SabaElLeil MK, Almousailleakh M, Wray J, Meloche S, Smith A: FGF stimulation of the Erk1/2 signalling cascade triggers transition of pluripotent embryonic stem cells from selfrenewal to lineage commitment.
Development 2007, 134:28952902. PubMed Abstract  Publisher Full Text

Masui S: Pluripotency maintenance mechanism of embryonic stem cells and reprogramming.
Int J Hematol 2010, 91:360372. PubMed Abstract  Publisher Full Text

Lanner F, Rossant J: The role of FGF/Erk signaling in pluripotent cells.
Development 2010, 137(20):335160. PubMed Abstract  Publisher Full Text

Stavridis MP, Lunn JS, Collins BJ, Storey KG: A discrete period of FGFinduced Erk1/2 signalling is required for vertebrate neural specification.
Development 2007, 134:28892894. PubMed Abstract  Publisher Full Text

Yuan H, Corbi N, Basilico C, Dailey L: Developmentalspecific activity of the FGF4 enhancer requires the synergistic action of Sox2 and Oct3.
Genes Dev 1995, 9:26352645. PubMed Abstract  Publisher Full Text

Loh YH, Zhang W, Chen X, George J, Ng HH: Jmjd1a and Jmjd2c histone H3 Lys 9 demethylases regulate selfrenewal in embryonic stem cells.
Genes and Dev 2007, 21:25452557. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Mangan S, Alon U: Structure and function of the feedforward loop network motif.
Proc Natl Acad Sci U S A 2003, 100:1198011985. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Kaern M, Elston TC, Blake WJ, Collins JJ: Stochasticity in gene expression: from theories to phenotypes.
Nat Rev Genet 2005, 6(6):45164. PubMed Abstract  Publisher Full Text

Raj A, van Oudenaarden A: Nature, nurture, or chance: stochastic gene expression and its consequences.
Cell 2008, 135(2):21626. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Shahrezaei V, Swain PS: The stochastic nature of biochemical networks.
Curr Opin Biotechnol 2008, 19(4):36974. PubMed Abstract  Publisher Full Text

Swain PS, Elowitz MB, Siggia ED: Intrinsic and extrinsic contributions to stochasticity in gene expression.

Graf T, Enver T: Forcing cells to change lineages.
Nature 2009, 462:587594. PubMed Abstract  Publisher Full Text

Gillespie DT: Exact Stochastic Simulation of Coupled Chemical Reactions.
J Physical Chem 1977, 81(25):23402361. Publisher Full Text

Kampen NV: Stochastic Processes in Physics and Chemistry. Institute of Theoretical Physics, University of Utrecht,. North Holland, The Netherlands; 1992.

Elf J, Ehrenberg M: Fast Evaluation of Fluctuations in Biochemical Networks With the Linear Noise Approximation.
Genome Res 2003, 13(11):24752484. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Paulsson J: Summing up the noise in gene networks.
Nature 2004, 427:415418. PubMed Abstract  Publisher Full Text

Takahashi K, Yamanaka S: Induction of Pluripotent Stem Cells from Mouse Embryonic and Adult Fibroblast Cultures by Defined Factors.
Cell 2006, 126(4):663676. PubMed Abstract  Publisher Full Text

Bourillot P, Aksoy I, Schreiber V, Wianny F, Schulz H, Hummel O, Hubner N, Savatier P: Novel STAT3 target genes exert distinct roles in the inhibition of mesoderm and endoderm differentiation in cooperation with Nanog.
Stem Cells 2009, 27(8):176071. PubMed Abstract  Publisher Full Text

Bourillot PY, Savatier P: Krüppellike transcription factors and control of pluripotency.
BMC Biol 2010, 8:125. PubMed Abstract  BioMed Central Full Text  PubMed Central Full Text

Niwa H, Miyazaki J, Smith AG: Quantitative expression of Oct3/4 defines differentiation, dedifferentiation or selfrenewal of ES cells.
Nat Genet 2000, 24(4):372376. PubMed Abstract  Publisher Full Text

Okita K, Yamanaka S: Induced pluripotent stem cells: opportunities and challenges.
Phil Trans R Soc 2011, 366:21982207. Publisher Full Text

Villasante A, Piazzolla D, Li H, GomezLopez G, Djabali M, Serrano M: Epigenetic regulation of Nanog expression by Ezh2 in pluripotent stem cells.
Cell Cycle 2011, 10(9):148898. PubMed Abstract  Publisher Full Text

Hankowski KE, Terada N: An Ezh way to turn off Nanog.
Cell Cycle 2011, 10(14):22534. PubMed Abstract  Publisher Full Text

Herberg M, Roeder I: Epigenetic Nanog regulation and the role of functional heterogeneity.

Krupinski P, Chickarmane V, Peterson C: Simulating the Mammalian Blastocyst  Molecular and Mechanical Interactions Pattern the Embryo.
PLoS Comput Biol 2011, 7(5):e1001128. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Wang J, Levasseur D, Orkin S: Requirement of Nanog dimerization for stem cell selfrenewal and pluripotency.
Proc Natl Acad Sci U S A 2008, 105(17):632631. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Shea M, Ackers G: The OR control system of bacteriophage λ. A physical chemical model for gene regulation.
J Mol Biol 1985, 181:211230. PubMed Abstract  Publisher Full Text

Buchler N, Gerland U, Hwa T: On schemes of combinatorial transcription logic.
Proc Natl Acad Sci U S A 2003, 100:513641. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Bintu L, Buchler N, Garcia H, Gerland U, Hwa T, Kondev J, Phillips R: Transcriptional regulation by the numbers: models.
Curr Opin Genet Dev 2005, 15:116124. PubMed Abstract  Publisher Full Text

Hasty J, Isaacs F, Dolnik M, McMillen D, Collins J: Designer gene networks: Towards fundamental cellular control.

Narula J, Gottgens B, Igoshin OA, M SA: Modeling Reveals Bistability and LowPass Filtering in the Network Module Determining Blood Stem Cell Fate.
PLoS Comput Biol 2010, 6:e1000771. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

JDesigner [http://sourceforge.net/projects/jdesigner/ webcite]

Oscill8, Bifurcation Program [http://sourceforge.net/projects/oscill8/ webcite]