We give here a brief introduction to the CFTP framework (see 111213 for full proofs).

The central idea behind CFTP is to find a time in the past such that the whole state space is mapped to the same state at the present, for a given sequence of random numbers. When that occurs, the state at the present can be considered to be a sample of the stationary distribution. More formally, consider a Markov process defined by the transition rule X_t+1 = ϕ(X_t), where X_t ≡ x(t) is shorthand for the state of the system at time t. Any Markov chain X−T−∞≡{X−∞,...,X−T} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGamWiGhIfaynaaDaaaleaacqGHsislcqWGubavaeaacqGHsislcqGHEisPaaGccqGHHjIUcqGG7bWEcqWHybawdaWgaaWcbaGaeyOeI0IaeyOhIukabeaakiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabhIfaynaaBaaaleaacqGHsislcqWGubavaeqaaOGaeiyFa0haaa@4370@, started from t = -∞ will have reached stationarity at time t = -T. If a chain with an unknown value X_-T is continued to run until t = 0, it will attain a value X₀ = ϕ^T (X_-T), which also comes from the stationary distribution. The CFTP algorithm searches for a time -T such that the composite function ϕ^T (X_-T) has a unique image for all arguments X_-T. This implies that the chain started at -T is equivalent to a chain started from t = -∞, since it will reach the same state X₀ regardless of its value at t = -T. Hence the sample X₀ comes from the stationary distribution. Starting from the past and running into the present might seem counterintuitive and unnecessarily complicated. However, it is key for the algorithm to work and it can be shown that starting at t = 0 and coupling into the future will not guarantee that the samples are unbiased.

For large state spaces it is infeasible to monitor all initial conditions at time -T. However, this can be done efficiently if one can find a partial ordering over the state space that is preserved by the transition rule 12:

X t ≽ Y t ⇒ X s ≽ Y s , ∀ s ≥ t , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeabaaaabaGaeCiwaG1aaSbaaSqaaiabdsha0bqabaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaakiab=1Ni6kabhMfaznaaBaaaleaacqWG0baDaeqaaaGcbaGaeyO0H4nabaGaeCiwaG1aaSbaaSqaaiabdohaZbqabaGccqWF9jIUcqWHzbqwdaWgaaWcbaGaem4CamhabeaakiabcYcaSaqaaiabgcGiIiabdohaZjabgwMiZkabdsha0jabcYcaSaaaaaa@4B7D@

where ≽ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUaaa@341D@ denotes the partial order, i.e., a binary relation which is reflexive, anti-symmetric and transitive, although it does not necessarily satisfy total comparability. Under these conditions, the whole state space can be monitored by checking for the coalescence of coupled Markov chains started at the upper and lower extremes of the state space 1116.

Two Markov chains are said to be coupled if they use the same sequence of random numbers and the same transition rule but are started from different initial conditions. Coupled chains that meet at a time T_c are said to coalesce and will have identical states for t > T_c. A necessary (but not sufficient) condition for the preservation of the partial ordering is that the transition function is either monotone or anti-monotone:

M o n o t o n i c : X t ≽ Y t ⇒ ϕ ( X t ) ≽ ϕ ( Y t ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyta0Kaem4Ba8MaemOBa4Maem4Ba8MaemiDaqNaem4Ba8MaemOBa4MaemyAaKMaem4yamMaeiOoaOJaeCiwaG1aaSbaaSqaaiabdsha0bqabaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaakiab=1Ni6kabhMfaznaaBaaaleaacqWG0baDaeqaaOGaeyO0H4Taeqy1dyMaeiikaGIaeCiwaG1aaSbaaSqaaiabdsha0bqabaGccqGGPaqkcqWF9jIUcqaHvpGzcqGGOaakcqWHzbqwdaWgaaWcbaGaemiDaqhabeaakiabcMcaPaaa@5867@

A n t i - M o n o t o n i c : X t ≽ Y t ⇒ ϕ ( X t ) ≼ ϕ ( Y t ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyqaeKaemOBa4MaemiDaqNaemyAaKMaeeyla0Iaemyta0Kaem4Ba8MaemOBa4Maem4Ba8MaemiDaqNaem4Ba8MaemOBa4MaemyAaKMaem4yamMaeiOoaOJaeCiwaG1aaSbaaSqaaiabdsha0bqabaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaakiab=1Ni6kabhMfaznaaBaaaleaacqWG0baDaeqaaOGaeyO0H4Taeqy1dyMaeiikaGIaeCiwaG1aaSbaaSqaaiabdsha0bqabaGccqGGPaqkcqWF8jsUcqaHvpGzcqGGOaakcqWHzbqwdaWgaaWcbaGaemiDaqhabeaakiabcMcaPiabcYcaSaaa@5F62@

for coupled chains X and Y. If the partial order is preserved, we can monitor only the paths started at the 'extremes' of the state space, since all the paths in between remain bounded by them. We therefore define upper and a lower coupled Markov chains that enclose all other paths:

Lower path  ( L t − T )  started from L − T = 0 ^ Upper path  ( U t − T )  started from U − T = 1 ^ , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabiGaaaqaaiabbYeamjabb+gaVjabbEha3jabbwgaLjabbkhaYjabbccaGiabbchaWjabbggaHjabbsha0jabbIgaOjabbccaGiabcIcaOiabhYeamnaaDaaaleaacqWG0baDaeaacqGHsislcqWGubavaaGccqGGPaqkcqqGGaaicqqGZbWCcqqG0baDcqqGHbqycqqGYbGCcqqG0baDcqqGLbqzcqqGKbazcqqGGaaicqqGMbGzcqqGYbGCcqqGVbWBcqqGTbqBaeaacqWHmbatdaWgaaWcbaGaeyOeI0Iaemivaqfabeaakiabg2da9iqbhcdaWyaajaaabaGaeeyvauLaeeiCaaNaeeiCaaNaeeyzauMaeeOCaiNaeeiiaaIaeeiCaaNaeeyyaeMaeeiDaqNaeeiAaGMaeeiiaaIaeiikaGIaeCyvau1aa0baaSqaaiabdsha0bqaaiabgkHiTiabdsfaubaakiabcMcaPiabbccaGiabbohaZjabbsha0jabbggaHjabbkhaYjabbsha0jabbwgaLjabbsgaKjabbccaGiabbAgaMjabbkhaYjabb+gaVjabb2gaTbqaaiabhwfavnaaBaaaleaacqGHsislcqWGubavaeqaaOGaeyypa0JafCymaeJbaKaacqGGSaalaaaaaa@8244@

where 0ˆ≼x≼1ˆ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafCimaaJbaKaatqvzynuttLxBI9gBaeHbJ12C5fdmaGabaiab=XNi5kabhIha4jab=XNi5kqbhgdaXyaajaaaaa@3970@, ∀x.

The preservation of the partial order implies two important properties for coupled chains:

Sandwiching: all paths started between L and U will have coalesced by the time L and U do,

L t − T ≼ X t − T ≼ U t − T , t > − T . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabhYeamnaaDaaaleaacqWG0baDaeaacqGHsislcqWGubavaaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaakiab=XNi5kabhIfaynaaDaaaleaacqWG0baDaeaacqGHsislcqWGubavaaGccqWF8jsUcqWHvbqvdaqhaaWcbaGaemiDaqhabaGaeyOeI0IaemivaqfaaOGaeiilaWcabaGaemiDaqNaeyOpa4JaeyOeI0IaemivaqLaeiOla4caaaaa@4B98@

Funneling: all paths will get closer if they are started further back into the past,

L t − T ≼ L t − T − S ≼ U t − T − S ≼ U t − T , t > − T , S > 0. MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeGabiqaaaqaaiabhYeamnaaDaaaleaacqWG0baDaeaacqGHsislcqWGubavaaWeuvgwd10u51MyVXgaryWyTnxEXadaiqaakiab=XNi5kabhYeamnaaDaaaleaacqWG0baDaeaacqGHsislcqWGubavcqGHsislcqWGtbWuaaGccqWF8jsUcqWHvbqvdaqhaaWcbaGaemiDaqhabaGaeyOeI0IaemivaqLaeyOeI0Iaem4uamfaaOGae8hFIKRaeCyvau1aa0baaSqaaiabdsha0bqaaiabgkHiTiabdsfaubaakiabcYcaSaqaaiabdsha0jabg6da+iabgkHiTiabdsfaujabcYcaSiabdofatjabg6da+iabicdaWiabc6caUaaaaaa@5A97@

If the state space is unbounded from above, we need to use Kendall's DCFTP construction. DCFTP works by introducing a time-evolving dominating process D with known stationary distribution, which provides a random upper bound to the state space. The original process X can then be generated as an adapted functional F MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xmHyeaaa@36AF@ of the dominating process and a mark process M:

D t − T ≽ X t − T = F ( D s , M s ) , − T ≤ s ≤ t . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiraq0aa0baaSqaaiabdsha0bqaaiabgkHiTiabdsfaubaatqvzynuttLxBI9gBaeHbJ12C5fdmaGabaOGae8xFIORaeCiwaG1aa0baaSqaaiabdsha0bqaaiabgkHiTiabdsfaubaakiabg2da9mrtHrhAL1wy0L2yHvtyaeXbnfgDOvwBHrxAJfwnaGqbaiab+ftigjabcIcaOiabhseaenaaBaaaleaacqWGZbWCaeqaaOGaeiilaWIaeCyta00aaSbaaSqaaiabdohaZbqabaGccqGGPaqkcqGGSaalcqGHsislcqWGubavcqGHKjYOcqWGZbWCcqGHKjYOcqWG0baDcqGGUaGlaaa@5C6E@

The mark process generates a uniform random number each time D is changed. These marks are used to update the original process X according to the adapted functional (3) in a process that is equivalent to the direct simulation of X 12. Heuristically, the DCFTP scheme works as follows. Since the dominating process is started from the stationary distribution at t = -T, Dt−T MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiraq0aa0baaSqaaiabdsha0bqaaiabgkHiTiabdsfaubaaaaa@30A2@ is equivalent to a process started from t = -∞. By the funneling property, all chains from the original process started from t = -∞ will be beneath the dominating process: X−T−∞≼D−T−∞ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabgkHiTiabdsfaubqaaiabgkHiTiabg6HiLcaatqvzynuttLxBI9gBaeHbJ12C5fdmaGabaOGae8hFIKRaeCiraq0aa0baaSqaaiabgkHiTiabdsfaubqaaiabgkHiTiabg6HiLcaaaaa@3FC9@. If we set U_-T = D_-T and L_-T = 0^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafCimaaJbaKaaaaa@2CD2@ and check that these two extreme paths coalesce, then all chains started from t = -T map to the same state at t = 0, due to the sandwiching property. It then follows that X0−T MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiwaG1aa0baaSqaaiabicdaWaqaaiabgkHiTiabdsfaubaaaaa@3047@ is equivalent to X0−∞ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiwaG1aa0baaSqaaiabicdaWaqaaiabgkHiTiabg6HiLcaaaaa@3087@ and the sample comes from the stationary distribution of X, due to the equivalence of the adapted functional and the original process. Note that if D can be chosen to be a constant process equal to the maximal element of the state space, we obtain the CFTP algorithm 13.

These results are summarized in the following theorem for general DCFTP algorithms 1213:

Theorem 1 (DCFTP) Consider a stationary dominating process D, for which 0^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafCimaaJbaKaaaaa@2CD2@is an ergodic atom, and an associated random mark process M. Suppose that the processes L≼X≼U MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCitaW0euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF8jsUcqWHybawcqWF8jsUcqWHvbqvaaa@3990@are produced from D and M according to the adapted functional (3) so that the sandwiching and funneling properties (1)–(2) are fulfilled. Suppose further that X converges weakly to an invariant distribution π as t → ∞. Then L and U will coalesce almost surely in finite time and, if coalescence is achieved, L₀ = U₀ is a sample from the stationary distribution π.

Proof See 13.

This section presents briefly the classic Gillespie algorithm (SSA) for the exact simulation of the Master Equation of chemical reaction networks 7.

Definition 2 (Chemical reaction network) A system of chemical reactions N MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7eaaa@3763@is fully specified by the tuple N={S,R,Φ,ν} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7Kaeyypa0Jaei4EaSNae8NeXpLaeiilaWIae83gHiLaeiilaWIaeuOPdyKaeiilaWIaeqyVd4MaeiyFa0haaa@4429@, where S MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8NeXpfaaa@376D@ = {S₁,...,S_m} is a set of m different molecular species interacting through r reaction channels R MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83gHifaaa@36A5@ = {R₁,...,R_r}. Each reaction R_i is described by a stoichiometry vector ν_i, which gives the change in the number of molecules of all species when reaction R_i occurs, and a propensity function Φ_i(x), which gives the state-dependent probability that reaction R_i occurs. The state of the system is given by X_t ≡ x(t) = (x₁(t),...,x_m(t)) ∈ ℕ^m, where each component x_i(t) indicates the number of molecules of S_i at time t.

Under the assumption that the molecules are confined to a well-stirred volume and held at constant temperature, we can formulate a ME governing the evolution of the system 7:

d P ( x , t ) d t = ∑ i = 1 r [ Φ i ( x − ν i ) P ( x − ν i ; t ) − Φ i ( x ) P ( x ; t ) ] MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqWGKbazcqWGqbaucqGGOaakcqWH4baEcqGGSaalcqWG0baDcqGGPaqkaeaacqWGKbazcqWG0baDaaGccqGH9aqpdaaeWbqaamaadmaabaGaeuOPdy0aaSbaaSqaaiabdMgaPbqabaGccqGGOaakcqWH4baEcqGHsislcqaH9oGBdaWgaaWcbaGaemyAaKgabeaakiabcMcaPiabdcfaqjabcIcaOiabhIha4jabgkHiTiabe27aUnaaBaaaleaacqWGPbqAaeqaaOGaei4oaSJaemiDaqNaeiykaKIaeyOeI0IaeuOPdy0aaSbaaSqaaiabdMgaPbqabaGccqGGOaakcqWH4baEcqGGPaqkcqWGqbaucqGGOaakcqWH4baEcqGG7aWocqWG0baDcqGGPaqkaiaawUfacaGLDbaaaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabdkhaYbqdcqGHris5aaaa@64DA@

The ME is a conservation equation for the probability distribution and the right hand side accounts for the rate of change of the probability of finding the system in state x.

A general procedure to obtain exact realizations of Markov processes first suggested by Doob 17 was applied to chemical reactions by Gillespie in his celebrated Stochastic Simulation Algorithm 7:

Algorithm 3 (SSA) Given a chemical reaction network N={S,R,Φ,ν} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7Kaeyypa0Jaei4EaSNae8NeXpLaeiilaWIae83gHiLaeiilaWIaeuOPdyKaeiilaWIaeqyVd4MaeiyFa0haaa@4429@, as in Definition 2, with initial state Xt0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaaaaa@2FD5@and stopping time T_s:

   k ← 0

   loop

      k ← k + 1

      V_k, V′k MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOvayLbauaadaWgaaWcbaGaem4AaSgabeaaaaa@2EA1@ ~ U(0, 1)

      for i = 1 to r do

         θi←∑j=1iΦj(Xtk−1) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiUde3aaSbaaSqaaiabdMgaPbqabaGccqGHqgcRdaaeWaqaaiabfA6agnaaBaaaleaacqWGQbGAaeqaaOGaeiikaGIaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAcqGHsislcqaIXaqmaeqaaaWcbeaakiabcMcaPaWcbaGaemOAaOMaeyypa0JaeGymaedabaGaemyAaKganiabggHiLdaaaa@42CC@

      end for

      t_k ← t_k-1 - (1/θ_r) log V_k

      if t_k > T_s then

         return Xtk−1t0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabdsha0naaBaaameaacqWGRbWAcqGHsislcqaIXaqmaeqaaaWcbaGaemiDaq3aaSbaaWqaaiabicdaWaqabaaaaaaa@34B0@

      else

         Xtk←Xtk−1+νi,Rtk←Ri|θi−1θr<V′k<θiθr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaqGaaeaacqWHybawdaWgaaWcbaGaemiDaq3aaSbaaWqaaiabdUgaRbqabaaaleqaaOGaeyiKHWQaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAcqGHsislcqaIXaqmaeqaaaWcbeaakiabgUcaRiabe27aUnaaBaaaleaacqWGPbqAaeqaaOGaeiilaWIaemOuai1aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAaeqaaaWcbeaakiabgcziSkabdkfasnaaBaaaleaacqWGPbqAaeqaaaGccaGLiWoajuaGdaWcaaqaaiabeI7aXnaaBaaabaGaemyAaKMaeyOeI0IaeGymaedabeaaaeaacqaH4oqCdaWgaaqaaiabdkhaYbqabaaaaOGaeyipaWJafmOvayLbauaadaWgaaWcbaGaem4AaSgabeaakiabgYda8KqbaoaalaaabaGaeqiUde3aaSbaaeaacqWGPbqAaeqaaaqaaiabeI7aXnaaBaaabaGaemOCaihabeaaaaaaaa@5D31@

      end if

   end loop

A run of the SSA uses the uniform random numbers V, V' to generate a random sequence of reactions ℜ = {Rt1,...,Rtn} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaei4EaSNaemOuai1aaSbaaSqaaiabdsha0naaBaaameaacqaIXaqmaeqaaaWcbeaakiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabdkfasnaaBaaaleaacqWG0baDdaWgaaadbaGaemOBa4gabeaaaSqabaGccqGG9bqFaaa@3BAE@, taking place at the random transition times {t₁,...,t_n} such that t_n <T_s <t_n+1. The path XTst0≡{Xt0,Xt1,...,Xtn} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabdsfaunaaBaaameaacqWGZbWCaeqaaaWcbaGaemiDaq3aaSbaaWqaaiabicdaWaqabaaaaOGaeyyyIORaei4EaSNaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaakiabcYcaSiabhIfaynaaBaaaleaacqWG0baDdaWgaaadbaGaeGymaedabeaaaSqabaGccqGGSaalcqGGUaGlcqGGUaGlcqGGUaGlcqGGSaalcqWHybawdaWgaaWcbaGaemiDaq3aaSbaaWqaaiabd6gaUbqabaaaleqaaOGaeiyFa0haaa@4959@ is an exact stochastic realization of Eq. (4). Note that the sequence of reactions ℜ uniquely determines XTst0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabdsfaunaaBaaameaacqWGZbWCaeqaaaWcbaGaemiDaq3aaSbaaWqaaiabicdaWaqabaaaaaaa@32A3@. For convenience, we have committed a slight of abuse of notation when using real valued indices to denote the state Xtk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAaeqaaaWcbeaaaaa@3042@ and reaction Rtk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOuai1aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAaeqaaaWcbeaaaaa@3036@ taking place at time t_k.

Henceforth, we represent compactly the SSA Markov process implemented by Algorithm 3 as:

X T s t 0 = G SSA ( N , X t 0 , T s ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabdsfaunaaBaaameaacqWGZbWCaeqaaaWcbaGaemiDaq3aaSbaaWqaaiabicdaWaqabaaaaOGaeyypa0ZenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8NbXF0aaSbaaSqaaiabbofatjabbofatjabbgeabbqabaGccqGGOaakcqWFneVtcqGGSaalcqWHybawdaWgaaWcbaGaemiDaq3aaSbaaWqaaiabicdaWaqabaaaleqaaOGaeiilaWIaemivaq1aaSbaaSqaaiabdohaZbqabaGccqGGPaqkcqGGUaGlaaa@5021@

For an arbitrary initial state Xt0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaaaaa@2FD5@, repeated runs of the SSA will produce convergent estimates (in the Monte Carlo sense) of the distribution P(x, t|Xt0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaaaaa@2FD5@, t₀), ∀t ∈ [t₀, T_s] 8. However, if one is interested in the stationary distribution π, running the SSA repeatedly from different initial conditions for a finite time T_s does not guarantee that P(x, T_s) will converge to π, unless the starting points Xt0 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaaaaa@2FD5@ are themselves drawn from π. Our Perfect Sampling algorithm addresses this issue.

We use the Pareto dominance relation, frequently used in economics, which is defined componentwise:

Lemma 4 (Partial order) Given x, y ∈ ℕ^m, the relation x≽y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiEaG3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH5bqEaaa@3719@if x_i ≥ y_i, ∀i is a partial order.

Proof The proof follows trivially from the properties of natural numbers:

Reflexivity: ∀x_i ∈ ℕ, x_i ≥ x_i, whence x≽x MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiEaG3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH4baEaaa@3717@

Anti-symmetry: ∀x_i, y_i ∈ ℕ, if x_i ≥ y_i and y_i ≥ x_i then x_i = y_i. This means that x≽y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiEaG3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH5bqEaaa@3719@ and y≽x MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCyEaK3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH4baEaaa@3719@ implies x = y

Transitivity: ∀x_i, y_i, z_i ∈ ℕ, if x_i ≥ y_i and y_i ≥ z_i then x_i ≥ z_i. And the same property applies to the vectors: x≽y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiEaG3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH5bqEaaa@3719@ and y≽z MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCyEaK3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH6bGEaaa@371D@ implies x≽z MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiEaG3euvgwd10u51MyVXgaryWyTnxEXadaiqaacqWF9jIUcqWH6bGEaaa@371B@.   □

Consider a system of chemical reactions as given by Definition 2 with state vector x(t) ∈ ℕ^m. To guarantee the preservation of the Pareto partial order under the SSA Markov process (5), we restrict ourselves to a class of chemical networks with the following properties:

(a) all reactions are uni-molecular birth-death processes with non-zero propensities, i.e., each reaction R_i will only modify one species S_j by adding or subtracting one molecule. The reactions can be divided into two subsets:

Birth : R + = { R i | ν i = ( 0 , ... , 0 , + 1 , 0 , ... , 0 ) ⇒ x + ν i ≽ x } , Death : R − = { R i | ν i = ( 0 , ... , 0 , − 1 , 0 , ... , 0 ) ⇒ x + ν i ≼ x } MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabyWaaaaabaGaeeOqaiKaeeyAaKMaeeOCaiNaeeiDaqNaeeiAaGMaeiOoaOdabaaabaaabaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83gHi1aaWbaaSqabeaacqGHRaWkaaaakeaacqGH9aqpaeaacqGG7bWEcqWGsbGudaWgaaWcbaGaemyAaKgabeaakiabcYha8jabe27aUnaaBaaaleaacqWGPbqAaeqaaOGaeyypa0JaeiikaGIaeGimaaJaeiilaWIaeiOla4IaeiOla4IaeiOla4IaeiilaWIaeGimaaJaeiilaWIaey4kaSIaeGymaeJaeiilaWIaeGimaaJaeiilaWIaeiOla4IaeiOla4IaeiOla4IaeiilaWIaeGimaaJaeiykaKcabaaabaGaeyO0H4nabaGaeCiEaGNaey4kaSIaeqyVd42aaSbaaSqaaiabdMgaPbqabaWeuvgwd10u51MyVXgarCWyTnxEXadaiuaakiab+1Ni6kabhIha4jabc2ha9jabcYcaSaqaaiabbseaejabbwgaLjabbggaHjabbsha0jabbIgaOjabcQda6aqaaaqaaaqaaiab=TrisnaaCaaaleqabaGaeyOeI0caaaGcbaGaeyypa0dabaGaei4EaSNaemOuai1aaSbaaSqaaiabdMgaPbqabaGccqGG8baFcqaH9oGBdaWgaaWcbaGaemyAaKgabeaakiabg2da9iabcIcaOiabicdaWiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabicdaWiabcYcaSiabgkHiTiabigdaXiabcYcaSiabicdaWiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabicdaWiabcMcaPaqaaaqaaiabgkDiEdqaaiabhIha4jabgUcaRiabe27aUnaaBaaaleaacqWGPbqAaeqaaOGae4hFIKRaeCiEaGNaeiyFa0haaaaa@A442@

(b) the system must be chemically reversible, i.e., every reaction must be reversible leading to an irreducible Markov process

(c) all death reactions must be linear, i.e.

Φ_i = k_jx_j   for   R_i ≡ X_j → ∅

(d) all birth reactions must have (anti-)monotonic, sub-linear propensity functions, i.e., ∀i, j, ∀x: ∂Φ_i(x)/∂x_j does not change sign and Φ_i can be bounded by a linear function (or a constant).

As shown below, the last two assumptions are related to domination by a linear network which is required to have a stationary distribution.

Although assumptions (a) – (d) might appear restrictive, the specified class of reactions is generic and encompasses the standard equations used in the modelling of genetic and regulatory networks, the cellular circuits where stochasticity is most significant. Note that assumption (c) is not unrealistic for models of gene regulatory networks, in which linear death terms due to the cellular environment are prevalent. Birth reactions in these models are usually represented through nonlinear, uni-molecular (compound) rate laws that appear from quasi steady-state approximations. These functional forms have been shown to work well in the stochastic setting 18. Our own simulations confirmed that they provide a good approximation in a wide range of parameters (results not shown). These compound rate laws are the key components that encode the positive and negative feedback in gene regulation. Classic examples are the sigmoid functions:

Monod positive feedback : m ( x ) = k x α θ α + x α , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabb2eanjabb+gaVjabb6gaUjabb+gaVjabbsgaKjabbccaGiabbchaWjabb+gaVjabbohaZjabbMgaPjabbsha0jabbMgaPjabbAha2jabbwgaLjabbccaGiabbAgaMjabbwgaLjabbwgaLjabbsgaKjabbkgaIjabbggaHjabbogaJjabbUgaRjabcQda6aqaaiabd2gaTjabcIcaOiabdIha4jabcMcaPiabg2da9KqbaoaalaaabaGaem4AaSMaemiEaG3aaWbaaeqabaGaeqySdegaaaqaaiabeI7aXnaaCaaabeqaaiabeg7aHbaacqGHRaWkcqWG4baEdaahaaqabeaacqaHXoqyaaaaaiabcYcaSaaaaaa@5E35@

Hill negative feedback : h ( x ) = k θ α θ α + x α , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabbIeaijabbMgaPjabbYgaSjabbYgaSjabbccaGiabb6gaUjabbwgaLjabbEgaNjabbggaHjabbsha0jabbMgaPjabbAha2jabbwgaLjabbccaGiabbAgaMjabbwgaLjabbwgaLjabbsgaKjabbkgaIjabbggaHjabbogaJjabbUgaRjabcQda6aqaaiabdIgaOjabcIcaOiabdIha4jabcMcaPiabg2da9KqbaoaalaaabaGaem4AaSMaeqiUde3aaWbaaeqabaGaeqySdegaaaqaaiabeI7aXnaaCaaabeqaaiabeg7aHbaacqGHRaWkcqWG4baEdaahaaqabeaacqaHXoqyaaaaaiabcYcaSaaaaaa@5CB9@

which are sub-linear, (anti-)monotonic functions.

As stated above, assumption (d) is related to domination. In general, the state space of chemical reaction networks is unbounded from above; hence we must use the DCFTP construction, which requires a dominating process D with known stationary distribution. Fortunately, it has been shown that any network of linear first order reactions has a stationary distribution which is multivariate Poisson 19. Moreover, it can be shown that 0^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafCimaaJbaKaaaaa@2CD2@ is an ergodic atom for the multivariate Poisson, as assumed in Theorem 1 13. It then follows that a dominating process for any reaction network N={S,R,Φ,ν} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7Kaeyypa0Jaei4EaSNae8NeXpLaeiilaWIae83gHiLaeiilaWIaeuOPdyKaeiilaWIaeqyVd4MaeiyFa0haaa@4429@ composed of uni-molecular, sub-linear, (anti-)monotonic birth-death processes, as defined above, can be found by 'linearizing' the original network; that is, by constructing a linearized version of this network N˜={S,R,Φ˜,ν} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGaf8xdX7KbaGaacqGH9aqpcqGG7bWEcqWFse=ucqGGSaalcqWFBeIucqGGSaalcuqHMoGrgaacaiabcYcaSiabe27aUjabc2ha9baa@4447@, with the same reactions and compounds but with linear propensities Φ˜i MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafuOPdyKbaGaadaWgaaWcbaGaemyAaKgabeaaaaa@2EE5@(x) ≥ Φ_i(x), ∀x, ∀i that bound the original Φ from above. Under conditions of stability, the ME of N˜ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGaf8xdX7KbaGaaaaa@3772@ will have a stationary distribution π˜ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiWdaNbaGaaaaa@2DA1@, given by a multivariate Poisson that can be obtained by solving a system of linear equations 19. The existence of the stationary distribution of the dominating linear network N˜ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGaf8xdX7KbaGaaaaa@3772@ guarantees the existence of the stationary distribution for the original network of reversible, uni-molecular nonlinear reactions N MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7eaaa@3763@.

The dominating process D is defined as the stationary SSA process (5) of the linearized network N˜ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGaf8xdX7KbaGaaaaa@3772@ with initial state sampled from π˜ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiWdaNbaGaaaaa@2DA1@:

D T t 0 = G SSA ( N ˜ , D t 0 , T ) , D t 0 ~ π ˜ , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabhseaenaaDaaaleaacqWGubavaeaacqWG0baDdaWgaaadbaGaeGimaadabeaaaaGccqGH9aqpt0uy0HwzTfgDPnwy1egaryqtHrhAL1wy0L2yHvdaiqaacqWFge=rdaWgaaWcbaGaee4uamLaee4uamLaeeyqaeeabeaakiabcIcaOiqb=1q8ozaaiaGaeiilaWIaeCiraq0aaSbaaSqaaiabdsha0naaBaaameaacqaIWaamaeqaaaWcbeaakiabcYcaSiabdsfaujabcMcaPiabcYcaSaqaaiabhseaenaaBaaaleaacqWG0baDdaWgaaadbaGaeGimaadabeaaaSqabaGccqGG+bGFcuaHapaCgaacaiabcYcaSaaaaaa@54AF@

with the sequence of reactions ℜ˜={R˜t1,...} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWefv3ySLgzgjxyRrxDYbqeguuDJXwAKbIrYf2A0vNCaGabbiqb=XriszaaiaGaeyypa0Jaei4EaSNafmOuaiLbaGaadaWgaaWcbaGaemiDaq3aaSbaaWqaaiabigdaXaqabaaaleqaaOGaeiilaWIaeiOla4IaeiOla4IaeiOla4IaeiyFa0haaa@4356@.

It has been shown 13 that a correct realization of the original (nonlinear) SSA process X for a network N MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xdX7eaaa@3763@ with monotonic propensities can also be obtained through an adapted functional F MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xmHyeaaa@36AF@ defined in terms of the dominating process D and a random mark process M={Mt1,...} MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWefv3ySLgzgjxyRrxDYbqeguuDJXwAKbIrYf2A0vNCaGabbiab=Xa8njabg2da9iabcUha7jabd2eannaaBaaaleaacqWG0baDdaWgaaadbaGaeGymaedabeaaaSqabaGccqGGSaalcqGGUaGlcqGGUaGlcqGGUaGlcqGG9bqFaaa@43E8@ where Mtk MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyta00aaSbaaSqaaiabdsha0naaBaaameaacqWGRbWAaeqaaaWcbeaaaaa@302C@ ~ U(0, 1):

X T t 0 = F ( D T t 0 , M ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCiwaG1aa0baaSqaaiabdsfaubqaaiabdsha0naaBaaameaacqaIWaamaeqaaaaakiabg2da9mrtHrhAL1wy0L2yHvtyaeHbnfgDOvwBHrxAJfwnaGabaiab=ftigjabcIcaOiabhseaenaaDaaaleaacqWGubavaeaacqWG0baDdaWgaaadbaGaeGimaadabeaaaaGccqGGSaaltuuDJXwAKzKCHTgD1jharCqr1ngBPrgigjxyRrxDYbacfeGae4hdW3KaeiykaKIaeiOla4caaa@5243@

The update rule for F MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8xmHyeaaa@36AF@ uses the ratio of the monotonic propensity functions of the original and dominating processes as follows:

X