Institute for Systems Theory and Automatic Control, University of Stuttgart, Pfaffenwaldring 9, Stuttgart, Germany

Institute for Applied Analysis and Numerical Simulation, University of Stuttgart, Pfaffenwaldring 57, Stuttgart, Germany

Abstract

Background

Stochastic biochemical reaction networks are commonly modelled by the chemical master equation, and can be simulated as first order linear differential equations through a finite state projection. Due to the very high state space dimension of these equations, numerical simulations are computationally expensive. This is a particular problem for analysis tasks requiring repeated simulations for different parameter values. Such tasks are computationally expensive to the point of infeasibility with the chemical master equation.

Results

In this article, we apply parametric model order reduction techniques in order to construct accurate low-dimensional parametric models of the chemical master equation. These surrogate models can be used in various parametric analysis task such as identifiability analysis, parameter estimation, or sensitivity analysis. As biological examples, we consider two models for gene regulation networks, a bistable switch and a network displaying stochastic oscillations.

Conclusions

The results show that the parametric model reduction yields efficient models of stochastic biochemical reaction networks, and that these models can be useful for systems biology applications involving parametric analysis problems such as parameter exploration, optimization, estimation or sensitivity analysis.

Background

The chemical master equation (CME) is the most basic mathematical description of stochastic biomolecular reaction networks

Due to its infinite dimension, the CME is usually not directly solvable, not even with numerical methods. A recent breakthrough in the numerical treatment of the CME was the establishment of the finite state projection (FSP) method by Munsky and Khammash

Despite this progress, the direct simulation of the CME remains a computational bottleneck for common model analysis tasks in systems biology. It is especially problematic for tasks which require the repeated simulation of the model using different parameter values, for example identifiability analysis, parameter estimation, or model sensitivity analysis. Thereby, while a single or a few evaluations of a CME model with the FSP or other approaches may still be computationally feasible, the necessity of many repeated simulations will quickly render higher-level analysis tasks infeasible.

Mathematical methods that approximate the behaviour of a high-dimensional original model through a low-dimensional reduced model are a common way to deal with complex models. Especially for linear differential equations, model order reduction is a well established field and several methods to compute reduced order models are available

Fortunately, model reduction methods where parameters from the original model are retained as adjustable parameters also in the reduced model are now being developed. These methods allow to compute a reduced model which uses the same parameters as the original model, and where the reduced model can directly be simulated with any choice of parameter values

The purpose of this paper is to introduce the application of these parametric model reduction methods to finite-state approximations of the chemical master equation, and to show possible usage scenarios of such an approach. The structure is as follows. In the following section, we introduce some background and notation concerning the modelling of chemical reaction networks and parametric model order reduction. We also show how the parametric model order reduction methods can in fact be applied to the CME. Afterwards, we apply the reduction technique on two reaction network models and corresponding parametric analysis tasks.

Methods

We start with some preparatory background on the chemical master equation (CME) and parametric model order reduction. This serves in particular to fix the notation used throughout the remainder of the article. Then the application of parametric model order reduction to the CME is introduced.

The chemical master equation

The structure of a biochemical reaction network is characterized completely by the list of involved species, denoted as _{1}
_{2}…,_{
n
}, and the list of reactions, denoted as

where

Reversible reactions can always be written in the form (1) by splitting the forward and reverse path into two separate irreversible reactions.

For a stochastic network model, the variables of interest are the probabilities that the network is in any of the possible states which are characterized by the molecular copy numbers of the individual species _{1},_{2}…,_{
n
}. We denote the molecular copy number of _{
i
}by

for

The transitions from one state to another are determined by chemical reactions according to (1). The changes in the molecule numbers are described by the stoichiometric reaction vectors

To avoid needlessly complicated cases, we assume _{
j
}≠_{
k
}for

The probabilities of the network being in any of the possible states _{
j
} that reaction

where

Taking the possible transitions and the corresponding reaction propensities together yields the chemical master equation (CME), a linear differential equation where the variables are the probabilities that the system is in each of the possible molecular states

for
_{0},_{0}(

Despite being linear, the CME is hard to solve numerically. This is due to the problem that the state space is for most systems infinite-dimensional, since all possible states

As a more direct approach, Munsky and Khammash

where the ^{(i)} are the system states for which the probabilities are computed in the projected model. The underlying assumption is that the probabilities for other states will be very low on the time scale of interest—otherwise the FSP may not yield good approximations to the solution of the CME. In particular we assume the time interval of interest to be given by [0,^{(i)} in

The equation to be solved with the FSP approximation is

where

We will frequently omit the parameter dependence of the solution (and other parametric quantities). Hence the solution

Here, we consider in addition an output vector

with
_{
i
}= 1 if
_{
i
}= 0, with

i.e. ^{(1)},…,^{(d)}) with

The basic motivation for the model reduction presented here is that we are interested in parametric analysis of the model, where the model (10) has to be solved many times with different values for the parameters

Order reduction of parametric models

Model order reduction of parametric problems is a very active research field in systems theory, engineering and applied mathematics. We refer to

Here, we apply the reduction technique for parametric problems presented in
^{
T
}

The gain of computational efficiency in repeated simulations comes from a separation of the simulation task into a computationally expensive “offline” phase and a computationally cheap “online” phase. In the offline phase, suitable projection matrices

Decomposition in parametric and non-parametric part

The reduction technique assumes a separable parameter dependence of the full system matrices and the initial condition. This means, we assume that there exist a suitable small constant
_{
A
} such that

and similarly for the system matrix _{0}. We assume that

for _{
A
}. The resulting quantities

and similarly for _{
r0}(_{
r
}(

From the reduced state _{
r
}(

Basis generation

Different methods for the computation of the projection bases

The generation of projection matrices

Intuitively,
_{
max
}of

The POD-Greedy procedure which is given in the pseudo-code below, starts with an arbitrary orthonormal initial basis

function

1. _{0}

2. while

(a)

(b)

(c) _{
N + 1} :=

(d) _{
N + 1} := [_{
N
},_{
N + 1}]

(e)

3. end while

Note that the algorithm is implemented such that the simulation of the full model, yielding

For concluding the basis generation, we set ^{
T
}

A theoretical underpinning for the POD-Greedy algorithm has recently been provided by the analysis of convergence rates

Extensions of the POD-Greedy algorithm exist, e.g. allowing more than one mode per extension step, performing adaptive parameter and time-interval partitioning, or enabling training-set adaptation

Reduced models of the parametrized chemical master equation

In this section, we describe how to apply the reduction method for parametrized models presented in the previous section to FSP models for the chemical master equation.

As discussed in the previous section, the first step in the proposed reduction method is a decomposition of the

Hence, concerning the notation given before, we have _{
A
}=^{[q]} and coefficient functions
^{[q]} in this decomposition comes from just the transition propensities corresponding to reaction

More generally, such a decomposition is also possible if reaction rate propensities can be decomposed into the product of two terms, with the first term depending on parameters only, and the second term on molecule numbers only. This case is for example encountered when the temperature-dependance of the reaction rate constant is relevant, and the temperature _{0} are usually not depending on parameters in this framework, a decomposition of _{0} is not considered.

The situation is more difficult for reaction propensities involving for example rational terms with parameters in the denominator. The denominator parameters can not be included in the reduced order model by the decomposition outlined in (20) and (21). If variations in these parameters are however not relevant to the planned analysis, then they can be set to their nominal value, and the decomposition can directly be done as described above. Alternatively, approximation steps can be performed, such as Taylor series expansion or empirical interpolation

Results

In this section, we present the study of two example networks with the proposed model reduction method. With these examples, the applicability of the reduced modeling approach especially for analysis tasks requiring repeated simulations with different parameter values is illustrated. The first network is a bistable genetic toggle switch, where cells may switch randomly between two states, based on the model in

Parameter estimation in a genetic toggle switch model

Network description

The genetic toggle switch considered here is an ovarian follicle switch model from
_{1}, _{2}, representing the gene products. The network reactions are specified in Table

**Reaction**

**Stoichiometry ****
v
**

**Propensity ****
ν
**

List of reactions and reaction propensity functions for the follicle switch model

Production of _{1}

(1,0)^{T}

Degradation of _{1}

(−1,0)^{T}

_{1}
_{1}

Production of _{2}

(0,1)^{T}

Degradation of _{2}

(0,−1)^{T}

_{2}
_{2}

**
k
**

**
V
**

**
M
**

**
u
**

**
V
**

**
M
**

**
u
**

Parameter values for the follicle switch model in Table

4

75

25

75

25

In
^{(off)} = (0,0)^{T} and the other close to

In the study
^{(off)} to ^{(on)}. Therefore, the system is truncated to a rectangle

The next step is to apply the decomposition of the matrix _{1} and _{2}. Considering these two parameters as fixed quantities, the truncated CME for the follicle switch can be written as

where ^{[i]}, ^{2} × 151^{2} = 22801 × 22801.

As initial condition we choose a probability distributed over some lower states

For the parametric model reduction, we consider only variations in the parameters _{1} and _{2}. These influence both the steady state level of gene activity in the on-state as well as the switching kinetics and are thus of high biological significance in the model. Hence we set
^{7} which corresponds to a time range of approximately 19 years, i.e. about three times the half-life time of the off-state estimated in

Some state plots from the simulation of the full model are shown in Figure
_{1} and _{2} to the on state with high values. The parameter influence is mainly reflected in the speed of the transition: for the parameter vector (_{1}, _{2}) = (0.005, 0.02) in the lower row, most of the probability is already arranged around the on-state at the end of the simulation time. In contrast, for the parameter vector (_{1}, _{2}) = (0.05, 0.005) in the upper row, a significant portion of the probability is still located around the off-state at this time point. Also, the transition paths are different: in the first case, the values for _{2} are lower than the values for _{1}during the transition, while in the second case, this relation is reversed.

Illustration of solution snapshots of the switch model

**Illustration of solution snapshots of the switch model.** Illustration of some solution snapshots _{1},_{2}) = (0.05,0.005) (upper row) and (_{1},_{2}) = (0.005,0.02) (lower row) at times ^{5}, 5 · 10^{6}, and 1 · 10^{7}from left to right.

As typical simulation time for a single trajectory of the full system, we obtain 98.2 seconds on a IBM Lenovo 2.53 GHz Dual Core Laptop.

Basis generation

We generated a reduced basis with the POD-Greedy algorithm, where the training set was chosen as the vertices of a mesh with 9^{2} logarithmically equidistant parameter values over the parameter domain
_{0} = 1 and

The POD-Greedy algorithm produces a basis of 33 vectors and the overall computation of the reduced basis takes 7.9 hours, the dominating computation time being spent in the error evaluations and POD computations. Some of the resulting orthonormal basis vectors are illustrated in Figure

Basis vectors for the switch model

**Basis vectors for the switch model.** Illustration of the first eight basis vectors for the switch model generated by the POD-Greedy algorithm.

Results of the POD-Greedy algorithm for the switch model

**Results of the POD-Greedy algorithm for the switch model.** Illustration of the error decay during the POD-Greedy algorithm (left) applied to the switch model and the selected parameters (right) being a small subset of the 81 training parameter points.

The final reduced model of dimension 33 can then be simulated in 0.135 seconds, corresponding to a computational speedup factor of more than 700.

Parameter estimation

We exemplify a possible application of the reduced order model in parameter estimation, where we assume that a distorted output _{1}] is available from population-averaged measurements. The task is to estimate the parameter values _{1} and _{2} from such a noisy measurement.

The reference parameter is
_{
meas
}(_{
ref
})(1 + 0.05_{
ref
}) and the noisy signal _{
meas
}(

Parametric analysis for the reduced switch model

**Parametric analysis for the reduced switch model.** Application of parametric reduced models for parametric analysis: Illustration of the clean and noisy signals _{ref}) and _{meas}(

We want to recover the values of the parameters _{1} and _{2} based on fitting the reduced parametric model’s output
_{
meas
}(

and estimate the parameters by

In such an optimization problem, typically many forward simulations are required for adjusting

In order to gain a deeper insight into the optimization problem (25), we plot the values of the error functional ^{2} = 441 trajectories is realized in less than one minute. This would be a significant computational effort when using a non-reduced model.

From the cost function plot, we observe a narrow area of parameters which seem to produce a similar output as the reference parameter _{
ref
}. This shows that the two model parameters are not simultaneously identifiable from the considered output, and indicates that there may exist a functional dependence between the parameters _{1} and _{2} such that the model yields similar outputs

Assuming a functional dependence of _{1}and _{2}we now consider the 1-dimensional optimization problem along the line _{2} =_{2,ref
} = 0.01. We would like to recover _{1}from the optimization problem. The corresponding value of the cost function is _{
ref
}) = 3330.68, indicating a significant contribution of the noise. This restricted optimization problem is well conditioned and the optimization with a standard active set algorithm by MATLAB’s command _{
est
} := (_{1,est
},0.01) with _{1,est
} = 0.0100204, using 27 evaluations of the cost function. This accounts to a relative error in the _{1}value of 0.204%, hence excellent recovery. We refrain from plotting the recovered output
_{
est
}) = 3329.56 implies _{
est
}) <_{
ref
}), which may stem from a slight approximation error in the reduced model or from the effects of the measurement noise.

The right plot in Figure

Sensitivity analysis in a stochastic oscillator

Network description

The second case study is built on a genetic oscillator model showing stochastic resonance, which was presented in
_{1}, _{2}, representing the gene products. The network reactions are specified in Table
_{1} and _{2}, coming from a Langevin approximation to the stochastic dynamics

**Reaction**

**Stoichiometry ****
v
**

**Propensity ****
ν
**

List of reactions and reaction propensity functions for the oscillator model adopted from

Production of _{1}

(1,0)^{T}

Degradation of _{1}

(−1,0)^{T}

_{3}
_{1}

Production of _{2}

(0,1)^{T}

Degradation of _{2}

(0,−1)^{T}

_{7}
_{2}

**
k
**

**
k
**

**
k
**

**
k
**

**
k
**

**
k
**

**
k
**

**
s
**

Parameter values for the oscillator model from Table

0.2

6.5

10

The network model in Table

The system is truncated to the rectangle

Similarly as in the switch example, the reaction propensity expressions contain rational terms in the parameters _{2}, and _{6}. These three cannot be decomposed directly, so we do the decomposition described in the methods section for the other five parameters only. With this decomposition, the truncated CME for the genetic oscillator can be written as

where ^{[i]}, ^{2} × 301^{2} = 90601 × 90601. The initial condition for (26) is chosen as a uniform distribution over the rectangle {0,…,50} × {0,…,50}:

The time scale of interest for the model in (26) is for 0 ≤

Some state plots are given in Figure
_{4}on the amplitude of the oscillations. The simulation time for the detailed model was in average 7.3 minutes on a Dell desktop computer with 3.2 GHz dual-core Intel 4 processor and 1 GB RAM, without including the computation time for the construction of the state transition matrix

Illustration of solution snapshots of the oscillator model

**Illustration of solution snapshots of the oscillator model.** Illustration of some solution snapshots _{4} = 15 (upper row) and _{4} = 30 (lower row) at times

Basis generation

For the basis generation, the parameter _{4} was assumed to vary within the interval [10, 100]. A reduced basis with the POD-Greedy algorithm was computed from a training set of 30 logarithmically equidistant parameters over the parameter domain (Figure
_{1} :=_{0}.

Parametric analysis results for the oscillator model

**Parametric analysis results for the oscillator model.** Sensitivity analysis of oscillation amplitude over a parameter interval. Blue line shows oscillatory amplitude over the parameter _{4} predicted from the reduced model. Red dots are validation results from a simulation of the original model. Triangles on the parameter axis indicate parameter values which were used in the construction of the reduced basis.

The POD-Greedy algorithm produces a basis of 109 vectors, with an overall computation time of 16.5 hours on the hardware as in the previous subsection. The first 20 basis vectors are shown in Figure
_{4}. The error decay curve is shown in Figure

Basis vectors for the oscillator model

**Basis vectors for the oscillator model.** First 20 basis vectors for the oscillator model.

Results of the POD-Greedy algorithm for the oscillator model

**Results of the POD-Greedy algorithm for the oscillator model.** Error decay curve for the oscillator model.

With the reduced basis

with
_{4} has been varied in the reduction process, the other parameters are no longer present as parameters in the reduced model, but just take their nominal values. While the same basis

Sensitivity analysis of the oscillation amplitude

As an application of the reduced order parametric model obtained in the previous section, we study the variations of oscillatory amplitude over a parameter range. Specifically, we consider 200 equally spaced values for the parameter _{4} in the interval [12, 40] and compute the probability that the amount of _{2} is larger than 100:

with _{4}. Due to the significant time savings from the reduced model, this sensitivity curve can be computed with a high resolution.

To evaluate the quality of the reduced model, we also computed the probability (29) using the original model (26) at two points within the considered interval for the parameter _{4}. As shown in Figure

Conclusions

In this paper, we have introduced the application of parametric model reduction methods to finite-state approximations of the chemical master equation. We have also presented two case studies where these methods are applied to CME models of different networks in order to make parametric analysis tasks computationally efficient. By this, it has become clear that parametric model reduction methods are a very useful tool for the analysis of stochastic biochemical reaction network described by the CME.

Especially analysis tasks where many repeated simulations of a network with different parameter values are required can profit significantly from parametric model reduction. This includes for example sensitivity analysis or parameter optimization tasks such as identifiability analysis or estimation. Moreover, the significant speedup of the simulation for the reduced model allows an interactive exploration of the network’s dynamics within the parameter space within a suitable graphical user interface.

This contribution is just a first step in the application of parametric model reduction methods to the CME. One particularly important aspect that we have not discussed here is the computation of error estimates for certifying that the simulation output of the reduced model is within some tolerance of the corresponding simulation output of the original model. To maintain computational efficiency, the error estimation should be done without actually simulating the original model. Error estimation methods have been developed for parametric model reduction of generic models

Competing interests

Both authors declare that they have no competing interests.

Authors contributions

SW and BH conceived of the study, performed the study, and wrote the manuscript. Both authors read and approved the final manuscript.

Acknowledgements

We thank Wolfgang Halter for programming support in the oscillator case study. The authors would like to thank the German Research Foundation (DFG) for financial support of the project within the Cluster of Excellence in Simulation Technology at the University of Stuttgart. BH also acknowledges the Baden-Württemberg Stiftung gGmbH for funding. This work was also supported by the German Research Foundation (DFG) within the funding programme Open Access Publishing.