BioProcess Engineering Group, IIM-CSIC, Eduardo Cabello, 6, 36208 Vigo, Spain

Abstract

Background

Systems biology allows the analysis of biological systems behavior under different conditions through

This work addresses the numerical solution of such dynamic optimization problems for spatially distributed biological systems. The usual nonlinear and large scale nature of the mathematical models related to this class of systems and the presence of constraints on the optimization problems, impose a number of difficulties, such as the presence of suboptimal solutions, which call for robust and efficient numerical techniques.

Results

Here, the use of a control vector parameterization approach combined with efficient and robust hybrid global optimization methods and a reduced order model methodology is proposed. The capabilities of this strategy are illustrated considering the solution of a two challenging problems: bacterial chemotaxis and the FitzHugh-Nagumo model.

Conclusions

In the process of chemotaxis the objective was to efficiently compute the time-varying optimal concentration of chemotractant in one of the spatial boundaries in order to achieve predefined cell distribution profiles. Results are in agreement with those previously published in the literature. The FitzHugh-Nagumo problem is also efficiently solved and it illustrates very well how dynamic optimization may be used to force a system to evolve from an undesired to a desired pattern with a reduced number of actuators. The presented methodology can be used for the efficient dynamic optimization of generic distributed biological systems.

Background

Living organisms can not be understood by analyzing individual components but analyzing the interactions among those components

In recent years the use of optimization techniques for the purpose of modeling has attracted substantial attention. In particular, mathematical optimization is the underlying hypothesis for model development in, for example, flux balance analysis

Despite the success of modeling efforts in systems biology, the truth is that only in few occasions those models have been used to design or to optimize desired biological behaviors. This may be explained by the difficulty on formulating and solving those problems but also in the limited number of software tools that may be used for that purpose

It should be noted, however, that many biological systems of interest are being modelled by sets of partial differential equations (PDE). This is particularly the case of reaction diffusion waves in biology (see the recent review by

The use of global optimization techniques provides guarantees, at least in a probabilistic sense, of arriving to the global solution. Unfortunately the price to pay is the number of cost function evaluations and the associated computational cost, which increase exponentially with the number of decision variables. This aspect is particularly critical for PDE systems as they are usually solved with spatial discretization techniques (e.g. finite element or the finite differences methods) and the result is a large scale dynamic system whose simulation may take from several seconds to hours.

In this concern, the use of surrogate models has been proposed as the alternative to reduce total computation times. The most promising techniques based on kriging or radial basis functions have been incorporated to global optimization solvers

This work presents the application of hybrid optimization techniques for the solution of complex dynamic optimization problems related to biological applications. Particular emphasis is paid to the efficiency and robustness of the proposed methodologies. In this regard, the use of a hybrid global-local methodology together with a control refining technique is proposed. In addition, the POD technique is used to reduce the dimensionality (and thus the computational effort) of the original distributed (full scale) models.

To illustrate the usage and advantages of the proposed techniques two challenging case studies will be considered. The first is related to bacterial chemotaxis and considers the achievement of two different objectives as formulated in

Methods

Dynamic optimization problem formulation

Dynamic optimization, also called open loop optimal control (OCP), considers the computation of a set of time-dependent operating conditions (usually called controls) which optimize a certain performance index of the dynamic behavior of the biological system, subject to a set of constraints. This problem can be mathematically formulated as follows: **u**( _{0},_{
f
}]

where

A given number of constraints must be considered when solving optimal control problem (1). These may be classified in three main groups:

· the system dynamics that, for the general case of distributed process systems, can be represented as a set of partial and ordinary differential equations (PDEs) of the form:

with ∇ being the gradient operator and **x**, **y**, **u**) and **x**, **y**, **u**) two given (possibly nonlinear) functions which may represent for instance chemical reactions. This system must be completed with appropriate initial and boundary conditions which, for the general case, read as follows:

where **n** is a unit vector pointing outwards the boundary
**x **in the surrounding media, Robin boundary conditions are recovered.

· the bounds for the control variables:

· and possibly other equality or inequality constraints, which must be satisfied over the entire process time (path constraints) or at specific times (point constraints), being a particular case of the later the final time constraints which must be satisfied at final time. These constraints can be expressed as:

where _{
k
} is a time point, being the final time _{
f
}, a particular case.

Numerical methods

Numerical methods for the simulation

Many biological systems of interest exhibit a nonlinear dynamic behavior which makes the analytical solution of models representing such systems rather complicated, if not impossible, for most of the realistic situations. In addition to nonlinearity, these processes may present a spatially distributed nature. As a consequence they must be described using PDEs which, in turns, makes the analytical approach even more difficult. Numerical techniques must be, therefore, employed to solve the model equations.

Most of numerical methods employed for solving PDEs, in particular those employed in this work, belong to the family of ^{a}

Depending on the selection of the _{
i
}(

The underlying idea is to discretize the domain of interest into a (usually large) number

Probably the most widely used approaches for this transformation are the finite difference and the finite element methods. The reader interested on an extensive description of these techniques is referred to the literature

However it must be highlighted that in many biological models, especially those in 2D and 3D, the number of discretization points (

Different techniques like the eigenfunctions obtained from the Laplacian operator, Chevyshev or Legendre polynomials, among others have been considered over the last decades - see
_{
i
}(

where _{
i
}corresponds with the eigenvalue associated with each global eigenfunction _{
i
}. The kernel ^{
′
}) in equation (9) corresponds with the two point spatial correlation function, defined as follows:

with _{
j
}) denoting the value of the field at each instant _{
j
}and the summation extends over a sufficiently rich collection of uncorrelated snapshots at

The dissipative nature of this kind of systems makes that the eigenvalues obtained from Eqn (9) can be ordered so that _{
i
}≤_{
j
} for _{
A
}=_{1}
_{2},…,_{
N
} which captures the relevant features of the system

In order to compute the time dependent coefficients in Eqn (8), the original PDE system (2) is projected onto each element of the POD basis set. In this particular case, such projection is carried out by multiplying the original PDE by each _{
i
}and integrating the result over the spatial domain, this is:

Substituting the Fourier series approximation (8) into Eqn (12) leads to:

The basis functions obtained from (9) are orthogonal and can be normalized so that:

Therefore, Eqn (13) can be rewritten as:

where _{
i
} is a row vector of the form
_{A} corresponds with the following column vector

where _{
A
}= [_{1};_{2};…;_{
N
}],
_{
A
}are known from Eq (9) while time dependent coefficients are computed by solving Eq (15), therefore the approximation of the original field _{A} can be increased to approximate the original state

Dynamic optimization methods

There are several alternatives for the solution of dynamic optimization problems from which the direct methods are the most widely used. These methods transform the original problem into a non-linear programming (NLP) problem by means of complete parameterization

While the complete parameterization or the multiple shooting approaches may become prohibitively expensive in computational terms, the control vector parameterization approach allows handling large-scale dynamic optimization problems, such as those related to PDE systems, without solving very large NLPs and without dealing with extra junction constraints

The control vector parameterization proceeds dividing the process duration into a number of elements and approximating the control functions typically using low order polynomials. The polynomial coefficients become the new decision variables and the solution of the resulting NLP problem (outer iteration) involves the system dynamics simulation (inner iteration).

Nonlinear programming methods may be largely classified in two groups: local and global methods. Local methods are designed to generate a sequence of solutions, using some type of pattern search or gradient and Hessian information that will converge to a local optimum. However the NLP arising from the application of the control vector parameterization method are frequently multimodal (i.e. presenting multiple local optima), due to the highly nonlinear nature of the dynamics

Over the last decade a number of researchers have proposed different techniques for the solution of multimodal optimization problems. Depending on how the search is performed and which information they are exploiting the alternatives may be classified in two major groups: deterministic and stochastic.

Global deterministic methods

The main drawbacks of global deterministic methods have motivated the use of stochastic methods that do not require any assumptions about the problem’s structure. They make use of pseudo-random sequences to determine search directions toward the global optimum. This leads to an increasing probability of finding the global optimum during the run time of the algorithm, although convergence may not be guaranteed. The main advantage of these methods is that, in practice, they rapidly arrive to the proximity of the solution.

The most successful approaches lie in one (or more) of the following groups: pure random search and adaptive sequential methods, clustering methods or metaheuristics. Metaheuristics are a special class of stochastic methods which have proved to be very efficient in recent years. They include both population (e.g., genetic algorithms) or trajectory-based (e.g., simulated annealing) methods. They can be defined as guided heuristics and many of them try to imitate the behavior of natural or social processes that seek for any kind of optimality

Despite the fact that many stochastic methods can locate the vicinity of global solutions very rapidly, the computational cost associated to the refinement of the solution is usually very large. In order to surmount this difficulty, hybrid methods and metaheuristics that have been recently developed which combine global stochastic methods with local gradient based methods in two phases

Finally, knowing that global optimization methods become prohibitively expensive with an increasing number of decision variables, a control refining technique has been used so as to obtain smoother control profiles. This technique consists of performing successive re-optimizations with increasing control discretization level. A detailed description of the mesh refining approach used is presented in

· Step 1: The problem is solved using a coarse control discretization level (for example, 5−10) with the hybrid optimization method.

· Step 2: The best solution found is transformed by multiplying the discretization level by for example 2−4 and the result is employed as the starting point for the local method.

· Step 3: Step 2 is repeated until the established number of refinements has been achieved.

Results and Discussion

It is well known that spatio-temporal patterns appear in biology from the molecular level to the supra-cellular level

The examples considered here are related to the computation of such stimuli which will originate a given desired pattern. The first example is related to the bacterial chemotaxis process while the second, the FitzHugh-Nagumo model, provides a qualitative description of some physiological processes, such as the neuron firing in the brain or the heart beat.

Case Study I: Bacterial chemotaxis

Some types of cells are highly motile, they are able to sense the presence of chemical signals (chemoattractants) and guide their movement in the direction of the concentration gradient of these signals

The chemotaxis of the bacteria Escherichia coli is one of the best understood chemotactic processes. These bacteria, under given stress conditions, secrete chemoattractants. Other cells respond to these secreted signaling molecules by moving up their local concentration gradients and forming different types of multicellular structures

The modeling of bacterial chemotaxis has received major attention during last decades. In contrast, only some works by Lebiedz and co-workers

Mathematical model

The model under consideration describes the bacterial chemotaxis in a closed long thin tube containing a liquid medium with a cell culture of E. coli and the chemoattractant species which is produced by the cells themselves. The two components (bacteria and chemoattractant) may be described by a coupled reaction-diffusion system of PDEs which, in its 1D version, reads as follows

with boundary and initial conditions of the form:

where

Formulation of the optimal control problem

The objective is to externally manipulate the system so as to achieve a particular cell distribution. With this aim, a non-zero chemoattractant flux is introduced in the boundary

Experimentally this can be achieved introducing in that boundary a semi-permeable membrane (impermeable to the cells but permeable to the chemoattractant). The boundary chemoattractant flux is controlled by fixing the concentration of this chemical species,

As in
_{
T,2}(

where _{
ξ
}equidistant points so that instead of (22) the following expression will be employed:

Note that the summation extends over all the discretized points. The optimal control problem (23) is subject to:

· The system dynamics described by Equations (16)-(18), (20) and (21).

· Bounds on the control variable

The sub-cases will be referred to as OCP1 (for the Gaussian distribution) and OCP2 (for the constant profile).

Results

The finite difference method is employed in this case study to numerically compute the solution of system (16)-(21). Usually, in highly nonlinear systems as the one considered here, the spatial discretization level as well as the order of the finite difference formula play a central role in the computation of an accurate numerical solution. In order to avoid numerical solutions with no physical meaning (spurious solutions), a comparison among different schemes was performed.

Figure
_{
ξ
}. From the figure, it is clear that using a low number of discretization points may result into large simulation errors thus leading to wrong conclusions about optimality. Note also that the solution seems to converge for _{
ξ
}> 101. On the other hand, one may also consider increasing the order of the finite differences formula and check whether it has a direct impact on the number of discretization points required to accurately represent the system dynamics. Figure
_{
ξ
}= 121 and fourth order formula with _{
ξ
}= 41. Since the results are almost indistinguishable, fourth order formula with _{
ξ
}= 41 is selected for optimization purposes as it provides the best compromise between accuracy and efficiency.

Analysis of simulation results, in terms of final time cell distribution as a function of (a) the spatial discretization level and (b) the order of finite differences formula.

**Analysis of simulation results, in terms of final time cell distribution as a function of (a) the spatial discretization level and (b) the order of finite differences formula.**

A multistart strategy of a sequential quadratic programming method (FSQP,

As explained in the “Numerical methods” section, in the control vector parameterization method the process duration is divided into a number of elements (discretization level). As a first approximation we selected a discretization level ^{−5} is achieved.

The corresponding histograms of solutions are presented in Figure

Histograms of solutions for the multistart of FSQP for the chemotaxis related examples.

**Histograms of solutions for the multistart of FSQP for the chemotaxis related examples.** Results obtained from 300 runs from randomly generated initial control profiles. A comparison of optimal solutions obtained by means of **(a)** and OCP2 **(b)** the best reported value was obtained with piecewise linear interpolation.

Let us analyze the results. First depending on the initial guess for the control, different solutions, with different objective function values, are obtained. Therefore the problem is multimodal and several orders of magnitude in

To avoid getting trapped in suboptimal solutions, the use of global optimization methods is suggested. As mentioned previously the NLP solver eSS has proved to efficiently deal with a wide range of optimization problems. Therefore it has been chosen as the global NLP solver for this problem.

As in the multistart approach, a discretization level _{1,BEST
}, while _{2,BEST
} was improved by one order of magnitude. Also the time required to reach those solutions is much lower as compared with the total time of the multistarts.

**Best value**

**Mean value**

**Worst value**

The values between parenthesis correspond with **
log
**

OCP1

2.59×10^{−4} (-3.59)

5.60×10^{−4} (-3.25)

2.39×10^{−3} (-2.62)

OCP2

2.92×10^{−9} (-8.53)

4.11×10^{−8} (-7.38)

1.49×10^{−7} (-6.83)

The best optimal control profiles obtained in the previous step (

For the OCP1, the hybrid approach with control refining allowed us to arrive to
^{−3}

(a) Optimal control profile obtained by the hybrid (

**(a) Optimal control profile obtained by the hybrid (****=14****), linear interpolation) technique.****(b)** Cell density distribution at final time.

Case Study II: The FitzHugh-Nagumo problem

Some physiological processes, such as the heart beating or the neuron firing, are related to electrical potential patterns. Their normal operation is associated to the formation of a traveling plane wave which spreads all over the tissue. Figure

The figures at the top show two snapshots of the

**The figures at the top show two snapshots of the****-field for the FHN system corresponding to (a) the front behavior and (b) the spiral behavior.** The figures at the bottom represent the _{2} = 100 and at different times corresponding to **(c)** the front behavior and **(d)** the spiral behavior.

Due to the obvious necessity of preventing and/or controlling such undesirable behaviors, many research efforts have been devoted to the modeling of such processes. Particularly successful was the one developed by Hodgkin and Huxley

It is worth mentioning that the control and stabilization of spatio-temporal fronts in biological system, and in particular the FHN system, has been successfully approached in the literature -see

Mathematical model

In this work, we consider a 2D version of the FHN model. The system is defined over the square spatial domain

with boundary conditions:

In Equations (24)-(26), **n **indicates a unit vector pointing outwards the surface. In this case study, the initial conditions take the form:

By setting the parameters _{2} = 100 to _{2} = 200).

The finite element method with a grid of around 2300 points has been employed to solve the boundary value problem (24)-(28). Coarser grids result into a front-type solution with low resolution while finer grids do not alter the solution. Note that, since two state variables are considered, such grid implies solving around 4600 ODEs which, for optimization purposes, is computationally involved. In order to overcome such limitation an accurate reduced order model derived by using the POD technique will be developed.

Reduced order model

As mentioned previously, the POD technique will be employed to obtain the reduced order model. In this methodology, five steps can be distinguished:

· Obtain a set of snapshots representative of the system behavior

· Obtain the POD basis

· Decide how many basis will be employed in the projection

· Project the model equations (24)-(28) over the selected POD basis

· Solve the resulting ODE set

This is a critical point in the POD technique. In order to obtain an accurate reduced order model, the snapshots must be representative of the system behavior. Unfortunately, there is no systematic approach to decide the conditions that better represent the system behavior. However, the idea is to capture as much information as possible from a limited set of snapshots that may be obtained either through simulation of the original model or through appropriate experimental setups.

In our case all the snapshots were obtained from simulation of system (24)- (26). The first set of snapshots aimed to capture the front-type behavior, to that purpose the simulation started with initial conditions (27)- (28) setting the control

Once the snapshots are available they are employed to construct the kernel ^{
′
}) as in Eqn (10). In fact two kernels (_{
v
}(^{
′
}) and _{
w
}(^{
′
})) will be constructed from the snapshots of the state variables _{
v
}=[_{
v1}
_{
v2},…,_{
vn
}] and _{
w
}=[_{
w1}
_{
w2},…,_{
wm
}]) are obtained.

This will determine the dimension of the reduced order model. The criteria used to compute the number of POD basis is based on the energy captured by them -see Eqn (11)- which is represented in Figure

(a) Energy captured by the POD basis.**(b)** Reduced order model solution for the FHN system (front behavior)

**(a) Energy captured by the POD basis.****(b)** Reduced order model solution for the FHN system (front behavior).

As explained in section

where

Initial conditions are also projected as follows:

As a result a system with 113 ODEs (more than 40 times lower than the classical finite element method) is obtained.

Finally, the solution of (29)- (31) is computed by a standard initial value problem solver. Figure

Optimal control problem formulation

The aim of this section is to design an open-loop optimal control policy (_{
a
}= 6) are available. In this regard, as shown in Figure

Distribution of the six actuators over the spatial domain.

**Distribution of the six actuators over the spatial domain.**

The optimal control problem is then formulated as follows: _{
k
}(_{
T
}(_{1},_{2})

Subject to:

· The reduced order model dynamics (29)-(31)

· Bounds on the control variables,

Results

Similarly to the previous case, a multistart approach of the FSQP method was selected to study the possible multimodal nature of the problem. As a first approximation we selected a discretization level

Results obtained are summarized in Figure

Histogram of solutions for the multistart of the FHN system.

**Histogram of solutions for the multistart of the FHN system (**
**).**

In order to illustrate the effects of falling into suboptimal solutions, one of the locally optimal control profiles (with _{10}(_{1}>100. The use of the hybrid technique is thus suggested so as to achieve the best possible solution in reasonable computational costs.

Figures (a) and (b) represent the

**Figures (a) and (b) represent the****-field final time spatial distribution after the implementation of an intermediate control profile from the multistart and the global optimal control profile, respectively.** Figures **(c)** and **(d)** represent the absolute error between the desired profile (Figure 4 **(a)**) and the profiles obtained with the optimal control.

As in the chemotaxis case study we choose here the NLP solver eSS to compute the optimal solution. In order to compare the results with those of the multistart, the control discretization was fixed to _{1} = 10, i.e. 60 decision variables and 10 optimization were performed to check the robustness of the solver. The best optimal profile found lead to a cost function value of

From that solution the FSQP method was used with a refining on the control discretization level (_{2} = 20), resulting into a NLP problem with 120 decision variables. After the optimization, a value of the objective function of

Finally, the optimal control profiles for the spatially independent currents are represented in Figure

Heat map of the optimal control profiles for the FitzHugh-Nagumo problem.

**Heat map of the optimal control profiles for the FitzHugh-Nagumo problem.**

Conclusions

The combination of advanced numerical optimization techniques with reduced order based models enables the possibility of efficiently solve dynamic optimization problems related to complex distributed biological systems.

The simulation of non-linear and distributed models by means of typical spatial discretization techniques is usually computationally intensive. In addition, non-linear dynamics often induce multimodality in the associated optimization problems. Therefore calling for global optimization methods which often require a large number of model simulations. These pose important constraints to the solution of dynamic optimization problems related to distributed biological systems.

This work has shown, with two illustrative examples, how these difficulties can be surmounted with the following procedure:

· Use spatial discretization techniques, such as the finite differences or the finite element method, to handle process simulation under different control conditions and generate the snapshots, i.e., numerical values of the spatio-temporal evolution of the state variables.

· Use these snapshots to obtain a more efficient dynamic representation (reduced order model) via the proper orthogonal decomposition approach. Such reduced order model will be employed instead of the complete model, in the following steps, to enhance the efficiency of the solution of the optimization problem.

· Solve the dynamic optimization problem with a coarse discretization and stepwise approximation of the control variables by means of a local NLP solver with a multistart approach (i.e. using multiple initial guesses). If and when the presence of multimodal objective function is confirmed from multistart local optimizations (typically involving 25-50 initial guesses), a hybrid stochastic-local optimization method such as the scatter search based approach should be used.

· Obtain smoother control profiles, if required, by means of a mesh refining technique or a piecewise linear interpolation of the control variables.

Endnote

^{a} For the sake of clarity and without loss of generality, the vector field **x**(

Competing interests

The authors declare that they have no competing interests.

Author’s contributions

All authors contributed to the conception and design of the work. JRB and EBC selected the numerical methods for optimization and case studies. CV and AA selected the numerical methods for simulation. CV and EBC performed the numerical computations. All authors contributed to the writing of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

This work has been partially financed by the FP7 CAFE project (KBBE-2007-1-212754), by the Spanish Ministry of Science and Innovation projects SMART-QC (AGL2008-05267-C03-01) and MULTISCALES (DPI2011-28112-C04-03), by CSIC intramural project BioREDES (PIE-201170E018) and by the Xunta de Galicia project IDECOP (08DPI007402PR). We also acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI) and the valuable advices provided by J. A. Egea.