There exist a number of different optimization techniques for parameter estimation. The choice of technique usually depends on the type of model equations (deterministic or stochastic), on the number of unknown parameters (moderate or large), as well as on the dependence of model solutions on parameters (linear or nonlinear, continuous or discontinuous). For a survey on optimization methods in biochemical models we refer to 216. In general, model (1) – being nonlinear in θ – leads to a least squares problem (2) that has several minima, first because the problem has more than one solution, and second because the fitness function (2) can have several stationary points that do not correspond to the lowest value of the fitness landscape (so-called local minima). Local search methods, like Levenberg-Marquardt (LM), easily get trapped in one of the local minima rather than finding the global minimum. To explore the whole search space one needs global search methods, like the Evolution Strategy used in 12. Unfortunately, these methods converge very slowly once near a minimum. In contrast, gradient-based methods are efficient optimizers 17 for nonlinear least-squares problems once a sufficiently good initial guess for the parameter values is available. In this paper we use the solutions from the global search in 12 as initial guesses for local optimization by the LM method 18. In this way, we reduce the chance of missing the global minimum and the determination of all the minima is precise and fast.

In general, any gradient-based optimization procedure seeks a correction δθ for the parameter vector, such that S(θ + δθ) ≤ S(θ) holds. The LM method 18 determines the correction as the solution of the equations

(J^T(θ)J(θ) + λI_m) δθ = -J^T(θ)Y(θ),

where λ ≥ 0 is a control parameter (see below), I_m is the identity matrix of size m and the Jacobian J(θ)=∂Y(θ)∂(θ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOsaOKaeiikaGIaeqiUdeNaeiykaKIaeyypa0tcfa4aaSaaaeaacqGHciITcqqGzbqwcqGGOaakcqaH4oqCcqGGPaqkaeaacqGHciITcqGGOaakcqaH4oqCcqGGPaqkaaaaaa@3CD3@ is the so-called 'sensitivity' matrix of size N × m. The entry J_{i, j} in J(θ) shows how sensitive the model response is at the i-th data point for a change in the j-th parameter. The LM method can be seen as the combination of two gradient-based approaches: Gauss-Newton and steepest descent 17. If λ = 0 in (3), it coincides with the Gauss-Newton method. However, when the matrix J^T(θ)J(θ) is (almost) singular, to solve (3), λ has to be positive and for large λ the LM method approaches the steepest descent method. During the optimization λ is adapted such that the algorithm strives to exploit the fast convergence of the Gauss-Newton method whenever this is possible 1819.

In order to solve (3), the singular value decomposition (SVD) 20 of the matrix J(θ) can be used, i.e.

J(θ) = U(θ) Σ (θ) V^T(θ),

where U(θ) is an orthogonal matrix of size N × m, such that U^T(θ)U(θ) = I_m, V(θ) is an orthogonal matrix of size m × m, such that V^T(θ)V(θ) = V(θ)V^T(θ) = I_m, and Σ(θ) is a diagonal matrix of size m × m which contains all singular values σ_i in non-increasing order. Then the correction δθ can be found as

δθ = -V(θ) (Σ²(θ) + λI_m)^-1 Σ(θ) U^T(θ) Y(θ).

Later, when we study the reliability of the parameters computed, the SVD will play an important role again.

In order to execute an LM optimization step, the vector of discrepancies Y(θ), the matrix J(θ) and its SVD have to be evaluated for each new estimate of θ. For this purpose, for Y and the entries of J one needs to resolve (1) and the additional system of variational equations (i = 1,2,...,m)

{ ∂ ∂ t ∂ y ∂ θ i = ∂ f ∂ θ i + ∂ f ∂ y ∂ y ∂ θ i , 0 < t ≤ T , ∂ y ( t , θ ) ∂ θ i = ∂ y 0 ( θ ) ∂ θ i , t = 0 , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaiqaaeaafaqaaeGacaaabaqcfa4aaSaaaeaacqGHciITaeaacqGHciITcqWG0baDaaWaaSaaaeaacqGHciITcqWH5bqEaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaOGaeyypa0tcfa4aaSaaaeaacqGHciITcqWHMbGzaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaOGaey4kaSscfa4aaSaaaeaacqGHciITcqWHMbGzaeaacqGHciITcqWH5bqEaaWaaSaaaeaacqGHciITcqWH5bqEaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaOGaeiilaWcabaGaeGimaaJaeyipaWJaemiDaqNaeyizImQaemivaqLaeiilaWcabaqcfa4aaSaaaeaacqGHciITcqWH5bqEcqGGOaakcqWG0baDcqGGSaalcqaH4oqCcqGGPaqkaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaOGaeyypa0tcfa4aaSaaaeaacqGHciITcqWH5bqEdaWgaaqaaiabicdaWaqabaGaeiikaGIaeqiUdeNaeiykaKcabaGaeyOaIyRaeqiUde3aaSbaaeaacqWGPbqAaeqaaaaakiabcYcaSaqaaiabdsha0jabg2da9iabicdaWiabcYcaSaaaaiaawUhaaaaa@7A2C@

respectively. We note that the costs for performing the SVD and computing the correction (5) are negligible in comparison with the computational costs for solving (1) and (6).

Thus, a single LM step requires the numerical solution of m + 1 coupled systems, each one consisting of n ODEs. Fortunately, these systems are coupled in a special way, namely, for each i = 1, 2,...,m, system (6) is a system of linear ODEs for ∂y∂θi MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqGHciITcqWH5bqEaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaaaa@33F0@, coupled only with (1). The system of equations (6) has the same stiffness as (1), so for numerical stability the same step size can be used for the time integration of (1) and (6) (note that ODE stiffness is determined by the eigenvalues of the Jacobian matrix ∂f∂y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqGHciITcqWHMbGzaeaacqGHciITcqWH5bqEaaaaaa@3217@ and is not related to parameter stiffness as described above). Therefore, the one-way coupling can be used to solve (1) and (6) efficiently.

Numerical integration of (1) and (6) requires a fast and reliable ODE solver. Search in the parameter space may lead to some values of θ such that the systems of ODEs become stiff 21. It is well known that for stiff ODE systems explicit schemes can give rise to numerical instability or, alternatively, extremely small time steps. Therefore, an implicit scheme is the best choice for time integration for stability reasons. Using an implicit scheme allows us to exploit the specific coupling between (1) and (6) in an efficient way. At each time step τ integrating first (1) provides the solution vector y. This requires the LU decomposition of the Jacobian matrix Im−τ∂f∂y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfaOaemysaK0aaSbaaeaacqWGTbqBaeqaaiabgkHiTiabes8a0naalaaabaGaeyOaIyRaeCOzaygabaGaeyOaIyRaeCyEaKhaaaaa@3768@. Using this LU decomposition the calculation of ∂y∂θi MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqGHciITcqWH5bqEaeaacqGHciITcqaH4oqCdaWgaaqaaiabdMgaPbqabaaaaaaa@33F0@ from (6) reduces to a simple forward substitution and backsubstitution. In our simulations we use a tailor-made code 22 based on the implicit multistep Backward Differentiation Formulas (BDF) 23.

When the unknown parameters have to obey certain constraints – linear or non-linear – some additional work is needed. If the correction δθ found by (5) leads to violation of some constraints, then by the introduction of Lagrange multipliers a modified correction can be found, which fits all constraints. For the constrained minimization problem we refer the reader to 22.

For additional modeling and numerical aspects of this method we refer the reader to Additional file 1 (Section 1).

Above we used θ* to denote the 'true' parameter vector, for which (1) describes reality with sufficient accuracy, and by θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ we denote the parameter vector which minimizes (2). Even having a 'right' model and an estimate θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ for the parameter vector which fits the data well, does not mean that the whole modeling problem is resolved successfully. It is important to know how reliable the obtained estimate is. This is the subject of a posteriori identifiability analysis 3424. One way to look at this is inspecting the fitness landscape S(θ) in the neighbourhood of θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@. Roughly speaking, if it is a sharp trough then the true parameter vector θ* and the obtained minimum θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ are close. If it is flat in one or more directions, like the surface for a 2-parameter case in Fig. 1(a), then the minimum found can be far apart from the true parameter vector. Near the minimum, where the gradient of S(θ) vanishes, this surface is approximated by the second derivative or Hessian of S(θ). If the model is linear in the parameters the Hessian is equal to J^T J. This linearity assumption and some statistics underlie the following rigorous analysis 141521.

We assume that the measurement errors in y˜i MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyEaKNbaGaadaWgaaWcbaGaemyAaKgabeaaaaa@2EE6@ are independent of each other and normally distributed and that the error distributions have zero mean and constant standard deviation σ. Then, θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ is a maximum likelihood estimate 1415. By assumption the model with the 'true' solution θ* describes reality, thus

y ˜ i ≈ y c i ( t i , θ ∗ ) + ϵ i , i = 1 , 2 , ... , N , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiqbdMha5zaaiaWaaSbaaSqaaiabdMgaPbqabaGccqGHijYUcqWG5bqEdaWgaaWcbaGaem4yam2aaSbaaWqaaiabdMgaPbqabaaaleqaaOGaeiikaGIaemiDaq3aaSbaaSqaaiabdMgaPbqabaGccqGGSaalcqaH4oqCdaahaaWcbeqaaiabgEHiQaaakiabcMcaPiabgUcaRmbvLH1qn1uy0Hws0fgBPngaryWyT1wAXadaiqGacqWF1pGSdaWgaaWcbaGaemyAaKgabeaakiabcYcaSaqaaiabdMgaPjabg2da9iabigdaXiabcYcaSiabikdaYiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabd6eaojabcYcaSaaaaaa@56D5@

where ϵ_i are the measurement errors, for which

θ ^ − θ ∗ ~ N m ( 0 , σ 2 ( J T ( θ ^ ) J ( θ ^ ) ) − 1 ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaacqGHsislcqaH4oqCdaahaaWcbeqaaiabgEHiQaaakiabc6ha+jabd6eaonaaBaaaleaacqWGTbqBaeqaaOWaaeWaaeaacqaIWaamcqGGSaalcqaHdpWCdaahaaWcbeqaaiabikdaYaaakmaabmaabaGaemOsaO0aaWbaaSqabeaacqWGubavaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabdQeakjabcIcaOiqbeI7aXzaajaGaeiykaKcacaGLOaGaayzkaaWaaWbaaSqabeaacqGHsislcqaIXaqmaaaakiaawIcacaGLPaaaaaa@4A66@

holds approximately 14. Here N_m(·,·) denotes the m-dimensional multivariate normal distribution. Notice that (7) holds exactly when y is linear in θ. Next we can define a region around θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ in which the 'true' parameter vector θ* lies with a certain probability 1 - α. This (1 - α)-confidence region is determined by the inequality

( θ ∗ − θ ^ ) T ( J T ( θ ^ ) J ( θ ^ ) ) ( θ ∗ − θ ^ ) ≤ m N − m S ( θ ^ ) F α ( m , N − m ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeiikaGIaeqiUde3aaWbaaSqabeaacqGHxiIkaaGccqGHsislcuaH4oqCgaqcaiabcMcaPmaaCaaaleqabaGaemivaqfaaOWaaeWaaeaacqWGkbGsdaahaaWcbeqaaiabdsfaubaakiabcIcaOiqbeI7aXzaajaGaeiykaKIaemOsaOKaeiikaGIafqiUdeNbaKaacqGGPaqkaiaawIcacaGLPaaacqGGOaakcqaH4oqCdaahaaWcbeqaaiabgEHiQaaakiabgkHiTiqbeI7aXzaajaGaeiykaKIaeyizImAcfa4aaSaaaeaacqWGTbqBaeaacqWGobGtcqGHsislcqWGTbqBaaGccqWGtbWucqGGOaakcuaH4oqCgaqcaiabcMcaPiabdAeagnaaBaaaleaacqaHXoqyaeqaaOGaeiikaGIaemyBa0MaeiilaWIaemOta4KaeyOeI0IaemyBa0MaeiykaKIaeiilaWcaaa@5F38@

where F_α(m, N - m) is the upper α part of Fisher's distribution with m and N - m degrees of freedom. Geometrically these confidence regions are given by the contours of S(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) (for linear models), cf. Fig. 1(a).

The ellipsoid defined by (8), is centered at θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ and has its principal axes directed along the eigenvectors of J^T(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@)J(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@). Using the SVD (4) for J(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@), we get

J T ( θ ^ ) J ( θ ^ ) = V ( θ ^ ) Σ 2 ( θ ^ ) V T ( θ ^ ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOsaO0aaWbaaSqabeaacqWGubavaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabdQeakjabcIcaOiqbeI7aXzaajaGaeiykaKIaeyypa0JaemOvayLaeiikaGIafqiUdeNbaKaacqGGPaqkcqqHJoWudaahaaWcbeqaaiabikdaYaaakiabcIcaOiqbeI7aXzaajaGaeiykaKIaemOvay1aaWbaaSqabeaacqWGubavaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabcYcaSaaa@4982@

and the eigenvectors of J^T(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@)J(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) are the columns of the matrix V(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@). So, the ellipsoid has its principal axes directed along the column vectors of the matrix V(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@). Moreover, the radii along these principal axes are inversely proportional to the corresponding singular values σ_i, the diagonal elements of Σ(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@). This all can be seen by using the following transformation (rotation)

z = V T ( θ ^ ) ( θ ∗ − θ ^ ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeCOEaONaeyypa0JaemOvay1aaWbaaSqabeaacqWGubavaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabcIcaOiabeI7aXnaaCaaaleqabaGaey4fIOcaaOGaeyOeI0IafqiUdeNbaKaacqGGPaqkcqGGSaalaaa@3CE0@

yielding

( θ ∗ − θ ^ ) T ( V ( θ ^ ) Σ 2 ( θ ^ ) V T ( θ ^ ) ) ( θ ∗ − θ ^ ) = z T Σ 2 ( θ ^ ) z = ∑ i = 1 m σ i 2 z i 2 . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeiikaGIaeqiUde3aaWbaaSqabeaacqGHxiIkaaGccqGHsislcuaH4oqCgaqcaiabcMcaPmaaCaaaleqabaGaemivaqfaaOWaaeWaaeaacqWGwbGvcqGGOaakcuaH4oqCgaqcaiabcMcaPiabfo6atnaaCaaaleqabaGaeGOmaidaaOGaeiikaGIafqiUdeNbaKaacqGGPaqkcqWGwbGvdaahaaWcbeqaaiabdsfaubaakiabcIcaOiqbeI7aXzaajaGaeiykaKcacaGLOaGaayzkaaGaeiikaGIaeqiUde3aaWbaaSqabeaacqGHxiIkaaGccqGHsislcuaH4oqCgaqcaiabcMcaPiabg2da9iabhQha6naaCaaaleqabaGaemivaqfaaOGaeu4Odm1aaWbaaSqabeaacqaIYaGmaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabhQha6jabg2da9maaqahabaGaeq4Wdm3aa0baaSqaaiabdMgaPbqaaiabikdaYaaakiabdQha6naaDaaaleaacqWGPbqAaeaacqaIYaGmaaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemyBa0ganiabggHiLdGccqGGUaGlaaa@6B2F@

On the other hand, since S(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@)/(N - m) is an unbiased estimator of σ², the equation for the ellipsoid can be rewritten as

∑ i = 1 m σ i 2 z i 2 = r σ 2 , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaabCaeaacqaHdpWCdaqhaaWcbaGaemyAaKgabaGaeGOmaidaaOGaemOEaO3aa0baaSqaaiabdMgaPbqaaiabikdaYaaakiabg2da9iabdkhaYnaaDaaaleaacqaHdpWCaeaacqaIYaGmaaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemyBa0ganiabggHiLdGccqGGSaalaaa@4195@

where rσ2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOCai3aa0baaSqaaiabeo8aZbqaaiabikdaYaaaaaa@3024@ ≈ mσ²F_α(m, N - m) is proportional to the variance in the measurement errors. This form is more convenient to deal with because z can be considered as a set of uncorrelated variables, and once the conclusion has been drawn for the identifiability of z, the problem can be transformed back, revealing us the quality of θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@.

Now, we assume that the model (1) is properly scaled, such that all parameter values are of the same order of magnitudes, and that we are interested only in the first few digits of the parameter values. Let us introduce the sphere given by

∑ i = 1 m z i 2 = r ϵ 2 , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaabCaeaacqWG6bGEdaqhaaWcbaGaemyAaKgabaGaeGOmaidaaOGaeyypa0JaemOCai3aa0baaSqaambvLH1qn1uy0Hws0fgBPngaryWyT1wAXadaiqGacqWF1pGSaeaacqaIYaGmaaGccqGGSaalaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabd2gaTbqdcqGHris5aaaa@4585@

where r_ϵ defines the level of accuracy one desires for the parameter estimates. For instance, if the parameters are of order O(1) and one is interested only in the first two digits to the right of the decimal point, then r_ϵ = 0.01. In order to be able to determine z_i accurately enough, the radius along the ellipsoid's i-th principal axis shouldn't exceed the radius of the sphere, which leads us to the following inequality

σ i ≥ r σ r ϵ . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdMgaPbqabaGccqGHLjYSjuaGdaWcaaqaaiabdkhaYnaaBaaabaGaeq4WdmhabeaaaeaacqWGYbGCdaWgaaqaambvLH1qn1uy0Hws0fgBPngaryWyT1wAXadaiqGacqWF1pGSaeqaaaaakiabc6caUaaa@4197@

A graphical representation of the ellipsoid and the sphere for the 2-dimensional case is given in Figure 1(b).

If only the first k largest singular values satisfy (12), then only the first k entries of z are estimated with the required accuracy and no sufficient information is available for the remaining components of z. Each of the first k entries of z defines a parameter or a linear combination of parameters which is well-determined. If a principal axis of the ellipsoid makes a significant angle with the axes in parameter space (i.e., there exists more than one significant entry in the eigenvector), this implies correlation between parameters in θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@.

To summarize, the level of noise in the data in combination with the accuracy requirement for the parameter estimates, defines the threshold for significant singular values in the matrix Σ. The number of singular values exceeding this threshold determines the number of parameter relations that can be derived from the experiment. How these relations relate to the individual parameters is described by the corresponding columns in the matrix V. The largest entries in these columns indicate the well-determined parameters. This method is illustrated on the basis of a simple enzymatic reaction in 2.

Finally, (11) indicates that having, for instance, two times more accurate data so that the standard deviation σ is halved, will decrease the radii along the ellipsoid's principal axis by a factor of 2. Therefore, in case of very small singular values σ_i (i.e. strongly elongated ellipsoids) more accurate data obtained by the experimentalist will not improve the quality of the corresponding parameter estimates by much. In such a case, one certainly needs additional measurements of a different type (e.g. different components, different time points, or in the case of PDEs different spatial points).

Another way of assessing the information from the confidence region is by looking at confidence intervals of the parameter estimates θ^i MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaadaWgaaWcbaGaemyAaKgabeaaaaa@2F22@ (i = 1, 2,...,m). From (8) one can derive dependent and independent confidence intervals. The dependent confidence interval is the intersection of the ellipsoidal region with the i-th parameter axis

{ θ i : | θ i − θ ^ i | ≤ r σ ( V ( θ ^ ) Σ 2 ( θ ^ ) V T ( θ ^ ) ) i i } , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaiWaaeaacqaH4oqCdaWgaaWcbaGaemyAaKgabeaakiabcQda6iabcYha8jabeI7aXnaaBaaaleaacqWGPbqAaeqaaOGaeyOeI0IafqiUdeNbaKaadaWgaaWcbaGaemyAaKgabeaakiabcYha8jabgsMiJMqbaoaalaaabaGaemOCai3aaSbaaeaacqaHdpWCaeqaaaqaamaakaaabaWaaeWaaeaacqWGwbGvcqGGOaakcuaH4oqCgaqcaiabcMcaPiabfo6atnaaCaaabeqaaiabikdaYaaacqGGOaakcuaH4oqCgaqcaiabcMcaPiabdAfawnaaCaaabeqaaiabdsfaubaacqGGOaakcuaH4oqCgaqcaiabcMcaPaGaayjkaiaawMcaamaaBaaabaGaemyAaKMaemyAaKgabeaaaeqaaaaaaOGaay5Eaiaaw2haaiabcYcaSaaa@58DD@

i.e. one assumes that all other parameters are exactly determined. The independent confidence interval is the projection of the ellipsoidal region onto the i-th parameter axis

{ θ i : | θ i − θ ^ i | ≤ r σ ( V ( θ ^ ) Σ − 2 ( θ ^ ) V T ( θ ^ ) ) i i } . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaaiWaaeaacqaH4oqCdaWgaaWcbaGaemyAaKgabeaakiabcQda6iabcYha8jabeI7aXnaaBaaaleaacqWGPbqAaeqaaOGaeyOeI0IafqiUdeNbaKaadaWgaaWcbaGaemyAaKgabeaakiabcYha8jabgsMiJkabdkhaYnaaBaaaleaacqaHdpWCaeqaaOWaaOaaaeaadaqadaqaaiabdAfawjabcIcaOiqbeI7aXzaajaGaeiykaKIaeu4Odm1aaWbaaSqabeaacqGHsislcqaIYaGmaaGccqGGOaakcuaH4oqCgaqcaiabcMcaPiabdAfawnaaCaaaleqabaGaemivaqfaaOGaeiikaGIafqiUdeNbaKaacqGGPaqkaiaawIcacaGLPaaadaWgaaWcbaGaemyAaKMaemyAaKgabeaaaeqaaaGccaGL7bGaayzFaaGaeiOla4caaa@597A@

Clearly, small independent confidence intervals for θ^i MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaadaWgaaWcbaGaemyAaKgabeaaaaa@2F22@ indicate that it is well-determined. However, in some cases considering only individual confidence intervals can be misleading. For instance, in the presence of strong correlations between parameters, the dependent confidence intervals underestimate the confidence region while the independent confidence intervals overestimate it.

From (7), the covariance matrix of θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ is given by

σ 2 ( J T ( θ ^ ) J ( θ ^ ) ) − 1 = σ 2 V ( θ ^ ) Σ − 2 ( θ ^ ) V T ( θ ^ ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaWbaaSqabeaacqaIYaGmaaGcdaqadaqaaiabdQeaknaaCaaaleqabaGaemivaqfaaOGaeiikaGIafqiUdeNbaKaacqGGPaqkcqWGkbGscqGGOaakcuaH4oqCgaqcaiabcMcaPaGaayjkaiaawMcaamaaCaaaleqabaGaeyOeI0IaeGymaedaaOGaeyypa0Jaeq4Wdm3aaWbaaSqabeaacqaIYaGmaaGccqWGwbGvcqGGOaakcuaH4oqCgaqcaiabcMcaPiabfo6atnaaCaaaleqabaGaeyOeI0IaeGOmaidaaOGaeiikaGIafqiUdeNbaKaacqGGPaqkcqWGwbGvdaahaaWcbeqaaiabdsfaubaakiabcIcaOiqbeI7aXzaajaGaeiykaKIaeiOla4caaa@53E8@

Then, by denoting B(θ^)=V(θ^)Σ−2(θ^)VT(θ^) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOqaiKaeiikaGIafqiUdeNbaKaacqGGPaqkcqGH9aqpcqWGwbGvcqGGOaakcuaH4oqCgaqcaiabcMcaPiabfo6atnaaCaaaleqabaGaeyOeI0IaeGOmaidaaOGaeiikaGIafqiUdeNbaKaacqGGPaqkcqWGwbGvdaahaaWcbeqaaiabdsfaubaakiabcIcaOiqbeI7aXzaajaGaeiykaKcaaa@4334@, the correlation coefficient between θ^i MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaadaWgaaWcbaGaemyAaKgabeaaaaa@2F22@ and θ^j MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaadaWgaaWcbaGaemOAaOgabeaaaaa@2F24@ can be computed by

ρ i j = B i j B i i B j j . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqyWdi3aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpjuaGdaWcaaqaaiabdkeacnaaBaaabaGaemyAaKMaemOAaOgabeaaaeaadaGcaaqaaiabdkeacnaaBaaabaGaemyAaKMaemyAaKgabeaacqWGcbGqdaWgaaqaaiabdQgaQjabdQgaQbqabaaabeaaaaGccqGGUaGlaaa@3F25@

We note that by computing individual confidence intervals and correlations between parameters, one is not able to assess the identifiability of linear combinations of parameters. This can be seen only by using the first approach, i.e. by inspection of the V and Σ matrix.

We apply the methodology described above to assess parameter determinability of gene circuit models for the gap gene network in early Drosophila development. Here, we provide a brief outline of gap gene circuit models. More detailed information can be found in 7825.

Segment determination occurs during the blastoderm stage of Drosophila development, between 1.5 and 3 hours after egg laying 26. During this stage, the embryo consists of a syncytium; there are no cell membranes between nuclei. These nuclei constitute the basic objects of the model. They are arranged in a row along the A-P axis. Nuclei divide rapidly and synchronously 27. Periods between mitotic divisions are called cleavage cycles, where cycle n occurs between mitoses n - 1 and n. The models considered here run from early cycle 13 (t = 0.0 min) to the onset of gastrulation at the end of cycle 14A (t = 71.1 min). Mitosis occurs at the end of cycle 13, between t = 16.0 min and t = 21.1 min 27.

Gene circuit models describe the change in concentrations of each gap gene product in each nucleus over time by the following system of ODEs

d g i a d t = R a Φ ( ∑ b = 1 N g W a b g i b + m a g i Bcd + h a ) − λ a g i a + D a ( g i + 1 a − 2 g i a + g i − 1 a ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqWGKbazcqWGNbWzdaqhaaqaaiabdMgaPbqaaiabdggaHbaaaeaacqWGKbazcqWG0baDaaGccqGH9aqpcqWGsbGudaWgaaWcbaGaemyyaegabeaakiabfA6agnaabmaabaWaaabCaeaacqWGxbWvdaqhaaWcbaGaemyyaegabaGaemOyaigaaOGaem4zaC2aa0baaSqaaiabdMgaPbqaaiabdkgaIbaakiabgUcaRiabd2gaTnaaBaaaleaacqWGHbqyaeqaaOGaem4zaC2aa0baaSqaaiabdMgaPbqaaiabbkeacjabbogaJjabbsgaKbaakiabgUcaRiabdIgaOnaaBaaaleaacqWGHbqyaeqaaaqaaiabdkgaIjabg2da9iabigdaXaqaaiabd6eaonaaBaaameaacqWGNbWzaeqaaaqdcqGHris5aaGccaGLOaGaayzkaaGaeyOeI0Iaeq4UdW2aaSbaaSqaaiabdggaHbqabaGccqWGNbWzdaqhaaWcbaGaemyAaKgabaGaemyyaegaaOGaey4kaSIaemiraq0aaSbaaSqaaiabdggaHbqabaGcdaqadaqaaiabdEgaNnaaDaaaleaacqWGPbqAcqGHRaWkcqaIXaqmaeaacqWGHbqyaaGccqGHsislcqaIYaGmcqWGNbWzdaqhaaWcbaGaemyAaKgabaGaemyyaegaaOGaey4kaSIaem4zaC2aa0baaSqaaiabdMgaPjabgkHiTiabigdaXaqaaiabdggaHbaaaOGaayjkaiaawMcaaiabc6caUaaa@7B68@

a and b denote regulated genes and regulators respectively. a and b are integer indices representing cad, hb, Kr, kni, gt as well as the terminal gap gene tailless (tll). gia MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4zaC2aa0baaSqaaiabdMgaPbqaaiabdggaHbaaaaa@2FFF@ denotes the concentration of the product of gene a in nucleus i. The Bcd gradient remains constant over time, and is not regulated by the other genes in the model. giBcd MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4zaC2aa0baaSqaaiabdMgaPbqaaiabbkeacjabbogaJjabbsgaKbaaaaa@325B@ denotes the concentration of Bcd protein in nucleus i. N_g = 6 is the number of genes in the model (excluding Bcd), and the function

Φ ( x ) = 1 2 ( x x 2 + 1 + 1 ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeuOPdyKaeiikaGIaemiEaGNaeiykaKIaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqaIYaGmaaGcdaqadaqaaKqbaoaalaaabaGaemiEaGhabaWaaOaaaeaacqWG4baEdaahaaqabeaacqaIYaGmaaGaey4kaSIaeGymaedabeaaaaGccqGHRaWkcqaIXaqmaiaawIcacaGLPaaaaaa@3E43@

is a sigmoid regulation-expression function.

During mitosis, protein production is shut down. Nuclei divide instantaneously at the end of mitosis and the distance between them is halved. Gap gene circuits cover the region from 35% to 92% A-P position, which includes 30 (cycle 13) and 58 (14A) nuclei. Therefore, system (17) consists of 180 and 348 ODEs during cycles 13 and 14A, respectively. Initial conditions are prescribed by maternal gradients of Bcd, Cad and Hb, and zero levels for all other gene products. We use no-flux boundary conditions at i = 0 and i = i_max.

In system (17) there are m = 66 unknown parameters. These include the genetic interconnection or regulatory weight matrix W of size N_g × N_g where the matrix elements Wab MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdggaHbqaaiabdkgaIbaaaaa@2FD1@ represent the regulation of gene a by gene b, while maternal coefficients m_a represent the regulatory effect of Bcd on gene a. Regulatory parameters represent repression (if < 0), activation (if > 0) or no interaction (if ≈ 0). Other parameters include promoter thresholds h_a, promoter strengths R_a, diffusion coefficients D_a, and decay rates λ_a. Estimates for these parameters have been obtained in previous studies by fits to quantitative expression data 11 using global search methods such as parallelized Lam Simulated Annealing 78 or the Stochastic Ranking Evolution Strategy (followed by downhill simplex direct search) 12 and using a first-improvement local search method with randomized order of examination 13. In the latter the initial parameter estimates are obtained by using a splitting strategy: parameters λ_a and D_a are estimated by assuming that the protein production is constant within certain spatio-temporal domains which reduces (17) to a system of linear equations uncoupled for each gene (the boundaries of production domains are obtained from data); parameters in the nonlinear part of the model are estimated by fitting the production term in (17) with the data given as input, as closely as possible, to the quadrilateral production regions.

The data set used for model fitting consists of N = 2702 measurements of protein concentrations at nuclear resolution (using multi-channel immunofluorescent antibody assays; available online 11). Measurements were taken at one time point during cycle 13 (T₀), and eight time points T_i (1 ≤ i ≤ 8) during cycle 14A (Figure 2a). Measurements for the concentrations of all gene products represented in the model at all time points are available, except for Cad at T₇ and T₈, and Tll before T₃. The level of measurement error in the data is less than 5%, see 28. Each data point represents concentration values which have been averaged across 9–62 embryos. Therefore, from the Central Limit Theorem (CLT) we assume that the experimental errors are approximately normally distributed.

The quality of the parameter estimates is measured by the root mean square (RMS) of the discrepancy vector

R M S ( θ ) = 1 N ∑ a = 1 N g ∑ i = 1 N c ∑ j = 0 N t α j a ( g i a ( T j , θ ) m o d e l − g i a ( T j ) d a t a ) 2 , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOuaiLaemyta0Kaem4uamLaeiikaGIaeqiUdeNaeiykaKIaeyypa0ZaaOaaaeaajuaGdaWcaaqaaiabigdaXaqaaiabd6eaobaakmaaqahabaWaaabCaeaadaaeWbqaaiabeg7aHnaaDaaaleaacqWGQbGAaeaacqWGHbqyaaGccqGGOaakcqWGNbWzdaqhaaWcbaGaemyAaKgabaGaemyyaegaaOGaeiikaGIaemivaq1aaSbaaSqaaiabdQgaQbqabaGccqGGSaalcqaH4oqCcqGGPaqkdaWgaaWcbaGaemyBa0Maem4Ba8MaemizaqMaemyzauMaemiBaWgabeaakiabgkHiTiabdEgaNnaaDaaaleaacqWGPbqAaeaacqWGHbqyaaGccqGGOaakcqWGubavdaWgaaWcbaGaemOAaOgabeaakiabcMcaPmaaBaaaleaacqWGKbazcqWGHbqycqWG0baDcqWGHbqyaeqaaOGaeiykaKYaaWbaaSqabeaacqaIYaGmaaaabaGaemOAaOMaeyypa0JaeGimaadabaGaemOta40aaSbaaWqaaiabdsha0bqabaaaniabggHiLdaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGobGtdaWgaaadbaGaem4yamgabeaaa0GaeyyeIuoaaSqaaiabdggaHjabg2da9iabigdaXaqaaiabd6eaonaaBaaameaacqWGNbWzaeqaaaqdcqGHris5aOGaeiilaWcaleqaaaaa@7936@

where N_t = 8 is the number of time classes, N_c is the number of nuclei and αja MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqySde2aa0baaSqaaiabdQgaQbqaaiabdggaHbaaaaa@3049@ is equal to zero for Tll at j = 0,1, 2 and for Cad at j = 7, 8, and is equal to one otherwise. A solution is considered to be 'good' if RMS < 12.0 and if there are no visible pattern defects in the model response 781213. It is important to note that the RMS only shows the quality of the fit of the model to the data but does not give any information about the quality of the parameter estimates. Our aim is to find the parameter estimates that give a good fit and to apply statistical analysis in order to investigate how reliable these estimates are.

The search space for parameters is defined by the linear constraints

10.0 ≤ R a ≤ 30.0 , 0.0 < D a ≤ 0.3 , 5.0 ≤ l n ( 2 ) λ a ≤ 20.0 , a = 1 , ... , N g , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeabaaaabaGaeGymaeJaeGimaaJaeiOla4IaeGimaaJaeyizImQaemOuai1aaSbaaSqaaiabdggaHbqabaGccqGHKjYOcqaIZaWmcqaIWaamcqGGUaGlcqaIWaamcqGGSaalaeaacqaIWaamcqGGUaGlcqaIWaamcqGH8aapcqWGebardaWgaaWcbaGaemyyaegabeaakiabgsMiJkabicdaWiabc6caUiabiodaZiabcYcaSaqaaiabiwda1iabc6caUiabicdaWiabgsMiJMqbaoaalaaabaGaemiBaWMaemOBa4MaeiikaGIaeGOmaiJaeiykaKcabaGaeq4UdW2aaSbaaeaacqWGHbqyaeqaaaaakiabgsMiJkabikdaYiabicdaWiabc6caUiabicdaWiabcYcaSaqaaiabdggaHjabg2da9iabigdaXiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabd6eaonaaBaaaleaacqWGNbWzaeqaaOGaeiilaWcaaaaa@6578@

and by the nonlinear constraints

∑ b = 1 N g ( W a b g m a x b ) 2 + ( m a g m a x Bcd ) 2 + ( h a ) 2 ≤ 10 4 , a = 1 , ... , N g , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaamaaqahabaWaaeWaaeaacqWGxbWvdaqhaaWcbaGaemyyaegabaGaemOyaigaaOGaem4zaC2aa0baaSqaaiabd2gaTjabdggaHjabdIha4bqaaiabdkgaIbaaaOGaayjkaiaawMcaamaaCaaaleqabaGaeGOmaidaaOGaey4kaSYaaeWaaeaacqWGTbqBdaWgaaWcbaGaemyyaegabeaakiabdEgaNnaaDaaaleaacqWGTbqBcqWGHbqycqWG4baEaeaacqqGcbGqcqqGJbWycqqGKbazaaaakiaawIcacaGLPaaadaahaaWcbeqaaiabikdaYaaakiabgUcaRmaabmaabaGaemiAaG2aaSbaaSqaaiabdggaHbqabaaakiaawIcacaGLPaaadaahaaWcbeqaaiabikdaYaaakiabgsMiJkabigdaXiabicdaWmaaCaaaleqabaGaeGinaqdaaaqaaiabdkgaIjabg2da9iabigdaXaqaaiabd6eaonaaBaaameaacqWGNbWzaeqaaaqdcqGHris5aOGaeiilaWcabaGaemyyaeMaeyypa0JaeGymaeJaeiilaWIaeiOla4IaeiOla4IaeiOla4IaeiilaWIaemOta40aaSbaaSqaaiabdEgaNbqabaGccqGGSaalaaaaaa@692E@

where gmaxb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4zaC2aa0baaSqaaiabd2gaTjabdggaHjabdIha4bqaaiabdkgaIbaaaaa@32CD@ and gmaxBcd MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4zaC2aa0baaSqaaiabd2gaTjabdggaHjabdIha4bqaaiabbkeacjabbogaJjabbsgaKbaaaaa@3527@ are the maximum values in the data set for proteins b and Bcd, respectively. Note that in 7813 threshold parameters h_a for genes Kr, Kni, gt, and hb are fixed to negative values representing a constitutively repressed state for the corresponding genes 78. Fixing some parameters to specific values may severely restrict the search space leaving some solutions out of consideration. Contrary to their approach, we include threshold parameters for these genes in the search by putting the constraints -10.0 ≤ h_a ≤ 0.0.

In order to make the analysis of parameter estimation easier, we scale all parameters used in (17) in the following way:

R ˜ a = 0.1 R a , D ˜ a = 10 D a , λ ˜ a = 10 λ a , W ˜ a b = 10 2 W a b , m ˜ a = 10 2 m a , h ˜ a = h a , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGbaaaabaGafmOuaiLbaGaadaWgaaWcbaGaemyyaegabeaakiabg2da9iabicdaWiabc6caUiabigdaXiabdkfasnaaBaaaleaacqWGHbqyaeqaaOGaeiilaWcabaGafmiraqKbaGaadaWgaaWcbaGaemyyaegabeaakiabg2da9iabigdaXiabicdaWiabdseaenaaBaaaleaacqWGHbqyaeqaaOGaeiilaWcabaGafq4UdWMbaGaadaWgaaWcbaGaemyyaegabeaakiabg2da9iabigdaXiabicdaWiabeU7aSnaaBaaaleaacqWGHbqyaeqaaOGaeiilaWcabaGafm4vaCLbaGaadaqhaaWcbaGaemyyaegabaGaemOyaigaaOGaeyypa0JaeGymaeJaeGimaaZaaWbaaSqabeaacqaIYaGmaaGccqWGxbWvdaqhaaWcbaGaemyyaegabaGaemOyaigaaOGaeiilaWcabaGafmyBa0MbaGaadaWgaaWcbaGaemyyaegabeaakiabg2da9iabigdaXiabicdaWmaaCaaaleqabaGaeGOmaidaaOGaemyBa02aaSbaaSqaaiabdggaHbqabaGccqGGSaalaeaacuWGObaAgaacamaaBaaaleaacqWGHbqyaeqaaOGaeyypa0JaemiAaG2aaSbaaSqaaiabdggaHbqabaGccqGGSaalaaaaaa@68F4@

for all genes a and b. Note that the choice of the scaling factors for R_a, D_a, and λ_a is based on the search ranges of the corresponding parameters. The choice of the scaling factors for regulatory weights Wab MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdggaHbqaaiabdkgaIbaaaaa@2FD1@ and maternal coefficients m_a is based on the fact that the maximum level of protein concentration for all genes in the data set is of order O(10²). Thus, all scaled parameters are of order O(1).

Least squares estimation of the 66 parameters of the gap gene circuit model (full search case) using the LM method yields a significant decrease of the RMS (19) in all simulations (see Table 1). There are only 5 (out of 80) initial parameter sets with RMS < 10.0 (best fit: RMS = 9.56). After using the LM method there are 71 final parameter sets with RMS < 10.0, among which there are 64 with their RMS evenly distributed between 8.37 and 9.43. None of these low-scoring parameter sets show any visible patterning defects (see Figure 2.1 in Additional file 1), while most solutions with larger RMS do. As it is difficult to make a distinction between these 64 parameter sets based on RMS values and expression patterns only, we take all of them into account for our analysis. We note that there is no guarantee that a better solution might have been missed by our parameter estimation procedure. However, since the initial points for the LM search were found by a global search method 12, it is likely that the search space for unknown parameters is explored sufficiently enough.

Parameter estimates found by the LM method significantly improve solution fits found in previous studies (see Figure 3) 781213. However, there are two problems, mentioned in 78, that remain unsolved with the new parameter estimates. The first one concerns the artificially high level of gap gene expression during early cycle 13. The model responses are much larger than the data values yielding large positive discrepancies. This is probably due to the lack of protein production delays in the model 7. The second one concerns the incorrect shift of the posterior Hb domain, which is due to the absence of the terminal gap gene huckebein (hkb) from our current models 78.

Many parameters have a broad range of possible values, meaning that they are not uniquely determined (see Figures 2.2-2.3 in Additional file 1). Classification of all parameter estimates for regulatory weights into 'activating', 'repressing' or 'no interaction' categories is shown in Figure 4(a). The resulting network topology is in very good agreement with the results obtained in 781213. Specifically,

(A1) Cad and Bcd activate gap genes hb, Kr, kni, and gt;

(A2) gap genes hb, Kr, kni, and gt show auto-activation;

(A3) Tll represses gap genes Kr, kni, and gt;

(A4) gap genes with mutually exclusive expression domains strongly repress each other; these correspond to weights WgtKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUealjabdkhaYbaaaaa@328D@, WKrgt MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdEgaNjabdsha0baaaaa@328D@, Whbkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33FE@, and Wknihb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdIgaOjabdkgaIbaaaaa@33FE@.

Previous results also suggested that pairs of overlapping gap genes (hb and gt, gt and kni, kni and Kr, as well as Kr and hb) either show no or weak repressive interactions among each other. Note that some of these weights differ slightly from earlier analyses 712. In all of these cases the difference is extremely slight and depends on the threshold chosen to categorize an interaction as 'very weak repression' or 'no interaction' (for example WkniKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdUealjabdkhaYbaaaaa@33E4@ or WKrkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33E4@ in Figure 4(a); see also scatter plots in Figure 2.2 in Additional file 1). It is therefore unlikely that such differences are biologically significant. The only activation between overlapping gap genes is predicted for the effect of Gt on hb. In addition, we find that Kni activates gt in a majority of solutions. In both of these cases, the significance of the interactions does not lie in their weak activating effect (which has no discernible biological function), but rather in the absence of repression 78.

We applied the statistical analysis introduced in the Methods section to the 64 parameter sets obtained by the LM method to assess the quality of our estimates. Ellipsoidal confidence regions corresponding to parameter estimates are given by (10). None of the parameter estimates lies in the ellipsoidal confidence regions of all other parameter sets. Note that this does not necessarily imply that there is no unique 'true' solution for the parameter vector, since the ellipsoidal confidence regions – or at least some of them – may still have a non-empty intersection.

For each parameter set θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@, the SVD (4) of the Jacobian J(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) yields the matrices V(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) and Σ(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@). In order to find the number of singular values in Σ(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) satisfying the accuracy inequality (12), i.e. to determine how many (combinations of) parameters can be determined, we need to quantify r_ϵ and r_σ. We are interested only in the first decimal digit of the scaled parameters, and therefore we take r_ϵ = 0.1. For α = 0.05 we obtain r_σ ≈ 9.4 RMS(θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@) (the choice of α does not make much difference here due to the large value of N).

Investigation of all parameter sets shows that on average, 15 singular values satisfy (12) meaning that at most 15 parameters or linear combinations of them can be determined with one digit accuracy. There is a set of parameters which have significant entries in the first 15 columns of all V matrices. It includes regulatory weights WKrcad MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdogaJjabdggaHjabdsgaKbaaaaa@33B0@, Wgtcad MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdogaJjabdggaHjabdsgaKbaaaaa@33EC@, Wknicad MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdogaJjabdggaHjabdsgaKbaaaaa@3543@, Wtllcad MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdsha0jabdYgaSjabdYgaSbqaaiabdogaJjabdggaHjabdsgaKbaaaaa@3557@, WKrhb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdIgaOjabdkgaIbaaaaa@326B@, promoter thresholds h_Kr, h_gt, h_tll, decay rate λ_cad, and promoter strength R_Kr. However, inspection of the first 15 columns of the V matrices shows that there is not a single parameter which can be determined individually with the chosen accuracy. Thus, each column has a number of significant entries implying that the principal axis of the confidence ellipsoid is at an angle with the corresponding axes in parameter space. This indicates the presence of correlations between parameters.

Dependent and independent confidence intervals for each parameter set can be computed by (13) and (14), respectively. We check if the corresponding confidence intervals for regulatory weights fall entirely into the 'repression', 'no interaction', or 'activation' categories. Results in Figure 4(a) do not change when only dependent confidence intervals are taken into account. However, including independent confidence intervals one can no longer make correct qualitative conclusions about many of the entries in the regulatory weight matrix.

The sizes of the independent confidence intervals give an indication about the determinability of the corresponding regulatory weights. There is a set of eight regulatory weights which have relatively small confidence intervals for all 64 parameter sets (see Figure 2.4 in Additional file 1). It includes WhbKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdUealjabdkhaYbaaaaa@326B@, Whbtll MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdsha0jabdYgaSjabdYgaSbaaaaa@3412@, WKrhb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdIgaOjabdkgaIbaaaaa@326B@, WKrKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdUealjabdkhaYbaaaaa@3251@, WKrkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33E4@, Wgthb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdIgaOjabdkgaIbaaaaa@32A7@, Wgtgt MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdEgaNjabdsha0baaaaa@32C9@, and Wgtkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@3420@. For instance, Figure 5(a) shows the confidence intervals for WhbKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdUealjabdkhaYbaaaaa@326B@. This regulatory weight is well determined qualitatively, i.e. the independent confidence intervals fall entirely into one category and therefore the type of the regulation can be concluded. The model predicts that Kr does not regulate hb. Note that the confidence intervals for these eight parameters in the scaled case are of order O(10^-1) and therefore they are not determinable with the chosen accuracy level r_ϵ = 0.1. In fact, they are determinable only if we choose r_ϵ = 1.0.

The remaining regulatory weights have larger independent confidence intervals (see Figure 2.4 in Additional file 1) and therefore they are not determined quantitatively. Among them are some regulatory weights for which qualitative conclusions can be deduced from the results. For example, panels (d) and (e) of Figure 5 show the confidence intervals for regulatory weights Wknihb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdIgaOjabdkgaIbaaaaa@33FE@ and WgtKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUealjabdkhaYbaaaaa@328D@, respectively. Although these two regulatory weights can not be determined quantitatively, there is a qualitative difference between them. The independent confidence intervals for WgtKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUealjabdkhaYbaaaaa@328D@ do not extend significantly into the positive part of the plane. Therefore, one can make a qualitative conclusion for this weight: the model predicts that Kr does not activate gt. Note that this is a weaker conclusion than predicting repression for this weight from Figure 4(a). In contrast, we cannot draw any qualitative conclusions about Wknihb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdIgaOjabdkgaIbaaaaa@33FE@. Thus, our analysis does not confirm the repression of kni by Hb inferred from Figure 4(a) (but does not contradict it either). To demonstrate that repression of kni by Hb is not strictly necessary to fit the data correctly, we fix this weight to zero while performing parameter estimation. The obtained parameter set has a RMS = 9.24 and produces patterns with no visible defects (see Figure 2.7 in Additional file 1).

Based on the confidence intervals, we summarize the qualitative conclusions for the most important regulatory weights in the gap gene system:

(B1) Cad and Bcd do not repress gap genes hb, Kr, and gt; no conclusions can he made for regulation of kni by Cad and Bcd;

(B2) gap genes hb, Kr, kni, and gt do not show auto-repression;

(B3) Tll does not activate gap gene gt; no conclusions can be made for regulation of Kr and kni by Tll;

(B4) gap genes with mutually exclusive expression domains gt and Kr do not activate each other; no conclusions can be made for regulatory interactions between hb and kni.

Interactions between overlapping gap genes are mostly weakly repressive or absent, and are largely consistent with Figure 4(a): confidence intervals for WhbKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdUealjabdkhaYbaaaaa@326B@, WKrhb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdIgaOjabdkgaIbaaaaa@326B@, WKrkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33E4@, Wgthb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdIgaOjabdkgaIbaaaaa@32A7@, and Wgtkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@3420@ indicate no interaction, while confidence intervals for Wknigt MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdEgaNjabdsha0baaaaa@3420@, and WkniKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdUealjabdkhaYbaaaaa@33E4@ suggest the absence of activation. Finally, confidence intervals for Whbgt MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdEgaNjabdsha0baaaaa@32A7@ indicate the absence of repression.

Obviously, our qualitative conclusions (B1)–(B4) are weaker than the conclusions (Al)–(A4) made from Figure 4(a) by considering only the values of parameter estimates. Note that for all genes, promoter thresholds h, promoter strengths R, diffusion coefficients D, and decay rates λ have extremely large independent confidence intervals (see Figure 2.5 in Additional file 1) meaning that all these parameters are not determinable.

The large difference between dependent and independent confidence intervals indicates the presence of correlations between parameters. Individual confidence intervals are not informative for understanding the reason of poor determinability of parameters when their estimates are correlated. Using (16), we find the correlation matrix for each parameter set. To detect the most significant correlations between parameters present in all correlation matrices, we calculate an averaged matrix – which we call the mean correlation matrix – whose entries are the mean values of the corresponding correlation coefficients in the individual correlation matrices. The obtained mean correlation matrix has a block diagonal structure such that each block corresponds to a given gene and contains the correlation coefficients between parameters for the same gene (see Figure 2.6 in Additional file 1). Panels (a),(b),(c), and (d) of Figure 6 show the blocks corresponding to gap genes hb, Kr, gt, and kni, respectively. Note that the correlations corresponding to the most significant entries in the mean correlation matrix (with absolute values greater than 0.5) are statistically present in all individual correlation matrices because corresponding standard deviations are relatively small (less than 0.2).

The main insight from the mean correlation matrix is that we observe strong correlations of regulatory parameters with promoter thresholds. For instance, regulation of hb, Kr, gt, and kni by Bcd and Cad, and auto-regulation are all strongly correlated with their corresponding h_a (see panels (a),(b),(c), and (d) of Figure 6). This may explain the poor determinability for these interactions. We checked this hypothesis by fixing promoter thresholds h_a for gap genes hb, Kr, gt, and kni in (17) to a value of -3.5, similar to the approach used in 7812. We find that also in this case, least squares estimation using the LM method yields a significant decrease of the RMS (see Table 1). There are 63 parameter sets with RMS < 10.0 (best fit: RMS = 8.66). Among these, there are 60 parameter sets which have no visible patterning defects (see Figure 3.1 in Additional file 1) and these were taken into account in the following analysis. The resulting regulatory network topology (see Figure 4(b)) largely corresponds to that obtained without fixing threshold parameters (full search case) with a few minor exceptions. WkniKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdUealjabdkhaYbaaaaa@33E4@, WKrkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33E4@, and Wgtkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@3420@ now all fall into the 'no interaction' category while the full search found mutual repression between Kr and kni, and activation of gt by Kni (compare panels (a) and (b) of Figure 4). As discussed above, these changes represent very small quantitative changes in the parameter values and depend on the (somewhat arbitrary) choice of cut-off between regulatory categories (compare scatter plots in Figures 2.2 and 3.2 in Additional file 1). Therefore, they are unlikely to be biologically significant, while all our main qualitative conclusions (Al)–(A4) on gap gene network topology are fully consistent with our results using fixed threshold parameters.

On the other hand, we observe significant improvement in determinability of some regulatory weights when we compute dependent and independent confidence intervals for each parameter set by (13) and (14), respectively (see Figure 3.4 in Additional file 1). As an example, Figure 5(c) shows the confidence intervals for the regulatory weight Whbcad MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdogaJjabdggaHjabdsgaKbaaaaa@33CA@ with fixed promoter thresholds. There is a quantitative improvement in the determinability of this parameter indicated by smaller independent confidence intervals in the case of fixed threshold parameters (compare panels (b) and (c) of Figure 5). But there are also qualitative changes. The model now predicts the activation of hb by Cad. Similarly, Figure 5(f) shows the confidence intervals for the regulatory weight WgtKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUealjabdkhaYbaaaaa@328D@ with fixed promoter thresholds. Comparison of the panels (e) and (f) of Figure 5 shows that there is no quantitative difference between the two approaches for this weight.

However, we see a qualitative improvement for the case of fixed threshold parameters. The independent confidence intervals in Figure 5(f) lie in the negative part of the plane for almost all parameter estimates and therefore, repression is now predicted for this weight while the plot in Figure 5(e) corresponding to full search case predicts only the absence of activation.

Based on the confidence intervals, we summarize the qualitative conclusions for the essential regulatory weights in the gap gene model in the case of fixed promoter thresholds:

(C1) Cad activates gap genes hb, Kr, kni, and gt;

(C2) Bcd does not repress gap genes hb, Kr, and gt; no conclusions can be made for regulation of kni by Bcd;

(C3) gap genes hb, Kr, and gt have auto-activation; gap gene kni does not have auto-repression;

(C4) Tll does not activate gap gene gt; no conclusions can be made for the regulation of Kr and kni by Tll;

(C5) mutually exclusive gap genes gt and Kr repress each other; no conclusions can be made for regulations between hb and kni.

For interaction among overlapping gap genes, the confidence intervals in the case of fixed promoter thresholds are fully consistent with those for the full search case, even though three of these interactions fall into different categories in the analysis based on parameter values only (compare panels (a) and (b) of Figure 4). This shows that confidence intervals can be used to check the significance of ambiguities in predicted interactions based on parameter classification alone. However, although conclusions (C1)–(C5) show qualitative improvement for some regulations in comparison with (B1)–(B4), they are still weaker than those drawn from classifying parameter values only (A1)–(A4).

Similar to the full search case, we compute the mean correlation matrix to detect the significant correlations between parameters (see Figure 3.6 in Additional file 1). The obtained mean correlation matrix also has a block diagonal structure. However, there is a number of significant entries in off-diagonal blocks. Panels (e),(f),(g), and (h) of Figure 6 show the diagonal blocks corresponding to gap genes hb, Kr, gt, and kni, respectively. In the absence of dominating correlations between regulatory parameters and thresholds h_a we can now identify biologically significant parameter correlations. Here we restrict ourselves to describe some correlations which can be interpreted in biological terms with the emphasis on those for which at least one parameter is 'sloppy':

• Strong negative correlation is present between Wknihb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdIgaOjabdkgaIbaaaaa@33FE@ and m_kni. That is, strong repression of kni by Hb needs to be overcome through increased activation by Bcd. Note that both parameters are poorly determined. In the circuit with Wknihb MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUgaRjabd6gaUjabdMgaPbqaaiabdIgaOjabdkgaIbaaaaa@33FE@ set to zero, Bcd actually represses kni (see Table 2.1 in Additional file 1). This contradicts genetic and molecular evidence indicating that both repression of kni by Hb and its activation by Bcd are present in the embryo 2930.

• There are complex correlations between the (very small, or absent) repressive effects of Hb on Kr and gt, and the activation of those two genes by Bcd. This confirms earlier results indicating that the balance between activation and repression from maternal genes is crucial for correct gap gene expression 31.

• The importance of the balance between activation and repression is highlighted by the following: repression of kni and gt by Tll can be compensated through increased activation by Cad, repression of kni by Kr can be compensated through increased activation by Bcd, while repression of kni by Gt can be overcome by increased kni auto-activation in the posterior of the embryo.

• Increased hb auto-activation is compensated through decreased activation of hb by Bcd indicating that broad maternal activation and auto-regulation are somewhat redundant.

• There is a strong positive correlation present between m_Kr and m_gt. This correlation is most likely indirect, due to repressive interaction between gt and Kr. Increased activation of Kr by Bcd must be balanced by increased activation of gt by Bcd to maintain balance of mutual repression between Kr and gt.

• There are correlations between activation of Kr and gt by Bcd and their respective promoter strengths and decay rates. Such correlations are to be expected as stronger expression or increased protein stability can compensate for weaker activation by Bcd.

We note that some of the 'sloppy' parameters, such as WgtKr MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdEgaNjabdsha0bqaaiabdUealjabdkhaYbaaaaa@328D@, WKrgt MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdEgaNjabdsha0baaaaa@328D@, Whbkni MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdIgaOjabdkgaIbqaaiabdUgaRjabd6gaUjabdMgaPbaaaaa@33FE@, and WKrtll MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aa0baaSqaaiabdUealjabdkhaYbqaaiabdsha0jabdYgaSjabdYgaSbaaaaa@33F8@ are not (strongly) correlated to any of other parameters and their sloppiness remains unclear. The last is completely uncorrelated parameter. Posteriorly Kr is strongly repressed by Gt and somewhat weaker by Hb and Kni. Apparently, due to these interactions repression of Kr by Tll is somewhat redundant in the model.

In summary, the above suggests that complex correlations between regulatory weights as well as correlations between those weights and promoter strength or protein decay rates are an unavoidable property of complex biological networks, as some interactions or changes in expression rate can always compensate for changes in others.

Poor determinability of most of the parameters in the gap gene model is due to correlations between parameters. Here we investigate whether these correlations are caused by shortcomings of the data or the model.

At first glance, it seems that insufficient accuracy of the data cannot be the reason for correlations. More accurate data would simply make the ellipsoid confidence region shrink but not rotate. Therefore, it cannot significantly improve the determinability of the parameters (see also 5). We checked this by assuming that a larger data set was available: Say we had measurements for all gene products, in all nuclei, at 71 uniformly distributed time points (instead of 9). With these choices the total number of measurements would be N = 21180. Suppose that we have found that one of our parameter estimates θ^ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafqiUdeNbaKaaaaa@2D9B@ minimizes the sum of squares (2). Since the Jacobian depends only on the model responses and not on the values of the data, we can generate a new Jacobian J˜(θ^) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOsaOKbaGaacqGGOaakcuaH4oqCgaqcaiabcMcaPaaa@3079@ including all 'ghost' data points. From the SVD of the corresponding J˜(θ^) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOsaOKbaGaacqGGOaakcuaH4oqCgaqcaiabcMcaPaaa@3079@ we get the matrices V˜(θ^) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOvayLbaGaacqGGOaakcuaH4oqCgaqcaiabcMcaPaaa@3091@ and Σ˜(θ^) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafu4OdmLbaGaacqGGOaakcuaH4oqCgaqcaiabcMcaPaaa@30E0@ which define new ellipsoidal regions. The ellipsoids are slightly rotated in comparison with the initial ones but not enough to make the principal axes of the ellipsoid get closer to the parameter axes, i.e. the correlations between parameters are not removed.

Each data point is actually a sample mean, obtained by averaging gene concentrations from individual embryos. Therefore, measurement errors most likely have a normal distribution with zero mean. However, their standard deviations may vary for different data points. Assume that for the i-th data point K_i measurements from individual embryos are used and assume that the standard deviation of this sample s_i is known. Then the normal distribution of the sample mean has a standard deviation which can be estimated by σi=siKi MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdMgaPbqabaGccqGH9aqpjuaGdaWcaaqaaiabdohaZnaaBaaabaGaemyAaKgabeaaaeaadaGcaaqaaiabdUealnaaBaaabaGaemyAaKgabeaaaeqaaaaaaaa@3663@. For the dataset we used, both s_i and K_i are available from the FlyEx database 11. Once all σ_i are found, we can use a weighted least squares estimation such that θ minimizes the sum

S ( θ ) = ∑ i = 1 N w i 2 ( y c i ( t i , θ ) − y ˜ i ) 2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4uamLaeiikaGIaeqiUdeNaeiykaKIaeyypa0ZaaabCaeaacqWG3bWDdaqhaaWcbaGaemyAaKgabaGaeGOmaidaaOGaeiikaGIaemyEaK3aaSbaaSqaaiabdogaJnaaBaaameaacqWGPbqAaeqaaaWcbeaakiabcIcaOiabdsha0naaBaaaleaacqWGPbqAaeqaaOGaeiilaWIaeqiUdeNaeiykaKIaeyOeI0IafmyEaKNbaGaadaWgaaWcbaGaemyAaKgabeaakiabcMcaPmaaCaaaleqabaGaeGOmaidaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabd6eaobqdcqGHris5aaaa@4F1E@

instead of (2). We take the weights w_i inversely proportional to σ_i such that the weighted least squares yields the maximum likelihood estimate. Also in this case, we find that the obtained parameter estimates have the same type of correlations as those obtained with an ordinary least squares fit (data not shown).

Correlations between parameters can be due to hidden dependencies in the data set. To investigate whether this is the case, we conduct an inverse experiment. We choose one of the parameter sets obtained by the LM search, with an RMS = 8.38, and we denote it by θ*. By integrating the model equations with θ* we generate an exact data set at the same data points as the initial data set. To the exact data values we add errors drawn from the normal distribution with zero mean and standard deviation equal to 8.5. From the exact and the perturbed data set, we compute RMS(θ*) = 8.17. The perturbed dataset is used for the parameter estimation by means of the LM search. By constructing this inverse problem, we make sure that the assumption about the independence of the measurement errors is correct. With 40 different initial values of θ from 12 we obtain 34 parameter estimates having RMS between 7.95 and 8.25. Inspection of the corresponding V matrices shows that parameters are not determinable due to the correlations, similar to the original problem.

We conclude that the observed correlations between parameters are a property of the model, not of the data. Since an explicit form of the dependence of the state vector on the parameters is not known, the use of reparametrization techniques is not feasible. Note that the majority of parameters in (17) appear in the argument of the sigmoid regulation-expression function Φ. If the model (17) is used to obtain only qualitative information, such as the signs of regulatory weights, then the particular mathematical form of this function is of no importance 25. However, it has to be studied if the choice of the sigmoid function affects the determinability of parameters.