Fraunhofer-Institut für Techno- und Wirtschaftsmathematik (ITWM; Fraunhofer Institute for Industrial Mathematics), Kaiserslautern, Germany

Department of Medicine, University of Helsinki, Helsinki, Finland

Department of Mathematical Sciences, University of Gothenburg, Gothenburg, Sweden

Wallenberg laboratory, Sahlgrenska Center for Cardiovascular Research, Department of Medicine, University of Gothenburg, Gothenburg, Sweden

Fraunhofer Chalmers Centre (FCC), Gothenburg, Sweden

Abstract

Background

When mathematical modelling is applied to many different application areas, a common task is the estimation of states and parameters based on measurements. With this kind of inference making, uncertainties in the time when the measurements have been taken are often neglected, but especially in applications taken from the life sciences, this kind of errors can considerably influence the estimation results. As an example in the context of personalized medicine, the model-based assessment of the effectiveness of drugs is becoming to play an important role. Systems biology may help here by providing good pharmacokinetic and pharmacodynamic (PK/PD) models. Inference on these systems based on data gained from clinical studies with several patient groups becomes a major challenge. Particle filters are a promising approach to tackle these difficulties but are by itself not ready to handle uncertainties in measurement times.

Results

In this article, we describe a variant of the standard particle filter (PF) algorithm which allows state and parameter estimation with the inclusion of measurement time uncertainties (MTU). The modified particle filter, which we call MTU-PF, also allows the application of an adaptive stepsize choice in the time-continuous case to avoid degeneracy problems. The modification is based on the model assumption of uncertain measurement times. While the assumption of randomness in the measurements themselves is common, the corresponding measurement times are generally taken as deterministic and exactly known. Especially in cases where the data are gained from measurements on blood or tissue samples, a relatively high uncertainty in the true measurement time seems to be a natural assumption. Our method is appropriate in cases where relatively few data are used from a relatively large number of groups or individuals, which introduce mixed effects in the model. This is a typical setting of clinical studies. We demonstrate the method on a small artificial example and apply it to a mixed effects model of plasma-leucine kinetics with data from a clinical study which included 34 patients.

Conclusions

Comparisons of our MTU-PF with the standard PF and with an alternative Maximum Likelihood estimation method on the small artificial example clearly show that the MTU-PF obtains better estimations. Considering the application to the data from the clinical study, the MTU-PF shows a similar performance with respect to the quality of estimated parameters compared with the standard particle filter, but besides that, the MTU algorithm shows to be less prone to degeneration than the standard particle filter.

Background

Measurement time uncertainties

Uncertainty in the time at which a measurement is taken is an often neglected source of random error. While in many application areas, this kind of error is generally small and indeed neglectable (due to automated measurements and precise timings), in others it may be of real influence, especially in the life sciences. As a prominent example, one may consider pharmacokinetic and pharmacodynamic (PK/PD) models which are used to describe the metabolic interactions and the effects of a chemical agent (like a drug or a labelled substance) over time inside an organism, respectively.

A typical population experiment in the PK/PD context consists in the analysis of the contents of the blood plasma of several individuals with respect to concentrations of certain molecules of interest. For this purpose, blood probes have to be taken from each individual at certain (fixed) time points after a certain event has occurred (e.g. a drug or a labelled substance has been applied). It is clear from the setting of the experiments that there is some variation in the real point in time when the blood probe has been taken: the true time when the measurement value has been obtained might be shortly before or after the intended time, and this true measurement time is not known to us. Since the inclusion of those time uncertainties in the model usually makes the analysis more difficult, it is standard to lump the time uncertainties with the measurement error. But especially at early times when concentrations change quickly, this may easily lead to wrong estimations, even if one assumes very high variances of the measurement error (we will demonstrate this later on a simple example). On the other hand, the inclusion of measurement time uncertainties (MTU) in algorithms aiming at inference making in complex models is not straightforward. In this article, we will present a modification of the Particle Filter (PF) algorithm (which we call MTU-PF) which is able to fully include a statistical model of the time uncertainties.

Inference in complex systems

The assessment of the effectiveness of a drug in a clinical study has been done in the past by the direct computation of relatively simple statistical values. The enormous increase in complexity of the underlying models, due to present developments in medicine and biology, for instance in the areas of personalized medicine or systems biology, increases also the need for more sophisticated model-based inference methods.

The estimation of unobservable internal variables or model parameters from data which have been obtained from blood or tissue samples at several time points can reveal information on the concentrations and effectiveness of the substance under question. If these data come from individuals which belong to two different (or even more) groups, e.g. test and control group, mixed effects are introduced in the underlying models. The inherent non-linearity and high variability of biological processes adds considerably to the difficulties one faces during the inference step. Inference in connection with dynamic models plays a major role in many other application areas. State and parameter estimation as well as model discrimination and validation are most common, but also optimal control problems should be mentioned.

It is often not enough to consider (independent) measurement noise

Parameter estimation with Maximum Likelihood approach

Parameter estimation in state space systems is a difficult problem. In a context where the system dynamics are modelled by Ordinary Differential Equations (ODEs) without correlated noise, the problem is most often considered as a (deterministic) optimization problem based on a Maximum Likelihood (ML) formulation. An overview of these approaches can be found in

Parameter estimation in a Bayesian context

In a Bayesian context, in contrast to the “classical” ML approach, a prior distribution is assigned to the parameter vector, hence the parameters can be treated as random variables. In this sense, parameter estimation is done by evaluating the so-called posterior distribution which can be computed (at least theoretically) by Bayes’ theorem given the observations (measurements) and the prior distribution. In the context of high-dimensional spaces, this requires the computation of high-dimensional integrals which is not possible to do analytically. For this purpose, Markov Chain Monte Carlo (MCMC) methods provide powerful tools for the computation of simulation-based approximations to the posterior distribution. Again, in the context of the joint estimation of dynamic states and fixed parameters, the design of good proposal densities is a very difficult problem which renders the use of standard MCMC methods like the Metropolis-Hastings sampler impractical for the purposes of parameter estimation in state space systems.

It has long been a wish to combine both (dynamic) SMC and (static) MCMC methods to provide a general tool for the joint estimation of dynamic states and static parameters. Only recently, Andrieu et al.

In the present article, even though the PMCMC approach might be the preferred method for parameter estimation in state space systems, we will concentrate solely on the SMC methods, since our modification affects only this part. However, to be able to do parameter estimation in a pure SMC context, we rely on an approach that is very often used to avoid problems with the estimation of constant parameters. This approach consists in the introduction of artificial dynamics in the parameters, that means the parameters are allowed to slightly change their values over time. In this way, and in a Bayesian context, the parameters can be treated exactly in the same way as the system states. After building an augmented system state by concatenating the parameter vector and the state vector, the joint estimation of states and parameters reduces to filtering of the augmented state vector which makes SMC methods directly applicable to the problem.

Particle filters for state and parameter estimation

Particle filters (

Non-Linear Mixed Effects models

Estimation in a Non-linear Mixed Effects model (NLME) involves the estimation of both global and individual parameters. With classical maximum likelihood estimation, the individual parameters are random variables equipped with a distribution while the global parameters remain constants with a “true” but unknown value. If the underlying model equations are non-linear, this leads to likelihood functions which are not analytically accessible and one has to rely on approximations. In the context where the system dynamics are modelled by ODEs, the most popular algorithm for NLME parameter estimation in the PK/PD context is the tool NONMEM (

On the other hand, in a Bayesian context, also the global parameters are equipped with a (prior) probability distribution, and the conceptual difference between global and individual parameters vanishes. The mixed effects model can then be considered as a hierarchical model with dependent parameters (

Aim of the article

Our goal is two-fold: Firstly, we want to show that the particle filter algorithm is applicable (with our modifications) also to more complex models when time uncertainties are formulated explicitly. Secondly, we want to show that the modification may even provide the possibility for further enhancement of the performance of the algorithm by presenting an adaptive time-stepping scheme which is only possible in the context of the new algorithm.

We do not claim that our MTU algorithm generally performs better or worse than the standard filter, nor that it should be the preferred method for estimation in non-linear mixed effects models. Rather, we provide a method which is usable for models where time uncertainties may play a major role. In these cases, it may indeed lead to better estimations. On the other hand, our method transfers the time-discrete particle filter approach, where updates based on the measurements very strictly depend on the measurement times, to a truly time-continuous approach, where updates to the filtering distributions can be performed at every point on the time-scale. Since we want to focus on the time uncertainties, we neglect discussing further issues like identifiability, model evaluation and model discrimination. In our application to the model of plasma-leucine kinetics, we try to avoid these issues by providing ad-hoc values to some of the parameters (especially to the variances of the system states).

Motivating example

Let us have a look at an example for illustrating the benefits of a separate modelling of measurement time uncertainties. Let us consider a state space system given by the ODE

with parameters

Furthermore, let the initial state

true value of

1

true value of

3

0.05

log(1)

0.1

_{
y
}

0.005

distribution of _{
j
}

_{0}

We assume that _{
j
} and that each measurement _{
j
}) and with a fixed variance _{
j
} are assumed to be known. In contrast, we will assume that in addition to the measurement value error, there is some uncertainty about the exact times where the measurements have been taken. If we attempt to take the _{
j
} which may be shortly before or after the intended time _{
j
} is given as a realization of a random variable _{
j
}. In our example, we assume that _{
j
} follows a truncated normal distribution given by the density

with normalizing constant

and given intended measurement times _{
j
} in dependence of the intended measurement time _{
y
}: in (b), the original standard deviation is used, while in (c) and (d), higher standard deviations are used which correspond to the cases with lumped value and time variations.

Assumed measurement distributions for the motivating example

**Assumed measurement distributions for the motivating example.** Measurement distribution resulting from (**a**) separate modelling of measurement time uncertainties and measurement value uncertainties, with _{y}=0.005, and (**b**-**d**) lumped time-and-value uncertainties with several different assumed lumped measurement variances _{y}. The dashed dark-green line depicts the nominal evolution of the state

Comparing Figures

Methods

We divide this section into three subsections. In the first subsection, we fix the state and observation model we want to consider. In the second subsection entitled “Standard case” we outline the standard particle filter algorithm in the context of time-continuous states with time-discrete measurements, and the various probability distributions involved. Although nothing is new in this subsection, it serves several purposes. Firstly, the time-continuous case is relatively rarely considered in the literature; secondly, the derivation of our modification needs a slightly more general formulation than it is standard for the discrete-time filter; and lastly, the comparison of our modified version with the standard case might more clearly reveal the differences between the two approaches. In the third subsection entitled “MTU particle filter”, we present our new modification of the particle filter. In the following section “Results and Discussion”, we compare the new MTU particle filter to the standard particle filter and to an alternative Maximum Likelihood estimation method on a simple artificial example. We also present an application of our MTU-PF method to a PK/PD study in a non-linear mixed-effects setting in direct comparison with the standard particle filter.

Note: a list of all used symbols with a short explanation can be found at the end of this paper.

The model

State process

Let _{0},_{0},

For each _{0},_{
t
}, i.e.

the state space restricted to the interval [_{0},_{0}, let _{
s,t
}(_{
s
}, d_{
t
}) be the Markov kernel of the process

An important special case for

with drift _{
s,t
} when a suitable discretization method is applied, for instance the Euler-Maruyama method.

Observations / measurements

Let the process _{1:M
} with values in measurable spaces _{
j
} depends on the state variable _{
j
} and on the observation time (measurement time) _{
j
} itself. We assume that, given the observation time _{
j
} and the state _{
j
} is independent of all other variables, and the conditional measure can be expressed via some conditional probability density

Observation / measurement times

The observation times (measurement times) _{
j
} for _{
j
} are themselves realizations of random variables _{
j
}. These variables model the uncertainty about exact observation times. In contrast to the observation variables _{
j
}, the observation times _{
j
} are never observed (measured). We assume that all information available to us is their probability distribution on the half axis [_{0},_{
j
}, we know both the densities _{
j
}.

In this article, we will only consider the simplest case where each variable _{
j
} is independent of all others. Dependencies between the _{
j
}’s, especially concerning the order of the observation times, may be considered natural but would lead to more complicated algorithms. However, order dependencies can easily be introduced via restrictions on the support of the variables. In general, the probability distribution of every single variable _{
j
} shall be given by a density _{
j
}(_{
j
}) with respect to the Lebesgue measure _{0},

In the following, we will consider the two cases mentioned, where either all _{
j
} are deterministic and known or all _{
j
} are random and unknown. Note that the first case formally coincides with the case that _{
j
} is random but observed. We will therefore stick to the notation

Standard case: measurement times deterministic and known

We will first consider the standard case, where the observation times _{
j
} are known. For simplicity, we assume here that the observation times _{1:M
} are strictly ordered increasingly, i.e. _{0}<_{1}⋯<_{
M
}.

The standard case of the particle filter is usually formulated for discrete-time Markov processes _{0} and at the times _{1},…,_{
M
} when measurements occur. Nevertheless, this case is included in our more general framework where _{
t
} is defined for all _{0}. One just focuses on the state variables for those times only. In view of the later generalization to random observation times, we will consider the fixed values _{
j
} as realizations of random variables _{
j
} and condition all occurring densities on them. As mentioned above this assumption leads to the same results as if we assumed the values _{
j
} to be given deterministically.

Full model and filter model

The full model is given by the joint density of the variables _{1:M
} (conditioned on the observation times _{1:M
}=_{1:M
}) with respect to the product measure

The filter at a given time _{
k
} is based on a reduced model. This model is given by the joint density of the variables _{1:k
} (conditioned on _{1:M
}=_{1:M
}) with respect to the product measure

This density is based on the state sequence _{1:k
} (given _{1:M
}=_{1:M
}) with respect to

and the filter density at time _{
k
} with respect to

with

For general (non-linear) models, the practical computation of the filter density is very difficult. Nevertheless, the particle filter computes a Monte Carlo approximation using the fact that the filter densities _{
k-1} given by the probabilities

for each set _{1:k-1} (and _{1:M
}), by use of the kernel

for each set _{
k
}:

for each set

Importance sampling

Another ingredient for the particle filter is sequential importance sampling. We assume that a second Markov chain

exists. We further assume that the pushforward measure

For sequential importance sampling, we need to be able to sample from the initial measure

for each

Using

we can then write the recursive formula (8) for the filter distribution at time _{
k
} as

for each _{1:M
} is assumed to be fixed) is not necessary. Sequential importance sampling is performed as follows. Draw a number

Then, for all

For suitable integrable functions _{1:k
}=_{1:k
}, given by

can then be approximated by

where

Note that if we can sample from the Markov kernels of

Resampling

If the number

where

are the normalized weights. It obtains its maximal value _{Threshold} (which is usually chosen to be

Resampling at some time _{
ℓ
} is based on given non-negative (unnormalized) selection weights

This is multinomial resampling. There exist procedures where each single particle is still selected with probability _{
ℓ
}:

(using (16)). The necessary correction is therefore achieved if the unnormalized weights

Note that in the original particle filter, the selection weights _{
ℓ
} are chosen to be the particle weights (before the replacement), i.e.

such that after the resampling step the unnormalized weights are all equal to 1. Nevertheless, in general their choice is free and may be based on the observations (which is used in the so-called auxiliary particle filter

Particle filter algorithm

The particle filter computes the state realizations and weights recursively through time. In its standard form, the particle filter can be stated as in algorithm 1.

Note that if one chooses

Data Likelihood

Model validation or discrimination is generally based on the data likelihood

Algorithm 1 Standard particle filter

for given observations _{1:k
}. Without resampling, the data likelihood could be approximated by the empirical mean of the unnormalized weights, i.e. by

because this is the empirical estimate for the above expectation. After a resampling step, this is not valid any longer. Nevertheless, in any case, the data likelihood can be computed recursively by the following estimate of the ratio

with initial estimate

MTU particle filter: Uncertain measurement times

We now assume that each observation time _{
j
} is a realization of a random variable _{
j
}. Its distribution is expressed via densities _{
j
} with respect to the Lebesgue measure _{
j
} themselves are not observed.

Full model

The full model in this case will include complete continuous state paths, since the observation times are now distributed over the complete time axis [_{0},_{
j
}∈[_{0},_{1:M
} and _{1:M
}, with respect to the product measure

Filter model

The filter at a given time _{0} is again based on a reduced model. This model is given by the joint density of the following variables: _{
j
} for which _{
j
}≤_{1:M
}. This density is given with respect to the product measure

Note that we cannot use the simple notation of the standard case where for filtering only the first _{
k
}, since neither the observations are ordered in time nor the times _{
j
} are fixed in advance. For this reason we have to include all measurements _{1:M
} also into the filter model. Note that even though we use the complete data _{1:M
}=_{1:M
} in the notation, only those _{
j
} have to be known at time _{
j
}≤

We will now derive formulas for the filter density. Since we assume that the observation times _{1:M
} are not observed, we use marginalization to get the joint density for _{1:M
} only, which is

with respect to the product measure

then

and further

where the last step is possible because the factor indexed by ^{′}≠_{
j
} at the time point

where the last step follows from the fact that _{
j
} is a probability density and therefore

holds. Inserting this into (24), we get

With a further marginalization, we get the joint density of _{
t
} and _{1:M
} for the filter model,

which is with respect to the product measure

where

is the data likelihood with respect to the measure

Effective computation of the filter distributions

In the following paragraph, we will show how the densities of the filter distributions given by (26) can be effectively computed. This is the basis for the formulation of our MTU particle filter method.

Let the observations _{0},

by the following system of ODEs

for each

and by

We will show that for each set _{
t
}, it holds that

where _{
j,t
} and _{
t
} to compute the filter distributions through time. From this, it follows immediately that we can also compute filter expectations. Indeed, for any real-valued measurable function _{
t
})|]<_{
t
}) given _{1:M
}=_{1:M
} with respect to the filtered state _{
t
} defined by

is given by the following equation:

To show our assertion, we consider the processes _{
j,t
} for _{
j,t
} is defined as

so, with (30),

holds. Thus, to prove (31), we have to show that for each set _{
t
},

holds (see (25) and (26)). It is enough to show the equality for the numerator, i.e.

since the equality of the denominator follows then immediately from the special case

Using the variable transformation

This is what we wanted to show.

Weights

Since for each _{
j,t
} and _{
t
} depend only on the process

It follows from (34) that

for each

The values of _{
t
} will serve as weights in the MTU particle filter. We will call _{
j,t
} the partial weights. Since in each discretization scheme which is applied to solve the integral in the formula (36) for _{
j,t
} the integrand has to be evaluated, we may run into practical problems if we use it as it is written in the formula. If the density

where the cumulative distribution function

is independent of

depends on the path _{
j
}, if it is computationally available.

Note that the definition of the filter distribution is dependent on the reference measure

Resampling

Special attention is needed for the computation of the weights after resampling steps have been applied. As mentioned earlier, resampling at time _{
ℓ
} is done by randomly generating a selection function _{
ℓ
}:_{1},…,_{
ℓ
} with _{0}≤_{1}⋯<_{
ℓ
}≤_{
ℓ
}

and

For each time _{
ℓ
} and for each particle

with

Since the process _{
t
} computes the uncorrected weights

Note that we have to select these products during a resampling step similarly to the selection of the states, i.e. at _{
ℓ
},

The MTU particle filter algorithm

A practical MTU particle filter can be obtained by any discretization scheme based on an (arbitrary) time discretization _{0} = _{0}
_{1}…_{
D
}. Similar to the standard particle filter, sampling need not necessarily be done from the state process

for each _{0},_{
t | s
}(_{
t
} | _{
s
}) exists for all states _{
t | s
}≡1 for all _{1},…,_{
ℓ
} in the notations of states and weights (see last paragraph).

Algorithm 2 MTU particle filter

The algorithm as it is written here has to be enriched with concrete discretization methods for the sampling and update steps. For instance, if the process

with drift _{
d
}: =_{
d
}-_{
d-1}. The sampling is then done by

where ^{
i
} is a sample from a standard normal distribution (with mean 0 and variance 1). Further, the update step can be done in the simplest case using Euler discretization:

and

Of course, better discretization schemes are possible. As we have already mentioned, if the antiderivative _{
j
} with _{
j
}(_{0})=0 (which is in fact the distribution function) of _{
j
} is available, then one should rather use

for the computation of the values

Adaptive stepsize

To be able to fully exploit our MTU particle filter method, the discretization stepsize must be chosen appropriately. One simple possibility is to use a very small stepsize throughout the complete procedure. A quite high computation time will result from that. This can be reduced if an adaptive stepsize is chosen. We propose to determine the stepsize _{
d
} online depending on the ESS estimate. The stepsize should decrease when the ESS drops rapidly, and it should increase again if the ESS estimate changes only marginally. In detail, the following procedure can be applied.

In each step of the algorithm, we obtain an initial guess of the stepsize by a linear interpolation between a maximal stepsize if the ESS had not changed since the last step, and a minimal stepsize if the ESS had dropped by the number

Algorithm 3 Determination of the adaptive stepsize

If we sample from the Markov kernels of

Note that this procedure cannot be performed when the measurement times are fixed (i.e. in the standard particle filter). In this case the ESS does not depend on the stepsize, and a reduction of it will not improve the ESS. The application of the MTU particle filter with distributed measurement times is essential to be able to use this adaptive stepsize procedure.

Data Likelihood

As mentioned earlier for the standard case of the particle filter algorithm, without resampling, the data likelihood could be approximated by the empirical mean of the unnormalized weights (see (19)). In our case, this would be

After each resampling step, we have to correct this formula. Using the same notation as in the paragraph on resampling, for each time _{
ℓ
} and for each particle

with

(see (41)). This can be seen by considering recursively the correction at the time of the resampling step

is the expected number of times the particle

We get algorithm 4 for the computation of the empirical estimate of the data likelihood, which needs to be done in parallel to the MTU particle filter algorithm.

Algorithm 4 Estimation of the data likelihood

Offline and online estimation

Two main cases of estimation procedures may be distinguished: the offline estimation procedure used for example for parameter estimation, and the online estimation procedure for control purposes. While in the offline case the measurements are completely available before estimation begins, we know only some of the measurement values at some certain time _{0}. Online estimation is nevertheless possible if the support of the measurement time densities is finite. The online estimation must then be delayed by the diameter of the respective supports.

Implementation

We have implemented the proposed algorithm in Mathematica as part of a Parameter Estimation Toolbox for Systems Biology developed by the Systems Biology group at the Fraunhofer Chalmers Centre (FCC) in Göteborg (Sweden). Furthermore, we have implemented it also in the statistical computing language R

Results and Discussion

Motivating example - results

In this section, we resume our motivating example and use it in a parameter estimation setting to compare our MTU filter to both the standard particle filter and to a state-of-the-art Maximum Likelihood (ML) method which is not based on Monte Carlo techniques. To this aim, we first create virtual “measurements” at four intended measurement times _{
t∈[0,10]} based on the “true” parameter values _{
j
} according to the density _{
j
}. Now, for each _{
j
}, we sample a measurement value _{
j
} from the distribution

1. ML estimation on “lumped” measurements,

2. Standard PF on “lumped” measurements, and

3. MTU-PF.

Measurement distribution with a set of possible measurements

**Measurement distribution with a set of possible measurements.** The dashed dark-green line depicts the nominal evolution of the state

Note that in both the standard PF and the ML estimation method, it is not possible to include the uncertainties in the measurement times directly. Rather, we have to lump these uncertainties with the uncertainties in the measurement values by increasing their variances

We have implemented the standard particle filter and the MTU particle filter in the statistical computing language R

ML estimation

The ML estimation is a standard method based on a Maximum Likelihood approach, that is, on the maximization of the likelihood function _{1:M
}) with respect to the parameter vector

Given measurements _{1:M
} at a total of _{
j
} which are assumed to be fixed and known, the likelihood function can be written as

where _{
j
} given all previous observations _{1:j-1} as given in (9), here explicitly based on the given parameter vector

and

for the prediction of the mean and covariance of the observation variables, respectively. The computation of these values can be achieved by some derivates of the Kalman filter, commonly used are the Extended Kalman Filter (EKF) or the Unscented Kalman Filter (UKF), more exactly those versions of these filters which are time-continuous in the states and time-discrete in the measurements (continuous/discrete EKF and UKF, respectively). The residuals _{
j
} are then defined by the differences between the measurements _{
j
} and their estimations

The assumption of normally distributed observations leads to an approximation of the likelihood function of the following form

where

The problem to finding maximum likelihood estimates of the model parameters takes the form of a nonlinear optimization problem:

Roughly, the estimation with this approach is done as follows:

1. Choose an initial guess ^{∗} for the parameter vector

2. Based on the current parameter vector ^{∗} and for each (intended) measurement time _{
j
}, compute residuals _{
j
} and estimates of the covariance matrices _{
j|j-1} using (an approximation to) the Kalman filter.

3. Compute the likelihood _{
j
} and _{
j|j-1}.

4. Use one step of some local optimization technique to find an improved parameter vector ^{∗} by this improved parameter vector

5. Repeat steps 2 to 4 until the parameter vector shows convergence.

The results of several runs of parameter estimations for different values of the lumped measurement variances _{0} for the states from the correct distributions (see Table _{
j
}. Figure

Parameters ** α** and

**Parameters**** and**** estimated by ML estimator.** Box plots of estimated parameters of 100 runs each. The medians are depicted with circles. The bottom and top of the box are the 0.25-quantile and the 0.75-quantile, i.e. 50% of the values lie within the box. The whiskers mark the 0.025-quantile and the 0.975-quantile, i.e. 95% of the values lie between the whiskers. The horizontal red lines denote the true parameter values.

prior for estimation of

log-

prior for estimation of

log-

stepsize for standard particle filter

10^{-2}

maximal stepsize for MTU particle filter

10^{-2}

minimal stepsize for MTU particle filter

10^{-6}

diffusion parameter for artificial parameter noise at time

5.43/(^{2}

The estimated parameters of all estimation runs are far away from the true values; even quite large variances

Estimation with particle filters

Tests with the particle filters (both standard and MTU) have been performed in the following way: First we perform an

Table

**Data log-likelihood**

**Data log-likelihood**

**
σ
**

**
α
**

**
β
**

**Estimation run**

**Simulation run**

The true parameters are

MTU

0.005

1.012

3.010

-4.327

2.398

standard

0.005

7.031

20.712

-24.084

-744.970

standard

0.01

9.102

26.919

-23.936

-890.746

standard

0.1

4.709

13.847

-9.081

-140.117

standard

0.25

1.425

4.171

-6.714

-4.618

standard

0.5

1.156

3.287

-5.538

-2.170

standard

0.75

1.318

3.604

-5.807

-3.160

standard

1

1.450

3.733

-6.227

-4.100

In Figure

Filtered state distributions based on simulation runs with estimated parameters for the motivating example

**Filtered state distributions based on simulation runs with estimated parameters for the motivating example.** MTU particle filter (**a**) and standard particle filter with several different assumed lumped measurement variances _{y} (**b**-**d**). Violet shaded area: Filtered state distributions based on empirical quantiles. Solid blue line: Median of filtered states. Dashed dark-green line: nominal evolution of the state

Figure

Simulated measurements corresponding to the filtered state distributions for the motivating example

**Simulated measurements corresponding to the filtered state distributions for the motivating example.** MTU particle filter (**a**) and standard particle filter with several different assumed lumped measurement variances **b**-**d**). Violet shaded area: Measurement distributions based on empirical quantiles. Solid blue line: Median of simulated measurements. Yellow circles: Measurements.

In addition to the significant improvement in the parameter and state estimation, the MTU particle filter has another benefit. Figure

Estimated effectice sample size during estimation run

**Estimated effectice sample size during estimation run.** MTU particle filter (**a**) and standard particle filter with several different assumed lumped measurement variances _{y} (**b**-**d**). The dotted blue line denotes the resampling threshold.

Application to the Plasma-Leucine Model with Population Data

In this section, we apply our MTU particle filter algorithm to a leucine kinetics model (Demant et al.

**Data used for estimation and simulation.** The data consist of one record for each patient, each record consisting of 4 lines in the following format:

patient id [COMMA] group (control or diabetes) [COMMA] leucine initial value [NEW LINE]measurement 1 time [COMMA] measurement 2 time [COMMA] …[COMMA] measurement _{
p
} time [NEW LINE]measurement 1 value[COMMA] measurement 2 value [COMMA] …[COMMA] measurement _{
p
} value [NEW LINE][NEW LINE]

where _{
p
} is the number of measurements for patient

Click here for file

The Leucine model

In

Multicompartmental model for apolipoprotein B-100 (apoB) and triglyceride (TG) metabolism in very low density lipoprotein (VLDL) subfractions

**Multicompartmental model for apolipoprotein B-100 (apoB) and triglyceride (TG) metabolism in very low density lipoprotein (VLDL) subfractions.** This multicompartmental model was developed in

Schematic depiction of the restricted model (leucine pool)

**Schematic depiction of the restricted model (leucine pool) [****].** This scheme is a subscheme of Figure

The secreted particles become denser and denser as triglycerides are delivered to target organs such as muscles and adipose tissue and the relative protein content is increased. As the density increases, the VLDL becomes an intermediate density lipoprotein (IDL) and finally a low density lipoprotein (LDL).

For our purposes we use only the part of the model concerning the leucine pool (compartments 1-4), see Figure _{1}.

The data are obtained from tracer/tracee experiments. Here the tracee (i.e. the concentration we are actually interested in) is the leucine amino acids in the apoB molecules. Additional labelled leucine (tracer) is injected as a bolus infusion. Knowledge about the kinetics (fluxes between compartments) of the tracee can be gained by studying the kinetics of the tracer. In the restricted model four compartments are considered: plasma leucine (1), intra-hepatic leucine (2), and two plasma protein pools (3 and 4). In the full model, additional compartments represent VLDL subfractions (compartments 5-11 in Figure

For each compartment _{
i
} and _{
i
} denote the mass of the tracee and the tracer, respectively. Similarly, let _{
i
} and _{
i
} denote the input for the tracee and the tracer, respectively. For tracer/tracee experiments, _{
i
} is assumed to be in steady state. If the concentration level of the labelled injection is small compared with the overall concentration levels, and if the model is linear, then approximately

where _{1}(^{T} and

where _{
j,i
} for

Throughout this paper, the time unit used is hours, all fractional transfer coefficients are given in the unit h^{-1}, and the amount of material in compartments is given in mg. In our model, only _{0,1}, _{1,2}, _{1,3}, _{2,1}, _{3,1}, _{3,4}, _{4,3} and _{11,2} are assumed to be non-zero, while additionally the following dependencies of the transfer coefficients are assumed to be valid:

The fractional transfer coefficient _{11,2} has to be fixed for the system to be identifiable. We set _{11,2}=0.01h^{-1}, as an estimated average from current results. We build stochastic differential equations (SDEs) from the resulting ordinary differential equations by adding noise terms which are given by standard Wiener processes _{1},…,_{4}, respectively. The leucine pool subsystem which we consider here (compartments 1-4) interacts with the surroundings only via an initial input into compartment 1 at time

We fix the diffusion parameters to be _{1}=_{2}=_{3}=_{4}=3. Initial conditions are given by

The patients get a bolus injection and therefore the input _{1}(

Practically, this means that only the initial condition _{1} is affected by it, and we can replace the initial condition for _{1} by

and set _{1}(

The same differential equations, without noise terms, are assumed to be valid for the states _{
i
} and the input _{1} of the tracee:

where

, _{1}(^{T}. It is assumed that the tracee input _{1}(_{1} is constant but unknown. We will therefore estimate it together with the transfer parameters. Since for the tracee steady state is assumed (i.e. d_{
i
}(_{1}(

The output is a measurement proportional to the ratio between the tracer and the tracee, disturbed by log-normal noise:

where we assume the value of the variance parameter to be _{
t
}). The parameter _{1} denotes the unknown proportion of plasma leucine that actually is in the plasma. The parameters _{1} and _{1} are not jointly identifiable, therefore we fix _{1}=0.65. More details concerning the deterministic model (without noise terms) can be found in

The mixed effects model

The model as presented in the last paragraph contains only flux parameters _{
j,i
} which are assumed to be the same for every individual. Neither does it account for individual differences between several persons, nor does it account for possible changes of the flux parameters when the persons under consideration are affected by a disease or a treatment. To be able to treat these differences in an appropriate way, we introduce group and patient specific parameters in the model; namely the transfer coefficient _{0,1} will be split into a group dependent and a patient dependent part. In this way, we introduce so-called mixed effects into the model. Mixed effects generally increase the difficulty in inference making. In the following test runs, we will use measurement data previously reported as individual data for a total of 34 subjects _{0,1} of plasma leucine is significantly different for people with and without diabetes. We therefore assume that the expected value of _{0,1} in each group differs and may be either _{
p
} modelling the parametric uncertainties among individuals, such that finally

where all _{
p
}’s are static and independently log-normally distributed:

for _{1},…,_{4} has to be considered separately for each patient

The aim of the estimation runs is, apart from estimating the remaining parameters, to show that the group dependent parameters _{
p
} which also are to be estimated. Our process _{
t
} is then given as an augmented state vector

The overall model is thus a non-linear mixed effects model with three levels of effects (parameters), namely global parameters, group dependent parameters (_{
p
}). Nevertheless, since the core of the model is linear, i.e. the states _{1},…,_{4} conditioned on all parameters, a Rao-Blackwellization concerning the linear parts of the model is possible, and the Kalman filter applied to this linear partial model can be used in combination with particle filtering for the non-linear parts

Estimation runs

Estimation and simulation runs have been performed with data from all 34 patients (19 from control group and 15 from diabetes group). The computer experiments have been done as follows. We first estimate parameters with the MTU particle filter. Separately, we estimate parameters with the standard particle filter under the same conditions and with the same seed for the random number generator. The initial distribution of the particles is then the same in both cases. For both runs, we compute an estimate of the effective sample size (ESS) and the data likelihood over time. Both estimates allow a performance comparison of the MTU versus the standard particle filter. Secondly, the empirical medians of the final parameter distributions are afterwards used to perform simulation runs in both cases. Both versions of the particle filter, this time with parameters fixed to the estimated values, are used to perform these simulations. In these simulation runs, the data are used for the computation of the data likelihood conditioned on the estimated parameters. The resulting distributions of the simulated measurements can be compared to the true measurements, both visually and by observing the data likelihood.

We used 10,000 particles and a resampling threshold of 7,500. Stepsizes in the MTU filter are between 10^{-7}h and 10^{-3}h, adaptively computed based on the ESS estimate. In the standard filter, we use a fixed stepsize of 10^{-3}h. The data contain measurements until time _{
t
} (i.e.

As mentioned earlier, Bayesian parameter estimation with particle filters requires the introduction of artificial dynamic noise for the parameters. It is standard to use normal increments with decaying variances _{
p
}’s are assumed to be positive, the application of a “log-normal” noise (based on a geometric Brownian motion) in place of the standard normal noise seems to be more appropriate for these parameters. In detail, it has been done as follows. The priors and the distributions of the artificial noise are chosen to be log-normal for all parameters with exception of the individual parameters _{
p
} which have normal priors and noise. The prior for the parameters _{1,2}, _{1,3}, _{3,1}, _{4,3} is _{1} is _{
p
} is

for _{1,2}, _{1,3}, _{3,1}, _{4,3}, _{1}, and according to

for _{
p
}. It is standard to decrease the variance of the artificial noise over time. In our case we have chosen the diffusion parameter _{
θ
} of each artificial noise variable to be dependent on time via a quadratic function

with parameter dependent coefficients _{
θ
} and _{
θ
}. Practically, _{
θ
} and _{
θ
} have been determined by fixing two interpolation points (_{0},_{
θ
}(_{0})) and (_{1},_{
θ
}(_{1})). It holds:

We found best performance with the following choices: For all parameters, we have chosen _{0}=0h, _{1}=2h and _{
θ
}(_{1})=_{
θ
}(_{0})/10 which means that the diffusion parameter has dropped to 10% of its initial value at time _{
θ
}(_{0})=0.5h for all parameters with exception of the _{
p
}’s which have higher initial diffusion _{
θ
}(_{0})=1h.

As mentioned earlier, we have performed two different estimation and simulation runs, one with the MTU particle filter with distributed measurement times, and for comparison one run with the standard particle filter. In the MTU particle filter, the distributions of the measurement times are truncated normal distributions with mean equal to the nominal value of the measurement time and with variance 0.001^{2}. The normal distribution is truncated at the time point 0.01h left and right from the mean value. In Figure

Development of estimated effective sample size and estimated data likelihood during estimation runs

**Development of estimated effective sample size and estimated data likelihood during estimation runs.** Standard particle filter (top) and MTU particle filter (bottom).

Predicted measurement distribution over time (standard particle filter) during simulation run. Patients from diabetes group

**Predicted measurement distribution over time (standard particle filter) during simulation run. Patients from diabetes group.** Development of predicted measurement distributions over time during simulation run with parameters estimated by the standard particle filter. Circles: Measurements. Solid line: Median of simulated measurements. Violet shaded area: Measurement distributions based on empirical quantiles.

Predicted measurement distribution over time (standard particle filter) during simulation run. Patients from control group

**Predicted measurement distribution over time (standard particle filter) during simulation run. Patients from control group.** Development of predicted measurement distributions over time during simulation run with parameters estimated by the standard particle filter. Circles: Measurements. Solid line: Median of simulated measurements. Violet shaded area: Measurement distributions based on empirical quantiles.

Predicted measurement distribution over time (MTU particle filter) during simulation run. Patients from diabetes group

**Predicted measurement distribution over time (MTU particle filter) during simulation run. Patients from diabetes group.** Development of predicted measurement distributions over time during simulation run with parameters estimated by the MTU particle filter. Circles: Measurements. Solid line: Median of simulated measurements. Violet shaded area: Measurement distributions based on empirical quantiles.

Predicted measurement distribution over time (MTU particle filter) during simulation run. Patients from control group

**Predicted measurement distribution over time (MTU particle filter) during simulation run. Patients from control group.** Development of predicted measurement distributions over time during simulation run with parameters estimated by the MTU particle filter. Circles: Measurements. Solid line: Median of simulated measurements. Violet shaded area: Measurement distributions based on empirical quantiles.

Estimated global and group parameters

**Estimated global and group parameters.** Box plots of estimated distributions for the global parameters and the group dependent parameters. The medians are depicted with triangles (standard particle filter) or circles (MTU particle filter). The bottom and top of the box are the 0.25-quantile and the 0.75-quantile, i.e. 50% of the values lie within the box. The whiskers mark the 0.025-quantile and the 0.975-quantile, i.e. 95% of the values lie between the whiskers. The values for _{1} have been scaled by a factor of 0.01 in order to fit into the plot.

Estimated individual parameters

**Estimated individual parameters.** Box plots of estimated distributions for the individual parameters. The medians are depicted with triangles (standard particle filter) or circles (MTU particle filter). The bottom and top of the box are the 0.25-quantile and the 0.75-quantile, i.e. 50% of the values lie within the box. The whiskers mark the 0.025-quantile and the 0.975-quantile, i.e. 95% of the values lie between the whiskers.

A comparison of the results of MTU-PF and standard particle filter shows that both algorithms exhibit very similar performance with respect to the quality of the estimated parameters, since in both cases the development of the data likelihood is very similar both for estimation and simulation. The estimated log-likelihood of the data at the final time is 137.239 for the MTU particle filter and 136.207 in the standard case, which is practically equal. The computation time of the MTU-PF is only slightly higher than the one of the standard filter. Visual inspection of the simulation runs shows that the simulated measurements of the model with parameters estimated by both filters fit the data in a similar way. This impression is supported by the values of the estimated data likelihood. The MTU particle filter has a final log-likelihood value of 157.622, similar to the one of the standard filter with a value of 155.952. The difference is insignificant.

In contrast to the insignificant differences concerning the resulting likelihoods between the MTU-PF and the standard PF, the development of the ESS estimate in the estimation runs differs remarkably. With the MTU particle filter, the ESS estimate shows a high value at all times and does not drop below a value of 7032.661 during estimation. This is only slightly lower than the resampling threshold of 7500 (see Figure

A look at the estimated values of the group parameters ^{-1} vs 0.557h^{-1}, MTU particle filter: 0.346h^{-1} vs 0.577h^{-1}). The good performance especially of the MTU particle filter strengthens the confidence in the obtained result and leads to the conclusion that the secretion rate _{0,1} is indeed lower for the diabetes patients than for the people from the control group.

Conclusions and future work

We proposed a new modification of the particle filter algorithm which works in continuous-time settings. It allows the direct inclusion of measurement time uncertainties in the underlying model. The modifications additionally allow the use of time-stepping strategies to improve the performance of the algorithm. The assumption of a random distribution of measurement times is natural in many applications.

The MTU-PF method is generally applicable. Even when measurement times may be assumed to be concentrated on single time points, our method can be used as a kind of regularization of the standard particle filter method if artificial distributions with highly concentrated masses around the measurement points are introduced.

We compared the performance of the MTU-PF to the standard PF and to an alternative Maximum Likelihood estimation method on a small artificial example. The results clearly show the advantage of the application of the MTU-PF in cases of uncertain measurement times.

We believe that our MTU particle filter is especially suitable for biological/medical applications where — compared to technical applications — the variance of the measurement values is relatively high due to biological variation and because relatively few consecutive measurements are possible. We provided an illustrative application from a PK/PD study. A comparison of our MTU particle filter with the standard filter showed that in this case our method is able to avoid weight degeneracy measured in terms of the Effective Sample Size (ESS) estimate. Even though the estimations in this application with both standard and MTU particle filters are reasonably good, the fit to the measurements is still not perfect. Whether that is due to our choices of model and parameters, or due to the known weaknesses of the general SMC method (which also our modification necessarily suffers from), remains to be evaluated in greater detail. In subsequent studies, we plan to apply the new algorithm to the complete liver-plasma model using additional measurements to be able to draw conclusions of greater medical relevance.

Another topic may also prove interesting for future work. Our experiments showed that the results are highly dependent on the choice of the development of the diffusion coefficients of the artificial noise necessary for Bayesian parameter estimation. While there is a general agreement that these coefficients should decrease over time, there is currently a lack of automated methods for making appropriate choices of the diffusion coefficients, both for initial values and dynamic development. However, to provide a really practical method for parameter estimation in non-linear mixed effects models (or even in models which adhibit only global parameters), our approach could also be combined with methods better suited to the estimation of fixed parameters, a good candidate being the PMCMC methods proposed in

In summary, we believe that the method presented in this article opens the door to even more efficient and reliable state sampling and parameter estimation methods based on the particle filter algorithm operating on continuous-time stochastic state space systems.

Notation

_{0},

_{
t
}, i.e.

_{0},

_{
s,t
}(_{
s
}, d_{
t
}) the Markov kernel of the process

_{1:M
} observation random variable with values in measurable spaces

_{
j
} (

_{
j
} random variables modelling the uncertainty about exact observation times

_{0},

_{
j
}(_{
j
}) probability density of _{
j
} with respect to

_{0}

_{
ℓ
}

_{
ℓ
}:

_{
t
} full density at time

_{
j
} are included for which _{
j
}≤

_{
j
}≤

_{
j,t
}(_{
j
}(_{
j
} | _{
t
}(_{
j
}(

_{
j,t
}, where

_{
t
}) given _{1:M
}=_{1:M
} with respect to the filtered state _{
t
}

_{0}=_{0}<_{1}…<_{
D
} time discretization

_{
d
}=_{
d
}-_{
d-1} stepsize

_{
d
}

_{
i
} mass of the tracee in compartment

_{
i
} mass of the tracer in compartment

_{
i
} input for the tracee in compartment _{1} denotes the influx into compartment 1)

_{
i
} input for the tracer in compartment

_{
j,i
} transfer coefficient of the tracers from compartment

_{
i
} diffusion parameter

_{
t
} log-normal noise (

_{
p
}, _{
p
} patient-dependent random factors modelling the parametric uncertainties among individuals (_{
p
}= exp(_{
p
}) with

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

JH and AK contributed to the underlying ideas of the method and its elaboration and implemented the algorithm. Both authors contributed equally to the initial manuscript. MJ and MA commented and refined the manuscript. MA provided the compartmental model and the data. MT acquired the data. MJ contributed to the discussion and supervised the project. All authors read and approved the final manuscript.

Acknowledgements

Thanks go to Michaela Höhne from the University of Kaiserslautern for her help in preparing the figures. Data was generated by Martin Adiels, Professor Marja-Riitta Taskinen, Helsinki University, and Professor Jan Borén, Göteborg University, and has been published elsewhere (