Abstract
Background
For a typical influenza infection in vivo, viral titers over time are characterized by 1–2 days of exponential growth followed by an exponential decay. This simple dynamic can be reproduced by a broad range of mathematical models which makes model selection and the extraction of biologicallyrelevant infection parameters from experimental data difficult.
Results
We analyze in vitro experimental data from the literature, specifically that of singlecycle viral yield experiments, to narrow the range of realistic models of infection. In particular, we demonstrate the viability of using a normal or lognormal distribution for the time a cell spends in a given infection state (e.g., the time spent by a newly infected cell in the latent state before it begins to produce virus), while exposing the shortcomings of ordinary differential equation models which implicitly utilize exponential distributions and delaydifferential equation models with fixedlength delays.
Conclusions
By fitting published viral titer data from challenge experiments in human volunteers, we show that alternative models can lead to different estimates of the key infection parameters.
Background
In the past decade, mathematical models of viral infection have been successfully applied to a number of problems on the periphery of the annual public health problem that is influenza [1]. In the laboratory, mathematical models have aided the development of efficient vaccine production techniques [2] and improved the quantitative characterization of antiviral drug action [3]. Mathematical models have also improved our understanding of the course of the disease within human [4] and animal hosts [5]. Because these models serve as a bridge between the microscopic scale (where virus interacts with cell) and the macroscopic scale (where the infection is manifested as a disease) they will inevitably be applied in the future to pressing public health questions such as the estimation of virulence and fitness for emerging strains, the spread of drug resistance and, more generally, the connections between viral genotypic information and clinical data.
The success of a withinhost virus infection model depends on an accurate representation of biological reality. This allows a model not only to describe the phenomenon under consideration, but also to make reliable predictions about unobserved consequences. For example, in 1995 a simple model of HIV dynamics was applied to describe the observed exponential clearance of virus under the administration of a drug suppressing viral production [6]. The primary result of this work, however, was not the description of viral clearance itself, but the prediction of dynamics in the absence of drug, i.e., that high viral clearance must be balanced by high viral production, which in turn allows for extremely rapid mutation of the virus strain. This conclusion had important implications for the development of therapy, specifically the necessity of a “drug cocktail”. For influenza infections, the primary clinical data available to a mathematical modeler is the viral titer over the course of an infection, usually obtained by a daily nasal wash collected from an infected patient. This data generally follows a simple functional form in time which can be reproduced by a variety of dynamical models. Thus, if meaningful information is to be extracted from such data, the model applied must already be a trusted simulator of the underlying infection kinetics. In this paper, we consider evidence from laboratory infection experiments which must inform the construction of a mathematical model, focusing specifically on the implementation of the time spent by a cell in each of the various stages of infection.
The basic viral infection model [4,7] assumes interaction of virus with cells in four different states (Figure 1), and is based on a coarsegrained view of the virus replication cycle. Cells that have not yet been infected by the virus, but are susceptible to infection, are considered target cells (T). The interaction of virus with target cells leads to these cells becoming latently (L) infected (i.e., infected but not producing virus). After infection, a time, t_{L}, passes — as the virus particle is unpacked, its genome is delivered to the cell nucleus, replication begins, and new particles assemble at the plasma membrane — before new virus particles are released and the cell enters the infectious (I) state. After a subsequent time, t_{I}, the infectious cell halts virus production and transitions into a state we will refer to as dead (D).
Figure 1. Basic viral infection model. The cellvirus interactions of infection (β) and viral production (p) are indicated along with the four possible states in which cells can be found. The lifetimes of the latent and infectious states for a particular cell, t_{L} and t_{I}, are considered random variables.
The implementation of a particular dynamical structure on this basic model requires a more detailed specification of the biological processes. The infection of cells (the transition of target cells to latentlyinfected cells) has been observed to be a Poisson process where the rate of infection is proportional to the local virus concentration [8] and it is implemented in the model as a continuous representation of that stochastic process. Virus production by infectious cells can be assumed to proceed at a constant rate and the infectivity of free virus is known to decrease exponentially in time [3,9], leading to a simple equation for virus dynamics. To complete the dynamical description, one must specify how a latently infected cell becomes infectious and for how long infectious cells produce virus. In other words, one must specify the distribution of the delays, t_{L} and t_{I}, between the states of infection.
In an epidemiological context, the problem of implementing generic delays between infected classes was first considered by Kermack and McKendrick in their seminal 1927 work on infectious disease dynamics [10]. Hethcote and Tudor [11] introduced a general approach to the problem, using a probability density function for the time spent in a given state, which has been applied frequently in the field of mathematical epidemiology (see, e.g., [1214] and references therein). Here, we will apply the same approach to withinhost influenza viral infections, resulting in a model with differential equations to describe target cell and virus dynamics, and integral equations to describe the latent and infectious cell populations (a similar approach was considered for HIV in [15]).
Mathematically, the simplest choice of delay distribution is exponential (shown in Figure 2 with a few other choices), because it reduces the model to a system of ordinary differential equations (ODEs). For that reason, it is the most commonlyused model type for both epidemiological and withinhost problems. In viral infections, however, the assumption of an exponential distribution seemingly conflicts with the biological evidence. For example, if the time of latent infection is chosen from such a distribution, the model would predict that a significant fraction of cells begin producing virus almost immediately after infection. In reality, however, there is always a minimum delay prior to viral release: endocytosis and the fusion of the viral envelope with the endosome takes, on average, half an hour [16]; the viral RNA enters the nucleus in most cells within the next one hour [17]; mRNA is transcribed in the nucleus, then transported back to the cytoplasm for translation and newly formed M1 matrix proteins are observed only three hours after infection, on average, and hemagglutinin four hours postinfection [17]; newly formed glycoproteins, matrix proteins and nucleocapsids then must assemble at the cell membrane, bud off and be cleaved from the sialic acid receptors [18]. Each of these steps and their timings depend on virus strain and cell type, and one can expect significant variation between cells, but a long delay without viral production is an essential characteristic of the infection cycle. Influenza virusinduced cell death is less well characterized: the mechanism of cell killing (apoptosis or necrosis) depends on cell type [19,20], and the timing of apoptosis in particular is strongly strain dependent [21]. In this situation, a broad freedom in selecting the distribution for infectious cell lifespans is warranted.
Figure 2. Example delay distributions. Four example delay types defining the time spent by a cell in a given state. The plotted function, P(t) corresponds to the probability that after some time t, the cell remains in its current state, i.e., has not yet transitioned to the next state. In each example, is the median time of transition to the next state. While exponentialdelays allow for instantaneous transitions (e.g., 10% of cells will have transitioned after 1.5 h has elapsed), the other three types enforce a minimum waiting period for almost all cells before transitioning from one state to the other (i.e., P(t) ≈ 1 for 0 < t < t_{min}). Enforcing this delay is more biologically realistic. The function P(t) appears explicitly in the viral infection model, Equation (1), and allows for generic delays between states of infection.
Despite questions about their biological appropriateness, ODE models have had success in describing in vivo infection data (for influenza see, for example, [4,5]). Models with nonexponential delays have been similarly successful, including those with Dirac delta function transition distributions, leading to a delaydifferential model [3,4,22]; and multicompartmental ODE models (with n sequential phases of infection) yielding delays with a gammafunction distribution [2325] Here, we consider a set of in vitro experiments which allows for some discrimination between models, namely the singlecycle viral yield assay. By fitting models with different transition distributions (Figure 2) to singlecycle assay data, we show that the correct implementation of delays is crucial to the success of a model in describing these assays. Using these results, we consider in vivo data from challenge experiments in humans to explore how the choice of delays affects the parameter values extracted when fitting the model to experimental data.
Methods
Model
The general viral infection model used in this paper is written, following [11], as
where T, L, I are the populations of cells in the target, latentlyinfected and infectious (virusproducing) states, respectively, and N the total number of cells in the system. V is the virus concentration, β and p are the rates of infection and virus production, respectively, and c is the viral clearance rate. The function P_{L}(t) is the probability that a cell remains in the latent state for at least a time t before transitioning to the infectious state, and P_{I}(t) is the probability that a cell remains in the infectious state for at least a time t before transitioning to the dead state (i.e., before it ceases to release virus). The transition profiles for different choices for the expression of P(t) are illustrated in Figure 2. f_{L} is the probability density function for the time a cell will spend in the latent state before transitioning to the infectious state (f_{L} = –dP_{L}/dt). Note that f_{I}(t) does not explicitly appear in the model.
Delay distributions
If an exponential distribution is chosen for both transitions (exponentialdelay), we have
where τ_{L} is the average time spent by a cell in the latent infection state, and τ_{I} is the average lifespan of an infectious cell. Equations (1a–1d) can then be written as a system of ordinary differential equations
When a Dirac delta function is used for both f_{L} and f_{I} (fixeddelay), such that the times spent by cells in the latent state and the infectious state are exactly τ_{L} and τ_{I}, respectively, Equations (1a–1d) reduce to a set of delay differential equations (DDE) [3].
Biologically, cells transition from one state to another only after some average time has elapsed. One also expects to see some variation in these transition times among cells as many of the processes rely on chains of stochastic intracellular events. Thus, normal distributions,
where σ_{L} and σ_{I} are the standard deviations, are a natural choice (normaldelay). The normalization constants, A_{i}, are necessary since the distributions must be truncated at zero. If one wishes to avoid renormalization, a good alternative is a lognormal distribution (lognormaldelay)
where erfc(x) is the complementary error function (i.e., erfc(x) = 1 – erf(x)) and the standard deviations σ_{L} and σ_{I} are dimensionless quantities.
To facilitate comparison between the various distributions, we utilize their median values. For the Dirac delta and lognormal distributions, the median is simply the value of the parameter itself, i.e.,
For exponential distributions, the median is
and for the truncated normal, the median is found by setting its cumulative distribution function to onehalf (there is no simple analytical expression).
Numerical simulation
Numerical evaluation of the model in Equation (1) was performed using a modified Euler technique. At every time step of length ∆t, newly infected cells, L_{new} = βTV ∆t, were removed from the target population. The passage of these L_{new} cells through the latent and infectious states was then calculated for all future times using f_{L}, P_{L} and P_{I} and added to that of previously transitioned cells. Virus dynamics at each time step were calculated according to the Euler approximation of Equation (1d). Simulations of singlecycle in vitro experiments were initialized with L(0) = N; simulations of in vivo infections were initialized with T(0) = N and V (0) = V_{0}.
Model fitting and parameter extraction
In any modelfitting exercise, a number of considerations must be made to ensure the reliable extraction of parameter values from experimental data. First, one must consider the question of parameter identifiability [36,57,58]: If the experimental system were to exactly reproduce the dynamics of the model equations, could the parameters be uniquely identified from the available observations? A number of techniques have been introduced to address identifiability for ODE models [37,59], but these are not directly applicable to the more general system considered here (Equation (1)). Nevertheless, for each experiment considered here we attempted to reduce consideration to an identifiable set of parameters. For the early phase singlecycle viral yield experiments, we fixed parameter values involving viral clearance and infectious cell death, and fitted only those parameters related to viral production and the transition of cells from the latent to infectious state. In the singlecycle, singlehistory experiment, independent information on viral clearance allowed for that parameter to be fixed in fitting. For the in vivo volunteer patient infection data, we have arbitrarily fixed the product of the viral production and infection rates in order to obtain a unique solution in the fitting procedure.
A second consideration when fitting experimental data is the question of model error: Is the mathematical model an appropriate representation of experimental data? To address this question, we performed least squares fitting of the model equations to the data sets using the Octave 3.2.4 [60] implementation of the LevenbergMarquardt algorithm, leasqr. For all data presented here, fits were performed to the viral titer data in logspace and the sum of squared residuals (SSR) was calculated as
In order to compare model systems with different numbers of parameters, we evaluated the Akaike information criterion (AIC) for each fit,
where k is the number of model parameters, n is the number of data points and AIC_{c} is the “corrected” form of the AIC for small sample sizes [61].
Finally, to account for measurement error, we calculated 95% confidence intervals for each reliably extracted parameter value by fitting 1000 bootstrap replicates [62]. Confidence intervals were not calculated for models fits with large error (high SSR and AIC_{c}) or for fits to volunteer patient infection data, where arbitrary assumptions were made to constrain the fitted parameter values.
Calculation of the basic reproductive number
For each fit to the volunteer patient infection data we numerically calculated an approximation of the basic reproductive number, R_{0}. In the ODE model, Equation (3), linear stability of the diseasefree equilibrium is guaranteed by R_{0}< 1, where
It can also be shown using the ODE that this quantity is equal to the commonlyquoted definition of the reproductive number: the number of secondary infections caused by one infectious cell, in a completely susceptible cell population. For other delay models, where an analytical form is not readily available, we calculated R_{0} numerically according to that statement, i.e., for a given set of parameters, we disallowed latent to infectious transitions, initialized the simulation with one infectious cell, and determined the number of cells in the latent state as t → ∞.
Results and discussion
General features of singlecycle viral growth
Singlecycle growth (SCG) viral yield experiments provide a unique view of viral replication. By initiating infection with a viral inoculum of high concentration (a multiplicity of infection (MOI) much larger than one), all cells are infected simultaneously, and the experimentalist effectively synchronizes the cells’ passage through the phases of latency, viral production and death. The resulting viral production curve can then be viewed as that of the average cell. This is in sharp contrast to “multiplecycle” yield experiments (MOI ≪ 1), where only a few cells are initially infected, leading to successive cycles of infection; the resulting exponential growth of both infected cells and virus over time effectively masks the dynamics of a single cell. Virus infections of humans and animals are similar to this latter experiment in that they are likely initiated by the infection of a only a few cells [26,27], leading to the exponential consumption of a large target cell population. Thus, SCG experiments demonstrate an artificial infection dynamic which would never occur in nature. However, their depiction of the average virus production of a cell makes them an invaluable tool for model building and for isolating specific components or parameters of the viral replication cycle.
In the original publications of the nine example SCG data sets plotted in Figure 3[9,2832] virus was plotted on a logarithmic scale, due to the extraordinary sensitivity of measurement techniques which can detect infectious virus over nearly 8 orders of magnitude. It is clear from Figure 3, however, that this perspective masks the most important features of SCG: a long delay without significant viral release followed by linear growth of the virus concentration. This simple dynamic can be summarized in the empirical expression,
where and p* can be considered approximations of the average latent infection period, τ_{L}, and the viral production rate, p, in the case where viral clearance is low (i.e., Equation (8) is the solution of Equation (1d) when I/N = 1 and c = 0). The fitted values of and p* for all nine experiments are given in Table 1.
Figure 3. Singlecycle viral yield examples. Examples of singlecycle growth experiments. Plotted lines are fits to the nonzero data points using Equation 8 (the last data point in G and I have been excluded from the fits) Virus units (VU), the parameters of the fit, and the details of each experiment are given in Table 1.
We will consider below the characterization of the SCG experiment using various dynamical models, but some simple analysis can be done using only the empirical relation above. For example, two experiments (Figure 3AB and Figure 3CD) considered the growth of influenza A/Udorn/307/72 (H3N2) and a counterpart strain possessing a single mutation in the NS1 gene (T215A and R83A, respectively) [28,29]. In each case, the experiment reveals a significant reduction of the approximate viral production rate p* for the mutant, without a significant change in the approximate latency period. This shows that the singlecycle experiment can highlight important biological characteristics of a virus strain, with very little mathematical analysis.
Characterizing the latent infection period from singlecycle growth assays
To determine the type of distribution which should be used to model the time spent by a newly infected cell in the latent phase, we performed model fits to two singlecycle data sets [28,32] which offered frequent sampling in time and a large range of virus measurements (Figure 4). Recently we have shown [33] that the ODE assumption of exponential delays yields a very poor fit to singlecycle viral yield data. Here, we considered three additional distributions for the waiting times: fixeddelay, normaldelay and lognormaldelay. We neglected the influence of infectious cell death by setting the infectious cell lifespan to be much longer than the duration of the experiment. This is an oversimplification of reality, particularly for later times. However, the lack of data spanning the period of virus decline, which occurs at later times than those measured, prohibits any real measure of infectious cell death. Similarly, this lack of viral decay information prohibits an effective characterization of the loss of virus infectivity. Therefore, we fitted the model using two values of viral clearance, zero and 0.2 h^{–1}. The latter value represents an approximate upperbound of the viral clearance value based on literature reports for in vitro experiments performed at 37°C [2,3].
Figure 4. Model fits to viral titers from singlecycle experiments. Model fits to viral titers from singlecycle experiments by (Left) Hale et. al. [28] (Figure 3A) and (Right) Sugiura and Kilbourne [32] (Figure 3I). Fitting was performed using an exponential (dotdashed), delta (dotted), normal (solid) or lognormal (dashed) distribution for the time spent by newly infected cells in the latent state before transitioning to the infectious state. Insets show the same fits and data on a linear virus scale (axis ranges are identical to those in Figure 3).
The results of model fits to the two data sets are collected in Table 2. The exponential and fixeddelay models clearly provide a poorer fit to the data than those with normal and lognormaldelays (for the A/Udorn/307/72 [28] data set, however, the small number of data points and fewer parameters, with respect to normal and lognormal models, do not allow for their formal exclusion based on AIC_{c}). While the exponentialdelay model (ODE) is unable to duplicate any feature of the dynamics, the fixeddelay model fits the data well at late times in the infection when the dynamics are dominated by linear growth, at least when viral clearance is neglected. It does a poor job, however, of describing the early stages of virus release, and is unable to provide the correct dynamics in the presence of viral clearance.
Both the normal and lognormaldelay models provide an adequate description of the data over the entire range of values and for both values of viral clearance, although the SSR and AIC_{c} values are smaller for fits using a normal distribution. When the fits to the logvalued virus are viewed in linearspace (inset graphs), the normal fits appear to be a more reasonable approximation of the data in that the linear SSRs of these fits, which depend most sensitively on the larger virus values, are also smaller. When viral clearance is neglected, the fitted values of the viral production rate, p, are close to the approximate values of p* (Table 1). Nonzero viral clearance leads to a larger fitted production rate, as expected. The fitted values for the median latent infection period, , vary depending on the distribution type, but are always as long as the approximate values of for both experiments (10.9 h for [28] and 10.8 h for [32]). The introduction of a nonzero viral clearance leads to even longer latent infection periods, ranging from 8 h to 24 h. The fitted standard deviations, σ_{L}, are between 1.5 and 3.0 h for the normal distribution and between 0.15 and 0.44 for the lognormal distribution.
Characterizing the infection cycle from a singlecycle, singlehistory yield assay
In 1968, an in vitro experiment was performed which, to our knowledge, is unique in the literature [9]. Like the experiments presented in the previous sections, a SCG experiment was prepared: ~ 10^{7} cells were incubated with a high titer (MOI = 10) of influenza A/PR/8 (H1N1) virus such that almost all cells were infected and then the infection medium was removed. Unlike typical SCG experiments, however, viral titer was not measured by sacrificing independent wells at each sampling time to titrate their overlay. Instead, the liquid overlay from the same well was removed in its entirety and replaced with fresh, virusfree medium and the infection was allowed to continue. Thus, titrations of the collected overlay medium provided a measure of the amount of virus being produced by a single cell culture at the time of collection. Application of this sampling protocol to SCG experiments — which we refer to as a singlecycle, singlehistory yield experiment (SCSH) — mitigates complications tied to the accumulation of virus, and brings into focus the viral production of the cell culture as a series of snapshots over time, all sharing a common kinetic history.
Using the model in Equation (1), we performed fits to the SCSH data set [9] (Figure 5), allowing both the lengths of the latent and of the infectious periods to vary freely. To simulate the removal of the overlay medium at each sampling time, we reduced the virus concentration by a factor of 10^{3} (consideration of larger reductions showed a negligible influence on the fit). The first three data points were excluded from consideration while fitting since these points were likely due to the desorption of excess virus into the overlay [9], a process not accounted for in our model. The value of viral clearance was held fixed at 0.26 h^{–1}, based on the results of an independent mockinfection experiment (Figure 7 of [9]). Four model types were considered: exponential, fixed, normal and lognormaldelay. The bestfit virus curves are shown in Figure 5. The fitted parameter, SSR and AIC_{c} values for each distribution are reported in Table 3. As in SCG experiments, the normaldelay and lognormaldelay models provided adequate representations of the data, while the fixeddelay (DDE) and exponentialdelay (ODE) models did not.
Figure 5. Results of a singlecycle, singlehistory yield experiment[9]with model fits.Top row: Experimentallymeasured viral titers (points) are plotted along with model fits to the data (lines) for four different delay distributions. The left panel shows both the exponentialdelay (solid) and fixeddelay (dashed) model fits. Discontinuities in the model dynamics result from removal of 99.9% of virus at each measurement time point, duplicating the experimental procedure. Bottom row: A transformation of the experimental data according to Equation (10) (points) shows the average viral production rate (i.e., the production rate weighted by the fraction of infectious cells) as a function of time; the model dynamics for same quantity, pI/N, are overplotted (lines). The model dynamics plotted in the bottom row were generated using the extracted parameters from the fits performed to the raw data in the top row. While the exponential and fixeddelay models offer a poor fit to the data, both the normal and lognormaldelay models adequately describe the data.
The unique attribute of the SCSH experiment is the view it provides of the viral production by the average infected cell over time. To investigate these kinetics, a transformation of the original data is required. Specifically, the experimental measure of accumulated virus ∆V over an interval ∆t can be converted to an average virus production rate for that time interval, i.e., the fraction of infectious cells multiplied by the (constant) viral production rate of a cell, pI/N. The theoretical relationship between these quantities is provided by the basic equation for viral dynamics, Equation (1d). If we assume that the number of infectious cells, I, remains constant over the interval between samplings of the overlay, this equation can be solved to obtain accumulated virus as a function of average production rate and viral clearance, c:
where we have used the fact that the virus concentration at the beginning of the time interval is zero. Rearranging this expression, the average viral production over an interval ∆t centered at time t can then be written
The resulting transformed data is shown in Figure 5. Viewed in this way, one can explicitly see the growth of the infectious cell population, including a steep rise from 4 h to 12 h as cells transition from latent to infectious, and a long decline between 24 h and 48 h as infectious cells cease viral production. The model dynamics for pI/N, generated using parameters from the fit to the raw data, agree well with the transformed data, validating the use of Equation (1d) and the assumptions made in the above transformation.
While the normaldelay and lognormaldelay models both lead to an adequate description of the experimental data, a consideration of the fitted parameter values for the median infectious cell lifespan, , reveals vastly different underlying dynamics. In the normaldelay case, the median infectious lifespan is predicted to be short but the associated standard deviation is large. The long decay of infectious cells at late times is therefore explained by a broad distribution in the times spent by cells in the infectious state. Using the lognormaldelay model, however, the best fit is nearly that of a fixeddelay (σ_{I} ≪ 1), with a long median infectious lifespan Under this assumption, the decay of infectious cells is thus completely determined by the long tail in the distribution of latently infected cell lifespans. With only a single data set, it is not possible to discriminate between these two extreme cases: short infectious lifespans on average with a broad distribution, versus a long average infectious lifespan with a narrow distribution of transitions. The results of SCG experiments, where viral titer is observed to grow linearly over 10 to 20 h (Figure 3), suggest that the former is unlikely since infectious cell death would lead to a turnover in the viral titer curve. It would be useful to duplicate this unique experiment in parallel with a typical SCG experiment such that these biologically distinct possibilities can be distinguished.
Effect of delay assumptions in fitting clinical data
We now consider the effect that the choice of delay distribution has on estimated parameter values when fitting a model to viral titer data from human patients experimentally infected with influenza. This type of data — two examples are shown in Figure 6, for others see [34] — generally shows an exponential increase of virus followed by an exponential decrease after the peak, which occurs 1 to 2 d postinfection. An empirical function capturing these basic characteristics can be written as
where λ_{g} and λ_{d} are the exponential growth and decay rates, respectively. V_{p} is the peak value of the viral titer, and t_{p} is the time of viral titer peak. The exact peak values can be determined by differentiating Equation (11). Fits of this function to the experimental data, along with the calculated SSR, are shown in Figure 6. The simple functional form of the empirical model in Equation (11) points to a fundamental problem in fitting a dynamical model to clinical viral data: only four independent parameters can be reliably extracted from such data. A closely related problem is whether or not the parameters of a given model can be uniquely identified from a particular experimental measurement, the question of parameter identifiability [3537]. It has been shown, for example, that the ODE model in Equation (3) is not identifiable when considering viral titer data alone [37]. Sparse experimental data, where the viral titer curve is not wellsampled, can introduce additional complications for model fitting. For example, in the influenza A/Texas/36/91 (H1N1) viral titer data [38] presented in Figure 6a, there is only a single data point (at 24 h) from which the growth rate of viral titer, λ_{g}, can be determined.
Figure 6. In vivo patient data. Viral titer over time for volunteer patients infected with (a) influenza A/Texas/36/91 (H1N1) [38], and (b) A/Bethesda/1/85 (H3N2) [39]. The solid curve is a fit of the empirical model in Equation (11) to each dataset. The fitted parameter values are given along with the SSR of the fit.
To mitigate some of these problems, we fitted the four delay models (exponential, fixed, normal and lognormaldelay) to the clinical data sets in Figure 6 under a number of constraints. We fixed the values of σ_{L} and σ_{I} to those obtained for SCSH experiment (Table 3) to allow for comparison of model systems with an equal number of parameters. Then, to allow for unique solutions in the fitting algorithm, we reduced the parameter space by fixing the product of the viral production and infection rates, pβ, to the value of 1 h^{–2}. This quantity is a measure of the infectivity of a viruscell system and is related to the characteristic infecting time of the system, which is time for a single infectious cell to cause the latent infection of one more in a completely susceptible cell population [33]. Under these constraints we were able to compare the effect of different delay assumptions on the fitted values of viral clearance, c, and production rate, p ; median latent and infectious cell lifespans, , and , and the basic reproductive number R_{0} (Table 4).
A comparison of the fitted parameter values shows a clear delineation between the results obtained under the assumption of exponentiallydistributed delays and those of the other three models, which enforce longer delays for all cells. The median lifespan of a latently infected cell, , is shorter under the exponential assumption (3.7 h vs. an average of 7.6 h for the A/Texas/36/91 (H1N1) data set [38] and 6.4 h vs. an average of 9.4 h for A/Bethesda/1/85 (H3N2) [39]). The same is true for the median infectious lifetime, . Extracted values for the viral production rate are larger for the exponentialdelay model, by a factor of ~ 2 in one data set and by an order of magnitude in the other. The estimated basic reproductive number is smaller for the exponentialdelay model, by a factor of 2–10, than the other three delay types. The fitted value of the viral clearance, c, was independent of delay choice for one data set (and equal to the viral decay rate, λ_{d}), but differed for the exponentialdelay model in the other.
Conclusions
Mathematical models of viral infections within a host or cell culture have helped shed light on several aspects of cellvirus interactions [6,40,41]. Most frequently, models have been used to extract values for the parameters controlling smallscale infection kinetics from experimental data [4,42,43]. This has allowed mathematical modeling to play an evergrowing role in virological and immunological studies, with an increasing number of publications in these fields incorporating some amount of modeling [4446]. At present, however, modeling work often lags experimentation and primarily serves an explanatory role. It is desirable that models take a more predictive role in the future, where the simulation of viral infections may aid, for example, in the prediction of virulence for new strains or the kinetic mechanisms of untested antiviral therapies. Before models can take on a predictive role, however, the mathematical implementation of viral infection dynamics must be tested against a diverse set of experimental conditions to ensure that biological reality is faithfully represented.
Here, we have investigated the implementation of the progress of a cell through the states of infection: from latently infected to infectious to dead. We have focused on the characterization of two times: the time a cell is latently infected but not yet releasing virus, and the time a cell is infectious (releasing virus) before infectioninduced death. We explored four different distributions for these state lifetimes: exponential, Dirac delta (fixeddelay), normal, and lognormal. The validity of each distribution was assessed by fitting the associated model to data from singlecycle growth (SCG) viral yield experiments. These experiments provide a unique view of the average dynamics of a single cell due to the synchronous infection of all cells. We have shown that ODE models which implement exponential delays and DDE models with fixed delays are unable to describe this experimental data, whereas normaldelay and lognormaldelay models both provide a good fit to the data. In addition to the classic SCG experiment, we have also considered delay dynamics for a “singlecycle, singlehistory” (SCSH) experiment [9], which reveals average cellular virus production as a function of time. We have shown that, like the SCG experiment, both normal and lognormaldelay models provide an adequate description of the dynamics while exponential and fixeddelay models do not. The origin of the inability of ODEs and DDEs to exhibit singlecycle dynamics can be seen quite clearly in the experimental data (Figures 3, 4, and 5): virus production begins only after a long delay following the infection of a cell, a feature which ODE models cannot replicate, and the transition of cells into and out of the infectious phase follows a smooth distribution which DDE models cannot reproduce.
The median values of the latent infection period, determined by fitting normal and lognormaldelay models to the data from these SCG experiments, ranged from 8 h to 24 h, which is significantly longer than the 4 to 6 h values typically quoted in the literature (e.g., [2]). There are a number of possible reasons for this discrepancy. First, the latent infection period in the model includes both the typical eclipse period prior to virus production and any additional time required for viral release. Second, viral production in a culture is likely detectable much earlier. For example, under the assumption of normal delays the median length of the latent phase was found in the SCSH experiment to be with σ_{L} = 4.6 h. Thus, 6% of cells (~ 600, 000) had already begun releasing virus after only 6 h; using an alternative definition, the phase of latency could be declared over much earlier. Finally, the model describes a system where each cell releases no virus until the start of the infectious phase at which point virus is produced at a constant rate common to all cells. This is obviously a simplified version of reality. In fact, there is significant evidence from flow cytometry fluorescence experiments that different cells produce virus at different rates, perhaps over several orders of magnitude [47]; it is also likely that the rate of virus production varies over the course of its infectious lifespan.
The median infectious cell lifespan, determined in fits of the normal and lognormaldelay models to the SCSH data, ranged from 6 to 18 h, but the application of these two distributions types implied disparate dynamical scenarios. Under the assumption of normal delays, the infectious lifespan was small but the distribution was broad. In contrast, the lognormal assumption predicted nearly a fixed infectious lifetime for all cells and the observed slow decline of infections cells was completely ascribed to a long tail in the transition from the latent to infectious phase. Biologically, the infectious cell lifespan is variously characterized in the literature, as is influenzainduced cell death in general. When cell death is due to apoptosis, for example in MDCK cell cultures [48], the median time of cell death ranges from 12 to 48 h after infection, depending on the influenza strain subtype. When virusinduced cell death is caused by necrosis, as in cultures of some lung and intestinal epithelial cells, this range increases significantly with cells living 2–3 d postinfection [20,49,50]. The death of cells during an infection in vivo is obviously much more difficult to measure. There is some evidence that influenzainduced cell death is caused by apoptosis [5153] but details of the timing and strain dependence are unknown. Future elaboration of SCSH experiments, in concert with the type of analysis performed here, could aid in the quantitative characterization of virusinduced cell death.
Using the results from our analysis of in vitro experiments, we considered the effect of delay distribution choice when fitting viral titer data from in vivo infections (experiments performed on human volunteers). To allow for the identification of some kinetic parameters, we restricted the analysis by assuming a fixed value for the viral infectivity (defined as the product of the infection and production rates, pβ) and used the values for the parameters σ_{L} and σ_{I} determined when fitting the in vitro data. Under these assumptions, we found a clear difference between the extracted parameter values for models enforcing a delay in transitions (fixed, normal and lognormaldelay) and the ODE model with exponential transitions. Specifically the ODE predicted larger virus production rates, shorter latent and infectious phase lifespans, and a lower value of the basic reproductive number, R_{0} (Table 4). Although this result depends on rather arbitrary assumptions — infectivity may vary by orders of magnitude and there is little reason to assume that parameter values determined in vitro should be the same in vivo — it demonstrates that the choice of delay distribution has a significant effect on the conclusions drawn from modelfitting of in vivo data. This analysis also suggests that while a fixeddelay model cannot reproduce the continuous dynamics of cell transitions observed in singlecycle experiments, it may be a reasonable substitute for the more complicated normal and lognormal models in some contexts.
Our analysis of the in vivo experiments demonstrates the difficulty in extracting reliable information from viral titer data alone. Consideration of the SSR values for the model fits to in vivo data (Table 4) shows that each model adequately describes the data, despite the arbitrary constraints imposed. This highlights both the weakness of using viral titer data for model selection and the difficulty in uniquely identifying parameters from such data, given a particular model. Precise statements about the parameters controlling an in vivo infection can only be made by either imposing constraints to reduce the considered parameter space (as we have done here), or by obtaining complementary data (for example information about infectious cell population dynamics). Collection of infected cell data over the course of an influenza infection has only recently been considered in animal models (e.g., [54]) and such data will likely be unobtainable for human infections. Full characterization of models using in vitro experiments will therefore remain an important direction of future research. The introduction of infection models with an explicit immune response and the parallel measurements of immune system quantities (which are easier to obtain than infected cell populations) is a promising direction not only for a more complete understanding of the influenza infection but also the complete parametrization of these models [5,54,55] This will, of course, depend on the development of simple models for which the added complexity is warranted by the available data.
In the past ten years, tools for dynamical measurements of in vitro viral infections have improved quickly. The measurement of infection within individual cells by fluorescence microscopy has become routine [47]. The spatial spread of virus infections on cell culture can be viewed in real time, both at the level of cellular deformation and at the level of individual virus particles [56]. Individual virus particles have been tracked as they enter a cell, and repeated observations have allowed for a statistical characterization of the timing of events in the early stages of infection [16]. These detailed views of the influenza infection will be invaluable for the construction of the next generation of viral infection models. Models, in turn, will provide a cohesive picture of the overall infection process and, crucially, make connections between the known smallscale details of the viruscell interaction and the infection at the level of the organism, where it is manifested as a disease.
List of abbreviations
ODE: Ordinary differential equation; DDE: Delay differential equation; SSR: Sum of squared residuals; AIC: Akaike information criterion; AIC_{c}: Akaike information criterion, corrected for small samples; SCG: Singlecycle growth; MOI: Multiplicity of infection; SCSH: Singlecycle, singlehistory growth; MDCK: Madin Darby canine kidney; VU: Virus units
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
BH conceived the study, participated in the design of the study, carried out the modeling, performed the analysis and drafted the manuscript. CAAB conceived of the study, participated in its design and coordination, and helped to draft the manuscript. All authors read and approved the final manuscript.
Acknowledgements
This study was supported by the Canadian Institutes for Health Research [funding reference number 86937] and by the Natural Sciences and Engineering Research Council of Canada (CAAB). The authors wish to thank Dr. Shingo Iwami for his careful reading of the manuscript and helpful comments.
This article has been published as part of BMC Public Health Volume 11 Supplement 1, 2011: Mathematical Modelling of Influenza. The full contents of the supplement are available online at http://www.biomedcentral.com/14712458/11?issue=S1.
References

Lenglet AD, Hernando V, Rodrigo P, Larrauri A, Donado JD, de Mateo S: Impact of flu on hospital admissions during 4 flu seasons in Spain, 20002004.
BMC Public Health 2007, 7:197. PubMed Abstract  BioMed Central Full Text  PubMed Central Full Text

Möhler L, Flockerzi D, Sann H, Reichl U: Mathematical model of influenza A virus production in largescale microcarrier culture.
Biotechnol. Bioeng 2005, 90:4658. PubMed Abstract  Publisher Full Text

Beauchemin C, McSharry J, Drusano G, Nguyen J, Went G, Ribeiro R, Perelson A: Modeling amantadine treatment of influenza A virus in vitro.
J. Theor. Biol 2008, 254:439451. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Baccam P, Beauchemin C, Macken C, Hayden F, Perelson A: Kinetics of influenza A virus infection in humans.
J. Virol 2006, 80:75907599. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Handel A, Longini I, Antia R: Towards a quantitative understanding of the withinhost dynamics of influenza A infections.
J. R. Soc. Interface 2010, 7:3547. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Ho D, Neumann A, Perelson A, Chen W, Leonard J, Markowitz M: Rapid turnover of plasma virions and CD4 lymphocytes in HIV1 infection.
Nature 1995, 373:123126. PubMed Abstract  Publisher Full Text

Smith AM, Ribeiro RM: Modeling the viral dynamics of influenza a virus infection.
Crit. Rev. Immunol 2010, 30:291298. PubMed Abstract

Dulbecco R, Vogt M: Plaque formation and isolation of pure lines with poliomyelitis viruses.
J. Exp. Med 1954, 99:167182. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Gaush C, Smith T: Replication and plaque assay of influenza virus in an established line of canine kidney cells.

Kermack W, McKendrick A: Contributions to the mathematical theory of epidemics, Part I.
Proc. Roy. Soc. A 1927, 115:700721. Publisher Full Text

Hethcote H, Tudor D: Integral equation models for endemic infectious diseases.
J. Math. Biol 1980, 9:3747. PubMed Abstract  Publisher Full Text

Feng Z, Xu D, Zhao H: Epidemiological models with nonexponentially distributed disease stages and applications to disease control.
Bull. Math. Biol 2007, 69:15111536. PubMed Abstract  Publisher Full Text

Wearing HJ, Rohani P, Keeling MJ: Appropriate models for the management of infectious diseases.

Lloyd AL: Destabilization of epidemic models with the inclusion of realistic distributions of infectious periods.
Proc. R. Soc. Lond., B, Biol. Sci 2001, 268:985993. Publisher Full Text

Banks H, Bortz DM, Holte SE: Incorporation of variability into the modeling of viral delays in HIV infection dynamics.
Math. Biosci 2003, 183:6391. PubMed Abstract  Publisher Full Text

Lakadamyali M, Rust MJ, Babcock HP, Zhuang X: Visualizing infection of individual influenza viruses.
Proc. Natl. Acad. Sci. U.S.A 2003, 100:92809285. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Martin K, Helenius A: Nuclear transport of influenza virus ribonucleoproteins: The viral matrix protein (M1) promotes export and inhibits import.
Cell 1991, 67:117130. PubMed Abstract  Publisher Full Text

Nayak DP, Hui EKW, Barman S: Assembly and budding of influenza virus.
Virus Res 2004, 106:147165. PubMed Abstract  Publisher Full Text

Takizawa T, Matsukawa S, Higuchi Y, Nakamura S, Nakanishi Y, Fukuda R: Induction of programmed cell death (apoptosis) by influenza virus infection in tissue culture cells.
J. Gen. Virol 1993, 74:23472355. PubMed Abstract  Publisher Full Text

Zhirnov O, Klenk HD: Human influenza A viruses are proteolytically activated and do not induce apoptosis in CACO2 cells.
Virology 2003, 313:198212. PubMed Abstract  Publisher Full Text

Mohsin MA, Morris SJ, Smith H, Sweet C: Correlation between levels of apoptosis, levels of infection and haemagglutinin receptor binding interaction of various subtypes of influenza virus: Does the viral neuraminidase have a role in these associations.
Virus Res 2002, 85:123131. PubMed Abstract  Publisher Full Text

Lee HY, Topham DJ, Park SY, Hollenbaugh J, Treanor J, Mosmann TR, Jin X, Ward BM, Miao H, HoldenWiltse J, Perelson AS, Zand M, Wu H: Simulation and prediction of the adaptive immune response to influenza A virus infection.
J. Virol 2009, 83:71517165. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Mittler JE, Sulzer B, Neumann AU, Perelson AS: Influence of delayed viral production on viral dynamics in HIV1 infected patients.
Math. Biosci 1998, 152:143163. PubMed Abstract  Publisher Full Text

Grossman Z, Feinberg M, Kuznetsov V, Dimitrov D, Paul W: HIV infection: How effective is drug combination treatment?
Immunol. Today 1998, 19:528532. PubMed Abstract  Publisher Full Text

Lloyd AL: Dependence of viral parameter estimates on the assumed viral life cycle: Limitations of studies of viral load data.
Proc. R. Soc. Lond., B, Biol. Sci 2001, 268:847854. Publisher Full Text

Keele BF, Learn HLGH, Hraber P, Giorgi EE, Grayson T, Sun C, Chen Y, Yeh WW, Letvin NL, Mascola JR, Nabel GJ, Haynes BF, Bhattacharya T, Perelson AS, Korber BT, Hahn BH, Shaw GM: Lowdose rectal inoculation of rhesus macaques by SIVsmE660 or SIVmac251 recapitulates human mucosal infection by HIV1.
J. Exp. Med 2009, 206:5111751134. Publisher Full Text

Zwart MP, Hemerik L, Cory JS, de Visser JAG, Bianchi FJ, Oers MMV, Vlak JM, Hoekstra RF, der Werf WV: An experimental test of the independent action hypothesis in virusinsect pathosystems.
Proc. R. Soc. Lond., B, Biol. Sci 2009, 276:22332242. Publisher Full Text

Hale B, Knebel A, Botting C, Galloway C, Precious B, Jackson D, Elliott R, Randall R: CDK/ERKmediated phosphorylation of the human influenza A virus NS1 protein at threonine215.
Virology 2009, 383:611. PubMed Abstract  Publisher Full Text

Min J, Krug R: The primary function of RNA binding by the influenza A virus NS1 protein in infected cells: Inhibiting the 2′5′ oligo(A) synthetase/RNase L pathway.
Proc. Natl. Acad. Sci. U.S.A 2006, 103:71007105. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Nagai T, Moriguchi R, Suzuki Y, Tomimori T, Yamada H: Mode of action of the antiinfluenza virus activity of plant flavonoid, 5,7,4′trihydroxy8methoxyflavone, from the roots of Scutellaria baicalensis.
Antiviral Res 1995, 26:1125. PubMed Abstract  Publisher Full Text

Kaverin N, Webster R: Impairment of multicycle influenza virus growth in Vero (WHO) cells by loss of trypsin activity.
J. Virol 1995, 69:27002703. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Sugiura A, Kilbourne E: Genetic studies of influenza viruses II. Plaque formation by influenza viruses in a clone of a variant human heteroploid cell line.
Virology 1965, 26:478. PubMed Abstract  Publisher Full Text

Holder BP, Liao LE, Simon P, Boivin G, Beauchemin CA: Design considerations in building in silico equivalents of common experimental influenza virus assays.
Autoimmunity 2011, in press. PubMed Abstract  Publisher Full Text

Carrat F, Vergu E, Ferguson NM, Lemaitre M, Cauchemez S, Leach S, Valleron AJ: Time lines of infection and disease in human influenza: A review of volunteer challenge studies.
Am. J. Epidemiol 2008, 167:775785. PubMed Abstract  Publisher Full Text

Xia X, Moog C: Identifiability of nonlinear systems with application to HIV/AIDS models.
IEEE Trans. Automat. Contr 2003, 48:330336. Publisher Full Text

Wu H, Zhu H, Miao H, Perelson A: Parameter identifiability and estimation of HIV/AIDS dynamic models.
Bull. Math. Biol 2008, 70:785799. PubMed Abstract  Publisher Full Text

Miao H, Xia X, Perelson A, Wu H: On identifiability of nonlinear ODE models and applications in viral dynamics.
SIAM Review 2011.
To Appear

Barroso L, Treanor J, Gubareva L, Hayden FG: Efficacy and tolerability of the oral neuraminidase inhibitor peramivir in experimental human influenza: Randomized, controlled trials for prophylaxis and treatment.
Antivir. Ther 2005, 10:901910. PubMed Abstract

Bjornson A, Mellencamp M, Schiff G: Complement is activated in the upper respiratory tract during influenza virus infection.
Am. Rev. Respir. Dis 1991, 143:10621066. PubMed Abstract

Nowak M, Bonhoeffer S, Hill A, Bohme R, Thomas H, McDade H: Viral dynamics in hepatitis B infection.
Proc. Natl. Acad. Sci. U.S.A 1996, 93:43984404. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Sidorenko Y, Reichl U: Structured model of influenza virus replication in MDCK cells.
Biotechnol. Bioeng 2004, 88:114. PubMed Abstract  Publisher Full Text

Perelson A, Neumann A, Markowitz M, Leonard J, Ho D: HIV1 dynamics in vivo: Virion clearance rate, infected cell lifespan, and viral generation time.
Science 1996, 271:15821586. PubMed Abstract  Publisher Full Text

Ciupe SM, Ribeiro RM, Nelson PW, Dusheiko G, Perelson AS: The role of cells refractory to productive infection in acute hepatitis B viral dynamics.
Proc. Natl. Acad. Sci. U.S.A 2007, 104:50505055. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Bauer AL, Beauchemin CA, Perelson AS: Agentbased modeling of hostpathogen systems: The successes and challenges.
Inf. Sci. (Ny) 2009, 179:13791389. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Forrest S, Beauchemin CA: Computer immunology.
Immunol. Rev 2007, 216:176197. PubMed Abstract  Publisher Full Text

Chavali AK, Gianchandani EP, Tung KS, Lawrence MB, Peirce SM, Papi JA: Characterizing emergent properties of immunological systems with multicellular rulebased computational modeling.
Trends Immunol 2008, 29:589599. PubMed Abstract  Publisher Full Text

Sidorenko Y, SchulzeHorsel J, Voigt A, Reichl U, Kienle A: Stochastic population balance modeling of influenza virus replication in vaccine production processes.
Chem. Eng. Sci 2008, 63:157169. Publisher Full Text

Price G, Smith H, Sweet C: Differential induction of cytotoxicity and apoptosis by influenza virus strains of differing virulence.
J. Gen. Virol 1997, 78:28212829. PubMed Abstract  Publisher Full Text

Seo SH, Goloubeva O, Webby R, Webster RG: Characterization of a porcine lung epithelial cell line suitable for influenza virus studies.
J. Virol 2001, 75:95179525. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Arndt U, Wennemuth G, Barth P, Nain M, AlAbed Y, Meinhardt A, Gemsa D, Bacher M: Release of macrophage migration inhibitory factor and CXCL8/interleukin8 from lung epithelial cells rendered necrotic by influenza A virus infection.
J. Virol 2002, 76:92989306. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Mori I, Komatsu T, Takeuchi K, Nakakuki K, Sudo M, Kimura Y: In vivo induction of apoptosis by influenza virus.
J. Gen. Virol 1995, 76:28692873. PubMed Abstract  Publisher Full Text

Ito T, Kobayashi Y, Morita T, Horimoto T, Kawaoka Y: Virulent influenza A viruses induce apoptosis in chickens.
Virus Res 2002, 84:2735. PubMed Abstract  Publisher Full Text

Uiprasertkul M, Kitphati R, Puthavathana P, Kriwong R, Kongchanagul A, Ungchusak K, Angkasekwinai S, Chokephaibulkit K, Srisook K, Vanprapar N, Auewarakul P: Apoptosis and pathogenesis of avian influenza A (H5N1) virus in humans.
2007, 13:708712. PubMed Abstract  PubMed Central Full Text

Saenz RA, Quinlivan M, Elton D, MacRae S, Blunden AS, Mumford JA, Daly JM, Digard P, Cullinane A, Grenfell BT, McCauley JW, Wood JLN, Gog JR: Dynamics of infection and pathology in influenza.
J. Virol 2010, 84:39743983. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Miao H, Hollenbaugh JA, Zand MS, HoldenWiltse J, Mosmann TR, Perelson AS, Wu H, Topham DJ: Quantifying the early immune response and adaptive immune response kinetics in mice infected with influenza A virus.
J. Virol 2010, 84:66876698. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Doceul V, Hollinshead M, van der Linden L, Smith G: Repulsion of superinfecting virions: A mechanism for rapid virus spread.
Science 2010, 327:873876. PubMed Abstract  Publisher Full Text

Bellman R, Åström K: On structural identifiability.
Math Biosci 1970, 7:329339. Publisher Full Text

Cobelli C, III JJD: Parameter and structural identifiability concepts and ambiguities: a critical review and analysis.
Am. J. Physiol. Regulatory Integrative Comp. Physiol 1980, 239:R7R24.

Bellu G, Saccomania MP, Audoly S, D’Angió L: DAISY: A new software tool to test global identifiability of biological and physiological systems.
Comput. Methods Programs Biomed 2007, 88:5261. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Eaton JW: GNU Octave. [http://www.octave.org] webcite
2010.

Burnham KP, Anderson DR: Multimodel inference: Understanding AIC and BIC in model selection.
Sociol. Methods Res 2004, 33:261304. Publisher Full Text

Efron B, Tibshirani R: Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy.
Stat. Sci 1986, 1:7797. Publisher Full Text