Department of Computer Science, University of Saskatchewan, Saskatoon, Saskatchewan, Canada

Department of Community Health and Epidemiology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada

Abstract

Background

Microcontact datasets gathered automatically by electronic devices have the potential augment the study of the spread of contagious disease by providing detailed representations of the study population’s contact dynamics. However, the impact of data collection experimental design on the subsequent simulation studies has not been adequately addressed. In particular, the impact of study duration and contact dynamics data aggregation on the ultimate outcome of epidemiological models has not been studied in detail, leaving the potential for erroneous conclusions to be made based on simulation outcomes.

Methods

We employ a previously published data set covering 36 participants for 92 days and a previously published agent-based H1N1 infection model to analyze the impact of contact dynamics representation on the simulated outcome of H1N1 transmission. We compared simulated attack rates resulting from the empirically recorded contact dynamics (ground truth), aggregated, typical day, and artificially generated synthetic networks.

Results

No aggregation or sampling policy tested was able to reliably reproduce results from the ground-truth full dynamic network. For the population under study, typical day experimental designs – which extrapolate from data collected over a brief period – exhibited too high a variance to produce consistent results. Aggregated data representations systematically overestimated disease burden, and synthetic networks only reproduced the ground truth case when fitting errors systemically underestimated the total contact, compensating for the systemic overestimation from aggregation.

Conclusions

The interdepedendencies of contact dynamics and disease transmission require that detailed contact dynamics data be employed to secure high fidelity in simulation outcomes of disease burden in at least some populations. This finding serves as motivation for larger, longer and more socially diverse contact dynamics tracing experiments and as a caution to researchers employing calibrated aggregate synthetic representations of contact dynamics in simulation, as the calibration may underestimate disease parameters to compensate for the overestimation of disease burden imposed by the aggregate contact network representation.

Background

Computational models of contagion can provide insight and foresight into the behavior of particular pathogens for specific populations. The representation of contacts between population members has a critical impact on simulation outcomes

More recently, researchers have used microelectronic devices such as motes

Researchers have employed diverse strategies for using micro-contact data in transmission models. In some cases

To help address and expand upon the questions raised in

Even the weighted “typical-day” techniques advanced in

Confirming with an extended temporal horizon and extending findings of

While static synthetic networks with contact distributions can produce results consistent with the full dynamic network, this apparent agreement can be due to an accumulation of counter-balancing inaccuracies leading to a deceptively correct outcome.

Overall, our findings suggest that longitudinal data collected over a prolonged period is required to accurately reconstruct dynamic or static contact networks for the purpose of simulating pathogen spread for at least some subpopulations, and that simulations based on summary networks tend to systematically overestimate the risk of infection. These findings are important because they suggest that traditionally calibrated disease models may contain biased disease parameter estimates to compensate for undetected inaccuracies in the underlying network and mixing models. These biases in disease parameter estimates might then lead to erroneous conclusions about the relative impact of interventions or the rate of disease propagation. Because such inaccuracies may not cancel in the same fashion when investigating intervention effects, additional caution should be employed when using such aggregation and generalization schemes. Unfortunately, while typical day

Methods

Given the H1N1 strain emergence and in anticipation of the significance of the 2009–2010 influenza season, the co-authors launched a previously-described

To study the impact of network representation on simulation outcomes, an H1N1 infection transmission model

The underlying disease parameters and distributions were held constant across scenarios. Within a given scenario, stochastics associated with exogenous and endogenous infection transmission and duration of different phases in the natural history of infection induced variability in simulation output.

Data collection

The Flunet experiment covered 57 weekdays and 33 weekends/holidays (including Christmas break). Participants were asked to carry a small wireless sensor (or “mote”) capable of short-range wireless communication

Transmission model

As the data collection occurred during the H1N1 outbreak, we used an agent based H1N1 SEIR transmission model to simulate the infection dynamics. This section includes a short description of the model; interested readers are referred to a detailed specification in

A susceptible agent contracting the infection from either exogenous or endogenous sources transitions to the Latent state. When an individual enters the Latent period, the model computed the duration for each of the subsequent four stages of illness (Figure

Simulation structure and flow

**Simulation structure and flow.** This figure demonstrates the agent state transition process (oval nodes) and model parameter sources (clouds) as a flow process. Parameters inform the probability of state transition changes.

Each infected agent experienced the four illness states sequentially with the passage of time. A person in the Asymptomatic Infectious or Symptomatic Infectious state was considered infective. At each time the infective person triggered potentially transmitting events (e.g. coughs or sneezes) with a specified likelihood. A potentially transmitting event had a given probability of transmission to each susceptible in contact. The simulation model did not consider H1N1 mortality, self-quarantine, antiviral administration, or hospitalization outcomes. While the H1N1 model employed has been previously used the capture the impact of vaccination

Connectivity patterns

This work seeks to analyze and quantify the impact of contact pattern representation on transmission outcomes in agent-based simulation models. Because we had recourse to data offering considerably greater temporal span than most other past contributions in this area, we could more readily examine the impact of varying levels of temporal aggregation on model outcomes. To do so, we looked at three different experimental manipulations, each associated with additional parameter variations focused on a particular type of network representation. A combination of manipulation and parameter variation is termed a scenario.

The first baseline manipulation focused on two reference scenarios, each representing different extremes in the aggregation spectrum. The first employed the Flunet empirical data directly, as it captures the contact patterns between agents with the greatest fidelity. In the dynamic baseline graph, edges have weights of 1 or 0 – that is, they correspond to the existence or absence of a connection during a given timeslot. The dynamic contact network can be visualized as a series of network where the nodes represent participants and each connections represents a pair of participants that were proximate to each other during a time step. This construct is often called a dynamic graph – drawing from the field of Graph Theory – where the participants are nodes, and the connections are the edges. This graph can either be realized in practice as a sparse dynamic graph with edges appearing and disappearing, or a time series of symmetric matrices with binary constituents, with 0 indicating the absence and 1 indicating the presence of an undirected edge.

The second baseline manipulation scenario collapsed the dynamic Flunet graph down to a single static graph. In the static graph scenario, edge weights are replaced by the time averaged contact density between a specific pair of participants over the entire study. This is trivially constructed as the sum of all the dynamic symmetric matrices divided by the number of timesteps. This aggregation approach captures an extreme form of aggregation, in which no changes are made in contact graph structure over time. If contact dynamics had no significant impact on the results, then simulations using this graph should echo the fully dynamic case.

The second manipulation examined typical day approaches. As a method of temporally extrapolating graphs collected over shorter time horizons, both

Reflecting the fact that many researchers lack recourse to high-fidelity empirical contact data for model integration, the third manipulation focused on fitted synthetic networks. Past contributions have generated random contact networks using small-world, scale free or other network topologies, often with contact probabilities represented as edge weights randomly drawn from other independent distributions. To analyze the impact of such an aggregate network representation on the spread of infection across the simulated population, the dynamic Flunet graph was reduced to distributions for edge weights and node degree and a set of new small-world contact graphs based on these distributions were created. To determine the impact of model fit on simulation results, several edge weight distributions were employed that provided increasingly accurate fit, at the cost of decreasing theoretical rigour.

Unweighted small-world connectivity graphs with 36 nodes were generated using the Watts and Strogatz model

Histogram for Distribution of Generated Aggregated Contact Duration

**Histogram for Distribution of Generated Aggregated Contact Duration.** This figure provides a heat map to demonstrating the distribution of generated aggregated contact durations used in the simulation. Each sub-graph represents the CCDF of contact duration for: **a**) the empirical distribution used in the FullA case; **b**) the two part power-law exponential distribution commonly used in practice; **c**) the three part power law-exponential-exponential distribution which provides a better fit to our data; and **d**) the best fit power law-exponential-exponential with outliers included as single empirical data points. The shade of the point represents the frequency with which a network contained exactly that contact duration-probability pair. Some slight fanning of the distribution in **b**, **c** and **d** at higher contact durations indicates that our network construction algorithm had good but not perfect reconstruction of contact durations when compared to the empirical baseline.

Weights were assigned to the edges of the unweighted small-world network by drawing the value for each edge from a distribution. For each of the scenarios within the synthetic network manipulation, we examined the effects of using three different fitted distributions, as well as drawing from the normalized empirical histogram of pairwise Flunet aggregated contact durations (ACDs).

Fitting of distributions was performed using linear regression in MATLAB. For linear regression to perform properly, data underwent log-log (power law fitting) or log-linear (exponential fitting) transformation prior to performing the piecewise fit. For fitted distributions, the fitted curves were required to achieve a R^{2} value exceeding 99%. Piecewise breakpoints were selected by iteratively changing the breakpoints, performing the regression, and manually selecting the point at which error began to increase sharply, but which still maintained a minimum R^{2} of 99% value for all the piecewise components.

Figure

Curve fitting methods of Aggregated Contact Duration

**Curve fitting methods of Aggregated Contact Duration.** This figure shows the four piece-wise fitting approaches for aggregated contact duration over a CCDF plot of the empirical data. Points represented by a (+) are indicative of the empirical probability of a contact duration. The solid red line corresponds to the two-piece power law-exponential distribution fit. The solid green line corresponds to the three-piece power law-exponential-exponential distribution fit. Text in blue or red corresponds to the break points for each fit section. Outliers (at the head and tail) not included in either fit are included in the SW3PTT case.

Infectious events were generated according to a Poisson process as described in

Detailed scenario description

The three primary experimental manipulations (baseline, typical day and synthetic network representations) are each represented by a set of scenarios describing the method for generating the contact graphs. Each scenario in turn is composed of a set of cases; for example, there are 57 possible typical day pairs to investigate using our methodology; by contrast, each baseline scenario is associated with just a single case. Each experimental case was simulated for 10,000 realizations, where a single realization is an agent-based simulation of the entire 92 day study period, yielding a total of 6.16 million realizations across all scenarios. Here we describe each scenario in detail.

1. **Full-Detailed Network (FullD):** FullD is the first scenario in the baseline experimental manipulations. The connectivity pattern in this case uses the complete contact information of participants throughout the study as a dynamic graph, preserving the chronological order of contacts. For each participant, his/her observed contacts with other individuals in the study at each of 265,000 thirty-second time-slots across the 3 months of the Flunet study were imported into the model to represent the connectivity pattern of the corresponding agent. In each realization, the model stepped through all timeslots sequentially, simulating infection transmission using the corresonding inter-agent connectivity graph.

2. **Full-Aggregated Network (FullA):** FullA provides an upper bound on the impact of temporal aggregation. The connectivity pattern of this scenario is similar to the FullD scenario, but the contacts are aggregated over time. To generate the connectivity pattern, the study-wide per-timeslot contact likelihood between any two given agents was calculated based on the Flunet database, and imported into the model. Assuming the probability of two given nodes contacting each other across the entire 92 day study period is

3. **Day-Detailed Network (DayD):** The Flunet study covered 57 weekdays and 34 weekends/holidays. The DayD scenario – the first scenario in the typical day manipulation – abstracts this as 57 cases, each related to a unique pair of weekday-weekend/holiday. The first 34 weekdays paired with each of the first 34 weekend/holidays, and the remainder of the weekdays are paired by repeating the first 23 weekend/holidays. Subsequently, each pair was used to generate the connectivity pattern between agents by replicating the weekday of the pair 57 times (for the non-holiday weekdays during the study period) and replicating weekend/holiday of the pair 34 times (during the weekends and holidays during the study period). Therefore, in each case of this scenario, the ordered contacts of a particular weekday-weekend pair were replicated to cover all remaining days of the simulation as well.

4. **Day-Aggregated Network (DayA):** This scenario – the second scenario in the typical day experimental manipulation – consists of a hybridization between DayD and FullA. Like DayD, it also consists of 57 cases, each based on one of the 57 weekday-weekend pairs. Similar to FullA, the contact network used is aggregated over time (here, over a day) and regenerated by each realization prior to simulation. For each of the 57 weekday-weekend pairs, we derived the weekday-specific per-timestep contact probability between any two given nodes based on the specific contact patterns seen in the weekday from that pair. A weekend-specific per-timestep contact probability was analogously derived for each pair of nodes. For each pair of participants, and each timeslot of each (non-holiday) weekday throughout the study period, samples were drawn from a Bernoulli distribution with the specified weekday-specific contact probability for that pair. Contacts in weekend timeslots were similarly defined.

5. **Small-World Network with Power Law-Exponential Fitted ACD (SW2P):** In the first scenario in the synthetic network experimental manipulation, the selected connectivity range and rewiring probability (described in the previous section) were used to construct 100 unweighted small-world networks. To determine the weights for edges, a distribution was generated using a two-piece Power Law-Exponential distribution fitted to the Flunet ACD curve based on ^{
th
} small-world network from the 2 piece distribution.

6. **Small-World Network with Power Law-Exponential-Exponential (SW3P):** To construct connectivity patterns in the second scenario of the synthetic network manipulation an approach similar to SW2P was followed, but a 3-piece Power Law-Exponential-Exponential distribution was used to fit the Flunet ACD empirical distribution. Here, 3 data points from the head and 4 data points from the tail of the Flunet distribution were removed to improve the fit. As with SW2P, 100 connectivity networks were created.

7. **Small-World Network with Power Law-Exponential-Exponential True Tail Replaced (SW3PTT):** In the curve-fitting for SW3P – the third scenario in the synthetic network manipulation – we cut 4 points from the tail to better fit the distributions. Although the eliminated tail portion is approximately 1% of all data points in the Flunet empirical distribution, their contact duration was substantially larger than that of other data points, resulting in a greater importance to the network-wide transmission of infection

8. **Small-World Network with Empirical ACD (SWE):** The fourth and final scenario in the synthetic network manipulation uses a specified connectivity range and rewiring probability to generate 100 small-world networks, where the weights for the edges in each network are drawn from the empirical Flunet ACD distribution.

Results

Flunet dataset characteristics

A preliminary analysis of the dataset is provided in

Flunet Findings

**Flunet Findings.** Flunet Findings **a**) Contact histogram by hour of day, **b**) CCDF of contact duration, **c**) Connectivity graph with threshold of 18 minutes per day average contact. Black nodes represent stationary nodes associated with a location, and are included in this graph for illustrative purposes only. **d**) Network span for close and all contacts.

Figure

To visually highlight the impact of cliques and place on the dataset, Figure

Given the importance of network structure, we consider the span of the network in Figure

Contact pattern representations effects

We sought to study how the representation of dynamic contact networks over time impacted simulated transmission of an influenza-like illness. In particular, we were interested in determining whether the choice of a sample day as a typical day had an impact on reliability of results, whether significant differences in outcome were observed when collapsing empirical contact durations into distributions, and whether representation of dynamic – rather than aggregate – contact networks significantly affected simulated outcomes.

To compare overall trends in the findings, a simulation outcome for each scenario described in the previous section was graphed as a box plot in Figure

Boxplot of Infected Count

**Boxplot of Infected Count.** The number of endogenous infections of each realization summed by case are displayed as boxplots, and the FullA and FullD scenarios are displayed as single-case lines. The two day cases are represented on the left, have means between FullA and FullD scenarios and substantial variance. The synthetically created networks move increasing fidelity with the FullA case from SW2P (the weakest contact duration fit) to SWE, based on the empirical distribution of contact durations.

Both typical day (DayA and DayD) distributions were associated with a much larger span and heavier tails than the other distributions, suggesting that contact dynamics variation between days was characterized by different distributions rather than the temporal variances in the disease model. Because daily contact density was not normally distributed, the means for the typical day scenarios are pulled higher by the long tails evident in Figure

Impact of network structure

In line with previous simulation-centric studies

It is common practice to calibrate models to historical data to raise confidence in a model’s predictive abilities

Impact of study period

By collecting data on a hypothesized typical day

Detailed Boxplot on Infected Count

**Detailed Boxplot on Infected Count.** The number of endogenous infections for FullD, FullA and DayD, indexed by case pairing from 1–57 and sorted from lowest total daily contact time across all participants to largest total contact time. Four bins of contact time are denoted with dotted lines corresponding to less than 30 hours, between 30 and 60 hours, between 60 and 90 and greater than 90 hours of total contact time. Both mean and variance increase with total contact time, and match poorly with the mean and variance of the FullD case.

While some stochastically induced variation is evident in the baseline scenarios (FullD, FullA), the variation between distributions when different days are chosen as typical and the variation among the realizations for many high contact duration days are substantially greater. The differences in endogenous infection counts between each day in the DayD scenario are easily accounted for when considering the probability of infection. On days with limited connectivity, there are both fewer transmission paths and fewer chances for an infectious event to result from a contact. As such, those few infectious events which do occur will tend to occur between few individuals, suppressing both the variance and the mean. In cases where many connections exist, the joint probability of infectious events and proximity increases, as do the number of individuals who could be infected. The impact of sampling error that can result from a typical day experimental design is apparent from the graph, but to more fully quantify this risk we attempted to determine how many of the 57 sampled day pairs could be viewed as typical.

We chose to define a typical day in a post-hoc fashion: typical days should lead to similar individual infection risks to the baseline network. To determine similarity, we computed Pearson’s correlation between each individual’s infection risk within the single case of the FullD scenario with that associated with each day-specific case in the DayD scenario. Only 3 day pairs (the 21^{st}, 54^{th}, and 56^{th}) were found to have significant (ρ > 0.7, p < 0.05), correlation with the FullD scenarios, and no day pairs with strong and significant (ρ > 0.8, p < 0.05) correlations were found. Individual infection rates are shown in Figure

Typical Day Infection Risk Per Person

**Typical Day Infection Risk Per Person.** Infection risk for 36 participants of three selected cases of DayD networks that exhibit the highest infection rate correlation with the FullD scenario. The FullD risk (denoted by o’s) demonstrates a moderate increase in risk across participants. The days which correlate by infection count (+, *, ∆) show a marked difference between high-infection count and low infection count individuals.

Based on our analysis, and subject to the limitation of our dataset, we concluded that typical day techniques can generate spurious rates of infection within closed populations and are unlikely to represent the richness of longitudinal datasets. Although we do not have the requisite study population to conclude that

Discussion

The results presented here simultaneously underscore the high rate of return in terms of model reliability of investing in the collection of micro-contact data, but suggest that not all such investments confer equivalent value. It is widely recognized that model aggregation (for example, in the imposition of random mixing assumptions) tends to overestimate the spread of contagion. The results presented here suggest the natural extension of this understanding to the network context, where simulations employing purely static networks – such as might be produced by traditional network reconstruction techniques based on contact tracing, diarying or survey instruments – are also demonstrated to be biased towards overestimation of contagion. While this overestimation is pronounced in our experiments, it appears likely that the degree of this overestimation will depend heavily on the characteristics of the contagion process – particularly the speed of transmission relative to the speed of contact formation and dissolution

In addition to highlighting the importance of fine-scale data collection, these results also suggest a minimum efficient scale for such data collection. Such results are important in that many healthcare researchers have sought to defray the cost and logistic burden of electronically mediated micro-data collection by restricting the duration of studies. While such study designs sidestep many challenges associated with power consumption and device failures, they do rely heavily on the assumption that the network dynamics observed during the data collection interval is in some sense representative of the network dynamics over the long term. While such assumptions hold greater plausibility when applied in highly structured settings such as secondary schools

Synthetic networks based on randomly generated networks and contact duration distributions are an appealing representation for extending results to larger populations. Our results highlight the need for care in the design of synthetic networks given that even high quality matches to the contact distribution using parametric distribution mixtures can lead to distortions of infection risks, primarily due to the impact of a few important outliers. When assessing the match fidelity between the synthetic results and results for the fully detailed data, it is easy to overlook an underestimation if it cancels out the overestimation due to an aggregated network whose outcomes misleadingly agree with historical or synthetic ground truth estimates. These biases could in turn distort the perceived trade-offs between interventions, or skew other important statistics regarding model dynamics. As is the case for random mixing models

The findings here have implications both for health research and epidemiological methodology. The results suggest that, while it is important to capture micro-contact data in many contexts, attempting to economize by reducing study duration may be penny wise and pound foolish. Use of micro-contact data appears to confer substantial benefits, but those benefits will be compromised – and potentially reversed – unless studies are of sufficient duration to capture day-to-day variability. While these results could be unique to smaller participant pools and flu-like infections, they likely apply to many important pathogens in high-risk populations of note such as institutional populations associated with long-term care facilities which are often not much larger than our participant pool.

We hope that the findings presented here will offer initial guidance in planning micro-contact studies, elevating the prospect that these studies will serve as a cost-effective means of enhancing health insight. Our results regarding the impact of network representation and parametric contact distribution approximations will also inform the data analysis methodologies used to analyze and generalize such data for transmission modeling. We have further described here a general methodology that can be replicated for future studies to assess the impact of aggregation on simulation results.

While our paper has made several important contributions to the field, as with all initial analyses, there are several shortcomings that should be addressed in future work. In fact, a major contribution of the paper is elucidating what the structure of future experiments should be to provide realistic samples of contact and disease dynamics for a given population. In particular, the role of sample size (

Previous contributions

The uncertainty over the apparent tradeoff between study size and duration is a strong motivation for further contact dynamic studies with substantially larger, heterogeneous populations observed over longer time periods. Larger longitudinal studies would permit the analysis of the relative sensitivity of different disease models to simplifications in sample size, population and study duration by permitting knock-out experiments where similar simulations could be performed over statistically significant subsets of the data, and compared against each other and the full data set.

Summary

We have demonstrated that the dynamics of empirical contact networks impact the outcome of simulation models, that aggregation over this data provides systematic overestimates of disease burden, that typical day data collection techniques employed elsewhere impose a risk. These findings are important as they inform both issues in microcontact study design and use of synthetic contact networks in calibrated models. However these findings are limited by the number of participants in the study and the single pathogen studied. In the future, we intend to investigate these impacts against larger, longer and more diverse datasets to determine if there is a point at which increasing study duration or sample size no longer materially affects simulation outcomes. We will also investigate the interaction between pathogen behavior and temporal sampling strategies. By providing this work we have established baseline insights into both the design of agent-based simulations of pathogen transmission and into the emerging discipline of contact dynamics acquisition and analysis.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MH designed and performed the simulations. WQ performed the synthetic network analysis. KGS and NDO conceived the study design and structure. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to acknowledge the Natural Sciences and Engineering Research Council of Canada for providing funding for this research and the University of Saskatchewan HPC Training Facilities for providing computational resources.

Pre-publication history

The pre-publication history for this paper can be accessed here: