A simulation study comparing aberration detection algorithms for syndromic surveillance
1 Public Health – Seattle and King County, 999 Third Avenue, Suite 500, Seattle WA, 98104, USA
2 Department of Epidemiology, University of Washington, Mail Box 357236, Seattle WA, 98195-7236, USA
3 Foundation for Healthcare Quality, 705 Second Avenue, Suite 703, Seattle WA, 98104, USA
BMC Medical Informatics and Decision Making 2007, 7:6 doi:10.1186/1472-6947-7-6Published: 1 March 2007
The usefulness of syndromic surveillance for early outbreak detection depends in part on effective statistical aberration detection. However, few published studies have compared different detection algorithms on identical data. In the largest simulation study conducted to date, we compared the performance of six aberration detection algorithms on simulated outbreaks superimposed on authentic syndromic surveillance data.
We compared three control-chart-based statistics, two exponential weighted moving averages, and a generalized linear model. We simulated 310 unique outbreak signals, and added these to actual daily counts of four syndromes monitored by Public Health – Seattle and King County's syndromic surveillance system. We compared the sensitivity of the six algorithms at detecting these simulated outbreaks at a fixed alert rate of 0.01.
Stratified by baseline or by outbreak distribution, duration, or size, the generalized linear model was more sensitive than the other algorithms and detected 54% (95% CI = 52%–56%) of the simulated epidemics when run at an alert rate of 0.01. However, all of the algorithms had poor sensitivity, particularly for outbreaks that did not begin with a surge of cases.
When tested on county-level data aggregated across age groups, these algorithms often did not perform well in detecting signals other than large, rapid increases in case counts relative to baseline levels.