Email updates

Keep up to date with the latest news and content from BMC Medical Informatics and Decision Making and BioMed Central.

Open Access Research article

Electronic immunization data collection systems: application of an evaluation framework

Christine L Heidebrecht1, Jeffrey C Kwong12345, Michael Finkelstein26, Sherman D Quan7, Jennifer A Pereira1, Susan Quach1 and Shelley L Deeks12*

Author Affiliations

1 Public Health Ontario, Suite 300, 480 University Avenue, Toronto, ON M5G 1 V2, Canada

2 Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

3 Institute for Clinical Evaluative Sciences, Toronto, Canada

4 Department of Family and Community Medicine, University of Toronto, Toronto, Canada

5 University Health Network, Toronto, Canada

6 Toronto Public Health, Toronto, Canada

7 Trillium Health Partners, Mississauga, Canada

For all author emails, please log on.

BMC Medical Informatics and Decision Making 2014, 14:5  doi:10.1186/1472-6947-14-5

Published: 14 January 2014



Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units.


The Centers for Disease Control and Prevention’s Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations.


Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers’ immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform.


Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

Immunization; Information systems; Data collection; Program evaluation