An evaluation of data quality in Canada’s Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011
1 School of Public Health and Health Systems, University of Waterloo, 200 University Avenue West, N2L 3G1, Waterloo, ON, Canada
2 Institute of Gerontology, University of Michigan, 300 North Ingalls, 48109, Ann Arbor, MI, USA
3 Ann Arbor VA Health Care Center, Geriatrics Research, Education and Clinical Center, 2215 Fuller Road, 48105, Ann Arbor, MI, USA
4 Institute for Aging Research, 1200 Centre Street, 02131, Boston, MA, USA
5 Saskatchewan Health Quality Council, Atrium Building, Innovation Place, 241 – 111 Research Drive, SK S7N 3R2, Saskatoon, Canada
6 IMS Brogan, Montreal 16720 Route Transcanadienne Kirkland, H9H 5M3, Quebec, Canada
7 Canadian Institute for Health Information, Home and Continuing Care, Ottawa, 495 Richmond Road, Suite 600, K2A 4H6, Ottawa, ON, Canada
BMC Medical Informatics and Decision Making 2013, 13:27 doi:10.1186/1472-6947-13-27Published: 26 February 2013
Evidence informed decision making in health policy development and clinical practice depends on the availability of valid and reliable data. The introduction of interRAI assessment systems in many countries has provided valuable new information that can be used to support case mix based payment systems, quality monitoring, outcome measurement and care planning. The Continuing Care Reporting System (CCRS) managed by the Canadian Institute for Health Information has served as a data repository supporting national implementation of the Resident Assessment Instrument (RAI 2.0) in Canada for more than 15 years. The present paper aims to evaluate data quality for the CCRS using an approach that may be generalizable to comparable data holdings internationally.
Data from the RAI 2.0 implementation in Complex Continuing Care (CCC) hospitals/units and Long Term Care (LTC) homes in Ontario were analyzed using various statistical techniques that provide evidence for trends in validity, reliability, and population attributes. Time series comparisons included evaluations of scale reliability, patterns of associations between items and scales that provide evidence about convergent validity, and measures of changes in population characteristics over time.
Data quality with respect to reliability, validity, completeness and freedom from logical coding errors was consistently high for the CCRS in both CCC and LTC settings. The addition of logic checks further improved data quality in both settings. The only notable change of concern was a substantial inflation in the percentage of long term care home residents qualifying for the Special Rehabilitation level of the Resource Utilization Groups (RUG-III) case mix system after the adoption of that system as part of the payment system for LTC.
The CCRS provides a robust, high quality data source that may be used to inform policy, clinical practice and service delivery in Ontario. Only one area of concern was noted, and the statistical techniques employed here may be readily used to target organizations with data quality problems in that (or any other) area. There was also evidence that data quality was good in both CCC and LTC settings from the outset of implementation, meaning data may be used from the entire time series. The methods employed here may continue to be used to monitor data quality in this province over time and they provide a benchmark for comparisons with other jurisdictions implementing the RAI 2.0 in similar populations.