- Research article
- Open access
- Published:
Feasibility of using self-reported patient data in a national diabetes register
BMC Health Services Research volume 15, Article number: 553 (2015)
Abstract
Background
In order to improve recruitment of patients to the Norwegian diabetes register for adults, a questionnaire was designed to collect data directly from patients. The main aim of this study was to assess the agreement of questionnaire data with data reported to the Register from health care personnel during routine consultations.
Methods
Patient data were obtained by sending a questionnaire with 27 of the 41 Register variables to 3714 members of the Norwegian Diabetes Association. Questionnaire data were compared with data already in the Register. Paired t-tests, percentages of total agreement, percentages of “positive” answers and kappa coefficients (k) were used for comparing data.
Results
Of the 1645 replies (44.3 %), the Register already had data on 324 patients for comparison. Response rate for most variables was better from patients (ranging from 76–100 %) compared with health care professionals (33–100 %). For 17 of 25 assessable variables including diabetes duration, height, weight, HbA1c, drug treatment and several diabetes complications, agreement was substantial or better with kappa >0.60. Data on family history of premature heart disease (k–0.59), foot examination (k = 0.26), foot ulcer (k = 0.32) and arterial surgery (k = 0.24) seemed to be difficult to answer by patients, whereas data on physical activity and self-monitoring of glucose seemed to be better when reported by patients.
Conclusions
Patient response rate was acceptable, and data had good concordance with data from health care professionals for most variables. However, registers using patient questionnaires should compare questionnaire data with data from professionals at regular intervals.
Background
The prevalence of diabetes has increased rapidly in Norway as well as worldwide [1, 2]. In Norway approximately 4 % [3] of the population have diabetes, and the majority (85–90 %) of these have type 2 diabetes. Diabetes management is complex and studies have revealed that a substantial portion of the patients not treated according to guidelines [4, 5]. The Norwegian diabetes register for adults (18 years or older) was established in 2006 with the purpose of improving diabetes care, reducing diabetes complications, and enabling comparisons of several clinical outcome measures between treatment centres. The Norwegian Government funds the Register. General practitioners, hospital outpatient clinics and diabetologists practicing outside hospitals are encouraged to report data collected electronically during consultations to the Register, but participation is voluntary. Patients have to give written consent before their data can be included in the Register.
In 2010 the Register was in an early phase of development and had data on only 4 % of the known diabetes population in Norway (approximately 11 % of the type 1 diabetes population and 3 % of the type 2 diabetes population). We therefore wanted to explore the feasibility of using a questionnaire to collect information directly from patients and thereby increase recruitment to the Register. We are not aware that any other national diabetes register has used this recruitment method. It is important that the quality of data in a register is high and there is a need for continuous validation [6].
Several studies have assessed the agreement between self-reported data and medical records [7–9], however only a few studies have included diabetic patients, and none has used register data for comparison. Register data collected during consultations may be more “interactive” than regular medical record data. Further, most studies contain only a few variables such as diabetes duration, use of insulin, other antidiabetic therapy or antihypertensive therapy, HbA1c and eye-examination [7–9]. Some studies are based on patient interviews rather than questionnaires [8, 9]. The main aim of this study was to assess the agreement of a comprehensive set of patient questionnaire data with data collected electronically for the diabetes register during routine consultations.
Methods
Data collection
Hospital outpatient clinics and almost all General Practitioners use electronic medical records. This allows the Norwegian diabetes register for adults to capture data electronically using a specially designed diabetes medical record and data capture application at outpatient clinics, and a pop-up window (a less complicated diabetes record and data capture system) at the General Practitioners’ offices. Data is registered during regular follow-up consultations by doctors or nurses, and is transferred electronically to the Register in January every year for the preceding year.
Patient data for comparison was obtained by distributing a questionnaire and one reminder to people with diabetes who were members of the Norwegian Diabetes Association in three counties in northern Norway. The Norwegian Diabetes association is a voluntary, independent patient organization. The Association provides information, advice and support to help people manage their diabetes as well as campaigning to improve the quality of diabetes care and supporting research. The members are people with diabetes and their family members, health care workers and others with a special interest in diabetes. The counties were selected because they already had a relatively high percentage of patients enrolled in the Register (10 % of their estimated diabetes population, compared to 4 % for the whole country) thus making comparisons between register data and self-reported data possible.
The questionnaire data were obtained between August 2010 and March 2011. Register data obtained from health care professionals were mostly from 2010, however data on diabetes duration, type of diabetes, height, family history of diabetes, completed educational course for type 2-diabetes, and ethnicity could be from both 2009 and 2010. A complete list of variables and definitions in the diabetes register can be viewed on the Noklus (Norwegian Quality Improvement of Primary Health Care Laboratories) home site [10], the organization responsible for the day-to-day operation of the Register.
Variables
The questionnaire included 27 of the 41 variables in the Register. Variables that were considered to be difficult for the patient to understand or remember, such as cholesterol-values and the presence of foot pulses, were excluded. The wording of some of the register questions was modified to make them more readily understood. In this process we were guided by phrases used in a Norwegian health study (HUNT) [11]. The questionnaire was piloted on patients attending one general practice, and some minor changes were made. An overview of variables in the questionnaire is shown in Table 1.
Statistics
We decided to compare data without using health care professionals’ data as the gold standard as there was some time-lag between collection of patients’ and health care professionals’ data. In addition some information, e.g. data on health habits, might be more reliable when obtained directly from patients. If the Register had data from more than one source, the most recent information was used in the comparison. Eye- and foot examination data were excluded if reporting dates indicated that results were unavailable to either patients or health care professionals and thus making comparison impossible. No data had to be excluded for other variables because of a time discrepancy.
A paired t-test with a level of significance of <0.05 was used to compare continuous data such as mean diabetes duration, height, weight and HbA1c. The percentages of agreement, percentages of “positive answers” and the kappa coefficient were used as measures of concordance for categorical data. Binominal confidence intervals were used to compare percentages of positive answers between the two groups such as percentages of eye- and foot-examinations. Data for all variables with differences between patients and health care professionals were divided by type of diabetes (type 1 and type 2, respectively) to see if differences were related to type of diabetes. The kappa coefficient was interpreted as follows: <0.21 was considered poor to slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement, and 0.81–1.00 almost perfect agreement [12]. The Kappa coefficient corrects for chance agreement, but is sensitive to extremes in prevalence [13].
Ethical approval
The Register is approved by the Norwegian Data Inspectorate. The Regional Committee for Medical and Health Research Ethics decided that approval for the project was unnecessary as the study was a quality assurance project.
Results
A total of 3714 members with diabetes of the Diabetes Association in the three counties received the questionnaire, and 1645 replied (44.3 %). There was no difference in age, sex or type of diabetes between responders and non-responders based on membership information. We found that the Register had data on 324 of these patients (178 women and 146 men) for comparison with the self-reported data. One hundred and eighty-one had register data only from hospital outpatient clinics, 128 only from general practices, and 15 had data from both general practices and clinics. Overall, 39 % had type 1 diabetes, and 61 % had type 2 diabetes. As expected when the majority of register data comes from hospital clinics, there were more patients with type 1 diabetes among the 324 patients in the comparison group than among the responders to the questionnaire. Mean age for type 1 and type 2 diabetes were 49.1 and 64.1 years respectively. For Register data from health care professionals, the response rate varied from 33 % for questions regarding self-monitoring of blood glucose to 100 % on insulin administration. The response rate from the patients varied from 76 % on questions concerning HbA1c-results to 100 % on insulin administration. For almost all variables the response rate was substantially higher from patients compared with information obtained from health care professionals in the Register.
Basic data
Tables 2 and 3 depict the agreement for continuous data, i.e. diabetes duration, height, weight and HbA1c. Self-reported data showed significantly longer mean diabetes duration (16.9 vs. 16.1 y) and lower mean weight (82.8 vs. 84.2 kg) compared with register data. The mean diabetes duration stated by patients was significantly longer compared to mean diabetes duration stated by health care professionals only for type 2 diabetic patients (11.5 y vs. 10.1 y) (data not shown).
There was no difference in mean HbA1c, but for 8 % of the patients the HbA1c difference was 1.0 % or larger (Table 3). Agreement was not related to type of diabetes. Fourteen percent of patients with type 1 diabetes and 31 % of patients with type 2 diabetes were unaware of their last HbA1c value.
The agreement was substantial (k = 0.65) with respect to educational courses for type 2 diabetes with a match (portion of agreement) for 98/119 patients (Table 4). Regarding type of diabetes there was a mismatch for nine patients, but the files indicated probable faulty reporting not only from patients but also from health care professionals. Agreement on family history was almost perfect for diabetes (k = 0.82), but moderate for premature heart disease (k = 0.59) (Table 4). Ethnicity could not be assessed as almost no patients were in the “non-white” category.
Health habits
The response rate from health care professionals compared with patients was lower for smoking habits and physical activity, and strikingly lower for self-monitoring of blood glucose (33 vs. 99 %). Agreement was almost perfect for smoking habits (k = 0.86), but moderate for physical activity (k = 0.43) and self-monitoring of blood glucose (k = 0.58) (Table 4).
Eye and foot-examination
One hundred and fifty-three of the 324 patients (47.2 %) in the study had comparable data on eye examinations from both datasets, and agreement was moderate (k = 0.58). The patients did not report significantly more examinations than health care professionals.
The agreement on foot-examination was fair (k = 0.26) (Table 4). A significantly higher percentage of foot-examinations were reported from health care professionals than from patients.
Treatment
Drug treatment generally showed a high level of agreement with kappa-values ranging from 0.83 to 0.93 (Table 4). In the 13 instances where only patients reported using lipid-lowering therapy, 11 had stated the name of the statin used. There was no difference in time-lag in registration between patients who had a match for lipid-lowering therapy and patients with a mismatch, and agreement regarding statin use was better for people with type 1 diabetes (k = 0.97, 95 % CI: 0.91, 1.00) compared to people with type 2 diabetes (k = 0.72, 95 % CI: 0.60, 0.85).
Complications
The agreement was substantial for coronary heart disease (k = 0.78), severe hypoglycemia (k = 0.77), laser-treated retinopathy (k = 0.75) and stroke (k = 0.70), but only fair for foot ulcer (k = 0.32) and arterial lower limb surgery (k = 0.24) (Table 4). There were 17 foot ulcers reported from patients and only 6 from health care professionals. The patients reported only 2 of the 8 arterial interventions reported by health care professionals, whereas health care professionals reported only 2 of 7 interventions stated by patients. Amputation was not assessed due to very few events (data not shown).
Discussion
Main findings
The results indicate that for the majority of variables there was good concordance between registry data and self-reported data. All the continuous variables (Table 2) and most categorical variables showed substantial to almost perfect agreement with k > 0.60 (Table 4). For family history of premature heart disease, physical activity, self-monitoring of blood glucose and eye-examination agreement was moderate. For foot examination, foot ulcer and arterial surgery agreement was fair. For almost all questions the response rate was better from the patients. The percentage of patients completing questionnaires was acceptable.
Comparison with other studies and explanation of the results
Basic data
The finding that type 2 diabetic patients tend to report longer mean diabetes duration is similar to the studies of Midthjell [7] and Wada [14], although these studies did not relate findings to type of diabetes. The explanation may be that the onset of type 1 diabetes is more acute and dramatic compared with type 2 diabetes and therefore more easily remembered. Self-reported weight was lower, and this has also been reported in another study that compared self-reported and actual measurements [15]. However, data in the Register may be a mixture of weight reported to health care professionals and actual measurements, and some of the differences may be explained by home measurements without shoes and clothes.
Although HbA1c (response rate 76 %) was the variable with the lowest response rate in the patient questionnaire, other studies have shown that only 24–48 % had knowledge of their HbA1c value [9, 16]. Our study also showed good concordance for HbA1c-values in contrast to a previous study with primary care patients [9]. Only 8 % had an absolute difference of ≥1 percentage points (Table 3) which exceeds the difference of 0.5–1.0 % found to be clinically important in a study assessing critical differences between HbA1c results [17]. Patients answering the questionnaire in our study were members of the Norwegian Diabetes Association and this may explain the high percentage who were aware of their last HbA1c level.
The findings on family history are similar to other studies with a substantial to almost perfect agreement on diabetes [18] but moderate agreement on premature heart disease [19]. Two more studies on family history of heart disease have shown slight to fair agreement [18, 20], suggesting that this is a difficult question for the patient to answer, and that discussion with a health care professional may be needed to get a valid answer.
Health habits
We found almost perfect agreement for smoking habits in line with a comparable study by Tisnado [21]. A Canadian study on 4530 patients that compared self-reported smoking habits with urine samples to document smoking [22] also reported good agreement.
For physical activity and self-monitoring of blood glucose the agreement was moderate, but might be more valid when reported on a questionnaire than directly to a health care professional.
Eye- and foot examination
Our study showed moderate agreement for eye examination, but could not replicate the findings by Fowles [8] and Beckles [23] that patients tend to report (significantly) more eye examinations than health care personnel because records tend to miss information provided outside the clinic. However, few datasets were available for comparison in our study.
The fair agreement for foot-examination in a study by Tisnado [21] is confirmed in our study. Our study showed a higher percentage of foot-examinations compared to other studies [24, 25], possibly since only “yes” or “no” responses could be included, excluding missing data (no response) which may indicate that the examination had not been performed. The response rate among the patients was relatively low, suggesting that the date (month/year) of the last foot-examination is difficult to remember.
Drug treatment
Our study showed almost perfect agreement for all medications, in line with other studies [7, 8, 21]. In instances of mismatch, patient data were probably correct since a correct drug name was stated. Although we checked for the influence of time lag, reporting date differences may still account for some of the mismatches, or an alternative explanation could be that the drug file in the electronic medical records had not been updated. A further investigation comparing the Register data with data in the Norwegian Prescription Database is planned.
Complications
Our study showed substantial agreement for coronary heart disease, severe hypoglycemia, laser treated retinopathy and stroke, and fair agreement (with large confidence intervals) for foot ulcer and arterial surgery. We have categorized myocardial infarction, angina and bypass/blocking into a yes/no for coronary heart disease whilst other studies have assessed how patients remember each of these coronary artery disease complications. Previous studies have shown an agreement on myocardial infarction from moderate to substantial [19, 21, 26–28] and slight to moderate for angina [19, 29]. Regarding stroke, previous studies have shown a great variation and differ from fair to almost perfect agreement [19, 26, 28, 29].
Our study had better agreement for laser treated retinopathy compared with previous studies that compared all types of retinopathy, probably since laser treatment was easier to remember [21, 30]. For severe hypoglycemia we are not aware of similar studies. Arterial surgery is not a well-known complication to patients, and the fair agreement may indicate that this is a difficult question for the patient to answer. Our study confirmed the fair agreement on foot ulcer reported by Tisnado due to more ulcers being reported by patients [21]. Patients may also report smaller ulcers, which may be unrelated to diabetes, or not reported to health care personnel.
Limitations and strengths of the study
Patient data were obtained from members of the Norwegian Diabetes Association who may have had better knowledge of their disease than the general diabetes population, and possibly more frequent follow-up by health care professionals as many had type 1-diabetes. Further, a discrepancy in reporting dates between self-reported data and data collected from health care professionals may have occasionally led to mismatches.
On the other hand, the study obtained a high response rate from the patients for the majority of variables in the Register, and we were able to examine agreement between self-reported data and data from health care professionals for several variables that have not been evaluated in previous studies (type of diabetes, self-monitoring of blood glucose, hypoglycemia and participation in an educational course). We also explored differences in reporting related to type of diabetes for all variables. Only variables with differences (diabetes duration and lipid lowering therapy) are further discussed in the article.
Conclusions
Our study obtained an acceptable response rate to a questionnaire with a comprehensive set of variables about diabetes care. There was substantial or near perfect agreement (kappa > 0.60) between self-reported data and data reported to the register by health care professionals for 17 of 25 assessable variables. We suggest that registers can consider using self-reported data to obtain these variables. Variables with moderate or poor concordance (kappa < 0.60) such as family history of premature heart disease, foot examination, foot ulcer and arterial surgery should be obtained from health care workers. Self-reported data on physical activity and self-monitoring of blood glucose might be more reliable than data reported to the register by health care workers. Registers using patient questionnaires should compare questionnaire data with data from professionals at regular intervals to ensure data quality.
References
Krokstad S, Knudtsen MS. Public health development-The HUNT-study, Norway. 2011. www.ntnu.no/documents/10304/1130562/folkehelse-i-endring-huntrapport-2011.pdf. Accessed 16 Jan 2015.
Wild S, Roglic G, Green A, Sicree R, Hilary K. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care. 2004;27:1047–53.
Folkehelseinstituttet. Diabetes I Norge – Folkehelse rapporten 2014. http://www.fhi.no/eway/default.aspx?pid=239&trg=Content_7242&Main_6157=7239:0:25,8904&MainContent_7239=7242:0:25,8906&Content_7242=7244:110410::0:7243:1:::0:0. Accessed 24 Sept 2015.
Jenssen TG, Tonstad S, Claudi T, Midthjell K, Copper J. The gap between guidelines and practice in the treatment of type 2 diabetes. A nationwide survey in Norway. Diabetes Res Clin Pract. 2008;80:314–20.
Eliasson B, Cederholm J, Nilsson P, Gudbjörnsdóttir. The gap between guidelines and reality: type 2 diabetes in a National Diabetes Register 1996-2003. Diabet Med. 2005;22:1420–6.
Arts DG, De Keizer NF, Scheffer GJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002;9(6):600–11.
Midthjell K, Holmen J, Bjørndal A, Lund-Larsen PG. Is questionnaire information valid in the study of a chronic disease such as diabetes? The Nord-Trøndelag diabetes study. J Epidemiol Community Health. 1992;46:537–42.
Fowles JB, Rosheim K, Fowler EJ, Craft C, Arrichiello L. The validity of self-reported diabetes quality of care measures. Int J Qual Health Care. 1999;11:407–12.
Harwell TS, Dettori N, McDowall JM, Quesenberry K, Priest L, Butcher MK, et al. Do persons with diabetes know their (A1C) number? Diabetes Educ. 2002;28:99–105.
Norwegian Quality Improvement of Primary Health Care Laboratories http://www.noklus.no. Accessed 16. Jan 2015).
Holmen J, Midthjell K, Krüger Ø, Langhammer A, Holmen TL, Bratberg GH, et al. The Nord-Trøndelag Health Study 1995-97 (HUNT 2): Objectives, contents, methods and participation. Nor epidemiol. 2003;13:19–32.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.
Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.
Wada K, Yatsuya H, Ouyang P, Otsuka R, Mitsuhashi H, Takefuji S, et al. Self-reported medical history was generally accurate among Japanese workplace population. J Clin Epidemiol. 2009;62:306–13.
Gorber SC, Tremblay M, Moher D, Gorber B. A comparison of direct vs. self-report measures for assessing height, weight and body mass index: a systematic review. Obes Rev. 2007;8:307–26.
Casagrande SS, Burrows NR, Geiss LS, Bainbridge KE, Fradkin JE, Cowie CC. Diabetes knowledge and its relationship with achieving treatment recommendations in a national sample of people with type 2 diabetes. Diabetes Care. 2012;35:1556–65.
Skeie S, Perich C, Ricos C, Araczki A, Horvath AR, Oosterhuis WP, et al. Postanalytical External Quality Assessment of Blood Glucose and Hemoglobin A1c: An International Survey. Clin Chem. 2005;51:1145–53.
Alves E, Lunet N, Correia S, Morais V, Azevedo A, Barros H. Medical record review to recover missing data in a Portuguese birth cohort: agreement with self-reported data collected by questionnaire and inter-rater variability. Gac Sanit. 2011;25:211–9.
Englert H, Müller-Nordhorn J, Seewald S, Sonntag F, Völler H, Meyer-Sabellek W, et al. Is patient self-report an adequate tool for monitoring cardiovascular conditions in patients with hypercholesterolemia? J Public Health (Oxf). 2010;32:387–94.
Klungel OH, de Boer A, Paes A, Seidell JC, Bakker A. Cardiovascular diseases and risk factors in a population-based study in the Netherlands: agreement between questionnaire information and medical records. Neth J Med. 1999;55:177–83.
Tisnado DM, Adams JL, Liu H, Damberg CL, Chen W, Hu FA, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44:132–40.
Wong SL, Shields M, Leatherdale S, Malaison E, Hammond D. Assessment of validity of self-reported smoking status. Health Rep. 2012;23:1–7.
Beckles GL, Williamson DF, Brown AF, Gregg EW, Karter AJ, Kim C, et al. Agreement between self-reports and medical records was only fair in a cross-sectional study of performance of annual eye examinations among adults with diabetes in managed care. Med Care. 2007;45:876–83.
Claudi T, Cooper J, Skogoy K, HAuske MF, Melbye H. Diabetic care in Norwegian general practice. A report of current status from Salten and some regions in Rogaland. Tidsskr Nor Laegeforen. 1997;117:3661–4.
Claudi T, Ingskog W, Cooper JG, Jenum AK, Hausken MF. Quality of diabetes care in Norwegian general practice. Tidsskr Nor Laegeforen. 2008;128:2570–4.
Muggah E, Graves E, Bennett C, Manuel DG. Ascertainment of chronic diseases using population health data: a comparison of health administrative data and patient self-report. BMC Public Health. 2013;13:16.
Merkin SS, Cavanaugh K, Longenecker JG, Fink NE, Levey AS, Powe NR. Agreement of self-reported comorbid conditions with medical and physician reports varied by disease among end-stage renal disease patients. J Clinical Epidemiol. 2007;60:634–42.
Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol. 2004;57:1096–103.
Bush TL, Miller SR, Golden AL, Hale WE. Self-report and medical record report agreement of selected medical conditions in the elderly. Am J Public Health. 1989;79:1554–6.
Maclennan PA, McGwin G, Searcey K, Owsley C. Medical record validation of self-reported eye diseases and eye care utilization among older adults. Curr Eye Res. 2013;38:1–8.
Acknowledgement
This project was funded by Center of clinical documentation and evaluation (SKDE), Postboks 6, 9038 Tromsø.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
KFL was project supervisor, responsible for study design, data analysis, interpretation of data and drafting the manuscript. GT participated in the study design, data analysis, interpretation of data and drafting the manuscript. JC, TR, and SS participated in the data analysis and interpretation of data of the manuscript. All authors have revised the article critically and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Løvaas, K.F., Cooper, J.G., Sandberg, S. et al. Feasibility of using self-reported patient data in a national diabetes register. BMC Health Serv Res 15, 553 (2015). https://doi.org/10.1186/s12913-015-1226-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12913-015-1226-0