Email updates

Keep up to date with the latest news and content from BMC Health Services Research and BioMed Central.

Open Access Research article

Effective population management practices in diabetes care - an observational study

Anne Frølich1*, Jim Bellows2, Bo Friis Nielsen3, Per Bruun Brockhoff3 and Martin Hefford4

Author Affiliations

1 Department of Integrated Healthcare, Bispebjerg Hospital, Copenhagen, Denmark

2 Care Management Institute, Kaiser Permanente, Oakland, California, USA

3 DTU Informatics, Technical University of Denmark, Kgs. Lyngby, Denmark

4 Hutt Valley District Health Board, Lower Hutt, New Zealand

For all author emails, please log on.

BMC Health Services Research 2010, 10:277  doi:10.1186/1472-6963-10-277

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6963/10/277


Received:9 April 2010
Accepted:21 September 2010
Published:21 September 2010

© 2010 Frølich et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Ensuring that evidence based medicine reaches patients with diabetes in the US and internationally is challenging. The chronic care model includes evidence based management practices which support evidence based care. However, despite numerous studies, it is unclear which practices are most effective. Few studies assess the effect of simultaneous practices implemented to varying degrees. The present study evaluates the effect of fifteen practices applied concurrently and takes variation in implementation levels into account while assessing the impact of diabetes care management practices on glycemic and lipid monitoring.

Methods

Fifteen management practices were identified. Implementation levels of the practices in 41 medical centres caring for 553,556 adults with diabetes were assessed from structured interviews with key informants. Stepwise logistic regression models with management practices as explanatory variables and glycemic and lipid monitoring as outcome variables were used to identify the diabetes care practices most associated with high performance.

Results

Of the 15 practices studied, only provider alerts were significantly associated with higher glycemic and lipid monitoring rates. The odds ratio for glycemic monitoring was 4.07 (p < 0.00001); the odds ratio for lipid monitoring was 1.63 (p < 0.006). Weaker associations were found between action plans and glycemic monitoring (odds ratio = 1.44; p < 0.03) and between guideline distribution and training and lipid monitoring (odds ratio = 1.46; p < 0.03). The covariates of gender, age, cardiac disease and depression significantly affected monitoring rates.

Conclusions

Of fifteen diabetes care management practices, our data indicate that high performance is most associated with provider alerts and more weakly associated with action plans and with guideline distribution and training. Lack of convergence in the literature on effective care management practices suggests that factors contributing to high performance may be highly context-dependent or that the factors involved may be too numerous or their implementation too nuanced to be reliably identified in observational studies.

Background

Diabetes mellitus is among the leading chronic diseases in the US and internationally [1,2]. Complications of diabetes can be reduced through appropriate medical care and behavior modification [3,4]. Ensuring that patients with diabetes receive evidence based care to control disease and reduce the risk of complications is a significant challenge; much evidence exists of the gap between optimal care and the actual care patients receive [5,6]. The chronic care model advocates an evidence based approach for disease management [7-9]. The model proposes several evidence based management practices organised in six interrelated elements supporting the implementation of evidence based medicine [10,11].

It is essential to know which combinations of practices are most effective at improving diabetes care. Results from meta-analyses, randomized controlled trials, reviews and observational studies evaluating the impact of management practices on diabetes care are conflicting, presenting divergent results; important issues regarding the chronic care model and the practices embedded within it remain unresolved [12-16].

A systematic review assessing the effect of management practices in diabetes care concluded that professional and organizational practices, individually and in combination, improved diabetes care processes [17]. Adding patient education and or improving the role of a nurse led to improved care outcomes. Few studies assess the effects of individual management practices when several practices are applied at the same time and analysed with multivariate statistical models [14-16,18-21].

Kerr has recommended that conflicting results among observational studies of care management could be addressed via a study design in which specific practices are carefully defined and the degree of practice implementation is consistently measured across diverse settings and compared to the outcomes delivered. The study reported here implements that approach [22].

We studied care management practices in a large U.S. integrated health care delivery system that has been recognized as a relatively high quality provider. It provides comprehensive care to several million members, including more than 500,000 adults with diabetes. Within this organisation, decentralized operating units result in variability in the way components of the chronic care model are implemented and in the relative emphasis on each component. Implementation varies between centers and for individual components within centers; consequently, local population care programs differ across the system.

The organisation has invested in standardized measurement of numerous quality indicators to enable comparisons over time and across operating units. Internal reporting shows variations in performance on process and outcome measures. Variability in population care practices and performance, together with standardized measurements, provide a ripe opportunity for learning about the impact of the chronic care model on diabetes care processes.

The purpose of this study was to identify management practices that affect glycemic and lipid monitoring in diabetes care in the context of several concurrent care management practices implemented at varying levels. We chose to use monitoring rates as our outcomes as they are solely a function of provider decision-making.

Methods

We conducted a cross-sectional observational study. We identified fifteen diabetes population care management practices by reviewing the literature on diabetes care and the Chronic Care Model. These are briefly described in Table 1. The extent of implementation of each practice was assessed at each medical center by a survey administered to a key informant. Survey items were mainly drawn or adapted from existing instruments, including the Assessment of Chronic Illness Care, the National Survey of Physician Organizations, and the Translating Research into Action for Diabetes instruments [23-25].

Table 1. Population care practices surveyed.

We interviewed one key informant at each of the 41 sites at the integrated health care delivery system. Most key informants were non-physician managers responsible for population based care or diabetes care; some were physician champions for diabetes care. Survey items elicited factual information rather than beliefs or opinions, and key informants were well positioned to provide the requested information.

We developed algorithms to summarize detailed survey information into fifteen summary scores representing distinct population management practices. Three population care experts blinded to the data weighted individual items to form summary practice scores ranging from a minimum of 0 to a maximum of 1. More extensive implementation resulted in a higher score. The range of implementation scores for 15 population management practices across 41 sites is shown in Table 2. Data was registered one year after the interviews took place, as we wanted to be sure that the practices were well-implemented and had exerted their effects.

Table 2. Variation in diabetes population care practice scores across sites.

Glycemic monitoring rate was defined as the percentage of members with diabetes who had one or more glycated hemoglobin (HbA1c) tests during one calendar year. Lipid monitoring rate was defined as the percentage of members with diabetes who had one or more low density lipoprotein-C (LDL-C) tests during one calendar year.

Person-level data on monitoring status were aggregated into strata defined by medical center (41 categories), age group (4 categories), gender, presence/absence of depression, and presence/absence of cardiovascular disease, and monitoring rates were calculated for all strata. Of the 1312 possible location-age-gender-comorbidity combinations, 1198 combinations were populated by one or more individuals. These 1198 strata were the units of observation for our analyses, with the outcome variables expressed as monitoring rates.

As we had only 41 medical centres and a large number of explanatory variables, we used stepwise regression analysis to obtain reasonably parsimonious models that were not overly parameterized. Care management practices were used as explanatory variables in a forward selection, stepwise logistic regression model, with random effects of medical centers and the observation level and dependent variables of glycemic and lipid monitoring rates. The medical center random effect captured excess variability between the 41 medical centers and played a major role in the statistical analysis in that it accounted for the medical center clustering effect of observations, hence providing the proper unbiased error estimates. The medical center random effect captured excess variability between the 41 medical centers and played a major role in the statistical analysis. The observation level random effect captured overdispersion in the data stemming from the fact that the effects of the independent variables and covariates do not completely explain the excess variation in outcomes [26]. The random effects associated with medical centres and observation levels were similar in both models, with rather high and comparable dispersion.

Each outcome was modeled separately, assessing the degree of association between various practices and the specific process measure. The models were fitted and compared by maximum likelihood methods using the "lmer" function of R. Even though we had a large number of explanatory variables and relatively few medical centers, our analysis did not demonstrate multicollinearity to an extent that would have made our models unstable.

Results

The 41 practice sites provided care for 553,556 adults with diabetes, 51% of which were male. Among patients with diabetes, 16% were also diagnosed with coronary artery disease (CAD) and 13% with depression. Four percent of patients with diabetes were 18 to 34 years of age, 18% were 35 to 49 years old, 38% were 50 to 64 years old, and 40% were aged 65 years and up. The mean HbA1c was 7.2% (55 mmol/mol) and the mean LDL-C level was 105.1 mg/dl (2.72 mmol/L).

The regression models showed that provider alerts significantly affected the likelihood of both glycemic and lipid monitoring. This care management practice, in which providers received reminders of appropriate care delivered as computerized prompts or paper chart attachments, had a strong effect on both glycemic and lipid monitoring rates, increasing the odds ratios for glycemic monitoring by 4.07 (p < 0.00001) and for lipid monitoring by 1.63 (p < 0.0006) (Table 3). Sites that scored highest on this practice had automated, computerized alerts integrated into electronic medical records.

Table 3. Parameter estimates, odds ratios, and P-values for the three significant management practices.

Two other practices affected monitoring rates. Guideline distribution and training increased the likelihood of glycemic monitoring; the odds ratio was 1.46 (p < 0.03). Action plans increased the likelihood of lipid monitoring; the odds ratio was 1.44 (p < 0.03).

The covariates of gender, age, CAD, and depression affected monitoring rates in both models. Gender differences for monitoring rates decreased with increasing age (Table 4). The combined effect of depression and age resulted in statistically higher odds ratios for both types of monitoring for members up to 64 years of age, while odds ratios were lower for those who were 65 or older (0.95 and 0.87) (Table 5). The combined effect of CAD and age only affected the likelihood of glycemic monitoring in adults between the ages of 50 and 64, the odds ratio was 1.18 (p < 0.0001). The combined effect of CAD and gender increased the likelihood of lipid monitoring significantly in both men and women.

Table 4. Parameter estimates, odds ratios, and P-values for age and sex and their interactions.

Table 5. Parameter estimates, odds ratios and P-values for age and sex and comorbidities the three significant management practices.

In summary, the models for glycemic and lipid monitoring showed highly consistent effects; we interpret this finding as genuine because the implementation level of provider alerts differed across sites for these outcomes.

Discussion

We assessed the individual effects of fifteen concurrently and variably implemented population care practices on two diabetes process of care measures. Provider alerts increased the likelihood of glycemic and lipid monitoring to a statistically significant degree. The effects of action plans and guideline distribution and training were significant at a level well below that of provider alerts.

Other studies support our finding that female gender is associated with lower quality of care for diabetes and/or cardiovascular disease [27-29]. Notably, we found that gender differences disappeared with increasing age. In our study, increasing age was markedly associated with higher monitoring rates, contradicting evidence that it is associated with poorer quality of care [30]. Overall, our data suggest that care might be more strongly affected by patient characteristics than by guideline recommendations.

Strengths of our study included the very large population managed at the care sites we studied and our ability to quantify the effect of practices taking implementation levels into account as recommended by Kerr [22]. Limitations to our study included the relatively narrow variation across sites; homogeneity of care management practices throughout the integrated health care delivery system stems from multiple factors, including clustering of sites within operationally consistent regions.

Other limitations are inherent in the observational study design, including the difficulty of measuring implementation levels and the large number of unmeasured factors and interactions that influence outcomes in real-world settings. Measuring implementation levels is a challenging endeavour about which relatively little has been written [14,15]. More interviews with different health professionals with a variety of competencies and understandings of provided care might have improved the understanding of the implementation level of the management practices. However, because the interviews covered diverse aspects of care, many respondents gathered information from colleagues in preparation for the interview, so the single interviews typically integrate input from several individuals. We generally found that we could assess the degree to which structures and processes were in place to support each practice, but not their quality, highlighting the possibility of measurement error. To understand the practices in greater detail, we also conducted a series of case studies on diabetes care at four sites [31]. However, we were unable to complete a detailed assessment at all 41 sites.

Our results are best interpreted in the context of previous related studies. Provider alerts creating point-of-care reminders have been previously identified as an effective strategy for improving diabetes care [14,16,17,32-34]. Varying effects of action plans and guideline distribution and training are shown in the literature [14,16,17,35-38].

Another study using multivariate statistical modelling, The Translating Research into Action for Diabetes (TRIAD) study of approximately 9000 patients, examined the association between intensity of disease management by physicians and diabetes process and outcomes measures [25]. The TRIAD study identified an association between physician reminders and both lipid and glycemic monitoring rates: however, the same effect was noted for two other practices, structured care management and performance feedback, for which we did not find an association with monitoring rates.

In a multivariate study conducted in the U.S. Veterans Health Administration, four other practices affected a composite process measure of quality (monitoring tests and physical examinations): the level of support for guideline efforts, regional office leadership for guideline use, hospital use of guideline performance data, and hospital culture [16]. Other studies assessing the impact of management practices on diabetes quality of care and relying on multivariate statistical models have identified a variety of practices as important to quality [15,16,18-21]. Financial incentives have attracted much attention but did not have significant impact on the screening rates for HbA1c values nor for LDL-Cholesterol. Recent studies show mixed results of financial incentives regarding quality of care improvements [39-41].The physician financial incentives in place at the time of this study were small (less than 3% of total physician income). In sum, the literature is characterized by inconsistent findings regarding effective management practices in diabetes care.

Several factors may explain the lack of conclusive evidence. Comparability between studies is poor. Definitions of care management practices vary, and the Cochrane Collaboration suggests aligning these [32]. However, even across our study sites, we were not able to ensure consistency in the operational definitions of care management practices because they had been developed at the local level within the integrated health care delivery system over many years.

Another cause of conflicting results in the existing literature may be the effect of organisational and cultural contexts on the frontline delivery of diabetes care. These factors are very challenging to measure and analyze, and we did not attempt to do so.

Future research identifying effective management practices should build on the methods we used, including measurement of implementation levels and multivariate statistical models to assess the effects of multiple practices implemented in combination. Future efforts should also seek to align definitions of management practices, implementation levels, and outcome variables. The effects of patient-level covariates and organisational context should be taken into consideration.

The purpose of research attempting to identify specific practices most associated with better outcomes is to inform future quality improvement efforts; scarce resources would arguably be most efficiently allocated to those practices. This vein of research essentially attempts to "reverse engineer" critical performance drivers so they can be disseminated broadly or further amplified. However, conflicting results from observational care management studies suggest that this approach may not be the best way to identify or prioritize improvement opportunities; context dependencies and implementation details may preclude identifying performance drivers via observational studies. Quality improvement teams should consider other approaches. Two sound alternatives are based on "learning from improvement": using rapid-cycle improvement methods and/or identifying high-performing sites with similar organisational contexts from which structures and processes can be adopted [42].

Conclusions

The diabetes care management practice of provider alerts improved glycemic and lipid monitoring rates in patients with diabetes. Action plans and guideline distribution and training also affected monitoring rates, although to a lesser degree. The covariates of gender, age, CAD and depression also impacted monitoring rates to a statistically significant degree, individually and in combination.

Our findings contribute to a knowledge base that contains somewhat sparse and often conflicting results. Observational studies with uncontrolled variation on multiple factors may be inadequate for identifying the practices that could contribute most to ongoing improvement in population care, and quality improvement teams should also consider other methods for identifying improvement opportunities.

Ethical approval

The study was reviewed and approved by the Institutional Review Boards for KP Northern California, KP Southern California, KP Northwest, KP Georgia, and KP Hawaii.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

AF designed the concept and conduct of the study and interpreted data, drafted the manuscript, and participated in the decision to submit it. JB designed the concept and conduct of the study, collected, analyzed and interpreted data, drafted the manuscript and participated in the decision to submit it. BFN conducted the data analysis and interpreted findings and helped to draft the manuscript. PBB conducted the data analysis and interpreted findings. MH interpreted data and drafted the manuscript. AF, JB, BFN, PBB, and MH read, commented, and approved the final version of the manuscript.

Acknowledgements

Jennifer Green provided English text revision and correction.

We want to thank the Rockwool Foundation who funded part of this study.

References

  1. Centers for Disease Control: Diabetes: successes and opportunities for population-based prevention and control.

    Atlanta, GA 2007. Publisher Full Text OpenURL

  2. Economic costs of diabetes in the U.S. In 2007

    Diabetes Care 2008, 31:596-615. PubMed Abstract | Publisher Full Text OpenURL

  3. Gaede P, Vedel P, Larsen N, Jensen GV, Parving HH, Pedersen O: Multifactorial intervention and cardiovascular disease in patients with type 2 diabetes.

    N Engl J Med 2003, 348:383-393. PubMed Abstract | Publisher Full Text OpenURL

  4. Patel A, MacMahon S, Chalmers J, Neal B, Billot L, Woodward M, Marre M, Cooper M, Glasziou P, Grobbee D, et al.: Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes.

    N Engl J Med 2008, 358:2560-2572. PubMed Abstract | Publisher Full Text OpenURL

  5. Institute of Medicine: Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001. Publisher Full Text OpenURL

  6. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States.

    N Engl J Med 2003, 348:2635-2645. PubMed Abstract | Publisher Full Text OpenURL

  7. Wagner EH, Austin BT, Von Korff M: Organizing care for patients with chronic illness.

    Milbank Q 1996, 74:511-544. PubMed Abstract | Publisher Full Text OpenURL

  8. Bodenheimer T, Wagner EH, Grumbach K: Improving primary care for patients with chronic illness: the chronic care model, Part 2.

    JAMA 2002, 288:1909-1914. PubMed Abstract | Publisher Full Text OpenURL

  9. Coleman K, Austin BT, Brach C, Wagner EH: Evidence on the Chronic Care Model in the new millennium.

    Health Aff (Millwood) 2009, 28:75-85. PubMed Abstract | Publisher Full Text OpenURL

  10. Walshe K, Rundall TG: Evidence-based management: from theory to practice in health care.

    Milbank Q 2001, 79:429-457.

    IV-V

    PubMed Abstract | Publisher Full Text OpenURL

  11. Shortell SM, Rundall TG, Hsu J: Improving patient care by linking evidence-based medicine and evidence-based management.

    JAMA 2007, 298:673-676. PubMed Abstract | Publisher Full Text OpenURL

  12. Choe HM, Mitrovich S, Dubay D, Hayward RA, Krein SL, Vijan S: Proactive case management of high-risk patients with type 2 diabetes mellitus by a clinical pharmacist: a randomized controlled trial.

    Am J Manag Care 2005, 11:253-260. PubMed Abstract | Publisher Full Text OpenURL

  13. Glasgow RE, Nutting PA, King DK, Nelson CC, Cutter G, Gaglio B, Rahm AK, Whitesides H: Randomized effectiveness trial of a computer-assisted intervention to improve diabetes care.

    Diabetes Care 2005, 28:33-39. PubMed Abstract | Publisher Full Text OpenURL

  14. Mangione CM, Gerzoff RB, Williamson DF, Steers WN, Kerr EA, Brown AF, Waitzfelder BE, Marrero DG, Dudley RA, Kim C, et al.: The association between quality of care and the intensity of diabetes disease management programs.

    Ann Intern Med 2006, 145:107-116. PubMed Abstract OpenURL

  15. Solberg LI, Asche SE, Pawlson LG, Scholle SH, Shih SC: Practice systems are associated with high-quality care for diabetes.

    Am J Manag Care 2008, 14:85-92. PubMed Abstract | Publisher Full Text OpenURL

  16. Ward MM, Yankey JW, Vaughn TE, BootsMiller BJ, Flach SD, Welke KF, Pendergast JF, Perlin J, Doebbeling BN: Physician process and patient outcome measures for diabetes care: relationships to organizational characteristics.

    Med Care 2004, 42:840-850. PubMed Abstract | Publisher Full Text OpenURL

  17. Renders CM, Valk GD, Griffin S, Wagner EH, Eijk JT, Assendelft WJ: Interventions to improve the management of diabetes mellitus in primary care, outpatient and community settings.

    Cochrane Database Syst Rev 2001, CD001481. PubMed Abstract | Publisher Full Text OpenURL

  18. Shojania KG, Ranji SR, McDonald KM, Grimshaw JM, Sundaram V, Rushakoff RJ, Owens DK: Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis.

    JAMA 2006, 296:427-440. PubMed Abstract | Publisher Full Text OpenURL

  19. Keating NL, Landrum MB, Landon BE, Ayanian JZ, Borbas C, Wolf R, Guadagnoli E: The influence of physicians' practice management strategies and financial arrangements on quality of care among patients with diabetes.

    Med Care 2004, 42:829-839. PubMed Abstract | Publisher Full Text OpenURL

  20. Fleming B, Silver A, Ocepek-Welikson K, Keller D: The relationship between organizational systems and clinical quality in diabetes care.

    Am J Manag Care 2004, 10:934-944. PubMed Abstract | Publisher Full Text OpenURL

  21. Jackson GL, Yano EM, Edelman D, Krein SL, Ibrahim MA, Carey TS, Lee SY, Hartmann KE, Dudley TK, Weinberger M: Veterans Affairs primary care organizational characteristics associated with better diabetes control.

    Am J Manag Care 2005, 11:225-237. PubMed Abstract | Publisher Full Text OpenURL

  22. Kerr EA: Clinical management strategies and diabetes quality: what can we learn from observational studies?

    Med Care 2004, 42:825-828. PubMed Abstract | Publisher Full Text OpenURL

  23. Bonomi AE, Wagner EH, Glasgow RE, VonKorff M: Assessment of chronic illness care (ACIC): a practical tool to measure quality improvement.

    Health Serv Res 2002, 37:791-820. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  24. Casalino L, Gillies RR, Shortell SM, Schmittdiel JA, Bodenheimer T, Robinson JC, Rundall T, Oswald N, Schauffler H, Wang MC: External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases.

    JAMA 2003, 289:434-441. PubMed Abstract | Publisher Full Text OpenURL

  25. The Translating Research Into Action for Diabetes (TRIAD) study: a multicenter study of diabetes in managed care

    Diabetes Care 2002, 25:386-389. PubMed Abstract | Publisher Full Text OpenURL

  26. Baggerly KA, Deng L, Morris JS, Aldaz CM: Overdispersed logistic regression for SAGE: modelling multiple groups and covariates.

    BMC Bioinformatics 2004, 5:144. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  27. Bird CE, Fremont AM, Bierman AS, Wickstrom S, Shah M, Rector T, Horstman T, Escarce JJ: Does quality of care for cardiovascular disease and diabetes differ by gender for enrollees in managed care plans?

    Womens Health Issues 2007, 17:131-138. PubMed Abstract | Publisher Full Text OpenURL

  28. Chou AF, Wong L, Weisman CS, Chan S, Bierman AS, Correa-de-Araujo R, Scholle SH: Gender disparities in cardiovascular disease care among commercial and medicare managed care plans.

    Womens Health Issues 2007, 17:139-149. PubMed Abstract | Publisher Full Text OpenURL

  29. Ferrara A, Mangione CM, Kim C, Marrero DG, Curb D, Stevens M, Selby JV: Sex disparities in control and treatment of modifiable cardiovascular disease risk factors among patients with diabetes: Translating Research Into Action for Diabetes (TRIAD) Study.

    Diabetes Care 2008, 31:69-74. PubMed Abstract | Publisher Full Text OpenURL

  30. Gnavi R, Picariello R, la Karaghiosoff L, Costa G, Giorda C: Determinants of quality in diabetes care process: The population-based Torino Study.

    Diabetes Care 2009, 32:1986-1992. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  31. MacPhail LH, Neuwirth EB, Bellows J: Coordination of diabetes care in four delivery models using an electronic health record.

    Med Care 2009, 47:993-999. PubMed Abstract | Publisher Full Text OpenURL

  32. Grant RW, Pirraglia PA, Meigs JB, Singer DE: Trends in complexity of diabetes care in the United States from 1991 to 2000.

    Arch Intern Med 2004, 164:1134-1139. PubMed Abstract | Publisher Full Text OpenURL

  33. Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H, Furuseth K: Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system.

    Int J Technol Assess Health Care 2000, 16:210-227. PubMed Abstract | Publisher Full Text OpenURL

  34. Ilag LL, Martin CL, Tabaei BP, Isaman DJ, Burke R, Greene DA, Herman WH: Improving diabetes processes of care in managed care.

    Diabetes Care 2003, 26:2722-2727. PubMed Abstract | Publisher Full Text OpenURL

  35. Benjamin EM, Schneider MS, Hinchey KT: Implementing practice guidelines for diabetes care using problem-based learning. A prospective controlled trial using firm systems.

    Diabetes Care 1999, 22:1672-1678. PubMed Abstract | Publisher Full Text OpenURL

  36. Davis DA, Thomson MA, Oxman AD, Haynes RB: Evidence for the effectiveness of CME. A review of 50 randomized controlled trials.

    JAMA 1992, 268:1111-1117. PubMed Abstract | Publisher Full Text OpenURL

  37. Handley M, MacGregor K, Schillinger D, Sharifi C, Wong S, Bodenheimer T: Using action plans to help primary care patients adopt healthy behaviors: a descriptive study.

    J Am Board Fam Med 2006, 19:224-231. PubMed Abstract | Publisher Full Text OpenURL

  38. Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, Miller RG, Thomas PA, Prokopowicz GP, Qayyum R, et al.: Effectiveness of continuing medical education.

    Evid Rep Technol Assess (Full Rep) 2007, 1-69. OpenURL

  39. Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M: Effects of pay for performance on the quality of primary care in England.

    N Engl J Med 2009, 361:368-378. PubMed Abstract | Publisher Full Text OpenURL

  40. Mehrotra A, Bodenheimer T, Dudley RA: Employers' efforts to measure and improve hospital quality: determinants of success.

    Health Aff(Millwood) 2003, 22:60-71. PubMed Abstract | Publisher Full Text OpenURL

  41. Greene SE, Nash DB: Pay for performance: an overview of the literature.

    Am J Med Qual 2009, 24:140-163. PubMed Abstract | Publisher Full Text OpenURL

  42. Berwick DM: The science of improvement.

    JAMA 2008, 299:1182-1184. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6963/10/277/prepub