Open Access Research article

Advancing current approaches to disease management evaluation: capitalizing on heterogeneity to understand what works and for whom

Arianne MJ Elissen1*, John L Adams2, Marieke Spreeuwenberg1, Inge GP Duimel-Peeters34, Cor Spreeuwenberg1, Ariel Linden56 and Hubertus JM Vrijhoef78

Author Affiliations

1 Department of Health Services Research, CAPHRI School for Public Health and Primary Care, MaastrichtUniversity, Duboisdomein 30, PO Box 616 6200MD, Maastricht, the Netherlands

2 Department of Research and Evaluation, Kaiser Permanente Center for Effectiveness and Safety Research, Pasadena, CA, USA

3 Department of General Practice, CAPHRI School for Public Health and Primary Care, Maastricht University, Maastricht, the Netherlands

4 Department of Patient and Care, Maastricht University Medical Centre, Maastricht, the Netherlands

5 Linden Consulting Group, Ann Arbor, MI, USA

6 Department of Health Management and Policy, School of Public Health, University of Michigan, Ann Arbor, MI, USA

7 TRANZO Scientific Centre for Care and Welfare, Tilburg University, Tilburg, the Netherlands

8 Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore

For all author emails, please log on.

BMC Medical Research Methodology 2013, 13:40  doi:10.1186/1471-2288-13-40

Published: 14 March 2013

Abstract

Background

Evaluating large-scale disease management interventions implemented in actual health care settings is a complex undertaking for which universally accepted methods do not exist. Fundamental issues, such as a lack of control patients and limited generalizability, hamper the use of the ‘gold-standard’ randomized controlled trial, while methodological shortcomings restrict the value of observational designs. Advancing methods for disease management evaluation in practice is pivotal to learn more about the impact of population-wide approaches. Methods must account for the presence of heterogeneity in effects, which necessitates a more granular assessment of outcomes.

Methods

This paper introduces multilevel regression methods as valuable techniques to evaluate ‘real-world’ disease management approaches in a manner that produces meaningful findings for everyday practice. In a worked example, these methods are applied to retrospectively gathered routine health care data covering a cohort of 105,056 diabetes patients who receive disease management for type 2 diabetes mellitus in the Netherlands. Multivariable, multilevel regression models are fitted to identify trends in clinical outcomes and correct for differences in characteristics of patients (age, disease duration, health status, diabetes complications, smoking status) and the intervention (measurement frequency and range, length of follow-up).

Results

After a median one year follow-up, the Dutch disease management approach was associated with small average improvements in systolic blood pressure and low-density lipoprotein, while a slight deterioration occurred in glycated hemoglobin. Differential findings suggest that patients with poorly controlled diabetes tend to benefit most from disease management in terms of improved clinical measures. Additionally, a greater measurement frequency was associated with better outcomes, while longer length of follow-up was accompanied by less positive results.

Conclusions

Despite concerted efforts to adjust for potential sources of confounding and bias, there ultimately are limits to the validity and reliability of findings from uncontrolled research based on routine intervention data. While our findings are supported by previous randomized research in other settings, the trends in outcome measures presented here may have alternative explanations. Further practice-based research, perhaps using historical data to retrospectively construct a control group, is necessary to confirm results and learn more about the impact of population-wide disease management.

Keywords:
Chronic disease management; Quality measurement; Evaluation methodology; Multilevel regression methods; Statistical heterogeneity