Email updates

Keep up to date with the latest news and content from BMC Health Services Research and BioMed Central.

Open Access Highly Accessed Research article

Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada

Michèle Tony1, Monika Wagner1, Hanane Khoury1, Donna Rindress1, Tina Papastavros2, Paul Oh3 and Mireille M Goetghebeur14*

Author Affiliations

1 BioMedCom Consultants inc, Montréal, Québec Canada

2 Workplace Safety Insurance Board of Ontario, Toronto, Ontario, Canada

3 Toronto Rehabilitation Institute, Toronto, Ontario, Canada

4 Centre Hospitalier Universitaire Ste Justine, Montréal Québec, Canada

For all author emails, please log on.

BMC Health Services Research 2011, 11:329  doi:10.1186/1472-6963-11-329

Published: 30 November 2011



Consistent healthcare decisionmaking requires systematic consideration of decision criteria and evidence available to inform them. This can be tackled by combining multicriteria decision analysis (MCDA) and Health Technology Assessment (HTA). The objective of this study was to field-test a decision support framework (EVIDEM), explore its utility to a drug advisory committee and test its reliability over time.


Tramadol for chronic non-cancer pain was selected by the health plan as a case study relevant to their context. Based on extensive literature review, a by-criterion HTA report was developed to provide synthesized evidence for each criterion of the framework (14 criteria for the MCDA Core Model and 6 qualitative criteria for the Contextual Tool). During workshop sessions, committee members tested the framework in three steps by assigning: 1) weights to each criterion of the MCDA Core Model representing individual perspective; 2) scores for tramadol for each criterion of the MCDA Core Model using synthesized data; and 3) qualitative impacts of criteria of the Contextual Tool on the appraisal. Utility and reliability of the approach were explored through discussion, survey and test-retest. Agreement between test and retest data was analyzed by calculating intra-rater correlation coefficients (ICCs) for weights, scores and MCDA value estimates.


The framework was found useful by the drug advisory committee in supporting systematic consideration of a broad range of criteria to promote a consistent approach to appraising healthcare interventions. Directly integrated in the framework as a "by-criterion" HTA report, synthesized evidence for each criterion facilitated its consideration, although this was sometimes limited by lack of relevant data. Test-retest analysis showed fair to good consistency of weights, scores and MCDA value estimates at the individual level (ICC ranging from 0.676 to 0.698), thus lending some support for the reliability of the approach. Overall, committee members endorsed the inclusion of most framework criteria and revealed important areas of discussion, clarification and adaptation of the framework to the needs of the committee.


By promoting systematic consideration of all decision criteria and the underlying evidence, the framework allows a consistent approach to appraising healthcare interventions. Further testing and validation are needed to advance MCDA approaches in healthcare decisionmaking.