Table 4

Main characteristics of the four selected HEE quality appraisal instruments
QHES instrument[21] BMJ guidelines[22] CHEC list[23] Philips guidelines[24]
Author; year; journal Chiou et al. 2003; Medical Care Drummond, Jefferson; 1996; British Medical Journal (BMJ) Evers et al. 2005; International Journal of Technology Assessment in Health Care Philips et al. 2006; Pharmacoeconomics
Affiliation of authors Academia and industry Academia Academia Academia
Published in a peer-reviewed journal Yes Yes Yes Yes
Number of references 35 48 30 22
Purpose To provide a grading system for assessing the quality of health economic evaluations To improve the transparency of reporting To develop a generally accepted criteria list for assessing the methodology of economic evaluation studies in systematic reviews To identify, review, and consolidate currently available guidelines in order to establish a synthesized and consistent quality assessment framework for decision analytic models
Development process Selection of criteria from 19 existing guidelines; Use of a conjoint analysis survey of 120 international experts to estimate weights for each criterion Not specified Selection of items from 15 existing guidelines; Use of a Delphi panel consisting of 23 international experts to generate the final criteria list Selection and formulation of items by reviewing and consolidating 15 existing guidelines for good practice in decision-analytic modeling in HTA
Time frame Before, during and after peer review Before, during, and after peer review After peer review Before, during, and after peer review
Target audience Producers and consumers of economic evaluations Producers and consumers of economic evaluations Consumers intending to conduct a systematic review of trial-based economic evaluations Producers and consumers of model-based economic evaluations
Preferred analytical technique Full economic evaluations: cost-minimization-, cost-effectiveness-, cost-utility-, cost-benefit-analysis Full economic evaluations: cost-minimization-, cost-effectiveness-, cost-utility-, cost-benefit-analysis Full economic evaluations based on clinical trials: cohort studies, case–control studies, randomized controlled clinical trials Full economic evaluations based on decision-analytic models
Standard reporting format included 16 questions which should be asked when appraising the quality of health economic evaluations Ten sections under the three headings of study design, data collection, and analysis and interpretation of results: study question, selection of alternatives, form of evaluation, effectiveness data, benefit measurement and valuation, costing, modeling, adjustments for timing of costs and benefits, allowance for uncertainty, and presentation of results 19 questions which should be asked when appraising the quality of health economic evaluations in systematic reviews 15 sections under the three key themes of structure, data, and consistency: statement of decision problem/objective, statement of scope/ perspective, rationale for structure, structural assumptions, strategies/comparators, model type, time horizon, disease states/pathways, cycle length, data identification, pre-model data analysis, data incorporation, assessment of uncertainty, internal consistency, and external consistency
Number of questions/criteria 16 35 19 61
Operationalization of the questions/criteria Yes/No Yes/No/Not clear/Not appropriate Yes/No Yes/No/Unclear/Not applicable
Use of a quality score Yes No No No
Assessment instruction No Yes Yes: http://www.beoz.unimaas.nl/chec/ webcite Yes
Pilot test of the guideline Yes: Ofman et al. [31] Not specified Yes, but no details given Yes

Langer

Langer BMC Health Services Research 2012 12:253   doi:10.1186/1472-6963-12-253

Open Data