Method for appraising model validity of randomised controlled trials of homeopathic treatment: multi-rater concordance study
1 British Homeopathic Association, Hahnemann House, 29 Park Street West, Luton LU1 3BE, UK
2 Royal London Hospital for Integrated Medicine, 60 Great Ormond Street, London WC1N 3HR, UK
3 LMHI Research Secretariat, Rue Taille Madame 23, B-1450 Chastre, Belgium
4 Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, MD 21201, USA
5 School of Public Health and Community Medicine, University of Washington, Seattle, WA 98195, USA
6 Center for Integrative Complementary Medicine, Shaare Zedek Medical Center, Jerusalem, Israel
7 Boiron, 20 Rue de la Liberation, 69110 Sainte Foy-lès-Lyon, France
8 (Formerly) Central Council for Research in Homeopathy, Department of AYUSH, Ministry of Health & Family Welfare, Government of India, New Delhi 110058, India
9 Samueli Institute, 1737 King Street (Suite 600), Alexandria, VA 22314, USA
10 Department of Clinical Medicine, Universidade Federal de Uberlândia, Uberlândia, Brazil
BMC Medical Research Methodology 2012, 12:49 doi:10.1186/1471-2288-12-49Published: 17 April 2012
A method for assessing the model validity of randomised controlled trials of homeopathy is needed. To date, only conventional standards for assessing intrinsic bias (internal validity) of trials have been invoked, with little recognition of the special characteristics of homeopathy. We aimed to identify relevant judgmental domains to use in assessing the model validity of homeopathic treatment (MVHT). We define MVHT as the extent to which a homeopathic intervention and the main measure of its outcome, as implemented in a randomised controlled trial (RCT), reflect 'state-of-the-art' homeopathic practice.
Using an iterative process, an international group of experts developed a set of six judgmental domains, with associated descriptive criteria. The domains address: (I) the rationale for the choice of the particular homeopathic intervention; (II) the homeopathic principles reflected in the intervention; (III) the extent of homeopathic practitioner input; (IV) the nature of the main outcome measure; (V) the capability of the main outcome measure to detect change; (VI) the length of follow-up to the endpoint of the study. Six papers reporting RCTs of homeopathy of varying design were randomly selected from the literature. A standard form was used to record each assessor's independent response per domain, using the optional verdicts 'Yes', 'Unclear', 'No'. Concordance among the eight verdicts per domain, across all six papers, was evaluated using the kappa (κ) statistic.
The six judgmental domains enabled MVHT to be assessed with 'fair' to 'almost perfect' concordance in each case. For the six RCTs examined, the method allowed MVHT to be classified overall as 'acceptable' in three, 'unclear' in two, and 'inadequate' in one.
Future systematic reviews of RCTs in homeopathy should adopt the MVHT method as part of a complete appraisal of trial validity.