Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Open Badges Debate

Evaluating treatments in health care: The instability of a one-legged stool

Bonnie J Kaplan1*, Gerald Giesbrecht1, Scott Shannon2 and Kevin McLeod3

Author Affiliations

1 Department of Paediatrics, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

2 Department of Psychiatry, University of Colorado, Denver, Colorado, USA

3 Bachelor of Health Science Program, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

For all author emails, please log on.

BMC Medical Research Methodology 2011, 11:65  doi:10.1186/1471-2288-11-65

Published: 11 May 2011



Both scientists and the public routinely refer to randomized controlled trials (RCTs) as being the 'gold standard' of scientific evidence. Although there is no question that placebo-controlled RCTs play a significant role in the evaluation of new pharmaceutical treatments, especially when it is important to rule out placebo effects, they have many inherent limitations which constrain their ability to inform medical decision making. The purpose of this paper is to raise questions about over-reliance on RCTs and to point out an additional perspective for evaluating healthcare evidence, as embodied in the Hill criteria. The arguments presented here are generally relevant to all areas of health care, though mental health applications provide the primary context for this essay.


This article first traces the history of RCTs, and then evaluates five of their major limitations: they often lack external validity, they have the potential for increasing health risk in the general population, they are no less likely to overestimate treatment effects than many other methods, they make a relatively weak contribution to clinical practice, and they are excessively expensive (leading to several additional vulnerabilities in the quality of evidence produced). Next, the nine Hill criteria are presented and discussed as a richer approach to the evaluation of health care treatments. Reliance on these multi-faceted criteria requires more analytical thinking than simply examining RCT data, but will also enhance confidence in the evaluation of novel treatments.


Excessive reliance on RCTs tends to stifle funding of other types of research, and publication of other forms of evidence. We call upon our research and clinical colleagues to consider additional methods of evaluating data, such as the Hill criteria. Over-reliance on RCTs is similar to resting all of health care evidence on a one-legged stool.