Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Debate

How confidence intervals become confusion intervals

James McCormack1, Ben Vandermeer2 and G Michael Allan3*

Author Affiliations

1 Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver BC, Canada

2 Alberta Research Centre for Health Evidence, University of Alberta, Edmonton Alberta, Canada

3 Evidence-Based Medicine, Department of Family Medicine, University of Alberta, Room 1706 College Plaza, 8215 - 112 Street NW, Edmonton AB, Canada

For all author emails, please log on.

BMC Medical Research Methodology 2013, 13:134  doi:10.1186/1471-2288-13-134

Published: 31 October 2013

Abstract

Background

Controversies are common in medicine. Some arise when the conclusions of research publications directly contradict each other, creating uncertainty for frontline clinicians.

Discussion

In this paper, we review how researchers can look at very similar data yet have completely different conclusions based purely on an over-reliance of statistical significance and an unclear understanding of confidence intervals. The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings. We describe three examples of controversy around the potential benefit of a medication, a comparison between new medications, and a medication with a potential harm. The examples include the highest levels of evidence, both meta-analyses and randomized controlled trials. We will show how in each case the confidence intervals and point estimates were very similar. The only identifiable differences to account for the contrasting conclusions arise from the serendipitous finding of confidence intervals that either marginally cross or just fail to cross the line of statistical significance.

Summary

These opposing conclusions are false disagreements that create unnecessary clinical uncertainty. We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

Keywords:
Confidence intervals; Evidence based medicine; Statistical analysis; Statistical significance