Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Research article

Undue reliance on I2 in assessing heterogeneity may mislead

Gerta Rücker12*, Guido Schwarzer12, James R Carpenter13 and Martin Schumacher1

Author affiliations

1 Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Germany

2 German Cochrane Centre, University Medical Center Freiburg, Germany

3 Medical Statistics Unit, London School of Hygiene and Tropical Medicine, London, UK

For all author emails, please log on.

Citation and License

BMC Medical Research Methodology 2008, 8:79  doi:10.1186/1471-2288-8-79

Published: 27 November 2008

Abstract

Background

The heterogeneity statistic I2, interpreted as the percentage of variability due to heterogeneity between studies rather than sampling error, depends on precision, that is, the size of the studies included.

Methods

Based on a real meta-analysis, we simulate artificially 'inflating' the sample size under the random effects model. For a given inflation factor M = 1, 2, 3,... and for each trial i, we create a M-inflated trial by drawing a treatment effect estimate from the random effects model, using <a onClick="popup('http://www.biomedcentral.com/1471-2288/8/79/mathml/M1','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1471-2288/8/79/mathml/M1">View MathML</a>/M as within-trial sampling variance.

Results

As precision increases, while estimates of the heterogeneity variance τ2 remain unchanged on average, estimates of I2 increase rapidly to nearly 100%. A similar phenomenon is apparent in a sample of 157 meta-analyses.

Conclusion

When deciding whether or not to pool treatment estimates in a meta-analysis, the yard-stick should be the clinical relevance of any heterogeneity present. τ2, rather than I2, is the appropriate measure for this purpose.