Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Research article

A random effects variance shift model for detecting and accommodating outliers in meta-analysis

Freedom N Gumedze1* and Dan Jackson2

Author Affiliations

1 Department of Statistical Sciences, University of Cape Town, Rondebosch, 7701, South Africa

2 MRC Biostatistics Unit, Institute of Public Health, Cambridge, UK

For all author emails, please log on.

BMC Medical Research Methodology 2011, 11:19  doi:10.1186/1471-2288-11-19

Published: 16 February 2011

Abstract

Background

Meta-analysis typically involves combining the estimates from independent studies in order to estimate a parameter of interest across a population of studies. However, outliers often occur even under the random effects model. The presence of such outliers could substantially alter the conclusions in a meta-analysis. This paper proposes a methodology for identifying and, if desired, downweighting studies that do not appear representative of the population they are thought to represent under the random effects model.

Methods

An outlier is taken as an observation (study result) with an inflated random effect variance. We used the likelihood ratio test statistic as an objective measure for determining whether observations have inflated variance and are therefore considered outliers. A parametric bootstrap procedure was used to obtain the sampling distribution of the likelihood ratio test statistics and to account for multiple testing. Our methods were applied to three illustrative and contrasting meta-analytic data sets.

Results

For the three meta-analytic data sets our methods gave robust inferences when the identified outliers were downweighted.

Conclusions

The proposed methodology provides a means to identify and, if desired, downweight outliers in meta-analysis. It does not eliminate them from the analysis however and we consider the proposed approach preferable to simply removing any or all apparently outlying results. We do not however propose that our methods in any way replace or diminish the standard random effects methodology that has proved so useful, rather they are helpful when used in conjunction with the random effects model.