Open Access Research article

Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons

Fujian Song*, Allan Clark, Max O Bachmann and Jim Maas

Author Affiliations

Norwich Medical School, Faculty of Medicine and Health Science, University of East Anglia, Norwich, Norfolk, NR4 7TJ, UK

For all author emails, please log on.

BMC Medical Research Methodology 2012, 12:138  doi:10.1186/1471-2288-12-138

Published: 12 September 2012

Abstract

Background

Indirect treatment comparison (ITC) and mixed treatment comparisons (MTC) have been increasingly used in network meta-analyses. This simulation study comprehensively investigated statistical properties and performances of commonly used ITC and MTC methods, including simple ITC (the Bucher method), frequentist and Bayesian MTC methods.

Methods

A simple network of three sets of two-arm trials with a closed loop was simulated. Different simulation scenarios were based on different number of trials, assumed treatment effects, extent of heterogeneity, bias and inconsistency. The performance of the ITC and MTC methods was measured by the type I error, statistical power, observed bias and mean squared error (MSE).

Results

When there are no biases in primary studies, all ITC and MTC methods investigated are on average unbiased. Depending on the extent and direction of biases in different sets of studies, ITC and MTC methods may be more or less biased than direct treatment comparisons (DTC). Of the methods investigated, the simple ITC method has the largest mean squared error (MSE). The DTC is superior to the ITC in terms of statistical power and MSE. Under the simulated circumstances in which there are no systematic biases and inconsistencies, the performances of MTC methods are generally better than the performance of the corresponding DTC methods. For inconsistency detection in network meta-analysis, the methods evaluated are on average unbiased. The statistical power of commonly used methods for detecting inconsistency is very low.

Conclusions

The available methods for indirect and mixed treatment comparisons have different advantages and limitations, depending on whether data analysed satisfies underlying assumptions. To choose the most valid statistical methods for research synthesis, an appropriate assessment of primary studies included in evidence network is required.

Keywords:
Indirect comparison; Mixed treatment comparison; Network meta-analysis; Inconsistency; Bias; Type I error; Statistical power; Simulation evaluation