Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Research article

Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study

Bolaji E Egbewale, Martyn Lewis and Julius Sim*

BMC Medical Research Methodology 2014, 14:49  doi:10.1186/1471-2288-14-49

PubMed Commons is an experimental system of commenting on PubMed abstracts, introduced in October 2013. Comments are displayed on the abstract page, but during the initial closed pilot, only registered users can read or post comments. Any researcher who is listed as an author of an article indexed by PubMed is entitled to participate in the pilot. If you would like to participate and need an invitation, please email info@biomedcentral.com, giving the PubMed ID of an article on which you are an author. For more information, see the PubMed Commons FAQ.

Issues regarding simulation study and conclusions

Jonathan Bartlett   (2014-07-28 09:50)  London School of Hygiene and Tropical Medicine email









The topic of covariate adjustment in randomised trials is an important one. However, I believe the simulation study and conclusions of Edgewale are flawed, and moreover my concerns mirror those of one of the paper's original reviewer's (Gillian Raab), which appear not to have been dealt with.

The authors investigate the performance of three methods for analysing randomised trials with a single continuous outcome and a corresponding baseline measure. They focus on the issue of baseline imbalance, and conduct a simulation study where trial data are generated such that there is, on average, an imbalance at baseline between the two treatment groups. From their simulation results, the authors conclude that ANCOVA is unbiased, whereas analysis of change scores and an unadjusted comparison of outcomes (ignoring baseline) are biased.

The problem is that the above approach for generating data does not correspond to how data arise in randomised trials. As the authors explain in the introduction, randomisation guarantees balance in expectation or on average, but not balance in any given study. But in the simulation study conducted, data are generated so that there is systematically (i.e. on average) imbalance at baseline. It is therefore unsurprising that in this case the ANOVA analysis (a t-test of outcomes ignoring baseline) is biased. All this demonstrates is that if one has a confounder, and one does not adjust for the confounder, estimates are biased. Put another way, the simulations show the following: if one performs repeated randomised trials where patients are more likely to be allocated to one of the treatment groups if they have higher baseline values, then a ANOVA or analysis of change scores is biased. But of course this is not how patients are allocated to groups in a simple randomised trial!

In a randomised trial, provided that the randomisation procedure is not compromised, all three of the methods considered by the authors are unbiased, at least according to the statistical definition of bias of an estimator as the difference between the expectation of the estimator and the true parameter value. The methods do however differ in terms of precision/efficiency, and previously it has been shown that ANCOVA is superior in this regard to the other two methods. All of these results can be found in the following paper:

Yang L, Tsiatis A (2001). Efficiency study of estimators for a treatment effect in a pretest-posttest trial. The American Statistician; 55: 314-321.

Competing interests

No competing interests.

top

Post a comment