• Top
• Objective
• Methods
• Results
• Conclusions

Objective

Analysing the results of heterogeneous studies of conceptually similar interventions presents a number of challenges. Some of these are specific to reviews of complex interventions and others are simply exacerbations of challenges that must be addressed in any systematic review, especially the challenge of exploring heterogeneity. The aim of this report is to describe and critically appraise methods used in the analysis of reviews published by the Cochrane EPOC Group.

Methods

We conducted a critical review of the analytic methods used in all completed EPOC reviews published in Issue 2, 2001 of The Cochrane Library. Descriptions of the analyses were extracted from the methods section and compared with results reported anywhere in the review. This was supplemented by discussion and correspondence with the reviewers and the editorial team as well as reference to relevant methodological literature. The analytic methods that were used were classified and critically appraised with respect to the potential for misrepresentation of the results of individual studies, misleading comparisons between studies and misleading summaries of effects across studies.

Results

24 EPOC reviews were published in Issue 2, 2001. The number of included studies ranged from zero to 55 (median 12). 19 reviews did not conduct quantitative analyses for one or more of the following reasons: 'clinical' heterogeneity (16 reviews), few included studies (5 reviews), missing data (2 reviews). Five reviews used a combination of quantitative and qualitative analyses. Only two reviews described in the methods section the main factors that were considered as possible sources of heterogeneity. Possibly misleading analytical errors were identified.

Conclusions

The potential to improve the analysis in reviews of complex interventions is limited by the number and quality of the included studies. Important improvements are possible despite these limitations. Similar improvements might be relevant for less complex interventions. Long-term improvements may depend more on improving the methods and reporting of original studies than on the methods used in reviews.

This abstract has been submitted by authors on behalf of the Cochrane EPOC editorial team.

Have something to say? Post a comment on this article!