Table 2

Challenges facing researchers conducting an IMPF
Identifying all relevant studies
 ·  Unavailability of IPD in some studies
 ·  Time-consuming and costly nature of obtaining, cleaning and analysing the IPD.
Issues within individual studies
 ·  Dealing with skewed continuous variables and possible outliers.
 ·  Inability of IPD to overcome deficiencies of original studies, such as being retrospective rather than prospective, being too small for a multivariable analysis, missing important confounders, missing participant data or being of low methodological quality, etc.
 ·  How to assess the quality of studies identified
 ·  Re-analysing individual study IPD before considering meta-analysis. For a summary of important issues for the analysis of single prognostic factor studies see Holländer and Sauerbrei [9]. The re-analysis of a single study as the preliminary or first step toward a meta-analysis is influenced by and has consequences for the meta-analysis strategy (15).
Heterogeneity between studies
 ·  Different definitions of disease or outcome; e.g. Noordzij et al.[44] note different definitions of hypocalcemia across studies, whilst the MeRGE [40] collaborators note the definition of acute myocardial infarction changed over time.
 ·  Different participant inclusion and exclusion criteria
 ·  Different methods of measuring the same prognostic factor, for example see difficulties described by Look et al [2].
 ·  For survival data different lengths of follow-up
 ·  Factors measured at different points in time or at different stages of disease across studies; e.g. the MeRGE [40] collaborators note that the timing of echocardiography was variable in their included studies, although within 2 weeks of the index acute myocardial infarction
 ·  Different (or out-dated) treatments strategies, especially when a mixture of older and newer studies are combined; e.g. Yap et al. [36] state that a large proportion of the patients in their included trials did not receive common post-myocardial infarction therapy such as β-blockers and ACE inhibitor.
 ·  Insufficent information about treatment for some of the studies.
Statistical issues for meta-analysis
 ·  Missing data, including: missing factor values and outcome data for some participants within a study, and unavailable factors in some studies
 ·  Inability to adjust prognostic effects for a consistent set of adjustment factors in each study
 ·  Different measurement techniques between studies may be acceptable for adjustment variables, but are critical for the variable of main interest
 ·  Insufficient information to separate patient outcomes more discretely, e.g. Thakkinstian et al. [37] could not separate chronic allograft nephropathy from graft rejection or acute rejection from chronic rejection
 ·  Imposed choice of cut-off levels when individual studies categorise their continuous variables and/or categorise their continuous outcomes in their provided IPD
 ·  Difficulty in using a continuous scale for continuous factors in meta-analysis when some studies give IPD values on a continuous cale and others do not (e.g. see Rovers et al. [43])
 ·  Considering whether it is sensible and/or possible to investigate differential prognostic effects in subgroups
 ·  Potential for study-level confounding when assessing whether study covariates (e.g. year of publication) modify the prognostic effect.
 ·  Difficulty of interpreting summary meta-analysis results in the presence of heterogeneity (and heterogeneous populations) across studies.
Assessment of potential biases
 ·  Potential for publication bias and availability bias
 ·  How to assess the robustness of IPD meta-analysis results to the inclusion/exclusion of studies only providing summary data; and how to combine IPD studies with summary data studies

Abo-Zaid et al.

Abo-Zaid et al. BMC Medical Research Methodology 2012 12:56   doi:10.1186/1471-2288-12-56

Open Data