Table 2 |
|
| Challenges facing researchers conducting an IMPF | |
| Identifying all relevant studies | |
| · | Unavailability of IPD in some studies |
| · | Time-consuming and costly nature of obtaining, cleaning and analysing the IPD. |
| Issues within individual studies | |
| · | Dealing with skewed continuous variables and possible outliers. |
| · | Inability of IPD to overcome deficiencies of original studies, such as being retrospective rather than prospective, being too small for a multivariable analysis, missing important confounders, missing participant data or being of low methodological quality, etc. |
| · | How to assess the quality of studies identified |
| · | Re-analysing individual study IPD before considering meta-analysis. For a summary of important issues for the analysis of single prognostic factor studies see Holländer and Sauerbrei [9]. The re-analysis of a single study as the preliminary or first step toward a meta-analysis is influenced by and has consequences for the meta-analysis strategy (15). |
| Heterogeneity between studies | |
| · | Different definitions of disease or outcome; e.g. Noordzij et al.[44] note different definitions of hypocalcemia across studies, whilst the MeRGE [40] collaborators note the definition of acute myocardial infarction changed over time. |
| · | Different participant inclusion and exclusion criteria |
| · | Different methods of measuring the same prognostic factor, for example see difficulties described by Look et al [2]. |
| · | For survival data different lengths of follow-up |
| · | Factors measured at different points in time or at different stages of disease across studies; e.g. the MeRGE [40] collaborators note that the timing of echocardiography was variable in their included studies, although within 2 weeks of the index acute myocardial infarction |
| · | Different (or out-dated) treatments strategies, especially when a mixture of older and newer studies are combined; e.g. Yap et al. [36] state that a large proportion of the patients in their included trials did not receive common post-myocardial infarction therapy such as β-blockers and ACE inhibitor. |
| · | Insufficent information about treatment for some of the studies. |
| Statistical issues for meta-analysis | |
| · | Missing data, including: missing factor values and outcome data for some participants within a study, and unavailable factors in some studies |
| · | Inability to adjust prognostic effects for a consistent set of adjustment factors in each study |
| · | Different measurement techniques between studies may be acceptable for adjustment variables, but are critical for the variable of main interest |
| · | Insufficient information to separate patient outcomes more discretely, e.g. Thakkinstian et al. [37] could not separate chronic allograft nephropathy from graft rejection or acute rejection from chronic rejection |
| · | Imposed choice of cut-off levels when individual studies categorise their continuous variables and/or categorise their continuous outcomes in their provided IPD |
| · | Difficulty in using a continuous scale for continuous factors in meta-analysis when some studies give IPD values on a continuous cale and others do not (e.g. see Rovers et al. [43]) |
| · | Considering whether it is sensible and/or possible to investigate differential prognostic effects in subgroups |
| · | Potential for study-level confounding when assessing whether study covariates (e.g. year of publication) modify the prognostic effect. |
| · | Difficulty of interpreting summary meta-analysis results in the presence of heterogeneity (and heterogeneous populations) across studies. |
| Assessment of potential biases | |
| · | Potential for publication bias and availability bias |
| · | How to assess the robustness of IPD meta-analysis results to the inclusion/exclusion of studies only providing summary data; and how to combine IPD studies with summary data studies |
Abo-Zaid et al. BMC Medical Research Methodology 2012 12:56 doi:10.1186/1471-2288-12-56
