Table 3

Simulation Results (Normal Distribution, Equal Experimental and Control Groups, Standard Deviation 40% of Control Mean Value).

% Bias

% Coverage

% Statistical Power

I2(%)





τ = 0s

τ = 0.5s

τ = 0s

τ = 0.5s

τ = 0s

τ 0.5s

τ 0.5s








Δ

n (exp/contr)

k

MD

SMD

RoM

MD

SMD

RoM

MD

SMD

RoM

MD

SMD

RoM

MD

SMD

RoM

MD

SMD

RoM

MD

SMD

RoM


SMD = 0.2

10/10

5

0

-4

0

0

-5

1

95

97

95

90

92

90

15

11

14

15

13

14

59

48

55

10

0

-5

0

1

-5

0

95

97

95

92

93

92

27

22

26

19

18

19

60

48

56

30

0

-5

0

1

-6

0

94

96

95

94

94

94

66

61

64

39

38

38

60

48

56


MD = 8

100/100

5

0

0

0

0

0

1

96

97

96

88

88

87

82

82

82

21

21

21

93

92

92

RoM = 1.08

10

0

0

0

0

0

0

96

96

96

92

92

91

99

99

99

27

27

28

93

92

92

30

0

0

0

0

0

0

96

96

96

94

94

94

100

100

100

56

58

59

93

92

92



SMD = 0.5

10/10

5

0

-4

0

0

-5

0

95

97

95

90

92

90

64

57

62

43

40

42

59

48

56

10

0

-5

0

0

-5

0

95

97

95

92

93

92

91

89

90

66

64

65

60

48

56

30

0

-5

0

0

-6

0

94

96

94

94

93

93

100

100

100

98

97

97

60

47

57


MD = 20

100/100

5

0

0

0

0

0

1

96

97

96

88

88

88

100

100

100

61

61

61

93

92

92

RoM = 1.2

10

0

0

0

0

0

0

96

96

96

92

92

91

100

100

100

85

86

86

93

92

92

30

0

0

0

0

-1

0

96

96

96

94

94

93

100

100

100

100

100

100

93

92

92



SMD = 0.8

10/10

5

0

-4

0

0

-5

0

95

97

95

90

92

90

95

94

95

75

72

73

59

47

56

10

0

-5

0

0

-5

0

95

96

95

92

92

92

100

100

100

95

95

95

60

47

57

30

0

-5

-1

0

-6

0

94

94

94

94

92

93

100

100

100

100

100

100

60

46

57


MD = 32

100/100

5

0

0

0

0

0

1

96

96

96

88

88

87

100

100

100

91

91

91

93

92

92

RoM = 1.32

10

0

0

0

0

0

1

96

96

96

92

92

91

100

100

100

100

100

100

93

91

92

30

0

0

0

0

-1

0

96

96

96

94

94

93

100

100

100

100

100

100

93

91

92


Results of 10,000 simulations per scenario with a standard deviation equal to 40% of the control mean, for each combination of effect size (0.2, 0.5, and 0.8 standard deviation units), number of patients (10/10 and 100/100 experimental/control patients per trial), and number of trials (5, 10, and 30). The "% Bias" columns show the bias of each effect measure (MD, SMD, RoM) expressed as percentages of the expected values (negative sign denotes less than expected value), with and without heterogeneity. The "% coverage" columns show the percentage of cases that the true value falls within the 95% confidence interval of the simulated result, with and without heterogeneity. The "% statistical power" columns show the percentage of cases that the 95% confidence interval of the simulated result yields a significant treatment effect (i.e. excluding zero for MD and SMD, and one for RoM), with and without heterogeneity. The I2 column shows the degree of heterogeneity only for the scenarios in which heterogeneity was introduced. (For all the scenarios without heterogeneity, the ratio of Q/(k-1) was close to unity as expected, corresponding to I2 = 0 [data not shown].)

Abbreviations for Table and Legend: contr – control, exp – experimental, I2 I2 heterogeneity measure, k – number of trials in each meta-analysis, n – number of experimental or number of control patients per trial, Q – Cochran's Q statistic for heterogeneity, MD – mean difference, RoM – ratio of means, s – standard deviation units, SMD – standardized mean difference.

Friedrich et al. BMC Medical Research Methodology 2008 8:32   doi:10.1186/1471-2288-8-32

Open Data