Skip to main content
  • Research article
  • Open access
  • Published:

The relative efficacy of nine osteoporosis medications for reducing the rate of fractures in post-menopausal women

Abstract

Background

In the absence of head-to-head trials, indirect comparisons of randomized placebo-controlled trials may provide a viable option to assess relative efficacy. The purpose was to estimate the relative efficacy of reduction of fractures in post-menopausal women, and to assess robustness of the results.

Methods

A systematic literature review of multiple databases identified randomized placebo-controlled trials with nine drugs for post-menopausal women. Odds ratio and 95% credibility intervals for the rates of hip, non-vertebral, vertebral, and wrist fractures for each drug and between drugs were derived using a Bayesian approach. A drug was ranked as the most efficacious if it had the highest posterior odds ratio, or had the highest effect size.

Results

30 studies including 59,209 patients reported fracture rates for nine drugs: alendronate (6 studies), denosumab (1 study), etidronate (8 studies), ibandronate (4 studies), raloxifene (1 study), risedronate (7 studies), strontium (2 study), teriparatide (1 study), and zoledronic acid (1 study). The drugs with the highest probability of reducing non-vertebral fractures was etidronate and teriparatide while the drugs with the highest probability of reducing vertebral, hip or wrist fractures were teriparatide, zoledronic acid and denosumab. The drugs with the largest effect size for vertebral fractures were zoledronic acid, teriparatide and denosumab, while the drugs with the highest effect size for non-vertebral, hip or wrist fractures were alendronate or risedronate. Estimates were consistent between Bayesian and classical approaches.

Conclusion

Teriparatide, zoledronic acid and denosumab have the highest probabilities of being most efficacious for non-vertebral and vertebral fractures, and having the greatest effect sizes. The estimates from indirect comparisons were robust to differences in methodology.

Peer Review reports

Background

Osteoporosis defined by low Bone Mineral Density (BMD) (i.e., 2.5 standard deviations below peak gender specific BMD), is a progressive disease with high prevalence affecting 1 in 3 women and 1 in 8 men by the time they reach 90 years of age [1]. The major concern with low BMD is the high risk of fractures to non-vertebral bones such as the wrist or to the hip. A hip fracture may require extended hospital stay, surgical repair and rehabilitation therapy, and is associated with increased risk of death [2]. In addition, osteoporosis can lead to vertebral fractures which are identified by clinical assessment through decreased patient height (i.e., stooped posture) or with compressed spinal vertebra that can be radiologically assessed [3].

To reduce the risk of fractures due to osteoporosis, drugs have been introduced to reduce the rate of bone loss and to increase the strength of the bones. The first bisphosphonate available in Canada was etidronate in 1995 followed by alendronate in 1998. In 2005, alendronate became generic, which introduced a large increase in the uptake of these drugs. In Canada in 2010 about 9% of the population age 50 years and over were receiving an osteoporosis drug. These drugs include the bisphosphonates (alendronate, etidronate, risedronate or ibandronate), Selective Estrogen Receptor Modulators (raloxifene) and anabolic agents (teriparatide). All of these have shown to be effective in reducing the rate of fractures relative to placebo [4]. Recent additions to pharmacotherapy for osteoporosis include denosumab, strontium and zoledronic acid. Accordingly, it would be clinically important to know an estimate of the relative treatment efficacy or ranking of the most efficacious drugs. A major gap in the evidence to identify the most efficacious drugs is the lack of randomized active-controlled trials, i.e., direct treatment comparison (DTC) evidence [4].

DTC evidence for osteoporosis is absent because such later stage III trials are more complex, expensive, and require larger sample sizes than earlier phase II randomized placebo-controlled trials (RPCT) [5]. Meanwhile, osteoporosis drugs have been approved for use or listed under reimbursement formularies based on RPCT evidence, even though there are skeptics on the benefits of RPCTs for estimating relative efficacy compared to currently available drugs and that RPCTs are unethical. In a recent New England Journal of Medicine debate, Stein [6] argued that RPCTs are unethical because of the withholding of proven therapies in the placebo allocation, while Rosen [7] argued that the therapies are only proven in high risk patients (prior fracture, BMD < -3, or higher fracture risk assessment) and the inclusion criteria that possess true equipoise should only include individuals who are at low risk or are non-responsive to mild therapies. However in Canada the Tri-Council Policy Statement on the Ethical Conduct for Research Involving Humans suggest that RPCTs are acceptable to establish existence of effect and adverse events of drugs with new pharmacological mechanisms [8]. In the absence of DTC evidence, indirect treatment comparisons (ITC) might be a promising technique that allows the synthesis of available RPCT evidence to make a suggestion on the effect of DTC [9]. The theoretical foundations of the ITC method were provided in 1997 by Bucher [10] for the pair wise division of odds ratios to produce a common odds ratio thereafter referred to as the Bucher Method (i.e., for 2 drugs A and C and placebo B, the odds ratio of A/B divided by odds ratio C/B produces an odds ratio of A/C). While DTC is the highest level of clinical evidence, there exists the rationale to use ITC analysis where DTC is absent and not likely to be forthcoming [10]. Even if DTC evidence was available, ITC evidence based on other trials may be useful because of differences in patient characteristics and study characteristics such as length of follow-up [11].

In the absence of DTC evidence for osteoporosis drugs, two ITC analyses have been conducted to assess the relative efficacy at reducing the rates of fractures in post-menopausal women [12, 13]. The first using a Bayesian analysis that looked at seven studies including four drugs zoledronic acid (1 study), alendronate (3 studies), ibandronate (1 study) and risedronate (2 studies). this indicated that zoledronic acid had the highest efficacy in preventing vertebral fractures [12]. The second and more comprehensive analysis included eight RPCTs which was an update involving the above four medications adding etidronate (1 study). Of the five medications analyzed zoledronic acid had the highest efficacy in preventing vertebral and hip fractures while risedronate had the highest efficacy in reducing non-vertebral non-hip fractures [13].

We believe we can build on this pioneering work. First, nine drugs are currently available in Canada, European or the United States for use with osteoporosis. The nine drugs include the above five (zoledronic acid, alendronate, ibandronate, and risedronate) in the recent ITC analysis plus four more drugs that were not previously included (denosumab, raloxifene, strontium, and teriparatide). In addition, there are key differences in patient characteristics across the studies such as age, BMD, and fracture history. Further adjustment for these factors might affect estimates of the relative efficacy between treatments.

The purpose of this paper is to build on the previous estimation of relative efficacy between osteoporosis drugs for the prevention of fractures. First, we update the literature on osteoporosis drugs to include recent additions in pharmacotherapy and recent RPCTs by conducting a multiple database systematic literature review. Second, we estimate the relative efficacy of reducing fractures of each drug versus placebo and between the drugs with Bayesian ITC analysis. Third, we conduct the ITC analysis using Bucher's method, a classical analysis approach. Finally, we estimated the relative efficacy after adjustment for baseline patient characteristics.

Methods

Literature Search

An electronic search of the following databases restricted to English was conducted from January 1990 to October 2009, and the search was continually updated by alerts until September 2010: EMBASE, Medline, Medline in Process, and Cochrane Database for Systematic Reviews, Evidence Based Reviews-American College of Physicians Journal, National Health Service (NHS) Database of Assessment of Reviews and Effectiveness (DARE), CINAHL. Specific searches were developed for each database with the aid of professional librarian and were based on MeSH headings and keywords: osteoporosis, fractures, and bones. Methodological filters for randomization were applied to Medline and EMBASE (see additional file 1 for the search strategy). We also conducted a bibliographic search on each article that was identified. Following the literature searches, all citations were incorporated into Reference Manager citation database software [14] and duplicates were identified and removed.

The inclusion criteria was that each article must have; 1) one of the nine osteoporosis drugs: alendronate, denosumab, etidronate, ibandronate, raloxifene, risedronate, strontium, teriparatide, or zoledronic acid, 2) have a RPCT design, 3) have only post-menopausal women as an inclusion criterion, and 4) report the rate of fractures as a primary or secondary outcome. Studies were excluded if they were studies that combined different trials, were subgroup analysis, or the outcomes were not fractures such as BMD.

Selection of trials for inclusion and data abstraction

At the first level of screening of the publications, the titles and abstracts of the citations that were obtained from the search strategy were reviewed for relevance and inclusion for full-text review. Of these articles passing to level two, the articles were reviewed as full text for relevance. After inclusion, data was abstracted to pre-specified abstraction forms and then entered into Microsoft Excel and the Bayesian meta-analysis software WinBUGS (B ayesian inference U sing G ibbs S ampling for Windows) [15]. For the literature retrieved based on the targeted review for systematic reviews and meta-analysis, the same 4 inclusion criteria applied. Literature screening was conducted by two independent reviewers, with consensus reached on all discrepancies.

Outcomes

The main outcomes were the rates of vertebral, non-vertebral, hip and wrist fractures. In addition, study characteristics (country, numbers of study centres, and patient follow up in years) and baseline patient characteristics (age in years, years since menopause, BMD of the hip reported as g/cm2, and history of fractures) were abstracted. Data abstraction was verified by a second independent reviewer. For each outcome, the unadjusted odds ratio for each drug versus placebo were estimate along with its 95% CrI. In addition, the adjusted for fracture between each drug comparator was estimated along with its 95% CrI (i.e., Odds ratio of A/B = odd ratio (A/B) divided by odds ratio of C/B).

Primary Statistical Analysis: Bayesian ITC estimate of relative efficacy versus placebo and other drugs

ITC was conducted for the unadjusted analysis using Bayesian methods in WinBUGS software version 1.4.3 [15], which performs Bayesian analysis using Markov Chain Monte Carlo methods (see additional file 2 for software code). We reported the analysis according to the R eporting O f B ayes used in clinical ST udies (ROBUST) criteria [16]. The outcome estimated was the mean and the 95% credibility interval of the posterior distribution of the odds ratio of the rate of fracture versus placebo and other drugs, for each fracture. For the Bayesian analysis, priors were predefined for the mean log odds ratio as a normal distribution with mean zero, and precision 0.001 representing weak prior information. Weak priors were chosen so that the final estimates for odds ratios are driven by the data, and not by any assumption made. For each outcome, we performed 100,000 simulations discarding the first 50,000 simulations to allow burn-in; two chains were run simultaneously. Convergence was assessed using all of the Geweke, Raftery-Lewis, Gelman-Rubin and Heidelberger-Welch tests. To make a comparison of all drugs to each in order to determine the most effect efficacious drug, the proportion of Markov chain iterations in which a drug had the highest odds ratio represented the probability of that drug being ranked the most efficacious. In addition, the effect size was estimated for each drug versus placebo, where effect size was defined as the ratio of the odds ratio for fracture of placebo versus drug divided by the standard error of the estimate of the odds ratio. A higher effect size indicates the drug has lower odds for fractures than placebo and/or that the standard error is small. Software code for WinBUGS is provided in appendix 2.

Assessing robustness: homogeneity and consistency of evidence

A number of steps were taken to assess the integrity of the ITC analysis [10, 17–20]. The assessments included; 1) assessing homogeneity in meta-analysis of each comparator and across comparators, and 2) checking the consistency of the ITC analysis between Bayesian and classical software, and 3) checking the consistency of the ITC analysis to DTC if available. If there is homogeneity within drugs and across drugs, and the ITC evidence is consistent across methodologies or with DTC evidence, then the ITC evidence in considered strong and free of bias [19].

Homogeneity with each drug and across each drug was assessed with Review Manager 5 software [21]. Heterogeneity was assessed with I2 with greater than 50% being moderate heterogeneity and greater than 70% being considerable heterogeneity as suggested by the Cochrane Handbook of Systematic Reviews [22]. Consistency of evidence was assessed by comparing the results of the Bayesian analysis to free software specifically created for ITC analysis [23]. This software package for ITC was released by the Canadian Agency for Drugs and Technologies in Health (CADTH) [24], a national agency in Canada that provides evidence based decisions and associated services for the national and provincial level governments. Checking consistency of ITC evidence versus DTC evidence was conducted by a search for meta-analysis of DTC evidence.

Adjustment for difference in baseline characteristics

Lastly, we checked whether differences in patient characteristics across drugs contributed to the relative efficacy estimates in the ITC analysis. We estimated the odds ratios for fracture reduction with classical meta-analysis with meta-regression with the log of the odds ratio as the dependent variable, and dummy variables were added for each of the drugs. Following the unadjusted results, we adjusted the ITC estimates with meta-regression to include the age in years, BMD in g\cm2, percent of subjects with history of a vertebral fracture. Meta-regression was conducted with STATA version 11.0 using the command metareg.

Results

Based on the literature review, 30 RPCTs that investigated the effect of drugs on the rate of fractures were identified. The results of the screening process are provided in the PRISMA diagram [25] in Figure 1 and the descriptions of the included studies are presented in Table 1. For the 9 drugs, 6 studies were for alendronate [26–31], 1 study for denosumab [32], 8 studies for etidronate [33–40], 4 studies for ibandronate [41–44], 1 study for raloxifene [45], 6 studies for risedronate [46–51], 2 studies for strontium [52, 53], 1 for teriparatide [54], and 1 for zoledronic Acid [55]. The participants in the studies included 59,209 patients. The participants had a mean age ranging in studies between 52 and 72 years of age, years since menopause ranged from 2.7 to 31.9 years, and the study durations were from 1 to 4 years. Baseline BMD in the hip ranged from 0.28 to 1.08 and the percentage of participants that had previous vertebral fractures were from 0% to 100%.

Figure 1
figure 1

PRISMA Flow Diagram describing selection process for included studies. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Flow Diagram describing selection process for included studies.

Table 1 Description of Study and Baseline Characteristics for Included Studies

Bayesian ITC estimate of relative efficacy versus placebo and other drugs

The estimates of relative efficacy of each drug versus placebo in the Bayesian meta-analysis is reported in table 2. For non-vertebral fractures, only alendronate OR = 0.81 (95%CrI: 0.66, 0.96) and risedronate OR = 0.77 (95%CrI: 0.60, 0.91) had significant reduction. Etidronate had the highest probability of being most efficacious (0.41) along with teriparatide (0.41). All other drugs had less than 0.10 probability of being most efficacious. However, the drugs with the highest effect size were Risedronate (16.4) and Alendronate (16.1), but these effect sizes were smaller than the effect sizes for vertebral fractures. Based on the probabilities of being most efficacious, etidronate and zoledronic acid are the most efficacious drugs. However since etidronate does not have significant effect versus placebo, teriparatide is the most efficacious drug. In the ITC head-to-head analysis (Table 3) there is not enough evidence to detect differences in efficacy between any of the drugs for non-vertebral fractures, although teriparatide, zoledronic acid and denosumab have the lowest numbers need to treat to prevent a non-vertebral fracture versus the other drugs.

Table 2 Odds Ratio for Fracture, Indirect Treatment Comparison Results of Drug versus Placebo Classical and Bayesian analysis
Table 3 Odds Ratio for Fracture, Indirect Treatment Comparison between drugs (Bayesian analysis)

For vertebral fractures, all drugs except etidronate had significant reductions in the odds of a fracture. The drugs with the highest probability of being most efficacious are teriparatide (0.30), zoledronic acid (0.40) and denosumab (0.20). However, the drugs with the highest effect size were also teriparatide (29.8), zoledronic acid (66.2) and denosumab (53.6) based on probabilities and effect size the three drugs are most efficacious. In addition, these 3 drugs also had the lowest number needed to treat versus the other drugs (Table 3). In the ITC head-to-head analysis, teriparatide had significant reduction in vertebral fracture versus ibandronate and raloxifene, while denosumab had significant reductions versus alendronate, raloxifene, and risedronate. Zoledronic acid had significant reductions versus alendronate, raloxifene, and risedronate (Table 3).

For hip fractures, only alendronate has a significant reduction in relative rate of fractures (OR = 0.59 (95%CrI: 0.29 to 0.99). The drugs that had the highest probability of most efficacious were teriparatide (0.44) and etidronate (0.19). The drugs with the highest effect size were alendronate (9.49) and risedronate (5.71). Based on probabilities and effect size it is unclear which drug might be ranked most efficacious out of the choices of teriparatide or alendronate. In the ITC head-to-head analysis, the relative efficacy of teriparatide versus alendronate was OR = 1.35 (95%CrI: 0.07, 5.71) which is a non-significant finding. There were no drugs that had a significant benefit for hip fractures versus the other drugs (Table 3).

For wrist fractures, there were no drugs that a significant protective effect versus placebo, although no wrist fracture data was available for denosumab, ibandronate or zoledronic acid. The drugs that had the highest probability of most efficacious were teriparatide (0.41) and risedronate (0.22). The drugs with the highest effect size were alendronate (1.80) and risedronate (1.37), although the magnitude of the effect size was considerably lower than for other fractures. Based on probabilities and effect size it is unclear which drug might be ranked most efficacious out of the choices of teriparatide or alendronate. In the ITC head-to-head analysis, the relative efficacy of teriparatide versus alendronate was OR = 1.69 (95%CrI: 0.04, 8.09) which is a non-significant finding. There were no drugs that had a significant benefit for wrist fractures versus the other drugs (Table 3).

Assessing robustness: homogeneity and consistency of evidence

There was no difference between the estimates of the odds ratio and confidence or credibility intervals between the classical ITC software and the Bayesian WinBUGS ITC analysis.

For non-vertebral fractures, the evidence was considered strong and free of bias because of low heterogeneity, and similarity of classical results to the Bayesian analysis. For non-vertebral fractures, the overall odds ratio across all drugs was OR = 0.81 (95% CI: 0.77, 0.86), (P < 0.01) indicating a protective effect of pharmacotherapy (Figure 2). There was no heterogeneity between types of drugs (I2 = 0), although low heterogeneity (I2 = 16%) existed for alendronate.

Figure 2
figure 2

Forest plot non vertebral fractures. Odds ratio of non vertebral fractures for drugs versus placebo using Classical meta-analysis approach.

For vertebral fractures, the evidence is considered less strong than the evidence from non-vertebral fractures because of increasing heterogeneity (Figure 3), and the classical analysis having smaller confidence intervals than the Bayesian analysis. In the classical meta-analysis, the overall effect across all drugs was a protective effect in preventing vertebral fractures, OR = 0.49 (95% CI: 0.41, 0.58), and there was considerable heterogeneity across all drugs (I2 = 84%), while there was no heterogeneity within drugs. All drugs except one provided significant predictive effects with the exception being etidronate, which produced a p-value of 0.10. Conversely, in the Bayesian analysis only risedronate and alendronate had significant odds ratios relative to placebo.

Figure 3
figure 3

Forest plot vertebral fractures. Odds ratio of vertebral fractures for drugs versus placebo using Classical meta-analysis approach.

For hip fractures, the evidence is considered less strong than the evidence from non-vertebral fractures because of decreased confidence intervals in the classical analysis (Figure 4). For hip fractures, there was an overall protective effect against hip fracture for all drugs, OR = 0.73 (95% CI: 0.63, 0.84), and absence of heterogeneity (I2 = 0%). Three drugs reported an independent statistical reduction in the rate of hip fracture, alendronate, OR = 0.62 (95% CI: 0.40, 0.96), denosumab OR = 0.60 (95% CI: 0.37, 0.98), and risedronate OR = 0.74 (95% CI: 0.58, 0.94). This is in contrast to the Bayesian analysis where only alendronate reported a significant reduction in the odds ratios for hip fracture.

Figure 4
figure 4

Forest plot hip fractures. Odds ratio of hip fractures for drugs versus placebo using Classical meta-analysis approach.

For wrist fractures, the evidence is considered weak because of increasing heterogeneity and differences in the classical versus Bayesian analysis when drugs were compared to placebo (Figure 5). For wrist fracture, there was not an overall protective effect OR = 0.88 (95% CI: 0.77, 1.01), and the heterogeneity was substantial (I2 = 64%). The only drug that had a significant protective effect alone was risedronate OR = 0.71 (95% CI: 0.56, 0.89). The analysis of alendronate alone had considerable heterogeneity (I2 = 79%). Removing Cummings and Greenspan to produce comparable ITC evidence reduced the heterogeneity to 0% and the odds ratio to OR = 0.44 (95% CI: 0.30 to 0.67) for alendronate versus placebo. Removing Cummings and Greenspan produced an overall odds ratio for all drugs OR = 0.82 (95% CI: 0.71, 0.94: I2 = 59%).

Figure 5
figure 5

Forest plot wrist fractures. Odds ratio of wrist fractures for drugs versus placebo using Classical meta-analysis approach.

Adjustment for difference in baseline characteristics

The estimates of the relative efficacy with meta-regression for each drug versus placebo for each type of fracture were similar to the estimates of Bayesian analysis for odds ratios. Unfortunately, when baseline characteristics were added to the regression equation, there were not enough studies for the analysis and no estimate could be provided. This lack of result was created by the addition of the baseline characteristics age, BMD and rate of prior vertebral fractures that created multi-collinearity that was detected by exploded confidence intervals for each drug effect. When we ran the regression with only the top two drugs for each fracture along with adding in any of age, BMD or rates of prior vertebral fractures, the latter effects were significant while the drug effects was not significant. This suggests that the differences across studies in baseline characteristics contribute more to variation in odds ratio of fractures across studies than changes in the drugs.

Discussion

The objective was to update the literature on the relative efficacy of different osteoporosis medications to prevent four types of osteoporosis-related fractures. Based on the combination of effect size and probability of being most efficacious, teriparatide zoledronic acid and denosumab are consistently ranked highest for reducing non-vertebral and vertebral fractures, the two most common types of fractures

Etidronate is also ranked high on probability of being most efficacious but there are reservations with this result. First, etidronate does not have a statistically significant odds ratio versus placebo for non-vertebral fracture, but was ranked highest for being efficacious. The higher ranking may be due to a wide confidence interval that covers a lower region of odds ratio creating a favourable relative result over that region of low odds ratio. This suggests a limitation with this analysis where a requirement may be that the odds ratio for different drugs should have similar widths. A second caution with the results for etidronate is that the trials were small resulting in small effect sizes and the trials were conducted prior to the year 2000. This suggests that there is a lack of current strong evidence for the efficacy of etidronate versus placebo. As a result of these two limitations, this analysis suggests that etidronate should not be considered among the most efficacious drugs based on current evidence.

In addition, the number needed to treat analysis that treating as few as 10 patients with teriparatide, zoledronic acid or denosumab will produce 1 less fracture than if the patients were on other drugs.

This work updates the most recent study for ITC analysis in osteoporosis medications which looked at vertebral, hip and nonvertebral nonhip fractures [13] for five drugs, zoledronic acid, alendronate, ibandronate, risedronate and etidronate. Based on that analysis zoledronic acid had a 0.79 probability of being the most efficacious for vertebral fractures. In our analysis, teriparatide (0.40) and etidronate (0.40) had the highest probability of being the most efficacious. In our analysis, we included more studies for etidronate, alendronate, and risedronate in addition to adding denosumab, raloxifene, strontium and teriparatide. Similarly, the earlier work reported that zoledronic acid had the highest probability of preventing hip fractures, while our analysis indicates the most efficacious drugs are teriparatide (0.44), and that zoledronic acid (0.11), etidronate (0.19), denosumab (0.12) and alendronate (0.10) could be the most efficacious treatment. One key difference between inclusions of different studies was that we analyzed wrist fractures specifically while the earlier work reported on nonvertebral nonhip fractures [13]. We report that risedronate does have a high probability of being most efficacious similar to earlier work but we estimated that teriparatide has the highest probability of preventing wrist fractures (0.44).

The other objective of this analysis was to compare the results across two statistical methods. The first method was based on Bayesian ITC analysis in WinBUGS, and the second method was the results from classical Bucher analysis with ITC specific software. The estimates differed only by the second decimal place when the results were statically significant. However, there are key differences in the interpretation of the results. Based on the classical analysis we generated confidence intervals around the odds ratio and provided a test of association. In the Bayesian analysis, we generated a posterior distribution of the credible intervals for the true values of the odds ratio. In this analysis these values are similar, indicating that the priors used in the analysis were uninformative.

The analysis is limited in that the results are based on ITC comparisons. However, a recent review of the results of DTC and ITC analysis, described that out of 44 meta-analysis that were available with studies for meta-analysis by ITC and studies for meta-analysis by DTC, the DTC was similar in all but 3 cases to the ITC estimates for the same drugs and outcomes [9]. Of the 3 cases where the results were statistically different, 2 cases had the relative clinical benefit in the same direction while the third had differences in dosage regime in the studies. This result was also reported by Bucher in 1997 [10] where the ITC results were similar in direction as the DTC estimates. In addition, Bucher and Song both reported that the magnitude of the ITC results was larger between comparators than DTC comparisons, and the level of significance between comparators was less in ITC than DTC. In our ITC analysis, non-significant differences were estimated between drugs but the true effect between drugs may be even smaller.

The other assessment of strength of evidence in the indirect comparisons beyond looking at different classical versus Bayesian analysis was to look at heterogeneity within drugs and across drugs. The heterogeneity between comparators and heterogeneity within one comparator was small, with the exception of alendronate for wrist fractures. This heterogeneity was explained by two studies [28, 29] for wrist fractures. These studies did not contribute to heterogeneity in the meta-analysis of vertebral fractures and non-vertebral fractures. However, these two studies included the one study [28] that was the longest study with duration of 4 years with a low risk patients and the largest study for alendronate, while the other study [29] was small single centre study with duration of 4 years with low risk patients and the largest study.

The interpretation of the heterogeneity, although not a major feature in this analysis, is an important factor for ITC analysis. Increased heterogeneity can be caused by differences in inclusion criteria or study design such as length of follow-up. These are also important factors for consideration for analysis of DTC studies [20]. Three studies assessed the effect of patient characteristics to explain the level of heterogeneity in ITC analysis. In 2 studies [56, 57] no baseline variables were significant while in the other study [58] the year of the study and baseline risk affected heterogeneity. Both of these factors may have also affected heterogeneity if the studies were randomized with an active comparator. In our analysis, we may not have enough power to detect the impact of baseline characteristics because of a low number of studies for each drug [22]. In addition, because of the high heterogeneity in the estimates of odds ratios for wrist fractures, the evidence for wrist fractures should be considered weak.

ITC is becoming a useful tool in the absence of DTC comparisons and increasing transparency of ITC analysis builds confidence for the evidence. In a review of 88 ITC analyses, many of the studies could have increased the believability of their results [9] but the missed elements would also concern DTC analysis. These include incomplete searches or not assessing heterogeneity within a comparator. In 40/88 analysis there was no specific searches for active comparison studies to allow the comparison to the ITC evidence. For osteoporosis, this search was conducted and we found no published meta-analysis of DTC evidence. In the future stronger evidence may come from head-to-head studies but this is unlikely, because based on this analysis differences between comparators are not significant and studies would require very large sample sizes. Alternatively, the treatment analysis could come for pooling patient level data to compare the effects directly but this is unlikely due to propriety, and this analysis would diminish the benefits of randomization.

Conclusion

In light of the lack of DTC evidence, the ITC analysis of RPCTs may be the strongest evidence that will be available that answers the important clinical question of determining the most efficacious treatment for preventing fractures. In this analysis, teriparatide, zoledronic acid and denosumab have the highest probabilities of being most efficacious for non-vertebral and vertebral fractures, and having the greatest effect sizes. The estimates from indirect comparisons were robust to differences in methodology.

References

  1. Osteoporosis Facts and Statistics. [http://www.osteoporosis.ca/index.php/ci_id/8867/la_id/1.htm]

  2. Wiktorowicz ME, Goeree R, Papaioannou A, Adachi JD, Papadimitropoulos E: Economic implications of hip fracture: health service use, institutional care and cost in Canada. Osteoporos Int. 2001, 12: 271-278. 10.1007/s001980170116.

    Article  CAS  PubMed  Google Scholar 

  3. Cauley JA, Hochberg MC, Lui LY, Palermo L, Ensrud KE, Hillier TA, Nevitt MC, Cummings SR: Long-term risk of incident vertebral fractures. JAMA. 2007, 298: 2761-2767. 10.1001/jama.298.23.2761.

    Article  CAS  PubMed  Google Scholar 

  4. Bilezikian JP: Efficacy of Bisphosphonates in Reducing Fracture Risk in Postmenopausal Osteoporosis. American Journal of Medicine 122(2 SUPPL)()(pp S14-S21), 2009 Date of Publication: 2009. 2009, S14-S21.

    Google Scholar 

  5. Boers M: EMEA guidelines for trials in osteoporosis: design implications. Neth J Med. 2002, 60: 310-314.

    CAS  PubMed  Google Scholar 

  6. Stein CM, Ray WA: The ethics of placebo in studies with fracture end points in osteoporosis. N Engl J Med. 2010, 363: 1367-1370. 10.1056/NEJMsb1006120.

    Article  CAS  PubMed  Google Scholar 

  7. Rosen CJ, Khosla S: Placebo-controlled trials in osteoporosis--proceeding with caution. N Engl J Med. 2010, 363: 1365-1367. 10.1056/NEJMsb1002227.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council, Social Sciences and Humanities Research Council of Canada: Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. 2010

    Google Scholar 

  9. Song F, Altman DG, Glenny AM, Deeks JJ: Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003, 326: 472-10.1136/bmj.326.7387.472.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Bucher HC, Guyatt GH, Griffith LE, Walter SD: The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997, 50: 683-691. 10.1016/S0895-4356(97)00049-8.

    Article  CAS  PubMed  Google Scholar 

  11. Eddy DM, Hasselblad V, Shachter R: A Bayesian method for synthesizing evidence. The Confidence Profile Method. Int J Technol Assess Health Care. 1990, 6: 31-55. 10.1017/S0266462300008928.

    Article  CAS  PubMed  Google Scholar 

  12. Jansen JP, Bergman GJ, Huels J, Olson M: Prevention of vertebral fractures in osteoporosis: mixed treatment comparison of bisphosphonate therapies. Curr Med Res Opin. 2009, 25: 1861-1868. 10.1185/03007990903035281.

    Article  CAS  PubMed  Google Scholar 

  13. Jansen JP, Bergman GJ, Huels J, Olson M: The efficacy of bisphosphonates in the prevention of vertebral, hip, and nonvertebral-nonhip fractures in osteoporosis: a network meta-analysis. Semin Arthritis Rheum. 2011, 40: 275-284. 10.1016/j.semarthrit.2010.06.001.

    Article  CAS  PubMed  Google Scholar 

  14. Reference Manager, Version 11 for Windows. Thomson ResearchSoft. Carlsbad, CA, USA. 2009, Carlsbad, CA, USA, Thomson ResearchSoft

  15. The Bayesian inference Using Gibbs Sampling(BUGS) Project: WinBUGS. [http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml]

  16. Sung L, Hayden J, Greenberg ML, Koren G, Feldman BM, Tomlinson GA: Seven items were identified for inclusion when reporting a Bayesian analysis of a clinical study. J Clin Epidemiol. 2005, 58: 261-268. 10.1016/j.jclinepi.2004.08.010.

    Article  PubMed  Google Scholar 

  17. Caldwell DM, Ades AE, Higgins JP: Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005, 331: 897-900. 10.1136/bmj.331.7521.897.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Edwards SJ, Clarke MJ, Wordsworth S, Borrill J: Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract. 2009, 63: 841-854. 10.1111/j.1742-1241.2009.02072.x.

    Article  CAS  PubMed  Google Scholar 

  19. Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, Bradburn M, Eastwood AJ: Indirect comparisons of competing interventions. Health Technol Assess. 2005, 9: 1-iv.

    Article  CAS  PubMed  Google Scholar 

  20. Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004, 23: 3105-3124. 10.1002/sim.1875.

    Article  CAS  PubMed  Google Scholar 

  21. Review Manager (RevMan) [Computer program] Version 5.0. 2008, Copenhagen, The Nordic Cochrane Centre, The Cochrane Collaboration

  22. Higgins JP, Green S, editors: Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2. 2009, The Cochrane Collaboration

    Google Scholar 

  23. Wells G, Sultan S, Chen L, Khan M, Coyle D, Wells GA, Sultan SA, Chen L, Khan M, Coyle D: Indirect treatment comparison [computer program]. Version 1.0. 2009, Ottawa, Canadian Agency for Drugs and Technologies in Health

    Google Scholar 

  24. The Canadian Agency for Drugs and Technologies in Health (CADTH). [http://www.cadth.ca/index.php/en/home]

  25. Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009, 151: 264-9. W64

    Article  PubMed  Google Scholar 

  26. Ascott-Evans BH, Guanabens N, Kivinen S, Stuckey BG, Magaril CH, Vandormael K, Stych B, Melton ME: Alendronate prevents loss of bone density associated with discontinuation of hormone replacement therapy: a randomized controlled trial. Arch Intern Med. 2003, 163: 789-794. 10.1001/archinte.163.7.789.

    Article  CAS  PubMed  Google Scholar 

  27. Black DM, Cummings SR, Karpf DB, Cauley JA, Thompson DE, Nevitt MC, Bauer DC, Genant HK, Haskell WL, Marcus R: Randomised trial of effect of alendronate on risk of fracture in women with existing vertebral fractures. Fracture Intervention Trial Research Group. Lancet. 1996, 348: 1535-1541. 10.1016/S0140-6736(96)07088-2.

    Article  CAS  PubMed  Google Scholar 

  28. Cummings SR, Black DM, Thompson DE, Applegate WB, Barrett-Connor E, Musliner TA, Palermo L, Prineas R, Rubin SM, Scott JC: Effect of alendronate on risk of fracture in women with low bone density but without vertebral fractures: results from the Fracture Intervention Trial. JAMA. 1998, 280: 2077-2082. 10.1001/jama.280.24.2077.

    Article  CAS  PubMed  Google Scholar 

  29. Greenspan SL, Parker RA, Ferguson L, Rosen HN, Maitland-Ramsey L, Karpf DB: Early changes in biochemical markers of bone turnover predict the long-term response to alendronate therapy in representative elderly women: a randomized clinical trial. J Bone Miner Res. 1998, 13: 1431-1438. 10.1359/jbmr.1998.13.9.1431.

    Article  CAS  PubMed  Google Scholar 

  30. Liberman UA, Weiss SR, Broll J, Minne HW, Quan H, Bell NH, Rodriguez-Portales J, Downs RW, Dequeker J, Favus M: Effect of oral alendronate on bone mineral density and the incidence of fractures in postmenopausal osteoporosis. The Alendronate Phase III Osteoporosis Treatment Study Group. N Engl J Med. 1995, 333: 1437-1443. 10.1056/NEJM199511303332201.

    Article  CAS  PubMed  Google Scholar 

  31. Pols HA, Felsenberg D, Hanley DA, Stepan J, Munoz-Torres M, Wilkin TJ, Qin-sheng G, Galich AM, Vandormael K, Yates AJ: Multinational, placebo-controlled, randomized trial of the effects of alendronate on bone density and fracture risk in postmenopausal women with low bone mass: results of the FOSIT study. Fosamax International Trial Study Group. Osteoporos Int. 1999, 9: 461-468. 10.1007/PL00004171.

    Article  CAS  PubMed  Google Scholar 

  32. Cummings SR, San MJ, McClung MR, Siris ES, Eastell R, Reid IR, Delmas P, Zoog HB, Austin M, Wang A: Denosumab for prevention of fractures in postmenopausal women with osteoporosis. N Engl J Med. 2009, 361: 756-765. 10.1056/NEJMoa0809493.

    Article  CAS  PubMed  Google Scholar 

  33. Lyritis GP, Tsakalakos N, Paspati I, Skarantavos G, Galanos A, Androulakis C: The effect of a modified etidronate cyclical regimen on postmenopausal osteoporosis: a four-year study. Clin Rheumatol. 1997, 16: 354-360. 10.1007/BF02242451.

    Article  CAS  PubMed  Google Scholar 

  34. Meunier PJ, Confavreux E, Tupinon I, Hardouin C, Delmas PD, Balena R: Prevention of early postmenopausal bone loss with cyclical etidronate therapy (a double-blind, placebo-controlled study and 1-year follow-up). J Clin Endocrinol Metab. 1997, 82: 2784-2791. 10.1210/jc.82.9.2784.

    CAS  PubMed  Google Scholar 

  35. Montessori ML, Scheele WH, Netelenbos JC, Kerkhoff JF, Bakker K: The use of etidronate and calcium versus calcium alone in the treatment of postmenopausal osteopenia: results of three years of treatment. Osteoporos Int. 1997, 7: 52-58.

    Article  CAS  PubMed  Google Scholar 

  36. Pacifici R, McMurtry C, Vered I, Rupich R, Avioli LV: Coherence therapy does not prevent axial bone loss in osteoporotic women: a preliminary comparative study. J Clin Endocrinol Metab. 1988, 66: 747-753. 10.1210/jcem-66-4-747.

    Article  CAS  PubMed  Google Scholar 

  37. Pouilles JM, Tremollieres F, Roux C, Sebert JL, Alexandre C, Goldberg D, Treves R, Khalifa P, Duntze P, Horlait S: Effects of cyclical etidronate therapy on bone loss in early postmenopausal women who are not undergoing hormonal replacement therapy. Osteoporos Int. 1997, 7: 213-218.

    Article  CAS  PubMed  Google Scholar 

  38. Storm T, Thamsborg G, Steiniche T, Genant HK, Sorensen OH: Effect of intermittent cyclical etidronate therapy on bone mass and fracture rate in women with postmenopausal osteoporosis. N Engl J Med. 1990, 322: 1265-1271. 10.1056/NEJM199005033221803.

    Article  CAS  PubMed  Google Scholar 

  39. Watts NB, Harris ST, Genant HK, Wasnich RD, Miller PD, Jackson RD, Licata AA, Ross P, Woodson GC, Yanover MJ: Intermittent cyclical etidronate treatment of postmenopausal osteoporosis. N Engl J Med. 1990, 323: 73-79. 10.1056/NEJM199007123230201.

    Article  CAS  PubMed  Google Scholar 

  40. Wimalawansa SJ: A four-year randomized controlled trial of hormone replacement and bisphosphonate, alone or in combination, in women with postmenopausal osteoporosis. Am J Med. 1998, 104: 219-226. 10.1016/S0002-9343(98)00029-1.

    Article  CAS  PubMed  Google Scholar 

  41. Adami S, Felsenberg D, Christiansen C, Robinson J, Lorenc RS, Mahoney P, Coutant K, Schimmer RC, Delmas PD: Efficacy and safety of ibandronate given by intravenous injection once every 3 months. Bone. 2004, 34: 881-889. 10.1016/j.bone.2004.01.007.

    Article  CAS  PubMed  Google Scholar 

  42. Chesnut CH, Skag A, Christiansen C, Recker R, Stakkestad JA, Hoiseth A, Felsenberg D, Huss H, Gilbride J, Schimmer RC: Effects of oral ibandronate administered daily or intermittently on fracture risk in postmenopausal osteoporosis. J Bone Miner Res. 2004, 19: 1241-1249. 10.1359/JBMR.040325.

    Article  CAS  PubMed  Google Scholar 

  43. Ravn P, Neugebauer G, Christiansen C: Association between pharmacokinetics of oral ibandronate and clinical response in bone mass and bone turnover in women with postmenopausal osteoporosis. Bone. 2002, 30: 320-324. 10.1016/S8756-3282(01)00665-2.

    Article  CAS  PubMed  Google Scholar 

  44. Recker R, Stakkestad JA, Chesnut CH, Christiansen C, Skag A, Hoiseth A, Ettinger M, Mahoney P, Schimmer RC, Delmas PD: Insufficiently dosed intravenous ibandronate injections are associated with suboptimal antifracture efficacy in postmenopausal osteoporosis. Bone. 2004, 34: 890-899. 10.1016/j.bone.2004.01.008.

    Article  CAS  PubMed  Google Scholar 

  45. Ettinger B, Black DM, Mitlak BH, Knickerbocker RK, Nickelsen T, Genant HK, Christiansen C, Delmas PD, Zanchetta JR, Stakkestad J: Reduction of vertebral fracture risk in postmenopausal women with osteoporosis treated with raloxifene: results from a 3-year randomized clinical trial. Multiple Outcomes of Raloxifene Evaluation (MORE) Investigators. JAMA. 1999, 282: 637-645. 10.1001/jama.282.7.637.

    Article  CAS  PubMed  Google Scholar 

  46. Fogelman I, Ribot C, Smith R, Ethgen D, Sod E, Reginster JY: Risedronate reverses bone loss in postmenopausal women with low bone mass: results from a multinational, double-blind, placebo-controlled trial. BMD-MN Study Group. J Clin Endocrinol Metab. 2000, 85: 1895-1900. 10.1210/jc.85.5.1895.

    CAS  PubMed  Google Scholar 

  47. Harris ST, Watts NB, Genant HK, McKeever CD, Hangartner T, Keller M, Chesnut CH, Brown J, Eriksen EF, Hoseyni MS: Effects of risedronate treatment on vertebral and nonvertebral fractures in women with postmenopausal osteoporosis: a randomized controlled trial. Vertebral Efficacy With Risedronate Therapy (VERT) Study Group. JAMA. 1999, 282: 1344-1352. 10.1001/jama.282.14.1344.

    Article  CAS  PubMed  Google Scholar 

  48. Hooper MJ, Ebeling PR, Roberts AP, Graham JJ, Nicholson GC, D'Emden M, Ernst TF, Wenderoth D: Risedronate prevents bone loss in early postmenopausal women: a prospective randomized, placebo-controlled trial. Climacteric. 2005, 8: 251-262. 10.1080/13697130500118126.

    Article  CAS  PubMed  Google Scholar 

  49. McClung MR, Geusens P, Miller PD, Zippel H, Bensen WG, Roux C, Adami S, Fogelman I, Diamond T, Eastell R: Effect of risedronate on the risk of hip fracture in elderly women. Hip Intervention Program Study Group. N Engl J Med. 2001, 344: 333-340. 10.1056/NEJM200102013440503.

    Article  CAS  PubMed  Google Scholar 

  50. Mortensen L, Charles P, Bekker PJ, Digennaro J, Johnston CC: Risedronate increases bone mass in an early postmenopausal population: two years of treatment plus one year of follow-up. J Clin Endocrinol Metab. 1998, 83: 396-402. 10.1210/jc.83.2.396.

    CAS  PubMed  Google Scholar 

  51. Reginster J, Minne HW, Sorensen OH, Hooper M, Roux C, Brandi ML, Lund B, Ethgen D, Pack S, Roumagnac I: Randomized trial of the effects of risedronate on vertebral fractures in women with established postmenopausal osteoporosis. Vertebral Efficacy with Risedronate Therapy (VERT) Study Group. Osteoporos Int. 2000, 11: 83-91. 10.1007/s001980050010.

    Article  CAS  PubMed  Google Scholar 

  52. Meunier PJ, Roux C, Seeman E, Ortolani S, Badurski JE, Spector TD, Cannata J, Balogh A, Lemmel EM, Pors-Nielsen S: The effects of strontium ranelate on the risk of vertebral fracture in women with postmenopausal osteoporosis. N Engl J Med. 2004, 350: 459-468. 10.1056/NEJMoa022436.

    Article  CAS  PubMed  Google Scholar 

  53. Reginster JY, Felsenberg D, Boonen S, ez-Perez A, Rizzoli R, Brandi ML, Spector TD, Brixen K, Goemaere S, Cormier C: Effects of long-term strontium ranelate treatment on the risk of nonvertebral and vertebral fractures in postmenopausal osteoporosis: Results of a five-year, randomized, placebo-controlled trial. Arthritis Rheum. 2008, 58: 1687-1695. 10.1002/art.23461.

    Article  CAS  PubMed  Google Scholar 

  54. Neer RM, Arnaud CD, Zanchetta JR, Prince R, Gaich GA, Reginster JY, Hodsman AB, Eriksen EF, Ish-Shalom S, Genant HK: Effect of parathyroid hormone (1-34) on fractures and bone mineral density in postmenopausal women with osteoporosis. N Engl J Med. 2001, 344: 1434-1441. 10.1056/NEJM200105103441904.

    Article  CAS  PubMed  Google Scholar 

  55. Black DM, Delmas PD, Eastell R, Reid IR, Boonen S, Cauley JA, Cosman F, Lakatos P, Leung PC, Man Z: Once-yearly zoledronic acid for treatment of postmenopausal osteoporosis. N Engl J Med. 2007, 356: 1809-1822. 10.1056/NEJMoa067312.

    Article  CAS  PubMed  Google Scholar 

  56. Boonen S, Lips P, Bouillon R, Bischoff-Ferrari HA, Vanderschueren D, Haentjens P: Need for additional calcium to reduce the risk of hip fracture with vitamin d supplementation: evidence from a comparative metaanalysis of randomized controlled trials. J Clin Endocrinol Metab. 2007, 92: 1415-1423. 10.1210/jc.2006-1404.

    Article  CAS  PubMed  Google Scholar 

  57. Nixon R, Bansback N, Brennan A: The efficacy of inhibiting tumour necrosis factor alpha and interleukin 1 in patients with rheumatoid arthritis: a meta-analysis and adjusted indirect comparisons. Rheumatology (Oxford). 2007, 46: 1140-1147. 10.1093/rheumatology/kem072.

    Article  CAS  Google Scholar 

  58. Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol. 2009, 62: 857-864. 10.1016/j.jclinepi.2008.10.001.

    Article  PubMed  Google Scholar 

Pre-publication history

Download references

Acknowledgements

This work was completed as a chapter in Robert Hopkins' PhD dissertation. No funding was obtained.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Robert B Hopkins.

Additional information

Competing interests

Alexandra Papaioannou is or has been a consultant or on a speaker's bureau for Amgen, Aventis Pharma, Eli Lilly, Merck Frosst Canada, Novartis, Procter & Gamble Pharmaceuticals, Servier and Wyeth-Ayerst; she has conducted clinical trials for Eli Lilly, Merck Frosst Canada, Novartis, Procter & Gamble Pharmaceuticals and Sanofi-Aventis; and she has received unrestricted grants from Amgen, Eli Lilly, Merck Frosst Canada, Procter & Gamble Pharmaceuticals and Sanofi-Aventis. Jonathan Adachi has been a consultant or on a speaker's bureau for Amgen, Astra Zeneca, Eli Lilly, GlaxoSmithKline, Merck Frosst Canada, Novartis, Nycomed, Pfizer, Procter & Gamble Pharmaceuticals, Roche, Sanofi-Aventis, Servier, Wyeth-Ayerst and Bristol-Myers Squibb; he has conducted clinical trials for Amgen, Eli Lilly, GlaxoSmithKline, Merck Frosst Canada, Novartis, Pfizer, Procter & Gamble Pharmaceuticals, Sanofi-Aventis, Roche, Wyeth and Bristol-Myers Squibb. Lehana Thabane provides biostatistics consultation to GlaxoSmithKline for design and methodological issues. Eleanor Pullenayegum, Ron Goeree, Feng Xie and Robert Hopkins have no competing interests.

Authors' contributions

RH carried out the literature review, data abstraction, statistical analysis and drafting of the manuscript. FX, EP, and LT provided guidance on the statistical analysis methodology. JD and AP provided guidance for the clinical interpretation of manuscript. All authors read, provided comments and approved the final manuscript.

Electronic supplementary material

12891_2011_1248_MOESM1_ESM.DOCX

Additional file 1:Literature Search Strategy. Strategy to find indirect treatment comparison controlled-trials for osteoporosis medications in Medline. (DOCX 14 KB)

12891_2011_1248_MOESM2_ESM.DOC

Additional file 2:WinBUGS ITC code for vertebral fractures. WinBUGS software code and data to conduct ITC analysis for vertebral fractures. (DOC 25 KB)

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Hopkins, R.B., Goeree, R., Pullenayegum, E. et al. The relative efficacy of nine osteoporosis medications for reducing the rate of fractures in post-menopausal women. BMC Musculoskelet Disord 12, 209 (2011). https://doi.org/10.1186/1471-2474-12-209

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2474-12-209

Keywords