Skip to main content
  • Research article
  • Open access
  • Published:

Evaluating the cost effectiveness of donepezil in the treatment of Alzheimer's disease in Germany using discrete event simulation

Abstract

Background

Previous cost-effectiveness studies of cholinesterase inhibitors have modeled Alzheimer's disease (AD) progression and treatment effects through single or global severity measures, or progression to "Full Time Care". This analysis evaluates the cost-effectiveness of donepezil versus memantine or no treatment in Germany by considering correlated changes in cognition, behavior and function.

Methods

Rates of change were modeled using trial and registry-based patient level data. A discrete event simulation projected outcomes for three identical patient groups: donepezil 10 mg, memantine 20 mg and no therapy. Patient mix, mortality and costs were developed using Germany-specific sources.

Results

Treatment of patients with mild to moderately severe AD with donepezil compared to no treatment was associated with 0.13 QALYs gained per patient, and 0.01 QALYs gained per caregiver and resulted in average savings of €7,007 and €9,893 per patient from the healthcare system and societal perspectives, respectively. In patients with moderate to moderately-severe AD, donepezil compared to memantine resulted in QALY gains averaging 0.01 per patient, and savings averaging €1,960 and €2,825 from the healthcare system and societal perspective, respectively.

In probabilistic sensitivity analyses, donepezil dominated no treatment in most replications and memantine in over 70% of the replications. Donepezil leads to savings in 95% of replications versus memantine.

Conclusions

Donepezil is highly cost-effective in patients with AD in Germany, leading to improvements in health outcomes and substantial savings compared to no treatment. This holds across a variety of sensitivity analyses.

Peer Review reports

Background

Alzheimer's disease (AD) is an incurable neurodegenerative disease characterized by cognitive decline, impairment of daily activities and neuropsychiatric symptoms. AD patients lose the ability to perform higher-level daily activities and decrease to being no longer able to perform basic daily necessities such as eating or grooming [1]. Mood swings, apathy, psychosis or agitation are behavioral symptoms commonly observed with AD patients. With increasing severity of the disease dealing with the patients' symptoms can become an increasing burden to caregivers.

A recent study reported the prevalence of dementia in Germany a under 1% of 60-64 year olds and significantly increasing to roughly 20% for those over the 85-89 year olds, up to three quarters of whom had AD [2]. Recent estimates for Germany placed the number of individuals with moderate of severe dementia at just over 1 million, with a projected increase of AD patients in Germany of over 2 million by 2050 [2]. Against the background of an aging population, the German Federal Government has recently intensified its focus on dementia. In 2008, the "Leuchtturmprojekt Demenz" with a budget of 13 million Euro was initiated to improve the evidence-based medical and care service provision for dementia patients [3].

Despite the fact that the benefit of cholinesterase inhibitors have been established by numerous studies [4, 5] and that they are a recommended treatment for AD [6], physicians may hesitate to treat patients accordingly given drug acquisition cost considerations [7]. Research on the cost effectiveness of cholinesterase inhibitors is therefore important to provide decision makers with the best possible economic evidence to determine whether concerns over drug acquisition costs are legitimate.

Over the last decade, numerous studies have measured the cost-effectiveness of treatments for AD [8, 9], most evaluating the cost-effectiveness of cholinesterase inhibitors. Eight studies have investigated donepezil [10–17], with all but one indicating that donepezil was cost-effective. In Germany, a recent study showed that donepezil was also cost-effective in the German setting, with a base case estimated cost-effectiveness ratio of €4,264 per CDR--Clinical Dementia Rating Scale gained [18].

Previous cost-effectiveness studies have modeled AD progression in terms of cognitive function alone, functional status alone, a single global severity measure, or progression to the need for "Full Time Care". Our study uses an alternative modeling approach to estimate disease progression in terms of correlated changes in cognition, behavior and function. The model was initially constructed for analyses set in the UK [19].

Methods

The discrete event simulation developed for the evaluation of donepezil's cost-effectiveness in the UK [19, 20] was adapted for Germany. The model calculates outcomes from the perspective of both the statutory health insurance and care insurance (Gesetzliche Krankenversicherung/Soziale Pflegeversicherung, GKV/SPV), and from the societal perspective. The GKV/SPV perspective encompasses direct medical costs borne by statutory healthcare insurance including drug costs, costs for monitoring and service provision as well as patient care costs borne by long-term care insurance. The societal perspective comprises both direct and indirect costs, the latter including costs of caregiver time. A discount rate of 3.0% was used for both costs and benefits [21]. In the base case analyses, the time horizon is 10 years in order to capture all potential benefits over the course of the disease.

Model overview

To allow for individual level modelling, discrete event simulation was used as the modeling technique, capturing heterogeneity in disease progression and other outcomes, as well as tracking correlated changes on multiple domains on continuous rather than aggregated discrete scales. The approach also allows for persistence with treatment to be captured, factoring in time-dependence and the impact of treatment discontinuation on both costs and disease progression in a realistic manner [22–24].

Previous economic evaluations in AD have measured outcomes using highly aggregated health states, and thus were not able to capture the benefits associated with treatment in adequate detail. Furthermore, they have often modeled the disease either based on single domains (e.g., MMSE--Mini-Mental State Examination) or global domains (e.g., CDR--Clinical Dementia Rating Scale), losing the ability to capture the effects of treatment on the full spectrum of AD symptoms. This was driven in part by limitations in data accessible to analysts and the difficulty of tracking progression on multiple measures using traditional Markov model techniques. In addition, most of these models were designed as cohort models with no ability to account for individual characteristics in predicting outcomes, variability in outcomes over the course of the disease or other relevant factors that might influence important determinants of long term outcomes, such as persistence with treatment. The shortcomings of modeling studies in AD have been extensively debated in the literature [8, 9, 25, 26].

Our study adopts an alternative approach in an attempt to overcome some of these limitations. First, it addresses limitations of existing models that focus on a single measure of disease severity alone to model the evolution of AD, by modeling the disease using measures of cognition, behaviour and function. Second, it is an individual simulation that is not encumbered by the limitations imposed by Markovian structures such as an inability to account for individual characteristics by relying on cohort mean values or the use of aggregate health states (e.g., mild, moderate, severe; full-time care, pre-full time care) instead of continuous measurement of disease progression. Finally, as the model employed in this study is an individual patient simulation, it allows for consideration of variation in patient characteristics and disease progression, allows for simulation of persistence with treatment, implementation of clinical stopping rules, and time varying treatment effects and is therefore able to capture disease progression and treatment effects with greater accuracy. The model has been built using ARENA (Version 11) software.

Figure 1 provides an overview of the model flow. First, simulated patients are created and individual, unique attributes are assigned. Each patient is then copied twice and the three identical patients are assigned to either no treatment, donepezil 10 mg or memantine 20 mg. Patients are then followed over the course of the simulation with their characteristics updated over time. The simulation measures disease severity based on cognition (using the MMSE), behaviour (using the Neuropsychiatric Inventory, NPI), activities of daily living (ADLs) and instrumental activities of daily living (IADL). In order to preserve the correlation amongst these measures of severity, integrated equations were developed which sequentially predict changes in MMSE, followed by NPI, ADLs, and IADLs. Changes in cognition are first predicted for a simulated individual, which then influence changes in NPI, ADLs, and IADLs. Furthermore, changes in ADL scores are used as part of the prediction of changes in IADL scores. Based on a given patient's characteristics at any point in time, including treatment status and current disease severity, costs, health utilities and caregiver outcomes are calculated and accumulated over the appropriate time period. The model also reports time spent with non-severe symptoms as defined by MMSE, NPI, ADL or IADL scores. MMSE scores below 10 were assumed to represent severe disease according to currently accepted definitions for severe cognitive impairment. For NPI, a cluster analysis of psychiatric symptoms using the NPI of 122 Alzheimer's disease patients in the US [27] was used as the basis for assigning a threshold of 28 for the NPI as representing highly symptomatic behavioral disturbances. IADL and ADL thresholds were arbitrarily set to their mid-point values of 50.

Figure 1
figure 1

Simplified representation of the Alzheimer's disease simulation flow.

In the simulation, patients can discontinue treatment either based on pre-defined stopping rules, or for other unrelated reasons. As there are currently no defined stopping rules in Germany (e.g., stopping treatment when MMSE scores fall below 10), this option has only been explored in the sensitivity analyses. Mortality is also modeled, although given that neither cholinesterase inhibitors nor memantine have been associated with improvements in survival, time of death is assigned to each individual prior to treatment assignment, thereby ensuring that survival is identical in all groups.

Data sources

Population

An individual patient data set constructed using baseline information from donepezil clinical trials [28–30] is sampled from, to create simulated patients. Characteristics carried in the model include patient age, sex, use of psychiatric medications, MMSE, NPI, ADL- and IADL scores, as well as caregiver age and sex. The trials chosen to provide the sample patients were those that had data on as many target variables as possible, and taken together include all AD severity levels [19].

The age and sex distributions of AD patients in Germany [31] were used to assign sampling weights to individuals in the data set in order to ensure that the age and sex profile of the simulated population was consistent with that of the German AD patient population. Furthermore, analyses are specified according to subgroups of interest, so that sampling is restricted to the relevant population (e.g., patients with mild to moderate AD, as defined by baseline MMSE scores).

Disease progression and treatment effects

To improve on existing economic evaluations by including the effects of disease on behavior and function, data were analyzed from the CERAD (Consortium to Establish A Registry for Alzheimer's Disease) registry [32], and seven donepezil clinical trials in AD [28–30, 33–36], including data from open label extensions of two of the studies [37, 38]. Trial data included 2,700 patients from the US, Canada, UK, France, and five Nordic countries, with up to 52 weeks of follow-up. The inclusion of trials in the current analyses was based on several criteria. Most importantly, to develop equations related to disease progression and treatment effects, access to patient level data was required. In selecting trials to be included in the patient level analyses, studies had to be Phase III or later, had to include a measure of baseline MMSE, and had to include at least one of the effectiveness outcomes included in the model. Studies conducted in special populations (e.g., women only, Apo-E subtypes, nursing home residents only); or of open label design only were excluded, as were dose finding studies.

While MMSE data over time were available from trial data the patterns of change observed in CERAD were more consistent with previous findings on progression of AD in untreated patients [39–41] and were therefore used to model the natural history of cognitive changes in the absence of treatment. A piecewise linear regression model was fitted to the annual rate of change in MMSE. This approach allows for a different slope in different intervals of the MMSE scale to reflect differences in the rate of change at different disease stages. Variables considered included patient age at baseline, sex, disease duration, baseline MMSE, and rate of decline in the first year (labeled PrevRate). The following MMSE equation was derived, retaining variables significant at the 0.05 level:

R a t e o f C h a n g e = - 5 . 4663 - 0 . 4299 P M 1 - 0 . 0042 P M 2 + 0 . 1415 P M 3 - 0 . 0791 P r e v R a t e + 0 . 0747 A g e + δ i

PM represents patients' previous MMSE measurement, partitioned over the MMSE scale. PM1--PM3 are calculated as: PM1 = min(PrevMMSE, 9), PM2 = max[0, min(PrevMMSE-9, 9)], and PM3 = max[0, min(PrevMMSE-18, 12)]. δi represents a random intercept parameter, allowing the pattern of decline to vary between patients. The MMSE scale itself ranges from 0 to 30.

To apply a treatment effect for donepezil, a similar model was fitted to the donepezil trial data to identify differences in rate of cognitive decline [19]. In the first 20 weeks of treatment, the estimated coefficients for treatment effect on annual rate of change was 6.16 and 2.47 over weeks 20 to 52. After 1 year, further treatment was assumed to simply maintain previous gains (i.e. the treatment term in the rate of change equation is set to 0). Note that these coefficients are not the sole determinant of treatment effect size given that rate of change in MMSE is also influenced by individuals' previous rate of decline and overall disease severity. Furthermore, the coefficients for treatment effect influence the annual rate of change, and are applied differentially depending on time on treatment and how long patients remain on treatment. In order to test the validity of the effect size calculations, simulated effect sizes at 6 months were compared to the observed effect sizes in the clinical trials [20], with the simulation resulting in an estimate of improvement of 1.92 points on the MMSE for donepezil versus no treatment, compared to the observed 1.88 point difference.

NPI was predicted based on the donepezil trials where NPI data were collected. It was modeled as change from NPI at baseline.

C h a n g e N P I = ( 5 . 74 - 0 . 64 D o n e p e z i l + 0 . 03 W e e k s - 0 . 59 N P I b a s e - 0 . 001 N P I ∙ W e e k s + 0 . 24 N P I r e c e n t - 1 . 74 W h i t e - 3 . 82 B l a c k + 2 . 34 P s y M e d + 0 . 12 M M S E b a s e - 0 . 22 M M S E r e c e n t + δ i ) ∙ 1 . 44

Donepezil represents the treatment effect of donepezil, Weeks stands for weeks of follow-up in the simulation, NPI base is the patient's baseline NPI, NPI recent is the patient's last NPI. White and Black are dummy variables for race (All Other Races was the reference), PsyMed is a dummy variable for patients treated with psychiatric medications at baseline, MMSE base represents the patient's MMSE at baseline, and MMSE recent represents the patient's previous MMSE. δ i represents a random intercept parameter, which allows the pattern of decline to vary between patients. Patient age and sex, as well as rate of MMSE decline were also tested as predictors, but failed to reach a significance level of 0.05. The equation for NPI was derived based on a normalized scale of 0 to 100, and is therefore multiplied by 1.44 to rescale it to the standard 0 to 144 range for the NPI.

As changes in NPI are influenced by patients' baseline and most current MMSE, the treatment effect of donepezil is realized both through the treatment coefficient and its influence on MMSE over time [19]. The treatment effect coefficient, therefore, only partially accounts for the impact of treatment on NPI changes, as patients on treatment will generally have better MMSE scores. For example, a patient who has a 1 point treatment effect on MMSE, will experience a total treatment effect on NPI of 1.44*[(-0.22 × 1) -0.64], or -1.27.

For the scales that measure function (ADL and IADL) standardized scales ranging from 0 (best function) to 100 (worst function) were created based on the available clinical trial data.

As with NPI, ADL and IADL equations predict change from baseline:

R a t e o f C h a n g e A D L = 1 . 35 - 0 . 81 D o n e p e z i l + 0 . 06 W e e k s - 0 . 79 A D L b a s e + 0 . 71 A D L p r e v i o u s + 0 . 12 M M S E b a s e + 0 . 09 A g e + 0 . 81 P s y M e d - 3 . 05 B l a c k - 0 . 49 M M S E r e c e n t + δ i
R a t e o f C h a n g e I A D L = 1 . 27 + 0 . 63 D o n e p e z i l + 0 . 17 W e e k s - 0 . 06 D o n e p e z i l ∙ W e e k s - 0 . 84 I A D L b a s e + 0 . 002 I A D L W e e k + 0 . 84 I A D L p r e v i o u s - 0 . 67 M a l e + 0 . 20 M M S E b a s e - 0 . 28 M M S E r e c e n t - 0 . 16 A D L b a s e + 0 . 18 A D L r e c e n t + δ i

Age stands for the patient's age at baseline in years. Potential predictors considered for inclusion (at 0.05 level) were treatment, time, baseline and most recent ADL/IADL, baseline and most recent MMSE, baseline and most recent NPI, age, sex, treatment, and use of anti-psychotic medications.

For ADL scores, donepezil's effect was modeled directly through the treatment effect and the terms for patients' most recent MMSE. For IADLs, treatment comes into play through the treatment term, as well as patients' most recent MMSE and ADL scores.

Additional technical details on the equations used in the simulation have been published elsewhere [19]. Standard errors and a validation that simulated results on treatment effect from the predictive equations and compare well to observed results on treatment effect are also available [20], although IADL treatment effect sizes were underestimated at 6 months. For NPI, the simulated treatment effect size was 1.75 points compared to the observed 1.68 for ADL, the simulated effect size was 2.55 points compared to an observed 2.59 and for IADLs, the simulated effect size was 1.69 points compared to an observed 3.79.

As head-to-head data were not available for memantine, the simulation predicts disease progression and treatment persistence for patients on memantine using the parameters for patients on donepezil, but modifies these parameters using the difference between 6-month placebo adjusted clinical trial results for memantine 20 mg and donepezil 10 mg in moderate to severe AD patients (Table 1). Data for memantine were extracted from a Cochrane meta-analysis [42].

Table 1 Changes in placebo-adjusted effectiveness outcomes for donepezil and memantine

Persistence

Patients can stop treatment in the simulation either by reaching the end of the user-specified treatment duration (10 years in base case), based on clinical stopping rules (e.g., MMSE falling below 10 as explored in the sensitivity analyses), or other non-specified reasons. The analyses are based on the assumption that patients who stop treatment lose all treatment benefits over the course of the subsequent 6 weeks [37].

Hazard ratios for premature treatment discontinuation are derived from the donepezil clinical trial data and applied to base discontinuation rates derived from actual practice data in the UK [43] (Table 2) as there are no equivalent data available for Germany. The hazard ratios are based on a Cox regression model in which MMSE and the rate of decline in MMSE were updated over time. Demographic variables were also tested as predictors of discontinuation but were not significant (0.05) and not retained.

Table 2 Baseline discontinuation rates for patients on donepezil and memantine

Mortality

German-specific survival data were obtained from the German Federal Statistical Office [44]. As there are no disease specific survival data for Germany available, gender-specific differences in survival for the UK and German populations aged 65 years and older were applied to survival times from the Medical Research Council's cognitive function and ageing study (MRC CFAS), and time to death functions derived based on patient age and gender at baseline [45]. Mortality was assumed to be unaffected by treatment.

Medical costs

Daily treatment costs of €4.20 for donepezil 10 mg and €3.83 for memantine 20 mg were derived from the Rote Liste®.[46]. Patients on active therapy were assumed to incur costs associated with biannual visits to their physician. Costs for outpatient service provision, such as GP or specialist visits, dementia tests, laboratory tests or imaging are based on the German tariff EBM 2008 [46].

Direct patient care costs per disease state are based on a 2007 publication on the cost effectiveness of donepezil in Germany [18, 47], and inflated to 2008 Euros, using the harmonized index of consumer prices for Germany [48]. It provides monthly costs by severity level which were interpolated to provide estimates for intermediate disease severity stages (Table 3). The same cost was applied regardless of location of care because health state costs included costs for both ambulatory care and nursing homes.

Table 3 Cost inputs

Caregiver time costs

Caregiver time was linked to disease severity parameters [20] based on an equation derived from two of the donepezil clinical trials where these data were available, and took the form of [29, 30]:

C a r e M i n u t e s P e r D a y = 76 . 41 + 1 . 8 A g e c g + 93 . 02 M a l e c g + 85 . 56 M a l e p a t i e n t - 6 . 47 M M S E + 0 . 58 N P I + 2 . 66 A D L + 2 . 61 I A D L + 20 . 55 P s y M e d

Age cg stands for the caregiver's age, Male cg is a dummy variable for the caregiver's sex, and Male patient for the patient's sex. Patient age and relation of patient to caregiver (spouse, child, or other) were other parameters tested but dropped for lack of significance with p > 0.10. The p-value for PsyMed was 0.25 but it was retained as it was a confounding variable (i.e., dropping it biased the values of the other coefficients).

In sensitivity analyses, an alternative assignment of caregiver time was used, with caregiver time calculated based solely on patient's MMSE score using estimates reported for Germany [47]. Caregiver time was valued at €5.21 per hour based on a published study [18, 47].

Location of care

Costs and time by location of care are accumulated based on the severity of disease patients experience over the course of the simulation. Similarly, time spent by patients in institutions is allotted as percentage of the time that the patient was alive. Institutionalization rate was calculated based on institutionalization rates of AD patients in Germany [50] which were reported as 42.9% for patients with MMSE scores below 20, and 0% for those with scores of 20 or higher. In order to produce a finer gradient, these results were fit to a simple linear regression which predicted the proportion of patients institutionalized as 64.35% - 2.86% × MMSE, with rates varying from 0% for those at the mildest stages of the disease to 50% for those with severe AD.

Health utilities

Patients' health utilities were estimated based on a previously published regression equation [51] which used the EQ-5D to derive health utilities for 272 AD patients in Nordic countries [19]. The NPI term in the published equation was based on the brief NPI, and was modified to reflect the full NPI range (0 to 144) used in the simulation. The final equation took the following form and is applied in the model by using patients' values (e.g., MMSE score) over the course of the simulation to calculate the appropriate QALYs to be assigned to that patient:

U t i l i t y = 0 . 408 + 0 . 010 M M S E - 0 . 004 N P I - 0 . 159 I n s t i t u t i o n a l i z e d + 0 . 051 C a r e g i v e r

MMSE represents the patient's current MMSE, NPI represents the patient's current NPI. Institutionalized is a dummy variable for whether the patient is institutionalized. Caregiver indicates whether the patient lives with their caregiver.

Caregiver utilities are assigned based on equations derived from the donepezil trial data where caregivers completed the SF-36 [20, 28–30]. Scores were transformed to health utilities [52] and a linear repeated measures model was used to develop the following equation:

C a r e g i v e r U t i l i t y = 0 . 90 - 0 . 003 A g e C G + 0 . 03 M a l e C G + 0 . 001 A g e p a t i e n t + 0 . 00 M M S E - 0 . 001 N P I - 0 . 001 A D L - 0 . 0004 I A D L - 0 . 01 P s y M e d

Patient sex and relation of patient to caregiver (spouse, child, or other) were other variables tested but dropped for lack of significance with p > 0.10. PsyMed and IADL had p-values of 0.20 but were retained as they were confounders.

Analyses

Base case analyses were run for patients with mild to moderate AD (26 ≥ MMSE ≥ 10) treated with donepezil 10 mg versus no treatment, and for patients with moderate AD (MMSE 10-19) on donepezil versus memantine 20 mg over a 10 year time horizon.

The following parameters were varied in the probabilistic sensitivity analyses: treatment effects on MMSE, NPI, ADL, and IADL, patient care costs, caregiver time regression parameters, patient and caregiver utility regression parameters, the proportion of patients living in the community by disease severity, and treatment discontinuation rates.

Standard errors were available for many parameters from the parameter source data, reflecting the study sampling error. Where standard errors were not available, 25% of the parameter mean was used to assign an assumed 95% confidence interval from which standard error estimates were derived. A normal distribution was assumed for parameters on continuous variables, while proportion parameters on discrete variables were assumed to be beta distributed.

Results

In patients with mild to moderately severe AD (26 ≥ MMSE ≥ 10), donepezil dominates no treatment from both the GKV/SPV perspective, with savings averaging €7,007 per patient (€7,323 undiscounted). From the societal perspective where savings increase to €9,893 per patient (€10,384 undiscounted) (Table 4), donepezil treatment is associated with an increase in QALYs averaging 0.13 per patient (0.14 undiscounted). For caregivers, donepezil treatment increases QALYs by 0.01 compared to caregivers of untreated patients (0.02 undiscounted). Donepezil also increases the amount of time patients spend with MMSE scores above 10 by an average of 24 weeks per patient, NPI scores below 28 by almost 6 weeks, and ADL/IADL scores below 50 by more than 7 and 3 weeks, respectively. In patients with moderate including moderately-severe AD (20 > MMSE ≥ 10), donepezil also dominates memantine, although savings are smaller, averaging €1,960 per patient (€2,097 undiscounted) from the GKV/SPV- and €2,825 per patient (€3,012 undiscounted) from the societal perspective. QALY gains are clearly smaller, with donepezil associated with an average QALY gain of 0.01 per patient versus memantine (0.01 undiscounted), and caregiver QALYs gained at less than < 0.001. The reduction in time patients spend institutionalized also falls by just over 10 days per patient. For patients with moderately severe AD, donepezil still dominates no treatment, although consistent with findings in the UK [20]. Cross reference Getsios 2010, per patients savings were lower at €8,043, as were both patient and caregiver QALYs gained, at 0.120 and 0.013, respectively. Memantine also led to lower overall costs and improved QALYs relative to no treatment, although both savings and QALYs gained, although savings were 35% lower compared to those with donepezil, and QALYs gained 13% lower.

Table 4 Base case results by disease severity for the 10 years following treatment initiation*

In one-way sensitivity analyses key parameters such as caregiver time, costs, utilities, institutionalization and treatment effects were varied. Regardless of the variation, donepezil remained dominant compared to both no therapy and memantine. Rates of treatment discontinuation and the duration of treatment had the strongest influence on the extent of savings and health benefits (Table 5).

Table 5 One-way sensitivity analyses

In probabilistic sensitivity analyses, donepezil dominates no treatment in almost all replications from both the health care payer and societal perspectives (Figure 2). Versus memantine, donepezil dominates in 70% of replications, and leads to savings in 95% of replications (Figure 3). For the analyses versus memantine, at a threshold of 10,000 Euro/QALY, donepezil was cost-effective from both perspectives in over 90% of replications.

Figure 2
figure 2

Cost-effectiveness scatter plot for patients with 26 ≥ MMSE ≥ 10: donepezil versus no treatment. Distribution of replications: 0.8% in upper right quadrant (donepezil leads to incremental costs and higher QALYs), 99.2% in lower right quadrant (donepezil dominant, leading to lower costs and higher QALYs).

Figure 3
figure 3

Cost-effectiveness scatter plot for patients with 20 > MMSE ≥ 10: donepezil versus memantine. Distribution of replications: 69.8% in lower right quadrant (donepezil dominant, leading to lower costs and higher QALYs), 25.2% in lower left quadrant (donepezil leads to lower costs and lower QALYs), 4.6% in upper left quadrant (donepezil dominated, leading to higher costs and lower QALYs), and 0.4% in the upper right quadrant (donepezil leads to higher costs and higher QALYs).

Discussion

The discrete event simulation developed for donepezil provides a flexible framework for the assessment of treating AD patients with donepezil. By integrating patients' individual characteristics, heterogeneity in the population, disease progression and outcomes can be captured. Disease progression was modeled not solely relying on 1 year RCT data, but also using longer term CERAD registry data, which allows for a more realistic representation of the disease course. The model allows for analysis of population subgroups, with different settings for time horizon, treatment duration, discontinuation rules, and treatment effects. Cost, utility and caregiver inputs can be specified for different severity ranges and locations of care, or can be specified using predictive equations, making adaptation of the model and incorporation of new data easier.

The analyses for Germany indicate that donepezil is clearly cost-effective in the treatment of patients with mild to moderately-severe AD. In the base case and all one-way sensitivity analyses, donepezil dominated no treatment and memantine in all scenarios evaluated. Results of the probabilistic sensitivity analyses were also highly favorable with donepezil dominating no treatment in virtually all replications, and leading to savings in the comparison with memantine in 95% of replications. With the availability of generic cholinesterase inhibitors, cost savings should be even greater, although the contribution of the cost of treatment with donepezil to overall costs is modest, representing less than 2.5% of total costs in patients with mild to moderately-severe AD.

Our model differs from most previous economic models in AD in that we model disease progression over several domains using continuous scales, rather than using a single domain and/or limiting outcomes to a small number of discrete health states. Although more favorable overall with donepezil predicted to dominate no treatment, the results of this study are much more in line with previous findings for Germany [19], where a base-case ICER of €4,264 per QALY for donepezil versus placebo was estimated. That study also indicated that starting treatment early leads to cost reductions and therefore improved cost-effectiveness. The more favorable predictions from our simulation are chiefly a result of the greater sensitivity of our model in capturing changes in cognition, and in the case of indirect costs, the consideration of not only cognition, but also patient function and behavioral symptoms.

The current simulation is not without limitations. For example, the longest-duration of head to head clinical trial data available was for 1 year versus placebo [29]. With the assumption that continued treatment after 1 year serves as a maintenance function only with no further slowing of the rate of disease progression, we have adopted a conservative approach consistent with most other modeling studies in this area. Furthermore, we assume that all benefits are lost within 6 weeks if treatment is discontinued. Comparisons with memantine are subject to even greater uncertainty, as no head-to-head trial data are available, requiring indirect comparisons based on pooled trial results. Clearly, incorporation of head to head clinical trial data versus memantine would strengthen comparisons and yield more robust results. In addition, the current analyses evaluate memantine monotherapy versus donepezil monotherapy. Although memantine also be used as an add-on treatment to cholinesterase inhibitors, this was not evaluated, as the focus of these analyses was on the cost-effectiveness of donepezil, and not memantine.

The source population for the trial data was not German, though it was weighted for the age and sex distribution of German AD patients when defining the simulated population. Other limitations of the data revolve around assigning costs and utilities associated with different degrees of disease severity. A recent review paper on health utilities [53] used in economic evaluations, noted the limited amount of data on severity specific health utilities in populations with AD, and the poor correlation between patient-based utilities, and those derived by caregiver proxy.

The cost data for Germany are based entirely on MMSE ranges (i.e., they do not consider behavior or function). Furthermore published German cost data for AD at the time of the analysis were scant, with only one publication providing suitable information [47]. A number of studies on dementia and AD costs were published in 2011 [52, 54–56]. Those studies that did report costs by severity of disease [52, 54, 55], all found that costs increase markedly with increased disease severity, consistent with the data used in the current model and therefore would not alter our conclusions. The one study that examined costs in patients with AD, for example [54] found that the annual costs of the disease average €13,080 per patient per year (€ 2009), but approached €25,000 per patient for those at the most severe stages of the disease, compared to well under €7,000 per patient for those at the most mild stages. Finally, although wide variation exists in the valuation of informal care, with donepezil dominant over no treatment and memantine, even when these costs are considered, the method of assigning costs to caregiver time would only influence the extent of savings associated with donepezil. Of note, however, the hourly costs assigned to caregiver time are substantially lower than those used in recent German costing studies in dementia [52, 54, 55].

Conclusions

These analyses indicate that donepezil is highly cost-effective in the treatment of mild to moderate AD in Germany, and is likely associated with significant cost savings when compared to untreated patients. While benefits over memantine are modest, the base case and sensitivity analyses results suggest a high likelihood that donepezil would lead to cost savings if used in place of memantine.

Funding

This research was funded by Eisai GmbH, Germany. Eisai GmbH was informed throughout the process of data collection and analyses, and contributed to the research through critical review of results and manuscript.

Abbreviations

CDR:

Clinical Dementia Rating Scale

MMSE:

Mini-Mental State Examination

AD:

Alzheimer's disease

NPI:

Neuropsychiatric Inventory

ADLs:

Activities of daily living

IADL:

Instrumental activities of daily living

CERAD:

Consortium to Establish a Registry for Alzheimer's Disease

MRC CFAS:

Medical Research Council's cognitive function and ageing study.

References

  1. Cummings JL: Alzheimer's disease. N Engl J Med. 2004, 351 (1): 56-67. 10.1056/NEJMra040223.

    Article  CAS  PubMed  Google Scholar 

  2. Ziegler U: DG: prevalence and incidence of dementia in Germany -- a study based on data from the public sickness funds in 2002. Gesundheitswenen. 2009, 71: 281-290. 10.1055/s-0028-1119384.

    Article  CAS  Google Scholar 

  3. BMG. [http://www.bmg.bund.de/cln_110/nn_1168278/SharedDocs/Standardartikel/DE/AZ/D/Glossarbegriff-Demenz.html]

  4. Birks J: Cholinesterase inhibitors for Alzheimer's disease. Cochrane Database Syst Rev. 2006, CD005593-1

  5. Hansen RA, Gartlehner G, Webb AP, Morgan LC, Moore CG, Jonas DE: Efficacy and safety of donepezil, galantamine, and rivastigmine for the treatment of Alzheimer's disease: a systematic review and meta-analysis. Clin Interv Aging. 2008, 3 (2): 211-225.

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Waldemar G, Dubois B, Emre M, Georges J, McKeith IG, Rossor M, Scheltens P, Tariska P, Winblad B: Recommendations for the diagnosis and management of Alzheimer's disease and other disorders associated with dementia: EFNS guideline. Eur J Neurol. 2007, 14 (1): e1-e26.

    Article  CAS  PubMed  Google Scholar 

  7. Melchinger H: Demenzerkrankungen. Chronische Versorgungsdefizite. Dtsch Arztebl. 2007, 104 (47): A3236-A3237.

    Google Scholar 

  8. Green C: Modelling disease progression in Alzheimer's disease: a review of modelling methods used for cost-effectiveness analysis. Pharmacoeconomics. 2007, 25 (9): 735-750. 10.2165/00019053-200725090-00003.

    Article  PubMed  Google Scholar 

  9. Cohen JT, Neumann PJ: Decision analytic models for Alzheimer's disease: state of the art and future directions. Alzheimers Dement. 2008, 4 (3): 212-222. 10.1016/j.jalz.2008.02.003.

    Article  PubMed  Google Scholar 

  10. Ikeda S, Yamada Y, Ikegami N: Economic evaluation of donepezil treatment for Alzheimer's disease in Japan. Dement Geriatr Cogn Disord. 2002, 13 (1): 33-39. 10.1159/000048631.

    Article  CAS  PubMed  Google Scholar 

  11. Fagnani F, Lafuma A, Pechevis M, Rigaud AS, Traykov L, Seux ML, Forette F: Donepezil for the treatment of mild to moderate Alzheimer's disease in France: the economic implications. Dement Geriatr Cogn Disord. 2004, 17 (1-2): 5-13. 10.1159/000074079.

    Article  CAS  PubMed  Google Scholar 

  12. Green C, Picot J, Loveman E, Takeda A, Kirby J, Clegg A: Modelling the cost effectiveness of cholinesterase inhibitors in the management of mild to moderately severe Alzheimer's disease. Pharmacoeconomics. 2005, 23 (12): 1271-1282. 10.2165/00019053-200523120-00010.

    Article  CAS  PubMed  Google Scholar 

  13. O'Brien BJ, Goeree R, Hux M, Iskedjian M, Blackhouse G, Gagnon M, Gauthier S: Economic evaluation of donepezil for the treatment of Alzheimer's disease in Canada. J Am Geriatr Soc. 1999, 47 (5): 570-578.

    Article  PubMed  Google Scholar 

  14. Jonsson L, Lindgren P, Wimo A, Jonsson B, Winblad B: The cost-effectiveness of donepezil therapy in Swedish patients with Alzheimer's disease: a Markov model. Clin Ther. 1999, 21 (7): 1230-1240. 10.1016/S0149-2918(00)80025-9.

    Article  CAS  PubMed  Google Scholar 

  15. Neumann PJ, Hermann RC, Kuntz KM, Araki SS, Duff SB, Leon J, Berenbaum PA, Goldman PA, Williams LW, Weinstein MC: Cost-effectiveness of donepezil in the treatment of mild or moderate Alzheimer's disease. Neurology. 1999, 52 (6): 1138-1145.

    Article  CAS  PubMed  Google Scholar 

  16. Wimo A, Winblad B, Engedal K, Soininen H, Verhey F, Waldemar G, Wetterholm AL, Mastey V, Haglund A, Zhang R, et al: An economic evaluation of donepezil in mild to moderate Alzheimer's disease: results of a 1-year, double-blind, randomized trial. Dement Geriatr Cogn Disord. 2003, 15 (1): 44-54. 10.1159/000066669.

    Article  CAS  PubMed  Google Scholar 

  17. Stewart A, Phillips R, Dempsey G: Pharmacotherapy for people with Alzheimer's disease: a Markov-cycle evaluation of five years' therapy using donepezil. Int J Geriatr Psychiatry. 1998, 13 (7): 445-453. 10.1002/(SICI)1099-1166(199807)13:7<445::AID-GPS794>3.0.CO;2-E.

    Article  CAS  PubMed  Google Scholar 

  18. Teipel SJ, Ewers M, Reisig V, Schweikert B, Hampel H, Happich M: Long-term cost-effectiveness of donepezil for the treatment of Alzheimer's disease. Eur Arch Psychiatry Clin Neurosci. 2007, 257 (6): 330-336. 10.1007/s00406-007-0727-1.

    Article  PubMed  Google Scholar 

  19. Getsios D, Blume S, Ishak KJ, Maclaine GD: Cost effectiveness of donepezil in the treatment of mild to moderate Alzheimer's disease: A UK evaluation using discrete-event simulation. Pharmacoeconomics. 2010, 28 (5): 411-427. 10.2165/11531870-000000000-00000.

    Article  PubMed  Google Scholar 

  20. Getsios D, Blume S, Ishak KJ, Maclaine GD: Supplemental Digital Content: Cost Effectiveness of Donepezil in the Treatment of Mild to Moderate Alzheimer's Disease. 2010, [http://download.lww.com/wolterskluwer_vitalstream_com/PermaLink/PCZ/A/PCZ_28_5_2010_01_24_GETSIOS_1153187_SDC1.pdf]

    Google Scholar 

  21. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). Allgemeine Methoden zur Bewertung von Verhältnissen zwischen Nutzen und Kosten. Version 1.0 vom 12.10.2009, [http://www.iqwig.de/download/Methodik_fuer_die_Bewertung_von_Verhaeltnissen_zwischen_Kosten_und_Nutzen.pdf]

  22. Caro JJ: Pharmacoeconomic analyses using discrete event simulation. Pharmacoeconomics. 2005, 23 (4): 323-332. 10.2165/00019053-200523040-00003.

    Article  PubMed  Google Scholar 

  23. Caro JJ, Moller J, Getsios D: Discrete event simulation: the preferred technique for health economic evaluations?. Value Health. 2010, 13 (8): 1056-1060. 10.1111/j.1524-4733.2010.00775.x.

    Article  PubMed  Google Scholar 

  24. Hollingworth W, Spackman DE: Emerging methods in economic modeling of imaging costs and outcomes a short report on discrete event simulation. Acad Radiol. 2007, 14 (4): 406-410. 10.1016/j.acra.2007.01.007.

    Article  PubMed  Google Scholar 

  25. Bosanquet N, Yeates A: Modelling the cost effectiveness of cholinesterase inhibitors in the management of mild to moderately severe Alzheimer's disease. Pharmacoeconomics. 2006, 24 (6): 623-625. 10.2165/00019053-200624060-00008. author reply 625-626

    Article  PubMed  Google Scholar 

  26. Oremus M: Systematic review of economic evaluations of Alzheimer's disease medications. Expert Rev Pharmacoecon Outcomes Res. 2008, 8 (3): 273-289. 10.1586/14737167.8.3.273.

    Article  PubMed  Google Scholar 

  27. Tun SM, Murman DL, Long HL, Colenda CC, von Eye A: Predictive validity of neuropsychiatric subgroups on nursing home placement and survival in patients with Alzheimer disease. Am J Geriatr Psychiatry. 2007, 15 (4): 314-327. 10.1097/01.JGP.0000239263.52621.97.

    Article  PubMed  Google Scholar 

  28. Mohs RC, Doody RS, Morris JC, Ieni JR, Rogers SL, Perdomo CA, Pratt RD: A 1-year, placebo-controlled preservation of function survival study of donepezil in AD patients. Neurology. 2001, 57 (3): 481-488.

    Article  CAS  PubMed  Google Scholar 

  29. Winblad B, Engedal K, Soininen H, Verhey F, Waldemar G, Wimo A, Wetterholm AL, Zhang R, Haglund A, Subbiah P: A 1-year, randomized, placebo-controlled study of donepezil in patients with mild to moderate AD. Neurology. 2001, 57 (3): 489-495.

    Article  CAS  PubMed  Google Scholar 

  30. Feldman H, Gauthier S, Hecker J, Vellas B, Subbiah P, Whalen E: A 24-week, randomized, double-blind study of donepezil in moderate to severe Alzheimer's disease. Neurology. 2001, 57 (4): 613-620.

    Article  CAS  PubMed  Google Scholar 

  31. Gesundheitsberichterstattung des Bundes. [http://www.gbe-bund.de/]

  32. Consortium to Establish a Registry for Alzheimer's Disease (CERAD). [http://cerad.mc.duke.edu/Default.htm]

  33. Rogers SL, Farlow MR, Doody RS, Mohs R, Friedhoff LT: A 24-week, double-blind, placebo-controlled trial of donepezil in patients with Alzheimer's disease. Donepezil Study Group. Neurology. 1998, 50 (1): 136-145.

    Article  CAS  PubMed  Google Scholar 

  34. Rogers SL, Doody RS, Mohs RC, Friedhoff LT: Donepezil improves cognition and global function in Alzheimer disease: a 15-week, double-blind, placebo-controlled study. Donepezil Study Group. Arch Intern Med. 1998, 158 (9): 1021-1031. 10.1001/archinte.158.9.1021.

    Article  CAS  PubMed  Google Scholar 

  35. Black SE, Doody R, Li H, McRae T, Jambor KM, Xu Y, Sun Y, Perdomo CA, Richardson S: Donepezil preserves cognition and global function in patients with severe Alzheimer disease. Neurology. 2007, 69 (5): 459-469. 10.1212/01.wnl.0000266627.96040.5a.

    Article  CAS  PubMed  Google Scholar 

  36. Winblad B, Kilander L, Eriksson S, Minthon L, Batsman S, Wetterholm AL, Jansson-Blixt C, Haglund A: Donepezil in patients with severe Alzheimer's disease: double-blind, parallel-group, placebo-controlled study. Lancet. 2006, 367 (9516): 1057-1065. 10.1016/S0140-6736(06)68350-5.

    Article  CAS  PubMed  Google Scholar 

  37. Doody RS, Geldmacher DS, Gordon B, Perdomo CA, Pratt RD: Open-label, multicenter, phase 3 extension study of the safety and efficacy of donepezil in patients with Alzheimer disease. Arch Neurol. 2001, 58 (3): 427-433. 10.1001/archneur.58.3.427.

    Article  CAS  PubMed  Google Scholar 

  38. Winblad B, Wimo A, Engedal K, Soininen H, Verhey F, Waldemar G, Wetterholm AL, Haglund A, Zhang R, Schindler R: 3-year study of donepezil therapy in Alzheimer's disease: effects of early and continuous therapy. Dement Geriatr Cogn Disord. 2006, 21 (5-6): 353-363. 10.1159/000091790.

    Article  CAS  PubMed  Google Scholar 

  39. Stern RG, Mohs RC, Davidson M, Schmeidler J, Silverman J, Kramer-Ginsberg E, Searcey T, Bierer L, Davis KL: A longitudinal study of Alzheimer's disease: measurement, rate, and predictors of cognitive deterioration. Am J Psychiatry. 1994, 151 (3): 390-396.

    Article  CAS  PubMed  Google Scholar 

  40. Mendiondo MS, Ashford JW, Kryscio RJ, Schmitt FA: Modelling mini mental state examination changes in Alzheimer's disease. Stat Med. 2000, 19 (11-12): 1607-1616. 10.1002/(SICI)1097-0258(20000615/30)19:11/12<1607::AID-SIM449>3.0.CO;2-O.

    Article  CAS  PubMed  Google Scholar 

  41. Mohs RC, Schmeidler J, Aryan M: Longitudinal studies of cognitive, functional and behavioural change in patients with Alzheimer's disease. Stat Med. 2000, 19 (11-12): 1401-1409. 10.1002/(SICI)1097-0258(20000615/30)19:11/12<1401::AID-SIM432>3.0.CO;2-X.

    Article  CAS  PubMed  Google Scholar 

  42. McShane R, Areosa Sastre A, Minakaran N: Memantine for Dementia (Review). 2007, The Cochrane Library: JohnWiley & Sons

    Google Scholar 

  43. Lyle S, Grizzell M, Willmott S, Benbow S, Clark M, Jolley D: Treatment of a whole population sample of Alzheimer's disease with donepezil over a 4-year period: lessons learned. Dement Geriatr Cogn Disord. 2008, 25 (3): 226-231. 10.1159/000114450.

    Article  CAS  PubMed  Google Scholar 

  44. Deutsches Statistisches Bundesamt. [http://www.destatis.de/jetspeed/portal/cms/Sites/destatis/Internet/DE/Navigation/Navigationsknoten__Startseite.psml]

  45. Xie J, Brayne C, Matthews FE: Survival times in people with dementia: analysis from population based cohort study with 14 year follow-up. BMJ. 2008, 336 (7638): 258-262. 10.1136/bmj.39433.616678.25.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Kassenärztliche Bundesvereinigung. [http://www.kbv.de/8144.html]

  47. Hallauer JF, Schons M, Smala A, Berger K: Untersuchung von Krankheitshosten bei patienten mit alzheimer-erkrankung in Deutschland. Gesundheitsökonomie und Qualitätsmanagement. 2000, 5: 73-79.

    Google Scholar 

  48. European Commission: Eurostat. [http://epp.eurostat.ec.europa.eu/portal/page?_pageid=2714,1,2714_61582070&_dad=portal&_schema=PORTAL]

  49. Rote Liste. [http://www.rote-liste.de/]

  50. Kulp W, Graf GVD, Schulenburg J-M: Pharmakoökonomische Aspekte der Behandlung von Demenz-Patienten. Pharmazie unserer Zeit. 2002, 31 (4): 410-416. 10.1002/1615-1003(200207)31:4<410::AID-PAUZ410>3.0.CO;2-Y.

    Article  Google Scholar 

  51. Jonsson L, Andreasen N, Kilander L, Soininen H, Waldemar G, Nygaard H, Winblad B, Jonhagen ME, Hallikainen M, Wimo A: Patient- and proxy-reported utility in Alzheimer disease using the EuroQoL. Alzheimer Dis Assoc Disord. 2006, 20 (1): 49-55. 10.1097/01.wad.0000201851.52707.c9.

    Article  PubMed  Google Scholar 

  52. Schwarzkopf LMP, Kunz S, Holle R, Lauterberg J, Marx P, Mehlig H, Wunder S, Leidl R, Donath C, Graessel E: Costs of care for dementia patients in community setting: an analysis for mild and moderate disease stage. Value Health. 2011, 14 (6): 827-835. 10.1016/j.jval.2011.04.005.

    Article  PubMed  Google Scholar 

  53. Shearer J, Green C, Ritchie CW, Zajicek JP: Health state values for use in the economic evaluation of treatments for Alzheimer's disease. Drugs Aging. 2012, 29 (1): 31-43. 10.2165/11597380-000000000-00000.

    Article  PubMed  Google Scholar 

  54. Reese JPHmP, Seeberg G, Henkel D, Hirzmann P, Rieke J, Baum E, Dannhoff F, Müller MJ, Jessen F, Geldsetzer MB, Dodel R: Cost and care of patients with Alzheimer's Disease: clinical predictors in German health care settings. J Alzheimers Dis. 2011, 27 (4): 723-736.

    PubMed  Google Scholar 

  55. Leicht HHS, Heider D, Bachmann C, Bickel H, van den Bussche H, Fuchs A, Luppa M, Maier W, Mösch E, Pentzek M, Rieder-Heller SG, Tebarth F, Werle J, Weyerer S, Wiese B, Zimmermann T, König HH: AgeCoDe study group: Net costs of dementia by disease stage. Acta Psychiatr Scand. 2011, 124 (5): 384-395. 10.1111/j.1600-0447.2011.01741.x.

    Article  CAS  PubMed  Google Scholar 

  56. Kiencke PRR, Grimm C, Daniel D: Cost of illness in Alzheimer's disease. Med Klin (Munich). 2010, 105 (a5): 327-333.

    Article  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The statistical analyses were carried out by UBC's statistical team under the direction of Jack Ishak.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Denis Getsios.

Additional information

Competing interests

DG, ST, SB (SH is a former employee) are all employees of a consulting company, which has conducted research on behalf of Eisai, including research on donepezil. They have not received any reimbursements, fees, funding, or salary from an organization that could gain or lose financially from the publication of this manuscript. Partial funding for this manuscript was provided by Eisai, and Eisai will be financing the processing charge. Eisai reviewed the manuscript and Grant Maclaine, one of the co-authors was an employee of Eisai at the time of manuscript preparation. Eisai placed no restrictions on the content of the manuscript. They do not hold any stocks or shares. The only data used in the study that are not publicly available are those from the donepezil clinical trials (although results from these trials are all publicly available). Ethics approval for these trials had been obtained.

GM was an employee of Eisai Europe Limited at the time of preparation of the manuscript. Eisai provided funding for manuscript development and will pay the manuscript processing charge. Data, other than those from the donepezil clinical trials, are publicly available. Results from the donepezil clinical studies have been separately published and ethics committee approvals were received for these studies.

Authors' contributions

SH participated in the drafting of the manuscript, data collection, contribution to model analyses and interpretation of results. DG participated in the Model concept/design, data analyses and interpretation, critical revision of article. SB contributed to model design and implementation, data analyses, and drafting of the manuscript. ST contributed to the data/model analyses and critical revision of article. GM contributed to the model concept/design, critical revision/approval of the article (funding approved in cooperation with German Eisai affiliate).

Authors’ original submitted files for images

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Hartz, S., Getsios, D., Tao, S. et al. Evaluating the cost effectiveness of donepezil in the treatment of Alzheimer's disease in Germany using discrete event simulation. BMC Neurol 12, 2 (2012). https://doi.org/10.1186/1471-2377-12-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2377-12-2

Keywords