A recent HTA review concluded that there was a need for RCTs of exercise referral schemes (ERS) for people with a medical diagnosis who might benefit from exercise. Overall, there is still uncertainty as to the cost-effectiveness of ERS. Evaluation of public health interventions places challenges on conventional health economics approaches. This economic evaluation of a national public health intervention addresses this issue of where ERS may be most cost effective through subgroup analysis, particularly important at a time of financial constraint.
This economic analysis included 798 individuals aged 16 and over (55% of the randomised controlled trial (RCT) sample) with coronary heart disease risk factors and/or mild to moderate anxiety, depression or stress. Individuals were referred by health professionals in a primary care setting to a 16 week national exercise referral scheme (NERS) delivered by qualified exercise professionals in local leisure centres in Wales, UK. Health-related quality of life, health care services use, costs per participant in NERS, and willingness to pay for NERS were measured at 6 and 12 months.
The base case analysis assumed a participation cost of £385 per person per year, with a mean difference in QALYs between the two groups of 0.027. The incremental cost-effectiveness ratio was £12,111 per QALY gained. Probabilistic sensitivity analysis demonstrated an 89% probability of NERS being cost-effective at a payer threshold of £30,000 per QALY. When participant payments of £1 and £2 per session were considered, the cost per QALY fell from £12,111 (base case) to £10,926 and £9,741, respectively. Participants with a mental health risk factor alone or in combination with a risk of chronic heart disease generated a lower ICER (£10,276) compared to participants at risk of chronic heart disease only (£13,060).
Results of cost-effectiveness analyses suggest that NERS is cost saving in fully adherent participants. Though full adherence to NERS (62%) was higher for the economics sample than the main sample (44%), results still suggest that NERS can be cost-effective in Wales with respect to existing payer thresholds particularly for participants with mental health and CHD risk factors.
Current Controlled Trials ISRCTN47680448
Keywords:Public health policy; Exercise referral; Mental health; Heart disease risk factors; Cost-effectiveness
Lack of adequate physical exercise is a contributing factor in CHD risk, musculoskeletal problems, diabetes and depression which places a huge economic burden on the NHS and wider society in the UK  with similar trends worldwide. There is growing interest in general practice (GP) exercise referral schemes. Referred patients access an individually tailored exercise programme, including; exercise classes, swimming, green gyms and yoga.
Economic evaluation of public health interventions places challenges beyond the economic evaluation of clinical interventions in terms of potential wider benefits, multiple stake holders bearing costs, and time horizons over which costs and benefits will be accrued [2,3]. It is helpful to view such public health interventions as complex interventions with multiple components (in this case, fidelity to specific exercise interventions, compliance (dose) and uptake of the intervention, and mechanisms of behaviour change) [4,5].
Internationally, cost-effectiveness evidence has been equivocal compared with other population level interventions . Matrix (2006),  reviewed the literature and modelled the cost-effectiveness of four interventions, pedometers, exercise referral, walking and cycling programmes in the community. Cost per QALY estimates for all four interventions were well below the NICE threshold of £20,000 to £30,000 per quality adjusted life year (QALY). Pavey et al. (2011),  showed a 51% probability of GP referral being cost-effective at £20,000 per QALY, 88% at £30,000 per QALY. In Australia, Cobiac et al. (2009),  modelled disability adjusted life years (DALYs) gained from six population level exercise promotion interventions. They found that pedometer use, mass media-based community campaigns, GP exercise referral, healthy transport and internet programmes were within the AUS $50,000 threshold.
There have been few prospective economic evaluations alongside pragmatic, community based RCTs of GP exercise referral schemes. Munro et al. (2004),  calculated a cost per QALY of €17,174 for a community-based exercise programme in over 65 year olds. Owen et al. (2011),  published a list of 200 public health interventions included in NICE guidance, of which exercise referral was one of the 85% of interventions under the £20,000 threshold. The above authors do not explore cost-effectiveness with respect to subgroups or explore how interventions address known inequalities in health, points recognised by Kelly et al. (2005) and Weatherly et al. (2009), [2,3]. In particular, how effective and cost effective ERS are for participants with a medical diagnosis which may benefit from exercise .
This economic evaluation was conducted alongside a pragmatic randomised controlled trial of the Welsh National Exercise Referral Scheme (NERS) [11,12]. Detailed effectiveness and headline economic results are published elsewhere . NERS was designed to encourage increased activity in patients at low or medium risk of coronary heart disease or with mild to moderate depression. The scheme was sponsored by the Welsh Government (WG) in partnership with local authorities, National Public Health Service and Local Health Boards. The aim of our economic evaluation was to assess the cost-effectiveness of NERS in Wales from a multi-agency public sector perspective and to use subgroup analysis to explore the effects of medical diagnosis, gender, age, inequalities, referral route and adherence on effectiveness and cost-effectiveness.
A pragmatic, randomised controlled trial, with nested economic evaluation was conducted in 12 of the 22 local health board (LHB) areas in Wales, UK (for full details see study protocol) [11,12]. Participants aged >16 years, sedentary with risk factors for coronary heart disease (CHD) or minor mental health problems, were recruited by health professionals in primary care and randomised to intervention or control group. The intervention group participants were offered a 16-week exercise programme delivered by qualified exercise professionals (EP) based in local authority leisure centres. Participants allocated to the control group received an information leaflet and preferential access to the scheme after 12 months.
Data collection and economic measures
Postal questionnaires were sent to all participants at baseline, 6 and 12 months. The questionnaire at 12 months included a willingness to pay question. Non-respondents received a reminder two weeks later. The Client Service Receipt Inventory  asked participants to recall their recent contacts with health care services in primary and secondary care, including prescribing. EQ-5D (3 L)  is a generic, preference based instrument for evaluating health-related quality of life. A minimal questionnaire pack was sent at baseline assessing demographics and activity levels to reduce participant burden and facilitate recruitment.
A breakdown of the budget for NERS for 13 local authorities initially providing NERS was obtained from WG. This budget included 50% of local authority exercise coordinators’ salaries, 100% of leisure centre exercise professionals’ (EP) salaries, training; promotional materials, administration, information technology, travel and equipment. Telephone interviews with NERS programme directors at WG and leisure centre managers were conducted to capture additional costs incurred. One local authority incurred costs, although it did not implement the scheme during the trial period.
Cost and cost-effectiveness analysis
The mean national and local authority cost of running NERS for one year was used to calculate the range of costs and a cost per participant in the Scheme. We varied our assumptions around the number of referrals based on three estimates of participation;
a. referrals to the trial from July 2007 to October 2008, (n = 4779);
b. participants who accessed NERS during the trial, those in the intervention group (n = 1080) plus people referred to NERS, but not eligible for the trial (n = 1493), minus an estimated 15% who would not attend, (n = 2349).
c. and an arithmetic mean between our upper and lower estimates, (n = 3530).
Cost-utility analysis was conducted in line with MRC guidelines for the evaluation of complex interventions,  and our standard operating procedure for economic evaluation alongside pragmatic RCTs, . When EQ-5D,  data was missing for one or two domains in the five domain scale, stochastic imputation (assuming data was missing at random) was used (n = 28 scores). Since health-related quality of life was assessed at 6 and 12 month follow-up only, the base-case analysis used area under the curve (AUC) data for the period between 6 and 12 months to estimate QALY change. Sensitivity analyses were conducted using: a) EQ-5D,  scores at 6 months for both intervention and control group as an estimate of baseline differences, and b) using control group mean EQ-5D scores at 6 months as a point estimate of EQ-5D,  score at baseline. Sensitivity analyses were conducted for: different intervention costs (£285, £385 and £579); referral to NERS by GP, nurse, or 50% referral by GP and 50% by nurse; participant payment of £1 per session, and of £2 per session (taking into account Wales Index of Multiple Deprivation; WIMD). Subgroup analyses compared the cost-effectiveness of NERS depending on gender, age group (<44, 44–60 or >60 years), adherence to the Scheme (16 weeks or <16 weeks), and referral reason (CHD and/or mental health risk factors). Cost-utility analyses were carried out using STATA SE 10. Costs and benefits were not discounted as the trial follow-up period was 12 months.
Characteristics of economic sample
The economic sample, for whom full cost and EQ-5D,  data were collected (n = 798), represented 55% of the main effectiveness trial sample . There were no statistically significant differences between the two samples with regard to gender of participants, or the WIMD . However, the economic sample contained fewer younger participants (18%) than the effectiveness sample (30%), and a higher proportion of individuals who were referred because of CHD risk (77%) compared to the main sample (71%). There were no significant differences between the control and intervention groups in the economic sample with regard to demographic characteristics, levels of physical activity at baseline, referral reasons, and the mean number of reported visits to health professionals in the previous six months (Additional file 1: Table S1).
The cost of the national exercise referral programme
Table 1 shows the total set up and annual operating costs of the NERS. Setup costs included salaries, meetings, printing and resources, translation costs and training for exercise professionals. Costs incurred by WG for the 6 pilot areas in 2006–07 are included as part of NERS development costs (£183,600). The estimated total setup costs incurred by the WG were £365,875; mean cost per local authority £28,144. Annual operating costs for NERS in 2007/08 was £1.36 million, mean per local authority was £104,602 (SD £27,226). Operating costs are inclusive of salaries for coordinators, exercise professionals, printing, administration, travel, staff management, additional training and room hire.
Table 1. NERS set up costs for 13 local authorities in Wales in 2006 – 08 and operational costs for 2007 – 08 based on national budget plus national and local authority additional costs (£)1
On an intention-to-treat basis, to estimate the cost per participant in NERS, we divided the total annual operating costs for NERS for 2007/08 (£1,359,832), by 52 (weeks), multiplied by 16 (16 week programme), then divided this figure by the three estimates of number of referrals to NERS across the participating local authorities. We report a base case ICER calculation using the mid point estimate of the cost per participant (£385, n = 3530), in addition to an upper (£579, n = 2349) and lower estimate (£285, n = 4779), (Table 1).
Leisure centre managers reported no lost revenues from providing NERS, as sessions were held at quieter times of the day, thus utilizing underused space.
Use of health services
National unit costs of health services used in the calculations can be accessed as Additional file 2: Table S2. Table 2 shows the frequency and total mean costs of service use by intervention and control groups, subdivided into primary care (including prescribing) and secondary care. For the 12 month study period there was no significant difference in NHS resource use between the intervention and control groups, apart from the costs of healthcare tests, higher for the control group (p = < 0.05).
Format: DOC Size: 47KB Download file
This file can be viewed with: Microsoft Word Viewer
Table 2. NERS resource use and costs over 12 months by group
Adherence to NERS
Approximately 62% completed the 16 week programme, 32% (n = 123) attended fewer than 16 weeks, and 8% (n = 30) did not attend at all. Table 3 breaks down adherence by reason for referral, gender, age and the WIMD, . Those at risk of CHD were more likely to adhere to the full programme than those with mental health problems, and those with both mental health problems and at risk of CHD. A higher percentage of men completed the programme than women. Those people living in areas of high deprivation were more likely to complete the programme (66%) compared with those living in areas of low deprivation (60%).
Table 3. Mean effectiveness outcomes by group over 12 months
Participants’ willingness to pay for exercise classes
At 12 month follow-up, 55% completed the willingness to pay question (n = 441). The amount people were willing to pay was £2.16 (SD £1.50) for those in the intervention group compared with £2.38 (SD £1.78) in the control. This had fallen from the baseline figure of £2.43 (SD £1.67) compared with £2.55 (SD £2.49) in the control (81% response, n = 648). There was a difference in willingness to pay (WTP) depending upon whether people lived in an area of low or high deprivation as measured by the WIMD. At 12 month follow-up people from areas of high deprivation were WTP £1.91 (SD £0.88) compared with £2.05 for those from areas of low deprivation. Similarly, in the control group people were willing to pay £2.09 (SD £1.53) and £2.32 (SD £1.78), respectively. In all groups people valued the intervention more highly if they were in the control group. When considering the WTP by employment status then students were willing to pay the most per session and those seeking work the least.
The base case analysis produced a mean difference in cost of £327 and a mean difference in QALY gains of 0.027 between groups, indicating NERS was more expensive and more effective compared to the control. The incremental cost-effectiveness ratio (ICER) was £12,111 per QALY gained. Figure 1 shows the cost-effectiveness plane with 1,000 bootstrapped ICER estimates. The figure demonstrates that the majority of simulations fell in the north-east quadrant of the cost-effectiveness plane, where the intervention is more expensive and more effective compared to the control condition.
Figure 1. Cost-effectiveness planes with 1,000 bootstrapped ICER estimates for NERS economic evaluation. Cost-effectiveness planes for base case (A), cost-effectiveness acceptability curves for three intervention costs of £285, £385 (base case) and £579 (B), and for participant subgroups with different referral reason to NERS (C), adherence to NERS (D), gender (E), and age of participants (F).
Probabilistic sensitivity analysis conducted for our base case demonstrated that there is a 77% probability of NERS being cost-effective at £20,000 per QALY and 89% at £30,000 per QALY gained (Table 4). To assess the uncertainty associated with cost of NERS, sensitivity analyses were performed assuming an annual cost of attending the programme of either £285 or £579 per person per year. The ICER point estimates were £8,556 and £19,296 per QALY, respectively, remaining below a £30,000 payer threshold. However, for an annual cost of attending the NERS programme of £579 per person, the probability of the scheme being cost-effective was lower than the base case at 54% and 70% at £20,000 and £30,000 thresholds, respectively.
Table 4. Results of cost-utility analysis
When EQ-5D,  data at 6 months was used as an estimate of baseline EQ-5D scores, the difference in QALY gain between the intervention and control groups was 0.054, resulting in a decrease in ICER estimate from £12,111 to £6,056. Using the control group mean EQ-5D,  score at 6 months as an estimate of baseline values for both intervention and control groups, produced a QALY difference of 0.046 and an ICER point estimate of £7,109. Probabilistic sensitivity analyses demonstrated that alternative methods of QALY calculation resulted in higher probabilities of NERS being cost-effective compared to our base case, which can be considered conservative (Table 4).
Pragmatic sensitivity analyses were conducted to compare referrals to NERS by a GP or a nurse. When all patients were referred by a GP this resulted in a higher ICER (£13,444) compared to referral by nurse (£12,667) or 50-50% by GP and nurse (£13,074) (probability of being cost-effective was 72%,76% and 75% respectively at £20,000 threshold; and 87%, 89% and 88% respectively at £30,000 threshold). When participant payments of £1 and £2 per session were considered in the cost-utility analysis, the cost per QALY fell from £12,111 to £10,926 and £9,741, respectively. The probability of NERS being cost-effective at £30,000 per QALY increased from 89% to 91% and 92%, with the £1 and £2 per session participants’ contributions (Table 4).
Since the participant sample used in our cost-utility analysis was heterogeneous with relation to age (16–88 years), gender, reason for referral and adherence to NERS, a range of subgroup analyses were conducted to identify population groups which would most benefit from joining the NERS Scheme (Table 4). The cost-effectiveness acceptability curves (CEACs), (Figure 1C-F) demonstrated NERS is more likely to be cost-effective in younger participants (<44 years) than in older individuals. Participants with a mental health risk factor alone or in combination with a risk of chronic heart disease were characterized by a lower ICER (£10,276) compared to participants at risk of chronic heart disease only (£13,060). Subgroup analysis by gender, gave a significantly lower ICER for women (£9,006) than for men (£38,333). Pronounced differences in ICER were found between fully adherent participants and those attending fewer than 16 sessions (Table 4). For participants adhering to NERS, the analysis resulted in a net saving of £18 per patient per year. The probability that NERS is cost-effective in adherent participants was 100% at £20,000 (Figure 1).
The main trial effectiveness findings  demonstrated that the NERS was effective in increasing weekly exercise and improved self-reported mental health at 12 months as compared to the control group. A larger effect was found for women versus men and for persons who completed the 16 week programme, as compared with those who were non-adherent. Results of cost-effectiveness analyses suggest that NERS is likely to be cost saving in fully adherent participants . Our base case analysis and sensitivity analyses, we see that the ICER for NERS under different assumptions falls largely below a payer threshold of £20-30,000. However we recognize the need for caution in interpretation of ICERs which have an upper 95% confidence limit above £20,000 (Table 4). From a methodological viewpoint, the NERS trial shows the benefits of incorporating subgroup analysis and analysis of how public health interventions may impact on inequalities in health through i.e. differing uptake and effects by women and men, and differing willingness to pay by areas of socioeconomic deprivation. Our conservative base case analysis is robust to a range of sensitivity analyses, leading to the conclusion that NERS is likely to be cost-effective, under the £20,000 to £30,000 threshold (89% probability at £30,000).
Decisions based on average measures of cost-effectiveness carry a risk of overlooking potential benefits for a given subset of the population. In the current climate of limited health care resources, there is a clear necessity to maximise population health by delivering healthcare interventions to those recipients who would most benefit, . The proposed ways to account for heterogenity between population groups included: reporting ICER estimates for different subgroups, incorporating covariate adjustment in cost-effectiveness analysis, and using alternative cost-effectiveness indicators, [20-24]. Our analysis revealed wide variations in ICERs between subgroups.
Our analysis shows that adherence to exercise was greater in those areas of high deprivation as measured by the WIMD, . We also found that those people living in areas of high deprivation placed a higher value on NERS than those living in areas of lower socioeconomic deprivation. Our study goes some way to address current thinking in public health economics about the need to consider equity implications of access and uptake, in addition to cost-effectiveness, .
Comparison with other literature
Economic evaluation of exercise referral schemes involving inactive adults in primary care have been conducted as a part of the NIHR Health Technology Assessment programme, . The base-case cost-effectiveness analysis of NERS in Wales generated comparable data with previous reports, [6-9,25] demonstrating that the exercise referral schemes may be cost-effective under the £30,000 per QALY threshold. However, lifetime modelling exercises indicate a greater uncertainty of the long term cost-effectiveness of such programmes, . Our study suggests that NERS would be also be cost-effective in individuals at risk of CHD and mental health issues, as well as in individuals aged 60 years and over. The NERS may be cost-saving assuming full adherence to the programme.
Strengths and limitations
There have been few rigorous prospective economic studies alongside RCTs of exercise referral interventions. Results of the NERS trial, as a pragmatic policy evaluation, are likely to have high external validity and generalisabilty, . Full details of the methodological design of the NERS trial and its acknowledged limitations are published elsewhere, [11,12]. The NERS trial set out to explore the potential effectiveness and cost effectiveness of a national exercise referral scheme, integrating motivational interviewing, and making use of existing leisure centres. The NERS trial took a pragmatic approach, in that it was powered to show a difference in reported exercise at 12 months, by telephone, using 7D-PAR, a well validated measure, rather than focussing on load i.e. exercise type, or intensity. The NERS trial did focus on patients with CHD risk factors, mental health problems or both, but did not specify exercise type. Indeed the exercise was tailored, and this tailoring may explain the relatively high adherence rate to the programme. The NERS trial differs from published trials of specific exercise programmes for people with specific diagnoses, and hence economic evaluation of the NERS trial in terms of cost-utility analysis differs from cost-utility analyses of specific exercise programmes for people with specific diagnoses, . The trial specifically recruited patients with CHD risk factors, mental health problems or both conditions. The trial found that NERS was effective in increasing physical activity among those referred with CHD risk factors. Although there was no increase in physical activity among those referred for mental health reasons, anxiety and depression were reduced. These effects were highly dependent on adherence to the programme. To these results , this paper adds cost per QALY estimates for NERS, for comparison with payer thresholds, such as that of NICE, and for comparison with those of other large, community based ERS, . The NERS trial allowed a fully integrated economic evaluation which took a wide perspective, measuring costs and benefits to the WG, local authority leisure centres as well as the NHS. To reiterate, the methodological design of the NERS trial was more about achieving external validity in terms of policy evaluation at a macro level, than about micro level evaluation of a specific exercise programme for a specific diagnosis or patient group, in a sports science context.
A limitation of our study was that baseline EQ-5D,  data was not available. However, on the small set of baseline measures which were taken, activity level, service use over the preceding 6 months, age and gender, the intervention and control groups were similar. Our economic analysis was based on a smaller number of participants, because not all participants completed the economic questionnaires. We acknowledge that using a smaller number of participants in subgroup analysis increases statistical uncertainty. Reported ICER point estimates have a rather explorative value and require further verification.
We cannot extrapolate beyond the 12 month follow up period of the trial which is a limitation common to many trials of public health interventions. A future modeling exercise may help address the question of whether such an exercise intervention could have longer term benefits.
The effectiveness and economic results of the NERS trial suggest that age, gender and adherence all have an important part to play in affecting weekly exercise behaviour and self reported, health related quality of life. Weatherly et al. (2009),  have stressed the need for economic evaluations of public health interventions to examine equity considerations. Our results indicate that people living in areas of higher deprivation who were offered NERS were likely to adhere to the complete programme, although more research would be needed to investigate if this was maintained in the long-term.
Though full adherence to NERS (62%) was higher for the economics sample than the main sample (44%), our base case analysis over a 12 month follow up period, is robust to a range of sensitivity analyses. ICERs were well below the NICE threshold of £20-30,000, though upper 95% confidence limits cross this boundary, indicating the need for caution in the interpretation of results. There is evidence for confidence that NERS is likely to be cost saving in fully adherent participants, leading to the overall conclusion that NERS can be cost-effective.
The authors declare that they have no competing interests.
RTE was responsible for design of the economic evaluation and overseeing the economic evaluation. PL was responsible for the day-to-day running of the economic evaluation. NH was responsible for the cost-effectiveness analysis. RTE, PL and NH produced a first draft of the manuscript. LR was trial manager, SM was principal investigator. LM and NW were part of the trial management team. All authors read and commented on drafts and approved the final manuscript and had full access to all the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. RTE acts as guarantor. All authors read and approved the final manuscript.
We thank all participants of the NERS programme and the staff at the local authorities who participated in the study. We thank the scheme’s coordinators, Elaine McNish and Jeannie Wyatt-Williams, and Janine Hale and Chris Roberts of the Welsh Government for providing cost information and on going support to facilitate this study. We thank Alison Shaw for clerical support, Carys Jones who helped with prescribing costs and Joanna Charles for assisting with the preparation of the manuscript. The research was independent and funded by the Welsh Assembly Government. RTE is supported by Public Health Wales. Additional support for LM and SM during write up was provided by The Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), a UKCRC Public Health Research: Centre of Excellence. Funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council (RES-590-28-0005), Medical Research Council, the Welsh Assembly Government and the Wellcome Trust (WT087640MA), under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged.
Securing Good Health for the Whole Population Final Report.
Economic appraisal of public health interventions.
Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Rice N, Sculpher M, Sowden A: Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations.
Developing and evaluating complex interventions: new guidance.
Health Econ 2012.
doi:10.1002/hec.2795 [Online First]
PLoS Med 2009, 14:e1000110.
doi:10.1371/journal.pmed.1000110 [Online First]
NICE Rapid review of the economic evidence of physical activity interventions.
Pavey TG, Anokye N, Taylor AH, Trueman P, Moxham T, Fox KR, Hillsdon M, Green C, Campbell JL, Foster C, Mutrie N, Searle J, Taylor RS: The clinical effectiveness and cost-effectiveness of exercise referral schemes: a systematic review and economic evaluation.
J Public Health 2012, 34:37-45. Publisher Full Text
Murphy S, Raisanen L, Moore G, Edwards RT, Linck P, Williams N, Din NU, Hale J, Roberts C, McNaish E, Moore L: A pragmatic randomised controlled trial of the Welsh National Exercise Referral Scheme: protocol for trial and integrated economic and process evaluation.
Murphy SM, Edwards RT, Williams N, Raisanen L, Moore G, Linck P, Hounsome N, Din NU, Moore L: An evaluation of the effectiveness and cost effectiveness of the National Exercise Referral Scheme in Wales, UK: a randomised controlled trial of a public health policy initiative.
User Guide version 2.
Department of Health NHS Reference Costs 2007–08.
Unit Costs of Health and Social Care 2009.
NHS Information Centre Prescribing Cost Analysis 2008 England. 2008.
Wales Index of Multiple Deprivation (WIMD) 2008.
PLoS Med 2011, 8:e1001058.
doi:10.1371/journal.pmed.1001058 [Online First]PubMed Abstract | Publisher Full Text | PubMed Central Full Text
The pre-publication history for this paper can be accessed here: