Email updates

Keep up to date with the latest news and content from BMC Complementary and Alternative Medicine and BioMed Central.

Open Access Highly Accessed Research article

Is complementary and alternative medicine (CAM) cost-effective? a systematic review

Patricia M Herman1*, Benjamin M Craig2 and Opher Caspi3

Author Affiliations

1 Program in Integrative Medicine, University of Arizona, Tucson, Arizona, USA

2 Department of Pharmacy, University of Arizona, Tucson, Arizona, USA

3 Recanati Center for Internal Medicine and Research, Rabin Medical Center (Beilinson Campus), Petah Tikva, Israel

For all author emails, please log on.

BMC Complementary and Alternative Medicine 2005, 5:11  doi:10.1186/1472-6882-5-11

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6882/5/11


Received:23 December 2004
Accepted:2 June 2005
Published:2 June 2005

© 2005 Herman et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Out-of-pocket expenditures of over $34 billion per year in the US are an apparent testament to a widely held belief that complementary and alternative medicine (CAM) therapies have benefits that outweigh their costs. However, regardless of public opinion, there is often little more than anecdotal evidence on the health and economic implications of CAM therapies. The objectives of this study are to present an overview of economic evaluation and to expand upon a previous review to examine the current scope and quality of CAM economic evaluations.

Methods

The data sources used were Medline, AMED, Alt-HealthWatch, and the Complementary and Alternative Medicine Citation Index; January 1999 to October 2004. Papers that reported original data on specific CAM therapies from any form of standard economic analysis were included. Full economic evaluations were subjected to two types of quality review. The first was a 35-item checklist for reporting quality, and the second was a set of four criteria for study quality (randomization, prospective collection of economic data, comparison to usual care, and no blinding).

Results

A total of 56 economic evaluations (39 full evaluations) of CAM were found covering a range of therapies applied to a variety of conditions. The reporting quality of the full evaluations was poor for certain items, but was comparable to the quality found by systematic reviews of economic evaluations in conventional medicine. Regarding study quality, 14 (36%) studies were found to meet all four criteria. These exemplary studies indicate CAM therapies that may be considered cost-effective compared to usual care for various conditions: acupuncture for migraine, manual therapy for neck pain, spa therapy for Parkinson's, self-administered stress management for cancer patients undergoing chemotherapy, pre- and post-operative oral nutritional supplementation for lower gastrointestinal tract surgery, biofeedback for patients with "functional" disorders (eg, irritable bowel syndrome), and guided imagery, relaxation therapy, and potassium-rich diet for cardiac patients.

Conclusion

Whereas the number and quality of economic evaluations of CAM have increased in recent years and more CAM therapies have been shown to be of good value, the majority of CAM therapies still remain to be evaluated.

Background

Complementary and alternative medicine (CAM) has a reputation for good value among health conscious consumers [1]. In the United States consumers spend over $34 billion per year on CAM therapies [2], dollars spent outside the conventional health care financing system. Such evidence on out-of-pocket expenditures is a testament to the widely held belief that CAM therapies have benefits that outweigh their costs. Regardless of public opinion, there is often little more than anecdotal evidence on the health and economic implications of CAM therapies.

The paucity of outcomes research in CAM has likely depressed access to CAM therapies by impeding their integration into financial mechanisms commonly found in conventional health care. Most US consumers who have health insurance coverage, either through public or private institutions, bear the entire cost of CAM therapies out-of-pocket [3]. Theoretically, CAM therapies seem effective and a good candidate for cost savings because they avoid high technology, offer inexpensive remedies, and harness the power of vis medicatrix naturae (the body's natural ability to heal itself). As such, a thorough and external review of economic and health outcomes of CAM is necessary for evidence-based consideration of CAM therapies as a covered expense. That being said, it is also known that affirmative evidence on economic and health outcomes is a necessary, but not sufficient step toward CAM coverage, and not the decision itself. Other factors such as historical demand, political expediency, consumer demand, and practitioner enthusiasm may also be considered in the decision to incorporate CAM into a health insurance policy [1,4,5].

The need for economic evaluations is also growing in conventional healthcare. An increasing number of health plans and hospitals have moved from a simple budgetary focus in formulary decisions to requiring detailed evidence on the economic value of considered therapies relative to alternatives [6,7]. Beyond their use in decisions concerning health insurance coverage, economic outcomes of both CAM and conventional therapies also influence health policy, justify licensure of practitioners, inform industry investment decisions, provide general evidence to consumers about potential economic benefits, and can guide future research efforts through identifying decision-critical parameters for additional research [8,9].

In their systematic review of CAM economic evaluations, White and Ernst [4] identified 34 economic evaluations of CAM conducted between 1987 and 1999; only eleven of which were full economic evaluations (ie, compared both economic and health outcomes between two or more alternatives) [10]. Quality was evaluated by noting whether cost data were collected prospectively and whether comparison groups were comparable – ie, assigned randomly. Unfortunately, their search strategy included the term "alternative medicine" but not "complementary medicine." Therefore, all single therapy studies in their review are of CAM therapies that are usually used as substitutes (alternatives) to conventional care (eg, acupuncture, homeopathy, and spinal manipulation). No studies of complementary therapies (those used in conjunction with conventional care) were included, despite the use of the term "complementary" in their conclusion that spinal manipulative therapy may have benefits for back pain, but "there was a paucity of rigorous studies that could provide conclusive evidence of differences in costs and outcomes between other complementary therapies and orthodox medicine [4]."

The objectives of this paper are: 1) to introduce concepts commonly applied in economic evaluations of health technologies (often called technology assessment) so that practitioners and CAM users can translate and benefit from published evidence; and 2) present a systematic review of the current scope and quality of economic evaluations of CAM. We begin with an overview of economic evaluation, including didactic examples from the CAM economic literature to help clarify the concepts presented. Readers familiar with this type of analysis can skip this section and proceed directly to the methods section.

In our systematic review we expand upon and update the initial review by White and Ernst. We evaluate study quality in more detail, using both additional study design criteria and quality of reporting criteria, and present a summary of the results from exemplary studies. While their review was the first of its kind, economic evaluations in the CAM literature have improved greatly in the last five years. We end the paper with a description of the attributes of CAM that make economic evaluation challenging and how these issues may be addressed. We hope that practitioners' interest in economic evaluation will continue to grow, leading to greater incorporation of this research into CAM trials.

What is an economic evaluation?

An economic evaluation is a comparison of outcomes among alternative ways of achieving common objectives. These analyses are conducted according to explicit, systematic, and consistent criteria, and take into account both the positive and negative consequences of each alternative. Consequences may include economic, clinical, and humanistic outcomes, known as the ECHO model [11]. Economic outcomes represent the consumption and production of resources and their monetary value from the perspective of a decision maker. Clinical outcomes are medical events that are professionally meaningful. Humanistic outcomes are a broad category of intangible personal attributes, typically collected through self-report. Humanistic outcomes include quality of life characteristics such as sense of safety, physical comfort, enjoyment, meaningful activity, relationships, functional competence, dignity, privacy, individuality, autonomy, and spiritual well-being. Conventionally, clinical and humanistic outcomes are considered health outcomes, and we follow this convention for the remainder of the article.

There are several forms of economic evaluations that can be performed (cost-effectiveness analysis being only one of these) and each differs based on the selection and measurement of health outcomes. The perspective (or point of view) taken for the analysis also influences the selection and measurement of consequences, because not all outcomes are important to all decision makers. Generally, there are three perspectives for economic analysis: individual (eg, patient), institutional (eg, health maintenance organization), or societal. The societal perspective accumulates all outcomes, while individual and institutional analyses are more selective. Regardless of perspective, the objective of an economic evaluation is to provide information on consequences relating to alternatives faced by a decision maker.

The most basic form of economic evaluation is a table that lists the individual economic and health outcomes of alternative interventions. This table is known as a cost-consequence study. Cost-identification studies and cost-minimization analyses only address economic outcomes and are discussed below in that section. The remaining forms of economic evaluations summarize economic and health outcomes into a single result (Table 1).

Table 1. Three forms of full economic evaluations

The advantages of performing cost-benefit and cost-utility analyses are that multiple outcomes are summarized into a single unit, either monetary units such as dollars (CBA) or QALYs (CUA) and that therapies with different sets of health outcomes can be compared based on the differences in the summary measures. Cost-benefit analysis has the additional benefit of directly indicating whether the therapy pays for itself.

The disadvantages of CBA and CUA come from the techniques required to produce a summary measure. Cost-benefit analysis requires putting a monetary value on all health outcomes (and ultimately on life), and cost-utility analysis assigns value to health outcomes based on their contribution to quality of life under the presumption of population-based preferences. An extensive literature addresses the methodological and theoretical issues involved in the construction of these summary measures. The process usually occurs in two steps. In the first step, health outcomes of the intervention are measured, and in the second the outcomes are valued in summary units and aggregated. Cost-benefit analyses often assess the monetary value of health outcomes based on willingness-to-pay using a technique called conjoint analysis [12-14]. Willingness-to-pay inherently places a lower values of life on individuals with low income, because they can not pay what they do not have. Cost-utility analyses have multiple methods to place quality of life values on health outcomes, also known as social tariffs. Summary measures of quality of life may not be sensitive enough to pick up short-term changes such as for acute conditions and will not pick up specific clinical outcomes like blood pressure control [15]. Examples of instruments used to capture these general health states include the EuroQoL (EQ-5D) [16] and the Health Utilities Index [17].

Cost-effectiveness analysis (CEA) is the current standard in the literature, and has the most straight forward interpretation. Under CEA, therapies useful for a specific disease or condition can be directly compared using a metric of effectiveness relevant to that condition, such as blood pressure control. Although these types of analyses do not allow a summary of multiple outcomes they tend to respond well to the most urgent questions, such as how much would it cost to reduce the number of gestational diabetes cases by 10%? Clearly, a reduction in gestational diabetes cases has measurable implications in quality of life and economic units, but the creation of a summary measure is not necessary to address the decision maker's question.

No matter the approach taken, it is recommended that the estimated outcomes (economic, clinical and humanistic) of health care alternatives used in economic evaluation are best estimated in pragmatic clinical trials that directly and realistically compare the therapies of interest [10]. Rarely are the results of placebo-controlled trials appropriate [1,4,18-20]. Also, since many CAM therapies target chronic disease, it is important that the study period be long enough to capture the full benefits and costs of each therapy, and that future costs and benefits be discounted to the present for comparison. Finally, all economic evaluations should include some type of sensitivity analysis to test the robustness of results to the various assumptions made [1,20,21].

What are economic outcomes?

Economic outcomes are the net bundle of resources forgone due to an intervention valued at the opportunity cost of those resources (the value of their next best use or "opportunity"). Since the cost of a therapy differs depending on whether you are a patient, a health plan, or a health care provider, the economic outcomes (ie, costs) of each therapy depend on the perspective of the study. Studies that only measure the economic outcomes of interventions are known as cost-identification studies. A study that describes the economic and health outcomes of a single therapy can also be called a cost-identification study. These studies inform full economic evaluations. That is, they provide the data needed to better design future studies that consider both the economic and health outcomes of two or more alternative therapies. A cost-minimization analysis (CMA) explicitly assumes equivalence in health outcome among alternative therapies, and examines only economic outcomes. In practice, it appears the same as a cost-identification study, but under the assumption of equivalence, a CMA is a full economic evaluation.

Table 2 has been summarized from other references [1,20,22] and gives a list of the types of economic outcomes and the perspective of analysis where each is considered. Note that these types of economic outcomes should be inclusive of both the full costs of the therapy and of any treatment for adverse effects, which can be expensive. In economic evaluations, the safety of a therapy is addressed through accounting for the cost of treating these adverse events as well as through their impact on clinical and quality of life outcomes.

Table 2. Economic outcomes to include in economic evaluation

It is recommended that economic outcome data are best collected prospectively as part of a pragmatic clinical trial [1,4,19,20]. Inclusion and exclusion criteria for cost data should be established in the protocol, as for clinical outcome measurements, but provision must be made to add extra categories of costs which only become apparent after the trial has commenced [1,20]. Many studies try to collect cost data retrospectively, often after a therapy has shown clinical effectiveness. However, retrospective data collection is seldom fertile, adapted, or exhaustive, and it is subject to bias [18,20].

Examples of the different forms of economic evaluations of CAM

Our systematic review of the CAM economic evaluation literature (presented below) revealed no cost-consequence studies and no cost-benefit analyses. However, we did find examples of a cost-identification study, cost-minimization analysis, cost-effectiveness analysis, and cost-utility analysis. These examples are presented below.

Cost-identification study

Frenkel and Hermoni, 2002 [23], performed a retrospective comparison of medication consumption costs from computerized medication charts three months before and three months after a homeopathic intervention for atopic and allergic disorders. The review was performed on 48 consecutive self-referred patients in one clinic over one year with a diagnosis of an atopic condition who agreed to a classical homeopathic treatment in addition to usual conventional care. Of the 31 medication users (prescription and non-prescription allergy-related medications) before the intervention, 27 reduced their use, two increased their use, and two had their medication level unchanged after the intervention. Of the 17 who had not used medication before the intervention, 4 began medication after the intervention. There was an average drop in 3-month medication costs after homeopathy of $14 (1998 US$) or 54% per person.

Cost-minimization analysis

Herron and Hillis, 2000 [24], retrospectively compared government payments to physicians for 1418 Quebec health insurance enrollees who practiced the Transcendental Meditation (TM) to payments for 1418 randomly selected and matched enrollees who did not. Long term health outcomes were assumed to be equal for both groups. Before starting meditation, the groups were similar in the yearly rate of increase in payments. After starting TM, annual physician payments for the meditation group declined 1 to 2% per year, while those for the non-TM group increased annually over the six year period. The difference in the annual change in payments was statistically significant at a rate between 5 and 13% per year.

Cost-effectiveness analysis

Franzosi et al, 2001 [25], prospectively gathered health and economic outcomes during the 3.5 year follow-up period of a large randomized open-label study (n = 5664) of omega-3 polyunsaturated fatty acids (n-3 PUFA) as secondary prevention for patients with recent myocardial infarction. The perspective was that of a third-party payer; accordingly only direct health care costs (hospital admissions, laboratory and diagnostic tests, and medications) were considered. The incremental number of life-years saved by n-3 PUFA treatment over the 3.5 years (discounted at 5%) was 0.0332 per patient. The incremental cost discounted over the same period was 817€ per patient. Therefore, the incremental cost-effectiveness ratio is 24,603€ (approximately $25, 415 in 1999 US$ [26]) per life-year saved.

Cost-utility analysis

Korthals-de Bos et al, 2003 [27], performed an economic evaluation alongside a randomized controlled trial to compare manual therapy, physiotherapy, and care by a general practitioner for neck pain. The study used the societal perspective and collected direct and indirect costs (including hours of help from family and friends, and hours of absenteeism from work or other activities) through the use of cost diaries kept by patients over one year. Data on each patient's overall health state were gathered at baseline and at one year using a survey instrument called the EuroQoL [16]. The utility of these health states were then calculated by using "society's" preferences for each of those health states. Society's preferences were estimated from a sample of the general population by the developers of the EuroQoL instrument. Using the comparison of manual therapy to general practitioner care, manual therapy had a lower one-year cost ($402, US$) than general practitioner care ($1241). The QALYs were 0.82 for manual therapy and 0.77 for general practitioner care. Since the costs were lower and the QALYs higher for manual therapy as compared to usual care, manual therapy is said to dominate general practitioner care and no cost-utility ratio is calculated.

Methods

The National Center for Complementary and Alternative Medicine (NCCAM) defines CAM as "a group of diverse medical and health care systems, practices, and products that are not presently considered to be part of conventional medicine [28]." We further defined CAM as including only those therapies that could be prescribed (or recommended) and/or performed by a CAM practitioner who does not also have a conventional medical license (eg, doctor of medicine – MD, or doctor of osteopathy – DO). Therefore, we did not include therapies such as chemotherapy regimens nor therapies requiring surgical implantation (such as neuroreflexotherapy [29]) as CAM therapies even though these therapies do appear in searches using the keywords complementary and/or alternative medicine. We also did not include well-accepted vitamin and mineral supplementation therapies such as calcium and vitamin D for osteoporosis, niacin for dyslipidemia, and vitamin B12 and folic acid for homocysteine reduction.

Search strategy

We searched the following electronic databases from January 1999 to October 2004: Medline, AMED, Alt-HealthWatch, and the Complementary and Alternative Medicine Citation Index via NCCAM and the National Library of Medicine (NLM). Searching was restricted to English language journals and human studies with the keywords: complementary medicine or alternative medicine, and costs or cost analysis or cost-benefit or cost-effective or economic analysis or economic evaluation.

We removed duplicates from the search results and selected papers that reported original data on specific CAM therapies from any form of standard economic analysis, analysis of costs, or economic modeling. Studies were then excluded if they were cited in the White and Ernst review [4], or if they were case studies or case series of five or fewer subjects.

Data analysis

The following data were extracted from each of the included studies: full citation information (author(s), date, title, journal, etc), form of economic evaluation (stated or inferred), the therapies being compared and whether the CAM therapies were being used in addition to usual care (complementary) or instead of usual care (alternative), the perspective of analysis (stated or inferred), the study design, the sample size, and summary results.

The studies were categorized as either full economic evaluations (defined as a comparison between two or more alternatives and considering both costs and consequences [10]) or partial economic evaluations (those studies that did not contain a comparison, or only addressed costs). Studies that estimated resource utilization were included as full economic evaluations even if resources were not valued.

We captured data on quality of the full economic evaluations using two approaches. The first approach was to gather from each study the data needed to assess quality according to a 35-item checklist developed by the BMJ Economic Evaluation Working Party [30]. This checklist was developed to improve the quality of published economic evaluations, and was chosen because it is thorough, and entails an objective assessment of whether essential components of an economic evaluation are reported in the article. Therefore, the checklist is mainly a measure of reporting quality and not necessarily of study quality. We also report available results from several other general reviews of economic evaluations of conventional therapies that use this checklist for comparison.

As the purpose of economic evaluations is to inform clinical practice and health policy decisions, the best evaluations are timely and use the best data available at the time [10]. On the other hand, an evaluation is only as good as the data upon which it is based. It has been suggested that the ideal situation for data collection is to collect economic data along side health outcomes in a randomized pragmatic trial [10]. Pragmatic trials offer a compromise between the goals of internal and external validity. To assess study quality, we went beyond White and Ernst's [4] criteria of randomization (to reduce bias by creating comparable groups) and prospective collection of economic outcome data (to ensure all costs are captured) to include two additional indicators of whether a pragmatic (effectiveness or "real world") rather than efficacy trial was conducted. The first is that the comparison group was usual care, and the second was that the study was not blinded and not mandatory – ie, that physicians and patients could react realistically to the therapy [10]. These criteria relate to the external validity or generalizability of the study. Other indicators of a study's generalizability, such as the determination of whether study participants could be assumed to represent a normal case load, were not used as they required detailed knowledge as to the appropriateness of the inclusion and exclusion criteria for each condition studied – a level of expertise not held by the study's authors.

Based on the study quality criteria, we report summarized results of the exemplary studies – ie, those meeting all four study quality criteria. If the health outcomes for one therapy are better than that of its alternative and the economic outcomes are better or equal (lower or equal costs), that therapy is said to dominate (be clearly better than) its alternative. This is also the case if both therapies have equal health outcomes and one has lower costs. In all other cases, the decision maker must elect whether the increase (loss) in health benefits is worth the increase (savings) in cost.

Results

The database search rendered 1765 potential studies to screen. Application of inclusion and exclusion criteria reduced the list to 56 economic evaluations [23-25,27,31-82]. The therapies compared, study design employed, sample size used, and a summary of study results are provided for each study in an appendix [see 1]. The list contains 39 full evaluations and 17 partial evaluations. The evaluations cover a range of CAM therapies applied to a variety of conditions (Table 3). Some therapies, such as acupuncture, homeopathy, and manual therapy, were studied mainly as alternative therapies (ie, as substitutes or alternatives for conventional care). Other therapies, such as guided imagery, were studied as complementary therapies (ie, used in addition to conventional care).

Additional File 1. Descriptions of included studies ordered by complementary and alternative medicine (CAM) modality, form of economic evaluation, and publication date. The appendix contains a table summarizing each of the 56 economic evaluations found in the systematic review. For each evaluation the following are reported: therapies compared, study population, study design and sample size, whether it was a full or partial economic evaluation, form of the evaluation, perspective, and summary results.

Format: DOC Size: 161KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Table 3. Types of complementary and alternative medicine (CAM) therapies studied for various conditions (full/partial economic evaluations)

Reporting quality checklist

Table 4 shows the results of the application of the BMJ 35-item quality checklist [30] to the 39 full economic evaluations. For comparison, Table 4 also contains comparable results from systematic reviews in conventional medicine [6,83,84].

Table 4. Reporting quality of complementary and alternative medicine (CAM) economic evaluations and comparable results of similar reviews in conventional medicine

Study design

These checklist items indicate whether essential components of the study design were reported. About half the studies stated the form of the economic evaluation, however, several were stated incorrectly and only one justified the form chosen. The bulk of the studies presented cost-effectiveness analyses (36 or 92%), five presented cost-utility analyses, and one was a cost-minimization study [24]. Only one-third of studies stated the perspective of the analysis, however, it could be determined from the costs included for all studies. Ten used a societal perspective, and the majority (33 or 85%) used some sort of institutional perspective (eg, health insurance company or hospital). Note that the totals by form and perspective add to more than 39. This is because individual studies can include analyses using more than one form of economic evaluation and can report costs from more than one perspective.

Data collection

These checklist items relate to the presence of information essential to the generalizability of study results. All studies that included health outcomes (ie, all except the one cost minimization study [24]) reported the source of their effectiveness estimates. In 36 of the 38 cases the source was a single study, often the economic evaluation itself. The two other studies were modeling studies [49,78] where reviews were used as the source of effectiveness estimates. Items 12 and 13 are appropriate for cost-utility analyses (where health states are valued in terms of utility) and there were four such studies [27,35,51,78], only one of which gave details on the subjects from whom the valuations were obtained [35]. Productivity changes (items 14 and 15) are appropriate for studies using the societal perspective. Eight studies included the costs of changes in productivity from improvement in back or leg pain [47,82], neck pain [27], migraine [32], anxiety [44], ankylosing spondilitis [51], psoriasis [49], and children's rhinopharyngitis [41]. All but one [49] reported these amounts separate from total costs. However, few discussed the relevance to the study of productivity changes.

About two-thirds of studies reported resource use quantities separate from unit costs, or described the methods used to estimate both quantities and unit costs. Whereas, almost all reported the currency used, only a minority (16 or 41%) reported the currency year. A smaller number reported the details of adjustments for inflation or currency conversion, but this was not often required in studies collecting and reporting data in the same year and currency. Models (one decision tree model [78] and two multiplicative-type or impact [49,57] models) were used in three studies and in all cases the details of the model were given and justified.

Analysis and interpretation of results

All studies stated the time horizon for costs and benefits and most (35 or 90%) reported a time horizon of one year or less. Items 23 through 25 apply only to the four remaining studies with time horizons longer than one year. The discount rate is reported in two of these studies (one with a time horizon of 42 months [25] and the other that included a 12-year projection [78]), but only one justified the choice of discount rate [78]. Two studies gave an explanation for why they did not discount costs and benefits, however, neither needed to – one had a one-year time horizon [35] and the other stated its time horizon as one course of chemotherapy [55]. Five studies performed sensitivity analyses [25,27,35,51,78]. In all cases the approach and the range of variables tested were stated, but the choice of variables to test was only justified in two cases [35,51].

In about one-third of studies there was some comparison of study results to that of other studies. In most cases this was done as a simple statement noting that the results were either similar, or that they were dissimilar and that this might be because of differences in study design. Incremental cost-effectiveness or cost-utility ratios are usually only required when one therapy offers clearly better health outcomes than the other, but at a higher cost. In the 13 studies where this was the case over one-half reported incremental analyses. In most cases the major outcomes of the studies were shown disaggregated, and the study question was answered. We did require that a proper research question be stated (see the answers to item 1) for it to be answered. In all cases, we felt that the conclusions followed the data, but in about one-third of cases the conclusions were not presented with the appropriate caveats. For example, if a study did not explicitly discuss its limitations, it was not included as meeting the last item.

Measures of study quality

Twenty-seven studies (69%) gathered cost data prospectively and 21 (54%) used randomly assigned comparison groups. In 32 studies (82%) the physicians and patients were not blinded to the treatment received and participation was not mandatory (a worksite intervention [57]), and therapies were compared to usual care in 34 (87%) of studies. Fourteen studies [25,27,32,34,35,50,51,53-55,68,74,76,82] met all study quality criteria, and a summary of their results is shown in Table 5.

Table 5. Summary of the results of complementary and alternative medicine (CAM) economic evaluations with exemplary study quality

Discussion

The number of economic evaluations of CAM has increased in recent years, even if we only count full evaluations of alternative therapies. Study quality has also increased, and although reporting quality can use improvement, it is on the whole similar to that seen in economic evaluations of conventional medicine. Nevertheless, there are still too few good quality evaluations to draw many conclusions about the cost-effectiveness of specific CAM therapies for particular conditions.

Potential reasons for paucity

A possible explanation for the paucity of studies is that there may be less of an incentive to perform economic evaluations of CAM. Consumers are already spending a large amount of their disposable income on CAM without formal proof of effectiveness or cost effectiveness. Economic evaluations are typically required for the incorporation of therapies under traditional financing mechanisms and for adjustment of coverage under these mechanisms. Therefore, the market for economic evaluation in CAM may be small due to reduced involvement of third-party payers in CAM financing.

Some CAM practitioners do not see the need for economic evaluations. An interesting study by Kelner et al [85] asked chiropractors, homeopaths, and Reiki practitioners about the need to demonstrate the effectiveness, safety and cost effectiveness of their therapies. The chiropractors agreed that high quality economic evaluations are essential to their practice, but Reiki practitioners could see no reason for this research, and the homeopaths were divided on these issues. There may be good reason why some practitioners resist economic evaluation. If studies are performed that show economic benefit of CAM therapies, third party reimbursement may follow which could reduce practitioner autonomy. Coverage may also be restricted to the standardized forms of botanical medicines, nutritional supplements, or protocols used in the studies [86]. This could dramatically change how CAM is practiced by decreasing the use of multidimensional multicomponent interventions, by institutionalizing care into conventional health care systems, and by limiting the individualization of care.

Relative quality of evaluations

The reporting quality was poor for certain items, but was comparable to the quality found by systematic reviews of economic evaluations in conventional medicine [6,83,84]. Although the BMJ checklist was mostly objective (ie, required the least amount of judgment compared to the other checklists available), a fair amount of interpretation was still required for many items. For example, in our review we interpreted Item 1 as whether the study stated either a specific research question or study objectives in terms of economic and health outcomes. Three-quarters of the full economic evaluations of CAM met this criterion. However, in Jefferson et al, 1998 [83], only 16% of the 43 economic evaluations of conventional medicine reviewed where identified as fulfilling Item 1. It is likely that Jefferson et al took a more restrictive interpretation of this quality criterion.

Several studies have shown that at least some aspects of quality in economic evaluations improve over time [6,87]. Our findings suggest a trend of quality improvement in these studies in CAM. We found that 69% (27 of 39) of the cohort of full economic evaluations collected cost data prospectively as compared to 45% (5 of 11) in White and Ernst's review. Similarly, we found that 54% (21 of 39) of our studies used randomization to create the comparison groups as compared to 45% (5 of 11) in White and Ernst's review.

We found that 14 (36%) of full economic evaluations met all four study quality criteria and were identified as exemplars. However, the evidence from these criteria must be interpreted cautiously; meeting all study quality criteria does not guarantee an adequate study design. Some aspects of what makes a good pragmatic trial could not be judged by what was reported. For example, pragmatic trials enroll patients typical of normal caseload in typical settings with average physicians following them under routine conditions [10]. Judgments as to whether these criteria were met were not possible because of vague reporting. It is also not generally agreed across all health economists that a pragmatic trial, even a well-designed one, can fully represent the real world of health care. These economists advocate for the collection of cost data using an observational study design.

A study may also be of "poor" quality because it applied the CAM therapy inappropriately. This can happen when a study is designed by researchers not familiar with a therapy. In response to this problem researchers and practitioners of several CAM therapies have begun development of standards for research and reporting. Reporting standards do not guarantee that the therapy was used appropriately, but they at least allow determination of what was done. One such set of reporting standards are the STRICTA recommendations for acupuncture [88]. Of the four full evaluations of acupuncture, two (one of which was included in Table 5[32]) met STRICTA reporting standards. As these types of guidelines are not yet available for all CAM therapies, we did not assess whether CAM therapies were applied appropriately in the studies reviewed.

Cost-effectiveness of CAM

The exemplary studies summarized in Table 5 indicate that a number of CAM therapies may be considered cost-effective compared to usual care for a number of conditions: acupuncture for migraine, manual therapy for neck pain, spa therapy for Parkinson's, complementary guided imagery for cardiac surgery patients, complementary relaxation therapy for patients with previous myocardial infarction, complementary self-administered stress management for cancer patients undergoing chemotherapy, complementary pre- and post-operative oral nutritional supplementation for lower gastrointestinal tract surgery, potassium-rich diet (rather than potassium supplements) for postoperative cardiac patients, and biofeedback for patients with "functional" disorders such as irritable bowel syndrome. Acupuncture and homeopathy were both found to be equivalent in terms of effects and costs to usual care for dyspepsia. The attractiveness of the other CAM therapies shown in Table 5 depends on whether the increased health benefits are worth the additional cost, or whether other aspects of the therapy make them attractive, such as patient preference. Only one of the studies summarized in Table 5 reported results of a CAM therapy being dominated by (clearly inferior to) usual care. The use of professionally-administered stress management for cancer patients undergoing chemotherapy was shown to have higher costs, but no additional health benefits over usual care. It is important for CAM that this contradictory evidence is also known for best clinical practice and the efficient use of CAM resources.

On the surface one might expect that therapies that substitute for usual care (alternative medicine) would be much more likely to be cost effective. In this sample of exemplar studies, of the nine study comparisons where CAM therapies were shown to be superior to usual care (better effects and lower costs, similar effects and lower costs, or better effects and similar costs), four were studies of complementary therapies. Therefore, there is evidence that even though complementary therapies are given in addition to usual care, they can improve clinical outcomes without increasing costs.

Issues specific to the economic evaluation of CAM

In many ways the economic evaluation of CAM therapies is similar to that of conventional medicine. However, there are a number of issues specific to CAM that must be considered. These issues can roughly be divided into three groups: those involved with the impact of economic evaluation on CAM in general, those involving the estimation of health outcomes (ie, issues involved with estimating the efficacy or effectiveness of CAM), and those specific to CAM's economic and humanistic outcomes. The first group of issues has already been addressed above under the potential reasons for paucity.

The methodological challenges involved in determining the clinical effectiveness of CAM have been discussed at length in a number of papers. These include the appropriateness of population-based studies when individualized treatments are used and individualized outcomes are expected [89-91], reductionist focus on one therapy for one outcome when that therapy comes from a holistic healing system [92-94], the difficulties with blinding when no appropriate placebo is available [94,95], and the requirement for randomization when most CAM users have strong preferences for their therapy of choice and will often either refuse to be randomized, or will bypass the randomization if it is not to their liking [94]. These challenges are relevant to economic evaluations since they are dependent on effectiveness studies for health outcomes. Also since humanistic and economic outcomes are ideally measured alongside health outcomes in the same trials [1,4,19,20], the challenges above are also relevant to their measurement.

However, there are several additional issues specific to CAM humanistic and economic outcome measurement which must be considered. First, although CAM therapies can be used to treat acute conditions, they are more commonly used to treat chronic disease, to prevent future disease (risk reduction), and to optimize health and well-being. Using CAM for those indications requires that long term studies be performed [96]. However, there are a number of challenges inherent in long term studies in addition to the increase in cost (eg, increased loss to follow-up through patient attrition) [97]. In our systematic review we found only two clinical trials that followed patients prospectively longer than one year: a five-year study of relaxation therapy for patients with a previous myocardial infarction [54], and a 3.5-year study of n-3 PUFA as secondary prevention for patients with previous myocardial infarction [25].

Economic evaluations in CAM must recognize that the process of healthcare itself can be effective for patients. Attributes of the process of using CAM that may have value include patient empowerment, the operationalization of patient preference for a particular type of intervention, the length and process of the consultation, and still having treatment options open when other medical approaches have failed [4,98]. Therefore, economic evaluation of CAM needs to measure and include this value where appropriate.

Optimizing health, maximizing wellness, and enhancing well-being are patient-centered outcomes – ones that by definition require subjective measurement [99]. Economic evaluation of CAM must include appropriate measurement of these humanistic outcomes to account for the full value of CAM therapies. Our systematic review found five studies where humanistic outcomes were captured. The more well-known instruments used to measure health status in these studies included the SF-6D [35] and the EuroQoL (EQ-5D), and health status was translated into quality of life units using population-based preferences [27,51]. Sensitivity of these instruments to the changes in quality of life is an important concern for the evaluation of CAM therapies. Although the use of the EuroQoL for manual therapy for neck pain [27] resulted in a statistically insignificant change in quality of life, two other studies demonstrated small, but statistically significant differences in quality of life using the SF-6D for acupuncture for chronic headache [35], and using the EuroQoL for spa therapy for ankylosing spondylitis [51]. Therefore, it is possible to measure a change in humanistic outcomes for CAM therapies with these instruments.

The collection of economic outcome data is complicated by that fact that in the United States and other countries many CAM therapies are available over the counter and/or are often paid for out-of-pocket. The lack of administrative claims data on CAM therapies in countries where these costs are not covered or reimbursed means that cost studies require primary data collection (eg, patient self-report instruments) [100]. In their study on manual therapy for neck pain, Korthals-de Bos and colleagues used weekly cost diaries to obtain economic outcomes [27]. The second, related challenge is that many over-the-counter products, such as certain botanical medicines and nutritional supplements, are not standardized and of inconsistent quality. Standardization and quality will affect both the costs of the therapy and its outcomes. Finally, since there is often no provider "gatekeeper" controlling access to CAM therapies, monitoring of patient use can be complicated and labor intensive.

Recommendations for future research

Despite the challenges described for economic evaluations of CAM therapies, these studies ought to be done. Every planned trial of CAM therapies should at least consider the feasibility of including an evaluation of economic impacts. Observational studies should also include these data, and as information accumulates regarding economic impacts, these costs and cost savings can be estimated more accurately. Although in the ideal every cost category shown in Table 2 should be measured and outcomes should include a measure of quality-adjusted life-years, the estimation of direct medical costs and savings associated with the therapy (eg, practitioner fees, lab fees, and the cost of herbs or other supplements prescribed) will be fairly straightforward for most studies, and the planned primary outcome of the study can serve as the measure of effects to determine cost effectiveness. Even if the clinical outcomes of a CAM therapy are similar or slightly less beneficial than those of usual care, a lower cost of care can still make these therapies attractive to decision makers. However, if no cost data are available, even highly effective therapies can be easily overlooked.

Limitations

The limitations of this study are similar to those of the other reviews. First, the reader was not blinded to journals and article authors, which may have influenced results. Second, our measures of study quality depend on the information reported in an article, and no attempt was made to judge the merits of clinical or modeling assumptions made in the analyses. Third, only one reader read all the papers and extracted all the data. This may have lead to inaccurate reporting of results, and/or a biased interpretation of study quality. To maximize accuracy, data extraction was performed at least twice for each paper with several months break between extractions. Also, the approach and assumptions used to determine study quality were discussed at length with the other authors. These discussions led to a homogeneous approach being taken to both the application of the reporting quality criteria and the definition as to what constitutes an economic evaluation.

Conclusion

As health care costs continue to rise, decision makers must allocate their increasingly scarce resources toward therapies which offer the most benefit per unit of cost. Economic evaluations inform evidence-based clinical practice and health policy. To be considered by these decision makers, CAM therapies and their outcomes must be known and compared to conventional approaches. However, CAM practitioners must themselves decide whether the cost of performing these studies is worth the potential impacts to their profession of being considered in managed care. Nevertheless, these evaluations will be done and they will be better done with practitioner involvement. Whereas the number and quality of these studies has increased in recent years and more CAM therapies have been shown to be good value, there are still not enough studies to measure the cost effectiveness of the majority of CAM. If CAM providers wish to increase the provision of therapies to improve population health, they must report the potential outcomes of CAM therapies widely and well.

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

PH had the main responsibility for the manuscript and for bringing together the concepts of CAM and economics. PH also read and evaluated the quality of all papers included in the review. BC ensured that the health economic concepts were presented appropriately. PH and OC conceived the idea for the paper. OC contributed clinical and methodological insights. All authors read and approved the final manuscript.

Acknowledgements

This project was supported by grant #T32 AT01287-03 from the National Center for Complementary and Alternative Medicine.

References

  1. White AR, Resch KL, Ernst E: Methods of economic evaluation in complementary medicine.

    Forsch Komplementarmed 1996, 3:196-203. OpenURL

  2. MacLennan AH, Wilson DH, Taylor AW: The escalating cost and prevalence of alternative medicine.

    Preventive Medicine 2002, 35:166-173. Publisher Full Text OpenURL

  3. Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, Kessler RC: Trends in alternative medicine use in the United States, 1990-1997.

    JAMA 1998, 280:1569-1515. Publisher Full Text OpenURL

  4. White AR, Ernst E: Economic analysis of complementary medicine: a systematic review.

    Complementary Therapies in Medicine 2000, 8:111-118. Publisher Full Text OpenURL

  5. Pelletier KR, Astin JA: Integration and reimbursement of complementary and alternative medicine by managed care and insurance providers: 2000 update and cohort analysis.

    Alternative Therapies 2002, 8:38-48. OpenURL

  6. Neumann PJ: Evidence-based and value-based formulary guidelines.

    Health Affairs 2004, 23:124-134. Publisher Full Text OpenURL

  7. Sullivan SD, Lyles A, Luce B, Grigar J: AMCP guidance for submission of clinical and economic evaluation data to support formulary listing in US health plans and pharmacy benefits management organizations.

    Journal of Managed Care Pharmacy 2001, 7:272-282. OpenURL

  8. Claxton K, Neumann PJ, Araki S, Weinstein MC: Bayesian value-of-information analysis: an application to a policy model of Alzheimer's disease.

    International Journal of Technology Assessment in Health Care 2001, 17:38-55. Publisher Full Text OpenURL

  9. Claxton K, Posnett J: An economic approach to clinical trial design and research priority-setting.

    Medical Economics 1996, 5:513-524. OpenURL

  10. Drummond MF, O'Brien B, Stoddart GL, Torrance GW: Methods for the economic evaluation of health care programmes. Second edition. Oxford, Oxford University Press; 1997:305.

  11. Gunter MJ: The role of the ECHO model in outcomes research and clinical practice improvement.

    American Journal of Managed Care 1999, 5:S217-S224. OpenURL

  12. Ratcliffe J, van Haselen RA, Buxton M, Hardy K, Colehan J, Partridge M: Assessing patients' preferences for characteristics associated with homeopathic and conventional treatment of asthma.

    Thorax 2002, 57:503-508. Publisher Full Text OpenURL

  13. Johannesson MJB: Economic evaluation in health care: is there a role for cost-benefit analysis?

    Health Policy 1991, 17:1-23. Publisher Full Text OpenURL

  14. O'Brien BGA: When do the "dollars" make sense? Toward a conceptual framework for contingent valuation studies in health care.

    Medical Decision Making 1996, 16:288-299. OpenURL

  15. Chapman RH, Berger M, Weinstein MC, Weeks JC, Goldie S, Neumann PJ: When does quality-adjusting life-years matter in cost-effectiveness analysis?

    Health Economics 2004, 13:429-436. Publisher Full Text OpenURL

  16. Dolan P: Modelling valuations for EuroQoL health states.

    Medical Care 1997, 35:1095-1108. Publisher Full Text OpenURL

  17. Coons SJ, Rao S, Keininger DL, Hays RD: A comparative review of generic quality-of-life instruments.

    Pharmacoeconomics 2000, 17:13-35. OpenURL

  18. Chaufferin G: Improving the evaluation of homeopathy: economic considerations and impact on health.

    British Homeopathic Journal 2000, 89:S27-S30. Publisher Full Text OpenURL

  19. Drummond MFDL: Economic analysis alongside clinical trials: revisiting methodological issues.

    International Journal of Technology Assessment in Health Care 1991, 7:561-573. OpenURL

  20. White A: Economic evaluation in acupuncture.

    Acupuncture in Medicine 1996, 14:109-113. OpenURL

  21. Ashcroft DM, Po ALW: Herbal remedies: issues in licensing and economic evaluation.

    Pharmacoeconomics 1999, 16:321-328. OpenURL

  22. Naglak M, Mitchell DC, Kris-Etherton P, Harkness W, Pearson T: What to consider when conducting a cost-effectiveness analysis in a clinical setting.

    Journal of the American Dietetic Association 1998, 98:1149-1154. Publisher Full Text OpenURL

  23. Frenkel M, Hermoni D: Effects of homeopathic intervention on medication consumption in atopic and allergic disorders.

    Alternative Therapies 2002, 8:76-79. Publisher Full Text OpenURL

  24. Herron RE, Hillis SL: The impact of the transcendental meditation program on government payments to physicians in Quebec: an update.

    American Journal of Health Promotion 2000, 14:284-291. OpenURL

  25. Franzosi MG, Brunetti M, Marchioli R, Marfisi RM, Tognoni G, Valagussa F, GISSI-Prevenzione Investigators: Cost-effectiveness analysis of n-3 polyunsaturated fatty acides (PUFA) after myocardial infarction: results from Gruppo Italiano per lo Studio della Sopravvivenza nell'Infarto (GISSI) - Prevenzione Trial.

    Pharmacoeconomics 2001, 19:411-420. OpenURL

  26. OANDA Corporation: FXConverter - 164 Currency Converter. [http://www.oanda.com/convert/fxhistory] webcite

  27. Korthals-de Bos IBC, Hoving JL, van Tulder MW, Rutten-van Molken MPMH, Ader HJ, de Vet HCW, Koes BW, Vondeling H, Bouter LM: Cost effectiveness of physiotherapy, manual therapy, and general practitioner care for neck pain: economic evaluation alongside a randomised controlled trial.

    BMJ 2003, 326:911-917. OpenURL

  28. National Center for Complementary and Alternative Medicine: What is complementary and alternative medicine (CAM)? [http://nccam.nih.gov/health/whatiscam] webcite

  29. Kovacs FM, Llobera J, Abraira V, Lazaro P, Pozo F, Kleinbaum D, KAP Group: Effectiveness and cost-effectiveness analysis of neuroreflexotherapy for subacute and chronic low back pain in routine general practice: a cluster randomized, controlled trial.

    Spine 2002, 27:1149-1159. Publisher Full Text OpenURL

  30. Drummond MF, Jefferson TO, BMJ Economic Evaluation Working Party: Guidelines for authors and peer reviewers of economic submissions to the BMJ.

    BMJ 1996, 313:275-283. OpenURL

  31. Yeh GY, Phillips RS, Davis RB, Eisenberg DM, Cherkin DC: Visit time as a framework for reimbursement: time spent with chiropractors and acupuncturists.

    Alternative Therapies in Health & Medicine 2003, 9:88-94. OpenURL

  32. Liguori A, Petti F, Bangrazi A, Camaioni D, Guccione G, Pitari GM, Bianchi A, Nicoletti WE: Comparison of pharmacological treatment versus acupuncture treatment for migraine without aura--analysis of socio-medical parameters.

    Journal of Traditional Chinese Medicine 2000, 20:231-240. OpenURL

  33. Cherkin DC, Eisenberg D, Sherman KJ, Barlow W, Kaptchuk TJ, Street J, Deyo RA: Randomized trial comparing traditional Chinese medical acupuncture, therapeutic massage, and self-care education for chronic low back pain.

    Archives of Internal Medicine 2001, 161:1081-1088. Publisher Full Text OpenURL

  34. Paterson C, Ewings P, Brazier JE, Britten N: Treating dyspepsia with acupuncture and homeopathy: reflections on a pilot study by researchers, practitioners and participants.

    Complementary Therapies in Medicine 2003, 11:78-84. Publisher Full Text OpenURL

  35. Wonderling D, Vickers AJ, Grieve R, McCarney R: Cost effectiveness analysis of a randomised trial of acupuncture for chronic headache in primary care.

    BMJ 2004, 1-5. OpenURL

  36. van Haselen RA, Graves N, Dahiha S: The costs of treating rheumatoid arthritis patients with complementary medicine: exploring the issue.

    Complementary Therapies in Medicine 1999, 7:217-221. Publisher Full Text OpenURL

  37. Frei H, Thurneysen A: Homeopathy in acute otitis media in children: treatment effect or spontaneous resolution?

    British Homeopathic Journal 2001, 90:180-182. OpenURL

  38. Jain A: Does homeopathy reduce the cost of conventional drug prescribing? A study of comparative prescribing costs in general practice.

    Homeopathy: the Journal of the Faculty of Homeopathy 2003, 92:71-76. Publisher Full Text OpenURL

  39. Trichard M, Lamure E, Chaufferin G: Study of the practice of homeopathic general practitioners in France.

    Homeopathy: the Journal of the Faculty of Homeopathy 2003, 92:135-139. Publisher Full Text OpenURL

  40. Becker-Witt C, Keil T, Roll S, Menke D, Vance W, Wegscheider K, Willich SN: Effectiveness and costs of homeopathy compared to conventional medicine - a prospective multicenter cohort study: April 3-4; London. Volume 1. The Royal London Homoeopathic Hospital; 2003::57-60.

  41. Trichard M, Chaufferin G: Cost-effectiveness study of treatment of recurrent acute rhinopharyngitis in 18 month-old to 4-year-old children by homeopathic general practitioners: April 3-4; London. Volume 1. The Royal London Homoeopathic Hospital; 2003::43-49.

  42. Slade K, Chohon BP, Barker PJ: Evaluation of a GP practice based homeopathy service.

    Homeopathy 2004, 93:67-70. Publisher Full Text OpenURL

  43. Van Wassenhoven M, Ives G: An observational study of patients seeking homeopathic treatment.

    Homeopathy 2004, 93:3-11. Publisher Full Text OpenURL

  44. Trichard M, Chaufferin G: Cost-effectiveness study of the treatment of anxiety disorders by homeopathic general practitioners: April 3-4; London. Volume 1. The Royal London Homoeopathic Hospital; 2003::62-63.

  45. Hess JA, Mootz RD: Comparison of work and time estimates by chiropractic physicians with those of medical and osteopathic providers.

    Journal of Manipulative and Physiological Therapeutics 1999, 22:280-291. OpenURL

  46. Burton AK, Tillotson KM, Cleary J: Single-blind randomised controlled trial of chemonucleolysis and manipulation in the treatment of symptomatic lumbar disc herniation.

    European Spine Journal 2000, 9:202-207. Publisher Full Text OpenURL

  47. Hemmila HM: Quality of life and cost of care of back pain patients in Finnish general practice.

    Spine 2002, 27:647-653. Publisher Full Text OpenURL

  48. Stano M, Haas M, Goldberg B, Traub PM, Nyiendo J: Chiropractic and medical care costs of low back care: results from a practice-based observational study.

    American Journal of Managed Care 2002, 8:802-809. OpenURL

  49. Shani J, Harari M, Hristakieva E, Seidl V, Bar-Giyora J: Dead-Sea climatotherapy versus other modalities of treatment for psoriasis: comparative cost-effectiveness.

    International Journal of Dermatology 1999, 38:252-262. Publisher Full Text OpenURL

  50. Brefel-Courbon C, Desboeuf K, Thalamas C, Galitzky M, Senard JM, Rascol O, Montastruc JL: Clinical and economic analysis of spa therapy in Parkinson's disease.

    Movement Disorders 2003, 18:578-584. Publisher Full Text OpenURL

  51. Van Tubergen A, Boonen A, Landewe R, Rutten-van Molken M, Van Der Heijde D, Hidding A, Van der Linden S: Cost effectiveness of a combined spa-exercise therapy in ankylosing spondylitis: a randomized controlled trial.

    Arthritis & Rheumatism 2002, 47:459-467. Publisher Full Text OpenURL

  52. Roth B, Stanley TW: Mindfulness-based stress reduction and healthcare utilization in the inner city: preliminary findings.

    Alternative Therapies in Health & Medicine 2002, 8:60-62, 64-66. OpenURL

  53. Tusek DL, Cwynar R, Cosgrove DM: Effect of guided imagery on length of stay, pain and anxiety in cardiac surgery.

    Journal of Cardiovascular Management 1999, 10:22-28. OpenURL

  54. van Dixhoorn JJ, Duivenvoorden HJ: Effect of relaxation therapy on cardiac events after myocardial infarction: a 5-year follow-up study.

    Journal of Cardiopulmonary Rehabilitation 1999, 19:178-185. Publisher Full Text OpenURL

  55. Jacobsen PB, Meade CD, Stein KD, Chirikos TN, Small BJ, Ruckdeschel JC: Efficacy and costs of two forms of stress management training for cancer patients undergoing chemotherapy.

    Journal of Clinical Oncology 2002, 20:2851-2862. Publisher Full Text OpenURL

  56. Mehl-Madrona L: Complementary medicine treatment of uterine fibroids: a pilot study.

    Alternative Therapies in Health & Medicine 2002, 8:34-36, 38-40, 42, 44-46. OpenURL

  57. Bittman B, Bruhn KT, Stevens C, Westengard J, Umbach PO: Recreational music-making: a cost-effective group interdisciplinary strategy for reducing burnout and improving mood states in long-term care workers.

    Advances in Mind-Body Medicine 2003, 19:4-15. OpenURL

  58. Lang EV, Rosen MP: Cost analysis of adjunct hypnosis with sedation during outpatient interventional radiologic procedures.

    Radiology 2002, 222:375-382. OpenURL

  59. Meurisse M, Defechereux T, Hamoir E, Maweja S, Marchettini P, Gollogly L, Degauque C, Joris J, Faymonville ME: Hypnosis with conscious sedation instead of general anaesthesia? Applications in cervical endocrine surgery.

    Acta Chirurgica Belgica 1999, 99:151-158. OpenURL

  60. Defechereux T, Meurisse M, Hamoir E, Gollogly L, Joris J, Faymonville ME: Hypnoanesthesia for endocrine cervical surgery: a statement of practice.

    The Journal of Alternative & Complementary Medicine 1999, 5:509-520. OpenURL

  61. Chrubasik S, Kunzel O, Black A, Conradt C, Kerschbaumer F: Potential economic impact of using a proprietary willow bark extract in outpatient treatment of low back pain: an open non-randomized study.

    Phytomedicine 2001, 8:241-251. OpenURL

  62. Stothers L: A randomized trial to evaluate effectiveness and cost-effectiveness of naturopathic cranberry products as prophylaxis against urinary tract infection in women.

    Canadian Journal of Urology 2002, 9:1558-1562. OpenURL

  63. Shabert JK, Winslow C, Lacey JM, Wilmore DW: Glutamine-antioxidant supplementation increases body cell mass in AIDS patients with weight loss: a randomized, double-blind controlled trial.

    Nutrition 2000, 16:71-73. Publisher Full Text OpenURL

  64. Blakeley JA, Ribeiro V: A survey of self-medication practices and perceived effectiveness of glucosamine products among older adults.

    Complementary Therapies in Medicine 2002, 10:154-160. Publisher Full Text OpenURL

  65. Chang CW, Chu G, Hinz BJ, Greve MD: Current use of dietary supplementation in patients with age-related macular degeneration.

    Canadian Journal of Ophthalmology 2003, 38:27-32. OpenURL

  66. Senkal M, Zumtobel V, Bauer KH, Marpe B, Wolfram G, Frei A, Eickhoff U, Kemen M: Outcome and cost-effectiveness of perioperative enteral immunonutrition in patients undergoing elective upper gastrointestinal tract surgery: a prospective randomized study.

    Archives of Surgery 1999, 134:1309-1316. Publisher Full Text OpenURL

  67. Zhou YP, Jiang ZM, Sun YH, Wang XR, Ma EL, Wilmore D: The effect of supplemental enteral glutamine on plasma levels, gut function, and outcome in severe burns: a randomized, double-blind, controlled clinical trial.

    Jpen: Journal of Parenteral & Enteral Nutrition 2003, 27:241-245. OpenURL

  68. Smedley F, Bowling T, James M, Stokes E, Goodger C, O'Connor O, Oldale C, Jones P, Silk D: Randomized clinical trial of the effects of preoperative and postoperative oral nutritional supplements on the clinical course and cost of care.

    British Journal of Surgery 2004, 91:983-990. Publisher Full Text OpenURL

  69. Gilbert DL, Pyzik PL, Vining EP, Freeman JM: Medication cost reduction in children on the ketogenic diet: data from a prospective study.

    Journal of Child Neurology 1999, 14:469-471. OpenURL

  70. MacCracken KA, Scalisi JC: Development and evaluation of a ketogenic diet program.

    Journal of the American Dietetic Association 1999, 99:1554-1558. Publisher Full Text OpenURL

  71. Mandel A, Ballew M, Pina-Garza JE, Stalmasek V, Clemens LH: Medical costs are reduced when children with intractable epilepsy are successfully treated with the ketogenic diet.

    Journal of the American Dietetic Association 2002, 102:396-398. Publisher Full Text OpenURL

  72. Goldhamer AC: Initial cost of care results in medically supervised water-only fasting for treating high blood pressure and diabetes.

    Journal of Alternative & Complementary Medicine 2002, 8:696-697. Publisher Full Text OpenURL

  73. Raynor HA, Kilanowski CK, Esterlis I, Epstein LH: A cost-analysis of adopting a healthful diet in a family-based obesity treatment program.

    Journal of the American Dietetic Association 2002, 102:645-656. Publisher Full Text OpenURL

  74. Norris W, Kunzelman KS, Bussell S, Rohweder L, Cochran RP: Potassium supplementation, diet vs pills: a randomized trial in postoperative cardiac surgery patients.

    Chest 2004, 125:404-409. Publisher Full Text OpenURL

  75. Mellgren A, Jensen LL, Zetterstrom JP, Wong WD, Hofmeister JH, Lowry AC: Long-term cost of fecal incontinence secondary to obstetric injuries.

    Diseases of the Colon & Rectum 1999, 42:857-867. OpenURL

  76. Ryan M, Gervitz R: Biofeedback-based psychophysiological treatment in a primary care setting: an initial feasibility study.

    Applied Psycholphysiology & Biofeedback 2004, 29:79-93. Publisher Full Text OpenURL

  77. Abidia A, Laden G, Kuhan G, Johnson BF, Wilkinson AR, Renwick PM, Masson EA, McCollum PT: The role of hyperbaric oxygen therapy in ischaemic diabetic lower extremity ulcers: a double-blind randomised-controlled trial.

    European Journal of Vascular and Endovascular Surgery 2003, 25:513-518. Publisher Full Text OpenURL

  78. Guo S, Counte MA, Gillespie KN, Schmitz H: Cost-effectiveness of adjunctive hyperbaric oxygen in the treatment of diabetic ulcers.

    International Journal of Technology Assessment in Health Care 2003, 19:731-737. Publisher Full Text OpenURL

  79. Burns EE, Blamey C, Ersser SJ, Barnetson L, Lloyd AJ: An investigation into the use of aromatherapy in intrapartum midwifery practice.

    Journal of Alternative & Complementary Medicine 2000, 6:141-147. OpenURL

  80. Dobson A, DaVanzo J, Consunji M, Gilani J, McMahon P, Sen N, Preston B: A study of the cost effects of daily multivitamins for older adults. Falls Church, VA, The Lewin Group, Inc; 2004:30.

  81. Kail K: Clinical outcomes of a diagnostic and treatment protocol in allergy/sensitivity patients.

    Alternative Medicine Review 2001, 6:188-202. OpenURL

  82. Larsen K, Weidich F, Leboeuf-Yde C: Can custom-made biomechanic shoe orthoses prevent problems in the back and lower extremities? A randomized, controlled intervention trial of 146 military conscripts.

    Journal of Manipulative and Physiological Therapeutics 2002, 25:326-331. Publisher Full Text OpenURL

  83. Jefferson TO, Smith R, Yee Y, Drummond MF, Pratt M, Gale R: Evaluating the BMJ guidelines for economic submissions.

    JAMA 1998, 280:275-277. Publisher Full Text OpenURL

  84. Gerard K, Seymour J, Smoker I: A tool to improve quality of reporting in published economic analyses.

    International Journal of Technology Assessment in Health Care 2000, 16:100-110. Publisher Full Text OpenURL

  85. Kelner MJ, Boon H, Wellman B, Welsh S: Complementary and alternative groups contemplate the need for effectiveness, safety and cost-effectiveness research.

    Complementary Therapies in Medicine 2002, 10:235-239. Publisher Full Text OpenURL

  86. McKernan SL: Why naturopathic medicine should not embrace the managed care model in medicine.

    Journal of Naturopathic Medicine 1997, 7:65-66. OpenURL

  87. Jefferson TO, Demicheli V, Vale L: Quality of systematic reviews of economic evaluations in health care.

    JAMA 2002, 287:2809-2812. Publisher Full Text OpenURL

  88. MacPherson H, White A, Cummings M, Jobst KA, Rose K, Niemtzow RC: Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations.

    Journal of Alternative & Complementary Medicine 2002, 8:85-89. Publisher Full Text OpenURL

  89. Tonelli MR, Callahan TC: Why alternative medicine cannot be evidence-based.

    Academic Medicine 2001, 76:1213-1220. OpenURL

  90. Caspi O, Bell IR: One size does not fit all: aptitude x treatment interaction (ATI) as a conceptual framework for CAM outcome research. Part I - what is ATI research?

    Journal of Alternative & Complementary Medicine 2004, 10:580-586. Publisher Full Text OpenURL

  91. Caspi O, Bell IR: One size does not fit all: aptitude x treatment interaction (ATI) as a conceptual framework for CAM outcome research. Part II - research designs and their applications.

    Journal of Alternative & Complementary Medicine 2004, 10:698-705. Publisher Full Text OpenURL

  92. Mason S, Tovey P, Long AF: Evaluating complementary medicine: methodological challenges of randomised controlled trials.

    BMJ 2002, 325:832-834. Publisher Full Text OpenURL

  93. Cassidy CM: Social science theory and methods in the study of alternative and complementary medicine.

    The Journal of Alternative & Complementary Medicine 1995, 2:19-40. OpenURL

  94. Nahin RL, Strauss SE: Research into complementary and alternative medicine: problems and potential.

    BMJ 2001, 322:161-164. Publisher Full Text OpenURL

  95. Caspi O, Millen C, Sechrest L: Integrity and research: introducing the concept of dual blindness. How blind are double-blind clinical trials in alternative medicine?

    Journal of Alternative & Complementary Medicine 2000, 6:493-498. OpenURL

  96. Levin JS, Glass TA, Kushi LH, Schuck JR, Steele L, Jonas WB: Quantitative methods in research on complementary and alternative medicine: a methodological manifesto.

    Medical Care 1997, 35:1079-1094. Publisher Full Text OpenURL

  97. Pincus T: Limitations of randomized clinical trials in chronic diseases: explanations and recommendations.

    Advances 2002, 18:14-21. OpenURL

  98. Meenan R: Developing appropriate measures of the benefits of complementary and alternative medicine.

    Journal of Health Services Research and Policy 2001, 6:38-43. Publisher Full Text OpenURL

  99. Sullivan M: The new subjective medicine: taking the patient's point of view on health care and health.

    Social Science and Medicine 2002, 56:1595-1604. OpenURL

  100. Goossens MEJB, Rutten-van Molken MPMH, Vlaeyen JWS, van der Linden SMJP: The cost diary: a method to measure direct and indirect costs in cost-effectiveness research.

    Journal of Clinical Epidemiology 2000, 53:688-695. Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6882/5/11/prepub