Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Debate

Researching complementary and alternative treatments – the gatekeepers are not at home

Vinjar Fønnebø1*, Sameline Grimsgaard2, Harald Walach3, Cheryl Ritenbaugh4, Arne Johan Norheim1, Hugh MacPherson5, George Lewith6, Laila Launsø1, Mary Koithan47, Torkel Falkenberg8, Heather Boon9 and Mikel Aickin7

Author affiliations

1 National Research Center in Complementary and Alternative Medicine, Faculty of Medicine, University of Tromsø, N-9037 TROMSØ, Norway

2 Clinical Research Center, University Hospital of North Norway, N-9038 TROMSØ, Norway

3 School of Social Sciences & Samueli Institute, University of Northampton, Boughton Green Rd, Northampton NN2 7AL, UK

4 Department of Family & Community Medicine, University of Arizona, 1450 North Cherry Avenue, Tucson, AZ 85719, USA

5 Department of Health Sciences, University of York, Heslington, York YO10 5DD, UK

6 Complementary Medicine Research Unit, Primary Medical Care, Aldermoor Health Centre, Aldermoor Close, Southampton SO16 5ST, UK

7 Program in Integrative Medicine, University of Arizona, PO Box 245153, Tucson, AZ 85724-5153, USA

8 Center for Studies of Complementary Medicine, Department of Nursing and the division of International Health (IHCAR), Department of Public Health Sciences, Karolinska Institutet, Alfred Nobels Allé 23, S-141 83 Huddinge, Sweden

9 Leslie Dan Faculty of Pharmacy, University of Toronto, 19 Russell Street Toronto, Ontario, M5S 2S2, Canada

For all author emails, please log on.

Citation and License

BMC Medical Research Methodology 2007, 7:7  doi:10.1186/1471-2288-7-7

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2288/7/7


Received:7 September 2006
Accepted:11 February 2007
Published:11 February 2007

© 2007 Fønnebø et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

To explore the strengths and weaknesses of conventional biomedical research strategies and methods as applied to complementary and alternative medicine (CAM), and to suggest a new research framework for assessing these treatment modalities.

Discussion

There appears to be a gap between published studies showing little or no efficacy of CAM, and reports of substantial clinical benefit from patients and CAM practitioners. This "gap" might be partially due to the current focus on placebo-controlled randomized trials, which are appropriately designed to answer questions about the efficacy and safety of pharmaceutical agents. In an attempt to fit this assessment strategy, complex CAM treatment approaches have been dissected into standardized and often simplified treatment methods, and outcomes have been limited.

Unlike conventional medicine, CAM has no regulatory or financial gatekeeper controlling their therapeutic "agents" before they are marketed. Treatments may thus be in widespread use before researchers know of their existence. In addition, the treatments are often provided as an integrated 'whole system' of care, without careful consideration of the safety issue.

We propose a five-phase strategy for assessing CAM built on the acknowledgement of the inherent, unique aspects of CAM treatments and their regulatory status in most Western countries. These phases comprise:

1. Context, paradigms, philosophical understanding and utilization

2. Safety status

3. Comparative effectiveness.

4. Component efficacy

5. Biological mechanisms.

Summary

Using the proposed strategy will generate evidence relevant to clinical practice, while acknowledging the absence of regulatory and financial gatekeepers for CAM. It will also emphasize the important but subtle differences between CAM and conventional medical practice.

Background

The use of complementary and alternative medicine (CAM) has increased considerably in Western industrialized nations over the last 25 years. In the USA, the expenditure is approximately $30 billion per annum, surpassing current out-of-pocket expenditures for conventional treatments by primary care physicians [1,2]. CAM treatment modalities include a variety of approaches (i.e., acupuncture, homeopathy, herbal medicine, massage, reflexology, Reiki healing etc.), many of them based on theories that differ markedly from conventional Western biomedicine. The assessment and management of illness often includes a more detailed interest in patients' wellbeing, as well as recommendations concerning lifestyle and their quality of life as a whole [3].

Patients are selective in their choice of CAM treatments [4], usually abstaining from exclusive use of CAM in acute life-threatening conditions. Rather, CAM is mainly used in addition to conventional care for chronic and some acute health conditions, for disease prevention and for maintaining wellness. For example, more than half of all breast cancer patients use some form of CAM complementary to conventional medicine, most often with a palliative intent [5,6].

Many physicians do not understand why large segments of their patients use CAM when research generally has failed to provide decisive evidence of efficacy. Patients seem to make these treatment choices based on the qualities of the provider, desire for "individualized" treatments, and their perception of overall effectiveness rather than efficacy [7-11].

There is apparently a "gap" between the results of randomized controlled trials (RCT) showing little or no effect and the widespread use and reports of beneficial outcomes of CAM treatment [12,13]. If we were to assume that patients are not completely misguided, then we would need to look closely at the research strategies utilized in the CAM field and try to understand the reasons for the gap.

The purpose of this paper is therefore to

1. Explore the strengths and weaknesses of conventional biomedical research strategies and methods as applied to CAM

2. Suggest a new research framework for assessing these treatment modalities.

Discussion

Gatekeeping and regulating the use of CAM interventions

A majority of CAM research to date has used the research strategy employed and developed by clinical pharmacologists to document, in a prescribed sequential pattern, the quality, dose, safety, efficacy, and eventual effectiveness of a drug prior to its general release. In this model, governmental regulatory offices (e.g., USA: FDA, EU: EMEA) act as gatekeepers. Health insurers, including governmental single-payer funders, only reimburse for drugs meeting the appropriate criteria. An important principle in this research-regulatory-utilization model is that research determines which drugs are approved for generalized clinical use and are paid for by health insurers. There is also a growing network of international Health Technology Assessment Boards that seek consensus about the use of medical procedures, and some suggest this approach should be extended to the area of diagnostic tests [14].

Previously, CAM researchers have largely assumed that these same pharmacological research methods and regulatory-reimbursement models can be followed in the evaluation of CAM. Much to the frustration of many clinical researchers, the availability of CAM treatments which are affordable with little or no reimbursement, however, does not seem to be amenable to the same rules as those we seek to apply to conventional medicine. Even if studies show that a CAM treatment has no effect, it does not necessarily disappear from the marketplace. We suggest that this is because the CAM market has had no statutory body (gatekeeper) that ensured the quality, safety, efficacy and effectiveness of CAM treatments

    before
they appear on the "market". Further, since few health insurers reimburse for CAM treatments, there are few financial gatekeepers. Millions of patients in developed countries have experienced the effect of CAM treatments provided in contrast to, or more likely in addition to, the care they have received in conventional medicine [15-19]. The situation is thereby characterized by widespread patient experience of treatment outcomes combined with little research and no regulatory or financial gate-keeping activity.

Why is a different research strategy needed?

In conventional medicine we rarely question the efficacy or effectiveness of the overall therapeutic approach to the treatment of sick people. We take for granted that seeking advice from a health care professional is a good way of dealing with diverse health problems. Research in conventional medicine therefore focuses on choosing the best tools for health professionals to use. These tools include drugs, diagnostic methods, and surgical procedures, and employ a methodology often resulting in a one-size-fits-all therapeutic prescription. Conventional research may, however, have overlooked that the clinical effect of most therapies are overestimated when studied under optimal circumstances on susceptible, cooperative patients. Despite this, the randomized controlled trial is an important method when making decisions about tools to include in the toolbox of conventional medicine.

Because "conventional" researchers have done most of the CAM research, it is not surprising that CAM research has traveled down the same path as conventional medicine. It has tested the specific efficacy of what conventional researchers believe to be the active components of a therapy, often discounting synergistic effects. In addition to providing a predominantly individualistic treatment approach, many CAM therapists hold that CAM treatments cannot be split up into parts that can be investigated separately. They argue that the total effect adds up to more than the sum of its parts.

If CAM is to be evaluated comprehensively, one needs to extend the research focus to all aspects of the treatment approach [20]. To study only the specific effect of needling in acupuncture isolated from other interventions initiated by the acupuncturist, or the effect of a single, isolated homeopathic remedy separated from other aspects of homeopathic practice, is to neglect other potentially important components of these interventions [21-25]. Furthermore, clinical research teams should only venture into this area with a thorough contextual and philosophical understanding of the CAM treatment paradigm and its clinical use. Plunging into studies of efficacy that involve isolated detailed components of a treatment approach without thoroughly understanding its context is destined to failure and irrelevance no matter what the results show. These issues point to the need for a different, and more complex, research strategy for the CAM field.

A suggested research strategy

The most important unique characteristics of CAM are the absence of gatekeepers and the complexity of individually tailored treatments. Our proposed strategy does not contain new methodological elements, but organizes existing elements in a way that is tailored to pragmatic clinical practice (Figure 1). The strategy is not meant as a strictly chronological sequence defining when specific research phases should occur, but as a framework to guide CAM research, illustrating the necessary building blocks required for a rigorous evidence base.

thumbnailFigure 1. Research strategies in drug trials and CAM (proposed). Phases that contrast the proposed phased research strategy in CAM (dark arrows) with that conventionally used in drug trials (light arrows).

Although we have developed this strategy within a CAM context, it is appropriate for any treatment approach that has developed in and from clinical practice, and whose specific treatment tools are unregulated. In conventional medicine this might include nursing, health psychology, counseling and some aspects of general practice. These areas struggle, as does CAM, to establish a rigorous research framework around which their research efforts can be organized [26-28].

Phase1: Context, paradigms, philosophical understanding and utilization (What is going on?)

The cornerstone of the suggested research strategy is to understand the process and assumptions within a particular therapy, often using an inductive research approach [29-34]. Researchers need to understand what the treatment procedure is, how many variations there are, what philosophical foundations underlie it, its ideas about health and disease, its contextual framework and key treatment components. They also need to know how many and what segments of the patient population use it, and for what conditions. Many research questions can be asked within phase I, including perceived benefit of the treatment, cost of treatment, qualifications of the providers etc.

Phase 2: Safety status (Is it safe?)

Safety issues are important in the treatment of any illness, but in the area of CAM they cannot be emphasized strongly enough. The inherent risks of the diseases or illnesses for which patients seek CAM are generally low. The treatments given should therefore carry a low risk of adverse effects. CAM treatments have often been claimed to be without risk, but adverse effects in CAM are more than occasional case reports. [35]. The methods of choice to study safety would be similar to the detection of adverse events associated with pharmaceutical treatments. The field of acupuncture has provided a thorough and rigorous risk assessment [36-39] based on these principles and the other fields within CAM need to follow suit [40-42].

Phase 3: Comparative effectiveness (What is the system effectiveness?)

Patients are seeking CAM as a treatment system. Research needs to examine the outcomes of these treatments both in combination with, and as alternative, to conventional care. Thus randomized, controlled pragmatic trials are needed [43,44], wherein the specifics of the system are not disassembled, but the system under study is allowed to function as it is clinically practiced, including the urgently needed evaluation of cost-effectiveness [45]. The pragmatic trial design has been used in the study of acupuncture treatment of chronic pain [46], and the methodology is also widely used in conventional medicine, for example in dietary intervention studies such as the Women's Health Initiative [47]. In these trials, patients are randomly assigned to treatment alternatives which may include alternative viable whole systems, conventional treatment or no treatment. The trials may also evaluate whole system interventions that are implemented under protocols that specify individualized treatments based on specific patient characteristics, following the patterns of care found in the community. While blinding of the treatment providers and patients/subjects with regard to treatment allocation is usually not feasible, blinding of the outcomes evaluators can ensure an unbiased comparison of the outcome assessment.

In conventional medicine, a treatment's efficacy is often determined before assessment of its effectiveness. It is considered unethical to include non-efficacious treatments in the real world treatment of patients. Patients are, however, already using CAM treatments and thus effectiveness studies can be used to guide decisions about the necessity of studying the efficacy of specific components. Researchers have voiced this view in several countries over the last few years [20,48,49]. It was even emphasized in the much-criticized recent meta-analysis of homeopathy [50]: "Clearly, rather than doing further placebo-controlled trials of homoeopathy, future research efforts should focus on the nature of context effects and on the place of homoeopathy in health-care systems".

Included in this phase, and bordering on the next phase would be studies to evaluate how limited a whole package of care can be while still retaining its overall effectiveness ("Occam's razor"). There are many components within a CAM treatment approach, but are there any that we can eliminate and at the same time retain or improve overall treatment effectiveness?

Phase 4: Component efficacy (What is the efficacy of a specific component of the therapy?)

This is the area that has received most attention and research money to date, and while it is important, it is not the starting point in our model. The methods of choice are often double-blind randomized controlled trials. The well-established documentation of acupuncture/acupressure stimulation of one acupuncture point in the treatment of chemotherapy-induced nausea/vomiting is such an example [51]. It is, however, important to recognize that results from such research cannot be used to document or disprove the effectiveness of a "whole system" treatment.

Phase 5: Biological mechanisms (How can treatment outcomes be explained biologically?)

We want, and need, to understand the pathways and mechanisms through which treatments exercise their influence [52]. This has been and is being explored for acupuncture as an anti-emetic [53]. We must however realize that treatment outcomes, both at the system and component level, can be documented before the biologic mechanisms are understood, as has been the case for several non-surgical conventional therapies. The most prominent example from conventional medicine is probably aspirin. The anti-inflammatory and pain-killing properties of aspirin were discovered long before it became known that aspirin influences prostaglandin synthesis.

Summary

CAM is not simply a new array of therapeutic tools that need to be evaluated; it presents other ways to think about disease and therapeutics, and consequently new ideas about how research should be strategically developed. In this article we have suggested two ways of taking this forward. First, the absence of statutory and financial gatekeepers for CAM presents several issues that need to be considered closely. Secondly, the structure of CAM research should be different, in subtle but important ways. We have provided some suggestions as to how this alternative research strategy could be structured keeping in mind that the ultimate goal for all approaches to treatment is to provide effective medical interventions at reasonable cost and without harm.

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

VF and SG conceived of the study. VF drafted the manuscript. All authors have contributed to the paper by offering textual elements and decisive references, contributing to the critical discussion that fostered the article, and a critical reading. All authors read and approved the final manuscript.

References

  1. Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL: Unconventional Medicine in the United-States - Prevalence, Costs, and Patterns of Use.

    New England Journal of Medicine 1993, 328:246-252. PubMed Abstract | Publisher Full Text OpenURL

  2. Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, van Rompay M, Kessler RC: Trends in alternative medicine use in the United States, 1990-1997 - Results of a follow-up national survey.

    Jama-Journal of the American Medical Association 1998, 280:1569-1575. Publisher Full Text OpenURL

  3. Steinsbekk A, Launso L: Empowering the cancer patient or controlling the tumor? A qualitative study of how cancer patients experience consultations with complementary and alternative medicine practitioners and physicians, respectively.

    Integr Cancer Ther 2005, 4:195-200. PubMed Abstract | Publisher Full Text OpenURL

  4. Caspi O, Koithan M, Criddle MW: Alternative medicine or "alternative" patients: a qualitative study of patient-oriented decision-making processes with respect to complementary and alternative medicine.

    Med Decis Making 2004, 24:64-79. PubMed Abstract | Publisher Full Text OpenURL

  5. Von RE, Pampallona S, Van WB, Cerny T, Hurny C, Bernhard J, Helwig S, Heusser P: Attitudes and Beliefs towards Disease and Treatment in Patients with Advanced Cancer Using Anthroposophical Medicine.

    Onkologie 2000, 23:558-563. PubMed Abstract | Publisher Full Text OpenURL

  6. Weis J, Bartsch HH, Hennies F, Rietschel M, Heim M, Adam G, Gartner U, Ammon A: Complementary medicine in cancer patients: Demand, patients' attitudes and psychological beliefs.

    Onkologie 1998, 21:144-149. Publisher Full Text OpenURL

  7. Wilkinson S, Gomella LG, Smith JA, Brawer MK, Dawson NA, Wajsman Z, Dai L, Chodak GW: Attitudes and use of complementary medicine in men with prostate cancer.

    J Urol 2002, 168:2505-2509. PubMed Abstract | Publisher Full Text OpenURL

  8. Schonekaes K, Micke O, Mucke R, Buntzel J, Glatzel M, Bruns F, Kisters K: [Use of complementary/alternative therapy methods by patients with breast cancer].

    Forsch Komplementarmed Klass Naturheilkd 2003, 10:304-308. PubMed Abstract | Publisher Full Text OpenURL

  9. Astin JA: Why patients use alternative medicine: results of a national study.

    JAMA 1998, 279:1548-1553. PubMed Abstract | Publisher Full Text OpenURL

  10. Boon H, Brown JB, Gavin A, Kennard MA, Stewart M: Breast cancer survivors' perceptions of complementary/alternative medicine (CAM): making the decision to use or not to use.

    Qual Health Res 1999, 9:639-653. PubMed Abstract | Publisher Full Text OpenURL

  11. Boon H, Brown JB, Gavin A, Westlake K: Men with prostate cancer: making decisions about complementary/alternative medicine.

    Med Decis Making 2003, 23:471-479. PubMed Abstract | Publisher Full Text OpenURL

  12. Fønnebø V, Launsø L: Looking for new knowledge in the field of curing and healing.

    Focus Altern Complement Ther 2005, 10:13-14. OpenURL

  13. Launsø L, Gannik DE: The need for revision of medical research designs. In Disease knowledge and society. Edited by Gannik DE and Launsø L. Copenhagen, Forlaget Samfundslitteratur; 2000. OpenURL

  14. Gluud C, Gluud LL: Evidence based diagnostics.

    BMJ 2005, 330:724-726. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  15. Guthlin C, Lange O, Walach H: Measuring the effects of acupuncture and homoeopathy in general practice: an uncontrolled prospective documentation approach.

    BMC Public Health 2004, 4:6. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  16. Witt CM, Ludtke R, Baur R, Willich SN: Homeopathic medical practice: long-term results of a cohort study with 3981 patients.

    BMC Public Health 2005, 5:115. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  17. Hamre HJ, Becker-Witt C, Glockmann A, Ziegler R, Willich SN, Kiene H: Anthroposophic therapies in chronic disease: the Anthroposophic Medicine Outcomes Study (AMOS).

    Eur J Med Res 2004, 9:351-360. PubMed Abstract OpenURL

  18. Steinsbekk A, Ludtke R: Patients' assessments of the effectiveness of homeopathic care in Norway: a prospective observational multicentre outcome study.

    Homeopathy 2005, 94:10-16. PubMed Abstract | Publisher Full Text OpenURL

  19. Norheim AJ, Fonnebo V: A survey of acupuncture patients: results from a questionnaire among a random sample in the general population in Norway.

    Complement Ther Med 2000, 8:187-192. PubMed Abstract | Publisher Full Text OpenURL

  20. Ritenbaugh C, Verhoef M, Fleishman S, Boon H, Leis A: Whole systems research: a discipline for studying complementary and alternative medicine.

    Altern Ther Health Med 2003, 9:32-36. PubMed Abstract OpenURL

  21. Walach H, Jonas WB, Lewith G: The role of outcomes research in evaluating complementary and alternative medicine. In Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. Edited by Lewith G, Jonas WB and Walach H. London, Churchill-Livingston; 2002:29-45. OpenURL

  22. Lewith G, Walach H, Jonas WB: Balanced research strategies for complementary and alternative medicine. In Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London, Churchill Livingston; 2002:3-27. OpenURL

  23. Jonas WB: Building an evidence house: challenges and solutions to research in complementary and alternative medicine.

    Forsch Komplementarmed Klass Naturheilkd 2005, 12:159-167. PubMed Abstract | Publisher Full Text OpenURL

  24. Wittman WW, Walach H: Evaluating complementary medicine: Lessons to be learned from evaluation research. In Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London, Churchill Livingston; 2002:93-108. OpenURL

  25. Paterson C, Dieppe P: Characteristic and incidental (placebo) effects in complex interventions such as acupuncture.

    BMJ 2005, 330:1202-1205. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  26. Lilienfeld SO, Lynn SJ and Lohr JM (Eds): Science and Pseudoscience in psychology. New York, The Guilford Press; 2003. OpenURL

  27. Baskin TW, Tierney SC, Minami T, Wampold BE: Establishing specificity in psychotherapy: a meta-analysis of structural equivalence of placebo controls.

    J Consult Clin Psychol 2003, 71:973-979. PubMed Abstract | Publisher Full Text OpenURL

  28. Horvath P: Placebos and common factors in two decades of psychotherapy research.

    Psychol Bull 1988, 104:214-225. PubMed Abstract | Publisher Full Text OpenURL

  29. Thomas KJ, Nicholl JP, Coleman P: Use and expenditure on complementary medicine in England: a population based survey.

    Complement Ther Med 2001, 9:2-11. PubMed Abstract | Publisher Full Text OpenURL

  30. Thomas KJ, Carr J, Westlake L, Williams BT: Use of non-orthodox and conventional health care in Great Britain.

    BMJ 1991, 302:207-210. PubMed Abstract | PubMed Central Full Text OpenURL

  31. Hanssen B, Grimsgaard S, Launso L, Fonnebo V, Falkenberg T, Rasmussen NK: Use of complementary and alternative medicine in the Scandinavian countries.

    Scand J Prim Health Care 2005, 23:57-62. PubMed Abstract | Publisher Full Text OpenURL

  32. Launso L, Brendstrup E, Arnberg S: An exploratory study of reflexological treatment for headache.

    Altern Ther Health Med 1999, 5:57-65. PubMed Abstract OpenURL

  33. Launso L, Rieper J: General practitioners and classical homeopaths treatment models for asthma and allergy.

    Homeopathy 2005, 94:17-25. PubMed Abstract | Publisher Full Text OpenURL

  34. Launso L: Use of alternative treatments in Denmark: patterns of use and patients' experience with treatment effects.

    Altern Ther Health Med 2000, 6:102-107. PubMed Abstract OpenURL

  35. Norheim AJ, Fønnebø V: Acupuncture adverse effects are more than occasional case reports.

    Complement Ther Med 1996, 4:14-20. Publisher Full Text OpenURL

  36. Norheim AJ, Fonnebo V: Adverse effects of acupuncture.

    Lancet 1995, 345:1576. PubMed Abstract | Publisher Full Text OpenURL

  37. Norheim AJ: Adverse effects of acupuncture: a study of the literature for the years 1981-1994.

    J Altern Complement Med 1996, 2:291-297. PubMed Abstract OpenURL

  38. MacPherson H, Thomas K, Walters S, Fitter M: The York acupuncture safety study: prospective survey of 34 000 treatments by traditional acupuncturists.

    BMJ 2001, 323:486-487. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  39. MacPherson H, Scullion A, Thomas KJ, Walters S: Patient reports of adverse events associated with acupuncture treatment: a prospective national survey.

    Qual Saf Health Care 2004, 13:349-355. PubMed Abstract | Publisher Full Text OpenURL

  40. MacPherson H, Liu B: The safety of Chinese herbal medicine: a pilot study for a national survey.

    J Altern Complement Med 2005, 11:617-626. PubMed Abstract | Publisher Full Text OpenURL

  41. Endrizzi C, Rossi E, Crudeli L, Garibaldi D: Harm in homeopathy: aggravations, adverse drug events or medication errors?

    Homeopathy 2005, 94:233-240. PubMed Abstract | Publisher Full Text OpenURL

  42. Thompson E, Barron S, Spence D: A preliminary audit investigating remedy reactions including adverse events in routine homeopathic practice.

    Homeopathy 2004, 93:203-209. PubMed Abstract | Publisher Full Text OpenURL

  43. MacPherson H: Pragmatic clinical trials.

    Complement Ther Med 2004, 12:136-140. PubMed Abstract | Publisher Full Text OpenURL

  44. Hammerschlag R: Methodological and ethical issues in clinical trials of acupuncture.

    J Altern Complement Med 1998, 4:159-171. PubMed Abstract OpenURL

  45. Thomas KJ, MacPherson H, Ratcliffe J, Thorpe L, Brazier J, Campbell M, Fitter M, Roman M, Walters S, Nicholl JP: Longer term clinical and economic benefits of offering acupuncture care to patients with chronic low back pain.

    Health Technol Assess 2005, 9:iii-x, 1. OpenURL

  46. Vickers AJ, Rees RW, Zollman CE, McCarney R, Smith CM, Ellis N, Fisher P, Van HR: Acupuncture for chronic headache in primary care: large, pragmatic, randomised trial.

    BMJ 2004, 328:744. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  47. Beresford SA, Johnson KC, Ritenbaugh C, Lasser NL, Snetselaar LG, Black HR, Anderson GL, Assaf AR, Bassford T, Bowen D, Brunner RL, Brzyski RG, Caan B, Chlebowski RT, Gass M, Harrigan RC, Hays J, Heber D, Heiss G, Hendrix SL, Howard BV, Hsia J, Hubbell FA, Jackson RD, Kotchen JM, Kuller LH, LaCroix AZ, Lane DS, Langer RD, Lewis CE, Manson JE, Margolis KL, Mossavar-Rahmani Y, Ockene JK, Parker LM, Perri MG, Phillips L, Prentice RL, Robbins J, Rossouw JE, Sarto GE, Stefanick ML, Van HL, Vitolins MZ, Wactawski-Wende J, Wallace RB, Whitlock E: Low-fat dietary pattern and risk of colorectal cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial.

    JAMA 2006, 295:643-654. PubMed Abstract | Publisher Full Text OpenURL

  48. Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A: Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT.

    Complement Ther Med 2005, 13:206-212. PubMed Abstract | Publisher Full Text OpenURL

  49. Walach H: Zirkulär statt hierarchisch: Methodische Prinzipien bei der Evaluation der therapeutischen Effekte von Komplementärmedizin und anderer komplexer Massnahmen.

    Informatik, Biometrie und Epidemiologie in Medizin und Biologie 2004, 35:229-242. OpenURL

  50. Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M: Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.

    Lancet 2005, 366:726-732. PubMed Abstract | Publisher Full Text OpenURL

  51. Ezzo J, Vickers A, Richardson MA, Allen C, Dibble SL, Issell B, Lao L, Pearl M, Ramirez G, Roscoe JA, Shen J, Shivnan J, Streitberger K, Treish I, Zhang G: Acupuncture-point stimulation for chemotherapy-induced nausea and vomiting.

    J Clin Oncol 2005, 23:7188-7198. PubMed Abstract | Publisher Full Text OpenURL

  52. Lewith GT, White PJ, Pariente J: Investigating acupuncture using brain imaging techniques: the current state of play.

    Evid Based Complement Alternat Med 2005, 2:315-319. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  53. Tatewaki M, Strickland C, Fukuda H, Tsuchida D, Hoshino E, Pappas TN, Takahashi T: Effects of acupuncture on vasopressin-induced emesis in conscious dogs.

    Am J Physiol Regul Integr Comp Physiol 2005, 288:R401-R408. PubMed Abstract OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/7/7/prepub