Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Research article

Spontaneous improvement in randomised clinical trials: meta-analysis of three-armed trials comparing no treatment, placebo and active intervention

Lasse Theis Krogsbøll, Asbjørn Hróbjartsson and Peter C Gøtzsche*

Author Affiliations

The Nordic Cochrane Centre, Rigshospitalet, Dept. 3343, Blegdamsvej 9, DK-2100 Copenhagen, Denmark

For all author emails, please log on.

BMC Medical Research Methodology 2009, 9:1  doi:10.1186/1471-2288-9-1

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2288/9/1


Received:6 August 2008
Accepted:5 January 2009
Published:5 January 2009

© 2009 Krogsbøll et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. It is not known to which extent spontaneous improvement, effect of placebo and effect of active interventions contribute to the observed change from baseline, and we aimed at quantifying these contributions.

Methods

Systematic review and meta-analysis, based on a Cochrane review of the effect of placebo interventions for all clinical conditions. We selected all trials that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. Clinical conditions that had been studied in less than three trials were excluded.

Results

We analysed 37 trials (2900 patients) that covered 8 clinical conditions. The active interventions were psychological in 17 trials, physical in 15 trials, and pharmacological in 5 trials. Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms. The standardized mean difference (SMD) for change from baseline was -0.24 (95% confidence interval -0.36 to -0.12) for no treatment, -0.44 (-0.61 to -0.28) for placebo, and -1.01 (-1.16 to -0.86) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to that of the active interventions were 24% and 20%, respectively, but with some uncertainty, as indicated by the confidence intervals for the three SMDs. The conditions that had the most pronounced spontaneous improvement were nausea (45%), smoking (40%), depression (35%), phobia (34%) and acute pain (25%).

Conclusion

Spontaneous improvement and effect of placebo contributed importantly to the observed treatment effect in actively treated patients, but the relative importance of these factors differed according to clinical condition and intervention.

Background

It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. An improvement will often be ascribed to the treatment, although at least two other factors often play a role.

One factor is spontaneous improvement [1]. Many clinical conditions are self-limiting, e.g. headache, acute low back pain and the common cold, and most chronic disease symptoms fluctuate in intensity, e.g. rheumatoid arthritis, chronic low back pain and psoriasis. Patients will often seek medical attention when their symptoms are worst, and they are most likely to be included in randomised trials at this time. For the purpose of this paper, we regarded regression to the mean effects as being part of the spontaneous improvement. Regression to the mean occurs, for example, when a patient can only be included in a trial if the symptoms are worse than some threshold value; for statistical reasons, the value will then likely be lower at a later time [1,2].

The second factor is the effect of placebo. Patients may feel reassured, change their expectation, or re-interpret their symptoms once a treatment has been commenced. A Cochrane systematic review did not find large effects of placebo, but some effect in trials with patient-reported continuous outcomes, especially pain [3-5].

We have not found any previous reviews of the three main factors affecting the clinical course of patients included in randomised clinical trials: spontaneous improvement, effect of placebos and effect of active interventions (Fig. 1). We aimed at quantifying their relative contribution to change from baseline in randomised trials.

thumbnailFigure 1. Illustration of approximate contributions of spontaneous improvement and effect of placebo to the estimated effect of active interventions.

Methods

The Cochrane review of the effect of placebo interventions involved a thorough search for trials including a no-treatment arm and a placebo arm. We selected all trials from the updated Cochrane review of placebo interventions [5] that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. In order to permit analyses of separate clinical conditions, we excluded conditions studied in less than three trials.

Potentially eligible trial reports were read in full by one author (LK), who made preliminary decisions on inclusion and choice of outcome, and extracted the data. The authors of the Cochrane review (AH and PCG) checked the selections and the extracted data. Disagreements were resolved by discussion.

In the Cochrane review, patient-reported outcomes were preferred to observer-reported ones. For this study, we selected the outcome that we found most relevant, disregarding whether it was patient- or observer-reported. We made this decision by consensus; there was very little disagreement. In seven cases, the chosen outcome was different from that in the original review. An example is the selection of the well-known observer-reported Bech-Rafaelsen Melancholia Scale instead of the patient-reported Befindlichkeits-Skala.

Data extraction was done using a pilot-tested chart. For each trial, pre- and post-treatment means, standard deviations and group sizes were extracted for the three arms. Additional information extracted was: clinical condition, acute or chronic problem, name and range of scale used, and type of intervention (physical, pharmacological or psychological).

Meta-analysis was done using Comprehensive Meta Analysis [computer program] version 2.2.030, July 2006.

Standardized mean differences (SMD) with 95% confidence intervals were calculated for each trial. SMD is the difference in means divided by the pooled standard deviation. SMD was calculated as Hedges' g, with adjustment for small sample bias. A negative SMD usually implies a positive effect of the intervention, e.g. a lower pain score means less pain. However, in four trials, a large clinical score meant a beneficial effect, and we therefore changed the sign of the SMD before the analysis in these cases. Thus, a negative SMD in our analyses always means a beneficial effect. When standard deviations were missing, we used those from similar trials.

Due to the clinical diversity of the included patients, we did not investigate one treatment effect, but rather the mean of many different treatment effects. There was also substantial methodological heterogeneity, e.g. some trials did not have adequately concealed treatment allocation. We therefore used a random effects model for the analyses. The degree of heterogeneity was investigated with I2, which describes the percentage of the variability in effect estimates that is due to heterogeneity rather than sampling error [6].

It was not straightforward how to do the analyses, as we needed to compare the effects in the three groups with the condition at baseline. We analyzed the three treatment arms separately by comparing the post-treatment values with the values at baseline. These data were paired, but we analyzed them as if they were independent, as the presentation of data in the articles did not allow paired analyses. Thus, we accepted a moderate loss of statistical power by handling the paired data as unpaired and assumed that the effect of the ignored correlations between pre- and post-intervention measurements was the same in all situations. It should be noted that this approach leads to overestimation of the sampling error, and therefore to underestimation of the heterogeneity.

It was not possible to determine group sizes both pre- and post-treatment for all trials. We therefore used post-treatment sizes in the analyses, which has the advantage that treatment arms with relatively more dropouts receive less weight.

Ten trials had more than one active treatment. In the meta-analysis, these were entered as separate treatment arms and therefore contributed relatively more than trials with only one active treatment arm. However, the same would occur in trials with skewed randomisation ratios, and overall, the numbers of patients contributing to the results of the three treatment arms were not much different.

Results

In- and exclusion of trials

There were 118 trials in the Cochrane review with continuous outcome data. We excluded 61 trials: seven were two-armed; in 27 trials, the clinical condition had been studied in less than 3 trials; and 27 trials did not have a baseline assessment. Almost all of the trials without a baseline addressed acute conditions, for example acute pain during a procedure. Though such trials often had pre-treatment assessments they did not involve an assessment of pain experienced during the procedure, or the treatment was given before the painful procedure was initiated. Thus, we identified 57 eligible trials. Data necessary for meta-analyses could not be obtained from 14 trials, so we initially included 43 trials [7-49] (Fig. 2).

thumbnailFigure 2. Selection of trials for the review.

We found that the estimates for four hypertension trials [46-49] were unreliable for our purpose. Three of the four trials had run-in periods of 4 to 8 weeks before randomisation, which eliminates the regression to the mean effect, and the changes from baseline were therefore very small and unstable.

We expected that the change from baseline in the no-treatment arm and the placebo arm would covary from trial to trial, so that when it was large in one arm, it also tended to be large in the other. We verified this, but with two clear outliers (Fig. 3, lower right corner). In one nausea trial [10], the placebo therapy consisted of talks about the child's daily life, which might have had a large reassuring effect. In the other trial [33], the smoking rate was monitored for one week before treatment in the placebo group, but not in the no-treatment group.

thumbnailFigure 3. Change from baseline in the no-treatment and the placebo arms of the 37 analysed trials. The results are shown as standardized mean differences.

Our overall results were very similar, whether or not we excluded the four hypertension trials and the two outliers, but we feel the results for nausea and smoking are more reliable without the outliers. We report below the results for 37 trials (2900 patients), after these six trials were excluded.

Characteristics of included trials

The 37 trials covered eight different clinical conditions. Most active interventions were of a psychological (17 trials) or physical nature (15 trials); 5 trials were of drugs. Typical psychological treatments were cognitive behaviour therapy and hypnosis, and physical treatment was often acupuncture. Only 10 trials investigated conditions defined by us as acute: depression [7-9], nausea [11,12], and acute pain [13-17], while 27 trials investigated chronic conditions: chronic pain [18-28], phobia [29-31], smoking [32,34], obesity [35-39] and insomnia [40-45]. Duration of treatment was highly variable, ranging from a few days to several months. The outcome was patient-reported in 26 trials and observer-reported in 11 trials.

Statistical analyses

Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms (Table 1). The SMD was -0.24 (95% confidence interval -0.36 to -0.12, I2 = 25%) for no treatment, -0.44 (-0.61 to -0.28, I2 = 57%) for placebo, and -1.01 (-1.16 to -0.86, I2 = 57%) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to the change from baseline in the active intervention groups were 24% (0.24/1.01) and 20% ((0.44-0.24)/1.01), respectively (shown approximately in Fig. 1), but with wide variation related to the studied clinical conditions and interventions (Fig. 4). The most pronounced spontaneous improvements, relative to the change from baseline in the actively treated groups, were seen in nausea 45%, smoking 40%, depression 35%, phobia 34% and acute pain 25% (Fig. 4). When combining the influence of spontaneous remission and placebo, the similar proportions were for nausea 73%, smoking 59%, depression 43%, phobia 74%, and acute pain 23% (Fig. 4).

Table 1. Standardized mean differences (SMD) for changes from baseline in the three treatment arms separately.

thumbnailFigure 4. Relative contributions of the spontaneous improvement, effect of placebo, and effect of active treatment to the change from baseline seen in the actively treated group.

The point estimates were very similar in trials with patient-reported and observer-reported outcomes (Table 2) whereas trials involving acute conditions tended to have larger improvements in all three arms compared with trials involving chronic conditions (Table 3), as expected.

Table 2. Standardized mean differences (SMD) for changes from baseline grouped by patient- and observer-reported outcome.

Table 3. Standardized mean differences (SMD) for changes from baseline grouped by acute or chronic condition.

Discussion

We found that both the spontaneous improvement and the effect of placebo contributed importantly to the observed treatment effect in actively treated patients. As noted above, we have not found other reviews that describe the relative contributions of spontaneous remission and placebo to the improvement clinicians note when they treat patients.

Our findings have two implications. First, they underline that it is a fallacy when patients and clinicians interpret an improvement that occurs after a treatment has been instituted as being caused by that treatment. In fact, we found that, on average, only about half of that improvement could be ascribed to the treatment in the trials we analysed.

Second, our findings show that it is wrong to describe the effect that is observed in a placebo arm of a randomised trial as the effect of placebo, as it includes the spontaneous improvement that would also have occurred without administration of a placebo [50]. This error is very common. We did a full-text search on "placebo effect" on the BMJ's website on 30 April 2008 and found the error in 90% of the articles, even in an obituary.

It is a limitation of our study that a quarter of the eligible trials did not report the data necessary for our meta-analyses. Furthermore, we had to use an unconventional meta-analytic method but find it reassuring that the overall effect of placebo was -0.28, as this agrees closely with our previous estimate of -0.24 in the Cochrane review [5] where we used standard meta-analytic methods. We would not expect more elaborate methods to yield results that differ importantly from those we have reported here.

We considered other approaches and also did more traditional meta-analyses, comparing treatment arms within each trial after treatment and calculating ratios between the three arms before these ratios were pooled, but as the denominators of the per trial ratios had a distribution that crossed zero, these ratios were very unstable because of "division almost by zero" effects. Furthermore, we could not use this standard approach for the spontaneous improvement, as this required comparison with baseline. We did not try to convert our unpaired analyses into paired ones, as this would have required estimations of correlations that were likely to vary between diseases and interventions.

The relative contributions of spontaneous improvement, effects of placebo, and effects of active treatment to the observed change from baseline varied considerably. The eight clinical conditions we analysed were either psychiatric diseases (depression and phobia), involved a high degree of patient cooperation (smoking and obesity) or involved subjective outcomes (acute and chronic pain, nausea, and insomnia); and the interventions were mostly non-pharmacological. It seems likely that spontaneous improvement is more important in trials that include patients with high symptom scores and that do not implement a placebo run-in period, particularly as the regression to the mean is likely to be more pronounced in such settings.

Our Cochrane review suggested that the effect of placebos is smaller when imitating pharmacological interventions and when outcomes are observer-reported [3-5]. It is therefore likely that the effect of placebo is comparatively less important in drug trials and in trials with observer-reported outcomes. The Cochrane review found a small effect of placebo on pain, which we reproduced in this review for chronic pain, but not for acute pain, possibly because we were unable to include many acute pain trials that provided no baseline data.

The active interventions we included seemed to be quite effective, which is surprising, as most of them were unconventional, and as many trials involved acupuncture. We recently did a systematic review of three-armed acupuncture trials and found a small analgesic effect of acupuncture, compared to placebo acupuncture, that seems to lack clinical relevance and could not be clearly distinguished from bias [51]. The apparent effects we noted of active treatments may therefore to some degree reflect bias, e.g. related to unconcealed allocation of patients and unsuccessful blinding.

A major problem related to the interpretation of the outcomes in no-treatment and placebo groups is the lack of blinding. Blinding is important to reduce reporting bias in experiments with subjective outcomes [52], but it is not possible to blind patients who receive no treatment. The lack of blinding favours placebo [52], as patients were often blinded with respect to placebo and active treatment. Patients in the placebo group may think they receive active treatment, or they may tend to please their doctors by exaggerating the improvement, and conversely, patients in the no-treatment group may tend to view their experiences more negatively, as they may feel deprived of treatment.

Conclusion

We conclude that both the spontaneous improvement and the effect of placebo contribute importantly to the observed treatment effect in actively treated patients, and that the relative importance of these factors differ according to clinical condition and intervention.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

PCG and AH coined the idea, initiated the project, and wrote the protocol with LK; LK extracted data that were checked by PCG and AH; LK and PCG did the analyses; LK wrote the first draft of the paper, PCG and AH the final version. Guarantors: PCG and AH.

Funding

No funding.

Acknowledgements

We thank the statistical peer reviewer, Jesse Berlin, and a statistician we consulted, Peter Dalgaard, for very valuable advice.

References

  1. Morton V, Torgerson DJ: Effect of regression to the mean on decision making in health care.

    BMJ 2003, 326:1083-4. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Gøtzsche PC: Rational Diagnosis and Treatment. Evidence-Based Clinical Decision-Making. 4th edition. Chichester, Wiley; 2007. OpenURL

  3. Hróbjartsson A, Gøtzsche PC: Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment.

    N Engl J Med 2001, 344:1594-1602. PubMed Abstract | Publisher Full Text OpenURL

  4. Hrobjartsson A, Gøtzsche PC: Placebo treatment versus no treatment.

    Cochrane Database Syst Rev 2003, (1):CD003974. PubMed Abstract | Publisher Full Text OpenURL

  5. Hróbjartsson A, Gøtzsche PC: Placebo interventions for all clinical conditions.

    Cochrane Database Syst Rev 2004, (3):CD003974. PubMed Abstract | Publisher Full Text OpenURL

  6. Higgins JPT, Green S, editors: [http://www.cochrane-handbook.org] webcite

    Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. 2008. OpenURL

  7. Nandi DN, Ajmany S, Ganguli H, Banerjee G, Boral GC, Ghosh A, Sarkar S: A clinical evaluation of depressives found in a rural survey in India.

    Br J Psychiatry 1976, 128:523-7. PubMed Abstract | Publisher Full Text OpenURL

  8. Röschke J, Wolf C, Müller MJ, Wagner P, Mann K, Grözinger M, Bech S: The benefit from whole body acupuncture in major depression.

    J Affect Disord 2000, 57:73-81. PubMed Abstract | Publisher Full Text OpenURL

  9. Sumaya IC, Rienzi BM, Deegan JF, Moss DE: Bright light treatment decreases depression in institutionalized older adults: a placebo-controlled crossover study.

    J Gerontol A Biol Sci Med Sci 2001, 56(6):M356-M360. PubMed Abstract | Publisher Full Text OpenURL

  10. Hawkins PJ, Liossi C, Ewart BW, Hatria P, Kosmidis VH, Varvutsi M: Hypnotherapy for control of anticipatory nausea and vomiting in children with cancer: preliminary findings.

    Psychooncology 1995, 4:101-6. Publisher Full Text OpenURL

  11. O'Brien B, Relyea MJ, Taerum T: Efficacy of P6 acupressure in the treatment of nausea and vomiting during pregnancy.

    Am J Obstet Gynecol 1996, 174:708-15. PubMed Abstract | Publisher Full Text OpenURL

  12. Werntoft E, Dykes A: Effect of acupressure on nausea and vomiting during pregnancy. A randomized, placebo-controlled, pilot study.

    J Reprod Med 2001, 46:835-9. PubMed Abstract OpenURL

  13. Cupal DD, Brewer BW: Effects of relaxation and guided imagery on knee strength, reinjury anxiety, and pain following anterior cruciate ligament reconstruction.

    Rehabil Psychol 2001, 46:28-43. Publisher Full Text OpenURL

  14. Forster EL, Kramer JF, Lucy SD, Scudds RA, Novick RJ: Effects of TENS on pain, medications, and pulmonary function following coronary artery bypass graft surgery.

    Chest 1994, 106:1343-8. PubMed Abstract | Publisher Full Text OpenURL

  15. Helms JM: Acupuncture for the management of primary dysmenorrhea.

    Obstet Gynecol 1987, 69:51-6. PubMed Abstract OpenURL

  16. Kober A, Scheck T, Greher M, Lieba F, Fleischhackl R, Fleischhackl S, Randunsky F, Hoerauf K: Prehospital analgesia with acupressure in victims of minor trauma: a prospective, randomized, double-blinded trial.

    Anesth Analg 2002, 95:723-7. PubMed Abstract | Publisher Full Text OpenURL

  17. Sanders G, Tepe R, Maloney P, Reinert O: The effect of spinal manipulation on subjects with acute low back pain: a comparison of visual analog pain scores and serum beta endorphin levels.

    J Manipulative Physiol Ther 1990, 13:58. OpenURL

  18. Alfano AP, Taylor AG, Foresman PA, Dunkl PR, McConnell GG, Conaway MR, Gillies GT: Static magnetic fields for treatment of fibromyalgia: a randomised controlled trial.

    J Altern Complement Med 2001, 7:53-64. PubMed Abstract | Publisher Full Text OpenURL

  19. Blanchard EB, Appelbaum KA, Radnitz CL, Michultka D, Morrill B, Kirsch C, Hillhouse J, Evans DD, Guarnieri P, Attanasio V, Andrasik F, Jaccard J, Dentinger MP: Placebo-controlled evaluation of abbreviated progressive muscle relaxation and of relaxation combined with cognitive therapy in the treatment of tension headache.

    J Consult Clin Psychol 1990, 58:210-5. PubMed Abstract | Publisher Full Text OpenURL

  20. Blanchard EB, Appelbaum KA, Radnitz CL, Morrill B, Michultka D, Kirsch C, Guarnieri P, Hillhouse J, Evans DD, Jaccard J, Barron KD: A controlled evaluation of the thermal biofeedback and thermal biofeedback combined with cognitive therapy in the treatment of vascular headache.

    J Consult Clin Psychol 1990, 58:216-24. PubMed Abstract | Publisher Full Text OpenURL

  21. Chenard JR, Marchand S, Charest J, Jinxue L, Lavignolle B: Evaluation of a behavioral intervention for chronic low-back pain: 'The interactional back school' [Évaluation d'un traitement comportmental de la lombalgie chronique: 'l école interactionelle du dos'].

    Science et Comportement 1991, 21:225-39. OpenURL

  22. Hong C, Chen Y, Pon CH, Yu J: Immediate effects of various physical medicine modalities on pain threshold of an active myofascial trigger point.

    J Musculoskeletal Pain 1993, 1:37-53. Publisher Full Text OpenURL

  23. Kotani N, Kushikata T, Suzuki A, Hashimoto H, Muraoka M, Matsuki A: Insertion of intradermal needles into painful points provides analgesia for intractable abdominal scar pain.

    Reg Anesth Pain Med 2001, 26:532-8. PubMed Abstract | Publisher Full Text OpenURL

  24. Leibing E, Leonhardt U, Köster G, Goerlitz A, Rosenfeldt JA, Hilgers R, Ramadori G: Acupuncture treatment of chronic low-back pain: a randomized, blinded, placebo-controlled trial with 9-months followup.

    Pain 2002, 96:189-96. PubMed Abstract | Publisher Full Text OpenURL

  25. Moffett JAK, Richardson PH, Frost H, Osborn A: A placebo controlled double blind trial to evaluate the effectiveness of pulsed short wave therapy for osteoarthritic hip and knee pain.

    Pain 1996, 67:121-7. PubMed Abstract | Publisher Full Text OpenURL

  26. Parker JC, Smarr KL, Buckelew SP, Stucky-Ropp RC, Hewett JE, Johnson JC, Wright GE, Irvin WS, Walker SE: Effects of stress management on clinical outcomes in rheumatoid arthritis.

    Arthritis Rheum 1995, 38:1807-18. PubMed Abstract | Publisher Full Text OpenURL

  27. Thomas VJ, Dixon AL, Milligan P: Cognitive-behaviour therapy for the management of sickle cell disease pain: an evaluation of a community based intervention.

    Br J Health Psychol 1999, 4:209-29. Publisher Full Text OpenURL

  28. Wojciechowski FL: Behavioral treatment of tension headache: a contribution to controlled outcome research methodology.

    Gedrag – Tijdschrift voor Psychologie 1984, 12:16-30. OpenURL

  29. Etringer BD, Cash TF, Rimm DC: Behavioral, affective and cognitive effects of participant modeling and an equally credible placebo.

    Behav Ther 1982, 13:476-85. Publisher Full Text OpenURL

  30. Lick J: Expectancy, false galvanic skin response feedback and systematic desensitization in the modification of phobic behavior.

    J Consult Clin Psychol 1975, 43:557-67. PubMed Abstract | Publisher Full Text OpenURL

  31. Rosen GM, Glasgow RE, Barrera M: A controlled study to assess the clinical effcacy of totally self-administrated systematic desensitization.

    J Consult Clin Psychol 1976, 44:208-17. PubMed Abstract | Publisher Full Text OpenURL

  32. Etter J, Lazlo E, Zellweger J, Perrot C, Perneger TV: Nicotine replacement to reduce cigarette consumption in smokers who are unwilling to quit: a randomized trial.

    J Clin Psychopharmacol 2002, 22:487-95. PubMed Abstract | Publisher Full Text OpenURL

  33. Sipich JF, Russell RK, Tobias LL: A comparison of covert sensitization and 'nonspecific' treatment in the modification of smoking behavior.

    J Behav Ther Exp Psychiatry 1974, 5:201-3. Publisher Full Text OpenURL

  34. Spanos NP, Mondoux TJ, Burgess CA: Comparison of multi-component hypnotic and non-hypnotic treatments for smoking.

    Contemp Hypnosis 1995, 12:12-19. OpenURL

  35. Antonio J, Colker CM, Torina G, Shi Q, Brink W, Kalman D: Effects of standardised guggulsterone phosphate supplement on body composition in overweight adults: a pilot study.

    Curr Ther Res 1999, 60:220-7. Publisher Full Text OpenURL

  36. Block J: Effects of rational emotive therapy on overweight adults.

    Psychotherapy: Theory, Research and Practice 1980, 17:277-80. Publisher Full Text OpenURL

  37. Colker CM, Kalman DS, Torina GC, Perlis T, Street C: Effects of Citrus aurantium extract, caffeine, and St. John's wort on body fat loss, lipid levels, and mood states in overweight healthy adults.

    Curr Ther Res 1999, 60:145-53. Publisher Full Text OpenURL

  38. Roongpisuthipong C, Panpakdee O, Boontawee A, Kulapongse S, Tanphaichitr V: Possible thermogenesis with dexfenfluramine.

    J Med Assoc Thai 1999, 82(2):150-159. PubMed Abstract OpenURL

  39. Senediak C, Spence SH: Rapid versus gradual scheduling of therapeutic contact in a family based behavioural weight control programme for children.

    Behav Psychother 1985, 13:265-87. OpenURL

  40. Ascher LM, Turner RM: Paradoxical intention and insomnia: an experimental investigation.

    Behav Res Ther 1979, 17:408-11. PubMed Abstract | Publisher Full Text OpenURL

  41. Espie CA, Lindsay WR, Brooks DN, Hood EM, Turvey T: A controlled comparative investigation of psychological treatments for chronic sleep-onset insomnia.

    Behav Res Ther 1989, 27:79-88. PubMed Abstract | Publisher Full Text OpenURL

  42. Lick JR, Heffler D: Relaxation training and attention placebo in the treatment of severe insomnia.

    J Consult Clin Psychol 1977, 45:153-61. PubMed Abstract | Publisher Full Text OpenURL

  43. Nicassio P, Bootzin R: A comparison of progressive relaxation and autogenic training as treatments for insomnia.

    J Abnorm Psychol 1974, 83(3):253-260. PubMed Abstract | Publisher Full Text OpenURL

  44. Tsay SL, Chen ML: Acupressure and quality of sleep in patients with end-stage renal disease: a randomised controlled trial.

    Int J Nurs Stud 2003, 40:1-7. PubMed Abstract | Publisher Full Text OpenURL

  45. Turner RM, Ascher LM: Controlled comparison of progressive relaxation, stimulus control, and paradoxical intention therapies for insomnia.

    J Consult Clin Psychol 1979, 47:500-8. PubMed Abstract | Publisher Full Text OpenURL

  46. Canino E, Cardona R, Monsalve P, Acuna FP, López B, Fragachan F: A behavioral treatment program as a therapy in the control of primary hypertension.

    Acta Cient Venez 1994, 45:23-30. PubMed Abstract OpenURL

  47. Frankel BL, Patel DJ, Horwitz D, Friedewald WT, Gaarder KR: Treatment of hypertension with biofeedback and relaxation techniques.

    Psychosom Med 1978, 40:276-93. PubMed Abstract | Publisher Full Text OpenURL

  48. Seer P, Raeburn JM: Meditation training and essential hypertension: a methodological study.

    J Behav Med 1980, 3:59-71. PubMed Abstract | Publisher Full Text OpenURL

  49. Yates RG, Lamping DL, Abram NL, Wright C: Effects of chiropractic treatment on blood pressure and anxiety: a randomized, controlled trial.

    J Manipulative Physiol Ther 1988, 11:484-8. PubMed Abstract OpenURL

  50. Hróbjartsson A: What are the main methodological problems in the estimation of placebo effects?

    J Clin Epidemiol 2002, 55:430-5. PubMed Abstract | Publisher Full Text OpenURL

  51. Madsen MV, Gøtzsche PC, Hróbjartsson A: Acupuncture treatment for pain. Systematic review of randomized clinical trials with acupuncture, placebo acupuncture and no-acupuncture groups.

    BMJ, in press. OpenURL

  52. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study.

    BMJ 2008, 336:601-5. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/9/1/prepub