Email updates

Keep up to date with the latest news and content from BMC Psychiatry and BioMed Central.

Open Access Highly Accessed Research article

Is computerised CBT really helpful for adult depression?-A meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity

Mirai So1*, Sosei Yamaguchi2, Sora Hashimoto3, Mitsuhiro Sado4, Toshi A Furukawa5 and Paul McCrone6

Author affiliations

1 Department of Psychological Medicine, Institute of Psychiatry, Kings College London, UK, Weston Education Centre, Cutcombe Rd, London, SE5 9RJ, UK

2 Department of Psychiatric Rehabilitation, National Institute of Mental Health, National Centre of Neurology and Psychiatry, Japan, 4-1-1 Ogawa-Higashi, Kodaira, Tokyo, 187-8553, Japan

3 Department of Psychology and Humanities, College of Sociology, Edogawa University, 474 Komaki, Nagareyama, Chiba, 270-0198, Japan

4 Department of Neuropsychiatry, Keio University School of Medicine, 35 Shinanomachi, Shinjuku-ku, Tokyo, 160-8582, Japan

5 Department of Health Promotion and Human Behavior/Department of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Yoshida Konoe-cho, Sakyo-ku, Kyoto, 606-8501, Japan

6 Centre for the Economics of Mental and Physical Health (CEMPH) P024, Institute of Psychiatry King’s College London, De Crespigny Park, London, SE5 8AF, UK

For all author emails, please log on.

Citation and License

BMC Psychiatry 2013, 13:113  doi:10.1186/1471-244X-13-113


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-244X/13/113


Received:15 October 2012
Accepted:15 March 2013
Published:15 April 2013

© 2013 So et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Depression is a major cause of disability worldwide, and computerised cognitive behavioural therapy (CCBT) is expected to be a more augmentative and efficient treatment. According to previous meta-analyses of CCBT, there is a need for a meta-analytic revaluation of the short-term effectiveness of this therapy and for an evaluation of its long-term effects, functional improvement and dropout.

Methods

Five databases were used (MEDLINE, PsycINFO, EMBASE, CENTRAL and CiNii). We included all RCTs with proper concealment and blinding of outcome assessment for the clinical effectiveness of CCBT in adults (aged 18 and over) with depression. Using Cohen’s method, the standard mean difference (SMD) for the overall pooled effects across the included studies was estimated with a random effect model. The main outcome measure and the relative risk of dropout were included in the meta-analysis.

Results

Fourteen trials met the inclusion criteria, and sixteen comparisons from these were used for the largest meta-analysis ever. All research used appropriate random sequence generation and Intention-to-Treat analyses (ITT), and employed self-reported measures as the primary outcome. For the sixteen comparisons (2807 participants) comparing CCBT and control conditions, the pooled SMD was −0.48 [95% IC −0.63 to −0.33], suggesting similar effect to the past reviews. Also, there was no significant clinical effect at long follow-up and no improvement of function found. Furthermore, a significantly higher drop-out rate was found for CCBT than for controls. When including studies without BDI as a rating scale and with only modern imputation as sensitivity analysis, the pooled SMD remained significant despite the reduction from a moderate to a small effect. Significant publication bias was found in a funnel plot and on two tests (Begg’s p = 0.09; Egger’s p = 0.01). Using a trim and fill analysis, the SMD was −0.32 [95% CI −0.49 to −0.16].

Conclusion

Despite a short-term reduction in depression at post-treatment, the effect at long follow-up and the function improvement were not significant, with significantly high drop-out. Considering the risk of bias, our meta-analysis implied that the clinical usefulness of current CCBT for adult depression may need to be re-considered downwards in terms of practical implementation and methodological validity.

Keywords:
CCBT; Web-based CBT; Cognitive-behavioural therapy; Depression; Systematic review; Meta-analysis

Background

Depression is recognised as a major cause of disability all over the world [1]. It has also been recently emphasised that it is not a highly recoverable disorder, even when treated with established pharmacotherapy [2], challenging previously widely-held ideas on its treatability [3]. Therefore, there is a great amount of expectation placed on evidence-based cognitive-behavioural therapy (CBT) [4] as an alternative or addition to pharmacotherapy [5,6]. It is hypothesised that reconstructing distorted cognition or inadaptable behaviour with CBT is likely to lead to the reduction of symptoms [7]. In fact, there has been increasing attention paid to CBT because of several of its advantages, including its significant effectiveness against mild-to-moderate depression [8], the enhancement of quality of life [9], increase in adherence to pharmacotherapy [3], comparative advantage for pregnant women [10] and patients’ preferences [11]. Also, CBT seems to be beneficial as an early intervention or relapse prevention measure [12] and CBT is recommended by the National Institute for Health and Clinical Excellence (NICE) in preference over routine pharmacotherapy as a treatment for milder depression [13]. Nevertheless, making CBT widely used requires addressing inevitable resource allocation problems, including accessibility and cost-effectiveness [14]. Therefore, the advantages and practicality of self-help treatments including computerised CBT (CCBT), i.e. self-help CBT using a programme on a website or on a computer without an online network, have been attractive, and it is believed that self-help CBT will be an efficacious intervention, especially for mild-to-moderate depression [15]. There are now even greater expectations of CCBT, owing to its increased potential due to technological progress in terms of interactivity, multimedia functions and flexibility [16]. Since the first randomised controlled trial (RCT) reported by Selmi in 1990 [17], the number of papers published on CCBT has increased markedly. Also, to date, there have been five meta-analyses [18-22] which specifically looked at the effect of CCBT on adult depression, and all of them found that CCBT was of benefit with moderate effect sizes.

However, these systematic reviews cannot be considered to provide definitive and compelling support for CCBT due to both the lack of two significant perspectives from clinical implementation and the four issues of methodological validity.

From the perspective of clinical implementation, one point is that they have never dealt with functionality. Indeed, all of the three RCTs carried out to date on the cost-utility of CCBT [23-25] could not find significant QALY increases due to the lack of functional improvement. In addition to functionality, on the other hand, it seems to be also insufficient that even long-term effectiveness has not been meta-analysed at all.

Considering methodological aspects, it seems that past meta-analyses have crucial limitations in quality. The first issue is the heterogeneity of both targeted disorders and intervention. In the former, three meta-analytic studies [18,19,21] dealt with depression and other disorders (such as anxiety disorders) as common target disorders, implying a critical bias in the results due to considerable diagnostic heterogeneity, as CBT for depression and that for other disorders, such as anxiety disorders, are theoretically different interventions. Indeed, in these three, the effect sizes of CCBT for anxiety were generally greater than CCBT for depression.

In contrast, two other meta-analyses [22,20] exclusively dealt with depression. Nevertheless, there still remain the heterogeneous problems of intervention. Gellatly et al. included not only ten CCBT research studies but also twenty nine studies with non-CCBT interventions, such as bibliotherapy, indicating that the results of this meta-analysis cannot be accepted as those of solely CCBT-intervention. Compared to this review, Andrews et al. used the largest number to date (eleven) of RCTs on depression-only CCBT. Even so, this meta-analysis included two inappropriate studies [17,26], as mentioned below under Results.

Secondly, the published systematic reviews have not paid due attention to the problem of dropouts. Some studies suggest that CCBT has higher attrition rates than other therapies. However, one meta-analysis by Waller and Gilbody [27] indicated that there was no significant difference between CCBT users and controls, but this meta-analysis had substantial diagnostic heterogeneity (two depression-specific studies and seven studies specific to other disorders). Since uneven attrition between or among intervention arms can be a significant cause of bias [28], more rigorous consideration needs to be given to this factor.

Thirdly, the published meta-analyses have not examined publication bias in the available literature on CCBT.

Lastly, there have been an increasing number of new studies published since the most recent systematic review.

Therefore, we conducted a meta-analysis of clinical effectiveness of single CCBT for adult depression, taking the above methodological factors into consideration with an additional evaluation of functional outcomes and long-term follow-up effects.

Methods

Identification and selection of studies

All RCTs completed and analysed by 11 July, 2011 were eligible for inclusion in this review. Five bibliographic databases were used [MEDLINE (1948 to July 2011), PsycINFO (1806 to July 2011), EMBASE (1980 to 2011), CENTRAL (Cochrane Library, 2011 latest issue), and CiNii (until July 2011)]. We also searched http://www.controlled-trials.com webcite. Multiple search terms were used (Appendix) and modified for each database, as necessary. The search was performed on 11 July, 2011.

We included 1) randomised trials 2) in which the effects of guided and unguided CCBT specific to depression 3) were compared with one or more control conditions 4) in individuals aged 18 years or older 5) with depression, and in which 6) reliable and standardised rating-scales were equally used both at baseline and follow-up. Also, we only included studies 7) with proper allocation, concealment, and single or greater blinding of outcome assessment; and 8) trials using medications or other psychotherapies were included. We excluded studies on 1) inpatients, because we excluded patients with severe symptoms from self-help intervention, and those with 2) comorbidities such as psychotic disorders, manic status, dementia and severe physical conditions. In fact, we had originally intended to distinguish between patients on waitlists from treatment as usual (TAU), because we considered there to be restrictions on administration of medications to patients on waitlists. Nevertheless, the proportion of subjects taking medication at waitlist baseline was very similar to that with TAU, and medication was mostly not controlled. Therefore, we decided to group together both of these, and checked the influence of this factor on outcomes through a subgroup analysis. This grouping seemed to be justified because the above past five meta-analyses had treated data likewise. Studies had to have a primary endpoint including a measure of depression at the outcome assessment immediately after intervention and at long follow-up (if applicable). We defined long follow-up as follow-up where the final assessment was more than six months after treatment, because this is a recovery period associated with low future recurrence of depression [29]. Function at post-treatment and the number of total dropouts were adopted as secondary endpoints.

Meta-analyses

Intervention effects were expressed using various types of rating scales for common outcomes, thus the effect sizes using standardized mean differences (SMDs) with 95% confidence intervals for post-treatment were computed, and then incorporated into the meta-analysis and presented with 95% confidence intervals. Where trials used a number of different tools to assess depression, we included the main outcome measure following our hierarchy, including the primary endpoint or endpoint first reported in the results.

Statistical heterogeneity was evaluated through a SMD forest plot. Cochrane’s Q statistic (chi-squared test) was performed with a significance level of 0.10. Furthermore, the I-squared (I [2]) statistic for heterogeneity was also used for confirmation of Cochrane’s Q statistic. A random-effect model was selected due to the large heterogeneity of each clinical design and participants. All meta-analyses were performed using Review Manager (RevMan ver. 5.1). Subgroup analyses were performed for the type of control (Waitlist and TAU). Also, we re-evaluated the clinical effectiveness through a sensitivity analysis by excluding Beck Depression Inventory I, BDI- I [30], and II [31], or according to the difference in attrition rates and imputation techniques. The reason for the former is that, particularly in CCBT studies primarily relying on self-rating scales, measurement bias is suspected due to differences between the scales employed, and it is necessary to avoid this giving rise to underestimation. The reason for the latter is that high dropout rates were expected [27], thus we also performed an analysis excluding research with attrition rates higher than 20% (such rates probably had an influence on the results irrespective of ITT according to Cochran handbook) [32], significantly higher dropout RRs or non-modern imputation processing.

A funnel plot was used as a test of the main outcome to detect publication or reporting bias through visual inspection. Begg’s [33] and Egger’s [34] tests were also conducted for statistical checking. When a significant small study effect was noted, we assessed its influence through the trim-and-fill method [35].

It seems that adequate missing value management is useful in carrying out appropriate CCBT evaluation, because we expect overall attrition rates to be high. Although it was physically unfeasible for us to collect all the original data without imputation, in the present meta-analysis we considered the potential impact on the review result through a sensitivity analysis in terms of the influence of imputation. Also, the modern imputation was defined as an imputation needing more complex processing than classic and comparatively simple imputations such as last observation carried forward (LOCF) or mean imputation (MI).

Results

Characteristics of included studies

Out of 4,888 studies initially screened, following the process shown in Figure 1, fourteen were identified as relevant to the investigation of the clinical-effectiveness of CCBT as the sole intervention (Table 1). All continuous mean and SD values of post-treatment primary outcome data were available, and data from all 14 were finally used for 16 comparisons in the meta-analysis, as mentioned below (Table 2). In this inclusion process, we excluded two significant studies by Selmi et al. [17] and Wright et al. [26] that were included in all of the five prior meta-analyses mentioned above, because we ultimately judged one trial by Selmi et al. to have been conducted without an ITT analysis, proper allocation concealment and random sequence generation. The other, by Wright et al., was not regarded as being a study on self-help, due to highly intensive assistance of eight standard CBT sessions.

thumbnailFigure 1. Process of study selection: PRISMA flow diagram.

Table 1. Selected characteristics of randomised controlled studies examining the effect s of CCBT for adult depression

Table 2. Data and analyses

There were more women than men subjects, and the mean age ranged from 22.6 years (Clarke 2009) to 55 years [44] (see the Characteristics of included studies table). The majority of CCBT programmes were based on standard CBT, while combined CBT with other therapies was used in two trials. All studies used a self-reported measure of depression as their primary outcome, and eight trials mainly used BDI. All studies stated that allocation concealment and ITTs were adequately performed. There were follow-up data beyond six months after the interventions for 7 trials, one study (Titov 2009) [48] finished the follow-up of only the intervention group, and thus control data were not available. Another study [43] presented the results of the follow-up at 18 months by transforming a waiting-list group into an intervention group after a first-phase trial, and therefore could not be included in the current meta-analysis. Ten clinical-effectiveness studies recorded functional change as well as mood. Also, all 14 studies showed the post-intervention attrition rate. Two studies (Clarke 2005 [49] and Titov 2009 [48]) that each had three branches in the original research were respectively divided into two comparisons, and they were identified as Clarke 2005a and Clarke 2005b (post-card reminder or telephone reminder with CCBT) in the former, and as Titov 2009a and Titov 2009b (either technician-assisted or clinician-assisted CCBT) in the later, thereby giving a total of 16 comparisons from 14 original studies for the meta-analysis.

Primary and secondary outcomes

We included 16 comparisons (2807 participants) in terms of the reduction in depression symptoms between CCBT and controls following treatment (Figure 2). The pooled SMD was −0.48 [95% CI −0.63 to −0.33], indicating a significant moderate effect. Reduction in depression symptoms at long follow-up (Figure 3): The pooled SMD from the five studies (976 participants) that provided long-term follow-up data was −0.05 [95% CI −0.19 to 0.09], indicating no significant difference between the groups. Improvement in function at post-treatment (Figure 4): The pooled SMD from the twelve comparisons that provided data on function at post-treatment was −0.05 [95% CI −0.31 to 0.22], indicating no significant difference between two groups. In addition, the pooled relative risk from all trials providing data on attrition at post-treatment (Figure 5) was 1.68 [95% CI 1.31 to 2.16], indicating a significant difference between the two groups.

thumbnailFigure 2. Comparison: CCBT versus control, Outcome: Reduction in depression symptoms at post-treatment.

thumbnailFigure 3. Comparison: CCBT versus control, Outcome: Reduction in depression symptoms at long follow-up.

thumbnailFigure 4. Comparison: CCBT versus control, Outcome: Improvement in function at post-treatment.

thumbnailFigure 5. Comparison: CCBT versus control, Outcome: Relative risk of attrition at post-treatment.

Subgroup analyses

We attempted to analyse the differences in effects between studies where the control was a waiting list and those where it was TAU (Figure 6). The control was a waitlist in nine comparisons [36,37,41,44-48], while it was TAU in seven comparisons [38-40,42,43,49]. The pooled SMD for waitlist-controlled trials was −0.63 [95% CI −0.83 to −0.45], indicating a moderate effect. By contrast, the pooled SMD for TAU-controlled trials was −0.23 [95% CI −0.37 to −0.09], indicating a small effect. There was a significant difference between two groups.

thumbnailFigure 6. CCBT versus control: subgroup analyses – type of control (Waitlist and TAU), Outcome: Reduction in depression symptoms at post-treatment.

Sensitivity analyses

Sensitivity analyses were conducted as show below.

Rating scales except BDI

For the seven comparisons that employed neither BDI -I nor -II as the primary outcome measure [37-39,45,47,49], the SMD was −0.32 [95% CI −0.48 to −0.17], indicating a small clinical effect in favour of CCBT.

Acceptable attrition rate (<20%)

For the seven comparisons that were reported to have an acceptable dropout rate (<20%) [37,40,43,45,46,48], the SMD was −0.59 [−0.85, -0.34], showing a moderate clinical effect in favour of CCBT.

Imputation techniques

For the nine comparisons without a significant difference in attrition rate between intervention and control at post-treatment [37,39,40,42-44,46,48], the SMD was −0.50 [95% CI −0.73 to −0.27], showing a moderate clinical effect in favour of CCBT. On the other hand, for the eight comparisons from seven trials with modern imputation techniques [38,39], [43-45,47,49], the SMD was −0.34 [95% CI −0.51 to −0.18], indicating a small clinical effect (of borderline significance) in favour of CCBT.

Publication bias

We explored publication bias, using a Funnel plot (Figure 7). The plot can be seen to be asymmetric, indicating a relationship between intervention effect and study design. In particular, this asymmetry suggests a publication bias toward larger effect size in smaller studies, since there was a marked concentration of studies shown on the left side in the lower part of the plot. It was inferred that smaller studies with larger effect sizes were more likely to be published, and thus they had a higher probability of demonstrating statistical significance. Also, there was significant statistical evidence for study bias using both Begg’s test (p = 0.009) and Egger’s test (p = 0.01). Using the trim and fill analysis, the SMD was −0.32 [95% CI −0.49 to −0.16].

thumbnailFigure 7. Funnel plot of comparison: CCBT versus control, outcome: reduction in depression symptoms at post-treatment. White circles: included comparisons. Black circles: imputed comparisons using the trim-and-fill methods. White diamond: pooled observed standard mean difference. Black diamond: pooled imputed standard mean difference.

Discussion

We analysed 16 comparisons from 14 publications, targeting the largest size and highest quality meta-analysis. Consequently, this review found that CCBT has apparently a moderate post-treatment effect size (SMD −0.48, 95% CI −0.63 to −0.33) for adult depressive symptoms compared with control conditions, indicating almost the same result as those of past meta-analyses. Nevertheless, we further found a possibility that this result may have to be re-considered downward in terms of practical implementation and research methodology. Thus, we would like to examine these two aspects.

Considering the lack of the endurance of effectiveness, functional improvement and the high dropout rate, our result inevitably casts doubt on the actual practicability of the current CCBT for depression.

To begin with, the attenuation of long-term effectiveness with CCBT seems to be a serious issue from a clinical point of view. In fact, although one of the past reviewers [20] implied this tendency, long follow-up has not been clearly reviewed in meta-analyses until today. In this context, ours is the first review to meta-analyse long follow-up outcomes. This was paradoxical from the perspective that it has been reported that the effect of standard face-to-face CBT on depression does not usually attenuate sharply after intervention even without maintenance sessions. For example, the latter view has been advocated by the Annual Review of Psychology [50], which mentions that the effectiveness of CBT appears to be at least more enduring than that of antidepressants for depressive patients. However, it is still unclear why such variance could arise according to differences in modality, while van Londen et al. raised this question in the context of bibliotherapy [51].

In addition to long follow-up outcomes, it also has not been meta-analysed until our study about whether CCBT can contribute to functional improvement, even though this outcome is critically important in view of evaluating cost-utility, which is referred to as a distinctive advantage of CCBT [15]. In our analysis, CCBT intervention did not provide a significant effect in terms of function. There are a few possible reasons for this. Firstly, current CCBT may not be fundamentally good enough to improve function. The attainment of social functions such as returning to work has been commonly recognised as being more difficult than simply reducing depressive symptoms [52]. Secondly, we may have to consider the scale-sensitivity of function. Revicki et al. also referred to the property of generic measures that improvements in those scores are less sensitive in less depressive patients [53]. They suggested that such generic scales were even more unlikely to change amongst mild-to-moderate depressive patients than in severe depressive patients, often resulting in little change in utility and problematic utility assessment.

The third issue with practical implementation is that more than half of included studies had high overall dropout rates. It is clear that the higher dropout is unavoidable, especially for depression remediation, in that poor motivation is one of the fundamental symptoms. Indeed, even in the NHS, the dropout rate from CCBT is also high, with up to 50% of users starting the programme for depression not completing it, and it seems that this needs to be addressed as a serious issue [54].

Despite the above substantial limitations of CCBT, it is still used on the premise that it is significantly effective, at least as measured immediately following treatment with it. However, by addressing methodological issues, our analysis further revealed some findings that may raise a more fundamental question of whether CCBT is really effective for adult depression even following treatment.

The first finding is the ambiguous definition of control conditions. In all previous systemic reviews of CCBT, there was little clarification of the influence of grouping results from studies with TAU and the waitlist as controls. Unlike research on medications or psychotherapy, all RCTs on CCBT effectively did not restrict the usage of medications for waitlist groups. Therefore, we had held that this confusion between groups without sufficient presentation is a considerable problem, and set up a protocol to separate subjects on waitlists from those undergoing TAU. However, we found that the proportion of patients taking medication at baseline for TAU groups was in the range from 0% to 76%, and the range for control groups was from 37% to 74%. When considering the virtually undistinguishable rates of medication intake, we concluded that it was difficult to clearly separate TAU from waitlist data, and that is why we classified TAU and waitlist subjects into the same control group in a post-hoc decision, adding a subgroup analysis on the influence of doing this.

In the subgroup analysis, our results showed that the effects were significantly greater when the control group was a waitlist as opposed to TAU. Only a meta-regression [20] had an identical finding to ours, although the analysis was conducted by using only four (10.2%) reliable studies with depression-specific CCBT intervention. In general, this type of difference seems rational because TAU is more therapeutically intensive than a waitlist. However, another likely cause is that the reason for this is due to the tendency for it to be fundamentally easier for an intervention group to indicate a greater effect size relative to a waitlist than active placebo in psychotherapy research [55]. Therefore, it has been recently recommended to not use waitlists in research designs because of overestimation of intervention. Either way, this issue should be treated more carefully in general RCT settings as well as in RCTs of CCBT.

The second issue was that a high attrition rate was also considered to lead to a significant bias despite the conduct of ITT throughout all included studies. In practice, Cochran states that attrition rates higher than 20% may even affect outcomes analysed using ITT [32]. Also, extremely uneven attrition between or among branches of intervention can be an impermissible cause of bias [28]. Only one meta-analysis by Waller and Gilbody has dealt with this attrition issue, finding that subjects treated with CCBT dropped out approximately twice as frequently as control subjects, but this finding was not statistically significant [27].

In relation to the high dropout rate, we focused on the fact that a variety of imputation techniques were implemented for ITT in order to cover attrition in each study, but there was no research on CCBT which examined this risk by this kind of imputation. Rickels and Schweizer mentioned that ITT takes account of dropouts, usually by LOCF [56]. However, Shao et al. [57] and Unnebrink et al. [58] claim that old-type imputations, such as LOCF, mean imputation and worst observation carry-forward (WOCF), can cause significant differences in results when the attrition rate is higher than 20%. By contrast, modern imputation can be thought as being more appropriate. Moreover, there can be significant differences even among imputations, and if so, this issue is serious for research especially where there is a high level of attrition. For example, Warmerdam demonstrated that newer imputation led to significantly different results [25]. Therefore, we investigated the probability of bias due to the method of imputation. In fact, when only trials with modern imputation techniques were included, the effect size decreased from moderate to mild. The influence of imputation has not been seriously discussed in psychotherapy, including self-help. In particular, research on CCBT should give more consideration to this because of its high attrition rate relative to other psychotherapies.

Thirdly, our study was the first to detect significant publication bias specific to CCBT, and this suggested the necessity of careful re-consideration in evaluating the usefulness of CCBT. Indeed, the trim-and-fill method suggested that the SMD reduced from −0.48 [95% CI −0.63 to −0.33] to −0.32 [95% CI −0.49 to −0.16], but still indicating significant effectiveness at least at post-treatment.

Finally, we cannot overlook the fact that there has been a remarkable dominance of self-rating scales used as the primary endpoints of past CCBT research. In our analysis, self-rating scales were used as the primary outcome in all studies. Although all the adopted scales were academically reliable as screening tools, excessive expectations as to self-rating measures could lead to significant bias in the results because self-report ratings from depressed patients are not necessarily a reliable or definitive estimate of the severity especially during the acute phase including before symptomatological improvement [59].

Our sensitivity analysis also demonstrated that the effect size at post-treatment reduced from moderate to small without BDI. This can be explained by the characteristic of BDI that the score tends to be significantly influenced by cognitive factors rather than other instrumentals due to the different conceptualisation of depression among scales [60,61]. Indeed, CCBT is more likely to improve BDI scores than other measures probably because CCBT programmes strategically target cognitive change. Further, it has been also discussed that BDI is inaccurate as a way of appraising treatment outcomes due to overreactivity [62,63]. The frequent use of BDI can be theoretically justified as an efficacy study aiming at the efficacious maximisation of intervention. Even so, in terms of generalisablity, we may need to keep in mind the risk of overestimation when using self-rating scales, including BDI, when actually adopting CCBT for clinical use.

It is seemingly reasonable to expect that self-help CCBT can be a clinically- and cost-effective intervention, considering prior wholly-supportive reviews; however, the use of CCBT, even for mild to moderate depression, may be less practical and efficacious than believed at present. This can be supported by the poor results of three available cost-utility analyses of depression CCBT [23-25]. Nevertheless, it would be too extreme to conclude that CCBT is an inefficacious intervention for adult depression for a few reasons. Firstly, we could distinguish indications for which CCBT is appropriate. In fact, it has been reported that applying CCBT to patients with a personality suitable for it [64] or to those from a technologically-literate generation [19] may contribute to better outcomes. Also, further development of CCBT in terms of sophistication and attractiveness accompanying the rapid progress of information technology [65] might enhance the effectiveness of and adherence to CCBT, such as in the format of a therapeutic computer game [66].

Our review has a few limitations. Firstly, we should have ideally recalculated the effect size (SMD) of each outcome from the original research data in order to enhance the review quality [32]. However, we could not do this due to physical and time restrictions. Secondly, we could not include unpublished data or data from on-going trials even though we attempted to collect them using several ways.

Conclusion

This review found that CCBT seems to improve depressive symptoms at post-intervention among adults following treatment. However, the effect at long follow-up and the improvement of function were not significant, and a considerable dropout rate was also found. Also, there was significant publication bias and other influential methodological problems including with self-rating, control condition and imputation. This may imply a probability of overestimation of the effect of CCBT and the need to further improve it. Nevertheless, it is possible that we do not need to be too pessimistic about CCBT, since we might be just midway on a long journey of low intensity intervention, of which CCBT is one part [67]. More careful research is required for CCBT to develop more substantially.

Appendix: The search terms used and search process

MEDLINE, EMBASE

1. exp Depression/

2. exp Depressive Disorder/

3. (Depression or depressive or depressed).tw.

4. exp Dysthymic Disorder/

5. Dysthymia.mp.

6. exp Depressive Disorder, Major/

7. exp Affect/

8. exp Mental Disorders/

9. or/1-9

10. exp Internet/

11. exp Therapy, Computer-Assisted/

12. exp Cognitive Therapy/

13. exp Primary Health Care/

14. CCBT.mp.

15. exp Mental Health Services/

16. exp Software/

17. User manual.mp.

18. exp Internet/

19. exp Bibliotherapy/

20. Bibliotherapy.tw.

21. exp Cognitive Therapy/

22. Self-help.tw.

23. Medical informatics computing.mp.

24. exp Medical Informatics Computing/

25. exp Multimedia/

26. exp Computer-Assisted Instruction/

27. Computer assisted instruction.mp.

28. exp Decision Making, Computer-Assisted/

29. Computer-assisted.tw.

30. Computer&.tw.

31. Internet.tw.

32. Therapy, computer assisted.tw.

33. Web-based.mp.

34. Web-based.tw.

35. Beating the blues.mp.

36. Beating the blues.tw.

37. Overcoming depression.tw.

38. or/10-37

39. exp Psychotherapy/

40. psychotherapy.mp.

41. exp Psychotherapy, Multiple/

42. exp Psychotherapy, Group/

43. exp Psychotherapy, Brief/

44. (Cognitive adj2 therap$).tw.

45. ((Behaviour& or behavior&) adj2 therap&).tw.

46. Cognitive Therapy/

47. CBT.mp.

48. exp Psychoanalysis/

49. psychoanalysis.mp.

50. TAU.tw.

51. Treatment as usual.tw.

52. Standard treatment.tw.

53. Medication.tw.

54. exp Drug Therapy/

55. Pharmacotherapy.mp

56. Control.tw.

57. Supportive therapy.mp.

58. Supportive therapy.tw.

59. exp Waiting Lists/

60. Waiting list$.tw.

61. exp Placebos/

62. placebo.tw.

63. or/39-63

64. exp Randomized Controlled Trial/

65. Randomized controlled trial.mp.

66. Randomized controlled trial$.tw.

67. exp Randomized Controlled Trials as Topic/

68. RCT.mp.

69. RCT&.tw.

70. Random Allocation.tw.

71. or/64-71

72. 9 and 38 and 63 and 71

PsychINFO

1. exp "Depression (Emotion)"/

2. exp Major Depression/

3. Depression.mp.

4. (Depression or depressive or depressed).af.

5. dysthymia.mp. or exp Dysthymic Disorder/

6. or/1-5

7. Internet.mp. or exp Internet/or exp Internet Usage/

8. exp Cognitive Therapy/

9. exp Computer Applications/

10. exp Computers/

11. exp Computer Assisted Therapy/

12. exp Computer Assisted Instruction/

13. Therapy, Computer-Assisted.mp.

14. exp Computer Software

15. Computer assisted.af.

16. exp Computer Assisted Therapy/

17. Ccbt.mp.

18. exp Self Help Techniques/

19. ccbt.af.

20. Bibliotherapy.mp. or exp Bibliotherapy/

21. exp Online Therapy/or self-help.mp.

22. Self-help.af.

23. Web-based.mp.

24. Beating the blues.af.

25. or/7-24

26. exp Brief Psychotherapy/

27. exp Group Psychotherapy/

28. exp Psychotherapy/

29. exp Analytical Psychotherapy/

30. exp Individual Psychotherapy/

31. exp Interpersonal Psychotherapy/

32. exp Psychodynamic Psychotherapy/

33. tau.af.

34. treatment as usual.af.

35. standard treatment.af.

36. supportive therapy.af.

37. psychotherapy.af.

38. medication.mp. or exp Drug Therapy/

39. Pharmacotherapy.mp.

40. antidepressant$.tw.

41. Control&.af.

42. Waiting list$.af.

43. placebo.af.

44. or/26-44

45. exp Clinical Trials/or exp Intervention/or randomized controlled trial.mp.

46. randomised controlled trial$.af.

47. rct.mp.

48. random allocation.af.

49. or/45-49

50. (Cognitive adj2 therap$).af.

51. ((Behaviour& or behavior&) adj2 therap&).af.

52. Cognitive Therapy.mp.

53. exp Cognitive Therapy/

54. exp Cognitive Therapy/

55. exp Cognitive Behavior Therapy/

56. cbt.mp.

57. or/50-56

58. 6 and 25 and 44 and 49 and 57

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MS worked on the whole of this study throughout the process of conceptualising and planning this research, conducting and selecting data, analysing the results and finally writing this paper as a corresponding author. SY carried out significant fundamental work, contributing to the design of the study, data collection and selection. SH also carried out significant work mainly in the process of data collection and selection. TF made a considerable contribution mainly to parts of advanced statistical analyses such as meta-analysing and processing publication bias. PM contributed to the whole work from initial idea generation through designing the protocol to checking the final version as MS’s supervisor. All authors read and approved the final manuscript.

Authors’ information

MS MD, PhD is a visiting researcher of King’s College London Institute of Psychiatry whose specialty is modern psychotherapies including CBT or interpersonal psychotherapy, and who is engaged as a supervisor of CBT trainees in Japan. SY PSW PhD is a flexible researcher at the National Institute of Mental Health, National Center of Neurology and Psychiatry (NCNP) in Japan. His specialty is psychiatric rehabilitation and thus he is highly interested in low-intensity psychological intervention, including the usefulness of CCBT and bibliotherapy. SH MA is a lecturer at the Department of Psychology and Humanities, Edogawa University College of Sociology. Within his research, he has been engaged in self-help intervention for Japanese employees. Mitsuhiro Sado MD MSc is an instructor at Keio University School of Medicine whose major area of interest is the economic evaluation of psychological interventions such as CBT. He has a significant interest in CCBT, especially from the perspective of cost-effectiveness. TF MD PhD is a professor at Kyoto University Graduate School of Medicine/School of Public Health. He is not only a leading CBT supervisor in Japan but also a prominent epidemiologist in the country. PM PhD is a professor at the Centre for the Economics of Mental and Physical Health (CEMPH) at the Institute of Psychiatry King's College London. His specialty is the economic evaluation of mental health, and he has actually evaluated the cost-effectiveness of CCBT in the UK. http://www.iop.kcl.ac.uk/staff/profile/default.aspx?go=10869 webcite

Acknowledgements

We are grateful to Professor Mathew Hotopf, Dr William E Lee and Dr Kimio Yoshimura for their great help, including in statistical analysis.

References

  1. Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B: Depression, chronic diseases, and decrements in health: results from the World Health Surveys.

    Lancet 2007, 370(9590):851-858. PubMed Abstract | Publisher Full Text OpenURL

  2. Busch FN, Sandberg LS: Combined Treatment of Depression.

    Psychiatric Clinics of North America 2012, 35(1):165-179. PubMed Abstract | Publisher Full Text OpenURL

  3. Pampallona S, Bollini P, Tibaldi G, Kupelnick B, Munizza C: Combined pharmacotherapyandpsychological treatment for depression: a systematic review.

    Arch Gen Psychiatry 2004, 61(7):714-719. PubMed Abstract | Publisher Full Text OpenURL

  4. Beck AT: Cognitive therapy and the emotional disorders. Oxford, England: International Universities Press; 1976:356. OpenURL

  5. Department of Health: Treatment choice in psychological therapies and counselling. Evidence-based clinical practice guideline. In. London: DOH; 2001. OpenURL

  6. Butler AC, Chapman JE, Forman EM, Beck AT: The empirical status of cognitive-behavioral therapy: A review of meta-analyses.

    Clin Psychol Rev 2006, 26(1):17-31. PubMed Abstract | Publisher Full Text OpenURL

  7. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, Addis ME, Gallop R, McGlinchey JB, Markley DK, Gollan JK: Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression.

    J Consult Clin Psychol 2006, 74(4):658-670. PubMed Abstract | Publisher Full Text OpenURL

  8. Cuijpers P, Smit F, Van Straten A: Psychological treatments of subthreshold depression: a meta-analytic review.

    Acta Psychiatr Scand 2007, 115(6):434-441. PubMed Abstract | Publisher Full Text OpenURL

  9. IsHak WW, Ha K, Kapitanski N, Bagot K, Fathy H, Swanson B, Vilhauer J, Balayan K, Bolotaulo NI, Rapaport MH: The Impact of Psychotherapy, Pharmacotherapy, and Their Combination on Quality of Life in Depression.

    Harv Rev Psychiatry 2011, 19(6):277-289. PubMed Abstract | Publisher Full Text OpenURL

  10. Yonkers KA, Wisner KL, Stewart DE, Oberlander TF, Dell DL, Stotland N, Ramin S, Chaudron L, Lockwood C: The management of depression during pregnancy: a report from the American Psychiatric Association and the American College of Obstetricians andGynecologists.

    Gen Hosp Psychiatry 2009, 31(5):403-413. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  11. Mergl R, Henkel V, Allgaier A-K, Kramer D, Hautzinger M, Kohnen R, Coyne J, Hegerl U: Are treatment preferences relevant in response to serotonergic antidepressants and cognitive-behavioral therapy in depressed primary care patients? Results from a randomized controlled trial including a patients’ choice arm.

    Psychother Psychosom 2011, 80(1):39-47. PubMed Abstract | Publisher Full Text OpenURL

  12. Cuijpers P, van Straten A, Smit F, Mihalopoulos C, Beekman A: Preventing the onset of depressive disorders: a meta-analytic review of psychological interventions.

    Am J Psychiatry 2008, 165(10):1272-1280. PubMed Abstract | Publisher Full Text OpenURL

  13. NICE: Common Mental Health Disorders: Identification and Pathways to Care. Clinical Guideline 123. London, UK: National Institute for Health and Clinical Excellence; 2011.

    [http://www.nice.org.uk webcite]

    OpenURL

  14. Lovell K, Richards D: Multiple Access Points and Levels of Entry (MAPLE): ensuring choice, accessibility and equity for cbt services.

    Behav Cogn Psychother 2000, 28(04):379-391. OpenURL

  15. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, Parry G, Rooney G, Sutcliffe P: Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation.

    Health Technol Assess 2006, 10(33):iii.

    xi-xiv, 1–168

    PubMed Abstract | Publisher Full Text OpenURL

  16. Kaltenthaler E, Parry G, Beverley C: Computerized cognitive behaviour therapy: a systematic review.

    Behav Cogn Psychother 2004, 32(01):31-55. Publisher Full Text OpenURL

  17. Selmi PM, Klein MH, Greist JH, Sorrell SP, Erdman HP: Computer-administered cognitive-behavioral therapy for depression.

    Am J Psychiatry 1990, 147(1):51-56. PubMed Abstract | Publisher Full Text OpenURL

  18. Andersson G, Cuijpers P: Internet-based and other computerized psychological treatments for adult depression: a meta-analysis.

    Cogn Behav Ther 2009, 38(4):196-205. PubMed Abstract | Publisher Full Text OpenURL

  19. Barak A, Hen L, Boniel-Nissim M, Shapira N: A Comprehensive Review and a Meta-Analysis of the Effectiveness of Internet-Based Psychotherapeutic Interventions.

    J Technol Human Serv 2008, 26(2–4):109-160. OpenURL

  20. Gellatly J, Bower P, Hennessy S, Richards D, Gilbody S, Lovell K: What makes self-help interventions effective in the management of depressive symptoms? Meta-analysis and meta-regression.

    Psychol Med 2007, 37(9):1217-1228. PubMed Abstract | Publisher Full Text OpenURL

  21. Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V: Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis.

    Psychol Med 2007, 37(03):319-328. PubMed Abstract | Publisher Full Text OpenURL

  22. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N: Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis.

    PLoS One 2010, 5(10):e13196. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  23. Gerhards SAH, de Graaf LE, Jacobs LE, Severens JL, Huibers MJH, Arntz A, Riper H, Widdershoven G, Metsemakers JFM, Evers SMAA: Economic evaluation of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.

    Br J Psychiatry 2010, 196:310-318. PubMed Abstract | Publisher Full Text OpenURL

  24. McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, Ilson S, Gray JA, Goldberg D, Mann A: Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial.

    Br J Psychiatry 2004, 185:55-62. PubMed Abstract | Publisher Full Text OpenURL

  25. Warmerdam L, Smit F, van Straten A, Riper H, Cuijpers P: Cost-utility and cost-effectiveness of internet-based treatment for adults with depressive symptoms: randomized trial.

    J Med Internet Res 2010, 12(5):e53. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  26. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, Otto MW: Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time.

    Am J Psychiatry 2005, 162(6):1158-1164. PubMed Abstract | Publisher Full Text OpenURL

  27. Waller R, Gilbody S: Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence.

    Psychol Med 2009, 39(5):705-712. PubMed Abstract | Publisher Full Text OpenURL

  28. Fewtrell MS, Kennedy K, Singhal A, Martin RM, Ness A, Hadders-Algra M, Koletzko B, Lucas A: How much loss to follow-up is acceptable in long-term randomised trials and prospective studies?

    Arch Dis Child 2008, 93(6):458-461. PubMed Abstract | Publisher Full Text OpenURL

  29. Thase ME, Sullivan LR: Relapse and Recurrence of Depression. A practical approach for prevention.

    CNS Drugs 1995, 4:261-277. Publisher Full Text OpenURL

  30. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J: An inventory for measuring depression.

    Arch Gen Psychiatry 1961, 4(6):561-571. PubMed Abstract | Publisher Full Text OpenURL

  31. Beck AT, Steer RA, Ball R, Ranieri WF: Comparison of Beck Depression Inventories-IA and-II in Psychiatric Outpatients.

    J Pers Assess 1996, 67(3):588-597. PubMed Abstract | Publisher Full Text OpenURL

  32. Intention to treat issues: Special topic in statistics.

    [http://www.mrc-bsu.cam.ac.uk/cochrane/handbook/ webcite]

    OpenURL

  33. Begg CB, Mazumdar M: Operating characteristics of a rank correlation test for publication bias.

    Biometrics 1994, 50(4):1088-1101. PubMed Abstract | Publisher Full Text OpenURL

  34. Egger M, Smith GD, Schneider M, Minder C: Bias in meta-analysis detected by a simple, graphical test.

    BMJ 1997, 315(7109):629-634. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  35. Duval S, Tweedie R: A nonparametric trim-and-fill method of accounting for publication bias in meta-analysis.

    J Am Stat Assoc 2000, 95:89-99. OpenURL

  36. Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, Ekselius L: Internet-based self-help for depression: randomised controlled trial.

    Br J Psychiatry 2005, 187:456-461. PubMed Abstract | Publisher Full Text OpenURL

  37. Christensen H, Griffiths KM, Jorm AF: Delivering interventions for depression by using the internet: randomised controlled trial.

    BMJ 2004, 328(7434):265. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  38. Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, Lynch F, Nunley S: Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program.

    J Med Internet Res 2002, 4(3):E14. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  39. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C: Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults.

    Cogn Behav Ther 2009, 38(4):222-234. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  40. de Graaf LE, Gerhards SAH, Arntz A, Riper H, Metsemakers JFM, Evers SMAA, Severens JL, Widdershoven G, Huibers MJH: Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.

    Br J Psychiatry 2009, 195(1):73-80. PubMed Abstract | Publisher Full Text OpenURL

  41. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M: Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.

    J Med Internet Res 2009, 11(2):e15. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  42. Perini S, Titov N, Andrews G: Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial.

    Aust N Z J Psychiatry 2009, 43(6):571-578. PubMed Abstract | Publisher Full Text OpenURL

  43. Ruwaard J, Schrieken B, Schrijver M, Broeksteeg J, Dekker J, Vermeulen H, Lange A: Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up.

    Cogn Behav Ther 2009, 38(4):206-221. PubMed Abstract | Publisher Full Text OpenURL

  44. Spek V, Nyklicek I, Smits N, Cuijpers P, Riper H, Keyzer J, Pop V: Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial.

    Psychol Med 2007, 37(12):1797-1806. PubMed Abstract | Publisher Full Text OpenURL

  45. van Straten A, Cuijpers P, Smits N: Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial.

    J Med Internet Res 2008, 10(1):e7. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  46. Vernmark K, Lenndin J, Bjarehed J, Carlsson M, Karlsson J, Oberg J, Carlbring P, Eriksson T, Andersson G: Internet administered guided self-help versus individualized e-mail therapy: A randomized trial of two versions of CBT for major depression.

    Behav Res Ther 2010, 48(5):368-376. PubMed Abstract | Publisher Full Text OpenURL

  47. Warmerdam L, van Straten A, Twisk J, Riper H, Cuijpers P: Internet-based treatment for adults with depressive symptoms: randomized controlled trial.

    J Med Internet Res 2008, 10(4):e44. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  48. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K: Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance.

    PLoS ONE [Electronic Resource] 2010, 5(6):e10939. Publisher Full Text OpenURL

  49. Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, Lynch F, Nunley S, Gullion C: Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders.

    J Med Internet Res 2005, 7(2):e16. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  50. Hollon SD, Stewart MO, Strunk D: enduring effects for cognitive behavior therapy in the treatment of depression and anxiety.

    Annu Rev Psychol 2006, 57(1):285-315. PubMed Abstract | Publisher Full Text OpenURL

  51. Van Londen L, Molenaar RP, Goekoop JG, Zwinderman AH, Rooijmans HG: Three- to 5-year prospective follow-up of outcome in major depression.

    Psychol Med 1998, 28(3):731-735. PubMed Abstract | Publisher Full Text OpenURL

  52. Kennedy N, Foy K, Sherazi R, McDonough M, McKeon P: Long-term social functioning after depression treated by psychiatrists: a review.

    Bipolar Disord 2007, 9(1–2):25-37. PubMed Abstract | Publisher Full Text OpenURL

  53. Revicki DA, Wood M: Patient-assigned health state utilities for depression-related outcomes: differences by depression severity and antidepressant medications.

    J Affect Disord 1998, 48(1):25-36. PubMed Abstract | Publisher Full Text OpenURL

  54. MacLeod M, Martinez R, Williams C: Cognitive Behaviour Therapy self-help: who does it help and what are its drawbacks?

    Cambridge Univ Press 2009, 37(1):61-72. OpenURL

  55. Watanabe N, Hunot V, Omori IM, Churchill R, Furukawa TA: Psychotherapy for depression among children and adolescents: a systematic review.

    Acta Psychiatr Scand 2007, 116(2):84-95. PubMed Abstract | Publisher Full Text OpenURL

  56. Rickels K, Schweizer E, Csanalosi I, Case G, Chung H: Long-term treatment of anxiety and risk of withdrawal. Prospective comparison of clorazepate and buspirone.

    Arch Gen Psychiatry 1988, 45(5):444-450. PubMed Abstract | Publisher Full Text OpenURL

  57. Shao J, Zhong B: Last observation carry-forward and last observation analysis.

    Stat Med 2003, 22(15):2429-2441. PubMed Abstract | Publisher Full Text OpenURL

  58. Unnebrink K, Windeler J: Intention-to-treat: methods for dealing with missing values in clinical trials of progressively deteriorating diseases.

    Stat Med 2001, 20(24):3931-3946. PubMed Abstract | Publisher Full Text OpenURL

  59. Prusoff Ba KGLPES: Concordance between clinical assessments and patients’ self-report in depression.

    Arch Gen Psychiatry 1972, 26(6):546-552. PubMed Abstract | Publisher Full Text OpenURL

  60. Moran PW, Lambert JM: A review of current assessment tools for monitoring changes in depression. In The Measurement of Psychotherapy Outcome in Research and Evaluation. Edited by Lambert MJ, Christensen ER, De Julio SS. New York: Wiley; 1983:263-303. OpenURL

  61. Lambert MJ, Hatch DR, Kingston MD, Edwards BC: Zung, Beck, and Hamilton Rating Scales as measures of treatment outcome: a meta-analytic comparison.

    J Consult Clin Psychol 1986, 54(1):54-59. PubMed Abstract | Publisher Full Text OpenURL

  62. Moran PW: A Review of Current Assessment Tools for Monitoring Changes in Depression. Brigham Young University: Department of Psychology; 1982. OpenURL

  63. Edwards BC, Lambert MJ, Moran PW, McCully T, Smith KC, Ellingson AG: A meta-analytic comparison of the Beck Depression Inventory and the Hamilton Rating Scale for Depression as measures of treatment outcome.

    Br J Clin Psychol 1984, 23(2):93-99. PubMed Abstract | Publisher Full Text OpenURL

  64. Vangberg HCB, Lillevoll KR, Waterloo K, Eisemann M: Does personality predict depression and use of an internet-based intervention for depression among adolescents?

    Depression Res Treatment 2012, 2012:11. OpenURL

  65. Mitchell N: Computerised CBT self-help for depression in Higher Education: Reflections on a pilot.

    Counselling and Psychotherapy Res 2009, 9(4):280-286. Publisher Full Text OpenURL

  66. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MFG: The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial.

    BMJ 2012, 18:344:e2598. OpenURL

  67. Clark DM: Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: the IAPT experience.

    Int Rev Psychiatry 2011, 23(4):318-327. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-244X/13/113/prepub