Email updates

Keep up to date with the latest news and content from BMC Psychiatry and BioMed Central.

Open Access Study protocol

Evaluating the efficacy and cost-effectiveness of web-based indicated prevention of major depression: design of a randomised controlled trial

Claudia Buntrock14*, David D Ebert12, Dirk Lehr1, Pim Cuijpers14, Heleen Riper134, Filip Smit456 and Matthias Berking12

Author Affiliations

1 Innovation Incubator, Division Health Training Online, Leuphana University, Lueneburg, Germany

2 Department of Psychology, Clinical Psychology and Psychotherapy, Philipps University Marburg, Marburg, Germany

3 GGZ inGeest, Regional Mental Health Service Centre, VU University Medical Centre, Amsterdam, The Netherlands

4 Department of Clinical Psychology and EMGO Institute for Health and Care Research, VU University, Amsterdam, The Netherlands

5 Department of Public Mental Health, Netherlands Institute of Mental Health and Addiction, Trimbos Institute, Utrecht, The Netherlands

6 Department of Epidemiology and Biostatistics, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands

For all author emails, please log on.

BMC Psychiatry 2014, 14:25  doi:10.1186/1471-244X-14-25


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-244X/14/25


Received:28 June 2013
Accepted:27 January 2014
Published:31 January 2014

© 2014 Buntrock et al.; licensee BioMed Central Ltd.

This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Major depressive disorder (MDD) imposes a considerable disease burden on individuals and societies. Web-based interventions have shown to be effective in reducing depressive symptom severity. However, it is not known whether web-based interventions may also be effective in preventing the onset of MDD. The aim of this study is to evaluate the (cost-) effectiveness of an indicated web-based guided self-help intervention (GET.ON Mood Enhancer Prevention) on the onset of MDD.

Methods/Design

A randomised controlled trial (RCT) will be conducted to compare the (cost-) effectiveness of the GET.ON Mood Enhancer Prevention training with a control condition exclusively receiving online-based psychoeducation on depression. Adults with subthreshold depression (N = 406) will be recruited from the general population and randomised to one of the two conditions. The primary outcome is time to onset of MDD within a 12-months follow-up period. MDD will be assessed according to DSM-IV criteria as assessed by the telephone-administered Structured Clinical Interview for DSM-IV (SCID). Time to onset of MDD will be assessed using life charts. Secondary outcomes include changes on various indicators of depressive symptom severity, anxiety and quality of life from baseline to post-treatment, to a 6-month and a 12-month follow up. Additionally, an economic evaluation using a societal perspective will be conducted to examine the intervention’s cost-effectiveness.

Discussion

This is one of the first randomised controlled trials that examines the effect of an indicated guided self-help web-based intervention on the incidence of major depression. If shown to be effective, the intervention will contribute to reducing the disease burden due to MDD in the general population.

Trial registration

German Clinical Trial Registration DRKS00004709.

Keywords:
Prevention; Indicated; Web-based; Subthreshold depression; Randomised controlled trial; Major depression; Cost-effectiveness

Background

Major depressive disorder (MDD) is highly prevalent [1,2] and has an incidence rate that is high relative to the number of prevalent cases [3]. The global point prevalence is estimated at 4.7% with an annual incidence rate of 3% [4]. Moreover, MDD is related to poorer quality of life [5,6], increased mortality [7], and substantial economic costs [8-10]. Currently, MDD ranks as the fourth disorder with the highest disease burden and is projected to be the leading cause of premature mortality and disability in high-income countries by 2030 [11].

The disease burden attributable to MDD might be reduced in two ways. The first approach is to treat existing cases. But despite the availability of effective MDD treatments, such as face-to-face cognitive-behaviour therapy, behavioural activation therapy or problem-solving therapy [12-14] less than half of depressed patients are recognised and treated [15]. Furthermore, it is estimated that approximately only one third of the disease burden caused by MDD could be averted assuming the hypothetical scenario of 100% coverage and full compliance to evidence-based treatments [16,17].

The second approach is reducing the development of new cases, which requires prevention. Preventive interventions might be capable of contributing to a further reduction in disease burden. A recent meta-analysis of 19 randomised controlled trials demonstrated that preventive interventions based on cognitive behaviour or interpersonal therapy were able to reduce the incidence of MDD by 22% [18]. One of these studies also showed the effectiveness of minimal contact cognitive-behavioural therapy for depression, based on the ‘Coping with Depression’ course [19].

Selective prevention aimed at high-risk groups and indicated preventive efforts that target individuals who show already detectable signs of MDD but who do not yet meet the diagnostic criteria for the disorder were particularly effective. Universal prevention aimed at the general population regardless of any risk profile showed only small effects.

Indicated prevention has been suggested to be more “efficient” than selective prevention [20]. “Efficiency” is here defined in terms of “impact”, that is the number of cases that would be prevented if the targeted risk indicator were fully blocked in the population and “effort” reflecting the number needed to be treated to prevent one new case of MDD. From a clinical point of view, indicated prevention is worthwhile for two reasons. First, subthreshold depression is a highly prevalent condition [21] and the burden posed on people affected and the community is considerable [22,23]. Second, subthreshold depression is a risk indicator for MDD, as the incidence rate of MDD is significantly increased in subjects with subthreshold depression compared to those without ranging from .15 in a general population up to .58 in general medical populations and high risk groups [24].

Despite their effectiveness, currently available indicated preventive face-to-face interventions face some serious limitations. These include (a) difficulties delivering interventions to the community en masse due to constraints in the workforce and health care resources [25,26], (b) limited availability of evidence-based interventions and clinicians in routine practice, especially in rural areas, and (c) low participation rates even if access to those interventions is at little or no costs [27]. Therefore, new approaches are needed to enhance the impact of indicated preventive interventions.

Using the Internet to provide (guided) self-help interventions may help to overcome some of the limitations of traditional preventive services. Web-based guided self-help strategies have several advantages over face-to-face approaches. These include: (a) interventions are more easily accessible at any time and place, (b) anonymity is assured when patients want to avoid stigmatisation, (c) a greater potential for the integration of acquired skills in daily life due to an emphasis on the participants’ active role in (guided) self-help interventions [28] (d) participants can work at their own pace and go through materials as often as they want, and (e) elimination of travel time and costs for both participants and clinicians. Finally (f), web-based interventions are easily scalable implying that only a small increase in therapeutic resources is required for reaching a greater proportion of the eligible population using these interventions. Thus, marginal costs per additional user are low due to an economies of scale effect.

Web-based interventions have shown to be well accepted by participants [29,30] and to be effective in the acute treatment of MDD [12] as well as in reducing depressive symptoms both in adults and adolescents [31,32]. However, although several web-based interventions are labelled as preventive interventions, i.e. ‘Colour your life’ [33,34], to the best of our knowledge, no study has yet investigated the (cost-) effectiveness of an indicated guided self-help web-based preventive intervention on the onset of diagnosed major depressive disorders.

Objective and research questions

The aim of this study is to evaluate whether a newly developed indicated guided self-help web-based intervention (GET.ON Mood Enhancer Prevention) is effective in preventing the onset of major depressive disorder when compared to an online psychoeducation-only control over a 12-months follow-up period. It is expected that depressive symptomatology will be reduced to a greater extend in the intervention group than in the control condition. It is hypothesised that GET.ON Mood Enhancer Prevention is superior in terms of cost-effectiveness, and QALY health gains compared to the psychoeducation-only control.

Methods/Design

Design

A two-armed randomised controlled trial (RCT) will be conducted to compare GET.ON Mood Enhancer Prevention with a psychoeducation-only control condition. Measurements will be taken at baseline, post-treatment (6 weeks), 6, and 12 months follow-up (after randomisation). Telephone-administered Semi-Structured Diagnostic Interviews (SCID) will be conducted at baseline, 6- and 12-months follow-up with the SCID/DSM-IV section for mood disorders (see Figure 1 for a detailed overview of the study design). The study is approved by the Medical Ethics Committee of the Philipps University Marburg (No. 2012-35 K).

thumbnailFigure 1. Overview of study procedure.

Inclusion and exclusion criteria

We will include adults (a) age 18 and above who (b) suffer from subthreshold depression (Centre for Epidemiological Studies Depression Scale (CES-D) ≥ 16) but do not meet DSM-IV criteria for a major depressive episode, (c) have Internet access, and (d) are willing to give informed consent. We will exclude subjects who (a) meet DSM-IV criteria for (a) current major depressive episode, (b) a bipolar disorder, or (c) a psychotic disorder. Additional exclusion criteria are: (d) a history of a major depressive disorder in the past six months (based on Kupfer [35]), (e) currently receiving psychotherapy for any kind of mental health problems, (f) being on a waiting list for psychotherapy, (g) receiving psychotherapy in the past six months, and (h) showing a notable suicidal risk, as indicated through a score greater 1 in the Beck Depression Inventory (BDI) Item 9 (“I feel I would be better off dead”).

Recruitment

Participants will be recruited from March 2013 to February 2014 via the GET.ON research website [36] that is announced in newspapers, on-air media and related websites. The research website provides information about the GET.ON Mood Enhancer Prevention training and details about the study. Additionally, a major health insurance company will support the recruitment by placing the information leaflet on its website. Individuals interested in participating in the study can apply online on the GET.ON research website by providing the research team with their e-mail address or by sending directly an e-mail to the research team. They do not need to be referred by their GP or other mental health care specialist.

Assessment of eligibility and randomisation

People who apply online for study participation receive an information letter via e-mail with detailed information about the study procedures. They will be informed that they can withdraw from the intervention and/or study at any time without any negative consequences. Applicants who still want to participate in the study are asked to complete online screening questionnaires including information about the severity of their depressive symptoms (CES-D ≥ 16), whether they are currently receiving any kind of treatment for any mental health disease, whether they are on a waiting list for such a treatment, whether they received such a treatment in the past six months, and whether they have a high suicidal risk (BDI Item 9 > 1). Subjects screened positive and who are willing to give informed consent are scheduled for a semi-structured clinical interview (SCID) conducted by telephone [37,38]. Participants meeting all of the inclusion and none of the exclusion criteria who have completed the baseline assessment and returned the informed consent form via post or e-mail will enter the study and will be randomly allocated to study conditions. Randomisation will take place at an individual level. Block randomisation will be used to ensure equity of ample sizes across study conditions. Random blocks will consist of two allocations each. The allocation will be done by an independent researcher not otherwise involved in the study using an automated computer-generated random numbers table.

Blinding

The research staff conducting the semi-structured clinical interviews at 6- and 12-months follow-up will be blinded to the condition the participants are assigned to. These include: (a) an explanation to the participants why it is important not to inform the interviewer about the condition they were assigned to; (b) a written reminder for the interviewer in the interview manual to ask the participant not to inform him/her about the randomisation status; (c) written and verbal reminders to the patient before each interview; and (d) a documentation after each assessment of whether or not the interviewer is still blind to treatment condition. With regard to the latter, the interviewer will be asked to guess each participant’s randomisation status and these guesses will be compared with the actual status. Cohen’s kappa will be computed to clarify whether hit rates differ from what can be expected from chance. In case of evidence for blinding break down, the interviewer will be changed to the second outcome interview.

Intervention

Get.ON mood enhancer prevention

The GET.ON Mood Enhancer Prevention training consists of six lessons. Participants are advised to do two lessons a week but at least one. Consequently, the training lasts 3 to 6 weeks. However, participants are not excluded from the intervention if they do not manage to complete one lesson a week. Lessons consist of text, exercises, and testimonials. Each lesson includes interactive elements such as audio and video clips. Audio sequences introduce relaxation exercises, whereas video clips are used to explain theoretical frameworks, such as the concept of behavioural activation, in a user-friendly way. A strong focus lies on transfer tasks (homework assignments) to integrate newly acquired strategies and techniques into daily life. As an optional component, participants can choose to receive a set of about 42 standardised text-messages supporting them to integrate the learned techniques into their everyday life. An example of such a text-message would be “Everyone has his own strategies to vanquish the inner temptation. What helps you?” In the beginning of each subsequent lesson, participants are invited to reflect on their experiences with the newly acquired skills. The contents are adaptively tailored to the specific needs of the individual participant by continuously asking participants to respond by choosing among various response options. Subsequent content is then tailored to the participant’s response. For example, participants are asked whether to work on an elective module or not and if so they can choose which module they want to work on.

GET.ON Mood Enhancer Prevention is based on elements from behaviour therapy (BT) [39] and problem-solving therapy (PST) [40]. These therapeutic elements are often found in psychological treatments for subthreshold depression [41]. Interventions using BT and the combination of BT and PST have been shown to effectively prevent the onset of major depressive disorder (i.e. [42,43]). In BT, a strong focus rests on daily pleasurable activity scheduling that is integrated in each lesson (Additional file 1: Screenshots of the GET.ON Mood Enhancer Prevention intervention page 3). The PST elements implemented in GET.ON Mood Enhancer Prevention have been used in various web-based interventions, such as the Dutch web-based “Alles onder Controle” course, which has been shown to be effective in reducing depressive symptomatology across several randomised controlled trials [44,45]. In the current study, PST consists of three steps. First, participants make a list of things that matter most to them in their lives. Second, participants list all their problems and worries and divide them into ‘manageable’ and ‘unmanageable’. Finally, they are invited to think about activities on how to solve the manageable problems. The main focus in problem-solving therapy is to tackle those problems that are manageable by means of a six-step procedure: (1) defining the problem, (2) defining the target state, (3) brainstorming about possible solutions and choosing the best one, (4) making a plan how to implement this solution, (5) actually putting the solution into practice, and (6) evaluating the outcome. Finally, participants make a plan for the future on how they are going to accomplish their goals and those things that are most important to them in their lives (Additional file 1 page 2). In addition to the BT and PST elements, in the last three lessons participants are offered three elective modules targeting sleep hygiene, relaxation techniques, and dealing with worrying thoughts, respectively.

Additional file 1. Screenshots of the GET.ON Mood Enhancer Prevention intervention.

Format: PDF Size: 310KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

During the training, participants are supported by an online-trainer. The total time a trainer spends on a participant is approximately two hours. Trained and supervised graduate students and health care professionals will provide guidance. Participants will communicate with their trainer trough the internal messaging function of the system on which GET.ON Mood Enhancer Prevention is implemented. The guidance provided by the trainers focus on supporting participants to work through the exercises.

Psychoeducational-only condition

The psychoeducational intervention is also web-based and it is implemented on the same platform as GET.ON Mood Enhancer Prevention. Psychoeducational interventions have been shown to be effective in reducing depressive symptoms [46]. In the current study, the psychoeducational intervention is based on the German S3-Guideline/National Disease Management Guideline Unipolar Depression [47]. It informs participants about the nature and evidence-based treatments of depression including information about symptoms and sources of help. They can go through the material as often as they want to. In this study, the psychoeducational intervention does neither require participants to do explicit homework assignments nor is any support by a trainer or other mental health care specialist offered to participants.

Sample size calculation

We assume an absolute risk reduction of at least 10% for the incidence of major depressive disorder (MDD) between intervention and control group as clinically relevant. Based on previous studies evaluating interventions directed at the prevention of MDD, we expect a mean incidence of MDD in the control group of 25% within the 12-months follow-up period [48-50]. Based on a power of 80%, an alpha of 0.05, a relative risk reduction of 40%, and an attrition rate of 20%, we will need 406 participants to demonstrate an absolute risk reduction of 10% between the groups using log rank survival analyses (calculated using PASS 12).

Outcome measurements

For an overview of assessment at baseline, post-treatment, 6-and 12-month follow-up see Table 1.

Table 1. Overview of outcome measurements

Primary outcome

The primary outcome is time to onset of MDD within a 12-months follow-up period. Major depressive disorder will be assessed according to DSM-IV criteria as assessed by the telephone-administered Structured Clinical Interview for DSM-IV (SCID) at 6 and 12 months [37,38]. The inter-rater agreement of the Axis I disorders is moderate to excellent [51]. The agreement between face-to-face and telephone SCID interviews as indicated by the kappa coefficient is considered to be excellent [52]. Time to onset of MDD will be assessed using life charts. Life events are recalled by using a calendar method after which the presence of depressive symptoms at each month during the follow-up period is determined. SCIDs will be conducted by trained psychologists who are blind to treatment condition. The interviews will be recorded to examine inter-rater reliability. Disagreement shall be solved by discussion and the agreed rating will be used for analysis. If this is not possible, the assessment will be rated by an experienced psychotherapist (gold standard) and this rating will be used for analysis.

Secondary outcomes

Self-report data will be collected using a secured online-based assessment system (AES, 256-bit encrypted).

Depressive symptomatology

The depressive symptom level will be assessed with the German version of the Center for Epidemiological Studies Depression Scale (CES-D) [53]. The CES-D is a self-report scale and consists of 20 items, each scored 0–3, covering four domains: depressive affect, somatic complaints/activity inhibition, positive affect, and interpersonal difficulties. The total score ranges from 0–60, with a higher score indicating more severe depressive symptoms. A cut-off of 16 is usually regarded as indicating clinically relevant depressive symptom severity. The reliability of the CES-D has been shown to be excellent (internal consistency of Cronbach’s α = .89) [53].

Quality of life

Health-related quality of life will be assessed with two multidimensional generic measures, i.e., the EuroQol [54] and the SF-12v1 Health Survey [55]. The EuroQol entails the EQ-5D and a visual analogue scale. The EQ-5D consists of five items covering five dimensions (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression), each of which is rated as causing ‘no problems’, ‘some problems’, or ‘extreme problems’. The SF-12v1 has 12 items covering eight health domains (physical functioning, role functioning (physical and emotional), bodily pain, general health, vitality, social functioning, and mental health). The SF-12 generates two summary scores, the physical and mental health summary scores, respectively.

Anxiety

Anxiety will be measured with the German version of the anxiety subscale of the Hospital Anxiety and Depression Scale (HADS-A) [56,57]. The anxiety subscale consists of seven questions and each is scored from 0-3 meaning that the total scores ranges from 0-21 where a score between 0-7 indicates no anxiety, between 8 and 10 possible anxiety, and above 11 or 12 a clinical anxiety disorder. Psychometric properties are well established (Cronbach’s α ranging from .63-.93) [58].

Problem-solving skills

Problem-solving ability (i.e., generalised appraisal, beliefs, expectancies, and emotional responses) will be measured with two subscales of the Social Problem-Solving Inventory-Revised (SPSI-R). The positive problem orientation (PPO) subscale will represent a constructive dimension whereas the negative problem orientation (NPO) subscale is viewed as a dysfunctional dimension. Both subscales have displayed strong psychometric properties in former studies (Cronbach’s α = .76; .83) [59].

Behavioural activation

Participants’ activation towards goals/values and pleasant activities and avoidance behaviours will be measured with the BADS-Short Form (BADS-SF) [60]. The BADS-SF entails 9 items comprising two subscales (activation and avoidance). The items are rated on a 7-point Likert-type scale. Higher scores indicate that the individual scores high on the area of interest. The BADS-SF shows good psychometric properties (Cronbach’s α = .82) [60].

Mastery (internal locus of control)

Internal locus of control will be measured with the Pearlin Mastery Scale [61]. The Pearlin Mastery Scale consists of 7 items and each is rated on a 4-point Likert scale. The higher the score, the more the individual perceives having control over situations (internal mastery). A lower score points to external mastery meaning that the individual generally has the feeling that things are out of his or her control. The psychometric properties of this scale are well established [61].

Worrying

Worrying will be assessed with the ultra-brief version of the Penn State Worry Questionnaire (PSWQ) [62]. The ultra-brief version consists of 3 items stemming from the standard version, with each item being rated on a 7-point scale. The total score range from 0-18 with higher scores indicating more worry. The ultra-brief version shows similar psychometric properties compared to the standard version (Cronbach’s α = .85) [62].

Insomnia severity

Insomnia severity will be measured because evidence shows that treating sleep problems can ease depressive symptoms. Insomnia severity will be assessed with the Insomnia Severity Index (ISI) [63]. The ISI measures the nature, severity, and impact of insomnia. It consists of 7 items; each is rated on a 5-point Likert scale resulting in a total score ranging from 0 to 28. Higher scores indicate more severe insomnia. The ISI is a valid and reliable instrument to detect cases of insomnia in a population-based sample. The internal consistency is excellent (Cronbach’ α = .90) [64].

Treatment credibility/patient expectancy

Training credibility and participants’ expectancy for improvement will be measured with the credibility and expectation questionnaire (CEQ). The CEQ consists of 6 items, which are rated on a 9- or sometimes 10-point Likert scale. The psychometric properties of the instrument are well established (Cronbach’s α = .86) [65].

Attitudes toward seeking professional psychological help

The influence of attitudes on mental health care service utilisation will be measured with the Attitudes Toward Seeking Professional Psychological Help Scale-SF (ATSPPH-SF) [66]. The ATSPPH-SF consists of 10 items that are rated on a 4-point Likert scale yielding a total score ranging from 0-30. High scores indicate positive treatment attitudes. The instrument showed good psychometric properties in a previous study. The internal consistency ranges from .82 to .84 [67].

Course evaluation

User satisfaction will be measured with a self-designed questionnaire that is based on the “Satisfaction with Psychotherapy” Questionnaire (ZUF-8, [68]), the German version of the Client Satisfaction Questionnaire (CSQ-8, [69]). This self-report measure consists of 8 items measuring the global client satisfaction with the web-based training. Previous research indicated a high internal consistency (Cronbach’s α = .91) [70].

Side-effects of psychotherapy

Side-effects of psychotherapy will be measured with the side-effects of psychotherapy inventory (INEP) [71]. The INEP consists of 15 items assessing any changes participants experienced after the completing of the web-based training in their social and/or work environment that they directly relate to their participation in the web-based training.

Key economic outcomes

Clinical endpoints In the cost-effectiveness analyses, the main outcome will be depression-free years gained. Depression-free years will be assessed by calculating the difference in follow-up lengths and the duration of any major depressive episode (i.e. period of time in weeks that a person met DSM-IV criteria). In the cost-utility analysis, quality-adjusted life years (QALYs) will be the clinical endpoint. QALYs will be obtained from the EQ-5D (EuroQol) and SF-6D (SF-12v1). The EuroQol will be used because it is a widely applied quality of life instrument and its reliability and validity is well established [54]. Theoretically, the EQ-5D generates 243 different health states. Index scores for each of these health states are available for various countries with “perfect health” and “death” being assigned values of 1 and 0, respectively. In this study, the index scores derived from a large general population sample in the UK will be used [72]. For the sensitivity analysis, the German index scores [73] will be applied. The EQ-5D, however, might suffer from a ceiling effect meaning that a large number of respondents suffering from mild manifestations of depressive symptoms report no problems. Hence, the SF-6D will also be used because it might be the more appropriate measure for milder conditions, i.e., subthreshold depression [74]. The SF-6D contains 6 dimensions (each with between 2 and 5 levels) and includes 7 items of the SF-12. The SF-6D generates 7500 different health states. Utility values will be derived using Brazier’s algorithm [75,76].

Costs Direct medical (i.e. health care service utilisation) and direct non-medical costs (i.e. all costs not directly related to medical services such as transportation) are measured with the TiC-P, which is a self-report questionnaire [77]. Indirect non-medical cost stemming from production losses due to absenteeism and presenteeism will be assessed with specific modules of the TiC-P. A catalogue of unit costs (i.e. inpatient, outpatient and rehabilitative services) and evaluation standards of the Association of Statutory Health Insurance Physicians [78] will be used to calculate the total care costs on an individual basis [79].

Statistical analyses

Clinical efficacy

The study will be conducted in agreement with the CONSORT statement. Differences in the time to onset of MDD will be analysed in a proportional log rank survival analysis over a follow-up period of 12 months after baseline. Time to onset will be the dependent variable in the survival analyses. Analyses will be done based on the intention-to-treat (ITT) principle. In addition, per-protocol analyses will be performed. For participants who are lost from the trial, available measurements will be used and then censored at the time of their last observation. Participants who miss the assessment at 6-months follow-up, but are then assessed at 12-months follow-up, will be asked about their current and past symptoms according to SCID diagnostic criteria since the diagnostic interview at baseline. This will enable us to assess the time to onset of a depressive episode and thus to censoring. One-sided tests will be used for testing unidirectional and two-sided tests for testing bidirectional hypotheses. For all analyses statistical significance will be set at p < .05. We will calculate the number needed-to-be-treated (NNT) with GET.ON Mood Enhancer Prevention to prevent one case of MDD as compared to the control group to estimate the clinical effect size.

In secondary analyses, hierarchical linear modelling will be used to assess differences in secondary outcomes, such as depressive symptom severity, anxiety, and quality of life. We will also explore the proportion of major depressive episodes in the intervention and control group by follow-up. For all mixed-model analyses, Cohen’s d will be calculated by standardising the differences between baseline and follow-up scores by the pooled standard deviation of baseline scores.

Economic analyses

It will be checked whether baseline differences exist between the intervention and control group. If necessary, statistical techniques will be used to correct for baseline differences [80]. In the cost-effectiveness analyses, the incremental cost-effectiveness ratio (ICER) will be stated as costs per depression-free years gained, whereas the ICER in the cost-utility analyses will represent the costs per quality-adjusted life year (QALY) gained. Bootstrapping will be used to test the robustness of the ICERs and quantify the uncertainty around the ratios that will be graphically represented on a cost-effectiveness plane. The bootstrapped ICERs will also be shown in a cost-effective acceptability curve disclosing the probability that GET.ON Mood Enhancer Prevention is cost-effective for a wide range of willingness-to-pay ceilings [81]. To test the robustness of the base-case findings, a multi-way sensitivity analysis will be done. An incremental net benefit regression analysis will be performed to ascertain which sub-groups benefit more from the intervention in terms of superior cost-effectiveness.

Discussion

Major depressive disorder is a highly prevalent disorder associated with a considerable loss of quality of life, increased mortality rates, and formidable economic costs. Due to limited accessibility and efficacy, treating existing cases only contributes to a partial reduction of disease burden. Thus, interventions preventing the onset of MDD should be used to complement treatment-focused interventions and further reduce the burden of this debilitating disorder. Available face-to-face preventive interventions face, however, limitations. Novel approaches are needed that go beyond the limits of traditional services. The Internet may attract people who do not participate in face-to-face interventions. Moreover, it potentially provides the opportunity to offer preventive interventions to the community en masse. To our knowledge, this is one of the first randomised controlled trials that examines the (cost-) effectiveness of a web-based intervention on the onset of major depression in subjects with subthreshold depression.

Limitations of this study include the following. Attrition is a common problem in web-based interventions [82]. However, providing guidance has been shown to reduce attrition rates [12] and the intensity of support offered in this study is considered to keep drop-out to a minimum. In addition, the psychometric properties of most of the secondary outcome measures used in this trial have not yet been tested in an online environment.

There are several strengths to this study. First, by conducting this trial a significant contribution to the literature is made as to the best of our knowledge no other study has yet been undertaken that investigates the effect on an indicated guided self-help web-based intervention on the incidence of MDD. Second, Semi-Structured Diagnostic Interviews (SCID) will be conducted two times within the 12-month follow-up period to assess the time to onset of depressive episodes. This frequency of assessments allows for a reasonable temporal precision of onset of depressive episodes according to DSM-IV criteria. Third, an economic evaluation will be performed alongside the randomised controlled trial. If shown to be effective, this web-based preventive intervention could be easily disseminated. As a low-threshold intervention, it would be better accepted among the target group. If shown to be cost-effective, GET.ON Mood Enhancer Prevention will be a valuable tool to efficiently reduce the disease burden attributable to MDD at population level.

Competing interests

Professor Berking is minority shareholder of Minddistrict GmbH, which will provide the platform for the web-based intervention.

Authors’ contributions

MB obtained funding for this study. MB, CB, JR, LB, DE, and DL contributed to the development of the GET.ON Mood Enhancer Prevention training. All authors contributed to the study design. FS contributed to the design of the economic evaluation study. CB drafted the manuscript. CB, DE, PC, FS, and MB contributed to the further writing of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The associated project of all authors is funded by the European Union (project number: EFRE: CCI 2007DE161PR001).

References

  1. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, De Girolamo G, Graaf R, Demyttenaere K, Gasquet I, Haro JM, Katz SJ, Kessler RC, Kovess V, Lepine JP, Ormel J, Polidori G, Russo LJ, Vilagut G, Almansa J, Arbabzadeh-Bouchez S, Autonell J, Bernal M, Buist-Bouwman MA, Codony M, Domingo-Salvany A, Ferrer M, Joo SS, Martinez-Alonso M, Matschinger H, et al.: Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project.

    Acta Psychiatr Scand Suppl 2004, 420:21-27. PubMed Abstract OpenURL

  2. Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshleman S, Wittchen HU, Kendler KS: Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: results from the national comorbidity survey.

    Arch Gen Psychiatry 1994, 51(1):8-19. PubMed Abstract | Publisher Full Text OpenURL

  3. Waraich P, Goldner EM, Somers JM, Hsu L: Prevalence and incidence studies of mood disorders: a systematic review of the literature.

    Can J Psychiatry 2004, 49(2):124-138. PubMed Abstract OpenURL

  4. Ferrari AJ, Somerville AJ, Baxter AJ, Norman R, Patten SB, Vos T, Whiteford HA: Global variation in the prevalence and incidence of major depressive disorder: a systematic review of the epidemiological literature.

    Psychol Med 2013, 43(3):471-481. PubMed Abstract | Publisher Full Text OpenURL

  5. Ustun TB, Ayuso-Mateos JL, Chatterji S, Mathers C, Murray CJ: Global burden of depressive disorders in the year 2000.

    Br J Psychiatry 2004, 184:386-392. PubMed Abstract | Publisher Full Text OpenURL

  6. Saarni SI, Suvisaari J, Sintonen H, Pirkola S, Koskinen S, Aromaa A, Lonnqvist J: Impact of psychiatric disorders on health-related quality of life: general population survey.

    Br J Psychiatry 2007, 190:326-332. PubMed Abstract | Publisher Full Text OpenURL

  7. Cuijpers P, Smit F: Excess mortality in depression: a meta-analysis of community studies.

    J Affect Disord 2002, 72(3):227-236. PubMed Abstract | Publisher Full Text OpenURL

  8. Berto P, D’Ilario D, Ruffo P, Di Virgilio R, Rizzo F: Depression: cost-of-illness studies in the international literature, a review.

    J Ment Health Policy Econ 2000, 3(1):3-10. PubMed Abstract | Publisher Full Text OpenURL

  9. Greenberg PE, Birnbaum HG: The economic burden of depression in the US: societal and patient perspectives.

    Expert Opin Pharmacother 2005, 6(3):369-376. PubMed Abstract | Publisher Full Text OpenURL

  10. Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, Beekman A: Costs of nine common mental disorders: implications for curative and preventive psychiatry.

    J Ment Health Policy Econ 2006, 9(4):193-200. PubMed Abstract | Publisher Full Text OpenURL

  11. Mathers CD, Loncar D: Projections of global mortality and burden of disease from 2002 to 2030.

    PLoS Med 2006, 3(11):e442. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  12. Richards D, Richardson T: Computer-based psychological treatments for depression: a systematic review and meta-analysis.

    Clin Psychol Rev 2012, 32(4):329-342. PubMed Abstract | Publisher Full Text OpenURL

  13. Cuijpers P, van Straten A, Andersson G, van Oppen P: Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies.

    J Consult Clin Psychol 2008, 76(6):909-922. PubMed Abstract | Publisher Full Text OpenURL

  14. Cuijpers P, van Straten A, Warmerdam L, Andersson G: Psychological treatment of depression: a meta-analytic database of randomized studies.

    BMC Psychiatry 2008, 8:36. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  15. Tylee A, Jones R: Managing depression in primary care.

    BMJ 2005, 330(7495):800-801. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  16. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H: Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders.

    Br J Psychiatry 2004, 184:526-533. PubMed Abstract | Publisher Full Text OpenURL

  17. Chisholm D, Sanderson K, Ayuso-Mateos JL, Saxena S: Reducing the global burden of depression: population-level analysis of intervention cost-effectiveness in 14 world regions.

    Br J Psychiatry 2004, 184:393-403. PubMed Abstract | Publisher Full Text OpenURL

  18. Cuijpers P, van Straten A, Smit F, Mihalopoulos C, Beekman A: Preventing the onset of depressive disorders: a meta-analytic review of psychological interventions.

    Am J Psychiatry 2008, 165(10):1272-1280. PubMed Abstract | Publisher Full Text OpenURL

  19. Willemse GR, Smit F, Cuijpers P, Tiemens BG: Minimal-contact psychotherapy for sub-threshold depression in primary care: randomised trial.

    British J Psychiatry 2004, 185:416-421. Publisher Full Text OpenURL

  20. Schoevers RA, Smit F, Deeg DJ, Cuijpers P, Dekker J, van Tilburg W, Beekman AT: Prevention of late-life depression in primary care: do we know where to begin?

    Am J Psychiatry 2006, 163(9):1611-1621. PubMed Abstract | Publisher Full Text OpenURL

  21. Cuijpers P: Minor depression: risk profiles, functional disability, health care use and risk of developing major depression.

    J Affect Disord 2004, 79(1–3):71-79. PubMed Abstract | Publisher Full Text OpenURL

  22. Backenstrass M, Frank A, Joest K, Hingmann S, Mundt C, Kronmuller KT: A comparative study of nonspecific depressive symptoms and minor depression regarding functional impairment and associated characteristics in primary care.

    Compr Psychiatry 2006, 47(1):35-41. PubMed Abstract | Publisher Full Text OpenURL

  23. Cuijpers P, Smit F, Oostenbrink J, de Graaf R, Ten Have M, Beekman A: Economic costs of minor depression: a population-based study.

    Acta Psychiatr Scand 2007, 115(3):229-236. PubMed Abstract | Publisher Full Text OpenURL

  24. Cuijpers P, Smit F: Subthreshold depression as a risk indicator for major depressive disorder: a systematic review of prospective studies.

    Acta Psychiatr Scand 2004, 109(5):325-331. PubMed Abstract | Publisher Full Text OpenURL

  25. Morgan AJ, Jorm AF, Mackinnon AJ: Email-based promotion of self-help for subthreshold depression: mood memos randomised controlled trial.

    Br J Psychiatry 2012, 200(5):412-418. PubMed Abstract | Publisher Full Text OpenURL

  26. Christensen H, Griffiths KM: The prevention of depression using the Internet.

    Med J Aust 2002, 177(Suppl):S122-S125. PubMed Abstract | Publisher Full Text OpenURL

  27. Cuijpers P, van Straten A, Warmerdam L, van Rooy MJ: Recruiting participants for interventions to prevent the onset of depressive disorders: possible ways to increase participation rates.

    BMC Health Serv Res 2010, 10:181. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  28. Klaw E, Humphreys K: Facilitating client involvement in self-help groups. In Psychologists’ desk reference. 2nd edition. Edited by Koocher G, Norcross J, Hill S. New York: Oxford University Press; 2005:502-505. OpenURL

  29. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M: Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.

    J Med Internet Res 2009, 11(2):e15. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  30. Cavanagh K, Seccombe N, Lidbetter N: The implementation of computerized cognitive behavioural therapies in a service user-led, third sector self help clinic.

    Behav Cogn Psychother 2011, 39(4):427-442. PubMed Abstract | Publisher Full Text OpenURL

  31. Spek V, Nyklicek I, Smits N, Cuijpers P, Riper H, Keyzer J, Pop V: Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial.

    Psychol Med 2007, 37(12):1797-1806. PubMed Abstract | Publisher Full Text OpenURL

  32. Calear AL, Christensen H, Mackinnon A, Griffiths KM, O’Kearney R: The YouthMood Project: a cluster randomized controlled trial of an online cognitive behavioral program with adolescents.

    J Consult Clin Psychol 2009, 77(6):1021-1032. PubMed Abstract | Publisher Full Text OpenURL

  33. Hoek W, Schuurmans J, Koot HM, Cuijpers P: Effects of Internet-based guided self-help problem-solving therapy for adolescents with depression and anxiety: a randomized controlled trial.

    PLoS One 2012, 7(8):e43485. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  34. de Graaf LE, Gerhards SA, Arntz A, Riper H, Metsemakers JF, Evers SM, Severens JL, Widdershoven G, Huibers MJ: Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.

    Br J Psychiatry 2009, 195(1):73-80. PubMed Abstract | Publisher Full Text OpenURL

  35. Kupfer DJ: Long-term treatment of depression.

    J Clin Psychiatry 1991, 52(Suppl):28-34. PubMed Abstract OpenURL

  36. GesundheitsTraining.Online http://www.geton-training.de webcite

  37. Rohde P, Lewinsohn PM, Seeley JR: Comparability of telephone and face-to-face interviews in assessing axis I and II disorders.

    Am J Psychiatry 1997, 154(11):1593-1598. PubMed Abstract | Publisher Full Text OpenURL

  38. Allen K, Cull A, Sharpe M: Diagnosing major depression in medical outpatients: acceptability of telephone interviews.

    J Psychosom Res 2003, 55(4):385-387. PubMed Abstract | Publisher Full Text OpenURL

  39. Lewinsohn PM: The behaviroal study and treatment of depression. In Progress in behavior modification. 1st edition. Edited by Hersen M, Eisler RM, Miller PM. New York: Academic Press; 1974. OpenURL

  40. D’Zurilla TJ, Nezu AM: Social problem solving in adults. In Advances in cognitive-behavioral research and therapy. 1st edition. Edited by Kendall PC. New York: Academic Press; 1982:201-274. OpenURL

  41. Cuijpers P, Smit F, van Straten A: Psychological treatments of subthreshold depression: a meta-analytic review.

    Acta Psychiatr Scandinavica 2007, 115(6):434-441. Publisher Full Text OpenURL

  42. Garber J, Clarke GN, Weersing VR, Beardslee WR, Brent DA, Gladstone TR, DeBar LL, Lynch FL, D’Angelo E, Hollon SD, Shamseddeen W, Iyengar S: Prevention of depression in at-risk adolescents: a randomized controlled trial.

    JAMA 2009, 301(21):2215-2224. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  43. van’t Veer-Tazelaar PJ, van Marwijk HW, van Oppen P, van Hout HP, van der Horst HE, Cuijpers P, Smit F, Beekman AT: Stepped-care prevention of anxiety and depression in late life: a randomized controlled trial.

    Arch Gen Psychiatry 2009, 66(3):297-304. PubMed Abstract | Publisher Full Text OpenURL

  44. Warmerdam L, van Straten A, Twisk J, Riper H, Cuijpers P: Internet-based treatment for adults with depressive symptoms: randomized controlled trial.

    J Med Internet Res 2008, 10(4):e44. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  45. van Straten A, Cuijpers P, Smits N: Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial.

    J Med Internet Res 2008, 10(1):e7. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  46. Donker T, Griffiths KM, Cuijpers P, Christensen H: Psychoeducation for depression, anxiety and psychological distress: a meta-analysis.

    BMC Med 2009, 7:79. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  47. German S3-Guideline/National Disease Management Guideline Unipolar Depression http://www.depression.versorgungsleitlinien.de/ webcite

  48. Allart-van Dam E, Hosman CM, Hoogduin CA, Schaap CP: Prevention of depression in subclinically depressed adults: follow-up effects on the ‘Coping with Depression’ course.

    J Affect Dis 2007, 97(1–3):219-228. PubMed Abstract | Publisher Full Text OpenURL

  49. Clarke GN, Hawkins W, Murphy M, Sheeber LB, Lewinsohn PM, Seeley JR: Targeted prevention of unipolar depressive disorder in an at-risk sample of high school adolescents: a randomized trial of a group cognitive intervention.

    J Am Acad Child Adolesc Psychiatry 1995, 34(3):312-321. PubMed Abstract | Publisher Full Text OpenURL

  50. Hagan R, Evans SF, Pope S: Preventing postnatal depression in mothers of very preterm infants: a randomised controlled trial.

    BJOG 2004, 111(7):641-647. PubMed Abstract | Publisher Full Text OpenURL

  51. Lobbestael J, Leurgans M, Arntz A: Inter-rater reliability of the structured clinical interview for DSM-IV Axis I disorders (SCID I) and axis II disorders (SCID II).

    Clin Psychol Psychother 2011, 18(1):75-79. PubMed Abstract | Publisher Full Text OpenURL

  52. Crippa JA, de Lima OF, Del-Ben CM, Filho AS, da Silva Freitas MC, Loureiro SR: Comparability between telephone and face-to-face structured clinical interview for DSM-IV in assessing social anxiety disorder.

    Perspect Psychiatr Care 2008, 44(4):241-247. PubMed Abstract | Publisher Full Text OpenURL

  53. Hautzinger M, Bailer M, Hofmeister D, Keller F: Allgemeine Depressionsskala. 2nd edition. Göttingen: Hogrefe; 2012. OpenURL

  54. Brooks R: EuroQol: the current state of play.

    Health Policy 1996, 37(1):53-72. PubMed Abstract | Publisher Full Text OpenURL

  55. Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, Bullinger M, Kaasa S, Leplege A, Prieto L, Sullivan M: Cross-validation of item selection and scoring for the SF-12 health survey in nine countries: results from the IQOLA project: international quality of life assessment.

    J Clin Epidemiol 1998, 51(11):1171-1178. PubMed Abstract | Publisher Full Text OpenURL

  56. Zigmond AS, Snaith RP: The hospital anxiety and depression scale.

    Acta Psychiatr Scand 1983, 67(6):361-370. PubMed Abstract | Publisher Full Text OpenURL

  57. Hermann-Lingen C, Buss U, Snaith RP: Hospital Anxiety and Depression Scale - Deutsche Version (HADS-D). 3rd edition. Bern: Verlag Hans Huber; 2011. OpenURL

  58. Bjelland I, Dahl AA, Haug TT, Neckelmann D: The validity of the hospital anxiety and depression scale: an updated literature review.

    J Psychosom Res 2002, 52(2):69-77. PubMed Abstract | Publisher Full Text OpenURL

  59. D’Zurilla TJ, Nezu AM, Maydeu-Olivares A: Social Problem-Solving Inventory-Revised (SPSI-R). New York: MHS; 2002. OpenURL

  60. Manos RC, Kanter JW, Luo W: The behavioral activation for depression scale-short form: development and validation.

    Behav Ther 2011, 42(4):726-739. PubMed Abstract | Publisher Full Text OpenURL

  61. Pearlin LI, Schooler C: The structure of coping.

    J Health Soc Behav 1978, 19(1):2-21. PubMed Abstract | Publisher Full Text OpenURL

  62. Berle D, Starcevic V, Moses K, Hannan A, Milicevic D, Sammut P: Preliminary validation of an ultra-brief version of the Penn State worry questionnaire.

    Clin Psychol Psychother 2011, 18(4):339-346. PubMed Abstract | Publisher Full Text OpenURL

  63. Bastien CH, Vallieres A, Morin CM: Validation of the insomnia severity index as an outcome measure for insomnia research.

    Sleep Med 2001, 2(4):297-307. PubMed Abstract | Publisher Full Text OpenURL

  64. Morin CM, Belleville G, Belanger L, Ivers H: The insomnia severity index: psychometric indicators to detect insomnia cases and evaluate treatment response.

    Sleep 2011, 34(5):601-608. PubMed Abstract | PubMed Central Full Text OpenURL

  65. Devilly GJ, Borkovec TD: Psychometric properties of the credibility/expectancy questionnaire.

    J Behav Ther Exp Psychiatry 2000, 31(2):73-86. PubMed Abstract | Publisher Full Text OpenURL

  66. Fischer EH, Farina A: Attitudes toward seeking professional psychological help: a shortened form and considerations for research.

    J Coll Stud Dev 1995, 36:368-373. OpenURL

  67. Elhai JD, Schweinle W, Anderson SM: Reliability and validity of the attitudes toward seeking professional psychological help scale-short form.

    Psychiatry Res 2008, 159(3):320-329. PubMed Abstract | Publisher Full Text OpenURL

  68. Schmidt J, Lamprecht F, Wittmann W: Satisfaction with inpatient management: development of a questionnaire and initial validity studies: [Article in German].

    Psychother, Psychosom Med Psychol 1989, 39:248-255. PubMed Abstract OpenURL

  69. Nguyen TD, Attkisson CC, Stegner BL: Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire.

    Eval Program Plann 1983, 6(3–4):299-313. PubMed Abstract OpenURL

  70. Attkisson CC, Zwick R: The client satisfaction questionnaire: psychometric properties and correlations with service utilization and psychotherapy outcome.

    Eval Program Plann 1982, 5(3):233-237. PubMed Abstract | Publisher Full Text OpenURL

  71. Nestoriuc Y, Rief W: Risiken und Nebenwirkungen von Psychotherapie [Risks and side-effects of psychotherapy]. In Risiken und Nebenwirkungen von Verhaltenstherapie [Risks and side-effects of behavioural therapy]. Edited by Linden M, Strauß B. Berlin: MWV; 2012:59-74. OpenURL

  72. Dolan P: Modeling valuations for EuroQol health states.

    Med Care 1997, 35(11):1095-1108. PubMed Abstract | Publisher Full Text OpenURL

  73. Greiner W, Claes C, Busschbach JJ, von der Schulenburg JM: Validating the EQ-5D with time trade off for the German population.

    Eur J Health Econ 2005, 6(2):124-130. PubMed Abstract | Publisher Full Text OpenURL

  74. Brazier J, Roberts J, Tsuchiya A, Busschbach J: A comparison of the EQ-5D and SF-6D across seven patient groups.

    Health Econ 2004, 13(9):873-884. PubMed Abstract | Publisher Full Text OpenURL

  75. Brazier JE, Roberts J: The estimation of a preference-based measure of health from the SF-12.

    Med Care 2004, 42(9):851-859. PubMed Abstract | Publisher Full Text OpenURL

  76. Brazier J, Rowen D, Hanmer J: Revised SF-6D scoring programmes: a summary of improvements.

    PRO Newsletter 2008, 40:14-15. OpenURL

  77. Hakkaart-van Roijen L, van Straten A, Donker M, Tiemens B: Manual Trimbos/iMTA questionnaire for Costs associated with Psychiatric illness (TiC-P). Rotterdam: Erasmus University; 2002. OpenURL

  78. Association of Statutory Health Insurance Physicians http://www.kvberlin.de/20praxis/30abrechnung_honorar/10ebm/index.html webcite

  79. Krauth C, Hessel F, Hansmeier T, Wasem J, Seitz R, Schweikert B: Empirical standard costs for health economic evaluation in Germany – a proposal by the working group methods in health economic evaluation.

    Gesundheitswesen 2005, 67(10):736-746. PubMed Abstract | Publisher Full Text OpenURL

  80. Manca A, Hawkins N, Sculpher MJ: Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility.

    Health Econ 2005, 14(5):487-496. PubMed Abstract | Publisher Full Text OpenURL

  81. Fenwick E, O’Brien BJ, Briggs A: Cost-effectiveness acceptability curves–facts, fallacies and frequently asked questions.

    Health Econ 2004, 13(5):405-415. PubMed Abstract | Publisher Full Text OpenURL

  82. Christensen H, Griffiths KM, Farrer L: Adherence in internet interventions for anxiety and depression.

    J Med Internet Res 2009, 11(2):e13. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-244X/14/25/prepub