Email updates

Keep up to date with the latest news and content from BMC Medical Informatics and Decision Making and BioMed Central.

Open Access Research article

Visual Aids for Multimodal Treatment Options to Support Decision Making of Patients with Colorectal Cancer

Sabine Hofmann1, Janina Vetter1, Christiane Wachter1, Doris Henne-Bruns1, Franz Porzsolt2 and Marko Kornmann1*

Author Affiliations

1 Department of General and Visceral Surgery, University of Ulm, Albert-Einstein-Allee 23, 89081, Ulm, Germany

2 Study Group of Health Services Research at the Department of General and Visceral Surgery, University of Ulm, Albert-Einstein-Allee 23, 89081, Ulm, Germany

For all author emails, please log on.

BMC Medical Informatics and Decision Making 2012, 12:118  doi:10.1186/1472-6947-12-118


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6947/12/118


Received:25 October 2011
Accepted:31 August 2012
Published:23 October 2012

© 2012 Hofmann et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

A variety of multimodal treatment options are available for colorectal cancer and many patients want to be involved in decisions about their therapies. However, their desire for autonomy is limited by lack of disease-specific knowledge. Visual aids may be helpful tools to present complex data in an easy-to-understand, graphic form to lay persons. The aim of the present study was to evaluate the treatment preferences of healthy persons and patients using visual aids depicting multimodal treatment options for colorectal cancer.

Methods

We designed visual aids for treatment scenarios based on four key studies concerning multimodal treatment of colorectal cancer. The visual aids were composed of diagrams depicting outcome parameters and side effects of two treatment options. They were presented to healthy persons (n = 265) and to patients with colorectal cancer (n = 102).

Results

Most patients and healthy persons could make immediate decisions after seeing the diagrams (range: 88% – 100%). Patients (79%) chose the intensive-treatment option in the scenario with a clear survival benefit. In scenarios without survival benefit, all groups clearly preferred the milder treatment option (range: 78% - 90%). No preference was seen in the scenario depicting equally intense treatment options with different timing (neoadjuvant vs. adjuvant) but without survival benefit.

Conclusions

Healthy persons’ and patients’ decisions using visual aids seem to be influenced by quality-of-life aspects rather than recurrence rates especially in situations without survival benefit. In the future visual aids may help to improve the management of patients with colorectal cancer.

Keywords:
Visual aids; Colorectal cancer; Quality of life; Side effects; Shared decision-making

Background

Colorectal cancer (CRC) is the second most common form of cancer in both genders today. The last decades have shown a rising incidence, while advances in treatment have led to decreasing mortality [1]. Multimodal treatment strategies have been established and are presently regarded as standard for many tumor stages. The wealth of possible treatment options requires a sophisticated planning of the therapeutic approach. Recommendations for the individual treatment of patients are nowadays often made by an interdisciplinary panel of specialists based on the staging and pathological results.

However, the relation between the doctor and patients has changed considerably in recent years due to patients’ increasing desire for autonomy. Many patients want to be involved in the process of decision-making about their therapies [2] and already have differentiated ideas about their therapy stemming from information often gathered in the internet [3]. However, these ideas may be unrealistic because patients lack sufficient disease-specific knowledge and experience with the potential progression of the disease, whereas physicians may be misled in regard to patients’ expectations. Both patients and physicians are forced to make vital decisions in the face of complex and sometimes confusing data with regard to effectiveness and toxicity of various potential treatment options.

To be able to participate in decision-making it is essential to provide patients with information based on current scientific evidence that is both easy to understand and detailed. In malignant diseases such as breast or prostate cancer decision aids have been tested to facilitate the patient’s decision-making process [4-6]. In colorectal cancer decision aids were used in the process of screening [7,8]. Only one study so far evaluated decision aids for palliative colorectal cancer treatment [9]. Therefore, it is presently completely unknown how healthy persons and patients would decide when confronted with multimodal treatment options in curative settings of colorectal cancer. In contrast to decision aids which by definition have to be publicly available we recently designed visual aids for several scenarios of multimodal potentially curative treatment options of colorectal cancer which could be used by the physician to present the outcome and side effects of various treatment options to the patient [10,11]. The aim of the present study was to evaluate the treatment preferences of healthy persons and patients for potentially curative multimodal treatment options in colorectal cancer using these visual aids.

Methods

Selection of studies for visual aids

Four key studies of colorectal cancer influencing treatment strategies were chosen as a basis for the visual aid scenarios [12-19]. All studies were prospective, randomized controlled trials (Table 1). In scenarios 1 – 3, a mild therapeutic option consisting of an operation alone or an operation with adjuvant mono-chemotherapy was presented in comparison with a more intensive therapeutic option consisting of an operation with mono-chemotherapy, double-chemotherapy, or radiation [12,14,15]. In scenario 4, the results of two equally intense options were presented comparing neoadjuvant chemoradiotherapy with adjuvant chemoradiotherapy in rectal cancer [19].

Table 1. Study characteristics

Design of the visual aids

There exists a huge literature about how to present information to healthy persons and patients [20-22]. One of the key findings was that graphical presentation including frequencies may be most appropriate to present results of clinical trials [23]. Our designed visual aids depicted separately the results of two treatment options evaluated in a randomized controlled trial according to therapeutic outcome and side-effects. They were presented as graphs as described previously [10,11]. This was achieved by four-colored square diagrams, each consisting of 100 small squares (four quadrants), representing a total of 100 patients or 100% (Figure 1). One colored square stands for one event in a patient; a cross means the coincidence of two events in one patient. The two quadrants on the left side show the first treatment option, while the second option is displayed in the quadrants on the right side. The two upper quadrants show the outcome results, such as mortality, local recurrence rates and distant metastases. The two lower quadrants display possible side effects. Survival differences between the two therapy arms are presented separately in the middle (Figure 1).

thumbnailFigure 1. Visual aid illustrating two treatment scenarios for patients with UICC stage III colon cancer based on the published data[12,13]. The results of an operation (OP) alone (A) are shown as mild treatment on the left, and the results of an OP plus adjuvant 5-FU chemotherapy (CHT) (B) are shown as intense treatment on the right. The illustrations are based on 4 quadrants showing 100 squares representing% or events. The relative incidence of local recurrence and distant metastasis are displayed in the two upper quadrants and important undesired events in the two lower quadrants. The difference in recurrence of the two treatment options is summarized in the central dotted circle followed by a statement about overall survival benefit. To better differentiate the criteria, each is represented in a different color. If several criteria applied to one patient (e.g., nausea and diarrhea), the first criterion is shown as background color. Further criteria are shown in the foreground with the sign (X) in the appropriate color.

Out of the recommended quality criteria for patient decision aids published [20,24] our visual aids fulfilled several important criteria including systematic development process [10,11], providing information about options, presenting probabilities, disclosing conflicts of interest, balancing the presentation of options, using plain language, and basing information on up to date scientific evidence. During presentation it was also possible to clarify values and to communicate with the presenter. There was no conflict of interest. In constrast to the requirement of decision aids our visual aids are not publicly available.

Feasibility testing using healthy persons

After establishing the graphs as described above and previously reported [10,11], a pilot experiment was performed to test the feasibility of the visual aids. The preferences of healthy persons with regard to the treatment options were also evaluated. Thus, the visual aids of the four scenarios were presented to 213 healthy persons as a powerpoint presentation in an auditorium after a 30-minute lecture about colorectal cancer. Each participant marked his/her preferred treatment option on a form using a pencil directly after the presentation of each study. They had the option to vote for treatment A or treatment B. In addition, participants had the possibility to mark a third option, “I don't know”, if they were not able to make a decision immediately. This experiment was carried out with the support of several social clubs (Landfrauen and protestant church) in the area which organized the recruitment and transport of the volunteers to the auditorium.

Evaluation of the visual aids by patients

After demonstrating feasibility an ethics request was filed as a precondition for the following study involving colorectal cancer patients. The study was approved by the ethics committee of the University of Ulm (no. 141/08).

After giving written informed consent, patients with colorectal cancer were questioned individually by one of the authors (J.V.) about their treatment preferences after being shown the colored print-out diagrams wrapped in transparent foil. Patients with colorectal cancer were asked to participate either during an outpatient visit at the time of follow-up 6 to 24 months after surgery (n = 52) or during inpatient treatment approximately 10 days following surgery of primary colorectal cancer (n = 50). An additional 52 healthy persons of the authors’ milieu were also questioned individually to rule out potential differences with regard to the mode of presentation (group vs. individual). Similar to the healthy persons in the auditorium, participants had the option to vote for treatment A or treatment B. They could also mark the third option, “I don't know”, if they were not able to make a decision during the individual presentation. All participants were requested to fill out a demographic questionnaire before the presentation.

Results

Design of the visual aids

The results of four prospective, randomized controlled trials [12,14,15,19] comparing two treatment options were presented as graphic scenarios displaying outcome parameters (mortality, recurrence rates, survival) and side effects (Figures 1, 2, 3, 4). In scenarios 1 – 3, a mild therapeutic option was compared with a more intensive therapeutic option, while two equally intense options with different timing (neoadjuvant vs. adjuvant) were presented in scenario 4. The characteristics of the scenarios and related studies are summarized in Table 1, while the visual aids are shown in Figures 1, 2, 3, 4.

thumbnailFigure 2. Visual aid illustrating two adjuvant treatment scenarios for patients with UICC stages II and III colon cancer[[14]]. The results of an operation (OP) plus 5-FU chemotherapy (CHT) are shown as mild treatment (A) on the left, and the results of an OP plus 5-FU and oxaliplatin combination CHT are shown as intense treatment (B) on the right. The relative incidence of death, local recurrence and distant metastasis are displayed in the two upper quadrants and important undesired events in the two lower quadrants. The difference of recurrence of the two treatment options is summarized in the central dotted circle followed by a statement about overall survival benefit.

thumbnailFigure 3. Visual aid illustrating two treatment scenarios for patients with UICC stages I to III rectal cancer based on the published results and side effects[[15-18]]. The results of the “new” operation (OP) alone (total mesorectal excision technique) are shown as mild treatment (A) on the left, and the results of the “new” OP plus neoadjuvant short-course (5x5 Gy) radiotherapy are shown as intense treatment (B) on the right. The relative incidence of death, local recurrence and distant metastasis are displayed in the two upper quadrants and important undesired events in the two lower quadrants. The difference of recurrence of the two treatment options is summarized in the central dotted circle followed by a statement about an overall survival benefit.

thumbnailFigure 4. Visual aid illustrating two equally intense treatment scenarios for patients with UICC stages I to III rectal cancer[[19]]. The results of neoadjuvant chemoradiotherapy followed by an operation (OP) are shown as pre-operative option (A) on the left, and the results of an OP followed by adjuvant chemoradiotherapy are shown as post-operative option (B) on the right. While all patients underwent neoadjuvant treatment in option A, only 72% of the patients underwent additional adjuvant treatment, depending on the pathological tumor stage. The relative incidence of death, local recurrence and distant metastasis are displayed in the two upper quadrants and important undesired acute and long-term side effects in the two lower quadrants. The difference of recurrence of the two treatment options is summarized in the central dotted circle followed by a statement about an overall survival benefit.

Groups of participants

In a pilot experiment the prepared visual aids were presented to 213 healthy persons as a powerpoint presentation in an auditorium after a 30-minute lecture about colorectal cancer. Demonstrating the feasibility of our visual aids in a next step patients with colorectal cancer (n = 102) were questioned individually. One of the authors (J.V.) interviewed patients either during inpatient treatment following an operation (n = 50) or during an outpatient visit within the follow-up program (n = 52). The visual aids were presented as colored print-out diagrams wrapped in transparent foil. In addition, another 52 healthy persons were questioned in this individual fashion to rule out a possible influence of the mode of presentation (auditorium vs. individual). The demographics of the participating groups are summarized in Table 2.

Table 2. Demographic data of participating groups

Decisions of the groups

The participants generated a total of 1468 preferences when shown the 4 scenarios – 408 patient preferences and 1060 preferences of healthy persons. The results for the three groups are summarized in Table 3. There were no differences in the decisions between the in- and outpatients (data not shown).

Table 3. Results of test person’s decisions

The first scenario (Figure 1) compared an operation alone with an operation and adjuvant mono-chemotherapy in colon cancer [12]. The results were characterized by a 5% survival advantage in the second group. Furthermore, an obviously lower rate for local recurrence and distant metastasis could be shown. Side effects were only documented for the chemotherapy group. The majority of patients and healthy persons questioned individually chose the intensive treatment (77% and 71%, respectively). Only half of the healthy persons from the auditorium group chose this option (49%).

The second scenario (Figure 2) displayed two adjuvant treatment options for colon cancer, comparing 5-FU mono-chemotherapy with 5-FU and oxaliplatin double-chemotherapy [14]. Despite better control of tumor recurrence by the intensive option, survival rates in the overall group (UICC II and III) were not affected according to the data published in 2004. All groups showed a strong preference for the mild treatment option, irrespective of the mode of presentation (range 81% - 90%, Table 3).

In the third scenario (Figure 3), the operation alone involving a total mesorectal excision (TME) was compared with short-course radiotherapy (5x5 Gy) followed by TME in rectal cancer [15]. The intensive option showed better local control and a somewhat lower frequency of distant metastases, but no effect on survival. The intensive option was associated with more side effects. Similar to scenario 2, all groups preferred the mild treatment option, regardless of the form of presentation (range 78% - 82%, Table 3).

In the fourth scenario (Figure 4), two equally intense treatment options comparing neoadjuvant chemoradiotherapy with adjuvant chemoradiotherapy in rectal cancer were presented [19]. The first option included all patients in the neoadjuvant treatment, whereas the second option included only 70% of the patients - depending on the tumor stage – who received adjuvant chemoradiotherapy. The advantage of the neoadjuvant option was the lower rate of local recurrence and distant metastasis (Figure 4). The groups of patients and healthy persons questioned showed no preference, choosing the adjuvant treatment option in 65% to 46%, respectively. There was no difference between the preference of any group.

Discussion

Three main forms of patient-physician relationships have been described [2]: First, the era of parternalism - doctor served as the guardian; second, the era of informed consent; and third, the era of shared-decision-making. Today many patients want to be well informed and demand to take an active role in the planning of their treatment reflecting a shift in the patient’s role in decision-making from the passive bystander to an active participant [2,3,6]. However, patients’ desire for autonomy is impaired by a lack of disease-specific knowledge and by the growing complexity of treatment options which have recently become available, especially in cancer therapy. Guidelines have been developed by expert consensus to give recommendations for a therapy or combination of therapies according to the disease stage to support medical professionals in their recommendations. However, since guidelines are standardized, they may not consider the patient’s individual preferences and thoughts about life planning. To be able to participate in the individual decision-making process it is essential to provide information for the patients based on current scientific evidence that is both easy to understand and detailed. No studies about the use of decision aids in neoadjuvant or adjuvant treatment of potentially curative colorectal cancer have been published to date [8]. Therefore, in analogy to decision aids we designed visual aids for neoadjuvant and adjuvant treatment of colorectal cancer depicting various therapy options. The design of our visual graphs basically followed the basic recommendations for decision aids standards except public availability [20,24]. Our aim was to evaluate the treatment preferences of patients for these multimodal treatment options using visual aids.

To test the feasibility of the visual aids the four designed graphs were presented to 213 healthy persons in an auditorium after a lecture about colorectal cancer. These preliminary results demonstrated that our visual aids were feasible, because most participants could make decisions immediately. Evaluation of these results further revealed that healthy persons generally preferred the milder treatment options. These options were often not recommended by our German guideline [25]. In order to rule out that healthy persons decide different from patients suffering from colorectal cancer facing a potentially lethal disease the second part of our study was carried out. Patients were asked individually for their preferences. We again asked healthy persons in an individual manner to rule out effects with regard to the mode of presentation.

Our results demonstrated that most patients and healthy persons can make a decision after being shown the visual aids. As expected the majority of patients (77%) and healthy persons (71%) chose the intensive treatment option in scenario 1 demonstrating a survival benefit when asked individually. We do not know why only 49% of healthy persons questioned in the auditorium chose the option accompanied with the survival benefit. It is possible therefore that the presenter and mode of presentation may have influenced the decision making to some extent, one possible limitation of our study. It is also possible that the lecture about colorectal cancer might have affected their opinion.

In contrast, the evaluation of the treatment choices without effect on overall survival revealed that the majority of the patients and healthy persons preferred the mild treatment options (78%-90%) independent of the mode of presentation (scenarios 2 and 3). When looking at the graphs of the visual aids these preferences seem not really surprising. However, they clearly contradict the opinion of specialists justifying intensive treatment by a sole reduction of recurrence rates and the present German guideline recommendations [25]. Our results suggest that participants - patients and healthy persons - placed more emphasis on quality-of-life aspects than on recurrence rates in decisions without difference in survival. This was not surprising to us and confirmed our previous results in healthy persons [10,11,26].

The fourth scenario depicted two equally intense options comparing different modes of application (neoadjuvant vs. adjuvant) in rectal cancer. The advantage of the neoadjuvant option is the significantly lower rate of local recurrence. This option is therefore recommended presently by the treatment guidelines and most physicians [25]. However, patients and healthy persons did not show a preference for the neoadjuvant therapy (28%-41%). Instead, no difference in the treatment preference was observed between the neoadjuvant and adjuvant options presented. This is probably related to the lack of a survival benefit and the possibility to avoid chemoradiotherapy at all when choosing the other option (adjuvant). While all patients received neoadjuvant chemoradiotherapy, only 72% of the patients received adjuvant chemoradiotherapy. The remaining 28% of the patients had tumors of lower stage that did not require multimodal treatment after final pathological examination [19].

In summary, our results showed: first, that patients and healthy persons can make decisions about presented multimodal treatment options using visual aids; second, that the decisions of patients and healthy persons are equal and independent of the mode of presentation when comparing mild versus intensive treatment without survival benefit; third, in decisions with survival benefit, patients and healthy persons prefer the beneficial option, but this may depend on the mode of presentation; and fourth, there is no preference for “equally” intense options without survival benefit.

Conclusions

We conclude from our results and experience gained during the project that patients’ decisions are primarily influenced by quality-of-live aspects and overall survival, often contrasting treatment recommendations made by specialists and national guidelines. Visual aids may be reasonable tools in the management of colorectal cancer patients in the future. They may be helpful to supply patients with additional information about the results and side effects of various disease-specific treatment options thereby increasing patients’ knowledge about the disease and thus gaining more confidence in the treatment and its consequences.

Competing interests

The authors declare that they have no competing interest.

Authors’ contributions

KM, PF - conception and design; VJ, WC, HBD, PF, KM - provision of study materials or patients; VJ, WC, KM - collection and assembly of data; HS, HBD, PF, KM - data analysis and interpretation; HS, PF, KM - manuscript writing. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to give special thanks to all patients and healthy persons who participated.

References

  1. Siegel R, Naishadham D, Jemal A: Cancer statistics, 2012.

    CA Cancer J Clin 2012, 62:10-29. PubMed Abstract | Publisher Full Text OpenURL

  2. Charles C, Whelan T, Gafni A: What do we mean by partnership in making decisions about treatment?

    BMJ 1999, 319:780-782. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  3. Frosch DL, Kaplan RM: Shared decision making in clinical medicine: past research and future directions.

    Am J Prev Med 1999, 17:285-294. PubMed Abstract | Publisher Full Text OpenURL

  4. Whelan T, Sawka C, Levine M, et al.: Helping Patients make informed choices: a randomized Trial of a Decision Aid for Adjuvant Chemotherapy in Lymph node-negative Breast Cancer.

    J Nat C Inst 2003, 95:581-587. Publisher Full Text OpenURL

  5. Auvinen A, Hakama M, Ala-Opas M, et al.: A randomized trial of choice of treatment in prostate cancer: the effect of intervention on the treatment chosen.

    BJU Int 2004, 93:52-56. PubMed Abstract | Publisher Full Text OpenURL

  6. Stacey D, Samant R, Bennet C: Decision making in oncology: a review of patient decision aids to support patient participation.

    CA Cancer J Clin 2008, 58:293-304. PubMed Abstract | Publisher Full Text OpenURL

  7. Miller DP, Spangler JG, Case LD, et al.: Effectiveness of a web-based colorectal cancer screening patient decision aid: a randomized controlled trial in a mixed-literacy population.

    Am J Prev Med 2011, 40:608-615. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  8. Stacey D, Bennett CL, Barry MJ, et al.: Decision aids for people facing health treatment or screening decisions.

    Cochrane Database Syst Rev 2011, 10:CD001431. PubMed Abstract | Publisher Full Text OpenURL

  9. Leighl NB, Shepherd HL, Butow PN, et al.: Supporting treatment decision making in advanced cancer: a randomized trial of a decision aid for patients with advanced colorectal cancer considering chemotherapy.

    J Clin Oncol 2011, 29:2077-2084. PubMed Abstract | Publisher Full Text OpenURL

  10. Kornmann M, Henne-Bruns D, Porzsolt F: Neoadjuvant treatment of rectal carcinoma: assessment of health care services by physicians and lay persons.

    J Clin Oncol 2008, 26:4866-4868. PubMed Abstract | Publisher Full Text OpenURL

  11. Kornmann M, Porzsolt F, Henne-Bruns D: Vergleichende Bewertung von Gesundheitsleistungen durch Laien am Beispiel der Therapie des Rektumkarzinoms.

    Zentralbl Chir 2008, 133:1-8. OpenURL

  12. Moertel CG, Fleming TR, Macdonald JS, et al.: Levamisole and fluorouracil for adjuvant therapy of resected colon carcinoma.

    N Engl J Med 1990, 322:352-358. PubMed Abstract | Publisher Full Text OpenURL

  13. Moertel CG, Laurie JA, Fleming TR, et al.: Fluorouracil plus levamisole as effective adjuvant therapy after resection of stage 3 colon cancer carcinoma: a final report.

    Ann Intern Med 1995, 122:321-326. PubMed Abstract | Publisher Full Text OpenURL

  14. André T, Boni C, Mounedji-Boudiaf L, et al.: Oxaliplatin, fluorouracil, and leucovorin as adjuvant treatment for colon cancer.

    N Engl J Med 2004, 350:2343-2351. PubMed Abstract | Publisher Full Text OpenURL

  15. Kapiteijn E, Marijnen CA, Nagtegaal ID, et al.: Dutch Colorectal Cancer Group: Preoperative radiotherapy combined with total mesorectal excision for resectable rectal cancer.

    N Engl J Med 2001, 345:638-646. PubMed Abstract | Publisher Full Text OpenURL

  16. Marijnen CAM, van de Velde CJH, Putter H, et al.: Impact of short-term preoperative radiotherapy on health-related quality of life and sexual functioning in primary rectal cancer: report of a multicenter randomized trial.

    J Clin Oncol 2005, 23:1847-1858. PubMed Abstract | Publisher Full Text OpenURL

  17. Peeters KCMJ, van de Velde CJH, Leer JWH, et al.: Late side effects of short-course preoperative radiotherapy combined with total mesorectal excision for rectal cancer: increased bowel dysfunction in irradiated patients - A dutch colorectal cancer group study.

    J Clin Oncol 2005, 23:6199-6206. PubMed Abstract | Publisher Full Text OpenURL

  18. Peeters KCMJ, Marijnen CA, Nagtegaal ID, Dutch Colorectal Cancer Group, et al.: The TME trial after a median follow-up of 6 years: increased local control but no survival benefit in irradiated patients with respectable rectal carcinoma.

    Ann Surg 2007, 246:693-701. PubMed Abstract | Publisher Full Text OpenURL

  19. Sauer R, Becker H, Hohenberger W, German Rectal Cancer Study Group, et al.: Preoperative versus postoperative chemoradiotherapy for rectal cancer.

    N Engl J Med 2004, 351:1731-1740. PubMed Abstract | Publisher Full Text OpenURL

  20. O’Conner H, Llewelyn-Thomas H, Stacey D: IPDAS (International Patient Decision Aids Standards) Collaboration Background Document.

    Presenting probabilities 2005.

    Ipdas.ohri.ca/IPDAS_Background.pdf

    OpenURL

  21. Lipkus IM: Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations.

    Med Decis Making 2007, 27:696-713. PubMed Abstract | Publisher Full Text OpenURL

  22. Peters E: Beyond comprehension: the role of numeracy in judgments and decisions.

    Curr Dir Psychol Sci 2012, 21:31-35. Publisher Full Text OpenURL

  23. Cheung YB, Wee HL, Thumboo J, et al.: Risk communication in clinical trials: a cognitive experiment and a survey.

    BMC Med Inform Decis Mak 2010, 10:55. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  24. Elwyn G, O'Connor A, Stacey D, et al.: International Patient Decision Aids Standards (IPDAS) Collaboration. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process.

    BMJ 2006, 333:417. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  25. Schmiegel W, Pox C, Reinacher-Schick A, et al.: S3 guidelines for colorectal carcinoma: results of an evidence-based consensus conference on February 6/7, 2004 and June 8/9, 2007 (for the topics IV, VI and VII).

    Z Gastroenterol 2010, 48:65-136. PubMed Abstract | Publisher Full Text OpenURL

  26. Wachter C, Porzsolt F, Henne-Bruns D, Kornmann M: Bewertung von Gesundheitsleistungen durch Laien am Beispiel der multimodalen Therapie des Kolonkarzinoms.

    Viszeralmedizin 2010, 26:131-138. Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6947/12/118/prepub