<?xml version='1.0'?>
<!DOCTYPE art SYSTEM 'http://www.biomedcentral.com/xml/article.dtd'>
<art>
   <ui>1472-6947-8-14</ui>
   <ji>1472-6947</ji>
   <fm>
      <dochead>Research article</dochead>
      <bibl>
         <title>
            <p>Risk communication formats for low probability events: an exploratory study of patient preferences</p>
         </title>
         <aug>
            <au id="A1" ca="yes">
               <snm>Dolan</snm>
               <mi>G</mi>
               <fnm>James</fnm>
               <insr iid="I1"/>
               <email>jdolan787@gmail.com</email>
            </au>
            <au id="A2">
               <snm>Iadarola</snm>
               <fnm>Stephen</fnm>
               <insr iid="I1"/>
               <email>stephen_iadarola@urmc.rochester.edu</email>
            </au>
         </aug>
         <insg>
            <ins id="I1">
               <p>Department of Medicine, University of Rochester and Unity Health System, 1555 Long Pond Rd, Rochester, New York, USA</p>
            </ins>
         </insg>
         <source>BMC Medical Informatics and Decision Making</source>
         <issn>1472-6947</issn>
         <pubdate>2008</pubdate>
         <volume>8</volume>
         <issue>1</issue>
         <fpage>14</fpage>
         <url>http://www.biomedcentral.com/1472-6947/8/14</url>
         <xrefbib>
            <pubidlist>
               <pubid idtype="pmpid">18402680</pubid>
               <pubid idtype="doi">10.1186/1472-6947-8-14</pubid>
            </pubidlist>
         </xrefbib>
      </bibl>
      <history>
         <rec>
            <date>
               <day>03</day>
               <month>8</month>
               <year>2007</year>
            </date>
         </rec>
         <acc>
            <date>
               <day>10</day>
               <month>4</month>
               <year>2008</year>
            </date>
         </acc>
         <pub>
            <date>
               <day>10</day>
               <month>4</month>
               <year>2008</year>
            </date>
         </pub>
      </history>
      <cpyrt>
         <year>2008</year>
         <collab>Dolan and Iadarola; licensee BioMed Central Ltd.</collab>
         <note>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<url>http://creativecommons.org/licenses/by/2.0</url>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</note>
      </cpyrt>
      <abs>
         <sec>
            <st>
               <p>Abstract</p>
            </st>
            <sec>
               <st>
                  <p>Background</p>
               </st>
               <p>Clear communication about the possible outcomes of proposed medical interventions is an integral part of medical care. Despite its importance, there have been few studies comparing different formats for presenting probabilistic information to patients, especially when small probabilities are involved. The purpose of this study was to explore the potential usefulness of several new small-risk graphic communication formats.</p>
            </sec>
            <sec>
               <st>
                  <p>Methods</p>
               </st>
               <p>Information about the likelihoods of cancer and cancer prevention associated with two hypothetical cancer screening programs were used to create an augmented bar chart, an augmented grouped icon display, a flow chart, and three paired combinations of these formats. In the study scenario, the baseline risk of cancer was 53 per 1,000 (5.3%). The risk associated with cancer screening option A was 38 per 1,000 (3.8%) and the risk associated with screening option B was 29 per 1,000 (2.9%). Both the augmented bar chart and the augmented grouped icon display contained magnified views of the differences in cancer risk and cancer prevention associated with the screening programs. A convenience sample of 29 subjects (mean age 56.4 years; 76% men) used the Analytic Hierarchy Process (AHP) to indicate their relative preferences for the six formats using 15 sequential paired comparisons.</p>
            </sec>
            <sec>
               <st>
                  <p>Results</p>
               </st>
               <p>The most preferred format was the combined augmented bar chart + flow diagram (mean preference score 0.43) followed by the combined augmented icon + augmented bar chart format (mean preference score 0.22). The overall differences among the six formats were statistically significant: Kruskal-Wallis Chi Square = 141.4, p &lt; 0.0001. The three combined formats all had statistically significant higher preferences scores than the single format displays (p &lt; 0.05).</p>
            </sec>
            <sec>
               <st>
                  <p>Conclusion</p>
               </st>
               <p>These findings suggest that patients may prefer combined, rather than single, graphic risk presentation formats and that augmented bar charts and icon displays may be useful for conveying comparative information about small risks to clinical decision makers. Further research to confirm and extend these findings is warranted.</p>
            </sec>
         </sec>
      </abs>
   </fm>
   <meta>
      <classifications>
         <classification type="bmc" subtype="user_supplied_xml" id="endnote"/>
      </classifications>
   </meta>
   <bdy>
      <sec>
         <st>
            <p>Background</p>
         </st>
         <p>Clear communication about the possible risks and benefits of proposed medical interventions has been an integral part of medical care since the doctrine of informed consent was adopted. In recent years, it has become increasingly more important due to changes in the accepted model of the doctor-patient relationship that promote more active involvement of patients in decisions about their care, the rise of evidence-based medical practice, and the increasing emphasis placed on preventive measures that are intended to reduce future health risks <abbrgrp><abbr bid="B1">1</abbr><abbr bid="B2">2</abbr></abbrgrp>. It is likely to become more important in the future as new information about personal health risks becomes available through work in the "new sciences" such as medical genomics, metabolic profiling, and proteomics <abbrgrp><abbr bid="B3">3</abbr></abbrgrp>. Richard Smith, former editor of the British Medical Journal, recently called risk communication the "main work of doctors" <abbrgrp><abbr bid="B4">4</abbr></abbrgrp>.</p>
         <p>Current recommendations for communicating information about uncertain future events emphasize the importance of presenting data in a balanced manner that avoids framing effects, provides baseline risk information, and uses graphic risk displays whenever possible. Recommended risk displays include bar charts, grouped icon displays (also known as dot or face diagrams), and flow diagrams <abbrgrp><abbr bid="B5">5</abbr><abbr bid="B6">6</abbr><abbr bid="B7">7</abbr></abbrgrp>. The use of multiple risk presentation formats has also been suggested <abbrgrp><abbr bid="B7">7</abbr><abbr bid="B8">8</abbr></abbrgrp>.</p>
         <p>Despite the importance of risk communication, there have been few empirical studies comparing different graphic formats for presenting probabilistic information to patients for use in shared decision-making <abbrgrp><abbr bid="B8">8</abbr><abbr bid="B9">9</abbr><abbr bid="B10">10</abbr><abbr bid="B11">11</abbr><abbr bid="B12">12</abbr></abbrgrp>. In comparative studies, both bar charts and icon displays have been more effective than simple numeric statements and several other graphic formats for comparing the outcomes of alternative decision options <abbrgrp><abbr bid="B13">13</abbr><abbr bid="B14">14</abbr></abbrgrp>. Ordered icon displays have also been shown to minimize decision biases due to vivid anecdotal information and to be well understood by patients over 75 years old <abbrgrp><abbr bid="B16">16</abbr><abbr bid="B17">17</abbr></abbrgrp>. Flow diagrams, developed by Gigerenzer and colleagues, have been shown to help people better understand risk forecasts as well as more complex probabilistic data, such as the change in risk of disease after a positive or negative test result <abbrgrp><abbr bid="B18">18</abbr><abbr bid="B19">19</abbr></abbrgrp>.</p>
         <p>A limitation of the currently available data is that most studies to date have compared risk displays using relatively common events with likelihoods of 20% or more. Outcomes with expected likelihoods of less than 5% occur frequently in medical decision making situations and in many cases have an important impact on medical decisions. Probably the most common of these low likelihood outcomes are risks of serious adverse events. For example, in 2004 the drug rofecoxib, a COX-2 inhibitor, was removed from the market in the United States due to concerns about an increase in the annual rate of myocardial infarctions from one per thousand to four per thousand, an absolute risk increase of three per thousand <abbrgrp><abbr bid="B15">15</abbr></abbrgrp>.</p>
         <p>Small risks are difficult to depict in bar charts and icon displays because it is hard to include all of the necessary information in a single display. Studies with both bar charts and icon displays have shown that graphs that do not include information about the chances of both experiencing and not experiencing the outcome can lead to misinterpretation of the data presented <abbrgrp><abbr bid="B12">12</abbr><abbr bid="B16">16</abbr></abbrgrp>. The need to properly illustrate this part-to-whole relationship is what makes it difficult to use these formats to convey information about low likelihood events</p>
         <p>A possible solution to this problem is to use graphs that include a special section designed to highlight information about low likelihood events. This approach has been used successfully in two studies that elicited probability estimates from patients using graphs that included a "magnified" zone for probabilities less than 1% <abbrgrp><abbr bid="B22">22</abbr><abbr bid="B23">23</abbr></abbrgrp>. Whether this approach can also be used to create effective graphic displays for conveying low likelihood information to clinical decision makers is currently unknown.</p>
         <p>This study was designed as the first step in a series of investigations designed to address the question of how to most effectively present information about low likelihood events to patients and other clinical decision makers. The purpose of this study was to explore the potential usefulness of several new small-risk graphic communication formats.</p>
      </sec>
      <sec>
         <st>
            <p>Methods</p>
         </st>
         <sec>
            <st>
               <p>Study population</p>
            </st>
            <p>The study population consisted of a convenience sample of middle-aged to elderly adults, recruited through word of mouth and distribution of informational materials, who volunteered to participate in a study of medical decision making. Patients were recruited from three separate sites in Rochester NY: a general maintenance shop, a weekly social gathering at an elderly retirement community, and a medical research institute contained within the University of Rochester. No patients had prior knowledge of the project or the study hypotheses. Formal health status was not assessed. However, subjects at the maintenance shop and research institute were working regularly at the time of the study and the subjects at the retirement community were well enough to attend a weekly social function. This study was a pilot study performed in connection with a larger project that was approved by the University of Rochester Research Subjects Review Board. The data were collected anonymously. All subjects gave consent to participate.</p>
         </sec>
         <sec>
            <st>
               <p>Study intervention</p>
            </st>
            <p>The study intervention was a computer-assisted comparison of six different graphic communication formats for displaying the differences in effectiveness of two hypothetical cancer screening programs. In the study scenario, the baseline risk of cancer was 53 per 1,000 (5.3%). The risk associated with cancer screening option A was 38 per 1,000 (3.8%) and the risk associated with screening option B was 29 per 1,000 (2.9%). These absolute risks and risk reductions are comparable to outcomes seen with currently recommended colorectal cancer screening tests <abbrgrp><abbr bid="B17">17</abbr></abbrgrp>. Although screening by itself does not reduce cancer risk, we did not include details about screening-related treatments to simplify the problem and focus attention on the graphic displays.</p>
            <p>Three of the formats studied were based on currently recommended graphic displays: a flow diagram, a vertical bar chart, and a grouped icon display. The bar chart and icon display were augmented to include both an overall depiction of the possible outcomes associated with each screening option and a magnified view of the differences between them in cancer risk and likelihood of cancer prevention. The resulting diagrams are shown in Figures <figr fid="F1">1</figr>, <figr fid="F2">2</figr>, <figr fid="F3">3</figr>.</p>
            <fig id="F1">
               <title>
                  <p>Figure 1</p>
               </title>
               <caption>
                  <p>The augmented bar chart</p>
               </caption>
               <text>
                  <p><b>The augmented bar chart</b>. The left hand panel is a standard bar chart showing the entire dataset. The right hand panel magnifies the differences between the two options so the magnitude of the differences can be seen more clearly.</p>
               </text>
               <graphic file="1472-6947-8-14-1"/>
            </fig>
            <fig id="F2">
               <title>
                  <p>Figure 2</p>
               </title>
               <caption>
                  <p>The augmented icon display</p>
               </caption>
               <text>
                  <p><b>The augmented icon display</b>. The left hand panel is a standard icon display showing the entire dataset. The right hand panel magnifies the differences between the two options so the magnitude of the differences can be seen more clearly. The red diamonds indicate patients with cancer, the green diamonds indicate patients without cancer, and the broken diamond symbol (<inline-formula><graphic file="1472-6947-8-14-i1.gif"/></inline-formula>) indicates cancers prevented through screening and screening-related interventions.</p>
               </text>
               <graphic file="1472-6947-8-14-2"/>
            </fig>
            <fig id="F3">
               <title>
                  <p>Figure 3</p>
               </title>
               <caption>
                  <p>The flow diagram</p>
               </caption>
               <text>
                  <p>
                     <b>The flow diagram.</b>
                  </p>
               </text>
               <graphic file="1472-6947-8-14-3"/>
            </fig>
            <p>The other three formats included in the study were paired combinations of the three single formats: a flow diagram plus an augmented icon display, a flow diagram plus an augmented bar chart, and an augmented icon display plus an augmented bar chart. An example of a combined format is shown in Figure <figr fid="F4">4</figr>.</p>
            <fig id="F4">
               <title>
                  <p>Figure 4</p>
               </title>
               <caption>
                  <p>Example paired display format</p>
               </caption>
               <text>
                  <p><b>Example paired display format</b>. This figure shows the combined format consisting of both an augmented bar chart and a flow diagram.</p>
               </text>
               <graphic file="1472-6947-8-14-4"/>
            </fig>
            <p>After a brief explanation of the study and a training exercise that involved judging the relative weights of three balls, the subjects were shown two of the six diagrams on a computer screen and asked to indicate if they liked both formats equally or if they preferred one to the other for comparing the effectiveness of the two screening options for preventing cancer. If one was preferred, they were asked to indicate their strength of preference using a dynamic slider that showed a range of preferences from equal to nine times more preferable. The relationships between different settings on the slider and relative strengths of preference were illustrated by linked horizontal bar and pie charts. An example of the screen used to make these comparisons is shown in Figure <figr fid="F5">5</figr>. This process was repeated until all six formats were compared, a total of 15 comparisons. Participants needed approximately 45 minutes to one hour to complete the study. The computer program used to perform these assessments is attached: see Additional file <supplr sid="S1">1</supplr>.</p>
            <suppl id="S1">
               <title>
                  <p>Additional file 1</p>
               </title>
               <text>
                  <p>Risk communication survey.swf. The instrument used to obtain the preferences from study subjects.</p>
               </text>
               <file name="1472-6947-8-14-S1.swf">
                  <p>Click here for file</p>
               </file>
            </suppl>
            <fig id="F5">
               <title>
                  <p>Figure 5</p>
               </title>
               <caption>
                  <p>Example preference comparison screenshot</p>
               </caption>
               <text>
                  <p><b>Example preference comparison screenshot</b>. This figure shows the screen used by the study subjects to make the comparisons among the risk presentation formats. The slider used to indicate their strength of preference, if any, is shown in the top panel. The magnitude of preference was indicated in the numeric box to the right and in the linked horizontal bar charts and pie chart below. The panel in the upper left is the menu screen used to move from one comparison to the next.</p>
               </text>
               <graphic file="1472-6947-8-14-5"/>
            </fig>
            <p>Quantitative preference scores summarizing the comparisons among the formats were calculated using the Analytic Hierarchy Process (AHP). To create the AHP-based numeric preference scores, each subject's comparisons were entered into a matrix and the normalized right eigenvector of the matrix was calculated. The resulting eigenvector values define the formats' preference scores. Because the eigenvector is normalized, the scores range from 0 to 1 and the sum of all scores equals 1. Scores are interpreted as indicating each format's relative strength of preference compared to the others, measured on a ratio scale.</p>
            <p>The reliability of each subject's comparisons was assessed by calculating the consistency ratio, a standard measure of the reliability of the pairwise comparisons used to generate the preference scale that is routinely used in AHP applications <abbrgrp><abbr bid="B22">22</abbr></abbrgrp>. Consistency ratios run from 0 to 1, with 0 indicating a perfectly consistent set of comparisons. Traditionally, consistency ratios &#8804; 0.10 have been considered acceptable; in practical applications this cut-off is often increased to &#8804; 0.20 <abbrgrp><abbr bid="B25">25</abbr></abbrgrp>. For this study the latter cutoff was used and defined <it>a priori</it>.</p>
         </sec>
         <sec>
            <st>
               <p>Statistical analysis</p>
            </st>
            <p>We analyzed the data to see if there were significant differences among the six presentation formats in subject preferences. Because the response data were not normally distributed, we tested this hypothesis using Kruskal-Wallis analysis of variance. The level of statistical significance was set <it>a priori </it>as an alpha level of 0.05. Differences between individual formats were tested using the Bonferroni multiple comparison test to maintain an alpha of 0.05. All statistical analyses were done using NCSS 2000 <abbrgrp><abbr bid="B26">26</abbr></abbrgrp>.</p>
         </sec>
      </sec>
      <sec>
         <st>
            <p>Results</p>
         </st>
         <p>The mean age of the study population was 56.4 years. There were 22 men and 7 women. The preference scores for the six presentation formats are summarized in the box plot shown in Figure <figr fid="F6">6</figr>. The most preferred format was the combined augmented bar chart + flow diagram with a mean preference score 0.43. This format is shown in Figure <figr fid="F4">4</figr>. The combined augmented icon display + bar chart was second most preferred with a mean preference score of 0.22. A small group of patients strongly preferred this format as indicated by the outlier circles on the box plot. The combined augmented icon display + flow chart had the third highest mean preference score, 0.15. The three single presentation formats were the least preferred with mean preference scores of 0.10 for the augmented bar chart, 0.07 for the flow diagram, and 0.04 for the augmented icon display.</p>
         <fig id="F6">
            <title>
               <p>Figure 6</p>
            </title>
            <caption>
               <p>Box plot showing the preferences scores for the six risk presentation formats</p>
            </caption>
            <text>
               <p><b>Box plot showing the preferences scores for the six risk presentation formats</b>. Abbreviations: BF = augmented bar chart + flow diagram; IB = augmented icon display + augmented bar chart; IF = augmented icon display + flow chart; Bars = augmented bar chart; Flow = flow diagram; Icons = augmented icon display. Box plot details: The horizontal line inside the box indicates the median value. The inner box indicates the inter-quartile range that runs between the 25<sup>th </sup>to 75<sup>th </sup>percentiles. The upper line extending from the box indicates the largest value between the 75<sup>th </sup>percentile and the point that is 1.5 times the inter-quartile range. The lower line extending from the box indicates the smallest point between the 25<sup>th </sup>percentile and 1.5 times the inter-quartile range. The circles represent values that lie outside these ranges.</p>
            </text>
            <graphic file="1472-6947-8-14-6"/>
         </fig>
         <p>The overall differences in preferences among the six formats were statistically significant: Kruskal-Wallis Chi Square = 141.4, p &lt; 0.0001. The Bonferroni multiple comparison test indicated that the mean preference scores for all three combined formats were significantly different from all other options and the mean preference score for the augmented bar chart was significantly different from the mean score of the augmented icon display, (p &lt; 0.05).</p>
         <p>The consistency indices for the comparisons ranged from 0.08 to 0.20 with a mean of 0.135. All subjects met the predefined standard for acceptable consistency of 0.20 or less.</p>
      </sec>
      <sec>
         <st>
            <p>Discussion</p>
         </st>
         <p>A recently issued set of guidelines for creating patient decision aids recommends the use of multiple risk presentation formats <abbrgrp><abbr bid="B7">7</abbr></abbrgrp>. These results support this recommendation. The most preferred format was a combined format and all three combined formats were more preferred than the three single format options included in the study.</p>
         <p>We were unable to find previous studies that have assessed patient preferences for single versus combined graphic risk presentation formats. In a study that compared text formats for presenting treatment effectiveness information, Sheridan and colleagues found that patient understanding was worse for a combined format than several single formats <abbrgrp><abbr bid="B18">18</abbr></abbrgrp>. The discrepancy between this result and ours may reflect differences in the way people process textual versus graphical information <abbrgrp><abbr bid="B19">19</abbr><abbr bid="B20">20</abbr></abbrgrp>. Alternatively, it could be due to the differences in the outcomes studied: preferences versus understanding.</p>
         <p>Given the limited scope of this study, we do not know why the patients preferred the combined formats so strongly. The cost-benefit theory of decision strategy choice posits that the way information is displayed affects a person's choice of how they will process the information based on anticipated effort and anticipated accuracy <abbrgrp><abbr bid="B19">19</abbr></abbrgrp>. It is possible that the increased variety of information presentation provided by the combined formats led the patients to perceive that they could be used to make more accurate decisions with less effort than the single format displays. Additional research is warranted to confirm our results and to address additional outcomes including understanding of the information presented, cognitive effort, the rationale underlying risk presentation preferences, and the effect of presentation preferences on the decision making process.</p>
         <p>This study also demonstrates the usefulness of using the Analytic Hierarchy Process as a technique for creating measurement scales in situations where no pre-existing scale exists. Although the AHP is most widely known as a multi-criteria decision making method, it is fundamentally a system of measurement that derives a ratio-level scale from a series of pairwise judgments <abbrgrp><abbr bid="B21">21</abbr><abbr bid="B22">22</abbr><abbr bid="B23">23</abbr><abbr bid="B24">24</abbr><abbr bid="B25">25</abbr><abbr bid="B26">26</abbr><abbr bid="B27">27</abbr><abbr bid="B28">28</abbr></abbrgrp>. In addition to producing a robust measurement scale, the AHP also yields an assessment of the consistency of the component judgments that can be used as an indicator of the reliability of the measurement process. The AHP therefore is a valuable assessment tool that can enhance our ability to investigate preferences and other important, but difficult to measure, concepts.</p>
         <p>This study has several limitations. First, the only outcome studied was patient preference. At least two studies have shown that the most preferred risk formats are not necessarily the same as the most effective formats for risk communication in medical decision making <abbrgrp><abbr bid="B13">13</abbr><abbr bid="B14">14</abbr></abbrgrp>. Assessing preferences as the initial step in evaluating new risk communication formats, however, has the advantage of providing easy to obtain information about an important dimension of a risk display that can be used to help plan additional investigations. Second, only two likelihoods for a single outcome were studied. We were, therefore, not able to determine if there is an interaction between the magnitude of the risk being presented and preferred display format. Third, the study population was a small volunteer convenience sample of people who were not facing a real decision. Fourth, the reproducibility of preferences over time was not tested. Finally, only one version of the newly created graphs was studied. Whether patient preferences are affected by different color schemes, axis formatting, the size of the display, and other design characteristics is unknown.</p>
      </sec>
      <sec>
         <st>
            <p>Conclusion</p>
         </st>
         <p>Despite these limitations, the results of this study suggest that patients may prefer combined, rather than single, graphic risk presentation formats and that augmented bar charts and icon displays may be useful for conveying comparative information about small risks to clinical decision makers. Further research to confirm and extend these findings is warranted.</p>
      </sec>
      <sec>
         <st>
            <p>Competing interests</p>
         </st>
         <p>The author(s) declare that they have no competing interests.</p>
      </sec>
      <sec>
         <st>
            <p>Authors' contributions</p>
         </st>
         <p>JD conceived and designed the study, performed the data analysis, and drafted the manuscript. SI created the data collection computer interface, recruited study subjects, administered the study interviews and compiled the results. Both authors read and approved the final manuscript.</p>
      </sec>
   </bdy>
   <bm>
      <ack>
         <sec>
            <st>
               <p>Acknowledgements</p>
            </st>
            <p>This study was supported by a grant from the National Cancer Institute: 1R01CA112366-01A1. The authors are indebted to the four peer reviewers for their insightful comments and suggestions regarding earlier drafts of this manuscript.</p>
         </sec>
      </ack>
      <refgrp>
         <bibl id="B1">
            <title>
               <p>Shared decisionmaking about screening and chemoprevention</p>
            </title>
            <aug>
               <au>
                  <snm>Sheridan</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Harris</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Woolf</snm>
                  <fnm>S</fnm>
               </au>
            </aug>
            <source>Am J Prev Med</source>
            <pubdate>2004</pubdate>
            <volume>26</volume>
            <issue>1</issue>
            <fpage>67</fpage>
            <lpage>80</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/j.amepre.2003.09.011</pubid>
                  <pubid idtype="pmpid" link="fulltext">14700715</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B2">
            <title>
               <p>Promoting informed choice: transforming health care to dispense knowledge for decision making</p>
            </title>
            <aug>
               <au>
                  <snm>Woolf</snm>
                  <fnm>SH</fnm>
               </au>
               <au>
                  <snm>Chan</snm>
                  <fnm>EC</fnm>
               </au>
               <au>
                  <snm>Harris</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Sheridan</snm>
                  <fnm>SL</fnm>
               </au>
               <au>
                  <snm>Braddock</snm>
                  <fnm>CH</fnm>
                  <suf>3rd</suf>
               </au>
               <au>
                  <snm>Kaplan</snm>
                  <fnm>RM</fnm>
               </au>
               <au>
                  <snm>Krist</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>O'Connor</snm>
                  <fnm>AM</fnm>
               </au>
               <au>
                  <snm>Tunis</snm>
                  <fnm>S</fnm>
               </au>
            </aug>
            <source>Ann Intern Med</source>
            <pubdate>2005</pubdate>
            <volume>143</volume>
            <issue>4</issue>
            <fpage>293</fpage>
            <lpage>300</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">16103473</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B3">
            <title>
               <p>Cardiovascular Proteomics: Tools to Develop Novel Biomarkers and Potential Applications</p>
            </title>
            <aug>
               <au>
                  <snm>Arab</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Gramolini</snm>
                  <fnm>AO</fnm>
               </au>
               <au>
                  <snm>Ping</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Kislinger</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Stanley</snm>
                  <fnm>B</fnm>
               </au>
               <au>
                  <snm>van Eyk</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Ouzounian</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>MacLennan</snm>
                  <fnm>DH</fnm>
               </au>
               <au>
                  <snm>Emili</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Liu</snm>
                  <fnm>PP</fnm>
               </au>
            </aug>
            <source>Journal of the American College of Cardiology</source>
            <pubdate>2006</pubdate>
            <volume>48</volume>
            <issue>9</issue>
            <fpage>1733</fpage>
            <lpage>1741</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/j.jacc.2006.06.063</pubid>
                  <pubid idtype="pmpid" link="fulltext">17084242</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B4">
            <title>
               <p>Communicating risk: the main work of doctors</p>
            </title>
            <aug>
               <au>
                  <snm>Smith</snm>
                  <fnm>R</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2003</pubdate>
            <volume>327</volume>
            <issue>7417</issue>
         </bibl>
         <bibl id="B5">
            <title>
               <p>Explaining risks: turning numerical data into meaningful pictures</p>
            </title>
            <aug>
               <au>
                  <snm>Edwards</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Elwyn</snm>
                  <fnm>G</fnm>
               </au>
               <au>
                  <snm>Mulley</snm>
                  <fnm>A</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2002</pubdate>
            <volume>324</volume>
            <issue>7341</issue>
            <fpage>827</fpage>
            <lpage>830</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1122766</pubid>
                  <pubid idtype="pmpid" link="fulltext">11934777</pubid>
                  <pubid idtype="doi">10.1136/bmj.324.7341.827</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B6">
            <title>
               <p>Strategies to help patients understand risks</p>
            </title>
            <aug>
               <au>
                  <snm>Paling</snm>
                  <fnm>J</fnm>
               </au>
            </aug>
            <source>BMJ</source>
            <pubdate>2003</pubdate>
            <volume>327</volume>
            <issue>7417</issue>
            <fpage>745</fpage>
            <lpage>748</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">200818</pubid>
                  <pubid idtype="pmpid" link="fulltext">14512489</pubid>
                  <pubid idtype="doi">10.1136/bmj.327.7417.745</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B7">
            <title>
               <p>International Patient Decision Aid Standards (IPDAS): beyond decision aids to usual design of patient education materials</p>
            </title>
            <aug>
               <au>
                  <snm>Holmes-Rovner</snm>
                  <fnm>M</fnm>
               </au>
            </aug>
            <source>Health Expect</source>
            <pubdate>2007</pubdate>
            <volume>10</volume>
            <issue>2</issue>
            <fpage>103</fpage>
            <lpage>7</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1369-7625.2007.00445.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">17524003</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B8">
            <title>
               <p>Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations</p>
            </title>
            <aug>
               <au>
                  <snm>Lipkus</snm>
                  <fnm>I</fnm>
               </au>
            </aug>
            <source>mdm</source>
            <pubdate>2007</pubdate>
            <volume>27</volume>
            <issue>5</issue>
            <fpage>696</fpage>
            <lpage>713</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmpid" link="fulltext">17873259</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B9">
            <title>
               <p>The visual communication of risk</p>
            </title>
            <aug>
               <au>
                  <snm>Lipkus</snm>
                  <fnm>IM</fnm>
               </au>
               <au>
                  <snm>Hollands</snm>
                  <fnm>JG</fnm>
               </au>
            </aug>
            <source>J Natl Cancer Inst Monogr</source>
            <pubdate>1999</pubdate>
            <issue>25</issue>
            <fpage>149</fpage>
            <lpage>163</lpage>
            <xrefbib>
               <pubid idtype="pmpid" link="fulltext">10854471</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B10">
            <title>
               <p>Patient comprehension of information for shared treatment decision making: state of the art and future directions</p>
            </title>
            <aug>
               <au>
                  <snm>Wills</snm>
                  <fnm>CE</fnm>
               </au>
               <au>
                  <snm>Holmes-Rovner</snm>
                  <fnm>M</fnm>
               </au>
            </aug>
            <source>Patient Education and Counseling</source>
            <pubdate>2003</pubdate>
            <volume>50</volume>
            <issue>3</issue>
            <fpage>285</fpage>
            <lpage>290</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1016/S0738-3991(03)00051-X</pubid>
                  <pubid idtype="pmpid" link="fulltext">12900101</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B11">
            <title>
               <p>A systematic review on communicating with patients about evidence</p>
            </title>
            <aug>
               <au>
                  <snm>Trevena</snm>
                  <fnm>LJ</fnm>
               </au>
               <au>
                  <snm>Davey</snm>
                  <fnm>HM</fnm>
               </au>
               <au>
                  <snm>Barratt</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Butow</snm>
                  <fnm>P</fnm>
               </au>
               <au>
                  <snm>Caldwell</snm>
                  <fnm>P</fnm>
               </au>
            </aug>
            <source>J Eval Clin Pract</source>
            <pubdate>2006</pubdate>
            <volume>12</volume>
            <issue>1</issue>
            <fpage>13</fpage>
            <lpage>23</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1111/j.1365-2753.2005.00596.x</pubid>
                  <pubid idtype="pmpid" link="fulltext">16422776</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B12">
            <title>
               <p>Design Features of Graphs in Health Risk Communication: A Systematic Review</p>
            </title>
            <aug>
               <au>
                  <snm>Ancker</snm>
                  <fnm>JS</fnm>
               </au>
               <au>
                  <snm>Senathirajah</snm>
                  <fnm>Y</fnm>
               </au>
               <au>
                  <snm>Kukafka</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Starren</snm>
                  <fnm>JB</fnm>
               </au>
            </aug>
            <source>J Am Med Inform Assoc</source>
            <pubdate>2006</pubdate>
            <volume>13</volume>
            <issue>6</issue>
            <fpage>608</fpage>
            <lpage>618</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1656964</pubid>
                  <pubid idtype="pmpid" link="fulltext">16929039</pubid>
                  <pubid idtype="doi">10.1197/jamia.M2115</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B13">
            <title>
               <p>Perception of quantitative information for treatment decisions</p>
            </title>
            <aug>
               <au>
                  <snm>Feldman-Stewart</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Kocovski</snm>
                  <fnm>N</fnm>
               </au>
               <au>
                  <snm>McConnell</snm>
                  <fnm>BA</fnm>
               </au>
               <au>
                  <snm>Brundage</snm>
                  <fnm>MD</fnm>
               </au>
               <au>
                  <snm>Mackillop</snm>
                  <fnm>WJ</fnm>
               </au>
            </aug>
            <source>Med Decis Making</source>
            <pubdate>2000</pubdate>
            <volume>20</volume>
            <issue>2</issue>
            <fpage>228</fpage>
            <lpage>238</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1177/0272989X0002000208</pubid>
                  <pubid idtype="pmpid" link="fulltext">10772360</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B14">
            <title>
               <p>Formats for improving risk communication in medical tradeoff decisions</p>
            </title>
            <aug>
               <au>
                  <snm>Waters</snm>
                  <fnm>EA</fnm>
               </au>
               <au>
                  <snm>Weinstein</snm>
                  <fnm>ND</fnm>
               </au>
               <au>
                  <snm>Colditz</snm>
                  <fnm>GA</fnm>
               </au>
               <au>
                  <snm>Emmons</snm>
                  <fnm>K</fnm>
               </au>
            </aug>
            <source>J Health Commun</source>
            <pubdate>2006</pubdate>
            <volume>11</volume>
            <issue>2</issue>
            <fpage>167</fpage>
            <lpage>182</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1080/10810730500526695</pubid>
                  <pubid idtype="pmpid" link="fulltext">16537286</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B15">
            <title>
               <p>Comparative Effectiveness and Safety of Analgesics for Osteoarthritis. Comparative Effectiveness Review No. 4</p>
            </title>
            <aug>
               <au>
                  <snm>Chou</snm>
                  <fnm>R</fnm>
               </au>
               <au>
                  <snm>Helfand</snm>
                  <fnm>M</fnm>
               </au>
               <au>
                  <snm>Peterson</snm>
                  <fnm>K</fnm>
               </au>
               <au>
                  <snm>Dana</snm>
                  <fnm>T</fnm>
               </au>
               <au>
                  <snm>Roberts</snm>
                  <fnm>C</fnm>
               </au>
            </aug>
            <publisher>Rockville, MD: Agency for Healthcare Research and Quality</publisher>
            <pubdate>2006</pubdate>
            <url>http://www.effectivehealthcare.ahrq.gov/reports/final.cfm</url>
         </bibl>
         <bibl id="B16">
            <title>
               <p>Foreground:background salience: explaining the effects of graphical displays on risk avoidance</p>
            </title>
            <aug>
               <au>
                  <snm>Stone</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Sieck</snm>
                  <fnm>W</fnm>
               </au>
               <au>
                  <snm>Bull</snm>
                  <fnm>B</fnm>
               </au>
               <au>
                  <snm>Yates</snm>
                  <fnm>J</fnm>
               </au>
               <au>
                  <snm>Parks</snm>
                  <fnm>S</fnm>
               </au>
               <au>
                  <snm>Rush</snm>
                  <fnm>C</fnm>
               </au>
            </aug>
            <source>Organizational Behavior and Human Decision Processes</source>
            <pubdate>2003</pubdate>
            <volume>90</volume>
            <issue>1</issue>
            <fpage>19</fpage>
            <lpage>36</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1016/S0749-5978(03)00003-7</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B17">
            <title>
               <p>Colorectal cancer screening: clinical guidelines and rationale</p>
            </title>
            <aug>
               <au>
                  <snm>Winawer</snm>
                  <fnm>SJ</fnm>
               </au>
               <au>
                  <snm>Fletcher</snm>
                  <fnm>RH</fnm>
               </au>
               <au>
                  <snm>Miller</snm>
                  <fnm>L</fnm>
               </au>
               <au>
                  <snm>Godlee</snm>
                  <fnm>F</fnm>
               </au>
               <au>
                  <snm>Stolar</snm>
                  <fnm>MH</fnm>
               </au>
               <au>
                  <snm>Mulrow</snm>
                  <fnm>CD</fnm>
               </au>
               <au>
                  <snm>Woolf</snm>
                  <fnm>SH</fnm>
               </au>
               <au>
                  <snm>Glick</snm>
                  <fnm>SN</fnm>
               </au>
               <au>
                  <snm>Ganiats</snm>
                  <fnm>TG</fnm>
               </au>
               <au>
                  <snm>Bond</snm>
                  <fnm>JH</fnm>
               </au>
               <etal/>
            </aug>
            <source>Gastroenterology</source>
            <pubdate>1997</pubdate>
            <volume>112</volume>
            <issue>2</issue>
            <fpage>594</fpage>
            <lpage>642</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1053/gast.1997.v112.agast970594</pubid>
                  <pubid idtype="pmpid">9024315</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B18">
            <title>
               <p>A randomized comparison of patients' understanding of number needed to treat and other common risk reduction formats</p>
            </title>
            <aug>
               <au>
                  <snm>Sheridan</snm>
                  <fnm>SL</fnm>
               </au>
               <au>
                  <snm>Pignone</snm>
                  <fnm>MP</fnm>
               </au>
               <au>
                  <snm>Lewis</snm>
                  <fnm>CL</fnm>
               </au>
            </aug>
            <source>J Gen Intern Med</source>
            <pubdate>2003</pubdate>
            <volume>18</volume>
            <issue>11</issue>
            <fpage>884</fpage>
            <lpage>892</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="pmcid">1494938</pubid>
                  <pubid idtype="pmpid" link="fulltext">14687273</pubid>
                  <pubid idtype="doi">10.1046/j.1525-1497.2003.21102.x</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B19">
            <title>
               <p>Information displays and decision processes</p>
            </title>
            <aug>
               <au>
                  <snm>Kleinmuntz</snm>
                  <fnm>D</fnm>
               </au>
               <au>
                  <snm>Schkade</snm>
                  <fnm>D</fnm>
               </au>
            </aug>
            <source>Psychological Science</source>
            <pubdate>1993</pubdate>
            <volume>4</volume>
            <issue>4</issue>
            <fpage>221</fpage>
            <lpage>227</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1111/j.1467-9280.1993.tb00265.x</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B20">
            <title>
               <p>Does evidence representation format affect judgment? An experimental evaluation of displays of data for judgment</p>
            </title>
            <aug>
               <au>
                  <snm>Sanfey</snm>
                  <fnm>A</fnm>
               </au>
               <au>
                  <snm>Hastie</snm>
                  <fnm>R</fnm>
               </au>
            </aug>
            <source>Psychological Science</source>
            <pubdate>1998</pubdate>
            <volume>9</volume>
            <issue>2</issue>
            <fpage>99</fpage>
            <lpage>103</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1111/1467-9280.00018</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B21">
            <title>
               <p>The Analytic Hierarchy Process</p>
            </title>
            <aug>
               <au>
                  <snm>Saaty</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <publisher>New York: McGraw-Hill Book Co</publisher>
            <pubdate>1980</pubdate>
         </bibl>
         <bibl id="B22">
            <title>
               <p>Axiomatic foundations of the analytic hierarchy process</p>
            </title>
            <aug>
               <au>
                  <snm>Saaty</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <source>Management Science</source>
            <pubdate>1986</pubdate>
            <volume>32</volume>
            <issue>7</issue>
            <fpage>841</fpage>
            <lpage>855</lpage>
         </bibl>
         <bibl id="B23">
            <title>
               <p>How to make a decision: The Analytic Hierarchy Process</p>
            </title>
            <aug>
               <au>
                  <snm>Saaty</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <source>Interfaces</source>
            <pubdate>1994</pubdate>
            <volume>24</volume>
            <fpage>19</fpage>
            <lpage>43</lpage>
         </bibl>
         <bibl id="B24">
            <title>
               <p>Decision Making for Leaders</p>
            </title>
            <aug>
               <au>
                  <snm>Saaty</snm>
                  <fnm>T</fnm>
               </au>
            </aug>
            <publisher>Pittsburgh, PA: RWS Publications</publisher>
            <edition>3</edition>
            <pubdate>2001</pubdate>
         </bibl>
         <bibl id="B25">
            <title>
               <p>The analytic hierarchy process in medical decision making: a tutorial</p>
            </title>
            <aug>
               <au>
                  <snm>Dolan</snm>
                  <fnm>JG</fnm>
               </au>
               <au>
                  <snm>Isselhardt</snm>
                  <fnm>BJ</fnm>
                  <suf>Jr</suf>
               </au>
               <au>
                  <snm>Cappuccio</snm>
                  <fnm>JD</fnm>
               </au>
            </aug>
            <source>Med Decis Making</source>
            <pubdate>1989</pubdate>
            <volume>9</volume>
            <issue>1</issue>
            <fpage>40</fpage>
            <lpage>50</lpage>
            <xrefbib>
               <pubidlist>
                  <pubid idtype="doi">10.1177/0272989X8900900108</pubid>
                  <pubid idtype="pmpid">2643019</pubid>
               </pubidlist>
            </xrefbib>
         </bibl>
         <bibl id="B26">
            <title>
               <p>Model world: When is a number a number?</p>
            </title>
            <aug>
               <au>
                  <snm>Gass</snm>
                  <fnm>S</fnm>
               </au>
            </aug>
            <source>Interfaces</source>
            <pubdate>2001</pubdate>
            <volume>31</volume>
            <issue>5</issue>
            <fpage>93</fpage>
            <lpage>103</lpage>
         </bibl>
         <bibl id="B27">
            <title>
               <p>The Analytic Hierarchy Process &#8211; An Exposition</p>
            </title>
            <aug>
               <au>
                  <snm>Forman</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Gass</snm>
                  <fnm>S</fnm>
               </au>
            </aug>
            <source>Operations Research</source>
            <pubdate>2001</pubdate>
            <volume>49</volume>
            <fpage>469</fpage>
            <lpage>486</lpage>
            <xrefbib>
               <pubid idtype="doi">10.1287/opre.49.4.469.11231</pubid>
            </xrefbib>
         </bibl>
         <bibl id="B28">
            <title>
               <p>Decision by objectives</p>
            </title>
            <aug>
               <au>
                  <snm>Forman</snm>
                  <fnm>E</fnm>
               </au>
               <au>
                  <snm>Selly</snm>
                  <fnm>M</fnm>
               </au>
            </aug>
            <publisher>Hackensack, NJ: World Scientific</publisher>
            <pubdate>2001</pubdate>
         </bibl>
      </refgrp>
      <sec>
         <st>
            <p>Pre-publication history</p>
         </st>
         <p>The pre-publication history for this paper can be accessed here:</p>
         <p>
            <url>http://www.biomedcentral.com/1472-6947/8/14/prepub</url>
         </p>
      </sec>
   </bm>
</art>
