Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Launching a virtual decision lab: development and field-testing of a web-based patient decision support research platform

BMC Medical Informatics and Decision Making volume 14, Article number: 112 (2014) Cite this article

Abstract

Background

Over 100 trials show that patient decision aids effectively improve patients’ information comprehension and values-based decision making. However, gaps remain in our understanding of several fundamental and applied questions, particularly related to the design of interactive, personalized decision aids. This paper describes an interdisciplinary development process for, and early field testing of, a web-based patient decision support research platform, or virtual decision lab, to address these questions.

Methods

An interdisciplinary stakeholder panel designed the web-based research platform with three components: a) an introduction to shared decision making, b) a web-based patient decision aid, and c) interactive data collection items. Iterative focus groups provided feedback on paper drafts and online prototypes. A field test assessed a) feasibility for using the research platform, in terms of recruitment, usage, and acceptability; and b) feasibility of using the web-based decision aid component, compared to performance of a videobooklet decision aid in clinical care.

Results

This interdisciplinary, theory-based, patient-centered design approach produced a prototype for field-testing in six months. Participants (n = 126) reported that: the decision aid component was easy to use (98%), information was clear (90%), the length was appropriate (100%), it was appropriately detailed (90%), and it held their interest (97%). They spent a mean of 36 minutes using the decision aid and 100% preferred using their home/library computer. Participants scored a mean of 75% correct on the Decision Quality, Knowledge Subscale, and 74 out of 100 on the Preparation for Decision Making Scale. Completing the web-based decision aid reduced mean Decisional Conflict scores from 31.1 to 19.5 (p < 0.01).

Conclusions

Combining decision science and health informatics approaches facilitated rapid development of a web-based patient decision support research platform that was feasible for use in research studies in terms of recruitment, acceptability, and usage. Within this platform, the web-based decision aid component performed comparably with the videobooklet decision aid used in clinical practice. Future studies may use this interactive research platform to study patients’ decision making processes in real-time, explore interdisciplinary approaches to designing web-based decision aids, and test strategies for tailoring decision support to meet patients’ needs and preferences.

Peer Review reports

Background

The 2014 Cochrane Collaboration’s review of 115 trials reported that patients who used decision aids had improved knowledge of the options, more realistic expectations of potential benefits/harms, and greater clarity about what matters most to them [1]. They were also more likely to participate in decision making, and to report improved communication with their doctors.

However, several gaps remain in our understanding of the processes whereby patients engage in decision making, and of the ways in which some features of decision aids might interact with those decision-making processes [1]-[4]. For example, the Cochrane Collaboration’s review noted that comparisons of simple and complex decision aid formats generated mixed effects, and highlighted the need for additional studies of interactivity and presentation formats [1].

Other studies have also raised questions about the potential for targeting or tailoring patient decision aids to address patients’ preferences for different levels of information and decision support [2]-[10]. For example, when patients are using a decision aid, particularly delivered online, to deliberate about a tough health care decision, do monitoring and blunting coping styles persist for both information seeking and the level of engagement in active deliberation? Are there groups of patients who seek detailed risk/benefit information (e.g., “monitors” [11]) and benefit from explicit, step-wise deliberative guidance, while other groups favor a gestalt-like overview of the key facts (e.g., “blunters”) and benefit more from implicit guidance? [12],[13] Would such “deliberative styles” tend to be trait-like, in that patients use the same deliberative style across a wide range of preventive, screening, acute-, and chronic-care health decisions? Or are these styles state-like, in that patients use different deliberative styles in accordance with what’s at stake in a particular health decision? For patients with chronic conditions, do their deliberative styles change over time as they gain decision-making skills? Furthermore, does the match/mismatch between the patients’ deliberative style and the type of decision aid they receive affect their information comprehension, decisional conflict, or decision quality?

One proposed approach to addressing these questions is to create a virtual decision support laboratory [3],[14]. Such a laboratory could include a web-based research platform that: a) presents a patient decision aid; b) has interactive features that allow patients to self-tailor the clinical information and deliberative support they receive to meet their decision-making needs and deliberative style; and c) incorporates data collection items for research. Researchers could then observe patients’ decision-making processes in real-time, and also could vary components of the decision aid and/or its interactive features in order to test various theories and design strategies.

There are several challenges here. An interdisciplinary approach is needed to design a web-based research platform with these capabilities [4]. Several studies of web-based patient decision aids have reported positive effects on information comprehension and acceptability to patients, but studies have also reported challenges in routine use [15]-[29]. The recent International Patient Decision Aid Standards (IPDAS) Collaboration’s report on Delivering Decision Aids Using the Internet recognized the potential gains of web-based decision support, but reported mixed results about the best practices for development, evaluation, and implementation [4]. In particular, it noted the paucity of studies reporting development and evaluation approaches derived from consumer health informatics and human factors fields. Therefore, the long-term goal of this planned program of research is to build a virtual decision lab and use it to investigate both fundamental questions about the design of interactive and tailored decision aids, as well as applied questions about the effects of a match/mismatch between patients’ deliberative styles and the types of decision support they receive on decision-making outcomes.

This paper describes the development and field-testing process used to create the virtual decision lab, which had three primary objectives. The first objective was to use an interdisciplinary, theory-based, patient-centered approach to designing a web-based decision support research platform containing three components – an introduction to shared decision making, a web-based patient decision aid with interactive features, and data collection items. The second objective was to test the feasibility of using a) this web-based research platform (in terms of recruitment, acceptability, and usage) and b) the web-based decision aid component (in terms of its performance compared with the videobooklet decision aid used in clinical practice), for planned future research studies.

Methods

Figure 1 illustrates the study design for the development and field-testing of the web-based research platform.

Figure 1
figure 1

Study design: integrating decision support and health informatics approaches for rapid-cycle development and field-testing.

Full size image

Development

To ensure that our design process addressed multiple users’ needs, we formed a stakeholder advisory panel consisting of four patients, two clinicians, two decision scientists, two decision counselors, and two health informaticians. The advisory panel selected three publications to guide development. From a decision science perspective, the Ottawa Decision Aid Development Workbook describes a protocol for theory-based, evidence-based, and patient-centered development of patients’ decision aids [30],[31]. Second, the International Patients’ Decision Aid Standards Collaboration’s guidelines provide 63 quality criteria for patient decision aids: 27 criteria are related to the content of a decision aid, 30 to its development process, and 6 to the evaluation of its effectiveness [2]. Third, from a consumer health informatics perspective, the Human-Computer Interaction Handbook describes methods for identifying users’ needs and for improving the design of web-based tools by engaging users in rapid iterative cycles of design, evaluation, and redesign [32].

The panel guided development of the web-based research platform, which contained the three components: a) an introduction to the decision-making situation and baseline data collection items; b) a patient decision aid containing up-to-date clinical information and guidance in four interactive deliberative steps, with embedded process data collection items; and c) a printable Personal Decision Summary and post-decision aid data collection items. Development proceeded in three phases.

Selecting the clinical context

Because the focus of this study was to rapidly develop a web-based research platform containing a patient decision aid, the stakeholder advisory panel sought a clinical context that did not involve a hypothetical, acute, or life-threatening decision, and for which there existed a high-quality decision aid that could provide up-to-date clinical content. From among several decision dilemmas considered, the panel selected chronic knee osteoarthritis as the clinical context for this initial study for three reasons.

First, prevalence, incidence, and surgical utilization rates for the management of chronic knee pain are high and variable. Worldwide, prevalence estimates range from 2,369 to 20,238 per 100,000 men over 45 years old, and incidence ranges from 5 to 1,263 per 100,000 person-years (21 to 27 million in the U.S.) [33]-[35]. Estimates are 1.5 - 2.0 times higher for women. The World Health Organization concluded that osteoarthritis is the 4th leading cause of years lost due to disability [34]. Previous studies have also observed variations in surgical rates. The U.S. National Arthritis Workgroup estimated that over 300,000 total knee replacement surgeries are performed annually in the U.S., and surgical rates vary by geographical location (3-fold), sex (2-fold), and race/ethnicity (2-fold) [36]-[41].

Second, for many patients the choice between nonsurgical or surgical therapies for this condition is “preference-sensitive”, in that it involves considering their informed preferences about the likelihood of risks and benefits [40],[41]. Mild osteoarthritis may be treated with one or a combination of pain medications, weight loss/exercise, physical therapy, etc. Patients with moderate disease may continue with non-surgical therapies and/or consider joint injections or total knee arthroscopy. For advanced osteoarthritis, multiple options are available for surgery (e.g., partial, total, bilateral knee replacements). Hence, knee osteoarthritis sufferers face repeated decisions about continuing, combining, or switching therapies.

Lastly, paper and video patient decision aids about knee osteoarthritis management have been rigorously developed and evaluated [1],[31],[41]-[44]. At the Dartmouth Center for Shared Decision Making, patients may self-refer or be referred by their clinician to view a videobooklet decision aid titled, “Treatment Choices for Knee Osteoarthritis” (© Health Dialog 2006) [44]. The Center monitors usage rates, as well as patients’ responses to the Osteoarthritis Decision Quality Index’s Knowledge Subscale, and the Preparation for Decision Making and Decisional Conflict scales (Table 1) [45]-[47].

Table 1 Properties of the outcome measures used to assess feasibility of the virtual decision lab
Full size table

Designing the web-based research platform components

The advisory panel outlined the tasks that a series of potential users would be asked to complete, and created an initial set of paper drafts, or “wireframes”, of the following three components of the web-based research platform.

Introduction and baseline data collection

The website opened with a welcome and log-on page inviting eligible patients to re-review the informed consent documentation, and to confirm consent by entering their study password. No personally identifiable information was collected by the study website. Optional voice-over audio was offered for patients who preferred to hear the text read to them.

In the Introduction, a narrator described how this health care decision has multiple treatment options, and discussed how patients can, if they choose, share in the decision-making process with their doctor(s). The narrator described the decision aid as a “resource to help you prepare for a discussion with your doctor about choosing the best treatment for your knee pain”. The narrator also explained that the website would provide an overview of the key information and decision-making steps, and that they could view more detailed information and engage in interactive decision-making activities if they chose. Finally, the narrator stated that the decision aid would anonymously collect responses into a printable Personal Decision Summary that they could discuss with their family and doctor(s).

Baseline questionnaires assessed participants’ self-reported characteristics. Socio-demographic characteristics included age, sex, race, and highest level of education. Clinical characteristics included knee pain, stiffness, and function measured by the Western Ontario McMaster Universities Arthritis Index (WOMAC® 3.1) [48]. Cognitive characteristics included decisional conflict, familiarity with the decision options, and baseline treatment preference measured by the Choice Predisposition Scale [47]-[50].

The patient decision aid and decision process data collection items

Based on the Ottawa Decision Support Framework, the advisory panel structured the patient decision aid in four “deliberative steps” [30].

Step 1: Information comprehension

The first deliberative step helps the patient gain a clear understanding of the clinical information (i.e., the condition, the treatment options, and the likelihood of positive/negative outcomes). To design Step 1 for this web-based decision aid, the advisory panel drew on theories of situated learning, gist/verbatim comprehension, and risk communication [51]-[54]. First, the narrator emphasized the importance of becoming well-informed. Next, the decision aid presented up-to-date clinical information about the natural history of knee osteoarthritis, the nonsurgical options, the surgical options, and the potential risks/benefits. A systematic review had recently been completed in the annual update of the videobooklet decision aid; a team of clinicians re-reviewed and critically appraised the original articles cited in the videobooklet and the systematic review, to confirm that the clinical information was up-to-date.

The decision aid presented the clinical information at an overview level in plain language, with available audio voiceover. Patients who desired additional detail could choose interactive “More Information” links to view definitions of medical terms, anatomical diagrams, and more detailed explanations of procedures, risks, and benefits. The decision aid then provided a side-by-side summary of the treatment options and attributes. Step 1 ended with two optional Personal Decision Activities where patients could: a) self-quiz their knowledge of the key facts, and b) document questions for their doctor.

Step 2: Values clarification

Once well-informed, Step 2 helps the patient consider the positive and negative attributes of each of the therapeutic options, identify which attributes matter most to them personally, and clarify the desirability/undesirability they would ascribe to each option. Design of Step 2 was guided by theories of conjoint measurement, judgment heuristics, and cognitive biases [55]-[57]. The narrator discussed the importance of considering whether some attributes are more important than others. Narrative examples illustrated how different patients may ascribe lesser or greater importance to the different attributes of particular procedures. For example, two narratives described how patients felt differently about the trade-off between spending 1–2 hours every week in physical therapy versus needing 1–2 months off work for surgical recovery.

Finally, Step 2 presented two interactive Personal Decision Activities in which the patient could: a) rate the importance of each option’s attributes on a 0- to 5-star scale (“Not Important” to “Very Important”); and b) indicate an initially-favored option that best matched the attributes they valued most.

Step 3: Considering social resources

Step 3 helps patients address the logistical considerations and social influences bearing on their decision. For this decision aid, integrated information and social cognition theories guided the design [58]-[60]. The narrator described strategies for managing positive and negative pressures to choose a particular option, and for communicating one’s preferences with others. The narrator also emphasized the importance of considering the personal and material resources involved in undergoing one’s preferred treatment, such as the potential need for transportation to physical therapy appointments or assistance while recovering from surgery.

Step 3 presented two interactive Personal Decision Activities in which the patient could: a) list who else might be involved in the decision process and identify what the patient would like their role to be (e.g., support person, surrogate decision-maker, etc.); and b) document specific questions they had for these individuals, as well as other people involved in the decision-making, treatment, or recovery processes.

Step 4: Forming an Action Plan

This final deliberative step helps the patient develop a feasible strategy for moving towards an informed, values-based, actionable choice. Prospect theory and availability heuristics [61] guided the design of strategies for encouraging the patient to integrate Steps 1–3 into forming individualized action plans. The narrator discussed strategies for creating a) short-term action items to address any gaps in information, clarity, or personal support, and b) a long-term plan, such as timelines illustrating treatments, follow-up visits, and recovery stages. Step 4 ended with an optional Personal Decision Activity where patients could interactively create their personal short- and/or long-term action plans.

Post-decision aid data collection

To address the study’s second objective, website tracking and five post-decision aid scales assessed the feasibility of using the newly-developed web-based research platform and the web-based decision aid component in future planned research studies.

Feasibility of the web-based research platform

To assess whether patients would use the study website and all of its components, the research platform tracked a) how many enrolled participants actually accessed the website, b) how many data collections items they responded to, and c) the time to completion. Next, to address questions about whether web-based decision support would be acceptable to patients, the platform presented the Ottawa Acceptability Scale (Table 1) [62]. For this web-based study, the scale was adapted into six multiple-choice items assessing patients’ subjective ratings of: ease of use, clarity of information, length, level of detail provided, ability to hold one’s interest, and satisfaction with “how the website prepared you for discussing this decision with your doctor(s)”.

Feasibility of the web-based decision aid component

Before using the research platform, the advisory panel sought to confirm that the web-based decision aid component would perform comparably with the videobooklet used in clinical practice. Therefore, the study website presented the three post-decision aid scales used at the Dartmouth Center for Shared Decision Making (Table 1). The Osteoarthritis Decision Quality Index, Knowledge Subscale contains 5 multiple-choice items assessing understanding of key facts about the treatment options [45]. The interactive capabilities of the web-based research platform allowed for adaption of the paper version to provide interactive corrective feedback (i.e., if an incorrect answer was selected, the correct answer was presented). The website then presented the 11-item Preparation for Decision Making Scale and the 10-item low-literacy Decisional Conflict Scale [46],[47].

In closing, the website summarized participant’s responses into their printable Personal Decision Summary, and provided links to references and related resources. It also thanked participants for participation in the study and invited them to make any final suggestions for improvement to the patient decision aid or the interactive features in an open text area.

User review and revision cycles

Six iterative cycles of review and revision refined the paper and online prototypes. Groups of five patients walked through paper drafts of each component and were asked to comment on the wording, format, and visual layout. The drafts were revised in accordance with their comments, and iteratively presented to a new set of five patients, then revised again. Once feedback reached saturation, the advisory panel re-reviewed the optimized paper drafts and approved them for programming.

Next, informaticians and programmers of the Dartmouth Digital Decision Aid Developer Program developed the three components into an interactive web-based prototype, using Ruby on Rails (v2.3, 2009) and a password-protected MySQL database. Four focus groups of patients (n = 4 each) iteratively reviewed the prototypes online (alpha testing). Finally, the advisory panel re-appraised the patients’ decision aid component of the research platform, using the IPDAS Collaboration’s criteria, and approved the research platform for initial field-testing in the clinic (beta testing).

Field test of feasibility for research use

Feasibility of the web-based research platform

The stakeholder advisory panel identified four a priori feasibility criteria related to website usage and acceptability. The research platform would be considered feasible if greater than 80% of eligible patients: a) could be enrolled in a project using both in-clinic and online recruitment strategies; b) could access the research website; c) would review the entire decision aid and complete all data collection items; and d) provide positive ratings on each of the six acceptability items.

Feasibility of the web-based decision aid component

Within the research platform, the advisory panel also identified three a priori criteria for considering the web-based decision aid component feasible for use in future research studies. It would be considered feasible if participants provided mean scores on the Osteoarthritis Decision Quality Index’s Knowledge Subscale, the Preparation for Decision Making Scale, and the Decisional Conflict Scale that were comparable to the mean scores observed when the videobooklet decision aid is used in clinical practice. Since this was the first use of the web-based version, blinded randomization was not used. Instead, performance of the web-based version was closely monitored and results were compared to the mean scores for the videobooklet decision aid during its clinical use over the previous two years.

The Dartmouth Committee for the Protection of Human Subjects provided ongoing ethical review and approval of this study. Participation involved a single viewing of the website between orthopaedic consultations and anonymous completion of the data collection items.

Study participants

Study participants were recruited in two ways to ensure that the decision aid was viewed by a diverse sample of patients and to confirm that patients could be successfully enrolled in clinical settings and online. Orthopaedic surgeons at Dartmouth-Hitchcock Medical Center recruited patients who: a) were over 18 years old and able to read, write, and speak English; b) had received a diagnosis of knee osteoarthritis and were eligible for either surgical or non-surgical management; and c) were at the point of making a treatment decision. Eligible individuals were referred to the research assistant, who further explained the purpose, process, risks, and benefits of participation in the study. Verbal informed consent constituted enrollment, because the collection of signatures would involve the collection of identifiable information. Participants were offered a private room, computer, and headphones at the clinic, or a flyer with instructions for viewing the website on a personal or public computer (e.g., at a library, etc.).

In addition, collaboration with Knowledge Networks (acquired by GfK Custom Research) facilitated recruitment of a stratified sample of socio-demographically and geographically diverse participants [63]. Knowledge Networks uses address-based sampling and provides Internet access where needed to maintain a probability-based panel representing 97% of U.S. households. For this study, Knowledge Networks screened KnowledgePanel members for study eligibility criteria through a self-reported questionnaire. Potentially-eligible members were referred to the research assistant and asked to confirm that they were still in the process of considering treatment options for their knee osteoarthritis. Eligible individuals who elected to participate were enrolled, and referred to the study website.

Data scoring and analysis

Baseline participant characteristics

Data analysis using Stata 10 (StataCorp® 2010) began with tabulations of the distributions of baseline characteristics of the study sample and comparison across recruitment methods. The research platform collected and scored responses to the questions about socio-demographic, clinical, and cognitive characteristics. For the Choice Predisposition Scale [49] the website scored the Leaning Scale as 0 “Unsure/No Preference” at the center; 1 to 4 in both directions for intermediate points on the scale; and 5 for the two extremes, “Strongly Prefer Surgical Treatments” at the left end and “Strongly Prefer Nonsurgical Treatments” at the right end. Scores of 4 or 5 were considered indicative of a baseline treatment preference.

Feasibility of the web-based research platform

The research team analyzed the feasibility of the research platform in terms of website usage and acceptability. First, the research platform tabulated the number of enrolled participants who accessed the website, and the number who completed all components and data collection items. The research team also performed content analysis on participants’ open-text suggestions for improvement. Second, for the adapted six-item Acceptability Scale [62], the website scored 0 for negative ratings (e.g., “Not at all Clear” or “Somewhat Unclear”) and 1 for positive ratings (e.g., “Clear” or “Very Clear”). The research team tabulated scores to determine whether more than 80% of respondents gave a positive response on each acceptability item, and to all six acceptability items.

Feasibility of the web-based decision aid component

The website automatically scored the Osteoarthritis Decision Quality Index Knowledge Subscale, the Preparation for Decision Making Scale, and the Decisional Conflict Scale according to their published scoring algorithms, (Table 1), then converted those scores to 0–100 units. For example, the 10 question-items of the baseline Decisional Conflict Scale assess patients’ perceived levels of: uncertainty (2 items), feeling informed (3 items), clarity about the personal value of the risks/benefits/side effects (2 items), and feeling supported in decision making (3 items) [47]. The website scored item-responses as either 0 “yes”, 2 “unsure”, or 4 “no”, then summed these scores, divided by 10, and multiplied by 25 to yield an individual’s score that could range from 0 (no decisional conflict) to 100 (high decisional conflict). For each scale, the analyst used unpaired t tests to compare the web-based decision aid’s mean scores with the videobooklet’s mean scores during clinical use over the previous two years.

Lastly, the analyst conducted sub-analyses to confirm that there were no differences in responses by recruitment method, and to assess whether participants’ baseline characteristics contributed to any observed results. Scatterplots were used to determine the appropriateness of a linear model of patient characteristics on the primary outcomes. Nonlinear distributions were transformed where possible and sensitivity analyses assessed the impact of outliers the primary outcomes. Univariate linear regression models assessed the likelihood that participants’ characteristics contributed to the results at the 5% significance level. Where associations were observed, multiple linear regression models assessed the relative contributions of each covariate.

Results

Development

Six cycles of iterative review and revision of the paper drafts (wireframes) resulted in a six-month development process that produced an optimized web-based patient decision aid for field testing in the clinic. The presentation of treatment options, risks, and benefits yielded the most feedback for revision. Patients recommended restructuring the presentation order, clarifying the wording of the clinical information, expanding the glossary of terms, and embedding additional “More Information” links. For the presentation of the decision support components, user feedback centered on minimizing text and adding additional “other” categories to the Personal Decision Activities. Review of the interactive features resulted in larger buttons/checkboxes and the inclusion of a status bar (e.g., “Step 3 of 4”) on the top of each page.

The advisory panel’s assessment of this optimized web-based patient decision aid prototype against the IPDAS Collaboration’s quality checklist [2] indicated that it met all 25 relevant criteria regarding the quality of the development process, and 21 of 22 criteria regarding content quality. The remaining criteria were not relevant because they referred to decisions regarding screening, the quality of testimonials, individualized risk estimates, or clinical effectiveness.

Field test

Recruitment and baseline participant characteristics

In the clinic, all eligible patients within a four-month period (n = 55) were offered participation in the study; one declined to speak with the research assistant due to fatigue from a prolonged series of clinical appointments. Web-based recruitment of the socio-demographically stratified sample took one week; among 80 eligible individuals, 93% elected to participate. No differences were observed between recruitment sites other than differences in race from the stratified sampling approach (i.e., all 54 patients recruited from the clinic identified themselves as Caucasian, while nationwide web-based recruitment included 19 people from Caucasian backgrounds and 53 people from African American, Hispanic, or “other” backgrounds).

Table 2 summarizes the participants’ characteristics. Overall (n = 126), study participants were primarily female, Caucasian, younger adults with college degrees and moderate knee pain. Most participants (86%) had searched the Internet for information prior to talking with their clinician; however, few (4%) had seen the videobooklet version of the decision aid.

Table 2 Study participants: socio-demographic, cognitive and clinical characteristics (N = 126)
Full size table

Feasibility of the web-based research platform

Table 3 summarizes the website usage and acceptability results. All participants (100%) preferred viewing the decision aid on a home or public computer rather than at the clinic. All participants viewed all components of the decision aid and completed all pre-/post-decisions data collection items. Mean viewing time was 36 minutes (minimum 12, maximum 90). Non-Caucasians spent more time viewing the website than Caucasians (mean = 47 versus 35 minutes, s = 42 versus 23, respectively; F 1,108 = 4.4, p F  = 0.04), and individuals less than 65 years old spent less time than individuals over 65 years old (mean = 34 versus 49 minutes, s = 22 versus 43, respectively; F 1,109 = 5.5, p F  = 0.02). All participants prepared a Personal Decision Summary for discussion with their doctor(s).

Table 3 Feasibility of the web-based patient decision support research platform: recruitment, usage, and acceptability (N = 126)
Full size table

More than 80% of participants provided positive ratings on all six acceptability items. Linear regression analyses revealed no differences across patient characteristics on the acceptability items. Open-text comments included, “I appreciate being able to re-review the information after my doctor’s visit”, “…at my own pace”, and “… in the privacy of my own home”. Multiple participants requested additional interactive features to create personalized risk estimates and treatment cost comparisons.

Feasibility of the web-based decision aid

Table 4 provides participants’ mean post-decision aid scores on the Osteoarthritis Decision Quality Index Knowledge Subscale, the Preparation for Decision Making Scale, and the Decisional Conflict Scale compared to the mean scores observed for patients who viewed the videobooklet decision aid at the Dartmouth Center for Shared Decision Making during the two years prior to the study (N = 243). For each of these three scales, participants who viewed the web-based decision aid reported higher mean scores than patients who viewed the videobooklet decision aid in clinical practice. The difference was not statistically significant for the Preparation for Decision Making Scale, but differences were statistically significant for the Knowledge Subscale and Decisional Conflict Scale. For the web-based version, there was also a statistically significant difference between these participants’ mean baseline and post-decision aid scores on the Decisional Conflict Scale (mean = 31.1 to 19.5, p <0.01).

Table 4 Feasibility of the web-based patient decision aid: knowledge, preparation for decision making, and decisional conflict scores
Full size table

Linear regression analyses identified two main effects on the Decision Quality Index Knowledge Subscale’s scores and two main effects on the Preparation for Decision Making Scale’s scores. On the Knowledge Subscale, Caucasians scored higher than non-Caucasians (mean = 69.0 versus 58.0, s = 17.0 versus 16.6, respectively; F 1,118 = 11.69, p F  < 0.001), and individuals who finished college scored higher than those without a degree (mean = 68 versus 60, s = 19.6 v 14.1, respectively; F 1,118 = 6.89, p F  = 0.009). On the Preparation for Decision Making Scale, females reported higher scores than males (mean = 73.0 versus 61.3, s = 22.3 versus 24.3, respectively; F 1,123 = 7.7, p F  = 0.006), and individuals less than 65 years old reported higher scores than those over 65 years old (mean = 72.2 versus 62.0, s = 21.8 versus 26.0, respectively; F 1,123 = 5.7, p F  = 0.02). No significant interactions were observed among participant characteristics and decisional conflict scores.

Discussion

This study demonstrates an initial interdisciplinary, patient-centered approach to developing an interactive web-based patient decision support research platform that appears feasible for studying the design and delivery of patient decision aids. During the development process, iterative patient-centered design cycles provided notable improvements to the platform’s content, language, format, and layout. Participants were successfully recruited in clinic and online, and were able to access the website, use the patient decision aid, and complete the data collection items. Positive results for usage and acceptability were observed.

Within the research platform, the interactive web-based version of the patient decision aid performed comparably to the videobooklet decision aid used in routine clinical care. On the Osteoarthritis Decision Quality Index Knowledge Subscale, participants who viewed the web-based version reported statistically significant higher mean scores, indicating improved knowledge. There was no significant difference in mean scores on the Preparation for Decision Making Scale, and these scores were similar to other published studies (mean scores between 66 and 78 across several clinical contexts) [1],[47],[64]-[67]. Mean Decisional Conflict scores were significantly higher for the web-based version than for the videobooklet; however, the post-decision aid mean for the web-based version (19.5) was below the threshold (25.0) associated with patients who proceed to making decisions [42] and reflected a significant improvement from baseline (31.0).

These results indicate that a web-based research platform supporting an interactive patient decision aid holds promise as a virtual research laboratory. However, the following limitations should be considered. First, the study sample may have contributed to a potential Type II error in terms of their high baseline familiarity. The selection of a chronic condition facilitated recruitment and minimized some confounding factors (e.g., anxiety from time-limited or life-threatening decisions), but may have constrained the ability to detect important subgroups who differ in their information comprehension, preparation for decision making, and decisional conflict. Different results may be observed with web-based research platforms and patient decision aids that focus on the first decision in a chronic condition, or on clinical situations that are acute, life threatening, or involve surrogate decision making. Furthermore, screening for low self-efficacy or minimal familiarity with the decision may facilitate tailoring the level of decision support for patients who need more/less-detailed clinical information and deliberative support.

Second, the advisory panel chose not to include video testimonials in this early version. Educational theory supports the use of videos to increase saliency, but current literature reviews regarding patients' decision support lack robust evidence that videos improve patients’ information comprehension or preparation for decision making [67]-[72]. Possible social matching/mismatching (by race, age, or sex) between the viewer and the individuals presented in videos could bias participants’ attitudes and treatment preferences. Notably, the web-based decision aid without testimonials performed comparably with the videobooklet version that contains testimonials. Now that this web-based decision aid has been field-tested, future studies may use this research platform to explore the purpose, structure, and optimal use of video testimonials, and other potential features of web-based patient decision aids. An interactive research program would allow such studies to proceed with large sample sizes, real-time data collection, and robust analyses [2].

Third, the IPDAS Collaborative’s report on Delivering Decision Aids Using the Internet distinguishes among patient decision aids that are a) posted in their original form; b) developed and evaluated using paper/video, then adapted for the Internet; and c) developed and evaluated as used on the Internet [4]. The report highlighted the challenges of combining clinical translational research and informatics approaches (e.g., how best to merge the efficacy-effectiveness-implementation pathway with the user-centered design approach), and noted that there are few published studies using such interdisciplinary approaches in a patient-centered manner. While this study designed and evaluated the decision aid as it would be used on the Internet, it falls short of a true user-centered design process, most notably in that it could not engage all users in identifying the initial purpose, goals, and tasks for the decision aid. Studies are needed to identify and evaluate best practices in engaging patients in the full user-centered design process.

Furthermore, the advisory panel elected to assess the feasibility of the research platform before engaging in more in-depth informatics and clinical studies. The intention was to ensure that participants could be successfully recruited to the research platform, use the platform, and receive comparable decision support prior to launching large randomized trials. Results indicate that this web-based research platform met the selected criteria for feasibility for research use, and subsequent studies are planned to test usability of the research platform and clinical effectiveness of the web-based patient decision aid. If the research platform is shown to be usable and effective, the virtual decision lab may then be made available to research teams to study best practices in design, optimization, dissemination, and continued improvement of web-based patient decision aids.

Fourth, the dissemination method used in this study was kept constant by providing access to a computer and headphones at the clinic. Not all clinics will have these resources; however, it is interesting to note that all participants indicated that they had access to and preferred to view the decision aid on a personal or public computer rather than at their clinicians’ office, even when a private room and computer was offered. Future research could compare dissemination methods and explore patients’ preferences for web-based decision support in and outside the clinical setting.

Conclusions

This project illustrates one strategy for integrating decision science and informatics methods into a rapid-cycle approach for designing web-based patient decision support research platforms. An interdisciplinary stakeholder advisory panel combined with early and repeated reviews from patients meaningfully contributed to the content, format, and features of the initial prototypes. Both clinical and online recruitment methods were successful; participants completed 100% of the data collection items, and provided positive acceptability ratings. Within the research platform, the web-based decision aid performed comparably with the existing videobooklet version as used in clinical practice. Several limitations were noted that should be considered in planning future studies; however, the overall results support the feasibility of developing and using a “virtual decision lab” for patient decision support research.

Implications

A virtual decision lab containing interactive patient decision aid(s) could be used to address a wide range of exploratory and experimental investigations [2]. A web-based research platform could be used, in real time, to address exploratory questions such as whether and how patients prefer a) to receive personalized clinical information, b) to receive lesser- or greater-detailed clinical information, or c) to proceed through the deliberative steps with lesser or greater interactive engagement at each step. Studies across clinical contexts and across time could assess whether these deliberative styles are state-like or trait-like, and whether patients’ decision support needs change as they become more skilled in the shared decision making process.

The virtual decision lab may also be used to test theory-based and interdisciplinary approaches to designing web-based patient decision aids. For example, does a user-centered approach to designing web-based tools lead to improved implementation and usage in clinical practice? Such a web-based decision lab could also be used in experimental studies testing whether important decision-making outcomes—such as information comprehension, preparation for engaging in shared decision making with one’s clinician(s)/family, or decision quality—are affected by a) the match/mismatch between patients’ deliberative styles and the design features of web-based decision support intervention, or b) the extent to which web-based decision support interventions are tailored to patients’ clinical profiles or stage of decision making.

The launch of a virtual decision research laboratory provides an opportunity to address fundamental and applied questions about how patients prefer to engage in shared decision making, and to rapidly test strategies for providing decision support interventions that meet their needs and preferences. Ultimately, these descriptive and experimental lines of investigation could contribute to the success of clinical efforts to provide targeted decision support interventions that are designed to improve patient-centered care [1],[2],[39],[40].

Abbreviations

IPDAS:

International Patient Decision Aids Standards Collaboration

MySQL:

My structured query language

WOMAC:

Western Ontario McMaster Universities Arthritis Index

References

  1. Stacey D, Legare F, Col NF, Bennett CL, Barry MJ, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R, Trevana L, Wu JHC: Decision aids for people facing health treatment or screening decisions.Cochrane Database Syst Rev 2014, 1:. http://www.cochrane.org/cochrane-reviews/citing-our-products.,

  2. Volk R, Llewellyn-Thomas H, Stacey D, Elwyn G: The international patient decision aids standards (IPDAS) collaboration’s quality dimensions: theoretical rationales, current evidence, and emerging issues.BMC Med Inform and Dec Making 2013, 13(Suppl 2). http://www.biomedcentral.com/bmcmedinformdecismak/supplements/13/S2.,

  3. Llewellyn-Thomas H: The Decision Research ‘lab’. Shared Decision-Making in Health Care. Edited by: Edwards A, Elwyn G. 2009, Oxford University Press, Oxford, 243-247.

    Google Scholar 

  4. Hoffman A, Volk R, Härter M, Li L, Saarimaki A, Stirling C: Delivering decision aids on the internet. BMC Med Inform and Dec Making. 2013, 13 (Suppl 2): S13-10.1186/1472-6947-13-S2-S13.

    Article  Google Scholar 

  5. Drake RE, Cimpean D, Torrey WC: Shared decision making in mental heatlh: prospects for personalized medicine. Dialogues Clin Neurosci. 2009, 11 (4): 455-463.

    PubMed  PubMed Central  Google Scholar 

  6. Molenaar S, Sprangers M, Oort F, Rutgers E, Luiten E, Mulder J, van Meeteren M, de Haes H: Exploring the black box of a decision aid: What information do patients select from an interactive CD-Rom on treatment options in breast cancer?. Pat Edu Counsel. 2007, 65 (1): 122-130. 10.1016/j.pec.2006.06.022.

    Article  Google Scholar 

  7. Tam KY, Ho SK: Understanding the impact of web personalization on user information processing and decision outcomes. Mis Quart. 2006, 30 (4): 865-890.

    Google Scholar 

  8. Keisler DJ, Auerbach SM: Optimal matches of patient preferences for information, decision-making and interpersonal behavior: evidence, models, and interventions. Patient Educ Couns. 2006, 61: 319-341. 10.1016/j.pec.2005.08.002.

    Article  Google Scholar 

  9. McCaffery K, Sheridan S, Nutbeam D, Clayman M, Kelly-Blake K, Rovner M, Rovner D, Smith S, Wolf M: Addressing health literacy in patient decision aids. BMC Med Inform and Dec Making. 2013, 13 (Suppl 2): S10-10.1186/1472-6947-13-S2-S10.

    Article  Google Scholar 

  10. Volk RJ, Jibaba-Weiss ML, Hawley ST, Kneuper S, Spann SJ, Miles BJ, Hyman DJ: Entertainment education for prostate cancer screening: a randomized trial among primary care patients with low health literacy. Patient Educ Couns. 2008, 73 (3): 482-489. 10.1016/j.pec.2008.07.033.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Witteman HO, Fuhrel-Forbis A, Wijeysundera HC, Exe N, Dickson M, Holtzman L, Kahn VC, Zikmund-Fisher BJ: Animated randomness, avatars, movement, and personalization in risk graphics.

  12. Miller SM: Monitoring and blunting: validation of a questionnaire to assess styles of information seeking under threat. J Pers Soc Psychol. 1987, 52: 34-53.

    Google Scholar 

  13. Llewellyn-Thomas HA: Patients’ health-care decision making: a framework for descriptive and experimental investigations. Med Decis Making. 1995, 15: 101-106. 10.1177/0272989X9501500201.

    Article  CAS  PubMed  Google Scholar 

  14. Llewellyn-Thomas HA: Decision-Making Needs of Older People: The Role of patients’ Decision Aids. 2005, The National Academies, National Academy of Sciences, Washington, D.C.

    Google Scholar 

  15. Miller DP, Spangler JG, Case LD, Goof DC, Singh S, Pignone MP: Effectiveness of a web-based colorectal cancer screening patient decision aid: a randomized controlled trial in a mixed-literacy population. Amer J Prevent Med. 2011, 40 (6): 608-615. 10.1016/j.amepre.2011.02.019.

    Article  Google Scholar 

  16. Thomson MD, Hoffman-Goetz L: Readability and cultural sensitivity of web-based patient decision aids for cancer screening and treatment: a systematic review. Med Inform Internet Med. 2007, 32: 263-286. 10.1080/14639230701780408.

    Article  CAS  PubMed  Google Scholar 

  17. Edwards A, Thomas R, Williams R, Ellner AL, Brown P, Elwyn G: Presenting risk information to people with diabetes: evaluating effects and preference for different formats by a web-based randomized controlled trial. Patient Educ Counsel. 2006, 63 (3): 336-349. 10.1016/j.pec.2005.12.016.

    Article  Google Scholar 

  18. Fagerlin A, Zikmund-Fisher BJ, Nair V, Derry HA, McClure JB, Greene S, Stark A, Hensley Alford S, Lantz P, Hayes DF, Wiese C, Claud Zweig S, Pitsch R, Jankovic A, Ubel PA: Women's decisions regarding tamoxifen for breast cancer prevention: responses to a tailored decision aid. Breast Cancer Res Treat. 2010, 119 (3): 613-620. 10.1007/s10549-009-0618-4.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Banegas MP, McClure JB, Barlow WE, Ubel PA, Smith DM, Zikmund-Fisher BJ, Greene SM, Fagerlin A: Results from a randomized trial of a web-based, tailored decision aid for women at high risk for breast cancer. Patient Educ Couns. 2013, 91 (3): 364-371. 10.1016/j.pec.2012.12.014.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Li LC, Adam PM, Backman CL, Lineker S, Jones CA, Lacaille D, Townsend AF, Yacyshyn E, Yousefi C, Tugwell P, Leese J, Stacey D: A proof-of-concept study of ANSWER, a web-based methotrexate decision aid for patients with rheumatoid arthritis.Arthritis Care Res 2014, [Epub ahead of print].,

  21. Ozanne EM, Howe R, Omer Z, Esserman LJ: Development of a personalized decision aid for breast cancer risk reduction and management. BMC Med Inform Decis Mak. 2014, 14: 4-10.1186/1472-6947-14-4.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Simon D, Kriston L, von Wolff A, Buchholz A, Vietor C, Hecke T, Loh A, Zenker M, Weiss M, Härter M: Effectiveness of a web-based, individually tailored decision aid for depression or acute low back pain: a randomized controlled trial. Patient Educ Couns. 2012, 87 (3): 360-368. 10.1016/j.pec.2011.10.009.

    Article  PubMed  Google Scholar 

  23. Durand MA, Wegwarth O, Boivin J, Elwyn G: Design and usability of heuristic-based deliberation tools for women facing amniocentesis. Health Expect. 2012, 15 (1): 32-48. 10.1111/j.1369-7625.2010.00651.x.

    Article  PubMed  Google Scholar 

  24. DuBenske LL, Gustafson DH, Shaw BR, Cleary JF: Web-based cancer communication and decision making systems: connecting patients, caregivers, and clinicians for improved health outcomes. Med Decis Making. 2010, 30 (6): 732-744. 10.1177/0272989X10386382.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Frosch DL, Kaplan RM, Felitti VJ: A randomized controlled trial comparing internet and video to facilitate patient education for men considering the prostate specific antigen test. J Gen Intern Med. 2003, 18: 781-787. 10.1046/j.1525-1497.2003.20911.x.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Frosch DL, Bhatnagar V, Tally S, Hamori CJ, Kaplan RM: Internet patient decision support: a randomized controlled trial comparing alternative approaches for men considering prostate cancer screening. Arch Intern Med. 2008, 168: 363-369. 10.1001/archinternmed.2007.111.

    Article  PubMed  Google Scholar 

  27. Sivell S, Edwards A, Manstead AS, Reed MW, Caldon L, Collins K, Clements A, Elwyn G, BresDex Group: Increasing readiness to decide and strengthening behavioral intentions: evaluating the impact of a web-based patient decision aid for breast cancer treatment options (BresDex:). Patient Educ Couns. 2012, 88 (2): 209-217. 10.1016/j.pec.2012.03.012.

    Article  PubMed  Google Scholar 

  28. Elwyn G, Kreuwel I, Durand MA, Sivell S, Joseph-Williams N, Evans R, Edwards A: How to develop web-based decision support interventions for patients: a process map. Patient Educ Couns. 2011, 82 (2): 260-265. 10.1016/j.pec.2010.04.034.

    Article  PubMed  Google Scholar 

  29. Evans R, Elwyn G, Edwards A, Watson E, Austoker J, Grol R: Toward a model for field-testing patient decision-support technologies: a qualitative field-testing study. J Med Internet Res. 2007, 9 (3): e21-10.2196/jmir.9.3.e21.

    Article  PubMed  PubMed Central  Google Scholar 

  30. O’Connor AM: Ottawa Decision Support Framework to address decisional conflict [http://decisionaid.ohri.ca/odsf.html]

  31. Jacobsen MJ, O’Connor AM: Population needs assessment: a workbook for assessing patients’ and practitioners’ decision making needs. [www.ohri.ca/decisionaid]

  32. Jacko JA, Sears A: Human-Computer Interaction Handbook. 2003, Lawrence Erlbaum & Associates, Mahwah

    Google Scholar 

  33. Altman R, Asch E, Bloch D, Bole G, Borenstein D, Brandt K, Christy W, Cooke TD, Greenwald R, Hochberg M, Howell D, Kaplan D, Koopman W, Longley A, Mankin H, McShane DJ, Medsger T, Meenan R, Mikkelsen W, Moskowitz R, Murphy W, Rothschild B, Segal M, Sokoloff L, Wolfe F: Development of criteria for the classification and reporting of osteoarthritis: classification of osteoarthritis of the knee. Diagnostic and therapeutic criteria committee of the American rheumatism association. Arthritis Rheum. 1986, 29: 1039-1049. 10.1002/art.1780290816.

    Article  CAS  PubMed  Google Scholar 

  34. World Health Organization: The Global Burden of Disease: 2004 Update. 2008, World Health Organization, Geneva

    Google Scholar 

  35. Lawrence RC, Felson DT, Helmick CG, Arnold LM, Choi H, Deyo RA, Gabriel S, Hirsch R, Hochberg MC, Hunder GG, Jordan JM, Katz JN, Kremers HM, Wolfe F, National Arthritis Data Workgroup: National arthritis data workgroup estimates of the prevalence of arthritis and other rheumatic conditions in the United States: part II. Arthritis Rheum. 2008, 58: 26-35. 10.1002/art.23176.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Kurtz S, Mowat F, Ong K, Chan N, Lau E, Halpern M: Prevalence of primary and revision total hip and knee arthroplasty in the United States from 1990 through 2002. J Bone Joint Surg Am. 2005, 87: 1487-1497. 10.2106/JBJS.D.02441.

    Article  PubMed  Google Scholar 

  37. Hawker GA, Wright JG, Coyote PC, Williams JI, Harvey B, Glazier R, Badley EM: Differences between men and women in the rate of use of hip and knee arthroplasty. N Engl J Med. 2000, 342: 1016-1022. 10.1056/NEJM200004063421405.

    Article  CAS  PubMed  Google Scholar 

  38. Weinstein JN, Bronner KK, Morgan TS, Wennberg JE: Trends and geographic variations in major surgery for degenerative diseases of the hip, knee, and spine.Health Aff 2004, (Suppl):VAR81–89. http://www.ncbi.nlm.nih.gov/pubmed/15471768.,

  39. O’Connor AM, Graham ID, Visser A: Implementing shared decision making in diverse health care systems: the role of patients’ decision aids. Patient Educ Couns. 2005, 57: 247-249. 10.1016/j.pec.2005.04.010.

    Article  PubMed  Google Scholar 

  40. O’Connor AM, Wennberg JE, Legare F, Llewellyn-Thomas HA, Moulton BW, Sepucha KR, Sodano AG, King JS: Toward the ‘tipping point’: decision aids and informed patient choice. Health Aff. 2007, 26: 716-725. 10.1377/hlthaff.26.3.716.

    Article  Google Scholar 

  41. Dartmouth Atlas Group: A Dartmouth Atlas Project Brief: Preference-Sensitive Care. 2007, The Dartmouth Atlas of Healthcare, Hanover, New Hampshire

    Google Scholar 

  42. Rivero-Santana A, Perestelo-Perez L, Perez-Ramos J, Gonzalez-Lorenzo M, Serrano-Aguilar P: Treatment Preferences in Patients With Knee or hip Osteoarthritis: An Overview. Osteoarthritis – Diagnosis, Treatment and Surgery. Edited by: Qian C. 2012, InTech, Shanghai: China

    Google Scholar 

  43. De Achaval S, Fraenkel L, Volk RJ, Cox V, Suarez-Almazor ME: Impact of educational and patient decision aids on decisional conflict associated with total knee arthroplasty. Arthritis Care Res. 2012, 64: 229-237. 10.1002/acr.20646.

    Article  Google Scholar 

  44. Treatment choices for knee osteoarthritis: decision aid summary. 2010. URL: http://decisionaid.ohri.ca/AZsumm.php?ID?1087 .,

  45. Sepucha KR, Stacey D, Clay CF, Chang Y, Cosenza C, Dervin G, Dorrwachter J, Feibelmann S, Katz JN, Kearing SA, Malchau H, Taljaard M, Tomek I, Tugwell P, Levin CA: Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC Musculoskelet Disord. 2011, 5 (12): 149-10.1186/1471-2474-12-149.

    Article  Google Scholar 

  46. Bennett C, Graham ID, Kristjansson E, Kearing SA, Clay KF, O'Connor AM: Validation of a preparation for decision making scale. Patient Educ Couns. 2010, 78: 130-133. 10.1016/j.pec.2009.05.012.

    Article  PubMed  Google Scholar 

  47. O’Connor AM: Validation of a decisional conflict scale. Med Decis Making. 1995, 15: 25-30. 10.1177/0272989X9501500105.

    Article  PubMed  Google Scholar 

  48. Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW: Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol. 1988, 15: 1833-1840.

    CAS  PubMed  Google Scholar 

  49. Connor AM: User Manual – Measures of Decision/Choice Predisposition. 1996, Ottawa Hospital Research Institute, Ottawa

    Google Scholar 

  50. Crump T, Llewellyn-Thomas HA: Assessing Medicare beneficiaries' strength-of-preference scores for health care options: how engaging does the elicitation technique need to be?. Health Expect. 2011, 14 (Suppl 1): 33-45. 10.1111/j.1369-7625.2010.00632.x.

    Article  PubMed  Google Scholar 

  51. Reyna VF, Rivers SE: Current theories of risk and rational decision making. Dev Rev. 2008, 28: 1-11. 10.1016/j.dr.2008.01.002.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Lave J, Wenger E: Situated Learning: Legitimate Peripheral Participation. 1991, University of Cambridge Press, Cambridge

    Book  Google Scholar 

  53. Slovic P: The Perception of Risk. 2000, Earthscan, Sterling

    Google Scholar 

  54. Wildavsky A, Dake K: Theories of risk perception: who fears what and why?. Amer Acad of Arts and Sci. 1990, 119: 41-60.

    Google Scholar 

  55. Tversky A, Kahneman D: Judgment under uncertainty: heuristics and biases. Science. 1974, 185: 1124-1131. 10.1126/science.185.4157.1124.

    Article  CAS  PubMed  Google Scholar 

  56. Zermelo E: Beweis, dass jede Menge wohlgeordnet werden kann ["Proof that every set can be well-ordered"]. Math Ann. 1904, 59: 514-516. 10.1007/BF01445300.

    Article  Google Scholar 

  57. Luce RD, Tukey JW: Simultaneous conjoint measurement: a new scale type of fundamental measurement. J Math Psych. 1964, 1: 1-27. 10.1016/0022-2496(64)90015-X.

    Article  Google Scholar 

  58. Tononi G: Consciousness as integrated information: a provisional manifesto. Biol Bull. 2008, 215: 216-242. 10.2307/25470707.

    Article  PubMed  Google Scholar 

  59. Bandura A: Human agency in social cognitive theory. Am Psychol. 1989, 44: 1175-84. 10.1037/0003-066X.44.9.1175.

    Article  CAS  PubMed  Google Scholar 

  60. Miller NE, Dollard J: Social learning and imitation. 1941, Yale University Press, New Haven

    Google Scholar 

  61. Kahneman D, Tversky A: Prospect theory: an analysis of decision under risk. Econometrica. 1979, 42 (2): 263-291. 10.2307/1914185.

    Article  Google Scholar 

  62. O’Connor AM, Cranney A: Users Manual - Acceptability Scale. 1996

    Google Scholar 

  63. Knowledge Networks, Knowledge Panel [http://www.knowledgenetworks.com/ganp/]

  64. Graham ID, O’Connor AM: User Manual – Preparation for Decision Making Scale [document on the Internet]. 1995, Ottawa Hospital Research Institute, Ottawa

    Google Scholar 

  65. Bunn H, O'Connor A: Validation of client decision-making instruments in the context of psychiatry. Can J Nurs Res. 1996, 28: 13-27.

    CAS  PubMed  Google Scholar 

  66. Simon D, Loh A, Härter M: Measuring (shared) decision-making–a review of psychometric instruments. Z Arztl Fortbild Qualitatssich. 2007, 101 (4): 259-267.

    PubMed  Google Scholar 

  67. Bekker HL, Winterbottom A, Butow P, Dillard A, Feldman-Stewart D, Fowler J, Jibaja-Weiss M, Shaffer V, Volk R: Using Personal Stories. In 2012 update of the International Patient Decision Aids Standards (IPDAS) collaboration’s background document. Edited by Volk R, Llewellyn-Thomas H. [http://ipdas.ohri.ca/resources.html]

  68. Shaffer VA, Owens J, Zikmund-Fisher BJ: The effect of patient narratives on information search in a web-based breast cancer decision aid: an eye-tracking study. J Med Internet Res. 2013, 15 (12): e273-10.2196/jmir.2784.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Hoffman S, Caro FG, Gottlieb AS, Kesternich I, Winter JK: Contributions of second opinions, outcome forecasts, and testimonials to patient decisions about knee replacement. Med Dec Making. 2014, 34 (5): 603-614. 10.1177/0272989X14527796.

    Article  Google Scholar 

  70. Ubel PA, Smith DM, Zikmund-Fisher BJ, Derry HA, McClure J, Stark A, Wiese C, Greene S, Jankovic A, Fagerlin A: Testing whether decision aids introduce cognitive biases: results of a randomized trial. Patient Educ Couns. 2010, 80 (2): 158-163. 10.1016/j.pec.2009.10.021.

    Article  PubMed  Google Scholar 

  71. Zikmund-Fisher BJ, Windschitl PD, Exe N, Ubel PA: 'I'll do what they did": social norm information and cancer treatment decisions. Patient Educ Couns. 2011, 85 (2): 225-229. 10.1016/j.pec.2011.01.031.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Peters E, Hart PS, Fraenkel L: Informing patients: the influence of numeracy, framing, and format of side effect information on risk perceptions. Med Decis Making. 2011, 31: 432-436. 10.1177/0272989X10391672.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Health Dialog and the Informed Medical Decision Making Foundation for providing the original source articles used to develop the “Treatment Choices for Osteoarthritis of the Knee” videobooklet. We would like to thank Knowledge Networks/GfK Custom Research for partnering with us to gain access to a diverse sample of patients. We would like to thank the study participants, who provided substantive feedback that improved the feasibility of the research platform and web-based decision aid. Lastly, we would like to thank the clinical team of the Dartmouth-Hitchcock Medical Center Department of Orthopaedics for their commitment to the continued improvement of patient-centered care.

Financial support for this study was provided primarily by a grant from the Robert Wood Johnson Foundation: Advancing Measurement of Equity and Patient-Centered Care to Improve Health Care Quality (RWJF63833). This project was also supported in part by grants from the Agency for Healthcare Research and Quality, and the National Institute of Arthritis and Musculoskeletal and Skin Diseases (P60AR062799 and P60AR048094). The funding agreements ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Author information

Authors and Affiliations

  1. Dartmouth Centers for Health and Aging, Department of Community and Family Medicine, The Geisel School of Medicine at Dartmouth, 46 Centerra Parkway (HB7250), Lebanon, 03766, NH, USA

    Aubri S Hoffman & Stephen J Bartels

  2. Department of Community and Family Medicine, The Geisel School of Medicine at Dartmouth, One Medical Center Drive (HB7250), Hanover, 03755, NH, USA

    Hilary A Llewellyn-Thomas & Anna N A Tosteson

  3. Department of Medicine, The Dartmouth Institute for Health Policy and Clinical Practice, The Geisel School of Medicine at Dartmouth, One Medical Center Drive (HB7505), Lebanon, 03755, NH, USA

    Anna N A Tosteson

  4. Department of Epidemiology, University of Ottawa, Ottawa, K1H 8M5, ON, Canada

    Annette M O’Connor

  5. Department of General Internal Medicine, Unit 1465, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, 77230, TX, USA

    Robert J Volk

  6. Department of Orthopaedics, Dartmouth-Hitchcock Medical Center, Lebanon, 03766, NH, USA

    Ivan M Tomek

  7. Collaboratory for Healthcare and Bioinformatics, The Geisel School of Medicine at Dartmouth, 46 Centerra Parkway, Suite 330, Lebanon, 03766, NH, USA

    Steven B Andrews

Authors
  1. Aubri S Hoffman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hilary A Llewellyn-Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna N A Tosteson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Annette M O’Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert J Volk
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ivan M Tomek
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Steven B Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stephen J Bartels
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aubri S Hoffman.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

AH and HT conceived of and designed the study, and drafted the primary manuscript. AH completed the data collection and statistical analyses. AT assisted in the study design and data analytic plan. SA assisted in the design of the study and programmed the web-based patient decision aid. All authors contributed to the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoffman, A.S., Llewellyn-Thomas, H.A., Tosteson, A.N.A. et al. Launching a virtual decision lab: development and field-testing of a web-based patient decision support research platform. BMC Med Inform Decis Mak 14, 112 (2014). https://doi.org/10.1186/s12911-014-0112-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-014-0112-8

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us