Measuring decision quality: psychometric evaluation of a new instrument for breast cancer chemotherapy
1 Department of Surgery, Lineberger Comprehensive Cancer Center, Sheps Center for Health Services Research, University of North Carolina, UNC Plastic Surgery CB 7195, 27599-7195 Chapel Hill, NC, USA
2 Department of Emergency Medicine, Stanford University, Palo Alto, USA
3 General Medicine Division, Massachusetts General Hospital, Harvard Medical School, Boston, USA
4 Institute for Health Policy Studies, University of California, San Francisco, CA, USA
5 Massachusetts General Hospital Cancer Center, Harvard Medical School, Boston, USA
6 Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, USA
7 Health Decision Sciences Center, Massachusetts General Hospital, Harvard Medical School, Boston, USA
BMC Medical Informatics and Decision Making 2014, 14:73 doi:10.1186/1472-6947-14-73Published: 20 August 2014
Women diagnosed with early stage (I or II) breast cancer face a highly challenging decision – whether or not to undergo adjuvant chemotherapy. We developed a decision quality instrument for chemotherapy for early stage breast cancer and sought to evaluate its performance.
Cross-sectional, mailed survey of recent breast cancer survivors, providers, and healthy controls and a retest survey of survivors. The decision quality instrument includes questions on knowledge and personal goals. It results in a knowledge score and concordance score, which reflects the percentage of patients who received treatments that match their goals. Hypotheses related to acceptability, feasibility, validity, and reliability of the survey instrument were examined.
Responses were received from 352 patients, 89 providers and 35 healthy controls. The decision quality instrument was feasible to implement with few missing data. The knowledge scores had good retest reliability (intraclass correlation coefficient (ICC) =0.75). Knowledge scores discriminated between providers and patients (mean difference 31.1%, 95% CI 26.9, 35.3) and between patients and healthy controls (mean difference 11.2, 95% CI 5.4, 17.1). Most providers reported that the knowledge items covered essential content. Two of the five goal items had a ceiling effect, and one goal had low content validity. The goal items had moderate retest reliability (ICC’s 0.57 to 0.78). In the multivariable model of treatment, none of the patient goals was associated with receipt of chemotherapy. Age and hormone receptor status were the only variables independently associated with chemotherapy. Most patients (77.6%) had treatment concordant with that predicted by the model. Patients who had concordant treatment had similar levels of confidence and regret as those who did not.
The Decision Quality Instrument is a reliable and valid measure of patient knowledge about chemotherapy, but its ability to measure concordance with patient goals is limited. In this sample, patient goals were not associated with treatment, and most patients reported they were not asked their preference, suggesting that goals were not adequately considered in decision making.