Email updates

Keep up to date with the latest news and content from BMC Health Services Research and BioMed Central.

Open Access Research article

Effect of response format for clinical vignettes on reporting quality of physician practice

Thao Pham1*, Carine Roy2, Xavier Mariette3, Fréderic Lioté4, Pierre Durieux5 and Philippe Ravaud2

Author Affiliations

1 Department of Rheumatology, CHU Conception, 13005 Marseille, France

2 Department of Epidemiology, biostatistics and clinical research, CHU Bichat, 13018 Paris, France

3 Department of Rheumatology, CHU Bicêtre, Le Kremlin-Bicêtre, France

4 Department of Rheumatology, CHU Lariboisière, 75 Paris, France

5 Department of public health and medical informatics, Hôpital Européen Georges Pompidou and Paris Descartes University, Paris, France

For all author emails, please log on.

BMC Health Services Research 2009, 9:128  doi:10.1186/1472-6963-9-128

Published: 28 July 2009

Abstract

Background

Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats – open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) – in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.

Methods

Study design: Prospective randomized study. Setting: Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment. Measurements: Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists. Statistical analysis: Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.

Results

Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).

Conclusion

In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.