Email updates

Keep up to date with the latest news and content from BMC Medical Education and BioMed Central.

Open Access Research article

Constructing a question bank based on script concordance approach as a novel assessment methodology in surgical education

Salah A Aldekhayel135, Nahar A ALselaim3, Mohi Eldin Magzoub3, Mohammad M AL-Qattan2, Abdullah M Al-Namlah1, Hani Tamim3, Abdullah Al-Khayal3, Sultan I Al-Habdan3 and Mohammed F Zamakhshary3456*

Author Affiliations

1 Plastic Surgery Division, King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Riyadh, Saudi Arabia

2 Plastic Surgery Division, King Saud University, Riyadh, Saudi Arabia

3 Department of Medical Education, King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Riyadh, Saudi Arabia

4 Pediatric Surgery Division, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia

5 King Abdullah International Medical Research Center, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia

6 Assistant Deputy, Minister of Health for Planning and Training, Saudi Arabia

For all author emails, please log on.

BMC Medical Education 2012, 12:100  doi:10.1186/1472-6920-12-100

Published: 24 October 2012

Abstract

Background

Script Concordance Test (SCT) is a new assessment tool that reliably assesses clinical reasoning skills. Previous descriptions of developing SCT-question banks were merely subjective. This study addresses two gaps in the literature: 1) conducting the first phase of a multistep validation process of SCT in Plastic Surgery, and 2) providing an objective methodology to construct a question bank based on SCT.

Methods

After developing a test blueprint, 52 test items were written. Five validation questions were developed and a validation survey was established online. Seven reviewers were asked to answer this survey. They were recruited from two countries, Saudi Arabia and Canada, to improve the test’s external validity. Their ratings were transformed into percentages. Analysis was performed to compare reviewers’ ratings by looking at correlations, ranges, means, medians, and overall scores.

Results

Scores of reviewers’ ratings were between 76% and 95% (mean 86% ± 5). We found poor correlations between reviewers (Pearson’s: +0.38 to −0.22). Ratings of individual validation questions ranged between 0 and 4 (on a scale 1–5). Means and medians of these ranges were computed for each test item (mean: 0.8 to 2.4; median: 1 to 3). A subset of test items comprising 27 items was generated based on a set of inclusion and exclusion criteria.

Conclusion

This study proposes an objective methodology for validation of SCT-question bank. Analysis of validation survey is done from all angles, i.e., reviewers, validation questions, and test items. Finally, a subset of test items is generated based on a set of criteria.

Keywords:
Plastic surgery; Script concordance approach; Question bank; Surgical education