BMC Medical Education - Latest Articles

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis

Marie Tarrant — 2009-07-07T00:00:00Z

Background: Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong.MethodSUsing item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. Results: The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD=0.88) functioning distractors. Only 52.2% (n=805) of all distractors were functioning effectively and 10.2% (n=158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. Conclusions: The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.

Neurophobia and its implications: evidence from a Caribbean medical school

Farid Youssef — 2009-07-01T00:00:00Z

Background: Neurology is regarded as a difficult component of the medical curriculum. This has been so marked that the term neurophobia and its effects are being investigated. Given the impact of neurological disorders worldwide, neurophobia has the potential to affect the diagnosis and management of such cases. Methods: A cross-sectional survey was done among clinical fourth and fifth year students at the Faculty of Medical Sciences, University of the West Indies, St. Augustine, Trinidad & Tobago. A survey tool successfully used in other schools was adapted to assess perceived level of difficulty, knowledge and interest in various medical sub-specialties including, neurology, cardiology, psychiatry, geriatrics, endocrinology, respiratory medicine, gastroenterology and pediatrics. Questions asked included: "What is your current level of interest in the following medical specialties?"; "What is your current level of knowledge in the given medical specialties?"; "Do you think the subject is easy or difficult?" and "Why do you think neurology is difficult?" Students were required to answer using a Likert scale and results were tabulated into mean scores and standard errors. Results: The response rate was 65% (167/255). Neurology was identified as the subject which students found most difficult (score 3.89 +/- 0.068) and had least knowledge of (2.32 +/- 0.075). These scores were significantly different from those observed for the other disciplines (p<0.001). The need to know basic neuroscience was identified as the biggest contributor to the difficulty associated with neurology (3.89 +/- 0.072) followed closely by the complex clinical examination associated with neurology (3.69 +/- 0.072). Greater clinical and practical exposure, more time being spent on the subject, and improved teaching skills of lecturers were put forward as suggestions for improving the teaching of neurology. Conclusion: This study provides empirical evidence that 'neurophobia' may indeed exist among the student population of the school. It suggests the need to re-visit the approach to neuroscience and neurology education and is consistent with similar trends worldwide.

An evaluation of the performance in the UK Royal College of Anaesthetists primary examination by UK medical school and gender.

Andrew Bowhay — 2009-06-29T00:00:00Z

Background: There has been comparatively little consideration of the impact that the changes to undergraduate curricula might have on postgraduate academic performance. This study compares the performance of graduates by UK medical school and gender in the Multiple Choice Question (MCQ) section of the first part of the Fellowship of the Royal College of Anaesthetists (FRCA) examination. Methods: Data from each sitting of the MCQ section of the primary FRCA examination from June 1999 to May 2008 were analysed for performance by medical school and gender. Results: There were 4983 attempts at the MCQ part of the examination by 3303 graduates from the 19 United Kingdom medical schools. Using the standardised overall mark minus the pass mark graduates from five medical schools performed significantly better than the mean for the group and five schools performed significantly worse than the mean for the group. Males performed significantly better than females in all aspects of the MCQ – physiology, mean difference = 3.0% (95% CI 2.3, 3.7), p < 0.001; pharmacology, mean difference = 1.7% (95% CI 1.0, 2.3), p < 0.001; physics with clinical measurement, mean difference = 3.5% (95% CI 2.8, 4.1), p < 0.001; overall mark, mean difference = 2.7% (95% CI 2.1, 3.3), p < 0.001; and standardised overall mark minus the pass mark, mean difference = 2.5% (95% CI 1.9, 3.1), p < 0.001. Graduates from three medical schools that have undergone the change from Traditional to Problem Based Learning curricula did not show any change in performance in any aspects of the MCQ pre and post curriculum change. Conclusion: Graduates from each of the medical schools in the UK do show differences in performance in the MCQ section of the primary FRCA, but significant curriculum change does not lead to deterioration in post graduate examination performance. Whilst females now outnumber males taking the MCQ, they are not performing as well as the males.

Training simulated patients: evaluation of a training approach using self-assessment and peer/tutor feedback to improve performance

Jennifer Perera — 2009-06-29T00:00:00Z

Background: Most medical schools use simulated patients (SPs) for teaching. In this context the authenticity of role play and quality of feedback provided by SPs is of paramount importance. The available literature on SP training mostly addresses instructor led training where the SPs are given direction on their roles. This study focuses on the use of peer and self evaluation as a tool to train SPs. Methods: SPs at the medical school participated in a staff development and training programme which included a) self-assessment of their performance while observing video-tapes of their role play using a structured guide and b) peer group assessment of their performance under tutor guidance. The pre and post training performance in relation to authenticity of role play and quality of feedback was blindly assessed by students and tutors using a validated instrument and the scores were compared. A focus group discussion and a questionnaire assessed acceptability of the training programme by the SPs. Results: The post-training performance assessment scores were significantly higher (p<0.05) than the pre-training scores. The degree of improvement in the quality of feedback provided to students was more when compared to the improvement of role play. The acceptability of the training by the SPs was very satisfactory scoring an average of 7.6 out of 10. The majority of the SPs requested the new method of training to be included in their current training programme as a regular feature. Conclusions: Use of structured self-reflective and peer-interactive, practice based methods of SP training is recommended to improve SP performance. More studies on these methods of training may further refine SP training and lead to improvement of SP performance which in turn may positively impact medical education.

Needs assessment of Wisconsin primary care residents and faculty regarding interest in global health training

Terese Bauer — 2009-06-24T00:00:00Z

Background: The primary objectives of this study were to assess Wisconsin's primary care residents' attitudes toward international health training, the interest among faculty to provide IH training, and the preferred modality of IH training. Methods: Surveys were administered using 505 residents and 413 medical faculty in primary care residencies in Wisconsin. Results from 128 residents and 118 medical school faculty members were collected during the spring of 2007 and analyzed. Results: In total, 25% of residents (128/505) and 28% of faculty (118/413) responded to the survey. A majority of residents (58%) and faculty (63%) were interested in global health issues. Among residents, 63% planned on spending professional time working abroad. Few residents (9%) and faculty (11%) assess their residencies as preparing residents well to address topics relating to international health. The survey indicates that adequate faculty in Wisconsin could provide mentorship in international health as 47% (55) of faculty had experience working as a physician internationally, 49% (58) of faculty spend more than 25% clinical time caring for patient from underserved communities and 39% (46) would be willing to be involved with developing curriculum, lecturing and/or mentoring residents in international health. Conclusions: Overall, the majority of the respondents expressed high interest in IH and few felt prepared to address IH issues indicating a need for increased training in this area. The findings of this survey are likely relevant as a prototype for other primary care residencies.

The effect of a brief social intervention on the examination results of UK medical students: a cluster randomised controlled trial

Katherine Woolf — 2009-06-24T00:00:00Z

Background: Ethnic minority (EM) medical students and doctors underperform academically, but little evidence exists on how to ameliorate the problem. Psychologists Cohen et al. recently demonstrated that a written self-affirmation intervention substantially improved EM adolescents' school grades several months later. Cohen et al.'s methods were replicated in the different setting of UK undergraduate medical education. Methods: All 348 Year 3 white (W) and EM students at one UK medical school were randomly allocated to an intervention condition (writing about one's own values) or a control condition (writing about another's values), via their tutor group. Students and assessors were blind to the existence of the study. Group comparisons on post-intervention written and OSCE (clinical) assessment scores adjusted for baseline written assessment scores were made using two-way analysis of covariance. All assessment scores were transformed to z-scores (mean=0 standard deviation=1) for ease of comparison. Comparisons between types of words used in essays were calculated using t-tests. The study was covered by University Ethics Committee guidelines. Results: Groups were statistically identical at baseline on demographic and psychological factors, and analysis was by intention to treat [intervention group EM n=95, W n=79; control group EM n=77; W n=84]. As predicted, there was a significant ethnicity by intervention interaction [F(4,334)=5.74; p=0.017] on the written assessment. Unexpectedly, this was due to decreased scores in the W intervention group [mean difference=0.283; (95% CI=0.093 to 0.474] not improved EM intervention group scores [mean difference=-0.060 (95% CI=-0.268 to 0.148)]. On the OSCE, both W and EM intervention groups outperformed controls [mean difference=0.261; (95%CI=-0.047 to -0.476; p=0.013)]. The intervention group used more optimistic words (p<0.001) and more "I" and "self" pronouns in their essays (p<0.001), whereas the control group used more "other" pronouns (p<0.001) and more negations (p<0.001).DiscussionCohen et al.'s finding that a brief self-affirmation task narrowed the ethnic academic achievement gap was replicated on the written assessment but against expectations, this was due to reduced performance in the W group. On the OSCE, the intervention improved performance in both W and EM groups. In the intervention condition, participants tended to write about themselves and used more optimistic words than in the control group, indicating the task was completed as requested. The study shows that minimal interventions can have substantial educational outcomes several months later, which has implications for the multitude of seemingly trivial changes in teaching that are made on an everyday basis, whose consequences are never formally assessed.

Are clinicians being prepared to care for abused women? A survey of health professional education in Ontario, Canada

C. Nadine Wathen — 2009-06-18T00:00:00Z

Background: The current project undertook a province-wide survey and environmental scan of educational opportunities available to future health care providers on the topic of intimate partner violence (IPV) against women. Methods: A team of experts identified university and college programs in Ontario, Canada as potential providers of IPV education to students in health care professions at the undergraduate and post-graduate levels. A telephone survey with contacts representing these programs was conducted between October 2005 and March 2006. The survey asked whether IPV-specific education was provided to learners, and if so, how and by whom. Results: In total, 222 eligible programs in dentistry, medicine, nursing and other allied health professions were surveyed, and 95% (212/222) of programs responded. Of these, 57% reported offering some form of IPV-specific education, with undergraduate nursing (83%) and allied health (82%) programs having the highest rates. Fewer than half of undergraduate medical (43%) and dentistry (46%) programs offered IPV content. Postgraduate programs ranged from no IPV content provision (dentistry) to 41% offering content (nursing). Conclusions: Significant variability exists across program areas regarding the methods for IPV education, its delivery and evaluation. The results of this project highlight that expectations for an active and consistent response by health care professionals to women experiencing the effects of violence may not match the realities of professional preparation.

Improving education in primary care: development of an online curriculum using the blended learning model

Linda Lewin — 2009-06-10T00:00:00Z

Background: Standardizing the experiences of medical students in a community preceptorship where clinical sites vary by geography and discipline can be challenging. Computer-assisted learning is prevalent in medical education and can help standardize experiences, but often is not used to its fullest advantage. A blended learning curriculum combining web-based modules with face-to-face learning can ensure students obtain core curricular principles. Methods: This course was developed and used at The Case Western Reserve University School of Medicine and its associated preceptorship sites in the greater Cleveland area. Leaders of a two-year elective continuity experience at the Case Western Reserve School of Medicine used adult learning principles to develop four interactive online modules presenting basics of office practice, difficult patient interviews, common primary care diagnoses, and disease prevention. They can be viewed at http://casemed.case.edu/cpcp/curriculum. Students completed surveys rating the content and technical performance of each module and completed a Generalist OSCE exam at the end of the course. Results: Participating students rated all aspects of the course highly; particularly those related to charting and direct patient care. Additionally, they scored very well on the Generalist OSCE exam. Conclusion: Students found the web-based modules to be valuable and to enhance their clinical learning. The blended learning model is a useful tool in designing web-based curriculum for enhancing the clinical curriculum of medical students.

Evidence of gender bias in True-False-Abstain medical examinations

Shona Kelly — 2009-06-07T00:00:00Z

Background: There is evidence that males and females differ in their attainment on a variety of assessments in general and in medical education. It has been suggested that the True-False-Abstain (TFA) format with negative marking is biased against females. Methods: Eight years worth of examination data from the first two years of an undergraduate medical curriculum was analysed. 359 courses were evaluated for statistically significant differences between the genders using ANOVA. Logistic regression was used to test if subject area, calendar year or exam format predicted that males or females do better (termed male advantage or female advantage). Results: Statistically significant differences between the genders were found in 111 (31%) of assessments with females doing better than males in 85 and males better in 26. Female advantage was associated with a particular year (2001), the Personal and Professional Development strand of the curriculum, in course assessment and short answer questions. Male advantage was associated with the anatomy and physiology strand of the curriculum and examinations containing TFA formats, where the largest gender difference was noted. Males were 16.7 times more likely than females to do better on an assessment if it had any questions using the TFA format. Conclusion: Although a range of statistically significant gender differences was found, they were concentrated in TFA and short answer formats. The largest effect was for TFA formats where males were much more likely to do better than females. The gender bias of TFA assessments in medical education is yet another reason why caution should be exercised in their use.

Investigating the barriers to teaching family physicians' and specialists' collaboration in the training environment: a qualitative study

Marie-Dominique Beaulieu — 2009-06-07T00:00:00Z

Background: Collaboration between physicians in different specialties is often taken for granted. However, poor interactions between family physicians and specialists contribute significantly to the observed discontinuity between primary and specialty care. The objective of this study was to explore how collaboration between family physicians and specialists was conceptualised as a competency and experienced in residency training curricula of four faculties of medicine in Canada. Methods: This is a multiple-case study based on Abbott's theory of professions. Programs targeted were family medicine, general psychiatry, radiology, and internal medicine. The content of the programs' objectives was analyzed. Associate deans of postgraduate studies, program directors, educators, and residents were interviewed individually or in focus groups (47 residents and 45 faculty members). Results: The training objectives related to family physicians-specialists collaboration were phrased in very general terms and lacked specificity. Obstacles to effective collaboration were aggregated under themes of professional responsibility and questioned expertise. Both trainees and trainers reported increasing distances between specialty and general medicine in three key fields of the professional system: the workplace arena, the training setting, and the production of academic knowledge. Conclusion: The challenges of developing collaborating skills between generalists and specialist physicians are comparable in many ways to those encountered in inter-professional collaboration and should be given more consideration than they currently receive if we want to improve coordination between primary and specialty care.