Using video-cases to assess student reflection: Development and validation of an instrument
1 Centre for Educational Development, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium
2 Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
3 Department of Educational Studies, Faculty of Psychology and Educational Sciences, Ghent University, Ghent, Belgium
4 Institute for Medical Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
5 Centre for Research and Innovation in Medical Education, Faculty of Medical Sciences, University of Groningen and University Medical Centre Groningen, Groningen, the Netherlands
Citation and License
BMC Medical Education 2012, 12:22 doi:10.1186/1472-6920-12-22Published: 20 April 2012
Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.
Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.
Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.
Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.