Research article

Self- and peer assessment may not be an accurate measure of PBL tutorial process

José Lúcio Martins Machado , Valéria Menezes Peixeiro Machado , Waldir Grec , Valdes Roberto Bollela and Joaquim Edson Vieira

UNICID – Universidade Cidade de São Paulo Medical School, Rua Cesário Galeno 448/475, CEP 03071-000, São Paulo, Brazil

author email corresponding author email

BMC Medical Education 2008, 8:55doi:10.1186/1472-6920-8-55

Published:	27 November 2008

Abstract

Background

Universidade Cidade de São Paulo adopted a problem-based learning (PBL) strategy as the predominant method for teaching and learning medicine. Self-, peer- and tutor marks of the educational process are taken into account as part of the final grade, which also includes assessment of content. This study compared the different perspectives (and grades) of evaluators during tutorials with first year medical students, from 2004 to 2007 (n = 349), from seven semesters.

Methods

The tutorial evaluation method was comprised of the students' self assessment (SA) (10%), tutor assessment (TA) (80%) and peer assessment (PA) (10%) to calculate a final educational process grade for each tutorial. We compared these three grades from each tutorial for seven semesters using ANOVA and a post hoc test.

Results

A total of 349 students participated with 199 (57%) women and 150 (42%) men. The SA and PA scores were consistently greater than the TA scores. Moreover, the SA and PA groups did not show statistical difference in any semester evaluated, while both differed from tutor assessment in all semesters (Kruskal-Wallis, Dunn's test). The Spearman rank order showed significant (p < 0.0001) and positive correlation for the SA and PA groups (r = 0.806); this was not observed when we compared TA with PA (r = 0.456) or TA with SA (r = 0.376).

Conclusion

Peer- and self-assessment marks might be reliable but not valid for PBL tutorial process, especially if these assessments are used for summative assessment, composing the final grade. This article suggests reconsideration of the use of summative assessment for self-evaluation in PBL tutorials.