Email updates

Keep up to date with the latest news and content from BMC Medical Education and BioMed Central.

Open Access Research article

Does mentoring new peer reviewers improve review quality? A randomized trial

Debra Houry1, Steven Green2 and Michael Callaham3*

Author Affiliations

1 Department of Emergency Medicine, Emory University School of Medicine, Atlanta, GA, USA

2 Department of Emergency Medicine, Loma Linda School of Medicine, Los Angeles, CA, USA

3 Department of Emergency Medicine, University of California, San Francisco, CA, USA

For all author emails, please log on.

BMC Medical Education 2012, 12:83  doi:10.1186/1472-6920-12-83

Published: 28 August 2012

Abstract

Background

Prior efforts to train medical journal peer reviewers have not improved subsequent review quality, although such interventions were general and brief. We hypothesized that a manuscript-specific and more extended intervention pairing new reviewers with high-quality senior reviewers as mentors would improve subsequent review quality.

Methods

Over a four-year period we randomly assigned all new reviewers for Annals of Emergency Medicine to receive our standard written informational materials alone, or these materials plus a new mentoring intervention. For this program we paired new reviewers with a high-quality senior reviewer for each of their first three manuscript reviews, and asked mentees to discuss their review with their mentor by email or phone. We then compared the quality of subsequent reviews between the control and intervention groups, using linear mixed effects models of the slopes of review quality scores over time.

Results

We studied 490 manuscript reviews, with similar baseline characteristics between the 24 mentees who completed the trial and the 22 control reviewers. Mean quality scores for the first 3 reviews on our 1 to 5 point scale were similar between control and mentee groups (3.4 versus 3.5), as were slopes of change of review scores (-0.229 versus -0.549) and all other secondary measures of reviewer performance.

Conclusions

A structured training intervention of pairing newly recruited medical journal peer reviewers with senior reviewer mentors did not improve the quality of their subsequent reviews.

Keywords:
Mentoring; Peer review; Scientific publication; Critical analysis; Journal peer reviewer