Caustic volleys and the sting of peer review: what’s the solution?

Posted by Biome on 19th April 2013 - 10 Comments


“The peer review process sometimes seems to reflect all the civility of being thrown to the lions in the Coliseum.”

- Gregory Petsko – Weill Cornell Medical College, USA

Increasing frustration with a peer review process that sometimes seems to reflect all the civility of being thrown to the lions in the Coliseum has led to a flood of commentaries with suggestions for reform. These range from a restrained comment from Raff et al. in Science, in which they point out that getting experimental work published can take as long as or longer than doing the work in the first place, and that the extra experiments demanded by referees frequently only strengthen the conclusions marginally, to the later invective against the tyranny of reviewer experiments (sic) in which Hidde Ploegh makes similar points in Nature. Current in many labs is a spoof carol with the first line “Wreck their scrawls with caustic volleys”, sung to the tune of to the tune of “Deck the Halls”. Suggestions for reform include Virginia Walbot’s comment on how to train postdocs not to be pit-bull reviewers and, among many others, the publication policy of eLife, whose stated raison d’etre is to change the peer-review process, as well as the policy operated by BMC Biology, whereby authors may opt out of re-review after revision of their papers.

It could reasonably be argued that none of this would ever have become necessary had the scientific community not lost sight of the fact that the responsibility of a reviewer is to review the paper as written, not to redesign the science the way he or she believes it should have been done. As Bob Horvitz has neatly put it: “…what is in the paper is fundamentally the responsibility of the authors, not of the reviewers”. Furthermore, journal editors should be willing to disregard unreasonable requests from reviewers, and not act as though the role of the journal was to set the direction of science and micromanage its conduct. Editors need to be more responsible, and journal policies could use improvement, but arguably the problem with peer review can only be fixed by an attitude adjustment on the part of reviewers: a recognition that follow-up and confirming experiments belong in future papers, combined with a humility that eschews showing off in favor of actually doing one’s job. As Walt Kelly’s philosophical possum Pogo so eloquently put it, “We have met the enemy, and he is us.”

Gregory Petsko

 

Panel discussion on peer review and painless publishing

 

As part of the celebrations of the tenth anniversary of BMC Biology’s publication, BioMed Central invited Gregory Petsko to chair a panel discussion on peer review with Hidde Ploegh and others in Boston on Sunday April 21, to provide a forum for the views of editors, reviewers, authors , and especially those who can claim to be all three, to put their points of view about the conduct of peer review and arrive at constructive solutions to the problems.

Some opinions voiced on the evening, from panelists Greg Petsko, Emilie Marcus, Hidde Ploegh, Josh Sanes and Laurie Goodman, are featured in this video.

 

Peer review experiments at BioMed Central

 

BioMed Central publishes a number of journals that have been experimenting with alternative peer review models, all aiming to improve transparency and address some of the problems discussed by Gregory Petsko above and by the participants at the panel discussion:

 

Editors-in-Chief: Eugene Koonin, Laura Landweber & David Lipman

Biology Direct offers a unique system of peer review: Authors are allowed to select suitable reviewers from the Editorial Board; and the peer-review process is made fully open to authors and readers, thus increasing the responsibility of the referees and eliminating sources of abuse in the refereeing process.

 

Editor: Miranda Robertson

BMC Biology operates a re-review opt-out policy: It offers authors who are asked to make revisions before a final decision is taken on their papers the opportunity to choose whether the paper is seen again by referees or whether the revisions and authors responses are judged by the editors alone.

 

Editor: Sabina Alam

BMC Medicine and all subject-specific medical BMC-series journals operate an ‘open peer review’ policy: Reviewers reveal their identity to the authors; and if an article is published, the complete pre-publication history, including all submitted versions, reviewers’ reports (and names) and authors’ responses, is provided with the published article.

 

  • Pingback: Panel discussion on peer review and painless publishing - Biome

  • Pingback: Peer review of peer reviewing - BioMed Central blog

  • Pingback: Can open peer review work? Biology Direct suggests it can… - BioMed Central blog

  • Ken

    Great article.

    I am an editor of a peer-reviewed journal, and it is a headache sometimes. In most cases, it’s relatively easy – if the reviewers are all happy or all unhappy, and their arguments are reasonable, and I feel much the same, then I can go with them.

    Sometimes, however, the reviewers are split. Part of my job is to see quality papers published. So, both “quality” and “published” are important. This means that I cannot
    simply reject stuff if there is a split – that would not be fair to authors. I have to give them every reasonable chance.

    I think too many editors go with the “lowest (un)common denominator” approach. Editors are able to make decisions also, and should weigh up their reviewers’ comments. Sometimes, a reviewer will miss the boat completely, and the review simply cannot be taken into account. I don’t want to waste the authors’ time by expecting them to defend themselves against stuff that is simply wrong.

    The concept of peer-review must also be taken literally. The reviewers are professional PEERS, which means that a single reviewer’s opinion must not automatically be given more weight than the authors’. After all, the choice of reviewers is non-random sample of 3 or 4, and that is not easily scientifically defensible.

  • Addisu Tadege

    Authors are being thrown away by the editors before the paper is reviewed by the peer reviewers and I do not think this is a right way of publishing articles in Biomed

  • Pingback: July 15, 2013 | kuspfyi

  • Pingback: Biome | Is peer review broken?

  • Pingback: Discovery notes – brief research findings, published fast - BioMed Central blog

  • Pingback: Innovations in peer review: join a discussion with our Editors - BioMed Central blog

  • Anthony Murawski

    “After all, the choice of reviewers is non-random sample of 3 or 4, and that is not easily scientifically defensible.”

    This sums it up precisely. Pre-publication peer review by the a few people chosen in some cases due to biases does not constitute true “peer review.” Meaningful peer review only occurs post-publication, when a large group of peers with a variety of views provide their perspective. I have seen several instances of rubbish published in the most “prestigious” medical journals, with the the peer-review process used as a form of censorship to prevent publication of well-founded, divergent views that would disprove claims made by the reviewers themselves in prior published articles. If an article is rejected for publication, a journal should provide a written justification to the authors so that scientific research becomes a transparent, depoliticized search for objective information.