Video: Is peer review broken?

Posted by Biome on 25th September 2013 - 5 Comments

To coincide with this year’s Experimental Biology conference, BioMed Central and BMC Biology organised a panel discussion to discuss the increasing frustration with the peer review process from the scientific community.

Chaired by Greg Petsko (who neatly summarised some of the problems before the discussion here) the panel brought together representatives of each role in the process: scientist, reviewer and Editor (both academic and professional).

 

“The thesis is that there is something seriously wrong with the scientific review process… which has become, if not outright broken, distorted to the point where it’s hindering people’s careers, and causing more problems than it is solving.”

Greg Petsko, Weill Cornell Medical College, USA

The discussion itself is summarised here, and this video brings together some opinions voiced on the evening, from Greg Petsko, Emilie Marcus, Hidde Ploegh, Josh Sanes and Laurie Goodman.

 

“I do think there are inefficiencies … it’s important that we look for what those inefficiencies are and identify ways to improve them… that preserve the value and quality of peer review.”

Emilie Marcus, CEO Cell Press

 

Emilie Marcus, CEO of Cell Press and Editor of Cell, gives her thoughts on the role of the Editor and the reviewer in the process, and current inefficiencies in the system.

Hidde Ploegh (whose high profile call in Nature to ‘End the wasteful tyranny of reviewer experiments’ was referenced throughout the discussion) reiterates his point that funding for science should not be being spent on time-consuming rounds of review in order to publish in high-profile journals, but to ‘get the scientific question answered’.

Josh Sanes, co-Editor-in-Chief of Neural Development, discusses the non-linear effect that journal prestige currently has on the career progression of young scientists.

 

“The hierarchy of journals has an effect that’s completely disproportionate to the real difference in quality of the papers that are published.”

Joshua Sanes, Harvard University

Throughout the discussion, questions and comments from the floor were forthcoming, and Laurie Goodman (Editor of GigaScience) provided an example of open peer review at it’s most beneficial (which Biome has covered in more detail here).

Goodman concludes the video with her thoughts on how opening up the process can be beneficial, but that a ‘one size fits all’ approach cannot meet the needs of differing scientific communities.

  • Pandelis Perakakis

    A lot of discussion on how reviewers should improve the way they evaluate papers, but do we really expect them to do so without proper recognition for their significant contribution to science?

    Anonymous reviewing has been repeatedly criticized as prone to bias and even unethical conduct, but more importantly it is problematic because it does not provide scholars with incentives to submit thorough and constructive criticisms.

    Another problem, also discussed here, is the role of journal editors in the review process. A serious problem is that journal editors are not always the most qualified to select adequate reviewers for each paper and in addition, they are influenced by their interest in raising their journal’s status by maintaining high rejection rates and selecting articles based on their citation expectancy.

    Representing an international, not-for-profit organization founded to improve the way scientific papers are evaluated, I support that it is time to challenge two fundamental ideas in the current peer review model:

    First, that scientists are not ready to openly assess each others’ papers if they are properly acknowledged for their work.

    Second, that peer review can only be arranged and handled by journal editors.

    Instead we propose a journal-independent, author-guided open peer review system that can complement journal peer review and can significantly improve and expedite the journal publication process.

    Our free, multidisciplinary open peer review platform called LIBRE will enable authors to directly invite expert peers to formally and openly evaluate their work posted in any online archive (libraries, repositories, preprint servers, etc). Based on these reviews, articles can be updated and submitted to any academic journal. Editors receiving these preprint reviewed papers will be able to make a better-informed decision on whether an article is methodologically sound and merits publication in their journal. This reduces risk-taking on behalf of journal editors and improves the quality of articles before they are even submitted for formal journal peer review.

  • Adam Etkin

    Peer review is not perfect, but when done properly it is pretty darn good and serves a crucial role in scholarly publishing. It seems to me that watching this video much of what is being discussed is what is “broken” in Academe, not peer review.

  • Claire Pillar

    An interesting discussion, thank you. I agree that sometimes reviewers are reviewing what the paper is not about, and clear parameters need to be set. However, do reviewers get enough recognition for the valuable work they do? The delays in publication are unfortunate, and we have the expertise and technology to chip away at this problem.

  • Dave Fernig

    Excellent discussion. I would add to Greg Pestko’s point on the desperate need for editors to start acting like editors and make editorial decisions that they also need to at on post publication peer review. We have far too many euphemistic corrections for what is clearly wrong and a general lack of concern over clear problems identified by the community. For example, only a tiny minority of clearly important concerns voiced at PubPeer.com have actually been taken on board by journal editors.

  • Pingback: Is peer review broken? | Ferniglab's Blog