Video: Is peer review broken?

Posted by Biome on 25th September 2013 - 20 Comments


To coincide with this year’s Experimental Biology conference, BioMed Central and BMC Biology organised a panel discussion to explore the increasing frustration with the peer-review process from the scientific community.

Chaired by Greg Petsko (who neatly summarised some of the problems before the discussion here) the panel brought together representatives of each role in the process: scientist, reviewer and Editor (both academic and professional).

 

“The thesis is that there is something seriously wrong with the scientific review process… which has become, if not outright broken, distorted to the point where it’s hindering people’s careers, and causing more problems than it is solving.”

Greg Petsko, Weill Cornell Medical College, USA

The discussion itself is summarised here, and this video brings together some opinions voiced on the evening, from Greg Petsko, Emilie Marcus, Hidde Ploegh, Josh Sanes and Laurie Goodman.

 

“I do think there are inefficiencies … it’s important that we look for what those inefficiencies are and identify ways to improve them… that preserve the value and quality of peer review.”

Emilie Marcus, CEO Cell Press

 

Emilie Marcus, CEO of Cell Press and Editor of Cell, gives her thoughts on the role of the Editor and the reviewer in the process, and current inefficiencies in the system.

Hidde Ploegh (whose high profile call in Nature to ‘End the wasteful tyranny of reviewer experiments’ was referenced throughout the discussion) reiterates his point that funding for science should not be being spent on time-consuming rounds of review in order to publish in high-profile journals, but to ‘get the scientific question answered’.

Josh Sanes, co-Editor-in-Chief of Neural Development, discusses the non-linear effect that journal prestige currently has on the career progression of young scientists.

 

“The hierarchy of journals has an effect that’s completely disproportionate to the real difference in quality of the papers that are published.”

Joshua Sanes, Harvard University

Throughout the discussion, questions and comments from the floor were forthcoming, and Laurie Goodman (Editor of GigaScience) provided an example of open peer review at it’s most beneficial (which Biome has covered in more detail here).

Goodman concludes the video with her thoughts on how opening up the process can be beneficial, but that a ‘one size fits all’ approach cannot meet the needs of differing scientific communities.

  • Pandelis Perakakis

    A lot of discussion on how reviewers should improve the way they evaluate papers, but do we really expect them to do so without proper recognition for their significant contribution to science?

    Anonymous reviewing has been repeatedly criticized as prone to bias and even unethical conduct, but more importantly it is problematic because it does not provide scholars with incentives to submit thorough and constructive criticisms.

    Another problem, also discussed here, is the role of journal editors in the review process. A serious problem is that journal editors are not always the most qualified to select adequate reviewers for each paper and in addition, they are influenced by their interest in raising their journal’s status by maintaining high rejection rates and selecting articles based on their citation expectancy.

    Representing an international, not-for-profit organization founded to improve the way scientific papers are evaluated, I support that it is time to challenge two fundamental ideas in the current peer review model:

    First, that scientists are not ready to openly assess each others’ papers if they are properly acknowledged for their work.

    Second, that peer review can only be arranged and handled by journal editors.

    Instead we propose a journal-independent, author-guided open peer review system that can complement journal peer review and can significantly improve and expedite the journal publication process.

    Our free, multidisciplinary open peer review platform called LIBRE will enable authors to directly invite expert peers to formally and openly evaluate their work posted in any online archive (libraries, repositories, preprint servers, etc). Based on these reviews, articles can be updated and submitted to any academic journal. Editors receiving these preprint reviewed papers will be able to make a better-informed decision on whether an article is methodologically sound and merits publication in their journal. This reduces risk-taking on behalf of journal editors and improves the quality of articles before they are even submitted for formal journal peer review.

  • Adam Etkin

    Peer review is not perfect, but when done properly it is pretty darn good and serves a crucial role in scholarly publishing. It seems to me that watching this video much of what is being discussed is what is “broken” in Academe, not peer review.

  • Claire Pillar

    An interesting discussion, thank you. I agree that sometimes reviewers are reviewing what the paper is not about, and clear parameters need to be set. However, do reviewers get enough recognition for the valuable work they do? The delays in publication are unfortunate, and we have the expertise and technology to chip away at this problem.

  • Dave Fernig

    Excellent discussion. I would add to Greg Pestko’s point on the desperate need for editors to start acting like editors and make editorial decisions that they also need to at on post publication peer review. We have far too many euphemistic corrections for what is clearly wrong and a general lack of concern over clear problems identified by the community. For example, only a tiny minority of clearly important concerns voiced at PubPeer.com have actually been taken on board by journal editors.

  • Pingback: Is peer review broken? | Ferniglab's Blog

  • Pingback: Peer Review is full of excuses | Library Attack!Library Attack!

  • Pingback: Being critical | BVet - Course Reflections

  • Pingback: Biome |

  • Pingback: The importance of blogs (and web 2.0) in exposing failings in peer review | Self assembly @ Keele

  • Pingback: Peer review under scrutiny - BioMed Central blog

  • JSP

    Having served on numerous federal and private grant review committees, and reviewed > 1000 papers in my career, my sense is that peer review has reached an unfortunate dichotomy: reviewers largely act either as advocates, championing a particular grant or manuscript whose P.I. or author they know well, or as competitors, trying to stall work by, as Hidde Plough so aptly put it in lapidary fashion, “the wasteful tyranny of reviewer experiments.” What is missing – in too many cases – is objectivity. A legitimate appeals process could help steer things towards back onto the track of impartiality.

  • BabyBoomerWriter

    Popularity contests of any kind rarely focus on more than superficial qualities. Naturally, prestige brands are under constant pressure to preserve their “luxury” status, sometimes at the cost of taking a risk on new ideas. We might note that this is the antithesis of what fuels entrepreneurial and innovative energy.

  • denis m

    I have reviewed 100s of papers & grant proposals and find that the major constraint to doing a quality review is sheer pressure of time. My univ admin also increasingly questions why I should devote time to such unpaid work – despite my obvious argument about reciprocity. A more compelling argument is the rise of open access journals where many of now are forced to publish by our funding bodies – these journals charge me $2-4k per article and yet expect me to provide quality refereeing for free. It is only my sense of responsibility and commitment to my research community that makes me continue as a reviewer (& journal editor) but this may not last in the current climate. I’m not at all convinced that paying referees is the way forward but some journals already do this and it would be interesting to see what other readers of this forum might think.

  • Pingback: Biome | The sting of peer review

  • Pingback: Is peer review broken? | PEER REVIEW WATCH

  • samir

    I think that peer review with all its shortcomings is here to stay. Obvious would you say !!!! The point that I want to make is what’s at stake are careers that hinge on a publication. Publication that could very well be stalled by a competitor.How many times have we received less than courteous reviews ( borderline insulting ) without ever knowing who dared treating us in such a way !!!!! Open peer review seems to be the solution .I said seems because in some cases it creates more problems that it solves but in a world where accountability has become the buzz word it is rather ludicrous to be judged and not know who’s the judge !!!!

  • Pingback: Epic Video Games for Science › From The Lab Bench

  • Pingback: Insert title here (thanks Kam and Ruth!) - BioMed Central blog

  • Pingback: We Can Fix Peer Review Now | Pubchase Blog

  • Pingback: Innovations in peer review:… « Another Word For It