The peer review crisis

Peer review in science, in which independent scientists who are experts on the subject assess the paper, is the current strategy for ensuring quality and control in scientific research and, therefore, it is essential for the academic world.  However, the peer review system has been criticized especially in recent years.

The main weaknesses are currently related to three aspects: the voluntary nature of the peer review, since 'peers' only participate in the process if they wish, the disparity of review criteria or guidelines produced by scientific journals, and a lack of tangible recognition of the reviewers for their service to the scientific community. All of the above makes the peer review process slow, highly subjective, and results in reviews of greatly varying quality.

This is a serious problem, not only for the scientific community, but also for the publishers, which in the last few years have openly recognized that it is increasingly difficult to secure participation of scientists as reviewers e.g. in the field of ecology, the non-acceptance rate for requests to review articles is 49%.  I can confirm this although it is not necessarily the lack of recognition but more a lack of time. A review of good quality requires time and I simply don't have much of it at my disposal and as a result the number of reviews I can provide is low. It also seems that the number of requests skyrocketed within the last years. 

A group of researchers from Trent University in Peterborough had a closer look at the history of the peer review process and the current situation for the leading journals in the field of ecology and evolution.

We provide historical context for the cultural lag that governs peer review that has eventually led to the system's current structural weaknesses (voluntary review, unstandardized review criteria, decentralized process). We argue that some current attempts to upgrade or otherwise modify the peer-review system are merely sticking-plaster solutions to these fundamental flaws, and therefore are unlikely to resolve them in the long term.

In addition to their historical research the colleagues conducted a survey in 38 selected journals and all of those confirmed that peer reviews were "not consistent", a cross-cutting issue in practically all scientific fields. The problem is that the definition of an ideal peer review is somewhat complex and, currently, even journals with highly organised systems and well defined guidelines struggle with weak reviews. Some mitigating measures have been suggested such as the privatization of peer reviews and making participation mandatory.

Applying correction factors to the h-index -the highest number of articles that an author has published and been cited at least the same number of times-, paying fees to the reviewers or offering them royalties (such as discounts on subscription fees or acknowledgement notes) are some of the proposals. Moreover, some editors are making efforts to homogenize review criteria between them, such as the British Ecological Society, and some review guidelines exist, although there is no agreed criteria on which is the most relevant.

A disparity of methods used by the journals to instruct their reviewers on peer reviews was evident in all cases. This ranged from the complete absence of guidelines and unclear criteria, to more formal systems with forms and defined criteria.

None of the measures proposed to date has the potential to resolve the problems in the long term, because they are partial and not holistic. In our opinion, a contemporary peer review process, in which the current needs of the scientific community are addressed, should be centralized in a platform -independent of journals, whose interests are above all financial- with clear review criteria and guidelines, adjusted according to the scientific field.

The researchers also propose that this centralization ought to be led by scientists, as this would facilitate the standardization of the process, as well as increasing its transparency and reliability.

I am not sure if the paper covers all issues the current peer review system faces. Although I have no doubt that the weaknesses the colleagues identified are root causes for this peer review crisis I am also not sure if the proposed ideas will help to rescue it. 

What do you think? 

Feel free to share your ideas and thoughts using the comments.