Portal de Periódicos da CAPES

Peer‐review warning: system error, reviewers not found

2015; Wiley; Volume: 13; Issue: 5 Linguagem: Inglês

10.1890/15.wb.009

1540-9309

Lluı́s Brotons,

Expert finding and Q&A systems

By providing an independent and critical evaluation of new research findings, the peer-review system aims at ensuring the credibility of to-be-published science. Despite the critical role of scholarly peer review, it has received many criticisms (Smith 2006). In addition, several issues – including an increasing number of submissions to journals, competition among and pressure for authors to publish their findings, and a generally inadequate recognition of the reviewer's role by academic institutions – are challenging this system through a syndrome termed the “tragedy of the reviewer commons” (Hochberg et al. 2009). Together with a commitment by journals to provide prompt decisions regarding the fate of submissions under consideration, reviewers are being exploited at faster rates, thereby making it increasingly difficult for journals to obtain a desired number of expert reviews. When a given manuscript is sent for external review and the reviewers of choice refuse to review the work, editors then look for alternate reviewers. However, the shortage of available reviewers has started to push journal editors toward dangerous waters. For example, one possible response to this shortage is the rejection of a submitted manuscript based on a single argument – namely, that a valid measure of a manuscript's quality is the willingness of reviewers to review it, and that if a large number of reviewers refuse to review a given manuscript beyond what is expected by chance, then that manuscript is of low quality and should not be reviewed. Recently, an editor of a prestigious ecological journal declined to publish a manuscript because 15 potential reviewers in a row had refused to review it. The editor's argument relied on the fact that for every 10 potential reviewers invited to review a manuscript submitted to this journal, seven typically decline; at the stated 7/10 or 0.7 refusal probability, the probability of consecutive refusal by 15 reviewers is 0.5% (where [0.7]15 = 0.005 or 0.5%). The editor then argued that this probability of refusal was much lower than would occur by chance, which the editor defined as any probability that was less than 5% (or, nine refusals in a row, where [0.7]9 = 0.04 or 4%). Reviewer refusal is often associated with external academic pressure; proper reviewing is time-consuming, and reviewers gain little recognition of their work in a competitive academic context (Fischer et al. 2012). Peer review depends on many additional factors that are also outside of the authors' control, such as the ability of the assigning editors to choose likely and appropriate reviewers. For example, if 15 potential reviewers were chosen whose expertise is well outside that of the manuscript, then one would expect a high probability of reviewer refusal, through no fault of the authors. Worryingly, if reviewer-refusal probabilities increase to values above 0.9, editors would be forced to search for at least 29 reviewers if they want to ensure that refusals based on consecutive negative responses did not originate by chance. Because journals usually secure at least two reviewers for each manuscript, it is easy to see how a large number of manuscripts could be potentially excluded from the peer-review process due to the lack of reviewers if this approach was implemented by other journals. Several alternatives to the traditional peer-review process have been proposed, some of which have already been put into practice. Entirely “open” peer-review systems such as F1000Research (http://f1000research.com) increase the transparency of the review process and ensure that research is communicated even before the review process has ended. Other initiatives include third-party review services such as Peerj (https://peerj.com), Axios Review (http://axiosreview.org), or “Peerage of Science” (https://www.peerageofscience.org). Such services have arisen – in part – as a response to the reviewer crisis but focus their solutions on an improved targeting of submitted manuscripts to a journal's scope and to reducing unnecessary reviews for a given manuscript. “Peerage of Science” also aims to recognize the most active and “best” reviewers, to allow for tangible benefits being received by reviewers when doing their work. Early testimonials from such services appear promising and are elements of a rapidly evolving publishing landscape (Hames 2014). Nevertheless, these alternatives, or any of the above strategies when implemented alone, will not easily transform our current peer-review system in the short term. Given these first signs of system collapse, urgent and pragmatic measures are needed to explicitly increase the pool of reviewers and strongly reduce their burden of work. Different solutions may be available and include the promotion of more early-career scientists into active reviewing, and the reduction of unnecessary reviews via improved inter-journal coordination. The peer-review process will also greatly benefit from multi-tiered review approaches. In these cases, an initial screening stage, involving a representative number of reviewers, could represent a first, simple, standardized assessment of the potential quality of a manuscript that (upon approval) could move into a second, stricter round of reviews. Reviewers would then have the possibility to better adjust their effort to the potential quality of the manuscripts, and journals would help to ensure fair decisions within the peer-review system. Finally, the scientific community needs to recognize the critical role of reviewers within the broader debate of where the peer-review process is heading. Given that it represents the cornerstone of science, a reviewer's efforts should start to be seriously included as part of that researcher's professional merit by his or her respective academic institution.