Portal de Periódicos da CAPES

Peering into Peer-Review

2011; Elsevier BV; Volume: 159; Issue: 1 Linguagem: Inglês

10.1016/j.jpeds.2011.02.012

1097-6833

Monica L. Helton, William F. Balistreri,

Health and Medical Research Impacts

Peer-review is defined as the expert assessment of submitted materials.1Bailar J.C. Patterson K. Journal peer review: the need for a research agenda.N Engl J Med. 1985; 312: 654-657Crossref PubMed Scopus (89) Google Scholar Because "the goal of this process is to ensure that the valid article is accepted, the messy article improved, and the invalid article rejected,"2Bornmann L. Daniel H.-D. The manuscript reviewing process: empirical research on review requests, review sequences, and decision rules in peer review.Libr Information Sci Res. 2010; 32: 5-12Crossref Scopus (37) Google Scholar quality reviews are essential to the peer-review process and assist in maintaining the integrity of the medical literature. Grainger believes that "a critical determinant of any successful journal or technical communication in general is the sound, reliable capability to readily access a talented, adept, accomplished and reliable reviewer pool."3Grainger D.W. Peer review as professional responsibility: a quality control system only as good as the participants.Biomaterials. 2007; 28: 5199-5203Crossref PubMed Scopus (43) Google Scholar Because The Journal of Pediatrics publishes articles focusing on a wide variety of specialties and subspecialties in the field of pediatrics, The Journal uses an extensive historical database of experts, to which more are added daily. However, because of heavy workloads and time constraints of potential reviewers, it is often challenging for the editors to secure a commitment to review a manuscript from appropriate reviewers. Editors of The Journal usually choose reviewers from these sources: (1) personal knowledge of experts in the field; (2) authors of articles referenced in the manuscript; (3) reviewers suggested by the authors; (4) literature searches; (5) requesting suggestions from the editorial board; and (6) Elsevier Editorial System (EES) report of manuscript/personal classification matches.4Balistreri W.F. Landmark, landmine, or landfill? The role of peer review in assessing manuscripts.J Pediatr. 2007; 151: 107-108Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar Occasionally, reviewers will be invited because another reviewer who declined or was uninvited suggests an individual as a good alternative candidate. Occasionally editors will choose reviewers from eTBlast, a freely available "similarity-based search engine," which allows them to find "authors that are the most published in the topic of [their] query" (http://etest.vbi.vt.edu/etblast3/). The Journal uses a single-blind review process; reviewers have access to the authors' names and affiliations, but the identity of reviewers is confidential. Manuscript submission and subsequent peer-review occurs through EES, the online submission and editorial system for The Journal (http://ees.elsevier.com/jpeds). The Journal invites authors to suggest 5 to 7 potential reviewers in their initial letter of submission. Reviewers who accept the invitation are e-mailed specific reviewer instructions (http://jpeds.com/authorinfo#guiderev), which ask them to consider the quality of the manuscript, the novelty and importance of the observation, appropriateness for the readership, and overlap between any other publications. After a thorough assessment of the manuscript, reviewers submit their confidential comments to the editors and anonymous comments to authors through EES. After all reviews have been received for a manuscript, the editor considers the reviewers' assessments and the editorial policies and criteria specific to The Journal and makes a decision. The Journal publishes <20% of submitted manuscripts, which are rarely approved for acceptance without at least one revision. Because conflicts of interest for reviewers are just as important as conflicts of interest for authors, when an invited reviewer is at the same department or institution as the authors, has conducted research with the authors, or is aware of anything that may be perceived as a conflict of interest, the reviewer should not accept the invitation to review.9Barbour V. Clark J. Peiperly L. Veitch E. Wong M. Yamey G. Making sense of non-financial competing interests.PLoS Medicine. 2008; 5: 1299-1301Google Scholar When reviewers believe that, despite the circumstances, a fair and objective review can be conducted, they must notify the editor of the potential conflict(s) of interest in their confidential comments to the editor. When the reviewer is uncertain whether objectivity can be attained, it is best to err on the side of caution and decline the invitation. Studies have found that editor-suggested reviewers (ESRs) were less likely than author-suggested reviewers (ASRs) to recommend acceptance.5Earnshaw J.J. Farndon J.R. Guillou P.F. Johnson C.D. Murie J.A. Murray G.D. A comparison of reports from referees chosen by authors or journal editors in the peer review process.Ann R Coll Surg Engl. 2000; 82: 133-135PubMed Google Scholar, 6Wager E. Parkin E.C. Tamber P.S. Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study.BMC. 2006; 4: 13Crossref Scopus (61) Google Scholar, 7Rivara F.P. Cummings P. Ringold S. Bergman A.B. Joffe A. Christakis D.A. A comparison of reviewers selected by editors and suggested by authors.J Pediatr. 2007; 151: 107-108Abstract Full Text Full Text PDF PubMed Scopus (26) Google Scholar Rivara et al found, in The Archives of Pediatrics and Adolescent Medicine, that 75% of ESRs recommended accept or revise, and 86% of ASRs recommended accept or revise.7Rivara F.P. Cummings P. Ringold S. Bergman A.B. Joffe A. Christakis D.A. A comparison of reviewers selected by editors and suggested by authors.J Pediatr. 2007; 151: 107-108Abstract Full Text Full Text PDF PubMed Scopus (26) Google Scholar In a sample of reviewers from 10 journals, Schroter et al found that ASRs tended to review manuscripts more favorably than ESRs.8Schroter S. Tite L. Hutchings A. Black N. Differences in review quality and recommendations for publication between peer reviews suggested by authors or by editors.JAMA. 2006; 295: 314-317Crossref PubMed Scopus (97) Google Scholar Because the habits of ASRs versus ESRs may affect how editors choose reviewers for manuscripts, we wanted to examine whether the same was true for The Journal by evaluating whether ASRs were more likely than ESRs to provide positive reviews. Additionally, we tested the hypothesis that ASRs are more likely than ESRs to accept invitations to review and concur less frequently with the editor's final decision. The editorial office of The Journal retrospectively evaluated the first 300 manuscripts submitted to The Journal that were assigned consecutive manuscript numbers in one year (2007). At the time of our assessment, all revisions had been received and final decisions made for these articles. Manuscripts that did not undergo external peer-review (ie, Editorials, Insights, and manuscripts that were editorially rejected without peer review) were excluded (n = 122). For the 178 peer-reviewed manuscripts, we recorded the following in Microsoft Excel 2003 software (Microsoft Corporation, Redmond, Washington): (1) whether the invited reviewers were suggested by the author or chosen by the editor; (2) the number of ASRs and ESRs who completed reviews; (3) the initial recommendation of the reviewer ("accept," "accept with revisions," or "reject"); and (4) the final decision of the editor ("accept" or "reject"). For our analyses, reviewer recommendations for "accept" included "accept" and "accept with revisions" recommendations. An independent statistician used the χ2 and the McNemar test for correlated proportions. All computations were done with SAS software version 9.2 (SAS, Cary, North Carolina). Approval of the study protocol was not required according to the institutional review board at Cincinnati Children's Hospital Medical Center. Of the total reviewers examined, 39.4% of invited reviewers (n = 873) accepted the invitation. ASRs accepted 37.2% of invitations (167/449) compared with 41.8% of ESRs (177/424; P = .17). Reviewers who were suggested by classification lists populated by EES or proposed as alternative candidates by reviewers who did not agree to review were excluded because they were chosen by neither the author nor the editor (n = 106). When evaluating manuscripts, 65.3% of ASRs recommended acceptance (109/167), whereas 54.2% of ESRs recommended acceptance (96/177; P = .04). Editors agreed with 49.5% (54/109) of the accept recommendations of ASRs (P < .0001) and with 55.2% (53/96) of ESRs (P < .0001). The evidence that ASRs are more likely than ESRs to accept an invitation to review is not statistically significant. However, ASRs are more likely to recommend acceptance of a submitted manuscript, although editors are less likely than both ASRs and ESRs to recommend acceptance of a manuscript. Regardless of whether a reviewer was suggested by an author or recommended by the editor, all reviewers should strive to provide a fair and balanced review, maintain confidentiality of the manuscript and its findings, and disclose any real or perceived biases. The rigorous assessment by the editors of The Journal appears to offset accept-recommendation advantages of ASRs. Editors weigh reviews on the basis of the expertise of the reviewer and thoroughness of the review, among other factors; the overall balance of the reviews is also considered before making a final decision on a manuscript. The study has limitations. The reviewers' initial recommendations on publication were included in this analysis. However, reviewers may recommend acceptance while giving a low priority for publication or recommend rejection with a high priority. Additionally, manuscripts are judged on the current medical literature and other manuscripts submitted to/published in The Journal. When reports with similar findings or topics have recently been submitted or published, manuscripts may be rejected because of lack of novelty. Although our findings could be caused by a variety of factors, the results support the peer review motto, "Reviewers advise; editors decide."4Balistreri W.F. Landmark, landmine, or landfill? The role of peer review in assessing manuscripts.J Pediatr. 2007; 151: 107-108Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar Because editors serve as the gatekeepers to medical literature, the final responsibility resides with the editors to ensure impartiality in the peer-review process.10Hojat M. Rosenzweig S. Journal peer review in integrative medicine discipline.Semin Integrative Med. 2004; 2: 1-4Crossref Scopus (3) Google Scholar We thank Rebecca Lindeman, BS, for data acquisition, Judy Bean, PhD, for statistical assistance, and the associate editors of The Journal of Pediatrics (Steven H. Abman, M.D., Stephen R. Daniels, MD, PhD, Alan H. Jobe, MD, PhD, Sarah S. Long, MD, Thomas R. Welch, MD, and Robert W. Wilmott, MD) for their valuable input. Finally, we thank the authors, reviewers, and readers of The Journal of Pediatrics for their time, effort, and support.