Portal de Periódicos da CAPES

Impact of Blinded Versus Unblinded Abstract Review on Scientific Program Content

2002; Lippincott Williams & Wilkins; Volume: 168; Issue: 5 Linguagem: Inglês

10.1016/s0022-5347(05)64315-7

1527-3792

Joseph A. Smith, RANDALL NIXON, Anton J. Bueschen, Dennis D. Venable, Hector H. Henry,

Academic Writing and Publishing

No AccessJournal of UrologyCLINICAL UROLOGY: Special Communication1 Nov 2002Impact of Blinded Versus Unblinded Abstract Review on Scientific Program Content JOSEPH A. SMITH, RANDALL NIXON, ANTON J. BUESCHEN, DENNIS D. VENABLE, and HECTOR H. HENRY JOSEPH A. SMITHJOSEPH A. SMITH Requests for reprints: Department of Urologic Surgery, Vanderbilt University Medical Center, A-1302 Medical Center North, Nashville, Tennessee 38232-8990. More articles by this author , RANDALL NIXONRANDALL NIXON More articles by this author , ANTON J. BUESCHENANTON J. BUESCHEN More articles by this author , DENNIS D. VENABLEDENNIS D. VENABLE More articles by this author , and HECTOR H. HENRYHECTOR H. HENRY More articles by this author View All Author Informationhttps://doi.org/10.1016/S0022-5347(05)64315-7AboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract Purpose: A procedure whereby reviewers are not informed of the author or institutional identity for submitted abstracts is sometimes considered a more equitable and impartial process for selection of the content for a scientific program. We performed a prospective randomized study to evaluate the impact of a reviewer blinding process on scientific program content. Materials and Methods: A total of 234 abstracts submitted for presentation at the 2001 meeting of the Southeastern Section of the American Urological Association were distributed for review and grading to 42 reviewers who were randomly assigned to either a blinded or unblinded category. Acceptance for presentation was based on combined raw scores for blinded and unblinded reviews. However, multiple statistical comparisons were performed to evaluate the program content if only the blinded or unblinded reviews had been used as criteria for program acceptance. The abstracts were divided into pediatrics; prostate cancer, bladder cancer and urinary diversion, female urology and urodynamics, endourology and laparoscopy, benign and malignant diseases of the kidney, and penis/erectile dysfunction and miscellaneous for review. Results: Statistically significant differences were observed in the variability of scores for blinded versus unblinded reviewers in the bladder cancer and prostate cancer groups but not in the other categories. For the pediatrics and urodynamics/female urology groups unblinded reviewers had statistically significant higher mean scores than blinded reviewers but this was not observed in other categories. Overall there was no clear pattern showing greater or less variability between reviewers for blinded versus unblinded reviews, nor was there a consistently observed difference in mean raw scores. Of the papers included in the program 60% would have been accepted for presentation by either blinded or unblinded review alone. This figure increased to 80% when considering the papers which were most highly scored by reviewers in each category. Conclusions: Although this study cannot be used to determine the quality of reviews based on blinding versus unblinding, it does not demonstrate any consistent increase in variability between reviewers whether they have been informed of abstract authors and institution. Furthermore, there was no identifiable tendency for blinded or unblinded reviewers to assign higher or lower raw scores to abstracts. However, blinded versus nonblinded review may have a substantial impact on program content. Only 60% of the presented abstracts would have been chosen by either method used alone. Additionally, 20% of the 5 abstracts in each category graded most highly by 1 review process would not have been accepted for inclusion on the program by the other process. These data may not be applicable to other circumstances, but the blinding process for abstract review may impact substantially on program content for section meetings. References 1 : What do peer reviewers do?. JAMA1990; 263: 1341. Crossref, Medline, Google Scholar 2 : Statistics notes: blinding in clinical trials and other studies. BMJ2000; 321: 504. Crossref, Medline, Google Scholar 3 : The effects of blinding on acceptance of research papers by peer review. JAMA1994; 272: 143. Crossref, Medline, Google Scholar 4 : Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA1998; 280: 237. Crossref, Medline, Google Scholar 5 : Multiple blinded reviews of the same two manuscripts. Effects of referee characteristics and publication language. JAMA1994; 272: 149. Crossref, Medline, Google Scholar From the Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, University of Alabama, Birmingham, Alabama, Louisiana State University Medical Center, Shreveport, Louisiana, and Cabarrus Medical Clinic, Concord, North Carolina, and Southeastern Section, American Urological Association© 2002 by American Urological Association, Inc.FiguresReferencesRelatedDetailsCited by Buchsbaum M (2019) Forty years of editorship of psychiatry researchPsychiatry Research, 10.1016/j.psychres.2019.01.071, VOL. 277, (4-9), Online publication date: 1-Jul-2019. Johnson D and Hermanowicz J (2017) Peer Review: From “Sacred Ideals” to “Profane Realities” Higher Education: Handbook of Theory and Research, 10.1007/978-3-319-48983-4_10, (485-527), . Somford M, van Deurzen D, Ostendorf M, Eygendaal D and van den Bekerom M (2015) Quality of research and quality of reporting in elbow surgery trialsJournal of Shoulder and Elbow Surgery, 10.1016/j.jse.2015.03.031, VOL. 24, NO. 10, (1619-1626), Online publication date: 1-Oct-2015. Wendler D and Miller F (2013) The ethics of peer review in bioethicsJournal of Medical Ethics, 10.1136/medethics-2013-101364, VOL. 40, NO. 10, (697-701), Online publication date: 1-Oct-2014. Lee C, Sugimoto C, Zhang G and Cronin B (2012) Bias in peer reviewJournal of the American Society for Information Science and Technology, 10.1002/asi.22784, VOL. 64, NO. 1, (2-17), Online publication date: 1-Jan-2013. (2009) Letters to the EditorBritish Journal of Occupational Therapy, 10.1177/030802260907200410, VOL. 72, NO. 4, (183-185), Online publication date: 1-Apr-2009. Lannin N, Gustafsson L, Cusick A, Walker M, Steultjens E, Fricke J, Turton A, Wågø Aas R, Logan P, Froude E, McCluskey A, Drummond A, Corr S, Fletcher-Smith J, Radford K, Pinnington L, Novak I and Wallen M (2009) Scholarly communication and concerns for our conferencesAustralian Occupational Therapy Journal, 10.1111/j.1440-1630.2009.00786.x, VOL. 56, NO. 2, (147-148), Online publication date: 1-Apr-2009. Baggs J, Broome M, Dougherty M, Freda M and Kearney M (2008) Blinding in peer review: the preferences of reviewers for nursing journalsJournal of Advanced Nursing, 10.1111/j.1365-2648.2008.04816.x, VOL. 64, NO. 2, (131-138), Online publication date: 1-Oct-2008. Hopewell S, Eisinga A and Clarke M (2007) Better reporting of randomized trials in biomedical journal and conference abstractsJournal of Information Science, 10.1177/0165551507080415, VOL. 34, NO. 2, (162-173), Online publication date: 1-Apr-2008. Salzl G, Gölder S, Timmer A, Marienhagen J, Schölmerich J and Grossmann J (2008) Poster Exhibitions at National ConferencesDeutsches Ärzteblatt international, 10.3238/arztebl.2008.0078 Cobo E, Selva-O'Callagham A, Ribera J, Cardellach F, Dominguez R, Vilardell M and Scherer R (2007) Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized TrialPLoS ONE, 10.1371/journal.pone.0000332, VOL. 2, NO. 3, (e332) Smith W, Cancel Q, Tseng T, Sultan S, Vieweg J and Dahm P (2018) Factors Associated With the Full Publication of Studies Presented in Abstract Form at the Annual Meeting of the American Urological AssociationJournal of Urology, VOL. 177, NO. 3, (1084-1089), Online publication date: 1-Mar-2007. Gleicher N (2007) A difficult decisionJournal of Assisted Reproduction and Genetics, 10.1007/s10815-006-9104-7, VOL. 24, NO. 1, (3-4), Online publication date: 2-Feb-2007. Poolman R, Keijser L, De Waal Malefijt M, Blankevoort L, Farrokhyar F, Bhandari M and On behalf of the Dutch Orthopedic A (2009) Reviewer agreement in scoring 419 abstracts for scientific orthopedics meetingsActa Orthopaedica, 10.1080/17453670710013807, VOL. 78, NO. 2, (278-284), Online publication date: 1-Jan-2007. Steers W (2018) Blind FaithJournal of Urology, VOL. 176, NO. 5, (1905-1906), Online publication date: 1-Nov-2006. Snodgrass R (2006) Single- versus double-blind reviewingACM SIGMOD Record, 10.1145/1168092.1168094, VOL. 35, NO. 3, (8-21), Online publication date: 1-Sep-2006. Bydder S, Marion K, Taylor M and Semmens J (2006) Assessment of abstracts submitted to the annual scientific meeting of the Royal Australian and New Zealand College of RadiologistsAustralasian Radiology, 10.1111/j.1440-1673.2006.01599.x, VOL. 50, NO. 4, (355-359), Online publication date: 1-Aug-2006. Cohen I and Patel K (2006) Peer Review Interrater Concordance of Scientific Abstracts: A Study of Anesthesiology Subspecialty and Component SocietiesAnesthesia & Analgesia, 10.1213/01.ane.0000200314.73035.4d, VOL. 102, NO. 5, (1501-1503), Online publication date: 1-May-2006. Burns K, Adhikari N, Kho M, Meade M, Patel R, Sinuff T and Cook D (2005) Abstract reporting in randomized clinical trials of acute lung injury: An audit and assessment of a quality of reporting score*Critical Care Medicine, 10.1097/01.CCM.0000178361.73895.24, VOL. 33, NO. 9, (1937-1945), Online publication date: 1-Sep-2005. Bhandari M, Templeman D and Tornetta P (2004) Interrater Reliability in Grading Abstracts for the Orthopaedic Trauma AssociationClinical Orthopaedics & Related Research, 10.1097/01.blo.0000127584.02606.00, VOL. 423, (217-221), Online publication date: 1-Jun-2004. Volume 168 Issue 5 November 2002 Page: 2123-2125 Advertisement Copyright & Permissions© 2002 by American Urological Association, Inc.Keywordspeer reviewprogram developmentMetrics Author Information JOSEPH A. SMITH Requests for reprints: Department of Urologic Surgery, Vanderbilt University Medical Center, A-1302 Medical Center North, Nashville, Tennessee 38232-8990. More articles by this author RANDALL NIXON More articles by this author ANTON J. BUESCHEN More articles by this author DENNIS D. VENABLE More articles by this author HECTOR H. HENRY More articles by this author Expand All Advertisement PDF downloadLoading ...