Methodological Rigor in Preclinical Cardiovascular Research: Contemporary Performance of AHA Scientific Publications
2021; Lippincott Williams & Wilkins; Volume: 129; Issue: 9 Linguagem: Inglês
10.1161/circresaha.121.319921
ISSN1524-4571
AutoresRichard G. Jung, Cameron Stotts, Dwipen Makwana, Pouya Motazedian, Pietro Di Santo, Cheng-Yee Goh, Louis Verreault‐Julien, Trevor Simard, F. Daniel Ramirez, Benjamin Hibbert,
Tópico(s)Viral Infectious Diseases and Gene Expression in Insects
ResumoHomeCirculation ResearchVol. 129, No. 9Methodological Rigor in Preclinical Cardiovascular Research: Contemporary Performance of AHA Scientific Publications Free AccessLetterPDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyRedditDiggEmail Jump toFree AccessLetterPDF/EPUBMethodological Rigor in Preclinical Cardiovascular Research: Contemporary Performance of AHA Scientific Publications Richard G. Jung, Cameron Stotts, Dwipen Makwana, Pouya Motazedian, Pietro Di Santo, Cheng-Yee Goh, Louis Verreault-Julien, Trevor Simard, F. Daniel Ramirez and Benjamin Hibbert Richard G. JungRichard G. Jung https://orcid.org/0000-0002-8570-6736 CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Vascular Biology and Experimental Medicine Laboratory (R.G.J., P.M., P.D.S., T.S., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Cellular and Molecular Medicine, Faculty of Medicine (R.G.J., T.S., B.H.), University of Ottawa, Ontario, Canada. , Cameron StottsCameron Stotts CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Department of Biochemistry, Microbiology, and Immunology, Faculty of Medicine (C.S.), University of Ottawa, Ontario, Canada. , Dwipen MakwanaDwipen Makwana CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. , Pouya MotazedianPouya Motazedian CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Vascular Biology and Experimental Medicine Laboratory (R.G.J., P.M., P.D.S., T.S., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Cumming School of Medicine, University of Calgary, Alberta, Canada (P.M.). , Pietro Di SantoPietro Di Santo CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Vascular Biology and Experimental Medicine Laboratory (R.G.J., P.M., P.D.S., T.S., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. School of Epidemiology (P.D.S.), University of Ottawa, Ontario, Canada. , Cheng-Yee GohCheng-Yee Goh https://orcid.org/0000-0001-8313-6904 CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. , Louis Verreault-JulienLouis Verreault-Julien https://orcid.org/0000-0001-9081-1832 CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. , Trevor SimardTrevor Simard CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Vascular Biology and Experimental Medicine Laboratory (R.G.J., P.M., P.D.S., T.S., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Cellular and Molecular Medicine, Faculty of Medicine (R.G.J., T.S., B.H.), University of Ottawa, Ontario, Canada. Department of Cardiovascular Diseases, Mayo Clinic School of Medicine, Rochester, MN (T.S.). , F. Daniel RamirezF. Daniel Ramirez Correspondence to: Daniel Ramirez, MD, MSc, University of Ottawa Heart Institute, 40 Ruskin St, H-1285A, Ottawa, Ontario, Canada K1Y 4W7, Email E-mail Address: [email protected] https://orcid.org/0000-0002-4350-1652 CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. and Benjamin HibbertBenjamin Hibbert Benjamin Hibbert, MD, PhD, University of Ottawa Heart Institute, 40 Ruskin St, H-4238, Ottawa, Ontario, Canada K1Y 4W7, Email E-mail Address: [email protected] https://orcid.org/0000-0003-0906-1363 CAPITAL Research Group (R.G.J., C.S., D.M., P.M., P.D.S., C.-Y.G., L.V.-J., T.S., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Vascular Biology and Experimental Medicine Laboratory (R.G.J., P.M., P.D.S., T.S., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Division of Cardiology (P.M., P.D.S., C.-Y.G., L.V.-J., F.D.R., B.H.), University of Ottawa Heart Institute, Ontario, Canada. Cellular and Molecular Medicine, Faculty of Medicine (R.G.J., T.S., B.H.), University of Ottawa, Ontario, Canada. Originally published15 Sep 2021https://doi.org/10.1161/CIRCRESAHA.121.319921Circulation Research. 2021;129:887–889is related toMeet the First AuthorsOther version(s) of this articleYou are viewing the most recent version of this article. Previous versions: September 15, 2021: Ahead of Print Meet the First Author, see p 824Preclinical cardiovascular research using animal models often provides the foundation for clinical research.1 Despite the importance of scientific integrity and reproducibility, methodological sources of bias and suboptimal reporting remain prevalent in preclinical cardiovascular research.2 Essential methodological study design elements (SDEs) include randomization, blinding, sample size or power estimation, and consideration of sex as a biological variable.3 We have previously demonstrated that the introduction of targeted journal initiatives such as that of Stroke was associated with improved reporting of SDEs in published preclinical research.1 In 2017, Circulation Research also introduced a checklist for preclinical studies in animals.4 Given the increased attention to methodological rigor and reproducibility in research, we have updated our analysis to evaluate SDEs in preclinical cardiovascular studies in American Heart Association (AHA) journals over the last 15 years (Circulation, Circulation Research, Stroke, Hypertension, and Arteriosclerosis, Thrombosis, and Vascular Biology [ATVB]).Full articles published between July 2006 and December 2020 reporting original data from in vivo experiments in nonhuman mammals relevant to cardiovascular disease were included, with detailed methods described previously elsewhere.1,2 Data from July 2006 to June 2016 has previously been published with additional data collected from July 2016 to December 2020 in an identical fashion for this analysis.1,2 Briefly, two independent reviewers reviewed each article and extracted SDEs using standardized case report forms. Categorical variables were reported as proportions (%) and compared by χ2 test. The primary outcome of the study was to evaluate changes in SDE reporting since the prior analysis.1,4 The cumulative number of SDEs reported was quantified on a yearly basis from 2006 to 2020 in a journal-specific fashion. All analyses were performed using SAS v9.4 (SAS Institute Inc, Cary, NC) using a 2-tailed α level of 0.05 for statistical significance. The figure was generated using GraphPad Prism 9 (GraphPad Software, La Jolla, CA).A total of 4669 preclinical cardiovascular research articles met inclusion criteria: 948 in Circulation, 806 in Circulation Research, 661 in Stroke, 1233 in ATVB, and 1021 in Hypertension. The majority of studies used mice (68.2%), followed by rats (22.2%), rabbits (2.0%), canines (1.5%), pigs (1.8%), other (1.3%), and combinations of animals (3.0%). The sex of the animals used was not reported in 18.5% of studies. Males were exclusively used in 57.1%, females in 10.1%, and both sexes in 14.3%.Since June 2016, there was a significant increase in the reporting of SDEs in all AHA journals (Figure [A] through [D]), although sample size estimation and the inclusion of both sexes remained low across all the journals. Temporal increases in the cumulative number of reported SDEs over time (2006–2020) were observed in Stroke and Circulation Research—two journals with submission checklists (Figure [E]). This updated analyses of AHA journals identifies improvements in the quality of reporting of SDEs since the 2017 report.3 The biggest gains observed were in Stroke and Circulation Research, which seemed to coincide with their introduction of standardized preclinical checklists.Download figureDownload PowerPointFigure. Reporting of individual study design elements (SDEs) in the 5 American Heart Association journals (ATVB, Circulation, Circulation Research, Hypertension, and Stroke).A, Randomization reported in the 5 journals, with significant changes across all AHA journals. B, Reporting of blinding in the 5 journals, with significant changes across all AHA journals. C, Reporting of sample size (SS) estimation in the 5 journals, with significant changes across all AHA journals. D, Reporting of both sex used in the 5 journals, with significant changes across all AHA journals except in Stroke. E, Each article was scored on a scale of 0 to 4 with randomization, blinding, SS estimation, and both sex used individually contributing 1 point, for a maximum of 4 points. The mean number of SDEs stratified by journals were then reported over time (2006–2020). Red represents Circulation. Blue represents Circulation Research. Green represents Stroke. Orange represents ATVB. Teal represents Hypertension. P<0.05 was considered statistically significant.Our study is not without limitations. We examined reported SDEs, which is influenced by the quality of reporting. The impact of reported SDEs examined on the reproducibility of preclinical research remains to be proven; however, preliminary data and data from the clinical realm suggest that this is to be expected. The SDEs selected in this study are broadly applicable, but their importance may vary depending on the nature of experiments and other SDEs may be more influential (eg, data handling, biological and experimental replicates, etc).Our updated review of preclinical research published in AHA journals demonstrates an overall improvement in the reported use of randomization, blinding, sample size estimation, and inclusion of both sexes in the last 5 years, especially in Stroke and Circulation Research. Despite this improvement, the reporting of SDEs remains poor. Initiatives to improve and standardize the reporting of methods and analyses of preclinical studies, including through the use of well-designed checklists, can have meaningful benefits and should be considered by other AHA journals.Nonstandard Abbreviations and AcronymsAHAAmerican Heart AssociationSDEstudy design elementDisclosures None.Footnotes*R.G. Jung and C. Stotts contributed equally.For Disclosures, see page 889.Correspondence to: Daniel Ramirez, MD, MSc, University of Ottawa Heart Institute, 40 Ruskin St, H-1285A, Ottawa, Ontario, Canada K1Y 4W7, Email [email protected]caBenjamin Hibbert, MD, PhD, University of Ottawa Heart Institute, 40 Ruskin St, H-4238, Ottawa, Ontario, Canada K1Y 4W7, Email [email protected]caReferences1. Ramirez FD, Jung RG, Motazedian P, Perry-Nguyen D, Di Santo P, MacDonald Z, Clancy AA, Labinaz A, Promislow S, Simard T, et al.. Journal initiatives to enhance preclinical research: analyses of stroke, nature medicine, science translational medicine.Stroke. 2020; 51:291–299. doi: 10.1161/STROKEAHA.119.026564LinkGoogle Scholar2. Ramirez FD, Motazedian P, Jung RG, Di Santo P, MacDonald ZD, Moreland R, Simard T, Clancy AA, Russo JJ, Welch VA, et al.. Methodological rigor in preclinical cardiovascular studies: targets to enhance reproducibility and promote research translation.Circ Res. 2017; 120:1916–1926. doi: 10.1161/CIRCRESAHA.117.310628LinkGoogle Scholar3. Ramirez FD, Motazedian P, Jung RG, Di Santo P, MacDonald Z, Simard T, Clancy AA, Russo JJ, Welch V, Wells GA, et al.. Sex bias is increasingly prevalent in preclinical cardiovascular research: implications for translational medicine and health equity for women: a systematic assessment of leading cardiovascular journals over a 10-year period.Circulation. 2017; 135:625–626. doi: 10.1161/CIRCULATIONAHA.116.026668LinkGoogle Scholar4. Bolli R. New initiatives to improve the rigor and reproducibility of articles published in circulation research.Circ Res. 2017; 121:472–479. doi: 10.1161/CIRCRESAHA.117.311678LinkGoogle Scholar Previous Back to top Next FiguresReferencesRelatedDetailsCited ByReeves M, Gall S and Raval A (2021) Hello Authors! We Are the Technical Reviewers and Are Here to Help You!, Stroke, 53:2, (307-310), Online publication date: 1-Feb-2022.Related articlesMeet the First AuthorsCirculation Research. 2021;129:822-824 October 15, 2021Vol 129, Issue 9Article InformationMetrics Download: 230 © 2021 American Heart Association, Inc.https://doi.org/10.1161/CIRCRESAHA.121.319921PMID: 34521221 Originally publishedSeptember 15, 2021 Keywordsanimalshypertensionthrombosissample sizebiasPDF download Advertisement SubjectsBasic Science Research
Referência(s)