Portal de Periódicos da CAPES

Distortion of journal impact factors in the era of paper mills

2023; Elsevier BV; Volume: 31; Issue: 6 Linguagem: Inglês

10.1016/j.ymthe.2023.05.008

1525-0024

Courtney Bricker-Anthony, Roland W. Herzog,

Meta-analysis and systematic reviews

Academia's obsession with the journal impact factor has been a subject of debate for some time. Most would probably agree that it is useful as a crude measure of a journal's prestige, quality, and general influence on a scientific or medical field but should not be overinterpreted. Nonetheless, some institutions go as far as disregarding a student's or faculty member's publications in journals with impact factors less than a certain number (often the magic number is 5) when it comes to performance evaluation, promotion, graduation, or hiring. Such overemphasis ignores that one journal with a lower impact factor may actually have more rigorous standards for acceptance of a paper than another with a higher impact factor. This situation may be observed for a variety of reasons, such as the degree of specialization of a journal or the ratio of review articles vs. original research papers. Another more nefarious contributor to a journal's impact factor, manipulated citations, is also growing and threatening to expose the deepening cracks in the foundation of academia's favorite metric. Initial efforts to manipulate citations were made by citation rings. Citation rings comprise groups of authors who form a pact to cite each other's papers.1Lockwood M. Editorial: citation malpractice.Proc. Math. Phys. Eng. Sci. 2020; 476: 20200746https://doi.org/10.1098/rspa.2020.0746Crossref PubMed Scopus (6) Google Scholar In the simplest version, the authors cite each other's work in individual papers, but it can become far more elaborate. These authors can create multiple fake reviewer accounts, allowing them to pose as various reviewers to favorably review their own papers and advocate for more citations of the ring's work in their critiques. Journal editors can also be involved; in one such case, a journal editor allegedly requested that authors cite many of his own articles, sometimes asking for the addition of more than 50 citations.1Lockwood M. Editorial: citation malpractice.Proc. Math. Phys. Eng. Sci. 2020; 476: 20200746https://doi.org/10.1098/rspa.2020.0746Crossref PubMed Scopus (6) Google Scholar However, at present, the majority of these efforts appear to be coming from paper mills. Paper mills are businesses that manufacture manuscripts on studies that have never been performed, sell authorship on the manuscripts, and submit them to journals on behalf of the authors, often to multiple journals at the same time.2Byrne J.A. Park Y. Richardson R.A.K. Pathmendra P. Sun M. Stoeger T. Protection of the human gene research literature from contract cheating organizations known as research paper mills.Nucleic Acids Res. 2022; 50: 12058-12070https://doi.org/10.1093/nar/gkac1139Crossref PubMed Scopus (4) Google Scholar,3Hvistendahl M. China's publication bazaar.Science. 2013; 342: 1035-1039https://doi.org/10.1126/science.342.6162.1035Crossref PubMed Scopus (163) Google Scholar Once the mill realizes that a journal is likely to accept their fake papers, they increase submissions to this journal and make sure to frequently cite accepted papers in the masses of subsequently manufactured publications. Rogue editors involved with paper mills can also seize control of special issues to publish a flood of fake papers that cite other fake papers.4Brainard J. Fast-growing open-access journals lose impact factors.Science. 2023; 379: 1283-1284https://doi.org/10.1126/science.adi0092Crossref PubMed Google Scholar Hence, fake studies obtain fake recognition in the scientific literature. Except for science and truth, everybody appears to win in this process, at least in the short run: journal impact factors rise, publishers collect substantial publication fees, authors publish "high-impact" papers and get promoted, and institutions celebrate high productivity. It remains to be seen how much damage this will do to the reputation of the scientific literature if this proliferation of false information is not stopped. Recognizing this problem, Clarivate, which owns the Web of Science database, has already begun to stop assigning impact factors to journals that practice rapid mass publications with limited scientific review, thus becoming easier targets for paper mills.4Brainard J. Fast-growing open-access journals lose impact factors.Science. 2023; 379: 1283-1284https://doi.org/10.1126/science.adi0092Crossref PubMed Google Scholar At Molecular Therapy, we have substantially bolstered our defenses over the last 2 years by onboarding dedicated staff and acquiring counteractive technology.5Bricker-Anthony C. Giangrande P.H. On integrity.Mol. Ther. Nucleic Acids. 2022; 30: 595https://doi.org/10.1016/j.omtn.2022.11.011Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar,6Frederickson R.M. Herzog R.W. Keeping them honest: fighting fraud in academic publishing.Mol. Ther. 2021; 29: 889-890https://doi.org/10.1016/j.ymthe.2021.02.011Abstract Full Text Full Text PDF PubMed Scopus (7) Google Scholar,7Frederickson R.M. Herzog R.W. Addressing the big business of fake science.Mol. Ther. 2022; 30: 2390https://doi.org/10.1016/j.ymthe.2022.06.001Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar In this arms race with paper mills, somewhat reminiscent of the Cold War, we are forced to continuously upgrade our arsenal of defenses.7Frederickson R.M. Herzog R.W. Addressing the big business of fake science.Mol. Ther. 2022; 30: 2390https://doi.org/10.1016/j.ymthe.2022.06.001Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar As we continue to learn about paper mills and their operations, more information about their origins has come forth. In a recent preprint,8Cabanac, G., Clausse, A., Jégou, L., and Maisonobe, M. The Geography of Retracted Papers: Showcasing a Crossref–Dimensions–NETSCITY Pipeline for the Spatial Analysis of Bibliographic Data.Google Scholar geographic analysis of Crossref and Dimensions data on 12,437 retracted manuscripts revealed exponential growth of retractions with affiliations in China and India from 2010 to 2020. Retraction Watch published an in-depth investigation of a Russian paper mill that sells authorship on papers already submitted to journals.9Perron B.E. Hiltz-Perron O.T. Victor B.G. Revealed: The Inner Workings of a Paper Mill. Retraction Watch.https://retractionwatch.com/2021/12/20/revealed-the-inner-workings-of-a-paper-mill/Date: 2021Google Scholar Another analysis of scientific articles from the top eight most productive nations showed publications from China accounted for 8% of the world's scientific articles, but also accounted for 24% of all retractions by 2017.10Tang L. Five ways China must cultivate research integrity.Nature. 2019; 575: 589-591https://doi.org/10.1038/d41586-019-03613-1Crossref PubMed Scopus (41) Google Scholar Aligning with our experiences with papermill papers submitted to Molecular Therapy, a cross-sectional analysis of retracted articles attributed to paper mills found nearly all authors were affiliated with Chinese institutions, most of which were hospitals.11Candal-Pedreira C. Ross J.S. Ruano-Ravina A. Egilman D.S. Fernández E. Pérez-Ríos M. Retracted papers originating from paper mills: cross sectional study.BMJ. 2022; 379: e071517https://doi.org/10.1136/bmj-2022-071517Crossref PubMed Scopus (19) Google Scholar In addition to considering the geography of paper mill papers, we need to consider the unique circumstances that likely precipitated their genesis, namely, money, prestige, and career advancement. From the 1990s until February 2020, many research institutions in China had cash-per-publication reward policies that paid authors (usually the first author) for publishing high-impact papers in journals indexed in the Web of Science.12Quan W. Chen B. Shu F. Publish or impoverish: an investigation of the monetary reward system of science in China (1999-2016).Aslib J. Inf. Manag. 2017; 69: 486-502https://doi.org/10.1108/AJIM-01-2017-0014Crossref Scopus (176) Google Scholar,13Mallapaty S. China bans cash rewards for publishing papers.Nature. 2020; 579: 18https://doi.org/10.1038/d41586-020-00574-8Crossref PubMed Scopus (44) Google Scholar These awards typically increased based on Journal Citation Report quartiles in the Web of Science database; thus, papers published in Science and Nature (both Q1 journals) fetched upward of $165,000 USD.12Quan W. Chen B. Shu F. Publish or impoverish: an investigation of the monetary reward system of science in China (1999-2016).Aslib J. Inf. Manag. 2017; 69: 486-502https://doi.org/10.1108/AJIM-01-2017-0014Crossref Scopus (176) Google Scholar The intent of these policies was to incentivize researchers to publish more high-impact research and propel Chinese research onto the international stage. While the number of papers published by researchers in China skyrocketed during this period,12Quan W. Chen B. Shu F. Publish or impoverish: an investigation of the monetary reward system of science in China (1999-2016).Aslib J. Inf. Manag. 2017; 69: 486-502https://doi.org/10.1108/AJIM-01-2017-0014Crossref Scopus (176) Google Scholar these policies also had unintended consequences. These incentives can encourage behaviors such as the submission of salami papers (multiple papers derived from one study),14Supak Smolcić V. Salami publication: definitions and examples.Biochem. Med. 2013; 23: 237-241https://doi.org/10.11613/BM.2013.030Crossref Scopus (59) Google Scholar also known as the "least publishable unit."15Broad W.J. The publishing game: getting more for less: meet the least publishable unit, one way of squeezing more papers out of a research project.Science. 1981; 211: 1137-1139https://doi.org/10.1126/science.7008199Crossref PubMed Scopus (208) Google Scholar Paper mills also exploited these policies to drive their business, with the ability to publish papers in Web of Science-indexed journals serving as a major selling point.3Hvistendahl M. China's publication bazaar.Science. 2013; 342: 1035-1039https://doi.org/10.1126/science.342.6162.1035Crossref PubMed Scopus (163) Google Scholar,16Liu X. Chen X. Journal retractions: some unique features of research misconduct in China.J. Sch. Publ. 2018; 49: 305-319https://doi.org/10.3138/jsp.49.3.02Crossref Scopus (13) Google Scholar Although these policies have ended, the pressure to publish in high-impact journals from institutions and funders worldwide has not waned; thus, paper mills have no reason to slow down. As we identify and reject paper mill manuscripts, we often see them published in other journals. Some journals appear to be preferred targets for these organizations, perhaps reflecting fewer capabilities to recognize such submissions. While we sympathize, given that our own journals were hit by paper mills before we had adequate countermeasures in place and understanding that no review system is perfect, we nonetheless appeal to journals and publishers to take this problem seriously by bolstering pre-screening processes and finding ways to collaborate and share information. We also desperately need a coordinated effort by publishers to dry out the flow of fabricated manuscripts. No short-term revenue considerations are worth eroding the trust in scientific publication. We also need to consider that these relentless paper mill activities add substantial confusion to entire areas of research. In agreement with Byrne and colleagues' 2019 study,17Byrne J.A. Grima N. Capes-Davis A. Labbé C. The possibility of systematic research fraud targeting under-studied human genes: causes, consequences, and potential solutions.Biomark. Insights. 2019; 14 (1177271919829162)https://doi.org/10.1177/1177271919829162Crossref PubMed Scopus (20) Google Scholar we find that studies targeting genes for knockdown or modulation, typically using miRNAs, lncRNAs, and circRNAs, are favorite topics among paper mills. Regulation of gene expression and macromolecular/cellular functions by non-coding RNA represents an immensely important topic in biomedical research. However, the injection of large numbers of fake papers complicates the differentiation between real and invented findings, thereby impeding meaningful progress in the field. The molecular biology of cancer and orthopedics/inflammation research are also among paper mill favorites, likely reflecting the size of the clientele seeking to publish in these areas. At present, the full scope of the influence of paper mills on the impact factor and the literature as a whole is still coming into focus. In a recent preprint, an analysis of papers published in high-impact cancer journals that contained incorrect nucleotide sequences, a potential indicator of paper mill involvement, revealed many of these articles are highly cited, with 92 articles from one journal fetching a whopping 8,048 citations.18Pathmendra P. Park Y. Enguita F.J. Byrne J.A. Verification of nucleotide sequence reagent identities in original publications in high impact factor cancer research journals.Cancer Biol. 2023; https://doi.org/10.1101/2023.02.03.526922Crossref Scopus (0) Google Scholar The impact factor is supposed to reflect general interest in papers published by a journal, not reward mass production of fake papers. It will take a strong coordinated effort by the scientific community, publishers, and institutions to make the system work again the way it was intended, rewarding exciting research that is anchored in the principles of scientific integrity.