Portal de Periódicos da CAPES

What’s Important: The Next Academic—ChatGPT AI?

2023; Wolters Kluwer; Volume: 105; Issue: 11 Linguagem: Inglês

10.2106/jbjs.23.00269

1535-1386

Andrew S. Bi,

Radiomics and Machine Learning in Medical Imaging

The use of artificial intelligence (AI) and natural language processing (NLP) tools in academic writing has become increasingly prevalent in recent years. One such tool, ChatGPT, is a large language model trained by OpenAI that can assist in generating text for various purposes including academic writing in orthopedic surgery. With its ability to analyze vast amounts of data and provide accurate information, ChatGPT can assist in literature review, drafting of manuscripts, and even generating research questions. This technology has the potential to improve the efficiency and accuracy of academic writing in orthopedic surgery, ultimately leading to the advancement of the field. You may be surprised to learn that I did not write the above paragraph. It was written by an AI chatbot with a natural language processor trained on the unlimited information from the internet. Could you tell? Chat-based Generative Pre-trained Transformer, or ChatGPT for short, is an AI natural language program launched by OpenAI of San Francisco, California, on November 30, 2022. It was fine-tuned from OpenAI's GPT-3.5 using the established machine-learning concepts of supervised deep learning and reinforcement learning in a process called reinforcement learning with human feedback (RLHF), with training drawing from all publicly available online data prior to 2022 (Fig. 1)1,2. ChatGPT is free and publicly available. Is it here to take our jobs?Fig. 1: ChatGPT's interface and list of disclaimers and limitations.While that may seem like a dramatic question, it can seem overwhelming to stay up-to-date and on the cutting edge as new technologies arise within orthopaedics, from the development of biomaterials to new surgical techniques. AI is no different, as it has already made substantial inroads within orthopaedic surgery, with the ability to create risk-prediction models, interpret advanced imaging, or extract large amounts of data from electronic health records3–6. With ChatGPT, academic writing may be next. Multiple news stories and viral videos have been published about ChatGPT writing final essays for college students or passing final exams at prestigious law and business schools7. Chat GPT-generated scientific abstracts have had varying degrees of success in fooling both human peer reviewers as well as AI-trained detection tools8,9. ChatGPT has already been used to assist in the writing of published case reports and has been tested to pass the United States Medical Licensing Examination (USMLE)10,11. ChatGPT has clearly arrived at the academic dinner table and will likely be here to stay in some form. I conducted a PubMed search for the term "ChatGPT" on March 17, 2023, and the search yielded 138 results: 5 from 2022 and 133 from 2023. The main concern that I see in moving forward with ChatGPT will be the accuracy of content, given the difference between writing scientific manuscripts that may be used to guide evidence-based medical care for real-life human patients versus generating a creative draft for, say, a college admissions essay. ChatGPT is currently available as a free research preview (i.e., beta version) as of its most recent update on February 12, 2023, with OpenAI offering a disclaimer of its limitations on its interactive screen: (1) "May occasionally generate incorrect information," (2) "May occasionally produce harmful instructions or biased content," and (3) "Limited knowledge of world and events after 2021" (Fig. 1)1. Although powerful and very nearly human in its training, it remains only as good as the data it is trained on, with the old adage "garbage in equals garbage out" holding true, similar to machine learning when trained on a flawed or nonapplicable database12. Given that the models were trained on internet data (albeit vast) written by humans, with variable levels of legitimacy and source control of that data, ChatGPT's output should not be taken as factual or even as expert opinion quite yet. However, given that it functions as a deep neural network with RLHF, it can be expected to continue to improve with continued interactions, even demonstrating more acute learning within a single conversation. I gave ChatGPT a simple prompt: "review the literature on anterior cruciate ligament ruptures." It took 32 seconds to produce a 334-word "mini review" on anterior cruciate ligament (ACL) injury (Fig. 2, left panel). Within the same "conversation," I gave ChatGPT a new prompt: "with 1 paragraph, review the literature on ACL reconstruction failures due to graft choice and provide references." It took 17 seconds to produce a 129-word paragraph with 3 references that could easily fit within a discussion section (Fig. 2, right panel). However, when attempting to confirm the citations, I found that all 3 references were inaccurate, clearly generated from authors, titles, and journals in the literature that ChatGPT had found, learned from, and regenerated in an amalgam of faulty references. I felt that I had almost gleaned a glimpse into ChatGPT's "nature"—that rather than simply pulling existing manuscripts as references (which would likely be easier), it appeared to have instead learned from countless citations how to generate its own, fabricated references. In this way, ChatGPT lacked an internal quality control as well as a recognition of its errors.Fig. 2: Examples of ChatGPT-produced writing from increasingly specific prompts.Given ChatGPT's limitations, orthopaedic surgeons' role in research and scientific writing remains secure, for now, and it can be viewed as another tool in our arsenal. With its rise in popularity and the upcoming release of ChatGPT Plus, it seems likely that ChatGPT will play an integral part in the creation of future manuscripts. This will put an increased onus on research teams, including academic orthopaedic surgeons, residents, and medical students, to ensure that the use of such tools is in accordance with ethical publishing standards, as well as on journal editors, reviewers, and copy editors to be vigilant with reference checking, fact checking, and the use of tools to identify AI writing, once available. Programs such as GPTZero, developed by a 22-year-old senior at Princeton, can be trained to identify AI-generated writing, which will likely prove to be essential, as manuscripts produced by ChatGPT will not be flagged by standard plagiarism-detection tools given that these manuscripts are spontaneously generated, novel content13. Rather than viewing AI as the enemy created to make us obsolete, orthopaedic surgeons might consider harnessing this new technology in a symbiotic manner, similarly to other workplaces in real estate, technology, or finance. With unlimited future possible uses, such as learning to generate patient- and problem-specific note templates in the clinic or writing letters to insurance companies to authorize imaging or treatments, ChatGPT could be an invaluable, time-saving tool, perhaps even an "AI partner," to a successful practice and academic orthopaedic surgeon. Its most useful aspect may be just that, as a creative partner to combat the dreaded blank-page syndrome.