What Should Radiology Residency and Fellowship Training in Artificial Intelligence Include? A Trainee’s Perspective— Radiology In Training
2021; Radiological Society of North America; Volume: 299; Issue: 2 Linguagem: Inglês
10.1148/radiol.2021204406
ISSN1527-1315
AutoresAli S. Tejani, Julia R. Fielding, Ronald M. Peshock,
Tópico(s)Radiology practices and education
ResumoHomeRadiologyVol. 299, No. 2 PreviousNext Reviews and CommentaryFree AccessPerspectivesWhat Should Radiology Residency and Fellowship Training in Artificial Intelligence Include? A Trainee’s Perspective—Radiology In TrainingAli S. Tejani , Julia R. Fielding, Ronald M. PeshockAli S. Tejani , Julia R. Fielding, Ronald M. PeshockAuthor AffiliationsFrom the Department of Radiology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75235.Address correspondence to A.S.T. (e-mail: [email protected]).Ali S. Tejani Julia R. FieldingRonald M. PeshockPublished Online:Mar 9 2021https://doi.org/10.1148/radiol.2021204406MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked In AbstractSummaryA standardized, holistic artificial intelligence (AI) curriculum is necessary to meet trainee demand and prepare them for effective AI tool use in their future practice.Dr Ali S. Tejani is an incoming 1st year resident in the Department of Radiology at the University of Texas Southwestern Medical Center. He is passionate about trainee education and hopes to expand access to artificial intelligence and imaging informatics resources.Download as PowerPointOpen in Image Viewer Artificial intelligence (AI) and machine learning (ML) have captured the imagination of researchers and clinicians alike. Of note, “AI” is a broad designation encompassing any technique that enhances the ability of machines to mimic human behavior, while “ML” represents a subset of AI techniques that enable models to improve performance with increasing exposure to data (1). AI has drawn the attention of trainees who prepare to join academic and private practices already implementing AI tools. Although programs have started to introduce technical ML concepts locally (ML theory, data curation, model development, computational methods), a standardized, holistic AI education beyond algorithm-focused lectures is lacking (2,3). Specifically, there is a need for a curriculum that introduces all trainees to factors crucial for clinical integration of AI tools; this will prepare tool deployers and users in addition to tool creators. The absence of a standardized curriculum leaves trainees to navigate AI technology without structure.Acknowledging commentary highlighting the need for an AI curriculum (4), we propose a framework that addresses the basics of ML, AI tool application to common clinical questions, and the regulatory, ethical, and economic implications on clinical practice. This curriculum would be offered in tandem with, or following, residency training either through dedicated rotations or a supplemental scholarly track. Regular content updates from local or national committees will ensure trainees are educated on clinically relevant paradigms and standards in this rapidly advancing arena.Meeting Trainee DemandOne of the primary requirements of a successful AI curriculum is sufficient trainee demand to justify investment of time and resources from residency programs. A 2019 multiprogram survey in Singapore (5) and a national survey in France (6) demonstrated that a majority of resident and faculty respondents desire further AI education. However, 74 of the 125 respondents in Singapore (59.2%) and 235 of the 270 respondents in France (87%) indicated that they believed instruction to be inadequate (5,6). More recently, 72% of respondents of the 2019 Spring Survey of the Association of Program Directors in Radiology indicated significant interest in imaging informatics education, a topic intimately related with clinical adoption of AI tools (7). However, 33% of respondents indicated absence of such a curriculum (7). These results underscore a consistent gap between trainee demand and program provision of AI and/or informatics education.Proposed ComponentsAll trainees should receive introductory lectures on the following content areas for a robust foundation on technical ML knowledge and clinical AI integration. However, advanced training should be encouraged for trainees with an interest beyond preliminary didactics.Defining Introductory TerminologyDefinition of common terms will ensure that trainees understand and use standard language. In addition to defining AI and ML, distinction should be made among different types of commonly employed ML techniques (supervised, unsupervised, reinforced, and semisupervised learning). Lecturers should cover federated learning techniques that allow for algorithm training across multi-institutional data sets, addressing issues of limited data set size associated with single institution–based models while maintaining data privacy (8).Basics of MLTrainees should be introduced to the basic concepts of data collection, annotation, and algorithm validation. AI tool integration will require all radiologists to function as guardians of accurately annotated, publicly accessible, and large data sets. Accordingly, instruction should include discourse on determining a sufficient caseload and the decision to validate AI performance on shared data sets that have been vetted by a reliable source. Trainees should receive a primer on addressing shortcomings of AI research, including validating AI apps to produce generalizable algorithms, maintaining transparency in algorithm training and/or testing methods, and obtaining a sufficient sample size with appropriate use of data augmentation techniques. Furthermore, trainees should understand the risks of selection bias, which can discriminate against underrepresented demographics in a given data set, and negative set bias, which may create imbalanced training data sets overrepresenting studies with “positive” imaging findings (9). Finally, didactics should cover performance metrics used to critically assess and compare AI tools, including receiver operator characteristic curves, areas under the curve values, and decision heatmaps.National RegulationEducation about algorithm regulation is arguably as important as algorithm creation, as new AI apps will require approval by the U.S. Food and Drug Administration before widespread clinical adoption. However, this content should concentrate on relevant topics for all trainees, specifically the definition of “Food and Drug Administration approval,” examples of successful versus rejected AI apps, and total product lifecycle–based regulatory frameworks to account for rapid algorithm updates (10). Trainees should have the opportunity to discuss the role of regulating bodies in ensuring patient safety, such as storage of data with protection against nonconsensual vendor use and maintaining thresholds of rigorous algorithm testing before clinical use. Further education regarding strategic options to navigate both domestic and foreign processes should be reserved for trainees or physician innovators actively involved in device approval.AI EthicsInherent biases and disparity in access to AI make this technology prone to misuse that may ultimately harm patients. The rapid development of AI necessitates mindfulness in considering the potential implications on privacy, data protection, patient representation in training and testing sets, and transparency in the details of algorithm development (9). Further discussion should prompt trainees to think about the role of AI in either informing physicians through a “second read” or generating unsupervised reports prone to false-positive or false-negative findings. Accordingly, this presents an opportunity to train future radiologists to actively combat automation bias by prioritizing the entire clinical picture, including evidence contradicting AI decisions. In addition, this topic should encourage discourse regarding the distribution of responsibility for AI-related patient harm. Radiologists will serve as stewards of responsible AI use in clinical practice, and preparation for this role requires thorough discussion of the pitfalls of AI technology to ensure responsible application to patient care.Economic Implications of AI on Business StrategyTrainees overseeing the clinical adoption of AI must understand the economic implications to justify practice investment. Successful AI implementation requires local champions who can cite tangible financial returns to earn leadership approval. Examples of citable returns on investment include flagging urgent findings to reduce image turnaround time, reducing examination times, automating structure and/or lesion segmentation, improving diagnostic accuracy (“second read”), and optimizing staffing based on predicted examination volume. Indirect returns offered by solutions include decreased patient safety events, reduced malpractice risk, and improved workflow efficiency.In addition, trainees must understand purchasing factors affecting the integration of apps into practice workflow. Topics of consideration include annual fixed versus per-case fee models versus packaging AI apps with vendor equipment. Additional discussion should emphasize the need to validate tools on enterprise-specific data before purchase to critically evaluate metrics posed by tool vendors. This content will prepare trainees to critically evaluate AI apps and demonstrate their value when justifying allocation of funds from the yearly budget for investment in AI tools. Finally, discussion of economic implications would be remiss without reviewing current and future Centers for Medicare and Medicaid Services reimbursement plans for AI apps.InformaticsA detailed curriculum in imaging informatics holds value of its own and is beyond the scope of this article. However, trainees will benefit from instruction on basic informatics terminology and health information systems that prepares them to address the integration of AI within clinical workflow. Trainees should discuss considerations in accommodating AI apps, such as the decision to host AI apps on specific equipment, directly integrate AI tools on enterprise-shared picture archiving and communication systems, apply algorithms accessed by means of a web browser, or load individual Digital Imaging and Communications in Medicine images into the institutional picture archiving and communication system after the application of AI tools. Fortunately, resources from national organizations, such as the National Imaging Informatics Curriculum and Course, provide supplemental material to introduce key informatics concepts to account for local limitations in informatics expertise.Scholarly ProjectsMotivated trainees will identify new applications of AI to address existing clinical or workflow inadequacies during their education. Accordingly, research projects provide an opportunity for learners to contribute to innovative solutions for real-world problems. However, projects should be tailored to trainee level of expertise in data science and ML, offering opportunities in all areas of imaging AI. These areas may include creating training data sets, developing algorithms, validating tools in clinical practice, integrating tools into clinical workflow, and assessing the impact of tool use through evaluation of clinical outcomes. Troubleshooting through algorithm development and the factors affecting implementation will allow trainees to grow into consulting roles for future app development. Furthermore, app development will teach trainees to examine future AI tools through a critical lens, scrutinizing shortcomings and projected benefits to workflow or clinical outcomes. Mentors will be key to guide trainees and advocate on their behalf; however, access to local mentors will vary across institutions. Navigating this limitation may require assistance from colleagues at other institutions who are willing to mentor trainees through virtual means. Another option may require adjunct support from the institution’s engineering department or research faculty well-versed in imaging AI, which could facilitate joint, multidisciplinary initiatives.In conclusion, it is clear that an increasing number of trainees are beginning to embrace a future of working with AI apps; however, attempting to learn about AI without appropriate guidance can prove to be an arduous task without a clear starting point. A structured, standardized AI curriculum is warranted to equip trainees with skills for the creation, regulation, and implementation of ML algorithms. Future work should examine barriers to curriculum implementation, such as obtaining program support and addressing unequal distribution of AI expertise and resources across training programs.Disclosures of Conflicts of Interest: A.S.T. disclosed no relevant relationships. J.R.F. disclosed no relevant relationships. R.M.P. disclosed no relevant relationships.References1. European Society of Radiology (ESR). What the radiologist should know about artificial intelligence - an ESR white paper. Insights Imaging 2019;10(1):44. Crossref, Medline, Google Scholar2. Lindqwister AL, Hassanpour S, Lewis PJ, Sin JM. AI-RADS: an artificial intelligence curriculum for residents. Acad Radiol 2020. https://doi.org/10.1016/j.acra.2020.09.017. Published online October 15, 2020. Crossref, Medline, Google Scholar3. Wiggins WF, Caton MT, Magudia K, . Preparing radiologists to lead in the era of artificial intelligence: designing and implementing a focused data science pathway for senior radiology residents. Radiol Artif Intell 2020;2(6):e200057. Link, Google Scholar4. Wood MJ, Tenenholtz NA, Geis JR, Michalski MH, Andriole KP. The need for a machine learning curriculum for radiologists. J Am Coll Radiol 2019;16(5):740–742. Crossref, Medline, Google Scholar5. Ooi SKG, Makmur A, Soon AYQ, . Attitudes toward artificial intelligence in radiology with learner needs assessment within radiology residency programmes: a national multi-programme survey. Singapore Med J 2019. https://doi.org/10.11622/smedj.2019141. Published online November 4, 2019. Google Scholar6. Waymel Q, Badr S, Demondion X, Cotten A, Jacques T. Impact of the rise of artificial intelligence in radiology: What do radiologists think?. Diagn Interv Imaging 2019;100(6):327–336. Crossref, Medline, Google Scholar7. Rozenshtein A, Griffith BD, Slanetz PJ, . “What program directors think” V: results of the 2019 spring survey of the association of program directors in radiology (APDR). Acad Radiol 2020. https://doi.org/10.1016/j.acra.2020.06.035. Published online August 7, 2020. Crossref, Google Scholar8. Sheller MJ, Edwards B, Reina GA, . Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci Rep 2020;10(1):12598. Crossref, Medline, Google Scholar9. Geis JR, Brady AP, Wu CC, . Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Radiology 2019;293(2):436–440. Link, Google Scholar10. Kohli A, Mahajan V, Seals K, Kohli A, Jha S. Concepts in U.S. food and drug administration regulation of artificial intelligence for medical imaging. AJR Am J Roentgenol 2019;213(4):886–888. Crossref, Medline, Google ScholarArticle HistoryReceived: Nov 24 2020Revision requested: Dec 07 2020Revision received: Dec 23 2020Accepted: Jan 05 2021Published online: Mar 09 2021Published in print: May 2021 FiguresReferencesRelatedDetailsCited ByArtificial Intelligence and Radiology EducationAli S. Tejani, Hesham Elhalawani, Linda Moy, Marc Kohli, Charles E. Kahn, Jr, 16 November 2022 | Radiology: Artificial Intelligence, Vol. 5, No. 1Teaching Artificial Intelligence Literacy: A Challenge in the Education of Radiology ResidentsClaudiaMello-Thoms2023 | Academic Radiology, Vol. 30, No. 7Artificial Intelligence Literacy: Developing a Multi-institutional Infrastructure for AI EducationJ.D.Perchik, A.D.Smith, A.A.Elkassem, J.M.Park, S.A.Rothenberg, M.Tanwar, P.H.Yi, A.Sturdivant, S.Tridandapani, H.Sotoudeh2023 | Academic Radiology, Vol. 30, No. 7Artificial Intelligence/Machine Learning Education in Radiology: Multi-institutional Survey of Radiology Residents in the United StatesNinad V.Salastekar, CharlesMaxfield, Tarek N.Hanna, Elizabeth A.Krupinski, DarelHeitkamp, Lars J.Grimm2023 | Academic Radiology, Vol. 30, No. 7Development and validation of an educational software based in artificial neural networks for training in radiology (JORCAD) through an interactive learning activityJorgeHernández-Rodríguez, María-JoséRodríguez-Conde, José-ÁngelSantos-Sánchez, Francisco-JavierCabrero-Fraile2023 | Heliyon, Vol. 9, No. 4Artificial Intelligence in Radiology Education: A Longitudinal ApproachVrushabGowda, Sheryl GillikinJordan, Omer AAwan2022 | Academic Radiology, Vol. 29, No. 5Artificial intelligence-based decision support system (AI-DSS) implementation in radiology residency: Introducing residents to AI in the clinical settingTinaShiang, ElisabethGarwood, Carolynn M.Debenedectis2022 | Clinical Imaging, Vol. 92Perceptions of US Medical Students on Artificial Intelligence in Medicine: Mixed Methods Survey StudyDavid ShalomLiu, JakeSawyer, AlexanderLuna, JihadAoun, JanetWang, LordBoachie, SafwanHalabi, BinaJoe2022 | JMIR Medical Education, Vol. 8, No. 4Accompanying This ArticleAuthor Interview: What Should Radiology Residency and Fellowship Training in Artificial Intelligence Include? A Trainee’s PerspectiveApr 11 2023Default Digital Object SeriesRecommended Articles ChatGPT and Other Large Language Models Are Double-edged SwordsRadiology2023Volume: 307Issue: 2Artificial Intelligence in Low- and Middle-Income Countries: Innovating Global Health RadiologyRadiology2020Volume: 297Issue: 3pp. 513-520The Economic Impact of the COVID-19 Pandemic on Radiology PracticesRadiology2020Volume: 296Issue: 3pp. E141-E144Radiology Report Value EquationRadioGraphics2018Volume: 38Issue: 6pp. 1888-1896Radiology Errors across the Diurnal CycleRadiology2020Volume: 297Issue: 2pp. 380-381See More RSNA Education Exhibits Anatomy of a Deep Learning Project for Breast Cancer Prognosis Prediction: From Collecting Data to Building a PipelineDigital Posters2019A Grassroots Approach To Forming A Diversity, Equity, And Inclusion Committee In An Academic Radiology Department: Early Successes And Lessons LearnedDigital Posters2021Optimizing the Mentor-Mentee Relationship: A Guide for MenteesDigital Posters2019 RSNA Case Collection Traumatic open globe rupture RSNA Case Collection2021NeurocysticercosisRSNA Case Collection2022Persistent fetal vasculature (PFV)RSNA Case Collection2020 Vol. 299, No. 2 Video SummaryAbbreviations Abbreviations: AI artificial intelligence ML machine learning Metrics Altmetric Score PDF download
Referência(s)