Portal de Periódicos da CAPES

The Potential of Artificial Intelligence to Analyze Chest Radiographs for Signs of COVID-19 Pneumonia

2020; Radiological Society of North America; Volume: 299; Issue: 1 Linguagem: Inglês

10.1148/radiol.2020204238

1527-1315

Bram van Ginneken,

AI in cancer detection

HomeRadiologyVol. 299, No. 1 PreviousNext Reviews and CommentaryFree AccessEditorialThe Potential of Artificial Intelligence to Analyze Chest Radiographs for Signs of COVID-19 PneumoniaBram van Ginneken Bram van Ginneken Author AffiliationsFrom the Department of Radiology, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands.Address correspondence to the author (e-mail: [email protected]).Bram van Ginneken Published Online:Nov 24 2020https://doi.org/10.1148/radiol.2020204238See related articleMoreSectionsPDF ToolsAdd to favoritesCiteTrack Citations ShareShare onFacebookXLinked In See also the article by Wehbe et al in this issue.Bram van Ginneken is professor of medical image analysis at Radboud University Medical Center. He also works for Fraunhofer MEVIS in Bremen, Germany, and is a founder of Thirona, a company that develops software and provides services for medical image analysis. He studied physics at Eindhoven University of Technology and at Utrecht University, where he obtained his doctorate in 2001 on computer-aided diagnosis in chest radiography. He pioneered the concept of challenges in medical image analysis.Download as PowerPointIt was about 1 year ago that a new coronavirus started to spread from Wuhan, China. The resulting pandemic is unprecedented in many ways, and one of them is the number of scientific publications it has generated. PubMed already lists over 70 000 papers on coronavirus disease 2019 (COVID-19). The first publication in Radiology, describing the CT appearance of COVID-19 pneumonia findings, dates from February 6, 2020. To date, Radiology has published 40 original research articles on this topic. These studies have had a substantial impact: the 12 most cited papers in Radiology from 2020 (using counts from Google Scholar) are all on COVID-19, and even the article ranked 12th has twice as many citations as the most cited Radiology article from 2019. (Interestingly, the 12 most-cited 2019 Radiology publications are all on applications of artificial intelligence [AI].)Of the Radiology COVID-19 articles published so far, 23 have focused on CT, and only six have focused on chest radiography. This is likely related to the fact that, as noted by the Fleischner Society (1) in their consensus statement on the role of imaging in patient care during the pandemic, "chest radiography is insensitive in mild or early COVID-19." This conclusion was based on evidence from the first article on chest radiographic findings in patients with COVID-19 (2). However, many countries encourage individuals with symptoms consistent with COVID-19 to quarantine at home. In such a scenario, patients presenting in a hospital may have more advanced disease, often with abnormalities visible on a chest radiograph. Because of its broad availability, low cost, and portability, chest radiography is a widely used tool to obtain an initial diagnosis while waiting for the results from molecular diagnostic tests. Radiographic imaging can also help assess disease progression or help detect other diseases.Many health care providers are overburdened during this pandemic and struggle with a lack of resources for image interpretation. AI could provide support in the reading process.In this issue of Radiology, Wehbe et al (3) present an AI algorithm, coined DeepCOVID-XR, that detects COVID-19 on single frontal chest radiographs. It is not the first article in Radiology to attempt to do this. In May 2020, Murphy et al (4) reported on a validation study of CAD4COVID–x-ray, a freely available CE-marked commercial solution; I was a coauthor of that study. In September 2020, Zhang et al (5) introduced CV19-Net. All three algorithms address the same task.A strength of DeepCOVID-XR is that it was trained on a large multicenter data set: Almost 15 000 images with over 4000 cases that were positive for COVID-19 originating from more than 20 sites across the Northwestern Memorial Health Care System, an organization operating in the Chicago, Ill, region. Nearly all images were anteroposterior bedside radiographs. The AI system was evaluated on data from one community hospital. This was a proper external validation set: No images from that hospital had been used to train the system. A set of 300 random cases (134 positive for COVID-19) were presented to five radiologists to allow a direct comparison between the AI system and human experts.A similar set-up was used in the other two studies. CAD4COVID–x-ray was evaluated on 454 images (223 positive for COVID-19) and compared with scores of six radiologists. However, this system was trained with only 416 images of COVID-19 suspects from only one other hospital, although the deep learning network was pretrained on pneumonia data from other sources. CV19-Net was trained with data from Henry Ford Health system and used a total of approximately 5000 images (about half of which were positive for COVID-19) for training. The negative cases were from patients diagnosed with pneumonia in 2019. A drawback of this study was that the test set came from hospitals that also provided most of the training data. On 500 randomly selected test images, equally balanced between positive and negative cases, CV19-Net was compared with readings from three radiologists.All three systems yielded promising results in terms of their area under the receiver operating characteristic curve (AUC), the most commonly used metric for systems that provide a continuous output for a binary classification task. The AUC is equivalent to the chance that a random positive image receives a higher score than a random negative image in the test set. DeepCOVID-XR reached an AUC of 0.88, comparable to the consensus of the five radiologists (AUC = 0.85, scores were obtained using a six-point scale). CV19-Net had an AUC of 0.94, outperforming each of the three readers who did not perform continuous scoring. CAD4COVID–x-ray achieved an AUC of 0.81 and slightly outperformed the radiologists at high-sensitivity cutoffs but performed slightly inferior to four of the six radiologists at their high specificity cutoff.An interesting strategy that Wehbe et al pursued was to design an ensemble of neural networks with diverse characteristics. They trained six different architectures that are popular today (DenseNet-121, ResNet-50, Inception, Inception-ResNet, Xception, and EfficientNet-B2) using two resolution levels (224 × 224 and 331 × 331) and two fields of view (the entire radiograph and the image cropped around the automatically segmented lung fields). Research has shown that zooming in on the lung fields may lead to slightly better performance for abnormality detection on chest radiographs (6). In this study, differences were minor, but the combined approach may be more robust when applied to unseen data.An important avenue for further research mentioned by Wehbe et al is to combine image analysis with additional input, such as demographics, vital signs, and laboratory data. A study using a simple scoring tool suggested such a multimodal approach can increase the AUC substantially compared with imaging alone (7). Additionally, AI analysis of chest radiographs may predict outcomes and guide patient care and interventions. A promising study on predicting intubation and mortality has just been published in Radiology: Artificial Intelligence (8). Such research requires the availability of large data sets where standardized outcomes and treatment parameters are available. Collecting such data remains extremely challenging when patient care strategies are continuously adapted while we learn more about COVID-19.To move forward and learn which approaches to automated analysis have the most potential, we should compare the performance of the various "nets" now published. Sharing the training and test data would facilitate this. However, making medical data publicly available can be a complicated process, and it is something that journals like Radiology do not require. Radiology editorial guidelines do request researchers to share their code unless their study reports on commercial software, as in Murphy et al (4). However, these guidelines do not specify what type of code should be shared, nor are reviewers encouraged to verify that the code produces the results reported in the article. As a result, the reusability of the shared code is often limited. Zhang et al (5) shared their code on GitHub, but their repository does not include the network weights; therefore, it cannot be used to process new images. Wehbe et al have made their code available on GitHub, together with network weights and instructions on how to apply the networks to new data and even how to train the system on additional images. My group is now working on comparing the results of CAD4COVID–x-ray and DeepCOVID-XR on the test set of Murphy et al (4). This would be the start of a more extensive external validation of several artificial intelligence tools that could contribute to the global fight against coronavirus disease 2019.Disclosures of Conflicts of Interest: B.v.G. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: institution received grants from Thirona, Delft Imaging, Siemens Healthineers, and MeVis Medical Solutions; institution receives royalties from Thirona, Delft Imaging, and MeVis Medical Solutions; owns Thirona stock. Other relationships: disclosed no relevant relationships.References1. Rubin GD, Ryerson CJ, Haramati LB, et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology 2020;296(1):172–180. Google Scholar2. Wong HYF, Lam HYS, Fong AHT, et al. Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19. Radiology 2020;296(2):E72–E78. Google Scholar3. Wehbe RM, Sheng J, Dutta S, et al. DeepCOVID-XR: An Artificial Intelligence Algorithm to Detect COVID-19 on Chest Radiographs Trained and Tested on a Large U.S. Clinical Data Set. Radiology 2021;299:E167–E176. Google Scholar4. Murphy K, Smits H, Knoops AJG, et al. COVID-19 on Chest Radiographs: A Multireader Evaluation of an Artificial Intelligence System. Radiology 2020;296(3):E166–E172. Google Scholar5. Zhang R, Tie X, Qi Z, et al. Diagnosis of COVID-19 Pneumonia Using Chest Radiography: Value of Artificial Intelligence. Radiology 2020. 10.1148/radiol.2020202944. Published online September 24, 2020. Google Scholar6. Baltruschat IM, Steinmeister L, Ittrich H, et al. When Does Bone Suppression and Lung Field Segmentation Improve Chest X-Ray Disease Classification? In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, April 8–11, 2019. Piscataway, NJ: IEEE, 2019; 1362–1366. Google Scholar7. Kurstjens S, van der Horst A, Herpers R, et al. Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med 2020;58(9):1587–1593. MedlineGoogle Scholar8. Li MD, Arun NT, Gidwani M, et al. Automated Assessment and Tracking of COVID-19 Pulmonary Disease Severity on Chest Radiographs Using Convolutional Siamese Neural Networks. Radiol Artif Intell 2020;2(4):e200079 https://doi.org/10.1148/ryai.2020200079. Google ScholarArticle HistoryReceived: Nov 8 2020Revision requested: Nov 16 2020Revision received: Nov 16 2020Accepted: Nov 16 2020Published online: Nov 24 2020Published in print: Apr 2021 FiguresReferencesRelatedDetailsCited ByTexture analysis of chest X-ray images for the diagnosis of COVID-19 pneumoniaWaldemarLeszczyński, WojciechKazimierczak, AdamLemanowicz, ZbigniewSerafin25 January 2024 | Polish Journal of Radiology, Vol. 89Evaluation of the clinical performance of an AI-based application for the automated analysis of chest X-raysJulius HenningNiehoff, JanaKalaitzidis, Jan RobertKroeger, DeniseSchoenbeck, JanBorggrefe, Arwed EliasMichael2023 | Scientific Reports, Vol. 13, No. 1Coronavirus diagnosis using cough sounds: Artificial intelligence approachesKazemAskari Nasab, JamalMirzaei, AlirezaZali, SarfenazGholizadeh, MeisamAkhlaghdoust2023 | Frontiers in Artificial Intelligence, Vol. 6Long-COVID diagnosis: From diagnostic to advanced AI-driven modelsRiccardoCau, GavinoFaa, ValentinaNardi, AntonellaBalestrieri, JosepPuig, Jasjit SSuri, RobertoSanFilippo, LucaSaba2022 | European Journal of Radiology, Vol. 148The role of artificial intelligence in plain chest radiographs interpretation during the Covid-19 pandemicDanaAlNuaimi, ReemAlKetbi2022 | BJR|Open, Vol. 4, No. 1AI and We in the Future in the Light of the Ouroboros Model: A Plea for PluralityKnudThomsen2022 | AI, Vol. 3, No. 4COVID-19 Chest X-ray Classification and Severity Assessment Using Convolutional and Transformer Neural NetworksTuanLe Dinh, Suk-HwanLee, Seong-GeunKwon, Ki-RyongKwon2022 | Applied Sciences, Vol. 12, No. 10Advances in Medical Technologies and Clinical PracticeJashasmitaPal, SubhalaxmiDas2022Development and prospective validation of COVID-19 chest X-ray screening model for patients attending emergency departmentsIgnatDrozdov, BenjaminSzubert, ElainaReda, PeterMakary, DanielForbes, Sau LeeChang, AbinayaEzhil, SrikanthPuttagunta, MarkHall, ChrisCarlin, David J.Lowe2021 | Scientific Reports, Vol. 11, No. 1Identification of Pneumonia Disease Applying an Intelligent Computational Framework Based on Deep Learning and Machine Learning TechniquesYarMuhammad, Mohammad DahmanAlshehri, Wael MohammedAlenazy, TruongVinh Hoang, RyanAlturki, MuhammadUsman2021 | Mobile Information Systems, Vol. 2021Lessons from the COVID-19 Pandemic on the Use of Artificial Intelligence in Digital Radiology: The Submission of a Survey to Investigate the Opinion of InsidersDanieleGiansanti, IvanoRossi, LisaMonoscalco2021 | Healthcare, Vol. 9, No. 3Application of artificial intelligence in chest imaging for COVID-19Eun YoungKim, Myung JinChung2021 | Journal of the Korean Medical Association, Vol. 64, No. 10Accompanying This ArticleDeepCOVID-XR: An Artificial Intelligence Algorithm to Detect COVID-19 on Chest Radiographs Trained and Tested on a Large U.S. Clinical Data SetNov 24 2020RadiologyRecommended Articles Artificial Intelligence for Clinical Interpretation of Bedside Chest RadiographsRadiology2022Volume: 307Issue: 1In Bed with AI: Aided Diagnosis of Supine Chest RadiographsRadiology2022Volume: 307Issue: 1Automated Assessment and Tracking of COVID-19 Pulmonary Disease Severity on Chest Radiographs Using Convolutional Siamese Neural NetworksRadiology: Artificial Intelligence2020Volume: 2Issue: 4Clinical and Chest Radiography Features Determine Patient Outcomes in Young and Middle-aged Adults with COVID-19Radiology2020Volume: 297Issue: 1pp. E197-E206Pulmonary Fibrosis: A Guide for the PerplexedRadiology: Cardiothoracic Imaging2021Volume: 3Issue: 1See More RSNA Education Exhibits A Comprehensive Imaging Review of COVID-19 Pneumonia with Focus on RSNA Expert Consensus, Fleischner Society Statement and ACR Recommendations: Challenges, Opportunities and Future Direction Digital Posters2020360-Degree Review Of Lessons Learned From The COVID-19 Pandemic: The Spectrum Of Imaging Findings, Impact On Radiology Departments And The Role Of Artificial IntelligenceDigital Posters2020Outsmarting AI: What Role Can The Radiologist Play In The Making And Deployment Of Artificial Intelligence ApplicationsDigital Posters2021 RSNA Case Collection Resolving COVID-19 pneumonia over timeRSNA Case Collection2020COVID-19 pneumonia and pulmonary embolismRSNA Case Collection2020COVID-19 PneumoniaRSNA Case Collection2020 Vol. 299, No. 1 Metrics Altmetric Score Open AccessThis article is made available via the PMC Open Access Subset for unrestricted re-use and analyses in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the COVID-19 pandemic or until permissions are revoked in writing. Upon expiration of these permissions, PMC is granted a perpetual license to make this article available via PMC and Europe PMC, consistent with existing copyright protections.PDF download