Taking Matters into Your Own Hands
2020; Radiological Society of North America; Volume: 2; Issue: 4 Linguagem: Inglês
10.1148/ryai.2020200150
ISSN2638-6100
Autores Tópico(s)Public Health Policies and Education
ResumoHomeRadiology: Artificial IntelligenceVol. 2, No. 4 PreviousNext CommentaryFree AccessTaking Matters into Your Own HandsSafwan S. Halabi Safwan S. Halabi Author AffiliationsFrom the Department of Radiology, Stanford University School of Medicine, 300 Pasteur Dr, MC 5105, Stanford, CA 94305.Address correspondence to the author (e-mail: [email protected]).Safwan S. Halabi Published Online:Jul 29 2020https://doi.org/10.1148/ryai.2020200150MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked In See also the article by Pan et al in this issue.Safwan S. Halabi, MD, is a clinical associate professor of radiology at the Stanford University School of Medicine and serves as the medical director for radiology informatics at Stanford Children's Health. Dr Halabi's clinical and administrative leadership roles are directed at improving quality of care, efficiency, and patient safety. His current academic and research interests include imaging informatics, deep/machine learning in imaging, artificial intelligence in medicine, clinical decision support, and patient-centric health care delivery.Download as PowerPointOpen in Image Viewer "One accurate measurement is worth a thousand expert opinions."– Admiral Grace Hopper* Bone age assessment became an early AI "poster child" that demonstrated the power of applying regression and machine learning techniques to a mundane and monotonous radiologic diagnostic task. In 2017, Lee et al and Larson et al (1,2) demonstrated the feasibility of curating bone age datasets to create a deep learning neural network model for skeletal age prediction. The deep learning–based automated software application performed with accuracy similar to that of a radiologist (1), and an automated report could be generated (2), which had tremendous implications in clinical practice. The work of curating bone age images and labels into a cohesive dataset served as the backbone of the first Radiological Society of North America Machine Learning Challenge (3). This challenge allowed for the public release of a medical imaging dataset for purposes of creating algorithms that could predict pediatric skeletal age from hand radiographs. The top performers in this challenge substantially outperformed the models that predated the challenge.The study by Pan et al describes a method to redefine the ground truth standard for image-based bone age assessment (4). Specifically, they propose an alternative approach by training on data from normal pediatric hand radiographs, using chronological age as ground truth, versus radiologist clinical interpretation utilizing previously described methods such as Gruelich and Pyle. The authors describe a method using a cohort of normal hand radiographs acquired in the emergency department as a proxy to the standard based on the 1950s Gruelich and Pyle atlas (5). Despite wide usage of the Gruelich and Pyle method for bone age assessment, it has been shown that this method is fraught with inter- and intraobserver variability (6–8). In addition, Gruelich and Pyle's patient population comprised a homogeneous cohort of White children from a limited geographic location (northern Ohio).The current study by Pan et al takes bone age assessment via deep learning one step further by evaluating a method that compares the current standard of reference to the true standard of chronological age (4). Despite the small number of examinations assessed, the authors prove that it is feasible to develop an algorithm that uses the local demographic chronological ages as the standard of reference as an alternative to antiquated methods such as Gruelich and Pyle. It is difficult to discern how diverse the patient population is in a specific hospital emergency department setting to generalize their method and algorithm to other sites. However, this study does show the feasibility of compiling datasets from multiple sites or using an overfitted model for use in a single center or health care system where the data were procured for the model development. The lead author, in another study, showed the power of combining or ensembling bone age assessment models to create better performing models (9).There are varying degrees of medical imaging data readiness (MIDaR), which were elegantly described by Harvey and Glocker in their MIDaR scale (10). This four-point MIDaR scale ranges from level D to level A. Level D (the lowest level of data readiness on the scale), or what can be referred to as "dirty" or "raw" data, represents data that contain patient-identifiable information, unverified in quantity and quality, and inaccessible to researchers. In contradistinction, a level A dataset is "structured, fully annotated, has minimal noise and, most importantly, is contextually appropriate and ready for a specific machine learning task (10)." Level A data (data veracity) are quite elusive, laborious to curate, and exist in low volumes. However, for certain machine learning tasks, such data can be discovered in plain sight.This concept of using data discovered in plain sight should open the door for many researchers to look internally for opportunities to combine the imaging and labels (eg, radiology reports, demographic data, clinical notes, etc) to create datasets that could be used to create clinically efficacious algorithms for diagnosis and prediction. Finding a standard of reference and curating level A data for the purposes of any diagnostic imaging task is difficult and elusive. This study promotes the idea of retrospectively looking at the continuum of the medical record to use as a post hoc prediction which would significantly expedite the ability to alert the radiologist and clinical teams if a certain patient falls within the expected range of normal studies or requires more attention because their attributes or findings don't match those of the patients who preceded them.The previously published nomograms, whether qualitative or quantitative, like Gruelich and Pyle, will become relics of the past as we enter the era of the "learning" or "evolving" nomograms that are based on data that are readily available in our electronic medical records. As Pan et al so eloquently demonstrated, diagnostic tools developed from internal sources were comparable to or superseded known standards that have been used for decades. These methods have tremendous potential to not only revolutionize the development of standards of reference to develop machine learning models, but also to help usher in an era of quantitative imaging that will enhance disease detection, surveillance, and intervention effectiveness.Disclosures of Conflicts of Interest: S.H. disclosed no relevant relationships.* Grace Brewster Murray Hopper was a computer pioneer and naval officer who received a master's degree and PhD in mathematics from Yale. She was one of the first three modern programmers, best known for her revolutionary contributions to the development of computer languages. Hopper had a reputation for being irreverent, sharp-tongued, and brilliant, and had long and influential careers in both the U.S. Navy and the private sector.References1. Lee H, Tajmir S, Lee J, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017;30(4):427–441. Crossref, Medline, Google Scholar2. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 2018;287(1):313–322. Link, Google Scholar3. Halabi SS, Prevedello LM, Kalpathy-Cramer J, et al. The RSNA pediatric bone age machine learning challenge. Radiology 2019;290(2):498–503. Link, Google Scholar4. Pan I, Baird G, Mutasa S, et al. Rethinking Greulich and Pyle: A deep learning approach to pediatric bone age assessment. Radiol Artif Intell 2020;2(4):e190198. Abstract, Google Scholar5. Greulich W, Pyle S. Radiographic Atlas of Skeletal Development of the Hand and Wrist. Stanford, Calif: Stanford University Press, 1999. Google Scholar6. Berst MJ, Dolan L, Bogdanowicz MM, Stevens MA, Chow S, Brandser EA. Effect of knowledge of chronologic age on the variability of pediatric bone age determined using the Greulich and Pyle standards. AJR Am J Roentgenol 2001;176(2):507–510. Crossref, Medline, Google Scholar7. Thodberg HH, Sävendahl L. Validation and reference values of automated bone age determination for four ethnicities. Acad Radiol 2010;17(11):1425–1432. Crossref, Medline, Google Scholar8. Johnson GF, Dorst JP, Kuhn JP, Roche AF, Dávila GH. Reliability of skeletal age assessments. Am J Roentgenol Radium Ther Nucl Med 1973;118(2):320–327. Crossref, Medline, Google Scholar9. Pan I, Thodberg HH, Halabi SS, Kalpathy-Cramer J, Larson DB. Improving automated pediatric bone age estimation using ensembles of models from the 2017 RSNA machine learning challenge. Radiol Artif Intell 2019;1(6):e190053. Link, Google Scholar10. Harvey H, Glocker B. A standardised approach for preparing imaging data for machine learning tasks in radiology. In: Ranschaert E, Morozov S, Algra P, eds. Artificial intelligence in medical imaging. Cham, Switzerland: Springer, 2019; 61–72. Crossref, Google ScholarArticle HistoryReceived: June 22 2020Revision requested: June 25 2020Revision received: June 30 2020Accepted: July 7 2020Published online: July 29 2020 FiguresReferencesRelatedDetailsAccompanying This ArticleRethinking Greulich and Pyle: A Deep Learning Approach to Pediatric Bone Age Assessment Using Pediatric Trauma Hand RadiographsJul 29 2020Radiology: Artificial IntelligenceRecommended Articles Point-of-Care Bone Age Evaluation: The Increasing Role of US in Resource-limited PopulationsRadiology2020Volume: 296Issue: 1pp. 170-171Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand RadiographsRadiology2017Volume: 287Issue: 1pp. 313-322The RSNA Pediatric Bone Age Machine Learning ChallengeRadiology2018Volume: 290Issue: 2pp. 498-503What Can We Learn from the RSNA Pediatric Bone Age Machine Learning Challenge?Radiology2018Volume: 290Issue: 2pp. 504-505Rethinking Greulich and Pyle: A Deep Learning Approach to Pediatric Bone Age Assessment Using Pediatric Trauma Hand RadiographsRadiology: Artificial Intelligence2020Volume: 2Issue: 4See More RSNA Education Exhibits Evidence Based Radiology in the age of Artificial Intelligence: The PICO/DATO ModelDigital Posters2020Digital Atlas for Radiological Evaluation of Bone Age in Male and Female Gender - Execution of a Digital Tool Based on the Author's Books Greulich & Pyle and Theodore KeatsDigital Posters2018Multimodality Imaging of Skeletal Dysplasias: What the Radiologist Needs to KnowDigital Posters2018 RSNA Case Collection Subperiosteal AbscessRSNA Case Collection2021Primary Bone LymphomaRSNA Case Collection2020Accessory Parietal Sutures RSNA Case Collection2021 Vol. 2, No. 4 Metrics Downloaded 850 times Altmetric Score PDF download
Referência(s)