Artigo Acesso aberto Revisado por pares

Evaluation of Commercial Next-Generation Sequencing Bioinformatics Software Solutions

2019; Elsevier BV; Volume: 22; Issue: 2 Linguagem: Inglês

10.1016/j.jmoldx.2019.09.007

ISSN

1943-7811

Autores

Rama R. Gullapalli,

Tópico(s)

Genetics, Bioinformatics, and Biomedical Research

Resumo

Next-generation sequencing (NGS) diagnostics continue to expand rapidly in clinical medicine. An ever-expanding menu of molecular biomarkers is deemed important for diagnostic, prognostic, and therapeutic assessment in patients. The increasing role of NGS in the clinic is driven mainly by the falling costs of sequencing. However, the data-intensive nature of NGS makes bioinformatic analysis a major challenge to many clinical laboratories. Critically needed NGS bioinformatics personnel are hard to recruit and retain in small- to mid-size clinical laboratories. Also, NGS software often lacks the scalability necessary for expanded clinical laboratory testing volumes. Commercial software solutions aim to bridge the bioinformatics barrier via turnkey informatics solutions tailored specifically for the clinical workplace. Yet, there has been no systematic assessment of these software solutions thus far. This article presents an end-to-end vendor evaluation experience of commercial NGS bioinformatics solutions. Six different commercial vendor solutions were assessed systematically. Key metrics of NGS software evaluation to aid in the robust assessment of software solutions are described. Comprehensive feedback, provided by the TriCore Reference Laboratories molecular pathology team, enabled the final vendor selection. Many key lessons were learned during the software evaluation process, which are described herein. This article aims to provide a detailed road map for small- to mid-size clinical laboratories interested in evaluating commercial bioinformatics solutions available in the marketplace. Next-generation sequencing (NGS) diagnostics continue to expand rapidly in clinical medicine. An ever-expanding menu of molecular biomarkers is deemed important for diagnostic, prognostic, and therapeutic assessment in patients. The increasing role of NGS in the clinic is driven mainly by the falling costs of sequencing. However, the data-intensive nature of NGS makes bioinformatic analysis a major challenge to many clinical laboratories. Critically needed NGS bioinformatics personnel are hard to recruit and retain in small- to mid-size clinical laboratories. Also, NGS software often lacks the scalability necessary for expanded clinical laboratory testing volumes. Commercial software solutions aim to bridge the bioinformatics barrier via turnkey informatics solutions tailored specifically for the clinical workplace. Yet, there has been no systematic assessment of these software solutions thus far. This article presents an end-to-end vendor evaluation experience of commercial NGS bioinformatics solutions. Six different commercial vendor solutions were assessed systematically. Key metrics of NGS software evaluation to aid in the robust assessment of software solutions are described. Comprehensive feedback, provided by the TriCore Reference Laboratories molecular pathology team, enabled the final vendor selection. Many key lessons were learned during the software evaluation process, which are described herein. This article aims to provide a detailed road map for small- to mid-size clinical laboratories interested in evaluating commercial bioinformatics solutions available in the marketplace. Next-generation sequencing (NGS) is rapidly gaining importance as a key driver of personalized medicine efforts in clinical practice.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar Clinical laboratories are the linchpin for the implementation of fast evolving NGS technologies into routine clinical practice.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,3Carter A.B. Considerations for genomic data privacy and security when working in the cloud.J Mol Diagn. 2019; 21: 542-552Abstract Full Text Full Text PDF PubMed Scopus (25) Google Scholar,4Luthra R. Chen H. Roy-Chowdhuri S. Singh R.R. Next-generation sequencing in clinical molecular diagnostics of cancer: advantages and challenges.Cancers (Basel). 2015; 7: 2023-2036Crossref PubMed Scopus (91) Google Scholar Research-based NGS studies have identified numerous genomic, transcriptional, and epigenetic biomarkers with potential clinical applications.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,5Cheng D.T. Mitchell T.N. Zehir A. Shah R.H. Benayed R. Syed A. Chandramohan R. Liu Z.Y. Won H.H. Scott S.N. Brannon A.R. O'Reilly C. Sadowska J. Casanova J. Yannes A. Hechtman J.F. Yao J. Song W. Ross D.S. Oultache A. Dogan S. Borsu L. Hameed M. Nafa K. Arcila M.E. Ladanyi M. Berger M.F. Memorial Sloan Kettering-Integrated Mutation Profiling of Actionable Cancer Targets (MSK-IMPACT): a hybridization capture-based next-generation sequencing clinical assay for solid tumor molecular oncology.J Mol Diagn. 2015; 17: 251-264Abstract Full Text Full Text PDF PubMed Scopus (1179) Google Scholar In pathology specifically, molecular biomarker assessment is becoming increasingly crucial for the process of diagnosis, prognosis, and therapeutic assessment of oncology patients.2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar With the availability of an increasing number of US Food and Drug Administration–approved, targeted cancer therapies, there is increasing demand from oncologists for multiplexed detection of cancer biomarkers as opposed to the previous single biomarker paradigms.6Rajeevan M.S. Li T. Unger E.R. Precision medicine requires precision laboratories.J Mol Diagn. 2017; 19: 226-229Abstract Full Text Full Text PDF PubMed Scopus (2) Google Scholar The ability to detect multiple biomarkers on a single NGS platform in a multiplexed manner (ie, many patients on a single run) is a key advantage of NGS technology.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar Although NGS technologies are undoubtedly powerful and have the potential to alter current clinical practice dramatically, there are many hurdles to their routine clinical implementation. The key challenge of clinical NGS is also the main strength of this technique (namely, the amount of sequencing data generated).3Carter A.B. Considerations for genomic data privacy and security when working in the cloud.J Mol Diagn. 2019; 21: 542-552Abstract Full Text Full Text PDF PubMed Scopus (25) Google Scholar,7Campbell W.S. Carter A.B. Cushman-Vokoun A.M. Greiner T.C. Dash R.C. Routbort M. de Baca M.E. Campbell J.R. A model information management plan for molecular pathology sequence data using standards: from sequencer to electronic health record.J Mol Diagn. 2019; 21: 408-417Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar,8Dove E.S. Joly Y. Tasse A.M. Knoppers B.M. Public Population Project in Genomics and Society (P3G) International Steering CommitteeInternational Cancer Genome Consortium (ICGC) Ethics and Policy CommitteeGenomic cloud computing: legal and ethical points to consider.Eur J Hum Genet. 2015; 23: 1271-1278Crossref PubMed Scopus (64) Google Scholar NGS is a data-intensive technique that generates a large amount of data at each step of the process: i) the raw data signal calls on the instrument, ii) FASTQ data files for the bioinformatics analysis, and iii) annotation data describe the generated sequencing variants.2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar,9Roy S. Coldren C. Karunamurthy A. Kip N.S. Klee E.W. Lincoln S.E. Leon A. Pullambhatla M. Temple-Smolkin R.L. Voelkerding K.V. Wang C. Carter A.B. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.J Mol Diagn. 2018; 20: 4-27Abstract Full Text Full Text PDF PubMed Scopus (226) Google Scholar Depending on the type of the NGS assay, the number of sequencing variants generated range from tens to thousands to millions (eg, panel sequencing versus whole-exome versus whole-genome sequencing, respectively).1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar,9Roy S. Coldren C. Karunamurthy A. Kip N.S. Klee E.W. Lincoln S.E. Leon A. Pullambhatla M. Temple-Smolkin R.L. Voelkerding K.V. Wang C. Carter A.B. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.J Mol Diagn. 2018; 20: 4-27Abstract Full Text Full Text PDF PubMed Scopus (226) Google Scholar In clinical oncology, the number of data variants generated is high because of the inherent genomic instability associated with biological carcinogenesis. With increasing interest in newer parameters of tumor assessment (eg, tumor mutational burden, copy number changes, and epigenetic markers), issues of NGS data management are complicated even further.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar,9Roy S. Coldren C. Karunamurthy A. Kip N.S. Klee E.W. Lincoln S.E. Leon A. Pullambhatla M. Temple-Smolkin R.L. Voelkerding K.V. Wang C. Carter A.B. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.J Mol Diagn. 2018; 20: 4-27Abstract Full Text Full Text PDF PubMed Scopus (226) Google Scholar Clinical NGS assays are evolving quickly and growing exponentially in scope.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,10Jennings L.J. Arcila M.E. Corless C. Kamel-Reid S. Lubin I.M. Pfeifer J. Temple-Smolkin R.L. Voelkerding K.V. Nikiforova M.N. Guidelines for validation of next-generation sequencing-based oncology panels: a joint consensus recommendation of the Association for Molecular Pathology and College of American Pathologists.J Mol Diagn. 2017; 19: 341-365Abstract Full Text Full Text PDF PubMed Scopus (370) Google Scholar Yet, routine availability of NGS assays and trained clinical personnel capable of interpreting high-throughput clinical data are highly uneven. NGS assay expertise is a niche subspecialization in a clinical laboratory, available mostly at major academic centers and large commercial clinical laboratories. The implementation of NGS assays requires significant financial resources and personnel for a routine clinical laboratory. Even with the availability of financial resources to purchase clinical NGS platforms, trained personnel with expertise in bioinformatics, clinical informatics, and data analytics are hard to come by.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar The lack of trained personnel capable of performing NGS bioinformatics analysis continues to be a major impediment to the widespread implementation of NGS testing.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar Currently, doctoral-level bioinformaticists work alongside molecular pathologists to enable clinical NGS assay implementation. The lack of a formal curriculum in genomics and bioinformatics in pathology residency training is a significant hurdle to the widespread adoption of NGS. A vast majority of the current practitioners of NGS analytics in a clinical laboratory are self-educated, often through a process of trial and error. Yet, because of the ever-increasing complexity of NGS testing, it is a challenge for any single individual to gain mastery over all of the aspects of the NGS assay (experimental and bioinformatics).1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar In the absence of adequate resources (financial and personnel), NGS assay options for many mid- to small-level institutions are restricted to send-out tests for biomarker assessment. Yet, standardized guidelines for pretherapy diagnostic assessment increasingly mandate the use of molecular biomarkers to guide therapies in newly diagnosed cancer patients.9Roy S. Coldren C. Karunamurthy A. Kip N.S. Klee E.W. Lincoln S.E. Leon A. Pullambhatla M. Temple-Smolkin R.L. Voelkerding K.V. Wang C. Carter A.B. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.J Mol Diagn. 2018; 20: 4-27Abstract Full Text Full Text PDF PubMed Scopus (226) Google Scholar,11Li M.M. Datto M. Duncavage E.J. Kulkarni S. Lindeman N.I. Roy S. Tsimberidou A.M. Vnencak-Jones C.L. Wolff D.J. Younes A. Nikiforova M.N. Standards and guidelines for the interpretation and reporting of sequence variants in cancer: a joint consensus recommendation of the Association for Molecular Pathology, American Society of Clinical Oncology, and College of American Pathologists.J Mol Diagn. 2017; 19: 4-23Abstract Full Text Full Text PDF PubMed Scopus (861) Google Scholar,12Lih C.J. Harrington R.D. Sims D.J. Harper K.N. Bouk C.H. Datta V. Yau J. Singh R.R. Routbort M.J. Luthra R. Patel K.P. Mantha G.S. Krishnamurthy S. Ronski K. Walther Z. Finberg K.E. Canosa S. Robinson H. Raymond A. Le L.P. McShane L.M. Polley E.C. Conley B.A. Doroshow J.H. Iafrate A.J. Sklar J.L. Hamilton S.R. Williams P.M. Analytical validation of the next-generation sequencing assay for a nationwide signal-finding clinical trial: molecular Analysis for Therapy Choice Clinical Trial.J Mol Diagn. 2017; 19: 313-327Abstract Full Text Full Text PDF PubMed Scopus (93) Google Scholar It is no longer an option to ignore a patient's molecular biomarker status before the initiation of cancer treatments. Middle- to small-sized clinical laboratories increasingly need to explore NGS assay options in their own setups. For a clinical laboratory with a preexisting non-NGS molecular component, the wet/bench portion of NGS technology is reasonably accessible (ie, NGS instrumentation and molecular technologist expertise). However, the bioinformatics portion of clinical NGS testing eludes easy solutions, irrespective of the institutional size. Some institutions have developed in-house solutions to bridge the bioinformatics gap.13Kang W. Kadri S. Puranik R. Wurst M.N. Patil S.A. Mujacic I. Benhamed S. Niu N. Zhen C.J. Ameti B. Long B.C. Galbo F. Montes D. Iracheta C. Gamboa V.L. Lopez D. Yourshaw M. Lawrence C.A. Aisner D.L. Fitzpatrick C. McNerney M.E. Wang Y.L. Andrade J. Volchenboum S.L. Furtado L.V. Ritterhouse L.L. Segal J.P. System for informatics in the molecular pathology laboratory: an open-source end-to-end solution for next-generation sequencing clinical data management.J Mol Diagn. 2018; 20: 522-532Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar Yet, this is not easy or feasible for smaller institutions.2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar,13Kang W. Kadri S. Puranik R. Wurst M.N. Patil S.A. Mujacic I. Benhamed S. Niu N. Zhen C.J. Ameti B. Long B.C. Galbo F. Montes D. Iracheta C. Gamboa V.L. Lopez D. Yourshaw M. Lawrence C.A. Aisner D.L. Fitzpatrick C. McNerney M.E. Wang Y.L. Andrade J. Volchenboum S.L. Furtado L.V. Ritterhouse L.L. Segal J.P. System for informatics in the molecular pathology laboratory: an open-source end-to-end solution for next-generation sequencing clinical data management.J Mol Diagn. 2018; 20: 522-532Abstract Full Text Full Text PDF PubMed Scopus (5) Google Scholar To overcome this shortcoming faced by smaller institutions, commercial cloud-based bioinformatics solutions have come into existence in the marketplace. Commercial vendors claim to provide solutions capable of bridging the lack of bioinformatics expertise in a clinical laboratory setting. In this article, evaluative assessments of the different commercial NGS bioinformatics solutions available in the market are described. At the University of New Mexico (UNM)/TriCore Reference Laboratories (TRL), routine oncology NGS testing is performed in various tumor categories (solid tumor and hematological). A major driver behind the need for a bioinformatics upgrade was the observation that the existing bioinformatics pipelines did not scale over time (2014 to current) and increasing testing volume (increasing annually). In addition, the lack of historical variant annotation data reduced the efficiency of the medical director's weekly sign-out process in the laboratory. The overall goal of this project was to assess NGS software in a rigorous and robust manner to differentiate the key bioinformatics features of solutions available in the marketplace. Thus, this evaluation experience may provide a useful roadmap to other small- to mid-sized laboratories interested in bringing NGS-based testing (and the bioinformatics software necessary) into their own setups. Six medical directors along with technical personnel in the laboratory drive the strategic long-term molecular diagnostics (MDx) agenda at UNM/TRL. The medical directors are collectively responsible for the planning, strategy, and execution of MDx projects originating in the division. A federated model of NGS bioinformatic analytics was adopted since starting clinical NGS assay services in 2014. A federated model implies that various components of the bioinformatics pipeline are executed by different individuals working in the MDx laboratory. TRL MDx technicians are trained to perform the initial manipulation of the raw NGS data (read mapping, BAM file generation, initial variant call reads, and quality assurance assessment). After the preliminary evaluation, each medical director in the laboratory (and clinical fellows) perform the higher-level variant assessment to issue the final NGS report associated with each patient. Cases with nondefinitive outcomes are reviewed collectively at a consensus meeting as necessary. Individuals with advanced bioinformatics expertise (R.R.G.) serve in a consultative role to drive the broad bioinformatics agenda in the division. The federated NGS bioinformatics model eliminated the need to hire specialized bioinformatics personnel, saving valuable resources for the institution. Because of the lack of scalability of the federated NGS workflow model over time (2014 to current), a decision was made to obtain additional specialized NGS software in January 2017. The key problems associated with the original bioinformatics pipeline included the inability to review historical NGS sign-out data (by various medical directors), a lack of access to comments of previously signed-out clinical cases, and a lack of historical quality control (QC) data across NGS assay runs. These shortcomings caused a reinvention of the wheel for case sign out every week, reducing the overall efficiency of the NGS sign-out process. The broad goal of the intended software purchase was to enable a scalable workflow to accommodate the increasing NGS needs of the MDx division as well as enable efficiencies of the physician sign-out process. The NGS information technology (IT) assessment project obtained an executive formal go-ahead in February 2017. A team of four individuals with expertise in technical project management were selected to drive the software evaluation process. This included the following: i) two medical directors, ii) one TRL MDx technical supervisor, and iii) one administrative specialist with expertise in project management to track the data collection during the review process. Executive management at TRL reviewed the consensus decision of the medical directors, and the project was green lighted in March 2017. At this initial planning stage, two key lessons were learned, which are worthy of mention: An IT project of this evaluative scope requires a key champion to drive forward the overall project, and adequate support staff is critical to track various aspects of the review during the entire software assessment process. It is a well-known fact that resources (human and financial) are often in short supply at small- to mid-level institutions. Yet, it is critically important to allocate adequate human resources upfront (at least one project management specialist) to ensure the successful completion of NGS software evaluation. A major challenge encountered in the initial phase of the project was determining the evaluation criteria of the NGS software assessment process.1Gullapalli R.R. Desai K.V. Santana-Santos L. Kant J.A. Becich M.J. Next generation sequencing in clinical medicine: challenges and lessons for pathology and biomedical informatics.J Pathol Inform. 2012; 3: 40Crossref PubMed Google Scholar,2Roy S. LaFramboise W.A. Nikiforov Y.E. Nikiforova M.N. Routbort M.J. Pfeifer J. Nagarajan R. Carter A.B. Pantanowitz L. Next-generation sequencing informatics: challenges and strategies for implementation in a clinical environment.Arch Pathol Lab Med. 2016; 140: 958-975Crossref PubMed Scopus (48) Google Scholar,14Gullapalli R.R. Lyons-Weiler M. Petrosko P. Dhir R. Becich M.J. LaFramboise W.A. Clinical integration of next-generation sequencing technology.Clin Lab Med. 2012; 32: 585-599Abstract Full Text Full Text PDF PubMed Scopus (50) Google Scholar To our knowledge, there are no descriptions of an NGS vendor assessment process previously reported in the literature. It was quickly learned that health care IT evaluation is nontrivial and a complicated process. Health care software assessment must involve not only the technical evaluation of the software features but also the potential impact on the end users (pathologists and technical staff). The initial stages mainly relied on the following resources to determine the assessment criteria of the in-house NGS software: i) official College of American Pathologists' molecular pathology checklists,15Molecular Pathology Checklist. College of American Pathologists (CAP), Chicago, IL2017: 35-51Google Scholar ii) Next-generation Sequencing: Standardization of Clinical Testing and additional Association for Molecular Pathology guidelines for bioinformatics pipelines,9Roy S. Coldren C. Karunamurthy A. Kip N.S. Klee E.W. Lincoln S.E. Leon A. Pullambhatla M. Temple-Smolkin R.L. Voelkerding K.V. Wang C. Carter A.B. Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the Association for Molecular Pathology and the College of American Pathologists.J Mol Diagn. 2018; 20: 4-27Abstract Full Text Full Text PDF PubMed Scopus (226) Google Scholar,10Jennings L.J. Arcila M.E. Corless C. Kamel-Reid S. Lubin I.M. Pfeifer J. Temple-Smolkin R.L. Voelkerding K.V. Nikiforova M.N. Guidelines for validation of next-generation sequencing-based oncology panels: a joint consensus recommendation of the Association for Molecular Pathology and College of American Pathologists.J Mol Diagn. 2017; 19: 341-365Abstract Full Text Full Text PDF PubMed Scopus (370) Google Scholar,16Gargis A.S. Kalman L. Berry M.W. Bick D.P. Dimmock D.P. Hambuch T. et al.Assuring the quality of next-generation sequencing in clinical laboratory practice.Nat Biotechnol. 2012; 30: 1033-1036Crossref PubMed Scopus (362) Google Scholar, 17Gargis A.S. Kalman L. Bick D.P. da Silva C. Dimmock D.P. Funke B.H. et al.Good laboratory practice for clinical next-generation sequencing informatics pipelines.Nat Biotechnol. 2015; 33: 689-693Crossref PubMed Scopus (113) Google Scholar, 18Lubin I.M. Aziz N. Babb L.J. Ballinger D. Bisht H. Church D.M. Cordes S. Eilbeck K. Hyland F. Kalman L. Landrum M. Lockhart E.R. Maglott D. Marth G. Pfeifer J.D. Rehm H.L. Roy S. Tezak Z. Truty R. Ullman-Cullere M. Voelkerding K.V. Worthey E.A. Zaranek A.W. Zook J.M. Principles and recommendations for standardizing the use of the next-generation sequencing variant file in clinical settings.J Mol Diagn. 2017; 19: 417-426Abstract Full Text Full Text PDF PubMed Scopus (15) Google Scholar and iii) a book, Evaluative Methods in Biomedical Informatics, authored by C.P. Friedman and Jeremy C. Wyatt.19Friedman C.P. Wyatt J. Evaluation Methods in Biomedical Informatics. Springer, New York, NY2005Google Scholar All these resources collectively provided an excellent starting point to enable a thorough assessment of commercial NGS software solutions described in this article. After surveying the literature, a list of key preassessment considerations was identified to help the team with the downstream evaluation process. Table 1 describes some of the key preliminary questions considered by the team before initiating the software review process. The major focus in this initial evaluation period related to the nature of the software itself and an assessment of the likely impact on the established NGS workflow within our laboratory (Table 1). Many of these questions are likely to be similar in nature for other laboratories as well. It is critical for the project leadership to thoroughly assess the basic laboratory NGS requirements in discussion with the key stakeholders at the institution (ie, executive management and laboratory personnel).Table 1A List of Potential Questions to Consider Before Software AssessmentsSoftware questionsLaboratory workflow questionsIs the software necessary? What are the key payoffs due to the acquisition?Is the software user friendly at all levels? (pathologists, supervisors, and technologists?)Windows or a Linux platform?How does the software improve the overall efficiency of the clinical sign out?Local on-site install or cloud based?Is the QC metrics collection improved?Is the software scalable for different kinds of assays in the future? (eg, DNA sequencing, RNA sequencing, and microbiome data)Is the transition likely to be disruptive? (for laboratories with existing NGS workflows)Reputation of the software in the user laboratory professional community?How does it help with impr

Referência(s)