Portal de Periódicos da CAPES

Validation for Clinical Use of, and Initial Clinical Experience with, a Novel Approach to Population-Based Carrier Screening using High-Throughput, Next-Generation DNA Sequencing

2014; Elsevier BV; Volume: 16; Issue: 2 Linguagem: Inglês

10.1016/j.jmoldx.2013.10.006

1943-7811

Stephanie Hallam, Heather H. Nelson, Valerie Greger, Cynthia L. Perreault-Micale, J. Davie, Nicole Faulkner, Dana Neitzel, Kristie Casey, Mark A. Umbarger, Niru Chennagiri, Alexander Krämer, Gregory J. Porreca, Caleb J. Kennedy,

Genetic and Kidney Cyst Diseases

Traditional carrier screening assays are designed to look for only the most common mutations within a gene owing to cost considerations. Although this can yield high detection rates in specific populations for specific genes (such as cystic fibrosis in Caucasians), they are suboptimal for other ethnicities or for patients of mixed or unknown ethnic background. Next-generation DNA sequencing provides an opportunity to provide carrier screening using more comprehensive mutation panels that are limited primarily by information about the clinical impact of detected sequence changes. We describe a next-generation DNA sequencing-based assay capable of reliably screening patient samples in a timely and comprehensive manner. The analytic accuracy in a research setting has been documented. Here, we describe the additional studies performed to ensure the accuracy (analytic validity) and robustness of our assay for use in clinical practice and provide data from our experience offering this testing. Our clinical experience using this approach to screen 11,691 in vitro fertilization patients has identified 449 mutant alleles: 447 in carriers and 2 in an affected individual. In total, we found 87 distinct mutations in 14 different genes. Approximately one quarter of the mutations found are not included in traditional, limited, mutation panels, including 16 known mutations unique to our panel, and novel truncating mutations in several genes. Traditional carrier screening assays are designed to look for only the most common mutations within a gene owing to cost considerations. Although this can yield high detection rates in specific populations for specific genes (such as cystic fibrosis in Caucasians), they are suboptimal for other ethnicities or for patients of mixed or unknown ethnic background. Next-generation DNA sequencing provides an opportunity to provide carrier screening using more comprehensive mutation panels that are limited primarily by information about the clinical impact of detected sequence changes. We describe a next-generation DNA sequencing-based assay capable of reliably screening patient samples in a timely and comprehensive manner. The analytic accuracy in a research setting has been documented. Here, we describe the additional studies performed to ensure the accuracy (analytic validity) and robustness of our assay for use in clinical practice and provide data from our experience offering this testing. Our clinical experience using this approach to screen 11,691 in vitro fertilization patients has identified 449 mutant alleles: 447 in carriers and 2 in an affected individual. In total, we found 87 distinct mutations in 14 different genes. Approximately one quarter of the mutations found are not included in traditional, limited, mutation panels, including 16 known mutations unique to our panel, and novel truncating mutations in several genes. Next-generation DNA sequencing (NGS) holds the promise of providing high-throughput, accurate carrier screening for multiple genes and multiple mutations in a highly efficient manner in clinical laboratories.1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar To date, NGS has found application in evaluating affected individuals for numerous multigene disorders and for elucidating the correct diagnosis and hence treatment of patients with various forms of cancer,2Rehm H.L. Disease-targeted sequencing: a cornerstone in the clinic.Nat Rev Genet. 2013; 14: 295-300Crossref PubMed Scopus (299) Google Scholar but carrier screening assay development has been slower because of concerns about accuracy3Desai A.N. Jere A. Next-generation sequencing: ready for the clinics?.Clin Genet. 2012; 81: 503-510Crossref PubMed Scopus (119) Google Scholar, 4Flaherty P. Natsoulis G. Muralidharan O. Winters M. Buenrostro J. Bell J. Brown S. Holodniy M. Zhang N. Ji H.P. Ultrasensitive detection of rare mutations using next-generation targeted resequencing.Nucleic Acids Res. 2012; 40: e2Crossref PubMed Scopus (99) Google Scholar and throughput in a clinical setting. Here, we describe the validation for clinical use of the first multigene carrier screening assay using NGS technology to be offered in the United States and provide information about our clinical experience thus far. The genes included in our assay have proven clinical validity (the association between mutations in the gene and the related disorder has been established).5American College of Obstetricians and Gynecologists Committee on GeneticsACOG committee opinion no. 486: update on carrier screening for cystic fibrosis.Obstet Gynecol. 2011; 117: 1028-1031Crossref PubMed Scopus (155) Google Scholar, 6Scott S.A. Edelmann L. Liu L. Luo M. Desnick R.J. Kornreich R. Experience with carrier screening and prenatal diagnosis for 16 Ashkenazi Jewish genetic diseases.Hum Mutat. 2010; 31: 1240-1250Crossref PubMed Scopus (120) Google Scholar In addition, carrier testing for these genes is recommended by the American College of Medical Genetics and Genomics, the American Congress of Obstetricians and Gynecologists, and/or are assessed routinely in persons of Ashkenazi Jewish descent because of the increased carrier frequency in this population and/or their clinical severity. The NGS panel validated the following diseases (gene symbols are shown in parentheses): Canavan disease (ASPA), cystic fibrosis (CFTR), glycogen storage disorder type 1a (G6PC), Niemann-Pick disease (SMPD1), Tay-Sachs disease (HEXA), Bloom syndrome (BLM), Fanconi anemia C (FANCC), familial hyperinsulinism (ABCC8), maple syrup urine disease type 1A (BCKDHA) and type 1B (BCKDHB), Usher syndrome type III (CLRN1), dihydrolipoamide dehydrogenase deficiency (DLD), familial dysautonomia (IKBKAP), mucolipidosis type IV (MCOLN1), and Usher syndrome type 1F (PCDH15). Additional genes and mutations, for which the gene or mutations are not readily evaluated by NGS, are assessed by alternate methodologies to ensure detection of clinically important mutations. The validations for these alternate methodologies followed standard, clinical laboratory procedures and are not described here. Clinical carrier screening assays traditionally have assessed a limited set of mutations, typically those prevalent in specific ethnic groups. NGS provides the possibility of finding a much larger set of sequence variants across many ethnic groups. Because NGS is not limited to a small number of mutations, there is an additional dilemma related to the interpretation and reporting of the variants detected. Certainly, this is one of the most challenging areas associated with the advent of NGS technology in the carrier screening arena. Extensive discussions and feedback from our clinical and genetic counselor advisory boards has indicated that specificity was an important factor in offering carrier screening using NGS. In addition, others advocating a responsible approach to offering full sequencing recommend reporting only those variants that are known to have a clinical impact7Khoury M.J. Coates R.J. Evans J.P. Evidence-based classification of recommendations on use of genomic tests in clinical practice: dealing with insufficient evidence.Genet Med. 2010; 12: 680-683Abstract Full Text Full Text PDF PubMed Scopus (62) Google Scholar (ie, no variants of unknown significance). After determination of the panel of genes to be assessed, it was essential to our clinical approach to rigorously establish the variants that must be detected for multifold reasons, including the following: i) to complete a comprehensive evaluation of all available information about each variant for each gene to determine the full list of mutations that were considered pathogenic; ii) to put this information into a variant database that could be curated, managed, and updated periodically to ensure that new information about variants could be added and that the panel would remain pertinent (this information then can be used for future cases); iii) to have a system that would integrate with the NGS data analysis pipeline to ensure rapid and consistent calling of clinically relevant mutations; and iv) to ensure that all variants that passed our filters for being pathogenic and clinically important could be either readily detected by the NGS assay or an alternate methodology to ensure detection. These alternate methodologies have all been used in clinical laboratories performing carrier testing for a number of years and hence are not discussed further (they were all validated before use in our laboratory). This latter consideration is important and one that is gaining awareness among those using or considering using NGS technology for clinical applications. Some genes, gene regions, or mutations are particularly problematic for NGS and vary with the technology and analysis method(s) used. For example, pseudogenes, GC-rich regions, homopolymers, large deletions, and complex insertions/deletions all can be problematic depending on the specific sample preparation, sequencing, and analysis method(s) used.8Schrijver I. Aziz N. Farkas D.H. Furtado M. Gonzalez A.F. Greiner T.C. Grody W.W. Hambuch T. Kalman L. Kant J.A. Klein R.D. Leonard D.G. Lubin I.M. Mao R. Nagan N. Pratt V.M. Sobel M.E. Voelkerding K.V. Gibson J.S. Opportunities and challenges associated with clinical diagnostic genome sequencing: a report of the Association for Molecular Pathology.J Mol Diagn. 2012; 14: 525-540Abstract Full Text Full Text PDF PubMed Scopus (122) Google Scholar Hence, our approach was to maximize the extent of what is addressable by NGS and to ensure detection of all clinically important mutations. Because NGS has the ability to detect additional sequence variants in genomic regions sequenced at high quality and depth, we established a pathway for assessment of novel, reportable variants whereby a patient's DNA sequence is scanned for variants that meet one or more of the following criteria: i) occurs at a conserved donor or acceptor splice site (±2 bases of intron), ii) generates a premature stop codon (nonsense mutation), or iii) generates a frame-shift in the protein sequence. When a stop codon or frame-shift mutation is present, the position of the mutation relative to the 3′ most truncating mutation previously described for the disorder also is taken into account. These mutations are reported as predicted to be pathogenic and further increase the detection rates for each disorder beyond what has been reported previously as pathogenic. Before beginning the validation of the clinical NGS assay, it was necessary to define acceptable analytic sensitivity and specificity criteria. To use an NGS assay for carrier screening, we required a high level of clinical confidence that the assay would not miss carriers. Hence, we were most concerned about the false-negative rate. All positive results were confirmed by Sanger sequencing, thereby eliminating false-positive results before clinical reporting. Studies detailing assay design and analytic accuracy in a research setting already have been described.1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar In brief, it was shown that NGS could achieve a false-negative rate of 2.52 × 10−4 (95% Wilson binomial confidence interval, 1.29 × 10−5 1.42 × 10−3) for single-nucleotide variants; a single false-negative call occurred in a sample previously characterized as aneuploid.9Locke D.P. Sharp A.J. McCarroll S.A. McGrath S.D. Newman T.L. Cheng Z. Schwartz S. Albertson D.G. Pinkel D. Altshuler D.M. Eichler E.E. Linkage disequilibrium and heritability of copy-number polymorphisms within duplicated regions of the human genome.Am J Hum Genet. 2006; 79: 275-290Abstract Full Text Full Text PDF PubMed Scopus (261) Google Scholar For insertions, deletions, or the more complex mutations that are indels, three false-negative results occurred. However, there were no false-negative results for any position in the sequence where a pathogenic mutation of interest occurred, as defined in our variant database, and/or that was not covered by an alternate methodology. Our goal therefore was to show a false-negative rate of zero in the validation studies described later. As mentioned earlier, in this study, Sanger sequencing was used as the comparator method for the analytic accuracy studies, despite the high cost and resource burden of fully sequencing each sample for each of the genes in the NGS panel. This can be leveraged for future validations. For the purpose of clinical testing, it is necessary to be able to Sanger sequence any amplicon for any gene that is assessed by NGS to confirm the presence of the mutation, and therefore we also validated Sanger sequencing for approximately 250 amplicons. To offer a clinical-grade carrier screening assay with high accuracy and precision (reproducibility), robustness, throughput, and a rapid turnaround time, it was necessary to complete a thorough validation. At the time our validation plan was set, there were no published guidelines on validation of NGS assays. Instead, we used a blended approach of professional society recommendations and the experience of the technical, clinical, and bioinformatics internal teams to devise an approach. Since completing these studies, one report8Schrijver I. Aziz N. Farkas D.H. Furtado M. Gonzalez A.F. Greiner T.C. Grody W.W. Hambuch T. Kalman L. Kant J.A. Klein R.D. Leonard D.G. Lubin I.M. Mao R. Nagan N. Pratt V.M. Sobel M.E. Voelkerding K.V. Gibson J.S. Opportunities and challenges associated with clinical diagnostic genome sequencing: a report of the Association for Molecular Pathology.J Mol Diagn. 2012; 14: 525-540Abstract Full Text Full Text PDF PubMed Scopus (122) Google Scholar has been published that discusses the issues related to NGS assay validation, and a second publication10Gargis A.S. Kalman L. Berry M.W. Bick D.P. Dimmock D.P. Hambuch T. et al.Assuring the quality of next-generation sequencing in clinical laboratory practice.Nat Biotechnol. 2012; 30: 1033-1036Crossref PubMed Scopus (365) Google Scholar has offered recommendations that are aligned with our approach. In addition, the College of American Pathologists checklists, used by many clinical laboratories to monitor practices in support of accreditation, have been updated to include items specific to NGS assays (http://www.cap.org/apps/docs/laboratory_accreditation/checklists/2012_checklist_brochure.pdf, last accessed July 27, 2013). The clinical assay described in this article has been in use for more than 1.5 years. To date, we have screened more than 11,000 patients (from in vitro fertilization clinics across the United States) for carrier status, and found almost 500 carriers in 14 disease-causing genes. We also have identified seven novel variants, including one in CFTR. We outline our clinical experience with this assay later, and discuss the lessons learned and describe the mutations detected thus far in this cohort of patients. Genomic DNA was purchased from the Coriell Cell Repositories (Camden, NJ) or isolated from whole blood by the Gentra Puregene method (Qiagen, Valencia, CA). DNA was quantitated using a NanoQuant Plate (Tecan, San Jose, CA). Nuclease-free water (Sigma, St. Louis, MO) and two DNA controls, NA11284, which contains mutations in the CFTR gene (p.F508del/p.R560T), and NA00502, which contains mutations in the HEXA gene (c.1278insTATC/c.1421+1 G>C), were purchased from Coriell Cell Repositories and were included in all runs. Next-generation DNA sequencing has been described previously.1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar Briefly, multiplex target capture using tiled, molecular inversion probes is followed by incorporation of molecular barcodes and Illumina (San Diego, CA) sequencing adapters. The product then is sequenced using the Illumina Hiseq2000 system. Data analysis proceeds using a combination of open-source and internally developed tools for sample demultiplexing, read alignment, genotype calling, and functional annotation. Subsequently, additional quality metrics are applied at the assay, sample, and variant levels, and nonreference calls are filtered using an in-house–developed database of variants. Details of reagents and equipment used for the validation studies are provided in Supplemental Tables S1 and S2. Three different types of quality scores were used to assess data in this validation: sample quality scores are indicative of the overall quality of a given sample's data, whereas the run quality scores are indicative of the overall quality of a run's data. Variant quality scores were used to assess sample-specific variant calls. Data that did not pass run or sample quality score criteria were marked as a “failed analysis” and were not evaluated further. Sample quality score 1 (SQS1) is a fraction of callable bases: a genomic position is defined as “callable” if the call for this position passes multiple quality filters including having a depth greater than 50. The fraction of callable bases is the number of bases considered callable in the sample divided by the number of bases targeted by the assay. It is also the number of bases that pass both QS1 and QS2 thresholds (see variant quality scores, later). The threshold is 0.9 (90% of captured bases are callable). In clinical practice, any sample with less than 99% of bases callable is repeated to obtain a more complete sequence. SQS2 is the number of uncallable variant database entries, that is, the number of Good Start Genetics, Inc. (GSG) variant database entries that were not callable for a given sample. The threshold is 10 (of 981) uncallable database positions. In clinical practice any uncalled position is sequenced by Sanger methodology. The run quality score 1 (RQS1) is the number of samples failing SQS1 or SQS2. The threshold is 20; RQS2 is the number of discordant genotype calls for control 1 (NA11284), the threshold is 0; RQS3 is the number of discordant genotype calls for control 2 (NA00502), the threshold is 0; RQS4, 5, and 6 are related to the detection of sample contamination from unexpected molecular barcodes, library barcode sequence cross-over rate, and mapped negative control reads, respectively; and RQS7 is the number of samples with zero associated reads, the threshold is 1. To identify low-confidence calls, we used thresholds on two quality scores (statistics) associated with each genotype call. These were depth of coverage (QS1) and strand bias (QS2). Thresholds for each of these parameters have been set empirically using data from previous analytic studies.1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar Any genotype call with a genomic position in the variant database that does not meet these criteria is flagged as uncallable for the sample under consideration. When runs included set 1 samples (fully characterized in a previous study1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar), data were subject to cross-validation analysis. All sequence changes that differed from reference, irrespective of the clinical relevance, were assessed. Runs were compared sample-by-sample. Each variant was given one of the following mutually exclusive labels: true positive if the variant was reported at the same genomic position with the same lesion (alternative sequence) in the same sample; false positive (FP) if the variant was detected at a given position and sample in the validation for clinical use data, but not in the previous study; and false negative (FN) if the variant was detected at a given position and sample in the previous study, but not in the validation for clinical use data. Each FP variant was subject to a manual review of next-generation sequence data (reads) and Sanger reference data (if available). Systematic sequencing artifacts that resulted in frequent FP variant calls were recorded and performance statistics were calculated for each run. PCR primers were developed for each region of interest. Briefly, 30-μL reactions were conducted with 100 ng of genomic DNA, 1 U of AmpliTaq Gold (Applied Biosystems, Grand Island, NY), and 1 μmol of each PCR primer in a PCR mix containing 5% dimethyl sulfoxide (vol/vol), 1 mol/L betaine, 2.5 mmol/L magnesium chloride, 1 μmol/L dNTPs (total), and 1× GeneAmp PCR Gold Buffer (Applied Biosystems, Brea, CA). Cycling conditions were as follows: 95°C for 10 minutes, 30× (95°C for 30 seconds, 60°C for 30 seconds, 72°C for 30 seconds), 72°C for 10 minutes, and 8°C forever. PCR primers used were as described by Jones et al,11Jones S. Zhang X. Parsons D.W. Lin J.C. Leary R.J. Angenendt P. et al.Core signaling pathways in human pancreatic cancers revealed by global genomic analyses.Science. 2008; 321: 1801-1806Crossref PubMed Scopus (3152) Google Scholar except M13 tails were removed. PCR products were purified using AMPure Beads (Beckman Coulter, Brea, CA), and chain termination bidirectional Sanger sequencing was performed on an ABI 3730xl according to standard protocols. Data were analyzed using mutation surveyor (SoftGenetics, State College, PA). Two sets of samples were used for the clinical validation. Set 1 consisted of samples thoroughly characterized by Sanger sequencing during previous studies.1Umbarger M.A. Kennedy C.J. Saunders P. Breton B. Chennagiri N. Emhoff J. Greger V. Hallam S. Maganzini D. Micale C. Nizzari M. Towne C. Church G.M. Porreca G.J. Next-generation carrier screening.Genet Med. 2013; ([Epub ahead of print])http://dx.doi.org/10.1038/gim.2013.83PubMed Google Scholar They consisted of a mix of HapMap/Human Variation Panel samples (to ensure representation of different ethnic groups) and samples containing known mutations for the genes of interest. A list of the set 1 samples is provided in Supplemental Table S3. A second set of samples, set 2, consisted of Coriell DNA with known mutation status, 10 DNA samples from external laboratories (known mutation status; various extraction methodologies), and 49 blood samples extracted at GSG. There was some overlap in the Coriell samples used between sets 1 and 2. A list of the set 2 samples is provided in Supplemental Table S4. Overall, 221 unique samples were contained within sets 1 and 2. The validation for clinical use (“clinical validation,” subsequently referred to as “CV”) for the NGS assay was performed as summarized in Table 1. Each study is described in more detail later. All studies passed the RQS criteria.Table 1Summary of Validation StudiesRun IDPurposeNotesSamplesCV1-1AccuracySet 1CV1-2AccuracySet 2CV2-1Lot-to-lot (lot 1 vs lot 2)Reagent lot 2 (different critical reagents; sub-lot of lot 1 MIPs)∗Molecular inversion probes (MIPs) are phosphorylated as a batch before use, which generates a sub-lot. CV2-1 used the lot 1 MIPs but a different sub-lot (compared with CV1-1). CV2-2 used a completely new lot of MIPs.Subset of set 1CV2-2Lot-to-lotReagent lot 1; MIP lot 2Subset of set 1CV4Limit of detectionCV4 and CV6-2 are on the same runSubset of set 1 samples at different concentrationsCV5Temperature assessmentCV5-2 and CV7 are on the same runBlood samples (not set 1 or 2)CV6-1ReproducibilityIntra-assay: day 1Subset of set 1 in duplicateCV6-2ReproducibilityInterassay: day 2Subset of set 1CV6-3ReproducibilityDifferent operator and instrument: day 3Subset of set 1 in duplicateCV7Blinded accuracySubset of sets 1 and 2Unless otherwise stated, reagent lot 1, operator 1, and instrument 1 were used. CV1-1 was used as the lot 1 run for comparison in reagent lot-to-lot studies.∗ Molecular inversion probes (MIPs) are phosphorylated as a batch before use, which generates a sub-lot. CV2-1 used the lot 1 MIPs but a different sub-lot (compared with CV1-1). CV2-2 used a completely new lot of MIPs. Open table in a new tab Unless otherwise stated, reagent lot 1, operator 1, and instrument 1 were used. CV1-1 was used as the lot 1 run for comparison in reagent lot-to-lot studies. The general acceptance criteria for these studies were as follows: i) all previously defined, clinically relevant mutations must be detected as expected (false-negative rate of zero at clinically relevant positions) for all samples that met the SQS1 and 2 criteria; ii) the number of failing runs must be one or less for the planned studies; iii) the overall sample fail rate (samples failing SQS1 or SQS2) must be <3% (with the exception of the limit of detection and temperature assessment studies). Additional criteria were applied when specified for each study described later. Note that false-positive results were not considered from an acceptance criteria perspective because Sanger sequencing would be used in clinical practice to determine the true status. Therefore, the main concern was whether the amount of Sanger sequencing generated could be managed in the laboratory given resource constraints. This is discussed further later. A summary of the expected versus observed results for all set 1 and 2 samples across all studies is provided in Supplemental Table S5. False-positive results are not shown in Supplemental Table S5 because these were assessed by Sanger sequencing and hence would be corrected before clinical result reporting. All runs passed the RQS criteria and hence this acceptance criterion was met. All set 1 and set 2 samples were run through the assay, in experiments CV1-1 and CV1-2, respectively. Assessing the accuracy data for all samples of known genotypes from a clinical perspective, all expected mutations were detected (ie, there were zero FNs). Therefore, our accuracy acceptance criteria were met. In addition, all next-generation sequenced base positions (irrespective of clinical relevance) were assessed for each sample in the CV1-1 data set and were compared with their respective counterparts in the known sequence to assess the concordance and detection rates. This was possible because all samples were previously fully characterized, as noted earlier. In other words, all base positions in the sequence for each sample were compared for genotype concordance. There were no FN and nine FP variant calls. Hence, the genotype concordance met the acceptance criterion of 99.999%, as shown:(1−9FP6,403,266callable bases)×100%=99.9998%(1) The primary purpose of the blood samples included in set 2 was to assess the overall quality and callability in samples extracted by our methodology versus the Coriell cell line DNA. The comparison of DNA source (blood or Coriell) using distribution of variant level parameters showed that there was no significant variation in quality distributions for depth of coverage (QS1, P = 0.20), strand bias (QS2, P = 0.59), or quality over depth (QS3, P = 0.48) (Figure 1). P values were calculated using the Wilcoxon rank-sum test with continuity correction. Note that three reportable mutations were detected by NGS in the unknown blood samples (set 2); all three were confirmed with Sanger sequencing. A subset of samples from set 1 were run in duplicate in a single assay (intrarun reproducibility). This same subset then was run on a separate day by the same operator using the same reagents and instruments (interrun reproducibility). The intrarun reproducibility study was replicated using a different instrument and operator (instrument and operator reproducibility). Note that although multiple samples can be processed simultaneously in an NGS run, validation studies require multiple runs to assess different conditions. Because the cost and time required for each run is high, it is necessary to find the most efficient manner possible to assess all required variables. Therefore, multiple parameters often were assessed in a run when possible and reasonable because of these cost constraints. This approach also is used for performing reagent prequalification; parameters subsequently are separated only when the assay parameters do not pass acceptance criteria. The use of set 1 samples for this study permitted cross-validation to previous data and showed zero FN results, therefore showing good precision of the assay. More information is provided in Table 2, which provides a summary of the FP and FN results for each CV run containing set 1 samples. The raw true positive, FP, and FN values represent all sequence changes observed across all regions detected, irrespective of whether the sequence change was clinically significant. The reported FP and FN values are based only on positions that are considered clinically important. Note that for each run, some samples were not assessed because they failed the callability criteria (SQS1). One FN variant was observed, a deletion mutation BCKDHA c.861_868delAGGCCCCG, in CV runs CV2-2 and CV6-1. It was unclear if this was owing to the quality of the DNA sample or the variant itself. Because this var