Portal de Periódicos da CAPES

External Validation of a Thiopurine Monitoring Algorithm on the SONIC Clinical Trial Dataset

2017; Elsevier BV; Volume: 16; Issue: 3 Linguagem: Inglês

10.1016/j.cgh.2017.08.021

1542-7714

Akbar K. Waljee, Kay Sauder, Yiwei Zhang, Ji Zhu, Peter Higgins,

Pancreatic and Hepatic Oncology Research

Several reports leverage patterns in electronic medical record data to create algorithms to optimize outcomes in healthcare.1Waljee A.K. Higgins P.D.R. Machine learning in medicine: a primer for physicians.Am J Gastroenterol. 2010; 105: 1224-1226Google Scholar, 2Waljee A.K. Joyce J.C. Wang S. et al.Algorithms outperform metabolite tests in predicting response of patients with inflammatory bowel disease to thiopurines.Clin Gastroenterol Hepatol. 2010; 8: 143-150Google Scholar, 3Waljee A.K. Sauder K. Patel A. et al.Machine learning algorithms for objective remission and clinical outcomes with thiopurines.J Crohns Colitis. 2017; 11: 801-810Google Scholar More recently, there has been an increasing call for the use of clinical trial data repositories.4Rosenbaum L. Bridging the data-sharing divide - seeing the devil in the details, not the other camp.N Engl J Med. 2017; 376: 2201-2203Google Scholar Open clinical trial data sharing has allowed us to evaluate the generalizability of algorithms to predict response to thiopurines in patients with inflammatory bowel disease (IBD). Yale University Open Data Access (http://yoda.yale.edu/) is a data-sharing platform that provides access to multiple clinical trial datasets. Previously, we internally validated machine learning algorithms for both clinical and biological remission among patients on thiopurines.2Waljee A.K. Joyce J.C. Wang S. et al.Algorithms outperform metabolite tests in predicting response of patients with inflammatory bowel disease to thiopurines.Clin Gastroenterol Hepatol. 2010; 8: 143-150Google Scholar, 3Waljee A.K. Sauder K. Patel A. et al.Machine learning algorithms for objective remission and clinical outcomes with thiopurines.J Crohns Colitis. 2017; 11: 801-810Google Scholar We sought to externally validate the previously developed algorithm to predict objective remission in the SONIC clinical trial of azathioprine, infliximab, or the combination of azathioprine and infliximab in Crohn's disease using open clinical trial data. Thiopurines continue to be used as monotherapy or as part of combination therapy for the treatment of IBD. Worldwide, thiopurines are frequently used as a monotherapy as they remain a low-cost option for steroid-sparing therapy. Experts in IBD rely on patterns in the complete blood count and differential (CBCD) and comprehensive chemistry (COMP) panels to monitor their patients, as thiopurine metabolites perform poorly in the evaluation of clinical response.5Osterman M.T. Kundu R. Lichtenstein G.R. et al.Association of 6-thioguanine nucleotide levels and inflammatory bowel disease activity: a meta-analysis.Gastroenterology. 2006; 130: 1047-1053Google Scholar Metabolites have also failed to show benefit in 2 prospective randomized controlled trials.6Dassopoulos T. Dubinsky M.C. Bentsen J.L. et al.Randomised clinical trial: individualised vs. weight-based dosing of azathioprine in Crohn's disease.Aliment Pharm Ther. 2014; 39: 163-175Google Scholar, 7Reinshagen M. Schutz E. Armstrong V.W. et al.6-thioguanine nucleotide-adapted azathioprine therapy does not lead to higher remission rates than standard therapy in chronic active crohn disease: results from a randomized, controlled, open trial.Clin Chem. 2007; 53: 1306-1314Google Scholar Patterns in the CBCD and COMP can be successfully used to predict objective remission with high accuracy, and have been internally validated.3Waljee A.K. Sauder K. Patel A. et al.Machine learning algorithms for objective remission and clinical outcomes with thiopurines.J Crohns Colitis. 2017; 11: 801-810Google Scholar To externally validate this algorithm for objective remission, we used data from the SONIC multicenter, randomized, double-blind controlled trial in patients with moderate-to-severe Crohn's disease, treated in 3 arms with infliximab plus azathioprine, infliximab, or azathioprine (ClinicalTrials.gov, NCT00094458). We applied our algorithm to predict objective remission on thiopurine monotherapy in IBD to the SONIC trial dataset to provide an external validation. We hypothesized that this algorithm would be minimally helpful (area under the receiver-operating characteristic curve [AuROC] ≈ 0.50) for patients in the infliximab alone arm, modestly helpful (AuROC ≈ 0.60) for patients in the combination (infliximab + azathioprine) arm, and quite helpful (AuROC ≈ 0.75) in the patients treated with azathioprine monotherapy. The initial trial randomized 508 patients, and details of this study can be found on the ClinicalTrials.gov website and in the corresponding publication.8Colombel J.F. Sandborn W.J. Reinisch W. et al.Infliximab, azathioprine, or combination therapy for Crohn's disease.N Engl J Med. 2010; 362: 1383-1395Google Scholar We selected only those patients who had objective inflammation at entry to the study, defined by a C-reactive protein (CRP) >6 mg/L or documented endoscopic ulceration (ULCRTN field = yes). Subjects who did not complete the week 26 visit (withdrawn) or those with no recorded CRP or ULCRTN value at week 26 (missing data) were not included in our analysis. In total, there were 244 inflamed subjects with complete data. Objective remission was defined by the absence of objective inflammation as defined by a CRP ≤6 mg/L and a response of "no" for the ULCRTN field at week 26. Subjects not in objective remission were defined based on a CRP >6 mg/L or ULCRTN field response of "yes." Of the 244 subjects analyzed, 77 achieved objective remission and 167 did not (Figure 1A). Independent predictor variables included all of the values from the CBCD and COMP panels, and the patient's age calculated as the exact number of years. Using the previously developed algorithm, we externally tested the performance of this algorithm in this dataset.3Waljee A.K. Sauder K. Patel A. et al.Machine learning algorithms for objective remission and clinical outcomes with thiopurines.J Crohns Colitis. 2017; 11: 801-810Google Scholar The performance of the algorithm was evaluated using the AuROC. Using a previously defined3Waljee A.K. Sauder K. Patel A. et al.Machine learning algorithms for objective remission and clinical outcomes with thiopurines.J Crohns Colitis. 2017; 11: 801-810Google Scholar cutoff of 0.564, we were able to predict which patients achieved objective remission on thiopurine monotherapy with an AuROC of 0.76 (Figure 1B). The performance characteristics of this algorithm were also evaluated for subjects in the combination (infliximab and azathioprine) arm (AuROC = 0.67), and for subjects in the infliximab monotherapy arm (AuROC = 0.60), with diminishing results, as expected. These thiopurine algorithms provide an accurate, rapid, low-cost, and generalizable alternative to thiopurine metabolites to evaluate objective remission in patients with IBD on thiopurine therapy. These algorithms have been implemented and automated in an Epic electronic medical record (Epic Systems Corporation, Verona, WI) and standard clinical workflow. In addition, we are now in the process of engaging other hospitals, predominantly in Taipei and Hong Kong where thiopurine use is more prevalent than in the United States, to do a prospective validation study using our algorithms. It should be noted that there are limitations to our study that include the retrospective nature of evaluating our algorithm, as well as the generalizability. We hope that these issues will be addressed in a prospective study. This study, YODA Project #2014-0401, used data from the Yale University Open Data Access Project, which has an agreement with Janssen Research & Development, LLC. The interpretation and reporting of research using this data are solely the responsibility of the authors and does not necessarily represent the official views of the Yale University Open Data Access Project or Janssen Research & Development, LLC.