Portal de Periódicos da CAPES

Data-driven methods distort optimal cutoffs and accuracy estimates of depression screening tools: a simulation study using individual participant data

2021; Elsevier BV; Volume: 137; Linguagem: Inglês

10.1016/j.jclinepi.2021.03.031

1878-5921

Parash Mani Bhandari, Brooke Levis, Dipika Neupane, Scott B. Patten, Ian Shrier, Brett D. Thombs, Andrea Benedetti, Kuan‐Pin Su, Chen He, Danielle B. Rice, Ankur Krishnan, Yin Wu, Marleine Azar, Tatiana Sanchez, Matthew J. Chiovitti, Nazanin Saadat, Kira E. Riehm, Mahrukh Imran, Zelalem Negeri, Jill Boruff, Pim Cuijpers, Simon Gilbody, John P. A. Ioannidis, Lorie A. Kloda, Roy C. Ziegelstein, Liane Comeau, Nicholas Mitchell, Marcello Tonelli, Simone N. Vigod, Franca Aceti, Rubén Alvarado, Cosme Alvarado‐Esquivel, Muideen O. Bakare, Jacqueline Barnes, Amar Bavle, Cheryl Tatano Beck, Carola Bindt, Philip Boyce, Adomas Bunevičius, Tiago Castro e Couto, Linda H. Chaudron, Humberto Corrêa, Felipe Pinheiro de Figueiredo, Valsamma Eapen, Nicolas Favez, Ethel Felice, Michelle Fernandes, Bárbara Figueiredo, Jane Fisher, Lluïsa García-Esteve, Lisa Giardinelli, Nadine Helle, Louise M. Howard, Dina Sami Khalifa, Jane Kohlhoff, Zoltán Kozinszky, Laima Kusminskas, Lorenzo Lelli, Angeliki Leonardou, Michaël Maes, Valentina Meuti, Sandra Nakić Radoš, Purificación Navarro García, Daisuke Nishi, Daniel Okitundu Luwa E‐Andjafono, Susan Pawlby, Chantal Quispel, Emma Robertson‐Blackmore, Tamsen Rochat, Heather Rowe, Debbie Sharp, Bonnie W.M. Siu, Alkistis Skalkidou, Alan Stein, Robert C. Stewart, Kuan‐Pin Su, Inger Sundström Poromaa, Meri Tadinac, S. Darius Tandon, Iva Tendais, Pavaani Thiagayson, Annamária Töreki, A. Torres, Thach Tran, Kylee Trevillion, Katherine Turner, Johann M. Vega‐Dienstmaier, Karen Wynter, Kimberly A. Yonkers,

Child and Adolescent Psychosocial and Emotional Development

Objective: To evaluate, across multiple sample sizes, the degree that data-driven methods result in (1) optimal cutoffs different from population optimal cutoff and (2) bias in accuracy estimates.Study design and setting: A total of 1,000 samples of sample size 100, 200, 500 and 1,000 each were randomly drawn to simulate studies of different sample sizes from a database ( n = 13,255) synthesized to assess Edinburgh Postnatal Depression Scale (EPDS) screening accuracy.Optimal cutoffs were selected by maximizing Youden's J (sensitivity + specificity-1).Optimal cutoffs and accuracy estimates in simulated samples were compared to population values.Results: Optimal cutoffs in simulated samples ranged from ≥ 5 to ≥ 17 for n = 100, ≥ 6 to ≥ 16 for n = 200, ≥ 6 to ≥ 14 for n = 500, and ≥ 8 to ≥ 13 for n = 1,000.Percentage of simulated samples identifying the population optimal cutoff ( ≥ 11) was 30% for n = 100, 35% for n = 200, 53% for n = 500, and 71% for n = 1,000.Mean overestimation of sensitivity and underestimation of specificity were 6.5 percentage point (pp) and -1.3 pp for n = 100, 4.2 pp and -1.1 pp for n = 200, 1.8 pp and -1.0 pp for n = 500, and 1.4 pp and -1.0 pp for n = 1,000.Conclusions: Small accuracy studies may identify inaccurate optimal cutoff and overstate accuracy estimates with data-driven methods.