Portal de Periódicos da CAPES

Milestones: Direct Observation May Be the Key to Accelerated Training

2013; Elsevier BV; Volume: 126; Issue: 12 Linguagem: Inglês

10.1016/j.amjmed.2013.05.004

1555-7162

Stephen J. Peterson, Christopher Nabors, William H. Frishman,

Medical Education and Admissions

The United States faces a worsening shortage of physicians, particularly in primary care.1Petterson S.M. Liaw W.R. Phillips Jr., R.L. Rabin D.L. Meyers D.S. Bazemore A.W. Projecting US primary care physician workforce needs: 2010-2025.Ann Fam Med. 2012; 10: 503-509Crossref PubMed Scopus (431) Google Scholar, 2Carek P.J. Peterson L. Shokar N.K. Johnson S.P. Knoll M.E. Mainous 3rd, A.G. Graduate medical education and primary care workforce: a CERA study.Fam Med. 2012; 44: 712-715PubMed Google Scholar The number of medical school positions has increased, but without a corresponding expansion of residency slots. As such, the number of American medical graduates is rising while the total number of postgraduate positions remains stable. Short of turning over primary care to nurse practitioners and physician assistants, we must be open to the idea of accelerating the pace of medical training. Emanuel and Fuchs3Emanuel E.J. Fuchs V.R. Shortening medical training by 30%.JAMA. 2012; 307: 1143-1144Crossref PubMed Scopus (113) Google Scholar recently made this argument, pointing out that reductions in medical school and residency duration are already underway. They suggested that certain aspects of third-year medicine residency, such as supervising interns and taking electives, are “hardly essential to becoming a knowledgeable practitioner.” And they concluded that “the third year of internal medicine or pediatric residencies or the research year in surgical specialties could be eliminated without compromising the clinical quality of trainees.” While their contentions may be largely correct, few would disagree that some medical residents require more than 2 years of supervised clinical experience before being deemed ready for fellowship or entry into independent practice. The challenge to the medical system, then, is to tailor training duration to actual clinical ability. In turn, educators must be able to assess trainee competence confidently and accurately. The Outcomes Project, which was introduced over 10 years ago, appeared to be the logical framework for such determinations. Yet full implementation has been slowed by a number of factors4Green M.L. Holmboe E. The ACGME toolbox: half empty or half full?.Acad Med. 2010; 85: 787Crossref PubMed Scopus (39) Google Scholar, 5Heard J.K. Allen R.M. Clardy J. Assessing the needs of residency program directors to meet the ACGME general competencies.Acad Med. 2002; 77: 750Crossref PubMed Scopus (57) Google Scholar, 6Malik M.U. Diaz V.V. Stewart C.M. et al.Barriers to implementing the ACGME outcome project: a systematic review of program director surveys.Journal of Graduate Medical Education. 2012; 4: 425-433Crossref PubMed Google Scholar and the premise of the Outcomes Project challenged.7Huddle T.S. Heudebert G.R. Taking apart the art: the risk of anatomizing clinical competence.Acad Med. 2007; 82: 536-541Crossref PubMed Scopus (95) Google Scholar, 8Whitcomb M.E. Redirecting the assessment of clinical competence.Acad Med. 2007; 82: 527-528Crossref PubMed Scopus (21) Google Scholar, 9Lurie S.J. Mooney C.J. Lyness J.M. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review.Acad Med. 2009; 84: 301-309Crossref PubMed Scopus (255) Google Scholar Lurie et al9Lurie S.J. Mooney C.J. Lyness J.M. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review.Acad Med. 2009; 84: 301-309Crossref PubMed Scopus (255) Google Scholar argued that with the exception of medical knowledge, assessments of residents could not reliably distinguish between levels of competence within the 6 general domains. On the other hand, Green and Holmboe4Green M.L. Holmboe E. The ACGME toolbox: half empty or half full?.Acad Med. 2010; 85: 787Crossref PubMed Scopus (39) Google Scholar have contended that “the inconsistent use and interpretation of [available instruments] by unskilled faculty” is more contributory to Lurie's findings than a lack of suitable evaluation tools. To address concerns with the Outcomes Project, the American Board of Internal Medicine and Accreditation Council for Graduate Medical Education (ACGME) jointly developed educational milestones.10Green M.L. Aagaard E.M. Caverzagie K.J. et al.Charting the road to competence: Developmental milestones for internal medicine residency training.J Grad Med Educ. 2009; 1: 5-20Crossref PubMed Google Scholar By subdividing the general competencies into more specific components, the milestones were intended to permit more convincing measures of clinical competence. Shortly after their introduction, our program, in collaboration with other members of the Educational Innovations Project (EIP),11Accreditation Council for Graduate Medical Education. Educational Innovation Project. Available at: http://www.acgme.org/acWebsite/RRC_140/140_EIPindex.asp. Accessed April 2, 2013.Google Scholar began to use them for clinical assessments. The EIP initiative used 8 milestones to guide direct observations of clinic performance that informed the entrustment decision for “ready for distance supervision.”12Heist K. Post J. Meade L. Brandenburg S. Milestones: do learners and teachers agree?.Am J Med. 2013; 126: 270-274Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar At the same time, our program broadly incorporated milestones within all clinical rotations, except certain subspecialty electives. Our initiative engaged faculty in all phases of curricular redesign, development, implementation, tool construction and use, and data management and consideration at higher-order summative evaluations (eg, clinical competency committee deliberations). For almost 3 years, we have considered curricular-milestones-based data at a series of landmark assessments (Entrustable Professional Activities13ten Cate O. Entrustability of professional activities and competency-based training.Med Educ. 2005; 39: 1176-1177Crossref PubMed Scopus (510) Google Scholar) to evaluate resident progress. Both faculty and residents have indicated through surveys and end-of-rotation feedback that milestones promoted a common understanding of what knowledge, skills, and attitudes should be displayed at particular stages of training, permitted greater specificity in performance feedback, and enhanced uniformity and fairness in promotion decisions. We successfully utilized curricular-milestones-based data to complete synthetic evaluations of trainee progress at the clinical competency level, where the use of curricular milestones allowed us to discern statistically significant differences in milestones-defined domains of clinical competence.14Nabors C. Peterson S.J. Forman L. et al.Operationalizing the internal medicine milestones-an early status report.J Grad Med Educ. 2013; 5: 130-137Crossref PubMed Google Scholar Surveyed committee members indicated that milestones improved the quality of information, promoted more uniform evaluation standards, and enhanced the ability of the committee to discriminate between specific strengths and weaknesses of individual residents. More recently, we conducted a pilot test of the Alliance for Academic Internal Medicine redesign group's recently released reporting milestones.15Caverzagie K.J. Iobst W.F. Aagaard E.M. et al.The internal medicine reporting milestones and the next accreditation system.Ann Intern Med. 2013; 158: 557-559Crossref PubMed Scopus (52) Google Scholar Using curricular-milestones-data organized by reporting milestone narrative, core competency and ACGME competency, we were able to effectively identify training attainment levels for a sample of our current residents. Two years ago, when Green and Holmboe4Green M.L. Holmboe E. The ACGME toolbox: half empty or half full?.Acad Med. 2010; 85: 787Crossref PubMed Scopus (39) Google Scholar asked whether the toolbox for competency-based assessment was half full or half empty, the answer was uncertain. Since then, the curricular milestones have identified the medical community's belief as to what knowledge, skills, and attitudes a trainee should display at particular times in medical training. These represent the “explicitly stated set of expectations that… link the ideals of the general competencies to the realities of measurement” called for by Lurie et al.9Lurie S.J. Mooney C.J. Lyness J.M. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review.Acad Med. 2009; 84: 301-309Crossref PubMed Scopus (255) Google Scholar Programs are now rapidly developing milestones-based evaluation tools within a unifying framework that allows for innovative approaches while also promoting standardization. Given the scope of the project, best practices and favored evaluation strategies should be identified quickly. The reporting milestones carry competency assessments one step further by providing the descriptive language by which curricular milestones-based data can be synthesized at higher-order summative evaluations to accurately mark the course of resident skill acquisition.16Nasca T.J. Philibert I. Brigham T. Flynn T.C. The next GME accreditation system—rationale and benefits.N Engl J Med. 2012; 366: 1051-1056Crossref PubMed Scopus (1030) Google Scholar In order for this new outcomes-based model of evaluation to reach its full potential, we believe that programs will require additional support for development of expert faculty evaluators who will be able to spend more time directly observing, guiding, and rating resident clinical activities. Program directors and clinical competency committees must then be able to fully consider the new stream of evaluation data. While calls for more resources in a time of belt tightening might seem whimsical, this training model has worked well within the EIP, where all program directors were guaranteed a minimum 0.75 Full-Time Equivalent. Our institution's EIP-related support made possible robust faculty participation that was critical to the development of a shared mental framework that guided evaluation tool construction and promoted focused and more numerous direct observations of clinical performance. This allowed us to integrate curricular milestones into our training 2 years ahead of the Next Accreditation System mandate while carrying out other quality improvement and patient safety initiatives at the same time.17Nabors C. Peterson S.J. Weems R. et al.A multidisciplinary approach for teaching systems-based practice to internal medicine residents.J Grad Med Educ. 2011; 3: 75-80Crossref PubMed Google Scholar, 18Nabors C. Peterson S.J. Lee W.N. et al.Experience with faculty supervision of an electronic resident sign-out system.Am J Med. 2010; 123: 376-381Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar, 19Nabors C. Peterson S.J. Aronow W.S. et al.Physician reporting of clinically significant events through a computerized patient sign-out system.J Patient Saf. 2011; 7: 155-161Crossref PubMed Scopus (8) Google Scholar We are convinced that the tools needed for accurate identification of trainees suitable for rapid advancement are at hand. As such, resources devoted to making sound promotion decisions would be well utilized. If each training program fast-tracks one fourth of its residents to graduate in 2 rather than 3 years, the nation's physician shortage will be mitigated and funds allocated to more effective outcomes assessments will be repaid. In short, the suite of evaluation tools now available to medicine residency program directors tips the half-full/half-empty balance far towards full. As residency programs integrate curricular and narrative milestones-based assessments into their training, they also should give thought to the next phase of outcomes-based assessment that will involve short-track pathways for suitably competent residents. Dr Nabors thanks Dr Howard Kerpen, Director of the Lorber Center for the Advancement of Medical Education, Long Island Jewish Medical Center, Lorber Professor of Medicine, Albert Einstein College of Medicine, Clinical Professor of Medicine, Hofstra North Shore-Long Island Jewish Health System, and the Foundation for Innovation in Medical Education for generously supporting our efforts to advance competency-based medical education.