Milestones for Nephrology Training Programs: A Modest Proposal
2013; Elsevier BV; Volume: 62; Issue: 6 Linguagem: Inglês
10.1053/j.ajkd.2013.06.022
ISSN1523-6838
AutoresChristina M. Yuan, Robert Nee, Kevin C. Abbott, James D. Oliver,
Tópico(s)Clinical Reasoning and Diagnostic Skills
ResumoThe Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System in order to improve future physician practice using the peer-review system, base training program accreditation decisions on trainee educational outcomes, and reduce administrative burdens associated with the present system.1Nasca T.J. Philibert I. Brigham T. Flynn T.C. The next GME accreditation system—rationale and benefits.N Engl J Med. 2012; 366: 1051-1056Crossref PubMed Scopus (1035) Google Scholar Specialty-specific milestones will be introduced to monitor the progress of trainees in achieving acceptable performance in the 6 competencies: Patient Care (PC), Medical Knowledge (MK), Professionalism (P), Interpersonal Communication Skills (IPCS), Practice-Based Learning and Improvement (PBLI), and Systems Based Practice (SBP). Each must be mastered to a level that allows the trainee to practice unsupervised. Milestones are goal behaviors that specialty and subspecialty trainees should achieve at defined times during their progression through training.1Nasca T.J. Philibert I. Brigham T. Flynn T.C. The next GME accreditation system—rationale and benefits.N Engl J Med. 2012; 366: 1051-1056Crossref PubMed Scopus (1035) Google Scholar Milestones standardize and clarify expectations for faculty and trainees.The Clinical Competency Committee, composed of key clinical faculty, will report on each trainee’s progress in achieving the milestones at least semi-annually.2Accreditation Council for Graduate Medical Education. Frequently asked questions about the Next Accreditation System (updated December 2012). http://www.acgme-nas.org/assets/pdf/NASFAQs.pdf. Accessed May 13, 2013.Google Scholar They will select a description of trainee behavior using a 1-9 rating scale spanning “Critical deficiencies” to “Aspirational” behavior (Box 1). The Clinical Competency Committee also will formally decide whether the trainee is progressing satisfactorily toward achieving competence for unsupervised practice.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google ScholarBox 1Example Question and Rating Options From ACGME Report Worksheet for Patient Care Milestone 1 (PC1) Assessment in Internal Medicine ResidentsGathers and synthesizes essential and accurate information to define each patient’s clinical problem(s).{Rating 1} Level 1: Critical Deficiencies•Does not collect accurate historical data•Does not use physical exam to confirm history•Relies exclusively on documentation of others to generate own database or differential diagnosis•Fails to recognize patient’s central clinical problems{Rating 2}{Rating 3} Level 2•Inconsistently able to acquire accurate historical information in an organized fashion•Does not perform an appropriately thorough physical exam or misses key physical exam findings•Does not seek or is overly reliant on secondary data•Inconsistently recognizes patients’ central clinical problem or develops limited differential diagnoses{Rating 4}{Rating 5} Level 3•Consistently acquires accurate and relevant histories from patients•Seeks and obtains data from secondary sources when needed•Consistently performs accurate and appropriately thorough physical exams•Uses collected data to define a patient’s central clinical problem(s){Rating 6}{Rating 7} Level 4: Ready for unsupervised practice•Acquires accurate histories from patients in an efficient, prioritized, and hypothesis-driven fashion•Performs accurate physical exams that are targeted to the patient’s complaints•Synthesizes data to generate a prioritized differential diagnosis and problem list•Effectively uses history and physical exam skills to minimize the need for further diagnostic testing{Rating 8}{Rating 9} Level 5: Aspirational•Obtains relevant historical subtleties, including sensitive information that informs the differential diagnosis•Identifies subtle or unusual physical exam findings•Efficiently uses all sources of secondary data to inform differential diagnosis•Role models and teaches the effective use of history and physical exam skills to minimize the need for further diagnostic testingAbbreviations: ACGME, Accreditation Council for Graduate Medical Education; exam, examination.Entrustable professional activities are complex task sets that define core specialty functions and, like milestones, permit standardized performance expectations.4ten Cate O. Entrustability of professional activities and competency-based training.Med Educ. 2005; 39: 1176-1177Crossref PubMed Scopus (524) Google Scholar, 5Carraccio C. Burke A.E. Beyond competencies and milestones: adding meaning through context.J Grad Med Educ. 2010; 2: 419-422Crossref PubMed Google Scholar They describe what the trainee must do to satisfactorily provide unsupervised specialty care. Multiple milestones in multiple competencies can be mapped to a given entrustable professional activity, and the Clinical Competency Committee may use performance with respect to the latter to document milestone achievement.Schema of more than 140 milestones and 16 entrustable professional activities have been proposed for internal medicine.6ABIM/ACGME Milestones Task Force. Developmental milestones for internal medicine residency training (draft), 2013. David Geffen School of Medicine at UCLA Graduate Medical Education. http://www.gme.medsch.ucla.edu/director/documents/milestones-framework-draft.pdf. Accessed February 7, 2013.Google Scholar, 7Alliance for Academic Internal Medicine Education Redesign Committee. Internal medicine end-of-training EPAs, 2013. http://www.im.org/AcademicAffairs/milestones/Pages/EndofTrainingEPAs.aspx. Accessed May 18, 2013.Google Scholar The ACGME has decided on 22 reporting milestones to be adopted in academic year 2013-2014.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Subspecialty milestones (including nephrology) will be developed in 2013-2014 and introduced in 2014-2015.8Accreditation Council for Graduate Medical Education. RRC news: internal medicine. July 2012. http://www.acgme.org/acgmeweb/Portals/0/PFAssets/PublicationsNewsletters/Internal_Medicine_Newsletter_Jul12_.pdf. Accessed February 7, 2013.Google Scholar Formal efforts to adapt the internal medicine milestones to the subspecialties are just beginning.9Alliance for Academic Internal Medicine. Internal medicine summit on subspecialty milestones (February 11-12, 2013, Alexandria, VA). http://www.im.org/AcademicAffairs/milestones/Pages/imsubspecialtysummit.aspx. Accessed February 22, 2013.Google Scholar, 10American Society of Nephrology. The future of nephrology training: 2013 ASN training program directors' retreat; Chicago, IL. http://www.asn-online.org/education/training/tpd/2013_TPD_Retreat_Agenda.pdf. Accessed May 22, 2013.Google ScholarThe ACGME-promulgated milestones for internal medicine have not yet been validated, and according to the ACGME, the “Ready for Unsupervised Practice” milestone is a target, but not yet a requirement, for graduation.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Data provided by the training programs will be used for prospective validation, although the ACGME does not indicate what outcomes will be measured. American Board of Internal Medicine (ABIM) board examination performance may be used, as well as analysis of the patterns and rates of progression toward milestone achievement for all specialty residents.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google ScholarOur program, which is small (3 fellows per year) and military based, has incorporated milestones into competency assessment for 3 years. We use quantitative objective milestones when possible. Our milestones (Table 1) and the rationale for selecting them are as follows.Table 1Proposed Milestones for Nephrology Fellowship TrainingCompetencyFY1 MilestonesFY2 MilestonesPatient Care (PC)•Progressive decline in chart audit deficiencies (defined as 50% of raters indicate that the lecture [when applicable] had the potential to change practice)•Reduction in 100% outpatient chart audit (mo 1-6) to 50% (mo 7-9) to 25% (mo 10-12), based on continued deficiency reduction (see above)•Preparation of independent research protocol; publication or presentation of abstract, case report, or journal articleProfessionalism (P)•Progressive improvement to satisfactory performance on 360° evaluations•Successful completion of Medical Ethics curriculum (including ≥75% correct on military medical ethics test)•Faculty-level performance on 360° evaluations by the second half of the year (≥7 ratings in all areas)Interpersonal and Communication Skills (IPCS)•Mini-CEX: scores progressively improve•Chart audit: progressive improvement in timeliness and consultation skills•Mini-CEX: scores progressively improve and are fully acceptable by end of year (≥6 on 1-9 scale)•Chart audit: fully acceptable in timeliness and consultation skills ( 50% “Yes” responses to the question “Will this presentation change your practice?” when applicable•Completion and presentation of nephrology-related, collaborative, multidisciplinary performance improvement projectSystems Based Practice (SBP)•Proper and timely completion of CMS Form 2728•Progressively improved administrative actions related to kidney disease•Correct completion of all administrative actions related to kidney diseaseAbbreviations: CMS, Centers for Medicare & Medicaid Services; FY, fellowship year; Mini-CEX, Mini Clinical Evaluation Exercise; OSCE, Objective Structured Clinical Examination; RRT, renal replacement therapy. Open table in a new tab First, milestones should be nephrology focused. As internal medicine specialty residency graduates, nephrology fellows have already been deemed “ready for unsupervised practice” in internal medicine. Therefore, nephrology milestones should be nephrology focused, not repetitions of milestones already achieved in internal medicine. As examples, they should include management of chronic kidney disease, end-stage renal disease (ESRD) preparation, dialysis therapy initiation and maintenance, and care of transplant donors and recipients.Second, milestones should be quantitative and objective, when possible, to avoid evaluator bias. Although milestone achievement can be assessed by use of entrustable professional activities, “satisfactory development” should be determined based on quantitative and objective assessment. This reduces the effect of rater bias by individual evaluators and the Clinical Competency Committee.Third, some milestones must be nephrology procedure focused. Definitive procedures include acute and chronic renal replacement therapy, continuous renal replacement therapy, kidney biopsy, temporary dialysis access placement, and microscopic urinalysis. Milestones must document and consider threshold numbers for competence in these procedures.11Barsuk J.H. Ahya S.N. Cohen E.R. McGaghie W.C. Wayne D.B. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice.Am J Kidney Dis. 2009; 54: 70-76Abstract Full Text Full Text PDF PubMed Scopus (109) Google ScholarFourth, more is not necessarily better. A large number of detailed milestones are not necessarily superior to simpler representative ones. Quality literature recommends simplicity and parsimony in metrics, using aggregate metrics when possible.12Brown M.G. How to design your own measurement system.in: Keeping Score: Using the Right Metrics to Drive World-Class Performance. Productivity Press, New York, NY1996: 3-16Google Scholar, 13Pronovost P.J. Bo-Linn G.W. Preventing patient harms through systems of care.JAMA. 2012; 308: 769-770Crossref PubMed Scopus (53) Google Scholar Faculty time constraints and number of evaluators also are limiting. Smaller numbers of milestones facilitate validation and monitoring.Fifth, outcome measures must be defined to determine milestone validity. At present, the only outcomes available are the ABIM Nephrology Board Examination and the American Society of Nephrology In-Training Examination (ITE). Both focus on the Medical Knowledge and Patient Care competencies. Other potential measures include postgraduate evaluations by peers and patients and quality assurance data supplied to the Centers for Medicare & Medicaid Services (CMS).Sixth, tools to assess milestones should be defined. Our program uses a number of tools to assess competency and document milestone achievement, as described in the following:• Outpatient clinic chart audits for general nephrology, maintenance dialysis, and transplantation. The audit tool (an Excel spreadsheet) has been shown to be associated with improvement in quality indicators (independent of reviewer).14Abbott K.C. Bohen E.M. Welch P.G. Yuan C.M. Analyzing process variation in chart review using a networked spreadsheet application in the Walter Reed Army Medical Center Nephrology Clinic.Mil Med. 2001; 166: 771-773PubMed Google Scholar It allows assessment of trends over time, in tabular and graphical form, and permits quantifiable identification of deficiencies in care, timeliness, or documentation (competencies PC and IPCS); enables compliance with specified practice guidelines (MK and PBLI), and monitors military administrative actions that restrict assignments in active-duty patients (SBP). A median of 1,554 charts have been audited per year for the past 5 years by 7-9 faculty members.• Outpatient clinic chart audits demonstrate progressive improvement and responsibility. Based on compliance with practice guidelines and reduction in deficiencies, audit is reduced from 100% to 25% of charts during the second year of fellowship (PC). Progressive improvement is demonstrated by achieving a <5% deficiency rate per month by 6 months within a given training year and declining thereafter.• Objective structured clinical examination (OSCE). First-year fellows must pass an OSCE demonstrating appropriate responses to and medical knowledge of 4 dialysis emergencies (PC). Second-year fellows must pass an OSCE using the College of American Pathologists urinalysis quality assurance survey (PC).• Conference presentation and evaluation. Fellows present at approximately 6 conferences yearly, mentored by faculty. Attendees evaluate these conferences (IPCS, MK, and P) for presentation and content and indicate whether the information is likely to change practice. The average number of evaluations per fellow per year is 32 ± 5 (SD).• Procedure logs. Fellows must successfully perform 5 native kidney biopsies, 2 transplant kidney biopsies, and 5 dialysis catheter placements and must have sufficient experience with continuous renal replacement therapy (PC). Thresholds were established in the early 1990s. Berns and O’Neill15Berns J.S. O'Neill W.C. Performance of procedures by nephrologists and nephrology fellows at U.S. nephrology training programs.Clin J Am Soc Nephrol. 2008; 3: 941-947Crossref PubMed Scopus (59) Google Scholar found that approximately one-third of nephrology training programs had not established minimum fellow procedure numbers. The majority of programs that had a threshold required 1-6 procedures per fellow.15Berns J.S. O'Neill W.C. Performance of procedures by nephrologists and nephrology fellows at U.S. nephrology training programs.Clin J Am Soc Nephrol. 2008; 3: 941-947Crossref PubMed Scopus (59) Google Scholar During the last 5 years, the median number of procedures per fellow in our program was as follows: native kidney biopsies, 9 (range, 5-19); transplant kidney biopsies, 7 (range, 5-13); and temporary dialysis catheters, 8 (range, 5-18).• Mini-Clinical Evaluation Exercise (Mini-CEX). Fellow performance in counseling patients and families before kidney biopsy and renal replacement therapy is assessed longitudinally using a faculty-administered Mini-CEX (IPCS, MK, PC, and P).• Research productivity. Fellows must perform a mentored independent research project during their second year and present or publish a meeting abstract, case report, or journal article (MK, IPCS, and P).• Multidisciplinary performance improvement project. Second-year fellows must perform and present results of a collaborative multidisciplinary nephrology-related performance improvement project (PBLI, P, and MK).• Ethics curriculum. Second-year fellows must complete 3 ethics exercises: pass a military medical ethics review with a post-test, write a reflection on the documentary D Tour16Granato J. D Tour. Autonomy16 Productions, San Francisco, CA2009Google Scholar regarding maintenance dialysis and transplantation from the patient perspective, and respond to a clinical vignette of ESRD in an elderly patient, with a written description of case management and literature references (P).• Completion of ESRD Medicare Evidence Form 2728. Accuracy and timeliness in completing ESRD Medicare Evidence Form 2728 are determined by social work services at end of first year (SBP).• 360° evaluations by nursing, peer, and ancillary staff. Comprehensive evaluations are administered twice during each training year (P and IPCS).These milestones inform faculty and the Clinical Competency Committee whether trainees are advancing toward, and finally achieving, “competency for unsupervised practice.” They permit specific identification of deficiencies and demonstration of amendment and improvement. Some (Mini-CEX, conference presentation, and performance improvement project) are entrustable professional activities. Although not comprehensive, they are representative of and likely to predict the capacity for unsupervised nephrology practice. All proposed milestones should be prospectively validated with a priori outcome measures.17Parker M.G. Nephrology training in the 21st century: toward outcomes-based education.Am J Kidney Dis. 2010; 56: 132-142Abstract Full Text Full Text PDF PubMed Scopus (19) Google Scholar Potential outcome measures include ITE and Nephrology Board Examination performance, end-of training procedure skills testing using simulation technology,11Barsuk J.H. Ahya S.N. Cohen E.R. McGaghie W.C. Wayne D.B. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice.Am J Kidney Dis. 2009; 54: 70-76Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar postgraduate peer and patient evaluations, and performance data supplied to the CMS. The ITE, given twice during fellowship, could be used to demonstrate progressive improvement. For sufficient trainee numbers to permit comparisons, training program outcome results would have to be combined. Programs could be stratified, milestones under investigation could be randomized in blocks, agreed-upon outcomes could be measured, and evidence-based milestones could be adopted. Ideally, the structure and results of the educational research process would be transparent to all. Otherwise, we will be clicking radio buttons in rating software, unsure of whether the results predict whether our trainees are ready for unsupervised practice. It is imperative that nephrology training programs have milestones of proven worth, not administrative millstones of dubious utility. The Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System in order to improve future physician practice using the peer-review system, base training program accreditation decisions on trainee educational outcomes, and reduce administrative burdens associated with the present system.1Nasca T.J. Philibert I. Brigham T. Flynn T.C. The next GME accreditation system—rationale and benefits.N Engl J Med. 2012; 366: 1051-1056Crossref PubMed Scopus (1035) Google Scholar Specialty-specific milestones will be introduced to monitor the progress of trainees in achieving acceptable performance in the 6 competencies: Patient Care (PC), Medical Knowledge (MK), Professionalism (P), Interpersonal Communication Skills (IPCS), Practice-Based Learning and Improvement (PBLI), and Systems Based Practice (SBP). Each must be mastered to a level that allows the trainee to practice unsupervised. Milestones are goal behaviors that specialty and subspecialty trainees should achieve at defined times during their progression through training.1Nasca T.J. Philibert I. Brigham T. Flynn T.C. The next GME accreditation system—rationale and benefits.N Engl J Med. 2012; 366: 1051-1056Crossref PubMed Scopus (1035) Google Scholar Milestones standardize and clarify expectations for faculty and trainees. The Clinical Competency Committee, composed of key clinical faculty, will report on each trainee’s progress in achieving the milestones at least semi-annually.2Accreditation Council for Graduate Medical Education. Frequently asked questions about the Next Accreditation System (updated December 2012). http://www.acgme-nas.org/assets/pdf/NASFAQs.pdf. Accessed May 13, 2013.Google Scholar They will select a description of trainee behavior using a 1-9 rating scale spanning “Critical deficiencies” to “Aspirational” behavior (Box 1). The Clinical Competency Committee also will formally decide whether the trainee is progressing satisfactorily toward achieving competence for unsupervised practice.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Gathers and synthesizes essential and accurate information to define each patient’s clinical problem(s).{Rating 1} Level 1: Critical Deficiencies•Does not collect accurate historical data•Does not use physical exam to confirm history•Relies exclusively on documentation of others to generate own database or differential diagnosis•Fails to recognize patient’s central clinical problems{Rating 2}{Rating 3} Level 2•Inconsistently able to acquire accurate historical information in an organized fashion•Does not perform an appropriately thorough physical exam or misses key physical exam findings•Does not seek or is overly reliant on secondary data•Inconsistently recognizes patients’ central clinical problem or develops limited differential diagnoses{Rating 4}{Rating 5} Level 3•Consistently acquires accurate and relevant histories from patients•Seeks and obtains data from secondary sources when needed•Consistently performs accurate and appropriately thorough physical exams•Uses collected data to define a patient’s central clinical problem(s){Rating 6}{Rating 7} Level 4: Ready for unsupervised practice•Acquires accurate histories from patients in an efficient, prioritized, and hypothesis-driven fashion•Performs accurate physical exams that are targeted to the patient’s complaints•Synthesizes data to generate a prioritized differential diagnosis and problem list•Effectively uses history and physical exam skills to minimize the need for further diagnostic testing{Rating 8}{Rating 9} Level 5: Aspirational•Obtains relevant historical subtleties, including sensitive information that informs the differential diagnosis•Identifies subtle or unusual physical exam findings•Efficiently uses all sources of secondary data to inform differential diagnosis•Role models and teaches the effective use of history and physical exam skills to minimize the need for further diagnostic testingAbbreviations: ACGME, Accreditation Council for Graduate Medical Education; exam, examination. Gathers and synthesizes essential and accurate information to define each patient’s clinical problem(s). {Rating 1} Level 1: Critical Deficiencies•Does not collect accurate historical data•Does not use physical exam to confirm history•Relies exclusively on documentation of others to generate own database or differential diagnosis•Fails to recognize patient’s central clinical problems {Rating 2} {Rating 3} Level 2•Inconsistently able to acquire accurate historical information in an organized fashion•Does not perform an appropriately thorough physical exam or misses key physical exam findings•Does not seek or is overly reliant on secondary data•Inconsistently recognizes patients’ central clinical problem or develops limited differential diagnoses {Rating 4} {Rating 5} Level 3•Consistently acquires accurate and relevant histories from patients•Seeks and obtains data from secondary sources when needed•Consistently performs accurate and appropriately thorough physical exams•Uses collected data to define a patient’s central clinical problem(s) {Rating 6} {Rating 7} Level 4: Ready for unsupervised practice•Acquires accurate histories from patients in an efficient, prioritized, and hypothesis-driven fashion•Performs accurate physical exams that are targeted to the patient’s complaints•Synthesizes data to generate a prioritized differential diagnosis and problem list•Effectively uses history and physical exam skills to minimize the need for further diagnostic testing {Rating 8} {Rating 9} Level 5: Aspirational•Obtains relevant historical subtleties, including sensitive information that informs the differential diagnosis•Identifies subtle or unusual physical exam findings•Efficiently uses all sources of secondary data to inform differential diagnosis•Role models and teaches the effective use of history and physical exam skills to minimize the need for further diagnostic testing Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; exam, examination. Entrustable professional activities are complex task sets that define core specialty functions and, like milestones, permit standardized performance expectations.4ten Cate O. Entrustability of professional activities and competency-based training.Med Educ. 2005; 39: 1176-1177Crossref PubMed Scopus (524) Google Scholar, 5Carraccio C. Burke A.E. Beyond competencies and milestones: adding meaning through context.J Grad Med Educ. 2010; 2: 419-422Crossref PubMed Google Scholar They describe what the trainee must do to satisfactorily provide unsupervised specialty care. Multiple milestones in multiple competencies can be mapped to a given entrustable professional activity, and the Clinical Competency Committee may use performance with respect to the latter to document milestone achievement. Schema of more than 140 milestones and 16 entrustable professional activities have been proposed for internal medicine.6ABIM/ACGME Milestones Task Force. Developmental milestones for internal medicine residency training (draft), 2013. David Geffen School of Medicine at UCLA Graduate Medical Education. http://www.gme.medsch.ucla.edu/director/documents/milestones-framework-draft.pdf. Accessed February 7, 2013.Google Scholar, 7Alliance for Academic Internal Medicine Education Redesign Committee. Internal medicine end-of-training EPAs, 2013. http://www.im.org/AcademicAffairs/milestones/Pages/EndofTrainingEPAs.aspx. Accessed May 18, 2013.Google Scholar The ACGME has decided on 22 reporting milestones to be adopted in academic year 2013-2014.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Subspecialty milestones (including nephrology) will be developed in 2013-2014 and introduced in 2014-2015.8Accreditation Council for Graduate Medical Education. RRC news: internal medicine. July 2012. http://www.acgme.org/acgmeweb/Portals/0/PFAssets/PublicationsNewsletters/Internal_Medicine_Newsletter_Jul12_.pdf. Accessed February 7, 2013.Google Scholar Formal efforts to adapt the internal medicine milestones to the subspecialties are just beginning.9Alliance for Academic Internal Medicine. Internal medicine summit on subspecialty milestones (February 11-12, 2013, Alexandria, VA). http://www.im.org/AcademicAffairs/milestones/Pages/imsubspecialtysummit.aspx. Accessed February 22, 2013.Google Scholar, 10American Society of Nephrology. The future of nephrology training: 2013 ASN training program directors' retreat; Chicago, IL. http://www.asn-online.org/education/training/tpd/2013_TPD_Retreat_Agenda.pdf. Accessed May 22, 2013.Google Scholar The ACGME-promulgated milestones for internal medicine have not yet been validated, and according to the ACGME, the “Ready for Unsupervised Practice” milestone is a target, but not yet a requirement, for graduation.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Data provided by the training programs will be used for prospective validation, although the ACGME does not indicate what outcomes will be measured. American Board of Internal Medicine (ABIM) board examination performance may be used, as well as analysis of the patterns and rates of progression toward milestone achievement for all specialty residents.3Accreditation Council for Graduate Medical Education and American Board of Internal Medicine. The Internal Medicine Milestone Project (v. 12/2012). http://www.acgme-nas.org/assets/pdf/Milestones/InternalMedicineMilestones.pdf. Accessed February 7, 2013.Google Scholar Our program, which is small (3 fellows per year) and military based, has incorporated milestones into competency assessment for 3 years. We use quantitative objective milestones when possible. Our milestones (Table 1) and the rationale for selecting them are as follows. Abbreviations: CMS, Centers for Medicare & Medicaid Services; FY, fellowship year; Mini-CEX, Mini Clinical Evaluation Exercise; OSCE, Objective Structured Clinical Examination; RRT, renal replacement therapy. First, milestones should be nephrology focused. As internal medicine specialty residency graduates, nephrology fellows have already been deemed “ready for unsupervised practice” in internal medicine. Therefore, nephrology milestones should be nephrology focused, not repetitions of milestones already achieved in internal medicine. As examples, they should include management of chronic kidney disease, end-stage renal disease (ESRD) preparation, dialysis therapy initiation and maintenance, and care of transplant donors and recipients. Second, milestones should be quantitative and objective, when possible, to avoid evaluator bias. Although milestone achievement can be assessed by use of entrustable professional activities, “satisfactory development” should be determined based on quantitative and objective assessment. This reduces the effect of rater bias by individual evaluators and the Clinical Competency Committee. Third, some milestones must be nephrology procedure focused. Definitive procedures include acute and chronic renal replacement therapy, continuous renal replacement therapy, kidney biopsy, temporary dialysis access placement, and microscopic urinalysis. Milestones must document and consider threshold numbers for competence in these procedures.11Barsuk J.H. Ahya S.N. Cohen E.R. McGaghie W.C. Wayne D.B. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice.Am J Kidney Dis. 2009; 54: 70-76Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar Fourth, more is not necessarily better. A large number of detailed milestones are not necessarily superior to simpler representative ones. Quality literature recommends simplicity and parsimony in metrics, using aggregate metrics when possible.12Brown M.G. How to design your own measurement system.in: Keeping Score: Using the Right Metrics to Drive World-Class Performance. Productivity Press, New York, NY1996: 3-16Google Scholar, 13Pronovost P.J. Bo-Linn G.W. Preventing patient harms through systems of care.JAMA. 2012; 308: 769-770Crossref PubMed Scopus (53) Google Scholar Faculty time constraints and number of evaluators also are limiting. Smaller numbers of milestones facilitate validation and monitoring. Fifth, outcome measures must be defined to determine milestone validity. At present, the only outcomes available are the ABIM Nephrology Board Examination and the American Society of Nephrology In-Training Examination (ITE). Both focus on the Medical Knowledge and Patient Care competencies. Other potential measures include postgraduate evaluations by peers and patients and quality assurance data supplied to the Centers for Medicare & Medicaid Services (CMS). Sixth, tools to assess milestones should be defined. Our program uses a number of tools to assess competency and document milestone achievement, as described in the following: • Outpatient clinic chart audits for general nephrology, maintenance dialysis, and transplantation. The audit tool (an Excel spreadsheet) has been shown to be associated with improvement in quality indicators (independent of reviewer).14Abbott K.C. Bohen E.M. Welch P.G. Yuan C.M. Analyzing process variation in chart review using a networked spreadsheet application in the Walter Reed Army Medical Center Nephrology Clinic.Mil Med. 2001; 166: 771-773PubMed Google Scholar It allows assessment of trends over time, in tabular and graphical form, and permits quantifiable identification of deficiencies in care, timeliness, or documentation (competencies PC and IPCS); enables compliance with specified practice guidelines (MK and PBLI), and monitors military administrative actions that restrict assignments in active-duty patients (SBP). A median of 1,554 charts have been audited per year for the past 5 years by 7-9 faculty members. • Outpatient clinic chart audits demonstrate progressive improvement and responsibility. Based on compliance with practice guidelines and reduction in deficiencies, audit is reduced from 100% to 25% of charts during the second year of fellowship (PC). Progressive improvement is demonstrated by achieving a <5% deficiency rate per month by 6 months within a given training year and declining thereafter. • Objective structured clinical examination (OSCE). First-year fellows must pass an OSCE demonstrating appropriate responses to and medical knowledge of 4 dialysis emergencies (PC). Second-year fellows must pass an OSCE using the College of American Pathologists urinalysis quality assurance survey (PC). • Conference presentation and evaluation. Fellows present at approximately 6 conferences yearly, mentored by faculty. Attendees evaluate these conferences (IPCS, MK, and P) for presentation and content and indicate whether the information is likely to change practice. The average number of evaluations per fellow per year is 32 ± 5 (SD). • Procedure logs. Fellows must successfully perform 5 native kidney biopsies, 2 transplant kidney biopsies, and 5 dialysis catheter placements and must have sufficient experience with continuous renal replacement therapy (PC). Thresholds were established in the early 1990s. Berns and O’Neill15Berns J.S. O'Neill W.C. Performance of procedures by nephrologists and nephrology fellows at U.S. nephrology training programs.Clin J Am Soc Nephrol. 2008; 3: 941-947Crossref PubMed Scopus (59) Google Scholar found that approximately one-third of nephrology training programs had not established minimum fellow procedure numbers. The majority of programs that had a threshold required 1-6 procedures per fellow.15Berns J.S. O'Neill W.C. Performance of procedures by nephrologists and nephrology fellows at U.S. nephrology training programs.Clin J Am Soc Nephrol. 2008; 3: 941-947Crossref PubMed Scopus (59) Google Scholar During the last 5 years, the median number of procedures per fellow in our program was as follows: native kidney biopsies, 9 (range, 5-19); transplant kidney biopsies, 7 (range, 5-13); and temporary dialysis catheters, 8 (range, 5-18). • Mini-Clinical Evaluation Exercise (Mini-CEX). Fellow performance in counseling patients and families before kidney biopsy and renal replacement therapy is assessed longitudinally using a faculty-administered Mini-CEX (IPCS, MK, PC, and P). • Research productivity. Fellows must perform a mentored independent research project during their second year and present or publish a meeting abstract, case report, or journal article (MK, IPCS, and P). • Multidisciplinary performance improvement project. Second-year fellows must perform and present results of a collaborative multidisciplinary nephrology-related performance improvement project (PBLI, P, and MK). • Ethics curriculum. Second-year fellows must complete 3 ethics exercises: pass a military medical ethics review with a post-test, write a reflection on the documentary D Tour16Granato J. D Tour. Autonomy16 Productions, San Francisco, CA2009Google Scholar regarding maintenance dialysis and transplantation from the patient perspective, and respond to a clinical vignette of ESRD in an elderly patient, with a written description of case management and literature references (P). • Completion of ESRD Medicare Evidence Form 2728. Accuracy and timeliness in completing ESRD Medicare Evidence Form 2728 are determined by social work services at end of first year (SBP). • 360° evaluations by nursing, peer, and ancillary staff. Comprehensive evaluations are administered twice during each training year (P and IPCS). These milestones inform faculty and the Clinical Competency Committee whether trainees are advancing toward, and finally achieving, “competency for unsupervised practice.” They permit specific identification of deficiencies and demonstration of amendment and improvement. Some (Mini-CEX, conference presentation, and performance improvement project) are entrustable professional activities. Although not comprehensive, they are representative of and likely to predict the capacity for unsupervised nephrology practice. All proposed milestones should be prospectively validated with a priori outcome measures.17Parker M.G. Nephrology training in the 21st century: toward outcomes-based education.Am J Kidney Dis. 2010; 56: 132-142Abstract Full Text Full Text PDF PubMed Scopus (19) Google Scholar Potential outcome measures include ITE and Nephrology Board Examination performance, end-of training procedure skills testing using simulation technology,11Barsuk J.H. Ahya S.N. Cohen E.R. McGaghie W.C. Wayne D.B. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice.Am J Kidney Dis. 2009; 54: 70-76Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar postgraduate peer and patient evaluations, and performance data supplied to the CMS. The ITE, given twice during fellowship, could be used to demonstrate progressive improvement. For sufficient trainee numbers to permit comparisons, training program outcome results would have to be combined. Programs could be stratified, milestones under investigation could be randomized in blocks, agreed-upon outcomes could be measured, and evidence-based milestones could be adopted. Ideally, the structure and results of the educational research process would be transparent to all. Otherwise, we will be clicking radio buttons in rating software, unsure of whether the results predict whether our trainees are ready for unsupervised practice. It is imperative that nephrology training programs have milestones of proven worth, not administrative millstones of dubious utility. The views expressed in this article are those of the authors and do not reflect the official policy of the Department of the Army, the Department of the Navy, the Department of Defense, or the United States government. Support: None. Financial Disclosure: The authors declare that they have no relevant financial interests.
Referência(s)