Portal de Periódicos da CAPES

The April Effect: A Multimedia Orientation Approach to Improve Rotation Transitions During Pediatric Residency

2016; Elsevier BV; Volume: 16; Issue: 3 Linguagem: Inglês

10.1016/j.acap.2016.02.005

1876-2867

Brad Sobolewski, Benjamin T. Kerrey, Gary L. Geis, Corinne Bria, M Mittiga, Javier A. Gonzalez del Rey,

Empathy and Medical Education

In the lay press, concerns about the transition from medical school to residency have been embodied in the so-called July effect, with reports of increased medical errors in the first month of the traditional academic calendar.1Sifferlin A. Fewer hours for doctors in training leading to more mistakes.2013http://healthland.time.com/2013/03/26/fewer-hours-for-doctors-in-training-leading-to-more-mistakes/Google Scholar This focus on the impact of novice trainees on the quality of care ignores the fact that transitions to unfamiliar clinical environments occur throughout the academic year, not just in July.2Jena A.B. Sun E.C. Romley J.A. Mortality among high-risk patients with acute myocardial infarction admitted to US teaching-intensive hospitals in July: a retrospective observational study.Circulation. 2013; 128: 2754-2763Google Scholar, 3Ravi P. Trinh V.Q. Sun M. et al.Is there any evidence of a "July effect" in patients undergoing major cancer surgery?.Can J Surg. 2014; 57: 82-88Google Scholar, 4Phillips D.P. Barker G.E. A July spike in fatal medication errors: a possible effect of new medical residents.J Gen Intern Med. 2010; 25: 774-779Google Scholar These transitions between rotations present many challenges, including different care environments, collaboration with a new set of providers, unique patient populations, varied severity of acute and chronic diseases, and differing learning objectives and clinical expectations. Pediatric residency programs must adhere to the Accreditation Council for Graduate Medical Education specifications for the quantity and specific type of rotation experiences, with each rotation including a formal orientation with accompanying explicit discussion of goals and objectives.5ACGME. Program requirements for graduate medical education in pediatrics. 2011. Available at: http://acgme.org/acWebsite/downloads/RRC_progReq/320_pediatrics_07012007.pdf. Accessed April 10, 2011.Google Scholar The rotation orientation is intended to mitigate the impact of transitions on patient care and resident learning. A well-structured orientation should explain resident roles and responsibilities, introduce them to critical personnel, and highlight the unique features of the rotation.6O'Brien B.C. Poncelet A.N. Transition to clerkship courses: preparing students to enter the workplace.Acad Med. 2010; 85: 1862-1869Google Scholar Orientations should be consistent, reproducible, and efficient, and they should provide a record of participation.7Andolsek K.M. Murphy G. Pinheiro S. et al.Efficacy and efficiency of webcast orientations versus live resident orientations: results of a 2-year survey.J Grad Med Educ. 2010; 2: 136-140Google Scholar Survey-based studies have shown that residents want an effective, personalized orientation that focuses on authentic clinical responsibilities.8Antonoff M.B. Swanson J.A. Acton R.D. et al.Improving surgery intern confidence through the implementation of expanded orientation sessions.Surgery. 2010; 148: 181-186Google Scholar, 9Brillman J.C. Sklar D.P. Viccellio P. Characteristics of emergency medicine resident orientation programs.Acad Emerg Med. 1995; 2: 25-31Google Scholar, 10Grover M. Puczynski S. Right from the start: the family practice orientation study.Fam Med. 1999; 31: 177-181Google Scholar, 11Fein J.A. Lavelle J. Giardino A.P. Teaching emergency medicine to pediatric residents: a national survey and proposed model.Pediatr Emerg Care. 1995; 11: 208-211Google Scholar In contrast, faculty indicate that they wish to promote group cohesion and introduce key hospital services and administrative resources and personnel.12Grover M. Puczynski S. Residency orientation: what we present and its effect on our residents.Fam Med. 1999; 31: 697-702Google Scholar, 13Harel Z. Riggs S. Vaz R. et al.A session of multiple choice questions in the orientation process to the adolescent medicine rotation.Teach Learn Med. 2004; 16: 365-367Google Scholar From the perspective of residency program and rotation directors, the heterogeneity of clinical rotations and work-hour regulations have made effectively orienting and transitioning residents to and from rotations more challenging.14Desai S.V. Feldman L. Brown L. et al.Effect of the 2011 vs 2003 duty hour regulation-compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial.JAMA Intern Med. 2013; 173: 649-655Google Scholar, 15Auger K.A. Jerardi K.E. Sucharew H.J. et al.Effects of the 2011 duty hour restrictions on resident education and learning from patient admissions.Hosp Pediatr. 2014; 4: 222-227Google Scholar, 16Auger K.A. Landrigan C.P. Gonzalez del Rey J.A. et al.Better rested, but more stressed? Evidence of the effects of resident work hour restrictions.Acad Pediatr. 2012; 12: 335-343Google Scholar While educators agree that it is important to ensure residents are prepared to transition into new roles, the development, implementation, and effectiveness of resident orientations are understudied aspects of training. Our objective was to apply the full process of curriculum development to create and evaluate a Web-based multimedia orientation to the pediatric emergency department (PED) that would supplement and reinforce the basic rotation introduction covered during the first week of the PL1 year. We present our experience with the development of this orientation, beginning with a clear identification of the problems we faced, followed by a targeted needs assessment, design, implementation, and finally impact assessment. The study was approved by the institutional review board at Cincinnati Children's Hospital Medical Center (CCHMC). CCHMC is an urban academic pediatric institution that is the major regional provider of emergency care to children, with approximately 85,000 annual visits to the emergency department (ED). The pediatric residency program has nearly 200 total residents, and approximately 120 are categorical pediatrics. Pediatric residents at CCHMC spend up to 4 nonconsecutive months in the ED, with individual rotations lasting either 2 weeks or 1 month. First-year pediatric residents are required to spend 1 month total in the ED; second and higher year residents rotate for a total of 3 months during their remaining years of training. Residents at all levels of training work 10-hour shifts. During a given month, there may be as many as 40 residents in the ED across all levels of training (categorical pediatrics, combined pediatric programs, and rotating residents from emergency medicine and family medicine). As is common within training institutions, clinical roles and responsibilities of residents grow in complexity over the course of training. At our institution, some faculty and residents in the ED shared concerns that our orientation process was suboptimal because it did not specifically address the unique personnel and patient care issues involved in caring for acutely ill and injured children. Before our work, the orientation to the ED rotation consisted of brief, informal introductions before the PL1 and PL2 years, as well as materials hosted on the hospital Intranet. Residents voiced concerns that they felt underprepared for the rotation, especially in the resuscitation area, an environment unique to the ED and likely not reproduced in any other area of residency. These concerns prompted an internally derived survey-based, anonymous needs assessment, which was voluntarily completed during a residents-only meeting in the spring of 2012 by a limited number (55 of the nearly 200 residents) of trainees, all of whom were rising PL2 and PL3 residents. Key ED faculty and the chief residents developed the survey, which consisted of Likert-response items and open-ended answer questions. Almost three-fourths (69%, 34 of 55) of respondents indicated that they had an inadequate orientation to the resuscitation area. Only 14% (8 of 55) were aware of the criteria on which they were being evaluated while rotating in the ED. Overall, 71% (39 of 55) indicated that the existing ED orientation was ineffective. On the basis of the needs assessment, we identified numerous challenges to orienting residents to the ED, including the following:•A large number of resident trainees each month, working a variable shift schedule.•An unfamiliar clinical environment.•A wide spectrum of illness and injury.•An expectation of frequent patient turnover within each shift, as well as receiving and giving sign-outs on patients whose disposition is undetermined at the time of sign-out.•Highly variable care teams, with more than 40 faculty, 11 ED fellows, and numerous allied health professionals.•Unique features of the ED rotation, including the evaluation and management of patients in the resuscitation area. Given the aforementioned challenges, we determined that a Web-based orientation would likely best serve our trainees' needs. Online sources of education can be leveraged in order to supplement the traditional curriculum, engage residents, and embed new learning before their arrival on an upcoming rotation. Online tools can provide rich learning experiences that allow learners to review materials on their own time, with straightforward methods to record access.7Andolsek K.M. Murphy G. Pinheiro S. et al.Efficacy and efficiency of webcast orientations versus live resident orientations: results of a 2-year survey.J Grad Med Educ. 2010; 2: 136-140Google Scholar, 17Khan B.H. Web-Based Instruction. Educational Technology, Englewood Cliffs, NJ1997Google Scholar, 18Kamel Boulos M. Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education.Health Info Libr J. 2007; 24: 2-23Google Scholar, 19Burnette K. Ramundo M. Stevenson M. et al.Evaluation of a Web-based asynchronous pediatric emergency medicine learning tool for residents and medical students.Acad Emerg Med. 2009; 16: S46-S50Google Scholar, 20Chenkin J. Lee S. Huynh T. et al.Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training.Acad Emerg Med. 2008; 15: 949-954Google Scholar, 21McKimm J. Jolie C. Cantillon P. ABC of learning and teaching Web based learning.BMJ. 2006; 326: 870-873Google Scholar, 22Solomon D.J. Ferenchick G.S. Laird-Fick H.S. et al.A randomized trial comparing digital and live lecture formats.BMC Med Educ. 2004; 4: 27Google Scholar, 23Cook D.A. Levinson A.J. Garside S. et al.Internet-based learning in the health professions.JAMA. 2008; 300: 1181-1196Google Scholar, 24Chang T.P. Pham P.K. Sobolewski B. et al.Pediatric emergency medicine asynchronous e-learning: a multicenter randomized controlled Solomon four-group study.Acad Emerg Med. 2014; 21: 912-919Google Scholar Thus, in anticipation of the 2012–2013 academic year, we used existing ED rotation goals and objectives and information from the survey to develop and implement an online orientation program to the ED rotation for residents. This program included:•A general video introduction to the ED.•A specific video introduction for the ED resuscitation area that included information on care team composition, resident role and responsibilities, and the correct performance of the rapid cardiopulmonary assessment.•A detailed presentation on the Pediatric Milestones–based ED rotation evaluation. The video-based components were filmed in the ED, with faculty and staff in various roles. Video footage was edited on Final Cut Pro X (Apple, Cupertino, Calif) and the final general and resuscitation area videos uploaded to Vimeo (White Plains, NY). The orientation videos were password protected and embedded on the orientation Web site, hosted via WordPress (WordPress, San Francisco, Calif) on a domain owned by the principal investigator (http://www.pemcincinnati.com/orientation). The entire orientation took 25 to 30 minutes to review. It was critical to assure that residents were held accountable for accessing the orientation. As such, we decreed that residents who did not attest to viewing the orientation would not be permitted to participate in care in the resuscitation area of the ED. Beginning with the July 2012 rotation block, an e-mail was sent to all PL1 and PL2 categorical pediatric residents scheduled to rotate through the ED, both 7 days and 48 hours before the start of their rotation. We embedded an e-mail form on the orientation Web site that allowed residents to attest to viewing of the orientation materials. We chose only the PL1 and PL2 residents because PL1s were entering the PED for the first time, and the PL2s would participate in the care of patients in the resuscitation area. PL3 and above residents were not included in the e-mail reminders because they had already rotated through the PED several times and had experience in the resuscitation area. Because of differing goals and objectives, visiting residents (family medicine) and emergency medicine residents were also excluded. Record of e-mail attestation from the Web site served as a proxy measure for resident access of the orientation materials. We assessed the impact of the orientation in 2 ways. First, research coordinators approached PL1s and PL2s in the ED and had them answer 10 single-answer questions related to principal concepts featured in the orientation, such as personnel roles and responsibilities, trauma team activation criteria, and ED-specific patient care processes. The 10 questions were randomly selected and sequenced from a bank of 25, which were developed by the study authors and piloted by the chief residents. The research coordinators approached residents in private during the first or second ED shift of their rotation, performed the knowledge testing, and provided answers for any incorrect responses. These pop quizzes were conducted between January and June of 2013. Using analysis of variance, we assessed for an association between the number of prior PED rotations and total score, suspecting that new PL1 residents who viewed the orientation would have similar knowledge to PL2 residents who had rotated previously. Second, after year 1 of our orientation implementation, we again surveyed the residents. We compared survey responses by chi-square test. During the 2012–2013 academic year, 94% of eligible residents (93 of 99) attested via e-mail that they viewed the orientation Web site before at least one ED rotation, including 96% of PL1s (50 of 52) and 91.5% of PL2s (43 of 47). Eighty-eight percent (87 of 99) of the residents rotated through the ED more than once; 31% (30 of 99) attested that they viewed the orientation before all of their ED rotations, including 67% (35 of 52) of PL1s and 17% (8 of 47) of PL2s. More than half (64%, 63 of 99) of the residents were approached for the pop quizzes (25 of 52 PL1s and 38 of 47 PL2s). The mean number of correct answers to knowledge questions by all residents was 7.2 of 10 (SD 2.1). The mean number of correct answers by PL2s (8.1 of 10, SD 1.7) was greater than that of PL1s (5.8 of 10, SD 2.1); P < .0001. The mean number of correct responses for residents in their first ED rotation was lower compared to those who had previously spent 1 or more months on the rotation (P = .001; Fig). Fifty-four (58%) of 93 residents who attested to viewing the orientation materials responded to our postsurvey, which asked the same questions as the presurvey. The postsurvey participants (PL1 and PL2 trainees) again provided their responses in an anonymous fashion. Furthermore, the pre- and postsurvey respondent populations were not identical. Sixty-five percent (35 of 54) indicated that they were aware of the specific criteria on which they were being evaluated, compared to 14% historically (8 of 56), (P < .0001). All (54 of 54) survey respondents felt that they had received an appropriate orientation to the resuscitation area compared to 32% (18 of 56) historically (P < .0001). In its first year, a Web-based and multimedia orientation to our PED was associated with improvements in resident exposure to orientation materials, resident knowledge of rotation expectations/evaluation criteria, and resident perception of the adequacy of the ED orientation. Since its inception, the orientation has been updated in an iterative fashion in response to changes at the institution. As noted earlier, most studies of residency rotation orientation effectiveness have been limited to survey-based methodology and do not address the process from start to finish, in contrast to our orientation program development process9Brillman J.C. Sklar D.P. Viccellio P. Characteristics of emergency medicine resident orientation programs.Acad Emerg Med. 1995; 2: 25-31Google Scholar, 12Grover M. Puczynski S. Residency orientation: what we present and its effect on our residents.Fam Med. 1999; 31: 697-702Google Scholar, 25Schott E. Evaluation of trauma service orientation.Conn Med. 2010; 74: 107-109Google Scholar One study has shown short-term increases in technical confidence among residents.26Hiraoka M. Kamikawa G. McCartin R. et al.A pilot structured resident orientation curriculum improves the confidence of incoming first-year obstetrics and gynecology residents.Hawaii J Med Public Health. 2013; 72: 387-390Google Scholar Multiple orientation curricula can be found online through repositories such as MedEdPORTAL, but they require modifications for local adaptation. Although many such curricula may include evaluation tools, data allowing an interpretation of the effects of implementation are lacking.27Frallicciardi A. Nowicki T. Herbst M.A.F. A Simulation-Based Emergency Medicine PGY-1 Resident Orientation Curriculum. MedEdPORTAL Publications, 2015Google Scholar, 28Nestor-Arjun B.M.M. Orientation to Emergency Pediatric Procedures. MedEdPORTAL Publications, Washington, DC2014Google Scholar There are limitations to our study and the interpretation of our findings. First, we were limited in our ability to assure that all residents watched the entirety of the videos. Content hosted on Vimeo or other online services lack detailed tracking beyond rudimentary view counts. Thus, future modifications will include content stratified by resident level and designed with adaptive modular platforms (such as Adobe Captivate) in order to better engage learners and track compliance. Second, we were not able to capture all of the available trainees in the knowledge assessment. However, we were reassured by the pop quiz scores of the PL1s that had never before rotated in the ED. Their results suggest that rotation-specific knowledge was acquired via exposure to the online orientation. It is important to note that residents with greater prior ED experience answered more questions correctly on average. We suspect that cumulative experience in the ED accounted for this measured difference. Third, we did not utilize a pretest–posttest methodology to establish the baseline knowledge of PL1s before the online orientation. Future studies should consider incorporating a pretest or use of a control group (ie, outside rotators such as family practice residents) to better assess the effectiveness of the orientation. Finally, it is important to note that our experience is an investigation of the impact of a pediatric residency rotation orientation at a single institution and these results may not be generalizable to all pediatric institutions or ED rotations. Though the content of our orientation was highly specific to our institution, the methodology is easily replicable and the process to create a rotation specific video can be accomplished affordably via commercially available software and equipment. Despite these limitations, several important lessons were learned that will inform our future endeavors and be of value to fellow educators who wish to develop similar orientation programs:•Involve key stakeholders in the process, including faculty (the rotation director), program leadership, ED care team personnel, chief residents, and the residents themselves.•Conduct a thorough needs assessment in order to identify your orientation's deficiencies, and plan to assess the impact of your new orientation.•If all of the trainees can't be in the same room at the same time, leverage the power of online tools and multimedia solutions.•Implement a system of accountability by assuring that residents know that the orientation is valuable and meaningful, and their participation is part of the professional expectations for the rotation. It is unlikely that residency work hours will increase in the foreseeable future. However, trainee concerns about transitions to new rotations will almost certainly persist. Developing methodologically sound plans for orientation to clinical rotations, in formats that are easily available and comprehensive, may mitigate some of the negative impact of having new residents each month in different areas of the hospital. Future study is needed to assess whether our, or similar online orientations, can reduce the impact of the July effect—and for that matter the September effect or even the April effect.