Portal de Periódicos da CAPES

Measuring Resident Hours by Tracking Interactions with the Computerized Record

2010; Elsevier BV; Volume: 123; Issue: 3 Linguagem: Inglês

10.1016/j.amjmed.2009.10.009

1555-7162

Daniel Shine, Ellen Pearlman, Brendan Watkins,

Healthcare Operations and Scheduling Optimization

The impact of resident duty hours on clinical and educational outcomes continues to concern professional and government oversight groups. Duty hours regulations have both stimulated and responded to a growing literature; but even after considerable study, the relationships remain uncertain among sleep, fatigue, effective education, hospital working conditions, hand-offs, patient safety, and resident burn-out. To be persuasive, reports investigating these relationships must accurately and completely measure the independent variable: resident duty hours. The almost universal practice of deriving duty hours from retrospectively completed time-cards, whether actual or computerized, has been criticized as potentially biased and poorly reproducible.Perspectives Viewpoints•Regulatory oversight of resident work schedules has obliged training programs to monitor resident hours closely. Work time is usually measured by self-report using resident time-cards, actual or computerized.•This study compares measurement of hours using paper, resident-completed time-cards to automated reports of resident interactions with the institution's electronic medical record (EMR).•EMR surveillance yielded information similar to time-cards with greater ease and timeliness.Regulation of resident duty hours began as a legislative initiative in New York 25 years ago.1NY Comp. Codes R. & Regs. Section 405.4.2002.Google Scholar This local issue appeared nationally in 2002 when the Occupational Safety and Health Administration, under petition by 3 concerned groups, first considered limiting hours for the sake of both resident and patient safety.2Public Citizen Health Research GroupMedical residents; work hours. Public Citizen, New York2003www.citizen.org/hrg/healthcare/articles.cfm?ID=6666Google Scholar, 3American Medical Student AssociationThe Resident Work Hour Issue.www.amsa.org/rwh/index.cfmGoogle Scholar, 4Committee of Interns and Residentswww.cirseiu.org/assets/Google Scholar Anticipating federal regulation, the Accreditation Council for Graduate Medical Education, a professional certification body, issued requirements the following year and subsequently to all training programs, regardless of specialty.5Philibert I. Friedmann P. Williams W.T. New requirements for resident duty hours ACGME. Work Group on Resident Duty Hours.JAMA. 2002; 288: 1112-1114Crossref PubMed Scopus (403) Google Scholar, 6Accreditation Council for Graduate Medical EducationInnovative Approaches for Resident Duty Hours.http://www.acgme.org/acWebsite/dutyHours/dh_innovative.aspGoogle Scholar, 7Institute of Medicinehttp://www.iom.edu/CMS/3809/48553/60449.aspxGoogle Scholar, 8Council Directive 93/104/EC.Official Journal of the European Community. 1993; L307: 18-24Google Scholar Most recently, the Institute of Medicine, an influential watchdog organization, has recommended that resident hours be further reduced to approach requirements in Europe. The European Working Time Directive requires that medical training be limited to 48 hours per week with 11 hours of rest in each 24-hour period.7Institute of Medicinehttp://www.iom.edu/CMS/3809/48553/60449.aspxGoogle Scholar, 8Council Directive 93/104/EC.Official Journal of the European Community. 1993; L307: 18-24Google ScholarLack of sleep impairs vigilance and task-oriented performance.9Denisco R.A. Drummond J.N. Gravenstein J.S. The effect of fatigue on the performance of a simulated anesthetic monitoring task.J Clin Monit. 1987; 3: 22-24Crossref PubMed Scopus (97) Google Scholar, 10Barger L.K. Cade B.E. Ayas N.T. et al.Extended works shifts and the risk of motor vehicle crashes among interns.N Engl J Med. 2005; 352: 125-134Crossref PubMed Scopus (714) Google Scholar, 11Ayas N.T. Barger L.K. Cade B.E. et al.Extended work duration and the risk of self reported percutaneous injuries in interns.JAMA. 2006; 296: 1055-1062Crossref PubMed Scopus (305) Google Scholar Less clear is the effect of fatigue on cognitive and team-based activities, such as rounding. Other workplace variables may bear as much on impairment and fatigue as do hours of sleep.12Arora V.M. Georgitis E. Siddique J. et al.Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities.JAMA. 2008; 300: 1146-1153Crossref PubMed Scopus (83) Google Scholar, 13Friesen L.D. Vidyarthi A.R. Baron R.B. Katz P.P. Factors associated with intern fatigue.J Gen Intern Med. 2008; 23: 1981-1986Crossref PubMed Scopus (27) Google ScholarAt least a dozen investigators have systematically examined changes in patient outcomes before and after Accreditation Council for Graduate Medical Education duty hours regulation implementation.14Fletcher K.E. Davis S.Q. Underwood W. et al.Systematic review: effects of resident work hours on patient safety.Ann Intern Med. 2004; 141: 851-857Crossref PubMed Scopus (174) Google Scholar, 15Landrigan C.P. Fahrenkopf A.M. Lewin D. et al.Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety.Pediatrics. 2008; 122: 250-258Crossref PubMed Scopus (136) Google Scholar, 16Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 984-992Crossref PubMed Scopus (219) Google Scholar, 17Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 975-983Crossref PubMed Scopus (261) Google Scholar, 18Shetty K.D. Battacharya J. Changes in hospital mortality associated with residency work-hour regulations.Ann Intern Med. 2007; 147: 73-80Crossref PubMed Scopus (145) Google Scholar Review of 7 such studies in 2004 found insufficient data to demonstrate effectiveness.14Fletcher K.E. Davis S.Q. Underwood W. et al.Systematic review: effects of resident work hours on patient safety.Ann Intern Med. 2004; 141: 851-857Crossref PubMed Scopus (174) Google Scholar Since then, one large study in pediatric patients detected significantly less resident "burn-out," but found no effect of the new regulations on hours of sleep or total hours of work. The authors reported a small increase in medication errors after the new hours.15Landrigan C.P. Fahrenkopf A.M. Lewin D. et al.Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety.Pediatrics. 2008; 122: 250-258Crossref PubMed Scopus (136) Google Scholar On the other hand, 3 recent studies demonstrated medical (but not surgical) mortality improvement among Medicare beneficiaries and in teaching hospitals (but not in non-teaching hospitals) comparing the years before and after reduction of resident hours.16Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 984-992Crossref PubMed Scopus (219) Google Scholar, 17Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 975-983Crossref PubMed Scopus (261) Google Scholar, 18Shetty K.D. Battacharya J. Changes in hospital mortality associated with residency work-hour regulations.Ann Intern Med. 2007; 147: 73-80Crossref PubMed Scopus (145) Google ScholarSuch disparity among studies examining the outcomes of regulation suggests that the extent (and perhaps therefore the effect) of decreasing resident hours may be local and variable. Only a credible method for measuring hours will confirm or refute this suggestion or provide a direct link between hours and outcomes. Several authors have suspected, and one has measured, substantial unreliability of time-card data.19Saunders D.L. Kehoe K.C. Rinehart V.H. Berg B.W. Self-reporting of internal medicine housestaff work hours.Hawaii Med J. 2005; 64: 14-16PubMed Google ScholarMaterials and MethodsWe aimed to validate self-report of duty hours (and of their distribution within the week) during a busy rotation in the medical intensive care unit using a more objective method: surveillance of all resident interactions with the electronic medical record.The Internal Medicine Residency Training Program at New York University Langone Medical Center rotates 164 residents through thirteen 28-day blocks per year at the University Hospital and two affiliated institutions. Typically, 2 blocks are spent in the medical intensive care unit. During 5 successive blocks between December 2008 and March 2009 (as previously for several years), we distributed index cards to all residents every 2 weeks during geriatrics, general medicine, intensive care, cardiology, hematology/oncology, and intensive care rotations at the University Hospital. In accordance with previous practice, residents were asked to be both truthful and precise in recalling the numbers of hours worked each day during the previous 2 weeks. Cards were distributed and collected on the same day.Residents were unaware that data also were collected from the electronic medical record system (Sunrise Clinical Manager, Eclypsis Corporation, Atlanta, Ga), capturing the times of all their interactions with the system. Interactions included entering orders, viewing results, writing notes, and opening notes to read them. We hypothesized that the data-rich and quickly changing clinical environment in our medical intensive care unit demanded frequent interaction with the electronic medical record; we proposed that residents could not therefore be engaged in patient care in this milieu without regular recourse to the computer.Residents were considered working if no 6-hour period passed without their contacting the institution's electronic medical record. We recorded a resident as having left work at the time of last computer interaction before a 6-hour hiatus. An Excel (Microsoft Corp, Redmond, WA) spreadsheet with preprogrammed calculated columns permitted rapid determination of the beginning and end of resident work days by the noted criteria. In addition, the spreadsheet automatically summed the number of violations specific to each of the 4 hour-related rules set by the Residency Review Committee in Internal Medicine following Accreditation Council for Graduate Medical Education standards.20Accreditation Council for Graduate Medical EducationInternal Medicine Program Requirements.http://www.acgme.org/acWebsite/RRC_140/140_prIndex.aspGoogle ScholarThe spreadsheet, written by one of our nontechnical authors (D.S.), required approximately 6 hours to construct and test, as well as 15 minutes to apply to each monthly data set. The downloaded electronic medical record data were pasted into the spreadsheet and required 10 minutes for our technical author (B.W.) to enter resident names and run each monthly data set.At the end of the 4-month study period, we compared total resident hours and occurrence of violations between electronic medical record surveillance and resident self-report. Fractional hours on time-cards were rounded to whole numbers. Violations were expressed as a fraction in which the numerator was the measured number of violations and the denominator was the number of opportunities for violation, based on the number of days for which each resident submitted time-cards. Table 1 defines the 4 violation types and the method for applying those definitions to time-cards and to surveillance downloads from the electronic medical record.Table 1Explanation and Details of Hours ViolationsBrief Regulation NameSummary of RegulationHow Regulation Was Applied to EMRHow Regulation Was Applied to Time-CardsViolations among Residents Submitting Cards (n=28)Violations Among Residents Not Submitting Cards (n=8)a Potential Violationsb Observed Violations by Time-Card (%)c Observed Violations by EMR (% of Potential)b/a vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)d Potential Violations on Dates with Computer Contacte Observed Violations by EMR (% of Potential)e/d vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)80-hResidents must not spend>80 h per week in a training site. Weekly hours may be averaged over the individual rotation.Starting with the first scheduled day in MICU on which there was computer activity, working periods were continuously summed until exactly 1 week later, when the process began again. Any week with summed working periods>80 h was considered in violation.Hours were tallied for each week, starting with days 1 and 8 on the card. Any week>80 h was considered in violation.715 (7%)1 (1.4%)P=.21*140 (0%)P=.4*27-hResidents must not spend>24 h on call. A 3-h period is allowed for signout of patients.Any continuous period>27 hours was considered in violation.Any continuous period>27 h was considered in violation.1549 (5.8%)9 (5.8%)P=1.0322 (6.3%)P=.6*24-hResidents must have 24 h away from all training sites in each week.Any week (as defined for 80-h violations above) in which there was not a single≥24-h period with no computer activity was considered in violation.Any week (days 1-7 or 8-14) on the time-card in which there was not a stated or easily calculable 24-h period off was considered in violation.710 (0%)2 (2.8%)P=.5*140 (0%)P=.7*10-hResidents must have 10 h away from all training sites after each shift.Time of first EMR interaction after a 6-h hiatus is considered a violation if it is > 10 h from the time of the last interaction.Time intervals > 10 h between reported end of a shift and reported start of another is considered a violation.4621 (0.2%)3 (0.6%)P=.4*9710 (3.1%)P=.06*EMR=electronic medical record; MICU=medical intensive care unit.Definitions of hours regulations, electronic medical record, and time-card criteria for violation used in the study, and distribution of violations and percentage occurrence using electronic medical record surveillance and time-card review. Open table in a new tab For residents submitting time-cards, we calculated potential violations and reported data only for those days on which the cards were submitted. We tested the significance of differences in occurrence of individual violation types between the 2 methods using the Fisher exact test (for<6 occurrences) and otherwise chi-square testing with continuity correction. For residents not submitting any cards, we calculated potential violations and reported hours data for days that were scheduled and on which electronic medical record reflected any interaction. No data were reported for days left undocumented by residents submitting incomplete time-cards.Thirty-six residents rotated through the medical intensive care unit during the 4 study months, accounting for 744 scheduled resident days. Twenty-eight residents returned time-cards, accounting for 462 days and ranging from 11% to 100% of scheduled days. Eighteen responding residents reported all their scheduled days, and 4 residents reported less than one half of their scheduled days. Eight residents did not respond at all. Relying only on electronic medical record data, we tested for significance differences in occurrence of each violation type between those residents who did and did not hand in any cards.ResultsResponding residents reported a total 4383 hours; the electronic medical record surveillance report returned 4062 for these same days, a 7.3% discrepancy. Considering only scheduled days for which time-cards had been submitted, there were 71 opportunities to violate the 80-hour work week rule and the same number of opportunities to violate the standard of 1 day off weekly. There were 154 opportunities to violate the 27-hour rule and 462 opportunities to violate the requirement for 10 hours off between shifts (Table 1). The 758 possible total violations (1.9% of which actually occurred according to time-cards) yielded 80% power to detect a significant (P < .05) difference from the time-cards method it frequency of violation by electronic surveillance was less than 0.5% or more than 4.5%.On review of time-cards, there were five 80-hour violations (7%), nine 27-hour violations (5.8%), no 24-hour violations (0%), and one 10-hour violation (0.2%). There were therefore a total of 15 time-card violations (1.9%). By electronic medical record surveillance there was one 80-hour violation (1.4%), nine 27-hour violations (5.8%), two 24-hour violations (2.8%), and three 10-hour violations (0.6%). Total violations by electronic medical record were 15 (1.9%). Occurrence of individual violations did not differ significantly between methods.Among the 8 residents who did not return time-cards, there were no 80-hour violations (0%), two 27-hour violations (6.3%), no 24-hour violations (0%), and three 10-hour violations (3.1%). Total violations were therefore 5 (5.2%). There was no significant difference in occurrence of any violation type between residents who did and did not return any cards, although occurrence of 10-hour violations was 3% among individuals who did not return cards and 0.6% among residents who did return cards (P=.06).Electronic medical record surveillance and time-card completion were approximately equivalent methods for measuring resident duty hours and occurrence of hours violations. Average total hours were 7.3% higher by time-cards, and total violations were identical. Electronic medical record surveillance had the advantages of ready availability and more complete data. The 8 noncompliant residents cannot be assumed to resemble the compliant 28 residents. It is easy to imagine, for example, that compliance in returning cards might have been lower in residents with more frequent hours violations. In fact, we found a trend toward more 10-hour violations among noncompliant residents.A 7.3% disparity was not unexpected. The Accreditation Council for Graduate Medical Education regulations require not only that residents go off duty at the correct time but also that they physically leave the training site. We reasoned that a resident using the computer is always at work (high positive predictive value), but that a resident not using the computer may be still in the hospital (lower negative predictive value). The electronic medical record surveillance method is thus likely only to underestimate duty hours. Detected violations by electronic medical record provide strong evidence that a problem exists; apparent compliance measured by electronic surveillance, on the other hand, cannot entirely verify compliance.LimitationsLimitations of this study include its performance at a single institution with an advanced electronic medical record and among only those residents who were assigned to the medical intensive care unit of the University Hospital. It is likely that residents rotating in areas where the patients are less ill will interact less frequently with the electronic medical record; underestimates of hours might therefore be greater than in a medical intensive care unit. The extent of additional underestimation will determine how much less useful this method is on a ward rotation than it seemed to be in an intensive care setting.Electronic medical record surveillance is easily employed in hospitals with developed systems. This method is inexpensive, provides information about residents who do not return cards, produces real-time results, and requires little special expertise. However, it is but one of several possible solutions to the alleged uncertainties of time-cards. Electronic monitoring of transmitter-fitted resident badges might yield times of coming and going, and the addition of global positioning technology might even monitor location within the hospital. Expense may be a limiting factor. Malfunction and failure to wear the badge recreates the time-cards problem of interpreting missing data. Signing in and out is a far cheaper alternative, but subject to noncompliance.ConclusionsOur finding that residents—compared with an independent and objective data source—accurately recorded their own hours and violations must be confirmed in other institutions and especially in less intensive clinical settings. This study suggests a possible alternative to time-cards but also provides validation for their use and therefore supports a method frequently used in published studies to measure the impact of resident duty hours on patient and resident outcomes. The impact of resident duty hours on clinical and educational outcomes continues to concern professional and government oversight groups. Duty hours regulations have both stimulated and responded to a growing literature; but even after considerable study, the relationships remain uncertain among sleep, fatigue, effective education, hospital working conditions, hand-offs, patient safety, and resident burn-out. To be persuasive, reports investigating these relationships must accurately and completely measure the independent variable: resident duty hours. The almost universal practice of deriving duty hours from retrospectively completed time-cards, whether actual or computerized, has been criticized as potentially biased and poorly reproducible. •Regulatory oversight of resident work schedules has obliged training programs to monitor resident hours closely. Work time is usually measured by self-report using resident time-cards, actual or computerized.•This study compares measurement of hours using paper, resident-completed time-cards to automated reports of resident interactions with the institution's electronic medical record (EMR).•EMR surveillance yielded information similar to time-cards with greater ease and timeliness. •Regulatory oversight of resident work schedules has obliged training programs to monitor resident hours closely. Work time is usually measured by self-report using resident time-cards, actual or computerized.•This study compares measurement of hours using paper, resident-completed time-cards to automated reports of resident interactions with the institution's electronic medical record (EMR).•EMR surveillance yielded information similar to time-cards with greater ease and timeliness. Regulation of resident duty hours began as a legislative initiative in New York 25 years ago.1NY Comp. Codes R. & Regs. Section 405.4.2002.Google Scholar This local issue appeared nationally in 2002 when the Occupational Safety and Health Administration, under petition by 3 concerned groups, first considered limiting hours for the sake of both resident and patient safety.2Public Citizen Health Research GroupMedical residents; work hours. Public Citizen, New York2003www.citizen.org/hrg/healthcare/articles.cfm?ID=6666Google Scholar, 3American Medical Student AssociationThe Resident Work Hour Issue.www.amsa.org/rwh/index.cfmGoogle Scholar, 4Committee of Interns and Residentswww.cirseiu.org/assets/Google Scholar Anticipating federal regulation, the Accreditation Council for Graduate Medical Education, a professional certification body, issued requirements the following year and subsequently to all training programs, regardless of specialty.5Philibert I. Friedmann P. Williams W.T. New requirements for resident duty hours ACGME. Work Group on Resident Duty Hours.JAMA. 2002; 288: 1112-1114Crossref PubMed Scopus (403) Google Scholar, 6Accreditation Council for Graduate Medical EducationInnovative Approaches for Resident Duty Hours.http://www.acgme.org/acWebsite/dutyHours/dh_innovative.aspGoogle Scholar, 7Institute of Medicinehttp://www.iom.edu/CMS/3809/48553/60449.aspxGoogle Scholar, 8Council Directive 93/104/EC.Official Journal of the European Community. 1993; L307: 18-24Google Scholar Most recently, the Institute of Medicine, an influential watchdog organization, has recommended that resident hours be further reduced to approach requirements in Europe. The European Working Time Directive requires that medical training be limited to 48 hours per week with 11 hours of rest in each 24-hour period.7Institute of Medicinehttp://www.iom.edu/CMS/3809/48553/60449.aspxGoogle Scholar, 8Council Directive 93/104/EC.Official Journal of the European Community. 1993; L307: 18-24Google Scholar Lack of sleep impairs vigilance and task-oriented performance.9Denisco R.A. Drummond J.N. Gravenstein J.S. The effect of fatigue on the performance of a simulated anesthetic monitoring task.J Clin Monit. 1987; 3: 22-24Crossref PubMed Scopus (97) Google Scholar, 10Barger L.K. Cade B.E. Ayas N.T. et al.Extended works shifts and the risk of motor vehicle crashes among interns.N Engl J Med. 2005; 352: 125-134Crossref PubMed Scopus (714) Google Scholar, 11Ayas N.T. Barger L.K. Cade B.E. et al.Extended work duration and the risk of self reported percutaneous injuries in interns.JAMA. 2006; 296: 1055-1062Crossref PubMed Scopus (305) Google Scholar Less clear is the effect of fatigue on cognitive and team-based activities, such as rounding. Other workplace variables may bear as much on impairment and fatigue as do hours of sleep.12Arora V.M. Georgitis E. Siddique J. et al.Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities.JAMA. 2008; 300: 1146-1153Crossref PubMed Scopus (83) Google Scholar, 13Friesen L.D. Vidyarthi A.R. Baron R.B. Katz P.P. Factors associated with intern fatigue.J Gen Intern Med. 2008; 23: 1981-1986Crossref PubMed Scopus (27) Google Scholar At least a dozen investigators have systematically examined changes in patient outcomes before and after Accreditation Council for Graduate Medical Education duty hours regulation implementation.14Fletcher K.E. Davis S.Q. Underwood W. et al.Systematic review: effects of resident work hours on patient safety.Ann Intern Med. 2004; 141: 851-857Crossref PubMed Scopus (174) Google Scholar, 15Landrigan C.P. Fahrenkopf A.M. Lewin D. et al.Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety.Pediatrics. 2008; 122: 250-258Crossref PubMed Scopus (136) Google Scholar, 16Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 984-992Crossref PubMed Scopus (219) Google Scholar, 17Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 975-983Crossref PubMed Scopus (261) Google Scholar, 18Shetty K.D. Battacharya J. Changes in hospital mortality associated with residency work-hour regulations.Ann Intern Med. 2007; 147: 73-80Crossref PubMed Scopus (145) Google Scholar Review of 7 such studies in 2004 found insufficient data to demonstrate effectiveness.14Fletcher K.E. Davis S.Q. Underwood W. et al.Systematic review: effects of resident work hours on patient safety.Ann Intern Med. 2004; 141: 851-857Crossref PubMed Scopus (174) Google Scholar Since then, one large study in pediatric patients detected significantly less resident "burn-out," but found no effect of the new regulations on hours of sleep or total hours of work. The authors reported a small increase in medication errors after the new hours.15Landrigan C.P. Fahrenkopf A.M. Lewin D. et al.Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety.Pediatrics. 2008; 122: 250-258Crossref PubMed Scopus (136) Google Scholar On the other hand, 3 recent studies demonstrated medical (but not surgical) mortality improvement among Medicare beneficiaries and in teaching hospitals (but not in non-teaching hospitals) comparing the years before and after reduction of resident hours.16Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 984-992Crossref PubMed Scopus (219) Google Scholar, 17Volpp K.G. Rosen A.K. Rosenbaum P.R. et al.Mortality among patients in Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.JAMA. 2007; 298: 975-983Crossref PubMed Scopus (261) Google Scholar, 18Shetty K.D. Battacharya J. Changes in hospital mortality associated with residency work-hour regulations.Ann Intern Med. 2007; 147: 73-80Crossref PubMed Scopus (145) Google Scholar Such disparity among studies examining the outcomes of regulation suggests that the extent (and perhaps therefore the effect) of decreasing resident hours may be local and variable. Only a credible method for measuring hours will confirm or refute this suggestion or provide a direct link between hours and outcomes. Several authors have suspected, and one has measured, substantial unreliability of time-card data.19Saunders D.L. Kehoe K.C. Rinehart V.H. Berg B.W. Self-reporting of internal medicine housestaff work hours.Hawaii Med J. 2005; 64: 14-16PubMed Google Scholar Materials and MethodsWe aimed to validate self-report of duty hours (and of their distribution within the week) during a busy rotation in the medical intensive care unit using a more objective method: surveillance of all resident interactions with the electronic medical record.The Internal Medicine Residency Training Program at New York University Langone Medical Center rotates 164 residents through thirteen 28-day blocks per year at the University Hospital and two affiliated institutions. Typically, 2 blocks are spent in the medical intensive care unit. During 5 successive blocks between December 2008 and March 2009 (as previously for several years), we distributed index cards to all residents every 2 weeks during geriatrics, general medicine, intensive care, cardiology, hematology/oncology, and intensive care rotations at the University Hospital. In accordance with previous practice, residents were asked to be both truthful and precise in recalling the numbers of hours worked each day during the previous 2 weeks. Cards were distributed and collected on the same day.Residents were unaware that data also were collected from the electronic medical record system (Sunrise Clinical Manager, Eclypsis Corporation, Atlanta, Ga), capturing the times of all their interactions with the system. Interactions included entering orders, viewing results, writing notes, and opening notes to read them. We hypothesized that the data-rich and quickly changing clinical environment in our medical intensive care unit demanded frequent interaction with the electronic medical record; we proposed that residents could not therefore be engaged in patient care in this milieu without regular recourse to the computer.Residents were considered working if no 6-hour period passed without their contacting the institution's electronic medical record. We recorded a resident as having left work at the time of last computer interaction before a 6-hour hiatus. An Excel (Microsoft Corp, Redmond, WA) spreadsheet with preprogrammed calculated columns permitted rapid determination of the beginning and end of resident work days by the noted criteria. In addition, the spreadsheet automatically summed the number of violations specific to each of the 4 hour-related rules set by the Residency Review Committee in Internal Medicine following Accreditation Council for Graduate Medical Education standards.20Accreditation Council for Graduate Medical EducationInternal Medicine Program Requirements.http://www.acgme.org/acWebsite/RRC_140/140_prIndex.aspGoogle ScholarThe spreadsheet, written by one of our nontechnical authors (D.S.), required approximately 6 hours to construct and test, as well as 15 minutes to apply to each monthly data set. The downloaded electronic medical record data were pasted into the spreadsheet and required 10 minutes for our technical author (B.W.) to enter resident names and run each monthly data set.At the end of the 4-month study period, we compared total resident hours and occurrence of violations between electronic medical record surveillance and resident self-report. Fractional hours on time-cards were rounded to whole numbers. Violations were expressed as a fraction in which the numerator was the measured number of violations and the denominator was the number of opportunities for violation, based on the number of days for which each resident submitted time-cards. Table 1 defines the 4 violation types and the method for applying those definitions to time-cards and to surveillance downloads from the electronic medical record.Table 1Explanation and Details of Hours ViolationsBrief Regulation NameSummary of RegulationHow Regulation Was Applied to EMRHow Regulation Was Applied to Time-CardsViolations among Residents Submitting Cards (n=28)Violations Among Residents Not Submitting Cards (n=8)a Potential Violationsb Observed Violations by Time-Card (%)c Observed Violations by EMR (% of Potential)b/a vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)d Potential Violations on Dates with Computer Contacte Observed Violations by EMR (% of Potential)e/d vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)80-hResidents must not spend>80 h per week in a training site. Weekly hours may be averaged over the individual rotation.Starting with the first scheduled day in MICU on which there was computer activity, working periods were continuously summed until exactly 1 week later, when the process began again. Any week with summed working periods>80 h was considered in violation.Hours were tallied for each week, starting with days 1 and 8 on the card. Any week>80 h was considered in violation.715 (7%)1 (1.4%)P=.21*140 (0%)P=.4*27-hResidents must not spend>24 h on call. A 3-h period is allowed for signout of patients.Any continuous period>27 hours was considered in violation.Any continuous period>27 h was considered in violation.1549 (5.8%)9 (5.8%)P=1.0322 (6.3%)P=.6*24-hResidents must have 24 h away from all training sites in each week.Any week (as defined for 80-h violations above) in which there was not a single≥24-h period with no computer activity was considered in violation.Any week (days 1-7 or 8-14) on the time-card in which there was not a stated or easily calculable 24-h period off was considered in violation.710 (0%)2 (2.8%)P=.5*140 (0%)P=.7*10-hResidents must have 10 h away from all training sites after each shift.Time of first EMR interaction after a 6-h hiatus is considered a violation if it is > 10 h from the time of the last interaction.Time intervals > 10 h between reported end of a shift and reported start of another is considered a violation.4621 (0.2%)3 (0.6%)P=.4*9710 (3.1%)P=.06*EMR=electronic medical record; MICU=medical intensive care unit.Definitions of hours regulations, electronic medical record, and time-card criteria for violation used in the study, and distribution of violations and percentage occurrence using electronic medical record surveillance and time-card review. Open table in a new tab For residents submitting time-cards, we calculated potential violations and reported data only for those days on which the cards were submitted. We tested the significance of differences in occurrence of individual violation types between the 2 methods using the Fisher exact test (for<6 occurrences) and otherwise chi-square testing with continuity correction. For residents not submitting any cards, we calculated potential violations and reported hours data for days that were scheduled and on which electronic medical record reflected any interaction. No data were reported for days left undocumented by residents submitting incomplete time-cards.Thirty-six residents rotated through the medical intensive care unit during the 4 study months, accounting for 744 scheduled resident days. Twenty-eight residents returned time-cards, accounting for 462 days and ranging from 11% to 100% of scheduled days. Eighteen responding residents reported all their scheduled days, and 4 residents reported less than one half of their scheduled days. Eight residents did not respond at all. Relying only on electronic medical record data, we tested for significance differences in occurrence of each violation type between those residents who did and did not hand in any cards. We aimed to validate self-report of duty hours (and of their distribution within the week) during a busy rotation in the medical intensive care unit using a more objective method: surveillance of all resident interactions with the electronic medical record. The Internal Medicine Residency Training Program at New York University Langone Medical Center rotates 164 residents through thirteen 28-day blocks per year at the University Hospital and two affiliated institutions. Typically, 2 blocks are spent in the medical intensive care unit. During 5 successive blocks between December 2008 and March 2009 (as previously for several years), we distributed index cards to all residents every 2 weeks during geriatrics, general medicine, intensive care, cardiology, hematology/oncology, and intensive care rotations at the University Hospital. In accordance with previous practice, residents were asked to be both truthful and precise in recalling the numbers of hours worked each day during the previous 2 weeks. Cards were distributed and collected on the same day. Residents were unaware that data also were collected from the electronic medical record system (Sunrise Clinical Manager, Eclypsis Corporation, Atlanta, Ga), capturing the times of all their interactions with the system. Interactions included entering orders, viewing results, writing notes, and opening notes to read them. We hypothesized that the data-rich and quickly changing clinical environment in our medical intensive care unit demanded frequent interaction with the electronic medical record; we proposed that residents could not therefore be engaged in patient care in this milieu without regular recourse to the computer. Residents were considered working if no 6-hour period passed without their contacting the institution's electronic medical record. We recorded a resident as having left work at the time of last computer interaction before a 6-hour hiatus. An Excel (Microsoft Corp, Redmond, WA) spreadsheet with preprogrammed calculated columns permitted rapid determination of the beginning and end of resident work days by the noted criteria. In addition, the spreadsheet automatically summed the number of violations specific to each of the 4 hour-related rules set by the Residency Review Committee in Internal Medicine following Accreditation Council for Graduate Medical Education standards.20Accreditation Council for Graduate Medical EducationInternal Medicine Program Requirements.http://www.acgme.org/acWebsite/RRC_140/140_prIndex.aspGoogle Scholar The spreadsheet, written by one of our nontechnical authors (D.S.), required approximately 6 hours to construct and test, as well as 15 minutes to apply to each monthly data set. The downloaded electronic medical record data were pasted into the spreadsheet and required 10 minutes for our technical author (B.W.) to enter resident names and run each monthly data set. At the end of the 4-month study period, we compared total resident hours and occurrence of violations between electronic medical record surveillance and resident self-report. Fractional hours on time-cards were rounded to whole numbers. Violations were expressed as a fraction in which the numerator was the measured number of violations and the denominator was the number of opportunities for violation, based on the number of days for which each resident submitted time-cards. Table 1 defines the 4 violation types and the method for applying those definitions to time-cards and to surveillance downloads from the electronic medical record. EMR=electronic medical record; MICU=medical intensive care unit. Definitions of hours regulations, electronic medical record, and time-card criteria for violation used in the study, and distribution of violations and percentage occurrence using electronic medical record surveillance and time-card review. For residents submitting time-cards, we calculated potential violations and reported data only for those days on which the cards were submitted. We tested the significance of differences in occurrence of individual violation types between the 2 methods using the Fisher exact test (for<6 occurrences) and otherwise chi-square testing with continuity correction. For residents not submitting any cards, we calculated potential violations and reported hours data for days that were scheduled and on which electronic medical record reflected any interaction. No data were reported for days left undocumented by residents submitting incomplete time-cards. Thirty-six residents rotated through the medical intensive care unit during the 4 study months, accounting for 744 scheduled resident days. Twenty-eight residents returned time-cards, accounting for 462 days and ranging from 11% to 100% of scheduled days. Eighteen responding residents reported all their scheduled days, and 4 residents reported less than one half of their scheduled days. Eight residents did not respond at all. Relying only on electronic medical record data, we tested for significance differences in occurrence of each violation type between those residents who did and did not hand in any cards. ResultsResponding residents reported a total 4383 hours; the electronic medical record surveillance report returned 4062 for these same days, a 7.3% discrepancy. Considering only scheduled days for which time-cards had been submitted, there were 71 opportunities to violate the 80-hour work week rule and the same number of opportunities to violate the standard of 1 day off weekly. There were 154 opportunities to violate the 27-hour rule and 462 opportunities to violate the requirement for 10 hours off between shifts (Table 1). The 758 possible total violations (1.9% of which actually occurred according to time-cards) yielded 80% power to detect a significant (P < .05) difference from the time-cards method it frequency of violation by electronic surveillance was less than 0.5% or more than 4.5%.On review of time-cards, there were five 80-hour violations (7%), nine 27-hour violations (5.8%), no 24-hour violations (0%), and one 10-hour violation (0.2%). There were therefore a total of 15 time-card violations (1.9%). By electronic medical record surveillance there was one 80-hour violation (1.4%), nine 27-hour violations (5.8%), two 24-hour violations (2.8%), and three 10-hour violations (0.6%). Total violations by electronic medical record were 15 (1.9%). Occurrence of individual violations did not differ significantly between methods.Among the 8 residents who did not return time-cards, there were no 80-hour violations (0%), two 27-hour violations (6.3%), no 24-hour violations (0%), and three 10-hour violations (3.1%). Total violations were therefore 5 (5.2%). There was no significant difference in occurrence of any violation type between residents who did and did not return any cards, although occurrence of 10-hour violations was 3% among individuals who did not return cards and 0.6% among residents who did return cards (P=.06).Electronic medical record surveillance and time-card completion were approximately equivalent methods for measuring resident duty hours and occurrence of hours violations. Average total hours were 7.3% higher by time-cards, and total violations were identical. Electronic medical record surveillance had the advantages of ready availability and more complete data. The 8 noncompliant residents cannot be assumed to resemble the compliant 28 residents. It is easy to imagine, for example, that compliance in returning cards might have been lower in residents with more frequent hours violations. In fact, we found a trend toward more 10-hour violations among noncompliant residents.A 7.3% disparity was not unexpected. The Accreditation Council for Graduate Medical Education regulations require not only that residents go off duty at the correct time but also that they physically leave the training site. We reasoned that a resident using the computer is always at work (high positive predictive value), but that a resident not using the computer may be still in the hospital (lower negative predictive value). The electronic medical record surveillance method is thus likely only to underestimate duty hours. Detected violations by electronic medical record provide strong evidence that a problem exists; apparent compliance measured by electronic surveillance, on the other hand, cannot entirely verify compliance. Responding residents reported a total 4383 hours; the electronic medical record surveillance report returned 4062 for these same days, a 7.3% discrepancy. Considering only scheduled days for which time-cards had been submitted, there were 71 opportunities to violate the 80-hour work week rule and the same number of opportunities to violate the standard of 1 day off weekly. There were 154 opportunities to violate the 27-hour rule and 462 opportunities to violate the requirement for 10 hours off between shifts (Table 1). The 758 possible total violations (1.9% of which actually occurred according to time-cards) yielded 80% power to detect a significant (P < .05) difference from the time-cards method it frequency of violation by electronic surveillance was less than 0.5% or more than 4.5%. On review of time-cards, there were five 80-hour violations (7%), nine 27-hour violations (5.8%), no 24-hour violations (0%), and one 10-hour violation (0.2%). There were therefore a total of 15 time-card violations (1.9%). By electronic medical record surveillance there was one 80-hour violation (1.4%), nine 27-hour violations (5.8%), two 24-hour violations (2.8%), and three 10-hour violations (0.6%). Total violations by electronic medical record were 15 (1.9%). Occurrence of individual violations did not differ significantly between methods. Among the 8 residents who did not return time-cards, there were no 80-hour violations (0%), two 27-hour violations (6.3%), no 24-hour violations (0%), and three 10-hour violations (3.1%). Total violations were therefore 5 (5.2%). There was no significant difference in occurrence of any violation type between residents who did and did not return any cards, although occurrence of 10-hour violations was 3% among individuals who did not return cards and 0.6% among residents who did return cards (P=.06). Electronic medical record surveillance and time-card completion were approximately equivalent methods for measuring resident duty hours and occurrence of hours violations. Average total hours were 7.3% higher by time-cards, and total violations were identical. Electronic medical record surveillance had the advantages of ready availability and more complete data. The 8 noncompliant residents cannot be assumed to resemble the compliant 28 residents. It is easy to imagine, for example, that compliance in returning cards might have been lower in residents with more frequent hours violations. In fact, we found a trend toward more 10-hour violations among noncompliant residents. A 7.3% disparity was not unexpected. The Accreditation Council for Graduate Medical Education regulations require not only that residents go off duty at the correct time but also that they physically leave the training site. We reasoned that a resident using the computer is always at work (high positive predictive value), but that a resident not using the computer may be still in the hospital (lower negative predictive value). The electronic medical record surveillance method is thus likely only to underestimate duty hours. Detected violations by electronic medical record provide strong evidence that a problem exists; apparent compliance measured by electronic surveillance, on the other hand, cannot entirely verify compliance. LimitationsLimitations of this study include its performance at a single institution with an advanced electronic medical record and among only those residents who were assigned to the medical intensive care unit of the University Hospital. It is likely that residents rotating in areas where the patients are less ill will interact less frequently with the electronic medical record; underestimates of hours might therefore be greater than in a medical intensive care unit. The extent of additional underestimation will determine how much less useful this method is on a ward rotation than it seemed to be in an intensive care setting.Electronic medical record surveillance is easily employed in hospitals with developed systems. This method is inexpensive, provides information about residents who do not return cards, produces real-time results, and requires little special expertise. However, it is but one of several possible solutions to the alleged uncertainties of time-cards. Electronic monitoring of transmitter-fitted resident badges might yield times of coming and going, and the addition of global positioning technology might even monitor location within the hospital. Expense may be a limiting factor. Malfunction and failure to wear the badge recreates the time-cards problem of interpreting missing data. Signing in and out is a far cheaper alternative, but subject to noncompliance. Limitations of this study include its performance at a single institution with an advanced electronic medical record and among only those residents who were assigned to the medical intensive care unit of the University Hospital. It is likely that residents rotating in areas where the patients are less ill will interact less frequently with the electronic medical record; underestimates of hours might therefore be greater than in a medical intensive care unit. The extent of additional underestimation will determine how much less useful this method is on a ward rotation than it seemed to be in an intensive care setting. Electronic medical record surveillance is easily employed in hospitals with developed systems. This method is inexpensive, provides information about residents who do not return cards, produces real-time results, and requires little special expertise. However, it is but one of several possible solutions to the alleged uncertainties of time-cards. Electronic monitoring of transmitter-fitted resident badges might yield times of coming and going, and the addition of global positioning technology might even monitor location within the hospital. Expense may be a limiting factor. Malfunction and failure to wear the badge recreates the time-cards problem of interpreting missing data. Signing in and out is a far cheaper alternative, but subject to noncompliance. ConclusionsOur finding that residents—compared with an independent and objective data source—accurately recorded their own hours and violations must be confirmed in other institutions and especially in less intensive clinical settings. This study suggests a possible alternative to time-cards but also provides validation for their use and therefore supports a method frequently used in published studies to measure the impact of resident duty hours on patient and resident outcomes. Our finding that residents—compared with an independent and objective data source—accurately recorded their own hours and violations must be confirmed in other institutions and especially in less intensive clinical settings. This study suggests a possible alternative to time-cards but also provides validation for their use and therefore supports a method frequently used in published studies to measure the impact of resident duty hours on patient and resident outcomes.