F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.21130.1 Correspondence Articles iCOMPARE, what value does it add to resident duty-hour discussions?

[version 1; peer review: 1 approved with reservations]

 Hopkins Zachary H. Conceptualization Resources Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-7248-3281 a 1 Secrest Aaron M. Supervision Writing – Review & Editing https://orcid.org/0000-0001-9297-0941 2 Dermatology, Broward Medical Center, Fort Lauderdale, Florida, 33316, USA Dermatology and Population Health Sciences, University of Utah Health, Salt Lake City, Utah, 84132, USA zhopkinsderm@gmail.com

No competing interests were disclosed.

 5 11 2019 2019 8 1856 29 10 2019 Copyright: © 2019 Hopkins ZH and Secrest AM 2019 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Discussions regarding resident duty-hour restrictions have been ongoing and heated. One influential argument for restrictions has been patient safety. Two trials, FIRST and iCOMPARE, were performed to investigate this relationship with surgical and medicine training, respectively. As the authors are approaching this discussion from a medicine-based perspective, iCOMPARE will serve as the primary basis of our discussion. Results from the iCOMPARE trial comparing flexible (28-hour shifts allowed) to the original 2011 ACGME shift requirements (maximum 16 hours) were recently published in the New England Journal of Medicine. This non-inferiority trial used 30-day post-hospitalization mortality as its primary endpoint. Results met qualifications for non-inferiority, and ACGME policy was changed to allow for 28-hour shifts for medicine residents. iCOMPARE results were highly lauded and used as primary justification for extending resident duty hours. Despite this sweeping impact, few have critically evaluated what this study actually adds to the literature. Herein, we argue that serious questions regarding trial design are apparent. Most importantly, the non-inferiority margins chosen were large, and represent an ambiguous marker of resident performance. Additionally, we question the lack of both patient consenting and direct patient-reported or patient-centered outcomes within the hospital stay. As more discussion arises in the medical literature surrounding patient-reported outcomes and shared decision making, we argue that the results of iCOMPARE disregarded the patient perspective or meaningful patient outcomes in an attempt to maintain status quo. Lastly, we discuss how iCOMPARE missed the broader question of actual duty-hour restrictions, and some practical methods already in practice at some programs, which may more directly balance resident work hours with patient care and resident learning.

 General Medial Education Resident Work Hours Resident Duty Hours Resident Wellness Medical Education iCOMPARE The author(s) declared that no grants were involved in supporting this work. <p>Mr. C was friendly and rapport came quickly. He ultimately needed thoracentesis and during the ensuing discussion he perfunctorily asked, “How long have you been on your shift?” After replying, “12 hours,” he responded, “I understand the necessity, but I want someone performing their best for this procedure. Can you wait until morning?” Fortunately, his case could be safely deferred until morning. This was neither the first, nor the last, time during my training a patient asked this question. </p> <p>Recently, the <italic toggle="yes">New England Journal of Medicine</italic> published the final primary endpoint analysis for iCOMPARE with an accompanying editorial <sup> <xref ref-type="bibr" rid="ref-1">1</xref>, <xref ref-type="bibr" rid="ref-2">2</xref> </sup>. This trial sought to evaluate the impact of flexible (program directed shift lengths up to 28 hours) versus 16-hour capped shifts on 30-day post-hospitalization patient mortality. As the only trial investigating duty-hour effects on internal medicine patients, its impact is large. Indeed, in the ACGME’s new duty-hour guidelines, results from the iCOMPARE and FIRST (similar study from surgical programs), are cited as a major justification for this change <sup> <xref ref-type="bibr" rid="ref-3">3</xref> </sup>. However, as with any practice changing study, critical appraisal of its results and application is required.</p> <p>First, iCOMPARE lacked informed consent from either patients or residents. This is concerning with a primary endpoint of patient mortality <sup> <xref ref-type="bibr" rid="ref-4">4</xref> </sup>. One participant institution’s IRB application stated that they were not involved in human research because the residency program itself was the research subject—this claim seems implausible <sup> <xref ref-type="bibr" rid="ref-4">4</xref> </sup>. Patients are both interested in, and concerned by, the length of their doctors’ shifts <sup> <xref ref-type="bibr" rid="ref-5">5</xref> </sup>. Indeed, in one study, a significant number of patients stated they would like to be informed of any clinician who had worked over 12 hours <sup> <xref ref-type="bibr" rid="ref-6">6</xref> </sup>. Patients and residents deserve to be informed of factors affecting their care, particularly in a trial with the primary endpoint involving their 30-day mortality. For residents, this is especially concerning given their vulnerable workforce position. A 2004 lawsuit against the National Resident Matching Program asserted that residents were “forced to participate in a system that ensures they work long hours for low wages.” <sup> <xref ref-type="bibr" rid="ref-7">7</xref> </sup> However, after a strong lobbying effort, national legislation exempted the residency match process from anti-trust laws. In this setting the potential for exploitation is high, and policies should err on the side of fair work conditions.</p> <p>Third, the 1% non-inferiority margin utilized suggests that 2,646 additional deaths is an acceptable justification for longer shifts. Extrapolating this margin to the U.S. population, nearly 80,000 deaths annually would be considered “non-inferior.” Understandably, limitations in recruiting and study design apply, especially for non-inferiority. However, we also question the appropriateness of a non-inferiority design for this question. A proposed ethical prerequisite for using non-inferiority analysis is that an experimental therapy “must have known advantages such as reduced cost, greater convenience, or fewer side effects to justify the randomization of patients to a therapy with unknown efficacy.” <sup> <xref ref-type="bibr" rid="ref-8">8</xref>– <xref ref-type="bibr" rid="ref-11">11</xref> </sup> Long duty-hours have been linked to several resident, patient, and community safety concerns <sup> <xref ref-type="bibr" rid="ref-12">12</xref>– <xref ref-type="bibr" rid="ref-15">15</xref> </sup>. Likewise, patients do not appear to feel comfortable with physicians who worked long hours <sup> <xref ref-type="bibr" rid="ref-5">5</xref>, <xref ref-type="bibr" rid="ref-6">6</xref> </sup>. With these concerns, we argue that iCOMPARE should have demonstrated <italic toggle="yes">superiority</italic> of unrestricted shift lengths. Indeed, one oft-used argument for extended shift hours is fewer handoffs and improved continuity. Inherent in the design appears to be a belief of superiority otherwise why put people through it? Furthermore, the hours worked between the two groups were the same, thus it is wholly expected that no difference in endpoint be observed <sup> <xref ref-type="bibr" rid="ref-16">16</xref> </sup>. Why not choose a restricted hour work week, as is done in Europe, for the comparator? <sup> <xref ref-type="bibr" rid="ref-17">17</xref> </sup> Or, if extended lengths of shift are believed to be better for patients, why not use this shift structure as the comparator? Lastly, we suspect superiority in patient clinical endpoints would be the only results acceptable to patients for continuing a training system of which they do not approve.</p> <p>Finally, the primary endpoint of iCOMPARE is inadequately justified. Residency training should have sufficiently robust supervision that shift length has minimal effect on 30-day mortality. Orders made by an intern pass through supervising residents, attendings, pharmacists, and nurses. While errors can and do get through, post-hospitalization mortality is buffered from the source of errors in question. This is supported by one study showing an increased length of hospital stay and increased number of ICU admissions with long resident work hours, but a non-significant change in hospital readmissions and within-hospital mortality <sup> <xref ref-type="bibr" rid="ref-18">18</xref> </sup>. Likewise, we suspect 30-day mortality is not the only issue for which patients are concerned. Quantifying errors and near-misses made within the hospital as well as the patient experience could more directly evaluate causal links with resident performance. Additionally, while markers of resident performance (testing performance, wakefulness, etc.) were evaluated, we argue that with work hours being equal and minimal implementation of 28 hour shifts occurring between the two groups, no meaningful comparison between these groups can be derived <sup> <xref ref-type="bibr" rid="ref-19">19</xref>, <xref ref-type="bibr" rid="ref-20">20</xref> </sup>. With no difference in hours worked and a large non-inferiority margin of an ill-defined primary endpoint, we argue that little can be gained in regards to the preceeding data.</p> <p>While the idea of flexible shift structure is sensical as policy, it severely diluted the applicability and generalizability of the results. Currently, policy extends further than data. A program could, in theory, make a majority of its rotations utilize a 28-hour shift structure and this would be within policy. However, the safety of this structure is untested, as this was not directly evaluated in iCOMPARE. Further, even if the results were to be taken at face value, the reciprocity of the results is seemingly ignored. This trial, as a non-inferiority trial, demonstrated that capped shifts, with the changes in care they bring, do not worsen patient mortality. However, what if even these are too long? Perhaps both capped and flexible structures as they currently stand with 80-hour work week restrictions are inferior to a more restricted system? We posit that this is a more pressing and sensible question.</p> <p>We believe that the results of iCOMPARE are critically lacking—we can and must do better ( <xref ref-type="table" rid="T1">Table 1</xref>). We do not argue in favor of 2011 ACGME requirements nor the new adjustments. Rather, we suggest that iCOMPARE itself was constructed to test an artificial argument, and did so poorly. Thoughtfully tested data is necessary to further duty hour discussions and guide further policy refinement as well as an openness to consider new paradigms. Let us work to cultivate a group of compassionate, empathetic, and well-trained physicians; armed with not only with technical prowess, but the emotional reserve to respond to and alleviate patient suffering. In contrast to this goal, the iCOMPARE study is sadly a self-fulfilling prophecy—a superficial justification to continue the status quo.</p> <table-wrap id="T1" orientation="portrait" position="anchor"> <label>Table 1. </label> <caption> <title>Summary of concerns and proposed changes.
Issues Concerns Possible Changes
Lack of informed consent Residency programs were considered the study "subjects." Patients and residents were not required to give consent. This reasoning is flawed 4 . -In trials investigating patient safety, patients should be informed 4 . -Residents likewise should have been informed and consented as participants in the trial. Medical students should be made aware of programs participation pre- MATCH and current residents should be consented and involved in decisions to participate.
Culture -“I survived residency living at the hospital, so should everyone else." -Belief that the old way of training is the best way of training. -Medical practice has changed, with some change to trainees as well. If this is ignored, relationships between program leadership will suffer. -Cultivate a culture supportive of resident concerns and needs. -Collaborate with residents for flexible solutions. -Improved work culture may lead to a better culture of care for patients (more compassionate, empathetic, and emotionally engaged).
Exploitation -Residents are a cheap labor force. -Restricting duty-hours would likely require hospitals to expand coverage. -Bias of avoiding expanded coverage is apparent in iCOMPARE conclusions. -The data support similar patient safety for both duty-hour groups. However, the conclusions favor the flexible group. -These data suggest that restricted duty-hours are likewise non-inferior to flexible/ extended hours. -Allow resident input into schedule design. Some services may benefit from different structures and schedule requirements. In this regard we agree with the ACGME’s decision to move to flexible program driven shift-length schedules -Reasonable duty-hour restrictions that allow programs the flexibility to apply changes in a less onerous way.
Non-inferiority design -There is little justification for choosing a non-inferiority design when existing data suggests the active “treatment group” is potentially dangerous. -Work hours were virtually the same between groups, so no real difference in patient outcomes is to be expected. -The bias was in favor of extended duty-hours. If one wants to argue that extended duty-hours are preferred, with associated physician and patient risks, superiority should be demonstrated. -Future studies should compare restricted duty-hours (i.e., 60 hours), as this strikes at the heart of the issue.
Obscure outcome -30-day patient mortality is not ideal for studying resident duty-hours. -Intern orders pass through residents, attendings, pharmacists, nurses, and the electronic record, inherently limiting significant mortality-altering errors. -No resident safety factors were followed (needlestick injuries, well- being, post-call motor vehicle accidents, etc.) -Resident safety outcomes such as needlestick injuries, motor-vehicle accidents, and well-being/mental health disorders should be tracked. -Patient experience should be tracked -In-hospital errors and near-misses should be tracked.
Lack of concern for resident well-being -Previous results from iCOMPARE resident surveys demonstrate resident dissatisfaction with the flexible group 19 . -Resident safety outcomes were not tracked. -Conclusion in favor of flexible group despite no superiority in results and increased concern for resident harm. -4+1 scheduling (1 week of outpatient care with at least 1 full weekend for every 4 weeks of inpatient care). -Robust resident wellness programs. -Variation in call length and day length on inpatient rotations. At least one in every 4 days should be a “golden day” with no admits and the ability to leave after rounding to complete work.
Data availability

No data are associated with this article.

 Acknowledgements

We would like to thank Dr. Vinay Prasad for his helpful editorial comments on this manuscript.

 Rosenbaum L Lamas D : Eyes Wide Open — Examining the Data on Duty-Hour Reform. N Engl J Med. 2019;380(10):969–970. 30855748 10.1056/NEJMe1817497 Silber JH Bellini LM Shea JA : Patient Safety Outcomes under Flexible and Standard Resident Duty-Hour Rules. N Engl J Med. 2019;380(10):905–914. 30855740 10.1056/NEJMoa1810642 6476299 Mallory K Brigham T Nasca T : ACGME’s 2017 Revision of Common Program Requirements. Patient Safety Network. 2017; Published August 2017. Accessed August 12:2019. Reference Source Shepherd L Macklin R : Erosion of informed consent in U.S. research. Bioethics. 2019;33(1):4–12. 30474129 10.1111/bioe.12532 Blum AB Raiszadeh F Shea S : US public opinion regarding proposed limits on resident physician work hours. BMC Med. 2010;8:33. 20515479 10.1186/1741-7015-8-33 2901227 Drolet BC Hyman CH Ghaderi KF : Hospitalized Patients’ Perceptions of Resident Fatigue, Duty Hours, and Continuity of Care. J Grad Med Educ. 2014;6(4):658–63. 26140114 10.4300/JGME-D-14-00128.1 4477557 Robinson S : Antitrust Lawsuit Over Medical Residency System is Dismissed. The New York Times. 2004;A00016. Reference Source Aberegg SK Hersh AM Samore MH : Do non-inferiority trials of reduced intensity therapies show reduced effects? A descriptive analysis. BMJ Open. 2018;8(3):e019494. 29500210 10.1136/bmjopen-2017-019494 5855198 Aberegg SK Hersh AM Samore MH : Empirical Consequences of Current Recommendations for the Design and Interpretation of Noninferiority Trials. J Gen Intern Med. 2018;33(1):88–96. 28875400 10.1007/s11606-017-4161-4 5756156 Garattini S Bertele’ V : Non-inferiority trials are unethical because they disregard patients’ interests. The Lancet. 2007;370(9602):1875–7. 17959239 10.1016/S0140-6736(07)61604-3 Prasad V : Non-Inferiority Trials in Medicine: Practice Changing or a Self-Fulfilling Prophecy? J Gen Intern Med. 2018;33(1):3–5. 28980180 10.1007/s11606-017-4191-y 5756169 Sen S Kranzler HR Krystal JH : A prospective cohort study investigating factors associated with depression during medical internship. Arch Gen Psychiatry. 2010;67(6):557–565. 20368500 10.1001/archgenpsychiatry.2010.41 4036806 Ayas NT Barger LK Cade BE : Extended work duration and the risk of self-reported percutaneous injuries in interns. JAMA. 2006;296(9):1055–1062. 16954484 10.1001/jama.296.9.1055 Barger LK Cade BE Ayas NT : Extended work shifts and the risk of motor vehicle crashes among interns. N Engl J Med. 2005;352(2):125–134. 15647575 10.1056/NEJMoa041401 Ware JC Risser MR Manser T : Medical resident driving simulator performance following a night on call. Behav Sleep Med. 2006;4(1):1–12. 16390281 10.1207/s15402010bsm0401_1 Schumi J Wittes JT : Through the looking glass: understanding non-inferiority. Trials. 2011;12(1):106. 21539749 10.1186/1745-6215-12-106 3113981 RodriguezJareno MC Demou E Vargas-Prada S : European Working Time Directive and doctors’ health: a systematic review of the available epidemiological evidence. BMJ Open. 2014;4(7):e004916. 25001394 10.1136/bmjopen-2014-004916 4091509 Ouyang D Chen JH Krishnan G : Patient Outcomes when Housestaff Exceed 80 Hours per Week. Am J Medicine. 2016;129(9):993–999.e1. 27103047 10.1016/j.amjmed.2016.03.023 4996740 Desai SV Asch DA Bellini LM : Education Outcomes in a Duty-Hour Flexibility Trial in Internal Medicine. N Eng J Med. 2018;378(16):1494–1508. 29557719 10.1056/NEJMoa1800965 6101652 Basner M Asch DA Shea JA : Sleep and Alertness in a Duty-Hour Flexibility Trial in Internal Medicine. N Engl J Med. 2019;380(10):915–923. 30855741 10.1056/NEJMoa1810641 6457111 10.5256/f1000research.23259.r88690 Reviewer response for version 1 Ropponen Annina 1 Referee Karolinska Institutet, Helsinki, Finland

Competing interests: No competing interests were disclosed.

 5 7 2021 Copyright: © 2021 Ropponen A 2021 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The authors of this commentary have chosen very dramatic words, which I would prefer being more moderate/humble. Studying working hours at hospitals is always tricky and especially if the data is gathered via self-reports. Only until recently, few studies exist utilizing objective, pay-roll based working hours. Furthermore, besides very long working hours among physicians or residents, also combinations of work shifts may cause difficulties for recovery and sleep. Hence the studies should not only focus on the length of working hours, but also on shift intervals between work shifts.

What comes to the informal consents, the procedures may vary. If only register data is utilized, that could be done even without a consent. Hence I would like to suggest to put more emphasis on discussing the design and analyses, than on the permissions for the study.

Are arguments sufficiently supported by evidence from the published literature or by new data and results?

Yes

Is the conclusion balanced and justified on the basis of the presented arguments?

Partly

Is the rationale for commenting on the previous publication clearly described?

Partly

Are any opinions stated well-argued, clear and cogent?

Partly

Reviewer Expertise:

Occupational epidemiology.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.