Keywords
Telemedicine, Quality Improvement, Surgical Antibiotic Prophylaxis, Surgical Site Infection, Proof-of-Concept, Observational Study.
-
Views
-
Downloads
How to cite this article
Montes de Oca A, Budelier TP, Wolfe RC et al. Protocol for a proof-of-concept study evaluating systematic quality improvement with Realtime event support (SQUIRES) [version 1; peer review: awaiting peer review]. F1000Research 2022, 11:800 (https://doi.org/10.12688/f1000research.122916.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Study Protocol
Protocol for a proof-of-concept study evaluating systematic quality improvement with Realtime event support (SQUIRES)
[version 1; peer review: awaiting peer review]
PUBLISHED 18 Jul 2022
OPEN PEER REVIEW
REVIEWER STATUS
Abstract
Corresponding author:
Troy S. Wildes
Competing interests:
No competing interests were disclosed.
Grant information:
The author(s) declared that no grants were involved in supporting this work.
Copyright:
© 2022 Montes de Oca A et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Montes de Oca A, Budelier TP, Wolfe RC et al. Protocol for a proof-of-concept study evaluating systematic quality improvement with Realtime event support (SQUIRES) [version 1; peer review: awaiting peer review]. F1000Research 2022, 11:800 (https://doi.org/10.12688/f1000research.122916.1)
First published: 18 Jul 2022, 11:800 (https://doi.org/10.12688/f1000research.122916.1)
Latest published: 18 Jul 2022, 11:800 (https://doi.org/10.12688/f1000research.122916.1)
Introduction
In the United States, adverse events associated with surgery account for approximately half of all adverse events in hospitals. According to a review of 14 retrospective studies, 5.2% of adverse events were preventable, 10.4% had severe consequences, and 3.6% had fatal consequences.1 One way to help avert or mitigate the risks of suboptimal care, is quality improvement (QI). QI in healthcare results in better patient outcomes, healthcare system performance, and professional development.2
There are multiple targets for QI in perioperative care. One example is surgical site infections (SSI), defined by the Centers for Disease Control and Prevention as a wound infection that occurs within 30 or 90 days of an operation depending on the operative procedure category.3 SSIs are among the most common preventable complications of surgical procedures, occurring in approximately 2-5% of patients undergoing inpatient surgery.4 They are associated with a mortality rate of 3% and are the most expensive healthcare-associated infection, accounting for an additional $3.3 billion in cost annually.5
Several risk factors for SSI have been identified. Reliable administration of indicated surgical antibiotic prophylaxis is of particular interest for anesthesiologists, as it is a modifiable intraoperative factor. A multicenter study involving 29 hospitals across the United States found an increased risk of SSIs when patients received prophylactic antibiotics outside of the recommended timing.6 Current guidelines propose that antibiotic prophylaxis should be administered within the one to two hours prior to incision, with specific timing dictated by the drug used and dosing dependent on additional patient parameters such as patient weight.7 Re-dosing during longer surgeries to maintain adequate tissue concentrations is also fundamental to achieving appropriate surgical antibiotic prophylaxis.4,8 Re-dosing intervals for antibiotic prophylaxis are half-life dependent, relating to each drug and to patient-specific elimination considerations such as kidney function.
The goal of this study is to improve perioperative quality metric performance in patients receiving anesthesia care provided by Washington University Department of Anesthesiology clinicians. In this protocol, we describe (1) the use of a telemedicine-augmented quality improvement program to enhance on-time dosage of surgical prophylactic antibiotics, while (2) monitoring SSI incidence in the target population. If this QI framework is found to be feasible and successful, we plan subsequently to study its application to additional perioperative quality targets, including many pertinent to surgical site infection incidence such as perioperative glucose monitoring and management, and perioperative temperature management.7
Aim 1 – Improvement in administration of surgical antibiotic prophylaxis
We hypothesize the telemedicine-augmented QI intervention will:
1a. Increase the rate of on-time pre-incision surgical antibiotic prophylaxis administration during the intervention period.
1b. Increase the rate of on-time pre-incision surgical antibiotic prophylaxis administration during the six weeks following completion of the quality improvement intervention.
1c. Reduce delays in re-dosing of antibiotic prophylaxis throughout the duration of the trial.
Methods
Study design, setting, and participants
This protocol has been prepared in accordance with SPIRIT guidelines.9,10 We will conduct a multi-center prospective before-and-after proof-of-concept study to evaluate a telemedicine-augmented quality improvement program. 65 operating rooms (OR) across seven OR facilities at hospitals in the BJC HealthCare system in St. Louis, Missouri, USA will take part in this trial. These hospitals and surgical facilities include Barnes-Jewish Hospital (BJH; Center For Advanced Medicine; Center For Advanced Medicine – South County), St. Louis Children’s Hospital, St. Louis Children’s Specialty Care Center, and Washington University and Barnes-Jewish Orthopedic Center in Chesterfield.
As a part of this project, Barnes-Jewish West County Hospital (BJWCH) and Progress West Hospital (PWH) are conducting a non-research QI project implementing the same procedures.
Patients undergoing procedures with anesthetic care in operating rooms at these sites will be included. Patients undergoing surgery in BJH operating rooms within the ongoing TECTONICS intraoperative Anesthesiology Control Tower telemedicine trial will be excluded from this study.11 Approximately 40,000 patients over an eight-month period will be enrolled for this proof-of-concept study. We anticipate roughly 40% will be enrolled from BJH, 30% will be enrolled from BJWCH, and 30% will be enrolled from the remaining hospitals. Information on the conduct of the quality improvement initiatives is provided below.
Quality improvement interventions
The SQUIRES program is designed as a telemedicine-enhanced QI infrastructure that can be leveraged to address various perioperative quality targets sequentially over time. The current study has been designed to study the impact of the SQUIRES program on an initial quality improvement target, the timely administration of surgical antibiotic prophylaxis (Aim 1). The eight-month quality improvement study will include a baseline observational phase, an educational intervention phase, a phase using real-time event detection paired with guidance from a remote telemedicine center, and a subsequent observational phase (Figure 1).
Figure 1. Phases of SQUIRES quality improvement program.
Aim 1 – Improvement in administration of surgical antibiotic prophylaxis
We hypothesize that the use of telemedicine-augmented quality improvement interventions will:
1a. Increase the rate of on-time pre-incision surgical antibiotic prophylaxis administration during the intervention period.
1b. Increase the rate of on-time pre-incision surgical antibiotic prophylaxis administration during the six weeks following completion of the quality improvement intervention.
1c. Reduce delays in re-dosing of antibiotic prophylaxis throughout the duration of the trial.
Baseline
The quality improvement program will begin with a six-week baseline phase. During this time, baseline information on preoperative patient characteristics, comorbidities, surgical and clinical history, perioperative information, and antibiotic administration will be collected. This data will be collected on every patient throughout each trial phase once they have entered a participating operating room. A version of AlertWatch® (AlertWatch, Ann Arbor, Michigan) decision-support software, customized for population-level quality improvement monitoring will be used by the telemedicine center. Specialized alerts customized for the quality improvement target, in this case timely prophylactic antibiotic administration, will be triggered based on OR events. Alerts generated by AlertWatch® will be autonomously collected during this baseline observational phase. Specific information on the use of telemedicine is described below.
Education
Following the baseline data collection phase, the quality improvement program will enter a six-week education phase. Anesthesiology clinicians who provide care at OR facilities designated for this study will be educated on the SQUIRES program mechanisms and the target clinical issue. This will include baseline education on relevance of the clinical problem, best practices, answers to common questions, real-time monitoring for relevant clinical events and the availability of guidance offered by the telemedicine center.
Educational materials with the most updated guidelines and recommendations about surgical antibiotic prophylaxis will be shared with anesthesiology clinicians through conferences, emails, postings, and institutional websites. This will include an educational video, a summary of the importance and relevance of antibiotic prophylaxis (Figure 2), the key components of successful on-time antibiotic prophylaxis (i.e., agent selection, dose, and timing requirements), and a discussion of common points of confusion (e.g., managing patients on prior antibiotic therapy, renal dosing adjustments).
Figure 2. Infographic for on-time delivery of prophylactic surgical antibiotics.
In addition to providing educational intervention, alerts generated by AlertWatch® will continue to be autonomously collected during this phase.
Telemedicine intervention
Following the education phase, the telemedicine center will subsequently monitor operating rooms included in this trial for protocol adherence for three-months. The telemedicine center is staffed by attending anesthesiologists, resident anesthesiologists, certified registered nurse anesthetists (CRNAs), and student registered nurse anesthetists (SRNAs) and is currently providing evidence-based support to clinicians in a subset of operating rooms and post-anesthesia care unit (PACU) beds at BJH.11–15
A station within the telemedicine center will be designated for this study. The telemedicine center receives patient information from multiple sources, including the electronic health record (EHR) and AlertWatch® decision-support software. A version of this decision-support software, customized for population monitoring of quality improvement initiatives, will be used by clinicians in the remote telemedicine center to identify potential metric adherence failures and provide clinical support (Figure 3). Monitoring of operating rooms by telemedicine center clinicians for this trial will occur during normal operational hours of the telemedicine center.
Figure 3. AlertWatch® decision-support software customized for clinical quality improvement.
Clinicians stationed in the remote telemedicine center will monitor operating rooms in real time using the Alertwatch® software, which registers alerts when surgical antibiotic prophylaxis failures occur. The telemedicine clinicians will contact OR clinicians accordingly to verify antibiotic dosing and documentation. Some antibiotic failure alerts may facilitate immediate guidance with resultant correction of the issue. Other alerts will be used as an opportunity to provide focused supplementary education, to collect information that may guide logistical improvements, and to fine-tune future systemic education. The telemedicine center will document the results of interactions with operating room clinicians directly into customized fields in the Alertwatch® software. Specific alerts relating to surgical antibiotic prophylaxis that have been designed and will be monitored by the telemedicine center are listed in Table 1.
Table 1. Antibiotic prophylaxis alerts registered in the AlertWatch® decision-support software.
In addition to monitoring operating rooms for alerts related to the antibiotic prophylaxis quality improvement initiative, the telemedicine center will monitor the operating room population for high-risk patient safety events (e.g., critically low doses of anesthesia during surgery) using alert condition rules designated for that purpose. High-risk patient safety alerts generated by the Alertwatch® decision-support software are outlined in Table 2. Should there be significant concern for a patient safety event based on one of these alerts, the telemedicine center will investigate and contact anesthesiology clinicians in the operating room through secured messaging platforms or phone calls to offer clinical support.
Table 2. Example of acute alerts generated by AlertWatch® decision-support software.
Clinician feedback for quality improvement
As stated above, the anesthesiology team in the telemedicine center will monitor ORs in real-time and communicate with teams within ORs when relevant. Should a surgical antibiotic prophylaxis fail event occur, clinicians will be contacted via secured messaging platforms or phone. At a time convenient to the clinician, information will be collected on their opinions as to the factors that promoted the failure. When possible, factors that can be corrected to resolve the failure (e.g., inaccurate charting) will be corrected in real time. Additionally, the telemedicine team will be available to answer any questions about antibiotic prophylaxis. To enhance system improvements, patterns of failures involving the surgical team will also be forwarded to surgical leadership for the purpose of improving patient care.
On a regular basis, clinicians providing anesthesia will be given feedback reports via institutional email. These reports will include information on successful surgical antibiotic prophylaxis administration rates and provide additional educational reinforcement. Periodic adjustments will be made to educational materials in response to both failures and their etiologic determinations.
Silent monitoring
At the conclusion of the monitoring and intervention phase, surgical antibiotic prophylaxis administration alerts generated by Alertwatch® (Table 1) will be silenced for the duration of the study. These alerts will continue to be logged silently for analysis of continued adherence of surgical antibiotic prophylaxis administration at the end of the study (Aim 1b). Monitoring of the operating rooms for potential patient safety events as outlined above will continue throughout the duration of the trial.
Aim 2 – Monitoring of surgical site infections
We hypothesize the telemedicine-augmented quality improvement intervention will:
2a. Decrease the rate of surgical site infections throughout the duration of the trial compared to baseline.
Throughout the trial, data on surgical site infection rates will be collected monthly across and within multiple surgical services. This data will be compared to baseline measures, and trends over time as they relate to surgical antibiotic prophylaxis will be described.
Descriptive statistics of surgical site infection rates (often described as a ratio of observed SSIs to expected SSIs) will be reported. Additional information on specific statistical analyses methods is described below.
Study size
We estimate 40,000 patients will have procedures with anesthesia care in one of the included operating rooms during this study. Based on previous research conducted at our institution, we estimate a baseline policy adherence rate between 90 and 95%.16 Using G*Power (version 3.1.9.7),17 we conducted a series of sensitivity power analyses with conservative assumptions to identify the minimum detectible effect range for this sample size.
At any time point during the trial, this sample size is powerful enough to detect, at an alpha level of 0.05, a minimum effect size delta of 0.5% to 1.0% given a baseline rate of 90% to 95% policy adherence (Figure 4 and Figure 5). For on-time re-dosing of antibiotic prophylaxis (Aim 1c), this sample size is powerful enough to detect, at an alpha level of 0.05, a minimum effect size delta varying between 0.8% and 1.3% given a baseline rate of 70% to 75% policy adherence from previous research conducted at our institution.16
Figure 4. Minimum detectible effect size(s) for sample with 0.90 baseline policy adherence.
Figure 5. Minimum detectible effect size(s) for sample with 0.95 baseline policy adherence.
Outcomes
Primary outcome
The primary outcome will be administration of on-time surgical antibiotic prophylaxis throughout the trial.
Secondary outcomes
Other outcomes will include on-time re-dosing of antibiotic prophylaxis and incidence of surgical site infections.
Statistical analyses
Baseline and perioperative variables will be reported using descriptive statistics, including proportions, median [IQR], and mean (SD), as appropriate. Differences in these variables pre/post intervention will be compared with paired tests: McNemar, Cochran’s Q, Wilcoxon signed-rank, and paired t tests, as appropriate. Trend analysis will also be conducted to evaluate relationships over the duration of the trial.
Additionally, we will conduct analyses using segmented regression to control for confounders such as procedural type, patient associated factors, and time. Differences in key performance indicators before and after the interventions will be evaluated. This approach will allow us to compare preintervention and postintervention changes over time, divergences in the outcome when an intervention begins, and trends observed with the intervention compared to trends projected without it.18
As a secondary outcome, we will evaluate on-time re-dosing of antibiotic prophylaxis and report using methods described above.
Data collection and management
Data collected from Alertwatch® will be automatically logged to a secure database. Data on patient outcomes, perioperative care metrics, preoperative patient characteristics, comorbidities, surgical and clinical history, anesthetic information, and immediate post-operative information will also be captured from EPIC Systems software (Verona, WI, USA). Data sharing and dissemination of outcomes are described below.
Data will be collected from both research and non-research QI sites and will be logged to a secure database. Permission to collect data from non-research QI sites (BJWCH and PWH) has been approved by an appropriate designee at each institution. Data from all sites will be used to address each aim using the statistical methods described above.
Safety monitoring
This intervention is low-risk and is a quality improvement project intended to improve adherence to current clinical protocols. We do not anticipate the occurrence of significant adverse events during this study. However, the primary investigator and the study team will review any adverse events identified by the departmental quality improvement program as potentially attributable to this study. The occurrence of any significant adverse events will be reported to the Human Research Protection Office (HRPO), and the study team and HRPO will decide together whether to halt the quality improvement study. No formal data-monitoring committee will be used. There will be no audit of trial conduct during the investigation. No interim data analysis is planned unless unanticipated safety issues are identified. There are no provisions for post-trial care or compensation to patients enrolled as part of this trial, as the intervention does not change existing care models.
Dissemination and implementation
Data from this study will not be shared with others outside the research team, Washington University stakeholders, and BJC HealthCare system stakeholders. Dissemination of the findings of this study will occur via educational materials, presentations at academic conferences, and journal publications.
Ethical considerations
This study has been approved and granted a waiver of informed consent for enrollment by the Human Research Protection Office at Washington University in St. Louis (HRPO# 202107031) and is registered at clinicaltrials.gov (NCT04983329, 30th July 2021). This study satisfies the criteria for a waiver of informed consent (the research could not practically be conducted without a waiver, the rights and welfare of patients are not adversely affected by their involvement in the study, there is minimal risk with the intervention, and there is no deception requiring additional disclosure) and will be conducted accordingly.
Strengths and limitations
This study has multiple strengths. A novel pragmatic approach will be used to address quality improvement initiatives in real-time utilizing remote telemedicine intervention. Adaptations to delivery of clinician education material will be made based on the etiology of fails during the trial. This study includes multiple hospitals and surgical facilities throughout the St. Louis, Missouri region. A population of both pediatric and adult patients undergoing surgical procedures of varying complexity will be studied.
This study also has important limitations. First, hospitals in this multicenter study are in the same geographical area and in the same health system, creating the need for external validity in differing geographic locales. Second, clinicians providing care to the patients in the studied operating rooms will be subject to bias, including parallel improvement efforts being conducted throughout hospital settings. Third, positive results of utilizing telemedicine for quality improvement initiatives may not be definitive. Confounders to quality improvement may exist that require further evaluation in future studies.
Conclusions
Quality improvement in healthcare is an important tool to help avert or mitigate the risks of suboptimal care. In this protocol, we describe a novel approach to delivering quality improvement initiatives. Beginning with on-time surgical antibiotic prophylaxis, we lay the framework for scalable, modular, quality improvement initiatives augmented by real-time population surveillance with telemedicine. Potential findings from this trial may include improved reliability in administration of surgical antibiotic prophylaxis and decreased rates of surgical site infections. As a scalable solution, the impact of this study may be vast. Following a successful implementation of this novel approach to quality improvement initiatives, we plan to study the impact of additional perioperative quality improvement initiatives using this model such as glucose management, temperature management, postoperative nausea and vomiting, as well as other contributors to surgical site infections.
Reporting guidelines
Figshare: SPIRIT Checklist for ‘Protocol for a proof-of-concept study evaluating Systematic QUality Improvement with Realtime Event Support (SQUIRES)’. https://doi.org/10.6084/m9.figshare.20054972.v1.10
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Montes de Oca A, Budelier TP, Wolfe RC et al. Protocol for a proof-of-concept study evaluating systematic quality improvement with Realtime event support (SQUIRES) [version 1; peer review: awaiting peer review]. F1000Research 2022, 11:800 (https://doi.org/10.12688/f1000research.122916.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Comments on this article Comments (0)