Protocol for a proof-of-concept observational study evaluating the potential utility and acceptability of a telemedicine solution for the post-anesthesia care unit

Aim 1 – Demonstrate the potential utility of a telemedicine center for the PACU, to assist with PACU functions

We hypothesize that clinicians in the telemedicine center for the PACU will:

Aim 2 – Identify barriers to and facilitators for the implementation of a telemedicine center for the PACU, as perceived by key stakeholders

We will assess attitudes of key stakeholders towards a telemedicine center for PACU. The key stakeholders will include PACU nurses, anesthesiologists, surgeons, hospital administrators, and PACU-telemedicine center clinicians.

Ethical statement

This proof-of-concept study has been approved and granted a waiver of informed consent for all patients and a waiver of written consent for participants enrolled by the Human Research Protection Office at Washington University in St. Louis (HRPO#201901180) and is registered at clinicaltrials.gov (NCT04020887, 16^th July 2019). It is infeasible to conduct this proof-of-concept study without a waiver of consent. Additionally, this study has been determined to involve no more than minimal risk to participants, as study participation would not deviate from or delay current standards of peri-anesthesia care.

Study setting, design, and participants

The study will be conducted at Barnes-Jewish Hospital (BJH) in St. Louis, Missouri, a large tertiary care academic medical center.

We will conduct a single center prospective before-and-after proof-of-concept study to evaluate a telemedicine center for the PACU. Adults (18 years and older) undergoing elective surgery at Barnes Jewish Hospital in St. Louis, Missouri will be enrolled. Approximately 500 patients will be enrolled in this study over a six-month duration, with an estimated 250 patients allocated to each phase of the trial. The first phase is an Observation phase and the next phase is an Interaction phase. More information on these phases is provided below.

Both the “Good ReseArch for Comparative Effectiveness” (GRACE) checklist⁷ and PICOTS framework⁸ (Table 1) were used in designing this study. The conduct and reporting of this observational study will follow the “Reporting of studies Conducted using Observational Routinely-collected health Data” (RECORD)⁹ statement and the “Strengthening the Reporting of Observational Studies in Epidemiology” (STROBE)¹⁰ statement guidelines for reporting observational studies.

Primary intervention: telemedicine center for PACU

Two bays in Barnes Jewish Hospital (BJH) in St. Louis, Missouri, will be equipped for telemedicine interaction (Figure 1). Video cameras and monitors have been installed in each of these bays to allow for remote monitoring, as well as two-way video communication during the interaction phase. The telemedicine center is staffed by attending anesthesiologists along with certified registered nurse anesthetists (CRNAs), anesthesiology residents, and student registered nurse anesthetists (SRNAs), and is currently providing evidence-based support to clinicians in the operating rooms^11–14.

Figure 1. Image of post-anesthesia care unit bay in Barnes-Jewish Hospital with two-way video communication.

A station in our telemedicine center will be designated for monitoring patients assigned to the two PACU bays during this proof-of-concept study. Patient information flows to the telemedicine center through the electronic health record (EHR), physiological waveform tracings, and direct video observation. A version of AlertWatch® (AlertWatch, Ann Arbor, Michigan) decision-support software, customized for the PACU environment (Figure 2), will assist clinicians in the telemedicine center in performing core PACU-related functions remotely (see Aim 1).

Figure 2. AlertWatch® decision-support software, customized for the post-anesthesia care unit (PACU) environment.

Aim 1 – Demonstrate the potential utility of a telemedicine center for the PACU, to assist with PACU functions

The assessments in relation to PACU functions will include:

Observation phase (three months). In the first three months (the Observation phase) of this proof-of-concept study, a telemedicine center for the PACU will monitor patients assigned to two PACU bays. Both the telemedicine center and nurses caring for patients in the PACU bays will separately document physiological derangements (Table 2), treatable symptoms (Table 3), or a situation requiring urgent medical intervention (telemedicine center only; Table 4) during the PACU stay. Clinicians in the telemedicine center will assess when the patient meets discharge criteria, based on the modified Aldrete scale¹⁵ and their clinical judgment. They will document the time that discharge criteria are met, the modified Aldrete scale score at this time, and any additional relevant information. If clinicians in the telemedicine center judge that they are unable to determine a patient’s readiness for discharge, they will document their reasons (Table 5). Clinical judgment will be used in determining appropriate discharge parameters for patients with pre-existing conditions. The telemedicine center clinicians will document each patient’s information outlined in Table 2–Table 5 directly into REDCap^TM (a secure web application for managing online surveys and databases) and AlertWatch. After a patient has been discharged from the PACU, the PACU nurse will fill out a form providing information outlined in Table 2–Table 5. This includes information on physiological derangements, treatable symptoms, and discharge information. This form will be collected by the research team, and the information in the form will be documented in REDCap. During this phase of the study, clinicians in the telemedicine center will not communicate with clinicians in the PACU (nurses or physicians), unless there is a patient safety event.

Interaction phase (three months). In the three months following the observation phase, clinicians in the telemedicine center will interact with patients and clinicians associated with the designated PACU bays using audio-visual technology. PACU clinicians and clinicians in the telemedicine center will become a “fused” team, and the telemedicine center will continue to document information on physiological derangements (Table 2), treatable symptoms (Table 3), situations requiring urgent medical intervention (Table 4), and discharge readiness (Table 5).

The telemedicine center clinicians will assess patients’ discharge readiness throughout their PACU stay. A modified Aldrete scale along with clinical judgment will guide the telemedicine center clinicians in determining readiness for discharge (Table 5). After discharge readiness has been determined by the telemedicine center, the attending anesthesiologist in the telemedicine center will document discharge readiness in AlertWatch and REDCap, and contact the relevant anesthesiologist. The telemedicine center for PACU will document when this information was communicated. At any point clinicians in the telemedicine center might decide to contact PACU clinicians (nurse or physician) if they have specific concerns regarding patients. If the telemedicine center clinicians feel that they cannot adequately assess a patient’s clinical status, they will notify the PACU clinicians. This will be documented together with a relevant explanation (Table 5).

Final determination and sign-off regarding discharge suitability will be made by the anesthesiologist in the PACU. With this proof-of-concept research project, there will be no change in relation to which clinicians have responsibility for decision making and clinical care. The telemedicine center clinicians will not write any orders in the medical record, and will provide opinions only to physicians and nurses who are responsible for patient care in the PACU. The responsibility to call for help when patients are deteriorating will remain with the PACU nurses, as is the current standard in that environment. The notion is that the telemedicine center will not lead to any decrement in the care that PACU patients are currently receiving from nurses and physicians in that environment.

The successful integration of the telemedicine center into each of the core PACU functions will be measured in the following ways:

    Physiological derangements – Success will be measured (in the observation phase) by the ability of the telemedicine center clinicians to identify physiological derangements as they are occurring in the PACU. The extent to which the telemedicine center clinicians can identify these physiological derangements will be measured by comparing PACU nurse and telemedicine center assessment surveys¹⁶ for each patient (Figure 3).

Figure 3. Overview of data collection methods and outcome measures during the interaction phase of a before-and-after proof-of-concept study for a telemedicine center for the post-anesthesia care unit (PACU).

    Symptom identification and management – Success will be measured (in the observation phase) by the ability of the telemedicine center clinicians to identify treatable symptoms as they arise in the PACU. The extent to which the telemedicine center clinicians can identify these treatable symptoms will be measured by comparing PACU nurse and telemedicine center assessment surveys¹⁶ for each patient (Figure 3).

    Emergency situations – Success will be measured (in the observation phase) by the ability of the telemedicine center clinicians to identify situations requiring emergency medical intervention as they are occurring in the PACU. By construction, any time the telemedicine center feels that an emergency situation is present, preserving patient safety mandates contacting the bedside clinician. During each such contact, the telemedicine center clinician will ask if the PACU nurse was already aware of the situation, disagreed with the assessment, and had already spoken to the supervising physician regarding it. The occurrence of emergency medical situations will be extracted from the electronic health record, and the agreement between telemedicine center and PACU nurse assessments will be quantitated.

    PACU discharge – Success will be measured by the ability of the telemedicine center clinicians to identify when patients are ready for discharge (observation phase [without communication] and interaction phase [active communication with patient and PACU clinicians]) (Figure 3). The impact of the telemedicine center on this key function will be examined based on feedback from key stakeholder focus groups (interaction phase; see Aim 2). The difference between sign-out times in the observation and the interaction phases will be compared.

Hand-off activity. The telemedicine center clinicians will participate in hand-off activities to and from the PACU. This includes ensuring appropriate transfer of information from operating rooms to the PACU. The telemedicine center clinicians will remotely join the hand-off conversations, and review patients’ medical history and intraoperative course to identify potential missed transfer of information.

During the observation phase, the telemedicine center clinicians will observe the hand-off workflow, gain familiarity with the current hand-off routine, and identify possible areas of missed information transfer where the telemedicine center clinicians may have adjunct utility. An example of potential adjunct utility would be communicating the importance of appropriate insulin and glucose management in the PACU for a patient with type I diabetes.

In the interaction phase of the study, the telemedicine center clinicians will try to fill gaps in information transfer during the hand-off procedure. In addition to remotely joining the hand-off conversation, the telemedicine center clinicians will share pertinent additional patient or procedural information, especially if this could inform the patient’s PACU medical treatment. After the completion of the hand-off procedure, the PACU nurse who interacted with the telemedicine center clinicians will complete a short survey¹⁶ to assess the telemedicine center’s involvement in that patient’s transfer of care.

The successful integration of the telemedicine center clinicians’ hand-off activity will be measured in the following way:

    Hand-off activity – Success will be measured (in the interaction phase) by the ability of the telemedicine center clinicians to join and contribute meaningfully to the hand-off discussion. The impact of the telemedicine center clinicians on this key function will be determined from feedback from key stakeholder focus groups (see Aim 2) and PACU nurse surveys¹⁶. These stakeholders will comment on utility of the telemedicine center’s involvement and provide suggestions for improvement. A binary assessment of hand-off adequacy will be provided by the PACU nurse hand-off survey¹⁶. The telemedicine center clinician will use a hand-off content checklist¹⁶ to record the number of mandatory items not discussed and number of recommended non-mandatory items discussed. For each of the observation and intervention phases, for 50 randomly selected cases a trained observer (not the participant in hand-off) will use the hand-off communication assessment tool¹⁶ of Weinger and others¹⁷ substituting the telemedicine center hand-off content checklist. A run-in phase of one month during the intervention will elapse before any of the 50 detailed communication evaluations are performed.

Aim 2 – Identify barriers to and facilitators for the implementation of a telemedicine center for the PACU, as perceived by key stakeholders

We will assess the attitudes of key stakeholders in order to identify barriers to and facilitators for implementation of a telemedicine center for the PACU. (Figure 3)

Stakeholder focus groups. We will conduct focus groups with stakeholders to gain insights regarding their perceptions of barriers and facilitators related to the above-noted PACU functions before and after the implementation and use of a telemedicine center for the PACU. We will also gather perspectives from the stakeholders on the role and impact of the telemedicine center on their individual and team workflows in the PACU and between units during care transitions. Focus group participants will include nurses, anesthesiologists, surgeons, hospital administrators, and PACU telemedicine center clinicians. Our focus groups will be homogeneous in order to understand the clinician workflow based on their professional role, and their use of the telemedicine center in supporting their role and responsibilities. Each focus group will comprise five to six participants. This will allow in-depth discussions of the workflow problems and unintended consequences caused by the implementation and use of the telemedicine center for the PACU. The focus group sessions will be guided by a semi-structured interview guide focused on the following themes: (1) PACU core functions, (2) PACU patient workflow, (3) PACU clinician activities and tasks, (4) tools and technologies used to support the PACU workflow, (5) major barriers to PACU functions, (6) use of a telemedicine intervention as a potential mechanism to support effective and efficient functioning of the PACU. We plan to conduct 6-8 focus group sessions (four pre-intervention during observation phase, and four post-intervention during interaction phase) or until data saturation is attained.

Study size

Patients are allocated to PACU bays according to the discretion of the nurse in charge of the PACU. Currently, approximately two patients per day are cared for in each bay in the participating PACU. Therefore, the telemedicine team will monitor approximately four patients per day over the course of the proof-of-concept study. We estimate that 500 patients will be included in this proof-of-concept study (250 per monitored phase) (Figure 4).

Figure 4. Allocation of sample size and post-anesthesia care unit (PACU) layout for proof-of-concept study.

Statistical methods

Primary outcome. This is a proof-of-concept study and will only address surrogate outcomes. The primary outcome (time to PACU discharge readiness) will use two comparison groups. First, historical controls will be drawn from the observation phase. A propensity score for inclusion into the study will be generated as a function of (minimally) surgery performed, day of week, time of day, age, and sex. 3:1 matched control patients will be included. The outcome will be analyzed with interrupted time series methods with flexible functions of calendar time used to adjust for secular trends; the study hypothesis is a non-zero discontinuity at telemedicine implementation. That is, if Y_i is the outcome for the ith patient at time t_i with covariate vector X_i, while the implementation time is t₀, and I() is the indicator function,

where f₁ and f₂ are smooth functions. Other patient factors known to strongly influence PACU length of stay (age, ASA physical status, number of co-morbidities, morbid obesity, obstructive sleep apnea, surgical specialty, primary anesthesia type, history of postoperative nausea and vomiting, preoperative pain, and scheduled case duration) will be included as covariates. The minimization criteria will be least squares or trimmed least squares or other robust criteria if there are substantial outliers. Outcomes will be examined for residual auto-correlation, and if non-negligible, auto-correlation robust standard errors (such as Newey-West errors) and an ARIMA model will be reported. Confidence intervals will be generated by non-parametric bootstrap sampling where possible. No adjustment will be made for matching, but bootstrap methods will respect the matched “units.” P-values will be generated both by likelihood ratio tests and by using non-deployment times as a null distribution; that is, we will run the same analysis looking for discontinuity at times remote from the true implementation time. We will conduct sensitivity analyses with transformations of the outcome variable. We will use an excluded run-in period of one month as a sensitivity analysis. Because hospital length of stay is unlikely to be meaningfully affected by a telemedicine center for the PACU, but does track overall acuity and surgical severity, we will use hospital length of stay as a control time series.

Contemporaneous control patients will also be gathered. A propensity score for study inclusion will be generated as a function of (minimally) surgery performed, calendar time, time of day, age, and sex. 3:1 matched control patients will be included. Differences will be analyzed by t-tests using permutation calibration. Confidence intervals on the difference in mean time to discharge readiness will be generated by nonparametric bootstrap. We will include a sensitivity analysis where the interrupted time series method includes historical and contemporaneous control patients with the treatment indicator T for study patients,

Based on data from our EHR, patients are currently in PACU for a mean of 150 min (standard deviation = 65 min) before they are determined to be suitable for discharge. Based on these values, with 250 patients in each phase (observation and interaction), this observational before and after study will have >70% power with an alpha <0.005 and > 90% power with an alpha <0.05 to detect a mean decrease in 20 min (from 150 min to 130 min) to PACU discharge readiness time. Statistical testing will be with appropriate statistical software. Using non-parametric bootstrap of historical data and a 3:1 control sampling ratio, the average standard error on the difference in means under the null hypothesis was 5.5 minutes, giving an anticipated 95% confidence interval width of 22 minutes. A somewhat larger standard error will be encountered when adjusting for covariates or secular trends; however, this suggests that we will be able to resolve differences in PACU readiness times of 20–25 minutes. This difference of approximately a third a standard deviation is usually regarded as a “small-moderate” sized effect.

Secondary outcomes. Hand-off quality assessment from the PACU nurse binary survey response will be analyzed using a logistic regression model adjusting for surgical service, age, and sex. Because observation resources are required for hand-off evaluations, no matching will be performed, and no contemporaneous controls will be gathered. Adjusted differences in rates of inadequate hand-off will be summarized with 95% confidence intervals and model-based p-values. Observed reported hand-off communication quality will be presented as a purely descriptive result.

The accuracy of physiologic, symptom, and status assessments is less straightforward to analyze. At the heart of the proposal is the belief that telemedicine assistance will detect some abnormalities not caught (or caught later) by the bedside team and detect that the patient has adequate status for PACU discharge before the bedside nurse. Using the bedside assessment as a gold standard is therefore limited. Similarly, although we believe that abnormalities detected by either bedside or telemedicine are unlikely to be false positives, we have no way of assuring that. We also cannot reliably determine the timing of the bedside nurse’s detection of an abnormality, as they may document it much later if they believe it does not require an immediate intervention.

Each status assessment event can occur multiple times for each patient; however, we are unlikely to accurately capture the bedside nurse’s impression of the number of times an event occurred. We will therefore binarize the presence of each assessment type and display confusion matrices (count tabulations) for each assessment type, which we will summarize with Jaccard indicies. The “null hypothesis” that these measures do not agree at all is not meaningful or the subject of this study. As described above, neither is a directional superiority hypothesis possible to evaluate. Final Aldrete scores will be assessed with pearson correlation, and a t-test of the difference in scores presented. Differences in ready-for-discharge times will be summarized as mean and standard deviation, with the null hypothesis of zero mean tested by t-test with a robust standard error.

Agreement of emergency medical status is unlikely to have enough events to be statistically compared. We will present cross-tabulations of (emergency detected by telemedicine center: yes/no) and (PACU nurse: disagree, investigate and agree, already aware, physician contacted). The absolute rate of telemedicine center false positives (team disagrees), true positives (team unaware), true positives (team aware), and false negatives (team aware > 15 minutes prior or t never detects) will be presented with 95% confidence intervals.

Data collection

Multiple sources will be utilized for data collection from which outcome measures will be extracted. Data from AlertWatch will be automatically logged to a secure database.

Preoperative patient characteristics, comorbidities, surgical and clinical history, as well as perianesthesia information will be captured using Epic Systems software (Verona, WI, USA). Prospective data will be collected from Epic Systems for the datapoints mentioned throughout the proof-of-concept study.

Relevant PACU information outlined in Table 2–Table 5 for patients in this study will be collected and entered into a REDCap database managed by Washington University. Data will not be shared with others outside the research team.

Methodological strengths and limitations

A strength of this study is its pragmatic approach as a real-world study with measurable aims. Feasibility will be determined, and information will be provided regarding logistical implications of establishing a telemedicine solution for the PACU. Many telemedicine solutions have been implemented without considering barriers and facilitators, such as cultural and political obstacles. This study proactively addresses these concerns, which might facilitate future successful implementation and generalization of similar telemedicine initiatives. Specific functions of the PACU have been detailed, and the methods of this study will allow assessment of the ability of the telemedicine center to facilitate the accomplishment of these functions.

This study also has important limitations. First, as a proof-of-concept, it will only include two PACU bays. Thus, its applicability to a large PACU will not be resolved. Second, PACU clinicians will be aware of the initiative, which could modify their behavior during the conduct of the study. Third, as the study design is observational with a before and after approach, improvements (for example in time to discharge) cannot be causally attributed to the intervention; there could be confounding explanations. Fourth, the current discharge criteria for the PACU do not have a firm evidential foundation (there is no gold standard measure for discharge readiness), and clinician gestalt plays an important role. This limitation can be addressed through development of rigorous, reliable and practical criteria. Finally, as a single center study, results will not necessarily be broadly generalizable.

Adverse events and safety monitoring

We do not anticipate the occurrence of significant adverse events during this study. However, the primary investigator and the study team will review any adverse events identified by the departmental quality improvement program as potentially attributable to this proof-of-concept study. The occurrence of any significant adverse events will be reported to the HRPO, and the study team and HRPO would decide together whether to halt the trial. No formal data-monitoring committee will be used. There will be no audit of trial conduct during the investigation. No interim data analysis is planned for this proof-of-concept trial unless unanticipated safety issues are identified. There are no provisions for post-trial care or compensation to patients enrolled as part of this trial, as the intervention in this proof-of-concept trial involves only the addition of real-time decision-support tools and does not change existing care models.

Dissemination

Dissemination of the findings of this study will occur via presentations at academic conferences, journal publications, and educational materials. Data from this study will not be shared with others outside the research team, as this study is a proof-of-concept designed to evaluate the potential utility and acceptability of a telemedicine solution for the post-anesthesia care unit and will only address surrogate outcomes.

Study status

This study transitioned from the observation phase to the interaction phase in September 2020.

Underlying data

No underlying data are associated with this article.

Extended data

Figshare: Supplemental Material for Proof-of-Concept PACU Telemedicine Protocol. https://doi.org/10.6084/m9.figshare.12944489.v1¹⁶

This project contains the following extended data in the file ‘PACU_Telemedicine_Supplement.docx’:

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).