Intervention- versus physiology-based risk assessment scores for predicting cardiac arrest: a pilot study

Introduction

Cardiac arrest in hospitalized patients outside the intensive care unit (ICU) carries a high mortality. Many studies have shown that these cardiac arrests are rarely sudden, as demonstrated by abnormal vital signs in the hours leading up to these events^1–7. The challenge lies in developing robust algorithms to predict cardiac arrests in order to better target interventions. In fact, one potential explanation for the failure of rapid response systems (RRS) to better impact patient outcomes is an inability to identify which patients actually require RRS intervention⁸.

The Modified Early Warning Score (MEWS) is one attempt at a risk prediction algorithm. This scale, which assigns point values for abnormal vital signs or mental status assessments, has been used to predict requirement for hospital admission in emergency department patients⁹ and to predict hospital mortality¹⁰. However, although a vital sign-based tool is appealing for general ward patients, it is limited by the quality of the data, specifically with respect to respiratory rate and mental status, which are poorly assessed and documented outside the intensive care unit. In addition, physiology-based tools do not take into account what has been done to the patient to maintain that physiology, such as the use of supplemental oxygen to maintain oxygen saturation or vasopressor agents to maintain blood pressure.

In the ICU setting, scales such as the Sequential Organ Failure Assessment (SOFA)¹¹ have been developed to account for both physiology and supportive measures, but have not been validated for use outside the ICU. Yet another ICU scale with the potential to fill this critical role for floor patients is the 28-item Therapeutic Interventions Scoring System (TISS-28)¹². This system was designed to quantify nursing workload in the intensive care setting, and as such does not include any physiology-based scales. Rather, the scale assesses the intensity of caregiver interventions such as the frequency of vital signs, use of invasive catheters and drains, and the frequency and intensity of certain treatments.

We hypothesized that a scoring system that includes supportive measures and interventions (such as the SOFA and/or TISS-28) would be superior to a system relying only on physiology (such as the MEWS) in detecting clinical deterioration preceding cardiac arrest on the floors.

Method

We conducted a retrospective chart review of patients between 2006 and 2008 at the University of Chicago Medicine, which is a tertiary-care, university teaching hospital with approximately 600 beds, without a RRS at the time of the study. The study protocol was approved by the University of Chicago Institutional Review Board.

A convenience sample of patients suffering cardiac arrest in the hospital were included in the analysis. Paper charts were reviewed and data sufficient to calculate the SOFA, TISS-28, and MEWS for each patient was extracted. This included vital signs, laboratory results for creatinine, bilirubin, arterial blood gases, vasoactive drug doses, and nursing notes related to mental status and care duties. When arterial gas data were not available, we estimated the PaO2 necessary for the SOFA using a validated conversion algorithm¹³.

The TISS-28 is comprised of 28 items with point values from one to eight, with higher scores equating to a higher nursing workload. The items are subdivided into component physiologic systems (Basic Monitoring, Cardiovascular, Respiratory, Interventions, Renal, Neurologic and Metabolic subscales) to isolate the dynamic aspects of the scale in the time preceding arrest; each subscale consists of one to several items. For example, the Neurologic subscale consists solely of the presence or absence of intracranial pressure monitoring (worth four points), whereas the Cardiovascular subscale consists of the presence of single or multiple vasoactive medications, the presence of a central line, and the presence of an arterial line (worth three, four, two, and five points, respectively). The Basic Monitoring scale consists of items related to the care of drains and wounds, and the frequency of vital sign checks and lab draws. The Intervention subscale encompasses recent procedures and studies the patient might undergo, such as endotracheal intubation, echocardiography, and trips to radiology or the operating room. The Respiratory and Renal subscales consist of items pertaining to the support of those particular organ systems (ventilatory management, diuresis and hemodialysis, respectively). Finally, the Metabolic subscale consists of items related to treatment of acid-base abnormalities and malnutrition.

MEWS, SOFA and TISS-28 scores were calculated for the calendar day of the arrest (pre-arrest) and the day prior to the arrest (baseline). MEWS scores were based on vital signs such as heart rate, respiratory rate, temperature, systolic blood pressure and patient responsiveness. Data obtained during or after the arrest was excluded. In accordance with studies using the SOFA score^14,15, we used the highest score for each organ system to calculate the total score, and then used the highest score from each time point. For the sake of consistency, we used this approach to calculate the TISS-28 and MEWS scores as well. Data were stored in a spreadsheet (Microsoft Excel, Redmond, WA) and analyzed using SPSS (Chicago, IL). Baseline and pre-arrest scores were compared using two-sided paired sample t-tests. Percentages of change in the various scales from baseline to pre-arrest were compared using chi-squared statistics. An alpha of 0.05 was used for all analyses.

Results

A total of twenty patients were included in the study. Demographic data are presented in Table 1. The mean age of subjects was 68±15 years. Eighty five percent of patients were black and sixty percent were female. Admission diagnoses for the 20 patients were highly variable and included dyspnea, subdural hematoma, and congestive heart failure, amongst others. For three subjects, cardiac arrest took place on the day of admission, and hence no baseline data were available. In the remaining 17 patients, the SOFA score increased from 1.29±0.40 at baseline to 1.76±0.45 (p=0.03) on the calendar day before the event, representing an increase of 36%, while the TISS-28 increased from 9.9 to 15.0 (p=0.04), representing a 52% increase. There was no significant change in the MEWS (see Table 2).

Analysis of the TISS-28 subscales revealed that the primary driver of the increase in TISS-28 was due to an increase in the Interventions score, which more than tripled (0.59±0.40 at baseline vs. 2.00±0.73 the day before arrest; p=0.03). There was no significant difference in the Basic Monitoring, Cardiovascular, and Respiratory subscales, despite a trend in the same direction (see Table 3).

Discussion

In this study of patients suffering cardiac arrest, we found that the measure of nursing care (TISS-28) changed to a greater degree than the physiology-based measures (MEWS) in the period preceding the arrest. While some patients experience sudden arrest, many are believed to decompensate over a period of hours or even days prior to their ultimate cardiac or respiratory arrest⁵. Schein and colleagues identified respiratory disorders as the most common antecedent to arrest, and noted that most patients experiencing arrest had "documented observations of clinical deterioration or new complaints within eight hours of arrest"⁷. In a retrospective, case-controlled study Hodgetts et al¹⁶ observed that abnormal breathing, heart rate, or systolic blood pressure were powerful predictors of arrest. In contrast, Kause⁴ identified hypotension and altered mental status as the primary predictors of arrest in their study examining patients from the United Kingdom and New Zealand.

The detection of patient deterioration by vital signs-based systems can be delayed by supportive measures, suggestive of low sensitivity. In many decompensating patients, vital signs may be altered by a variety of supportive measures instituted by caregivers. These supportive measures will delay the detection of deterioration by physiologic based scoring systems such as the MEWS, but would be detected by a scoring system of supportive measures such as the TISS. Intervention and laboratory based scoring systems may be more sensitive or earlier predictors of patient deterioration than any scoring system limited to only one of these dimensions.

Studies of RRS have varied in their ability to demonstrate an effect on cardiac arrest and mortality^17–19. The systems in most of these studies rely on nursing or medical staff to notice the sometimes subtle changes in patients’ well-being in order to activate the RRS. The attraction of simple severity of illness scales like those studied here is that, in conjunction with an electronic medical record, they can be calculated automatically. Such a system could prompt caregivers to consider RRS consultation when scale scores (i.e., caregiver duties) increase or pass some threshold, and this might better direct RRS resources to the sickest patients.

Our study demonstrates that the intensity of caregiver intervention, particularly those interventions related to procedures and cardiovascular and respiratory systems, is superior to MEWS in detecting patient decompensation. In our patients, the liver and coagulation components of the SOFA score changed less than the other parts of the SOFA score, though it is possible that the indicators of liver and coagulation function (i.e. bilirubin and platelet count) would be more useful in a larger sample of patients in which these parameters changed significantly. Similarly, the CNS component of the TISS-28, the measurement of intracranial pressure, was not performed on any patient in our study at any time point. Using only the presence or absence of intracranial pressure monitoring as a metric for CNS deterioration would likely miss a patient’s deteriorating mental status as a sign of clinical worsening.

Our study has several limitations. First, our sample size is small. It is possible that the MEWS would prove to be a significant predictor in a larger study. However, as an exploratory study of the utility for intervention-based assessment, either in addition to or in place of physiology-based predictors, these data show promise for the TISS-28 as a measure of clinical deterioration. Second, our study is a retrospective case series. These findings will need to be validated in a larger prospective, controlled trial. Third, although the TISS-28 describes caregiver interventions across subscales and organ systems, some subscales are more comprehensive than others (e.g., as mentioned above, the Neurologic subscale consists of one item, whereas other subscales cover a broader range of potential interventions). While changes in mental status are widely believed to foreshadow decompensation, variable and generally incomplete documentation of mental status frustrates abstracting SOFA and MEWS from charts.

Assessment of the central nervous system is probably sub-par in all three clinical scales. Whereas the TISS-28 focuses exclusively on ICP monitoring, the SOFA CNS subscale consists solely of the Glasgow Coma Scale (GCS) and the MEWS CNS variable captures patients’ responsiveness to stimulation. Each of these measures probably misses the subtle mental status alterations that precede clinical decompensation. Furthermore, the various CNS measures in this study were the most likely variables to have missing data. For the GCS, when data were lacking, we inferred a value of 15 (normal), and did the same for the CNS measure of the MEWS. This approach may lower the total scale scores, but we believe this is the most conservative approach to handling these missing data.

In conclusion, our data show that an intervention-based scoring system changes more than physiology-based scoring systems in the period prior to cardiac arrest. This may be due to support used to maintain normal vital signs in the early period of decompensation, which is not captured by vital sign-based scoring systems. This finding needs to be validated prospectively.