Keywords
Global Trigger Tool, patient safety, adverse events detections, quality of care, medical errors, harm
Global Trigger Tool, patient safety, adverse events detections, quality of care, medical errors, harm
We verified and adjusted some typos and added more clarification as suggested by Professor Donev. These include adjustments to the Figure 1 title and Tables 3 and 4 legend.
See the authors' detailed response to the review by Brent C. James
See the authors' detailed response to the review by Doncho Donev
See the authors' detailed response to the review by James M. Naessens
Safety is one of the domains of quality in healthcare. Improving the safety of patients is a political priority worldwide, as studies on the safety of patients have drawn attention to the high rates of health care-related harm1–3. Improving patient safety requires effective and reliable methods to identify and monitor adverse events (AEs) so that learning can take place and improvements can be made.
Even though several methods to detect AEs are available, there is no universally recognized method that reliably provides a comprehensive overview of the extent of the problem. These methods include incident reporting, clinical records (CRs) review and automated extraction using hospital administrative data, for example Patient Safety Indicators (PSI) as developed by the Agency for Healthcare Research and Quality (AHRQ). Incident reporting is the most commonly used method to detect AEs in hospitals, but is based on voluntary reporting. Despite considerable efforts by local hospitals, reporting systems only detect a limited number of AEs4. The effectiveness of automated extraction using hospital administrative data for detecting AEs depends on the accuracy of data compilation. CRs review, as used in the Harvard Medical Practice Study, is very labor intensive, thereby limiting its use4. As a result, health services, governments and researchers have focused on developing harm detection tools.
This paper is one of the first to report the findings of the application of the Global Trigger Tool, developed by the Institute for Healthcare Improvement (IHI), across the whole health system in Sicily. In 2015, the Sicilian Health System adopted IHI GTT5 to assess the number, types and severity levels of AEs.
The measurement instrument used in our study is the Italian version of the IHI GTT6. The Italian version was adapted to be appropriate to the regional context7. The triggers are grouped into the same seven categories as the original version (care, medication, surgical, intensive care, obstetric, pediatrics and emergency care); however, changes to some triggers have been introduced. We did not consider triggers and AEs that were present on admission and we added three new triggers: change in procedure anesthesia, duration of surgery greater than 6 hours, and hospital stay greater than five days after delivery7.
89 of medicine, surgery, obstetrics and intensive care wards has participated in the study. Some of these has participated from June 2015 to June 2018, while others did not continue to the end of the observation period. Others were added during the study. Some departments participated in the study from June 2015 to June 2018, while others did not continue until the end of the observation period. Others were added during the study.
We used a random method of records selection, as recommended by the IHI GTT. In each ward, 10 inpatient CRs were randomly and monthly selected, one every ten. (Table 1) From the ordered sequence of the numbering of the CRs of the period under evaluation, a CR was selected every 10 (ie 10th, 20th, 30th, 40th, 50th, etc). In the case in which the number of patients discharged during that month was less than 100, we proceded to remove the previously selected CRs and to select another CR in the same way (one CR every 10). Eligibility criteria were an admission lasting more than 24 hours, and all the administrative data completed. In the Intensive Care Unit (ICU), all patient CRs, discharged during the reference period, were reviewed.
As per the IHI protocol, each review team was composed of three individuals: two with clinical knowledge and expertise on patient clinical documentation, and a physician whose role was to authenticate the findings and the severity rating of the AEs.
The total number of the reviewers was 199 divided into a 71 team. Where possible, the review team remained consistent over time.
We excluded triggers and/or events that took place outside the time of the patient admission to the hospital and we considered only triggers and AEs that occurred during hospitalization. The two clinical reviewers audited all the CRs on their own, independently. We used five worksheets: general care, medication, surgical, obstetric and intensive care, with some changes in accordance with the IHI GTT. The third reviewer was always a physician. The physician did not review the CRs, but had to authenticate the consensus of the two primary record reviewers. The physician authenticated the findings of the adverse events, the rating of severity and provided answers to questions of the record reviewers about findings in a specific record. We have used the IHI GTT definition of an adverse event: unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalization, or that results in death.
Triggers and AEs present at the time of hospitalization were excluded in this study.
The CRs were examined following the order of the sections described in the IHI GTT. The revison time should have been no longer than 20 minutes. The “20-minute rule” was applied to all records regardless their size5. The reviewers entered data into a specially developed dedicated IT-platform, developed by our IT team (based on Jawascript HTML and PHP)8
From June 2015 to June 2018, 18,008 CRs from 105 wards of 44 Sicilian public hospitals were examined. In this study, we analyzed just 14,706 CRs relating to patients discharged from 89 medicine, surgery, obstetrics and intensive care wards, without including the CRs of the emergency and pediatric wards. In 5,574 (37.9%) CRs at least one trigger in each patient was found. In 7 CRs, the reading of the discharge diagnosis aroused interest by reviewers and an AE was detected and the triggers were not looked for. AEs were determined in 1,542 CRs (Table 1). The identification of triggers allowed us to identify corresponding AEs (Table 2).
In this study, 37.9% (n=5,574) of all CRs examined had at least 1 positive trigger. Of those, 2,778 CRs had a single positive trigger (49.8% of all CRs with positive triggers) while 2,796 CRs had more than one positive trigger (51.2% of all CRs with positive triggers).
CRs with triggers (n=5,574) are significantly present in surgery wards (n=1,709; 35.4%), medicine wards (n=1,517; 34.3%) and ICU (n=1,672; 84.7%). while CRs with triggers in obstetrics wards are significantly less frequent (n=676; 20.3%). CRs with isolated triggers are more common in medical wards (n=1,085; 23.7%) and rarer in ICU (n=272; 13.7%) (Table 1).
We detected 1542 AEs in 975 CRs (i.e. patients) with AEs. In 652 patients (66.8%) a single AE was present. In the remaining 323 (33.2%) were more than one AEs.
This analysis allowed us to highlight how isolated triggers are not always a good indicator of AEs. A Receiver Operating Characteristic (ROC) curve analysis demonstrates that the presence of two triggers in a CR has a high probability that an AE having occurred (Figure 1). In CRs with a high frequency of triggers, a corresponding number of AEs was not always detected. As indicated in Table 3, on the contrary, some triggers were associated with a large number of AEs. For example, the trigger C11 Health care–associated Infection was detected in 504 CRs. However, often, a single CR with trigger C11, presented more than one AE.
ROC curve shows that the presence of two triggers in clinical records indicates an adverse event with a high probability.
Triggers and AEs were analyzed when isolated triggers were identified (Table 4). For example, the isolated trigger C01 (Blood products use) was present in 483 cases, but identified only two AEs, and the trigger M05 (Rising BUN or serum creatinine >2 times the baseline) did not identify any AEs. AEs were classified using the 2009 edition of the WHO International Classification for Patient Safety (ICPS)9, and a clinical classification developed by our group (Table 5).
International Classification for Patient Safety (ICPS) - WHO ed. 2009 | Clinical classification | ||
---|---|---|---|
INCIDENT TYPE | AEs, N (%) | INCIDENT TYPE | AEs, N (%) |
Healthcare Associated Infection | 742 (48.1) | Healthcare Associated Infection | 742 (48.1) |
Clinical Process/Procedure | 697 (45.2) | Surgical complications | 175 (11.3) |
Medication/IV Fluids | 89 (5.7) | Pressure ulcers | 172 (11.2) |
Patient Accidents | 12 (0.1) | Acute kidney injury | 133 (8.6) |
Blood/Blood Products | 2 (0.1) | Procedure complications* | 109 (7.1) |
TOTAL | 1542 (100) | Hypoglycemia | 62 (4.0) |
Delivery complications | 47 (3.0) | ||
In-hospital Stroke | 18 (1.2) | ||
Anesthetic complications | 17 (1.1) | ||
Hemorrhage | 5 (0.3) | ||
Various | 62 (4.0) | ||
TOTAL | 1542 (100) |
The most frequent type of AEs observed: in-hospital related infections; surgical complications; pressure ulcers; acute kidney injury; and procedure complications.
The evaluation of the quality and safety of health systems is difficult, but has become a priority of healthcare funders and organizations. Outcome, management and patient satisfaction indicators are available to measure the different dimensions of health care quality, but reliable measurements of safety have been elusive. Many methodologies and indicators, such as the PSI developed by AHRQ, the review of health documentation incident reporting and prospective clinical surveillance methodology are currently used.
The documentation and study of AEs, i.e. where they occur, and the type and degree of harm, is essential to promote specific opportunities for interventions improvement and to evaluate effectiveness of any intervention over time.
The IHI GTT is one methodology proposed to detect and monitor AEs and provide information to implement improvement. At present, compared to other methods, it may be the best methodology to use4. A systematic review reported the use of GTT methodology in 15 countries in 44 hospitals, with 79,004 clinical records examined10. The data are an underestimation, as the report did not include some comprehensive Swedish and Norwegian studies11. Recently, papers have been published in Italy, Austria, China and Russia12–16. A critical appraisal of the studies and their results is difficult, as the methodologies use are heterogeneous, protocols are often locally adapted to the local context, the populations studied are different, and the skills of the reviewers vary. We adapted the IHI GTT to the local context in Sicily for this study and did not consider triggers and AEs identified at the admission of the patient as well as modifying some triggers. In this study, the triggers were analyzed both when associated with other triggers and when isolated. In both cases the correlation with AEs was analyzed.
In this study, 37.9% (n=5,574) of all CRs examined had at least 1 positive trigger. Of those, 2,778 CRs had a single, positive trigger (49.8% of all CRs with positive triggers) while 2,796 CRs had more than one positive trigger (51.2% of all CRs with positive triggers).
The connection between the number of CRs with triggers and the number of CRs examined is not always reported in the literature and when reported it is not always clear. Xu et al.17 report that during review of 240 clinical records, 51.0% triggers (26/51) were identified 206 times. Mortaro et al.12 report that during review of 1,320 clinical records, a total of 130 triggers were detected.
In our study, few AEs were identified by isolated triggers and many isolated triggers are not associated with AEs. Not very useful for identifying AEs, some triggers may be direct measures of "near misses". Today's trigger could be tomorrow's adverse event.
In general, it would appear that little attention is paid to triggers if they are not related to an AE. Instead, many triggers of the IHI GTT protocol could be considered to be a measure of near misses and potential AEs. These include, decrease of Hb or Ht >25%, readmission within 30 days, transfer to higher level of care, clostridium difficile–positive stool, PTT >100 seconds, INR > 6, glucose < 50 mg/dl, rising BUN or serum creatinine >2 times baseline, blood loss >500 mL (after vaginal delivery) or >1000 mL (Cesarean section), and readmission to ICU.
In a systematic review, de Vriess et al.18 reported that in 8 studies that included 74,485 CRs, the median overall incidence of in-hospital AEs was 9.2%. Another systematic review reported 44 hospitals with 79,004 CRs, had an incidence between 7 and 51%10. In the Sicilian public hospitals, 1,542 AEs were detected in 975 clinical records, corresponding to an incidence of 6.6% of CRs examined, and to 17.5% of CRs with triggers.
The results of this study are not comparable with other studies due to the diversity of detection protocols. This is also demonstrated by the wide frequency variability of AEs reported in the literature (7–51%)10.
In this study the most significant difference, compared to the other studies, is the exclusion of triggers and AEs present at the time of the patient admission in the hospital.
ICUs have the highest incidence of AEs, both with respect to CRs examined (30.4%) and those with triggers (35.9%) (Table 1). This could be due to patients being transferred to ICU and the cause of the AE was in another clinical setting. We analyzed the AEs comparing them to the triggers to allow for their identification:
■ Most AEs are associated with general care triggers (from C1 to C14) (n=6,103) (Table 2). If triggers are isolated, AEs are more frequently associated with care triggers (n=80) (Table 4).
■ The triggers related to general care have been identified 7,497 times, with an AE in 6,103 (81.4%).
■ Medications-related triggers (From M1 to M11) have been identified 2,878 times, with an AE in 1,525 (53%).
■ Surgery-related triggers (form S1 to S12) have been identified 323 times, with an AE in 148 (45.8%).
■ Obstetrics-related triggers (from P1 to P8) have been identified 492 times, with an AE in 61 (12.4%).
■ Intensive-care-related triggers (from I1 to I4) have been identified 1,817 times with an AE in 1,924 (i.e. it is very common for triggers to identify more AEs in the same patient) (Table 1 and Table 2).
However, if isolated triggers are considered, the intensive-care-related triggers were detected 46 times and they were correlated with only one AE (Table 4). This observation suggests that isolated triggers rarely allow to identify an AE and that the strength of the IHI’s GTT methodology is linked to the association of triggers. It is evident that the detection of many triggers in a CR is associated with a high probability of AEs. The analysis of the ROC curve (Figure 1) shows that it is sufficient to detect two triggers in a CR because it can be almost certain that in that CR there may be an AE.
We have classified the AEs using the ICPS 2009 classification and a clinical classification, developed by our group. In both classifications, hospital acquired infections are the most frequent AEs present (Table 5), observed in the ICU in 625 clinical records (84.3%). Surgical complications (n=175) were observed in 55.9% (n=99) in ICU, i.e. they were AEs in patients undergoing surgery and then transferred to ICU due to the onset of a complication. In 44.6% (n=80), the AEs are represented by hemorrhagic complications (intra- and post-operative hemorrhages or hematomas). Pressure ulcer lesions were detected in 172 cases, usually in the ICUs (n=112 - 65.1%). Also, the complications from procedures (n=109) were observed mainly in the ICU (n=78; 71.5%). In total, 33 complications from procedures are related to orotracheal intubation, 22 at central venous catheter, and 10 at childbirth analgesia. The complications of child birth (n= 47) were represented more frequently in 59.5% (n=28) by bleeding and in 29.8% (n=14) by lacerations.
Our study has some several limitations. The first concerns the inter-rater reliability assessment of review teams that is not available. The second limitation is the underlying quality of CRs, which may have affected the results. However, all reviewers received the same training and each team followed the same protocols to ensure reliability. Another limitation was the limited number of hospital wards that were included in this study.
The Global Trigger Tool is an effective method to identify the risk of AEs and track improvement of care. It provides to clinical teams an understanding of the patient safety issues that are present in their clinical area, as well as opportunities to improve. With active involvement of clinical teams, it places patient safety in the centre of clinical activity and fosters a culture of safety. It also provides an effective way to assess the quality of the clinical records. The drawback is that the process is labor intensive, particularly if the clinical records are paper based. The introduction of electronic medical records would allow a quicker process with the automation of the identification of triggers and the possibility to link triggers together in the identification of adverse events and near misses, especially where there has been more than one trigger19,20. Finally, we conclude that the analysis and monitoring of some triggers, as potential indicators of near misses, is important to prevent adverse events. Today's trigger could be tomorrow's adverse event.
Since the data used in this study was gathered during routine practice and is used for analysis of hospital procedures, no ethical approval was obtained. Every patient gave written informed consent on admission to hospital for the use of their data for scientific research. This consent is the "Information on the processing of personal data" with reference to the Italian law n. 196/2003 and Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (UE).
Harvard Dataverse: Replication data for Developing and implementing the Global Trigger Tool methodology across a large health system in Sicily, https://doi.org/10.7910/DVN/YQNKCC21
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
It was possible to realize this project thanks to the involvement and passion of the 199 reviewers, doctors, nurses and obstetrics, the cooperation of quality and patients safety managers of the Sicilian public hospitals, Antonino Drago and Rosario Raineri for their support on data analysis.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: I’m doctor specialist in Social medicine and organization of health care services and professor emeritus in Social Medicine and Public Health from the Faculty of Medicine in Skopje.
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health Services Research, Patient Safety and Quality of Care Measurement, Biostatistics
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health Services Research, Patient Safety and Quality of Care Measurement, Biostatistics
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Partly
Are all the source data underlying the results available to ensure full reproducibility?
No source data required
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Clinical quality improvement; patient safety; high-reliability organizations
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Partly
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Partly
Are all the source data underlying the results available to ensure full reproducibility?
No source data required
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: Dr. Naessens was an invited speaker at the initial meeting of Sicilian hospital personnel when this project was being implemented. His travel expenses were reimbursed, but no honorarium was necessary.
Reviewer Expertise: Health Services Research, Patient Safety and Quality of Care Measurement, Biostatistics
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |||
---|---|---|---|
1 | 2 | 3 | |
Version 4 (revision) 15 Jul 20 |
|||
Version 3 (revision) 09 Oct 19 |
read | read | |
Version 2 (revision) 11 Sep 19 |
read | ||
Version 1 07 Mar 19 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)