Diagnostic accuracy of Cipto Triage Method in the emergency department

Ethics and consent

This study had ethical approval from Ethical Committee of Cipto Mangukusumo Hospital/Faculty of Medicine Universitas Indonesia protocol number 21-11-1250, approval date January 17^th, 2022.²⁰ All study participants or their representative had given written consent. Individual data were treated confidentially. All methods were carried out in accordance with relevant guidelines and regulations. Study reporting followed the Standard for Reporting of Diagnosis Accuracy Studies (STARD) guidelines.^6,20

Study design

This cross-sectional diagnostic accuracy study was conducted in the ED of Dr. Cipto Mangunkusumo Hospital an academic referral hospital in Jakarta, Indonesia. The ED manages approximately 30,000 visits annually. The clinical spectrum of ED cases ranges from simple tropical diseases to terminally ill non-communicable diseases.

Participants

Inclusion criteria were patients aged 18 years and above who visited the ED, underwent routine triage assessment and were treated in the ED treatment zones. The exclusion criteria were trauma, pregnancy-related and psychiatric-related emergencies. Exclusion of these conditions was due to the different clinical syndrome knowledge that was not covered during the triage training. Potential eligible populations were identified by the EMO based on these criteria during the data collection period. All eligible male and female patients had equal chance to participate in the study. Adequate representation of males and females was expected based on demographic information of ED patients from previous study.⁴ Sex of participants was defined based on information from citizen identification card.

Test method

The Cipto Triage Method was performed (extended data 2²⁰ by collecting information like chief complaint, focused history, and physical examination related to the chief complaint, simple vital signs (level of consciousness, respiratory rate, heart rate, and behavioural pain scale), and discriminators (extended data 3²⁰). Discriminators can modify the risk of severity or urgency level such as comorbidity for medical cases, mechanism of injury for trauma cases, previous history of pregnancy-related diseases for obstetric cases and previously identified congenital or comorbid conditions for neonates and children.

After collecting the specific information, the triage nurse then categorised the patients into one of three categories of acuity. The resuscitation category was used if the patient was in critical condition and needed immediate medical assessment or life-saving intervention. The urgent category includes patients who exhibit signs and symptoms that need urgent medical assessment but do not need immediate life-saving intervention. The non-urgent category includes patients who exhibit signs and symptoms that do not need urgent medical assessment or intervention but might need clinical evaluation in the ED before getting discharged.

Patients who are categorised in the resuscitation triage category are sent to the red zone, and the response time is immediate; patients who are categorised in the urgent triage category are sent to the yellow zone, and the response time is appropriate if treatment is given in less than 30 minutes; patients who are categorised in the non-urgent triage category are sent to the green zone, and the response time is appropriate if treatment is given in less than 60 minutes.

The triage nurse decision using CTM was considered as the index test to evaluate the implementation of the nurse-based triage programme recommended by a previous study.⁴ This policy claimed to allocate human resources by replacing physicians from the triage counter to the ED treatment zones and focusing on appropriate response time. However, triage nurses must have comparable accuracy to physicians in assessing clinical acuity.

The EMO was a general physician working in the ED under the direct supervision of an emergency medicine specialist. Assessment by the EMO was considered the reference standard because of his/her additional competence in clinical diagnosis, allowing more precise information collection in CTM. The reference standard was applied regardless of the index test results.

Training

Before the study, a three-week online (triage principle, step by step triage process, communication at triage counter, medical evacuation at triage, the role triage officer in resuscitation) and hands-on (role play, observeration) combined education programme called ‘proactive triage training’ was conducted (extended data 4²⁰). Nurses who performed triage and the EMO participated in the training (November – December 2021).

Training participants who provided written consent as raters were included in this study. Reliability analysis after the training was performed (January 2022) using 30 case-based scenarios conducted for the nurse (20 raters) and EMO (10 raters) before the data collection process (triage assesment of the ED patients).

The scenarios were created and validated by three expert panels: HH (Internal Medicine and Emergency Medicine Specialist), SS (Anaesthesiologist and Emergency Medicine Specialist) and IAA (Orthopaedic Surgeon and Emergency Medicine Specialist). All scenarios were written in Bahasa (extended data 5²⁰) and never used in any previous ED training.

Data collection

Data collection was performed during routine ED services (April 2022–July 2022). Eligible patients underwent triage assessment based on the CTM by a triage nurse and were then referred to the EMO in the ED treatment zone. The EMO further performed an independent triage assessment based on the CTM with additional clinical examination to establish a clinical diagnosis. Triage decisions from nurses were written on the triage assessment form, whereas the EMO wrote on a separate medical record form (extended data 6²⁰). Written consent from the patients or their representatives was obtained before the written triage decision was made by the EMO. No potential harm was expected during the data collection. Patient disposition from triage nurses to the EMO was relatively rapid (immediate response time for resuscitation triage and less than five minutes for urgent or non-urgent triage). The EMO triage assessment was performed without interrupting any business processes in the ED. Furthermore, there was no intervention between the index and reference tests that could modify triage assessment.

This study was single-blinded, meaning that triage nurses were blinded from the triage decision made by EMO. However, EMOs were aware of the triage decision by nurses because patients were sent from the triage counter to the appropriate ED zone followed by the disposition process according to triage category on the daily basis of ED service.

Analysis

Before performing data collection for the diagnostic accuracy of the triage, we used 30 case-based scenarios for reliability analysis among nurses and EMOs who performed the index test and the reference standard test (extended data 5²⁰). The Krippendorff alpha coefficient was used to estimate inter-rater reliability among the nurses and EMO groups; to incorporate the reliability of more than two raters and ordinal-type data (three triage categories).⁷ The reliability analysis was performed using SPSS version 25.0.

Summary statistics provided descriptive information about the nurses, EMO and participant characteristics. The diagnostic accuracy of index test was estimated using three contingency tables with nominal outcomes for each triage category (resuscitation vs. other triage categories, urgent vs. other triage categories and non-urgent vs. other triage categories). Sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio (LR+) and negative likelihood ratio (LR-) were calculated. Furthermore, variability analysis by stratification of study participants based on age (<60 years old and ≥60 years old) in each triage category was performed.

Age stratification was performed because it is a strong predictor of clinical outcome, but critical signs and symptoms in geriatric patients are often vague and could mislead the triage decision.^8,9 The diagnostic accuracy analysis was performed using STATA ver. 17.0.

Patients who did not undergo a reference standard test were considered to have missing data. Inability of the nurse to determine triage categories was considered an indeterminate result of the index test. Both were excluded from the study.

Sample size measurement

Sample size estimation was based on sensitivity (for resuscitation and urgent categories) and specificity (for non-urgent triage category) estimations using Buderer’s formula.¹⁰ The highest estimation number was used as the sample size.

Since the concept of ED triage was to screen clinical acuity, we propose that higher sensitivity is more important than specificity for resuscitation (sensitivity, 80%) and urgent (sensitivity, 80%) triage categories.¹¹ For the non-urgent triage category, higher specificity is more important (specificity, ≥80%) to prevent under-triage of the patients. The prevalence of ED based on triage categories from a previous study was 15% resuscitation, 64% urgent, and 21% non-urgent.⁴

The predetermined confidence level, power of the study and desired absolute precision were 95%, 80%, and 10%, respectively. The diagnostic performance variation of each triage category in different age groups (<60 years and ≥60 years) was determined. Thus, the final expected research sample was the original sample size multiplied by two (from the two age group categories), which was 820. The final sample size was 902 participants after anticipating 10% of the potential missing values during participant selection.

Triage nurses and EMOs performed triage assessment for all eligible ED patients during the study period. Simple random sampling will then be performed by the researchers using random generator in Stata ver. 17.0 to obtain participants from all eligible ED patients based on sample size measurement (Figure 1).

Figure 1. Sample selection process.