Home Browse A study on the efficacy of APACHE-IV for predicting mortality and...
ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
▬
✚
Research Article

A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran

[version 1; peer review: 2 approved]
Mohammad Ghorbani
https://orcid.org/0000-0002-5219-9096
1, Haleh Ghaem2, Abbas Rezaianzadeh3, Zahra Shayan4, Farid Zand5, Reza Nikandish6
Mohammad Ghorbani
https://orcid.org/0000-0002-5219-9096
1, Haleh Ghaem2, [...] Abbas Rezaianzadeh3, Zahra Shayan4, Farid Zand5, Reza Nikandish6
PUBLISHED 20 Nov 2017
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

Abstract

Background: Clinical assessment of disease severity is an important part of medical practice for prediction of mortality and morbidity in Intensive Care Unit (ICU). A disease severity scoring system can be used as guidance for clinicians for objective assessment of disease outcomes and estimation of the chance of recovery. This study aimed to evaluate the hypothesis that the mortality and length of stay in emergency ICUs predicted by APACHE-IV is different to the real rates of mortality and length of stay observed in our emergency ICU in Iran.
Methods: This was a retrospective cohort study conducted on the data of 839 consecutive patients admitted to the emergency ICU of Nemazi Hospital, Shiraz, Iran, during 2012-2015. The relevant variables were used to calculate APACHE-IV.  Length of stay and death or discharge, Glasgow coma score, and acute physiology score were also evaluated. Moreover, the accuracy of APACHE-IV for mortality was assessed using area under the Receiver Operator Characteristic (ROC) curve.
Results: Of the studied patients, 157 died and 682 were discharged (non-survivors and survivors, respectively). The length of stay in the ICU was 10.98±14.60, 10.22 ± 14.21 and 14.30±15.80 days for all patients, survivors, and non-survivors, respectively. The results showed that APACHE-IV model underestimated length of stay in our emergency ICU (p<0.001). In addition, the overall observed mortality was 17.8%, while the predicted mortality by APACHE-IV model was 21%. Therefore, there was an overestimation of predicted mortality by APACHE-IV model, with an absolute difference of 3.2% (p=0.036).
Conclusion: The findings showed that APACHE-IV was a poor predictor of length of stay and mortality rate in emergency ICU. Therefore, specific models based on big sample sizes of Iranian patients are required to improve accuracy of predictions.

Keywords

APACHE-IV, Mortality, Length of stay, Intensive care unit, emergency

Corresponding author: Haleh Ghaem
Competing interests: No competing interests were disclosed.
Grant information: This study was financially supported by Shiraz University of Medical Sciences (grant number 94-7636).
Copyright:  © 2017 Ghorbani M et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
How to cite: Ghorbani M, Ghaem H, Rezaianzadeh A et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran [version 1; peer review: 2 approved]. F1000Research 2017, 6:2032 (https://doi.org/10.12688/f1000research.12290.1) First published: 20 Nov 2017, 6:2032 (https://doi.org/10.12688/f1000research.12290.1) Latest published: 20 Nov 2017, 6:2032 (https://doi.org/10.12688/f1000research.12290.1)

Introduction

Clinical assessment of disease severity is an important part of medical practice to predict mortality and morbidity in Intensive Care Unit (ICU)1. An acceptable goal in ICU is saving the lives of critically ill patients, since not all patients admitted to an ICU have a normal life after leaving and some will not survive because of disease severity2.

Specialties of ICUs should predict patient outcomes to focus more on efficient use of ICU beds for critically ill patients2. Disease severity scoring systems can be used as a guidance for clinicians in the objective assessment of disease outcomes and estimation of the chance of recovery2. Acute Physiology And Chronic Health Evaluation (APACHE), introduced in 1981, considers various parameters, including vital signs, physiological variables, neurological score, urine output, age, and comorbid conditions3. The latest version of APACHE-IV is calculated based on 129 variables derived within the first 24 h of ICU admission1, which were assessed from over 110,588 patients admitted to more than 104 ICUs across the USA4,5. Some studies have suggested the superior advantage of APACHE-IV compared to other risk scoring systems6,7.

Evaluation of clinical outcomes and effectiveness of care in ICU patients is influenced by predictive scoring models that compute measures of disease severity and the associated probability of death. APACHE is a logistic regression model involving both physiological and laboratory parameters. It is a commonly used ICU stratification instrument, which is known as an accurate predictor of mortality. Yet, model accuracy decreases over time and requires updating occasionally. A study conducted in 2012 indicated that APACHE-III performance was inadequate even with a predicted mortality of only 2% higher than the observed mortality rate (16% vs. 14%)8. A similar study conducted on APACHE-IV showed that the ICU’s outcome prediction by the model is different to observed values in clinical setting between the predicted and the observed mortality rate9.

To our knowledge, no study has been conducted to evaluate the accuracy of APACHE-IV for predicting mortality and length of stay in emergency ICUs in Iran. This study aimed to evaluate the hypothesis that the mortality and length of stay in emergency ICUs predicted by APACHE-IV is different than that observed in reality.

Methods

This was a retrospective cohort study conducted on the medical records of 839 consecutive patients admitted in the emergency ICUs in Nemazi Hospital, Shiraz, Iran, between July 2012 and July 2015. The patients of this study were selected from all patients referred to the ICUs of the Center during the study period using convenient sampling method. The total number of patients admitted during this period was 839. The inclusion criterion was minimum 24 hour admission in the ICU and there was no exclusion criterion for this study. The Namazi Hospital is a tertiary referral hospital affiliated to Shiraz University of Medical Sciences, Shiraz, Iran. All the experimental procedures and study protocol of the study were approved by the local ethics committee of Shiraz University of Medical Sciences (protocol no. 94-7636), which were in complete accordance with the ethical standards and regulations of human studies of the Declaration of Helsinki (2014).

The medical records of 839 consecutive patients admitted to the emergency ICUs of Nemazi Hospital were analyzed. The variables used to calculate APACHE-IV score included age, sex, dates of admission, discharge or death, systolic and diastolic blood pressure, body temperature, heart rate, respiratory rate, glucose, blood urea nitrogen, serum sodium, creatinine, blood hematocrit, white blood cells, serum albumin and bilirubin, urine output during the first 24 h of ICU admission, pH, fraction of inspired oxygen, partial pressure of carbon dioxide, partial pressure of oxygen, and bicarbonate5.

Death or discharge and length of stay in ICU were followed up by referring to patients’ medical records. Additionally, APACHE-IV score, Glasgow coma score (GCS), and acute physiology score (APS) were calculated according to www.cerner.com (the authors registered as a user in order to calculate all the parameters).

Statistical analysis

Qualitative variables were expressed as number and percentage, and quantitative variables as mean ± standard deviation. Student’s t-test, Mann–Whitney U, Wilcoxon rank test, and Chi-square tests were used where appropriate to compare survivors and non-survivors regarding demographic and clinical variables. In addition, Spearman’s correlation coefficient was used to examine the relationship between APACHE-IV score and length of stay in ICU. Finally, accuracy of APACHE-IV for mortality was assessed using area under the Receiver Operator Characteristic (ROC) curve with an attribution of ‘good’ > 0.80. The data are expressed as mean ± SD for all variables. All statistical analyses were carried out using Stata (version 13, Windows). As the distribution of the quantitative variables was not normal, Mann Whitney U test was used for comparisons of the difference between the survivor and non-survivor groups. For the sex variable the Chi-square test was used. p ≤ 0.05 was considered to be statistically significant.

Results

This study was conducted on 839 patients among whom, 157 died and 682 were discharged (non-survivors and survivors, respectively). The length of stay in ICU was 10.98±14.60, 10.22±14.21, and 14.30±15.80 days in all patients, survivors, and non-survivors, respectively. Demographic information and the clinical features of the patients are summarized in Table 1.

Table 1. Demographic and clinical features of the 839 patients.

For the comparisons of quantitative variables Mann Whitney U test was used and for the sex variable the Chi square test was used. P < 0.05 represents significant difference between the survivor and non-survivor groups.

CharacteristicsTotal (n=839)Survivors
(n=682)		Non-survivors
(n=157)		P-value
Age (years), mean ± SD 48.83±19.6546.72±19.0558.03±19.610.001
Sex, n (%)
Male
Female
452 (53.9)
387 (46.1)
374 (82.7)
308 (79.6)
78 (17.3)
79 (20.4)		0.243
ICU length of stay (days), mean ± SD10.98±14.6010.22±14.2114.30±15.800.001
Glasgow coma score, mean ± SD10.43±4.1411.03±3.937.77±4.000.001
APACHE-IV score, mean ± SD52.93±29.4846.34±22.8881.81±36.990.001
Acute physiology score, mean ± SD51.65±29.2145.06±22.5080.52±36.860.001

The results showed no significant difference between the two groups regarding sex (p=0.243). However, the two groups were significantly different with respect to the means of age (p≤0.001), ICU length of stay (p≤0.001), GCS (p≤0.001), APACHE-IV score (p≤0.001), and APS (p≤0.001) (Table 1).

Evaluation of APACHE-IV score

Outcome variables have been summarized in Table 2. Accordingly, mean ± SD of observed length of stay in ICU was 10.98±14.60 days. However, predicted ICU length of stay by the APACHE-IV model was 5.43±2.50 days (p<0.001). This indicated that APACHE-IV underestimated ICU length of stay in our emergency ICU. Additionally, the overall observed mortality was 17.8%, while the predicted mortality by APACHE-IV was 21%. Thus, mortality was overestimated by APACHE-IV model with an absolute difference of 3.2% (p=0.036).

Table 2. Outcome variables for the total cohort (n=839).

For the comparisons of stay length Mann Whitney U test was used and for the mortality rate the Chi square test was used. P < 0.05 represents significant difference between the survivor and non-survivor groups.

CharacteristicsObservedPredictedP-value
ICU length of stay, mean ± SD10.98±14.605.43±2.50<0.001
Mortality, n (%)157 (17.8)177 (21.0)0.036

ROC curve for APACHE-IV score and observed mortality has been depicted in Figure 1. Accordingly, area under the curve of the APACHE-IV score was 0.81, 95% CI (0.77, 0.84). These values were statistically significant and could be an appropriate predictor for observed mortality. Nevertheless, there was a significant weak correlation between APACHE-IV score and observed ICU length of stay (r=0.175, p<0.0001).

68e36b28-8606-4b08-91d0-8194077d472a_figure1.gif

Figure 1. ROC curve for APACHE-IV score and observed mortality.

Dataset 1.Data for the study on efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran.

Discussion

A retrospective cohort study was conducted among 839 patients referred to ICU at Namazi Hospital in Shiraz, Iran. The study results showed that APACHE-IV underestimated the length of stay in our emergency ICU. In addition, the overall observed mortality was 17.8%, while the predicted mortality by APACHE-IV was 21%. Thus, there was an overestimation of predicted mortality by APACHE-IV, with an absolute difference of 3.2% (p=0.036).

Several factors may contribute to poor performance of APACHE-IV in emergency ICU. APACHE-IV is a good benchmark to determine disease severity; however, the present study results indicated that it did not function well to predict the risk of mortality and length of stay in emergency ICU. Other studies also reported this score not to be predictive of mortality10,11. The poor estimate may be attributed to various reasons. Firstly, the estimations were achieved based on American rather than our own patients’ data. Generally, predictive scoring systems function appropriately in populations where scores are derived from the same population data. Therefore, many experts recommend external validation at national, regional, or institutional levels. For example, APS3 has several customized versions for seven geographic regions12,13.

Secondly, in America, where APACHE was calibrated, patients go from ICU to ‘step down’, a halfway ward, before moving to general wards. In Iran, patients directly go to general wards, and consequently, they have to stay in ICUs for a longer time period than American patients.

Thirdly, even if scores are achieved by patients’ data, they must be calibrated over time. This is because case-mix varies, quality of care improves, and types of disease changes over time. In general, accurate calibration is a key characteristic that should be ensured for all risk scoring systems. Calibration may weaken over time, especially due to the effects of altered patient interventions and case-mix. This often results in overestimation of death or mortality14.

The findings of the present study revealed that APACHE-IV score based on our data would be an appropriate predictor for the observed mortality, while this relationship was not confirmed by the APACHE-IV score according to the American database9. Moreover, our findings showed a similar relationship between APACHE-IV score and ICU length of stay with the study conducted on the United States database9. Overall, a large patient’s database should exist in order for APACHE-IV to correctly predict outcomes (i.e. mortality and ICU length of stay).

Strengths and limitations of the study

The strength of this study should be noted. This study is the first study in Iran that demonstrated that predictions of mortality and ICU length of stay should be based on data obtained from Iranian and not American patients. However, this study had some limitations, the first of which being the intrinsic shortcomings of its retrospective design (inability to confirm causation, and dependence on medical records). Another study limitation was its small sample size; however, to date, our cohort of 839 patients is the largest reported study of patients admitted to emergency ICUs in Iran.

In conclusion, the findings of this study suggested that the American based APACHE-IV score is a poor predictor of length of stay and mortality in emergency ICU in Iran. Therefore, specific models based on big sample sizes of our patients from Iran are required to improve the accuracy of predictions of mortality and ICU length of stay for our country.

Data availability

Dataset 1: Data for the study on efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. doi, 10.5256/f1000research.12290.d17798715

Competing interests

No competing interests were disclosed.

Grant information

This study was financially supported by Shiraz University of Medical Sciences (grant number 94-7636).

Acknowledgements

The present article was extracted from the thesis written by Mohammad Ghorbani. The authors would like to thank Ms. A. Keivanshekouh at the Research Improvement Center of Shiraz University of Medical Sciences for improving the use of English in the manuscript.

References

  • 1.  Saleh A, Ahmed M, Sultan I, et al.: Comparison of the mortality prediction of different ICU scoring systems (APACHE II and III, SAPS II, and SOFA) in a single-center ICU subpopulation with acute respiratory distress syndrome. Egyptian Journal of Chest Diseases and Tuberculosis. 2015; 64(4): 843–8. Publisher Full Text
  • 2.  Rahimzadeh P, Taghipur Anvari Z, Hassani V: Estimation of mortality rate of patients in surgical intensive care unit of Hazrat-Rasul hospital of Tehran using the APACHE II standard disease severity scoring system. Hakim Res J. 2008; 11(1): 22–8. Reference Source
  • 3.  Knaus WA, Zimmerman JE, Wagner DP, et al.: APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med. 1981; 9(8): 591–7. PubMed Abstract | Publisher Full Text
  • 4.  Zimmerman JE, Kramer AA, McNair DS, et al.: Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med. 2006; 34(10): 2517–29. PubMed Abstract | Publisher Full Text
  • 5.  Zimmerman JE, Kramer AA, McNair DS, et al.: Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006; 34(5): 1297–310. PubMed Abstract | Publisher Full Text
  • 6.  Kuzniewicz MW, Vasilevskis EE, Lane R, et al.: Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest. 2008; 133(6): 1319–27. PubMed Abstract | Publisher Full Text
  • 7.  Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, et al.: External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care. 2011; 26(1): 105.e11–8. PubMed Abstract | Publisher Full Text
  • 8.  Paul E, Bailey M, Van Lint A, et al.: Performance of APACHE III over time in Australia and New Zealand: a retrospective cohort study. Anaesth Intensive Care. 2012; 40(6): 980–94. PubMed Abstract
  • 9.  Zimmerman JE, Kramer AA: Outcome prediction in critical care: the Acute Physiology and Chronic Health Evaluation models. Curr Opin Crit Care. 2008; 14(5): 491–7. PubMed Abstract | Publisher Full Text
  • 10.  Hanisch E, Brause R, Paetz J, et al.: Review of a large clinical series: Predicting death for patients with abdominal septic shock. J Intensive Care Med. 2011; 26(1): 27–33. PubMed Abstract | Publisher Full Text
  • 11.  Chan T, Bleszynski MS, Buczkowski AK: Evaluation of APACHE-IV Predictive Scoring in Surgical Abdominal Sepsis: A Retrospective Cohort Study. J Clin Diagn Res. 2016; 10(3): Pc16–8. PubMed Abstract | Free Full Text
  • 12.  Metnitz PG, Moreno RP, Almeida E, et al.: SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med. 2005; 31(10): 1336–44. PubMed Abstract | Publisher Full Text | Free Full Text
  • 13.  Moreno RP, Metnitz PG, Almeida E, et al.: SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005; 31(10): 1345–55. PubMed Abstract | Publisher Full Text | Free Full Text
  • 14.  Kelley MA, Manaker S, Finlay G: Predictive scoring systems in the intensive care unit. UpToDate. 2012. Reference Source
  • 15.  Ghorbani M, Ghaem H, Rezaianzadeh A, et al.: Dataset 1 in: A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. F1000Research. 2017. Data Source

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 20 Nov 2017
Comment
Author details Author details
Competing interests
Grant information
Article Versions (1)
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
 - -
Citations
SEE MORE DETAILS
CITE
how to cite this article
Ghorbani M, Ghaem H, Rezaianzadeh A et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran [version 1; peer review: 2 approved]. F1000Research 2017, 6:2032 (https://doi.org/10.12688/f1000research.12290.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 20 Nov 2017
Views
5
Cite
Reviewer Report 27 Nov 2017
Ali Yadollahpour, Department of Medical Physics, Faculty of Medicine, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran;  Bioelectromagnetic Clinic, Ahvaz Imam Khomeini Hospital, Ahvaz, Iran 
Approved
VIEWS 5
https://doi.org/10.5256/f1000research.13304.r28373
The manuscript entitled "A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran" was reviewed. This is a well conducted study with appropriate methodology. The data provided are of ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Yadollahpour A. Reviewer Report For: A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran [version 1; peer review: 2 approved]. F1000Research 2017, 6:2032 (https://doi.org/10.5256/f1000research.13304.r28373)
The direct URL for this report is:
https://f1000research.com/articles/6-2032/v1#referee-response-28373
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Report a concern
Views
9
Cite
Reviewer Report 22 Nov 2017
Nelson Elias, Vila Velha Hospital, Vila Velha, Brazil 
Jose Eduardo Grandi Ribeiro, Vila Velha Hospital, Vila Velha, Brazil 
Approved
VIEWS 9
https://doi.org/10.5256/f1000research.13304.r28177
Classification:
Research - retrospective cohort study with a critical review of the corresponding literature in Apache IV scoring systems
 
Originality and Relevance:
a)         Pertinent for the journal - F1000Research
b)         Original
c)         Contribution ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Elias N and Grandi Ribeiro JE. Reviewer Report For: A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran [version 1; peer review: 2 approved]. F1000Research 2017, 6:2032 (https://doi.org/10.5256/f1000research.13304.r28177)
The direct URL for this report is:
https://f1000research.com/articles/6-2032/v1#referee-response-28177
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Report a concern

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 20 Nov 2017
Comment

Open Peer Review

Reviewer Status

Alongside their report, reviewers assign a status to the article:

Approved
The paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved
Fundamental flaws in the paper seriously undermine the findings and conclusions

Reviewer Reports

Invited Reviewers
1 2
Version 1
20 Nov 17 		read read
  1. Nelson Elias, Vila Velha Hospital, Vila Velha, Brazil
    Jose Eduardo Grandi Ribeiro, Vila Velha Hospital, Vila Velha, Brazil
  2. Ali Yadollahpour, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran; Ahvaz Imam Khomeini Hospital, Ahvaz, Iran

Comments on this article

All Comments(0)

Add a comment
Sign up for content alerts

Browse by related subjects

    keyboard_arrow_left Back to all reports
    keyboard_arrow_left Back to all reports
    Alongside their report, reviewers assign a status to the article:
    Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
    Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
    Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
    Sign In
    If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

    The email address should be the one you originally registered with F1000.
    Email address not valid, please try again

    You registered with F1000 via Google, so we cannot reset your password.

    To sign in, please click here.

    If you still need help with your Google account password, please click here.

    You registered with F1000 via Facebook, so we cannot reset your password.

    To sign in, please click here.

    If you still need help with your Facebook account password, please click here.

    Code not correct, please try again
    Email us for further assistance.
    Server error, please try again.