ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

The development of novel score model as a predictor of in-hospital mortality of adult cardiac surgery patients in Indonesia

[version 1; peer review: 1 approved with reservations]
PUBLISHED 10 Oct 2024
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

Abstract

Introduction

In recent years, the number of cardiac surgeries has been on the rise. Several predictive models have been developed to assess the risk of mortality for these patients. EuroSCORE II, a widely used model, demonstrates good discrimination and calibration for the European population, but its accuracy may vary in the Indonesian population. The purpose of this study was to develop a scoring model tailored to Indonesian population, which may have better accuracy in assessing in-hospital mortality risk among adult cardiac surgery patients.

Method

A retrospective study was conducted using medical records of adult cardiac surgery patients from four participating hospitals. Potential risk factors were included as variables and analyzed using bivariate and multivariable logistic regression with the L-backward method. Bootstrapping was applied to enhance the model’s validity. Receiver operating characteristic (ROC) curves were created for each model, and the area under the curve (AUC) was calculated to assess discrimination ability, while the Hosmer-Lemeshow test was used to evaluate calibration.

Results

We extracted data from 4,875 patients, with a mean age of 50.41 years, and most patients were men (63.1%). The majority of patients were in NYHA class I-II. The in-hospital mortality rate was 6.5%. From 62 potential variables, 13 variables were included in the final model. Our new model demonstrated strong discrimination and calibration (AUC 0.7564; Hosmer-Lemeshow p = 0.9510).

Conclusion

The newly developed scoring model exhibited good discrimination and calibration, making it a promising tool for predicting in-hospital mortality in adult cardiac surgery patients in Indonesia.

Keywords

in-hospital mortality, cardiac surgery, model building, risk prediction, retrospective study

Introduction

The trend of cardiac surgery is showing an increase in Indonesia. Cardiac surgery procedures have shown an increase from year to year in Indonesia.1 Cardiac surgery has been regarded as one of the most expensive surgical procedures in a hospital and is associated with high mortality and morbidity rates.2 To minimize perioperative mortality, patient selection has been advocated, using risk assessment systems developed over the last decades. Decision-making for surgery requires more than just clinical experience and intuition. This decision-making process is highly complex, and scoring systems were developed as a way to objectively assess patient conditions, determine patient mortality risk profiles, and improve patient outcomes.3

Several risk stratification models have been developed for cardiac surgery patients. The three most commonly used scoring systems are the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II,4 Society of Thoracic Surgeons (STS) score,2 and Parsonnet score.5 EuroSCORE II has been validated in various population settings and remains the standard for many institutions.6 Among the various scoring systems, EuroSCORE II is the most frequently studied in Indonesia. Results from several studies in Indonesia have found that EuroSCORE II shows good discriminatory but weak calibration performance.79

Existing risk models shows suboptimal performance in Indonesia population. These models generally predict outcomes more accurately in the settings where they were initially developed. Socio-economic conditions, living standards, health resources, and geographic and ethnic characteristics may affect the applicability of these models in different regions.10 In clinical practice, most predictive scores show lower accuracy when validated on new population. The two common possible explanations for this are that the predictive scores were not adequately developed as well as major differences between the populations in which they were developed and validated.11

To this date, there is no known risk stratification model that has been developed from a local database or with characteristics that align with the Indonesia population. The main issue that we faced in Indonesia is the lack of a widely applicable scoring system for assessing in-hospital mortality risk in cardiac surgery patients. We propose to develop a mortality risk model for cardiac surgery patients from local database derived from the Indonesian population.

Methods

We conducted a retrospective study using medical record data from adult cardiac surgery patients who underwent procedures at four tertiary referral hospitals in Indonesia (Dr. Sardjito General Hospital in Yogyakarta, Harapan Kita National Heart Center Hospital in Jakarta, Dr. Kariadi General Hospital in Semarang, and Abdoel Wahab Sjahranie Regional Hospital in Samarinda). The subjects were adult patients (aged 18 to 85 years) who underwent cardiac surgery procedures between January 1, 2016, and December 31, 2020. Heart transplant patients, aortic dissection patients, and patients with incomplete primary medical records were excluded from the study. It was estimated that between 75 and 3500 subjects were required for each potential predictor. Thus, a minimum of 3500 subjects were required for this study. This study complies with the Declaration of Helsinki and received ethical approval from the Medical and Health Research Ethics Committee of the Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia on October 30, 2019 (No: KE/FK/1277/EC/2019)

The selection of potential predictors and definitions for each variable was conducted through a focused group discussion. Selected candidate variables were based on variables present in EuroSCORE II, CARE, Parsonnet, STS scores. We also considered other risk factors that not included in existing scores but proven or considered strongly associated with post-cardiac surgery mortality risk, and potentially suitable as candidate variables in model development.

The statistical analysis began with bivariate analysis using t-tests for normally distributed numeric variables; otherwise, Mann-Whitney tests were performed. Chi-square tests were performed for categorical variables. Candidate variables with p < 0.25 or clinically significant associations that cannot be ignored, such as age and gender, and occurrence rates exceeding 1% of the total sample size, were included in multivariate analysis. Multivariate analysis was performed using logistic regression with backward elimination method. Models with significant variables underwent bootstrapping. Odds ratios (OR) and 95% confidence intervals (CI) were calculated from bootstrapped coefficients. ROC (receiver operating characteristic) curves were constructed for each model, and the AUC (area under the curve) values were assessed and compared to determine the discriminative performance of each model. The final model is the best fit model for in-hospital mortality. Model calibration was tested using the Hosmer-Lemeshow test, comparing the model’s predicted mortality with actual mortality. A risk prediction model with a p-value > 0.05 is considered well-calibrated. The analysis was performed using STATA 15 software.

Results

A total of 4,940 samples from 4 participating hospitals met the initial inclusion criteria. Of these, 65 subjects were excluded (53 patients with aortic dissection and 12 patients with incomplete primary outcome data), resulting in a final sample size of 4,875 patients (Figure 1).

6731bfb2-fb08-4a2a-abef-6422219e5d0e_figure1.gif

Figure 1. Study subjects flow diagram.

Compared to the EuroSCORE II database, our population was younger, with median age of 50.4 years, with higher proportion of female patients (36.9 %), and more patients had NYHA class I-II,. The in-hospital mortality rates for cardiac surgery patients at Dr. Sardjito General Hospital, Dr. Kariadi General Hospital, Abdoel Wahab Regional Hospital, and Harapan Kita National Heart Center were 8.6% (74 patients), 8.7% (113 patients), 9.3% (30 patients), and 4.18% (101 patients), respectively. These translated into overall mortality rate of 6.5%, which is higher than the mortality rate in the EuroSCORE II database. The demographic characteristics of the study subjects are shown in Table 1.

Table 1. The demographic characteristics of the study subjects.

VariablesTotalSardjito HospitalKariadi HospitalAbdul Wahab HospitalHarapan Kita Hospital
Age50.4 ± 13.447.1 ± 14.447.0 ± 13.649.9 ± 12.453.4 ± 12.2
Sex
 Male3074 (63.1)451 (53.3)704 (54.4)203 (63.2)1714 (71)
 Female1801 (36.9)395 (46.7)588 (45.5)118 (36.8)700 (29)
BMI23.9 ± 5.323.1 ± 4.822.6 ± 4.723.6 ± 4.824.7 ±5.7
LVEF58.3 ± 13.560.4 ± 1258.5 ± 13.657.4 ± 13.757.6 ± 13.8
NYHA class
 I-II3218 (66)663 (78.4)887 (68.6)161 (50.2)1507 (62.4)
 III-IV165734183 (21.6)406 (31.4)160 (49.8)907 (37.6)
Diabetes Mellitus
 No3719 (76.3)687 (81.2)1098 (84.9)262 (81.6)1671 (69.2)
 Yes1156 (23.7)159 (18.8)195 (15.1)59 (18.4)743 (30.8)
Outcome
 Alive4557 (93.5)772 (91.3)1180 (91.3)291 (90.7)2313 (95.8)
 Died318 (6.5)74 (8.6)113 (8.7)30 (9.3)101 (4.2)
Procedures
 CABG2041 (41.9)95 (11.2)410 (31.7)151 (47.2)1385 (57.4)
 Valve1835 (37.6)384 (45.4)615 (47.6)97 (30.2)739 (30.6)
 ASD433 (8.9)179 (21.1)152 (11.8)30 (9.3)72 (2.9)
 VSD97 (2)45 (5)30 (2.3)9 (2.8)13 (0.5)
 Combination311 (6.4)134 (15.8)74 (5.7)32 (9.9)71 (2.9)
 Others158 (3.2)9 (1.5)12 (0.9)2 (0.6)135 (5.6)

Of the 62 variables included in bivariate analysis, 36 variables had p < 0.25. The L-backward multivariable logistic regression analysis was performed and 19 variables were retained. These 19 variables were used as base model for our model building process.

After L-backward multivariable logistic regression analysis was performed, we further selected variables with p < 0.05 or with high β coefficient Model selection was based on AUCs of ROC curves and AIC values. The final model consisted of 13 variables (Table 2).

Table 2. Bivariate and multivariate analysis of candidate variables.

VariablesOutcomespBase modelFinal model
AliveDiedBpBp
Age50.18 ± 13.29853.62 ± 13.763< 0.0010.041< 0.0010.038< 0.001
BMI < 18.5 kg/m2950 (92)83 (8)0.027*0.934< 0.0010.987< 0.001
History of cardiac surgery105 (77.8)30 (22.2)< 0.0011.537< 0.0011.473< 0.001
History of PCI100 (75.8)32 (24.2)< 0.0011.4360.0011.5010.001
Severe immobility158 (77.1)47 (22.9)< 0.0011.074< 0.0011.286< 0.001
Two procedures or more850 (87.7)119 (12.3)< 0.0010.882< 0.0010.928< 0.001
CPB4139 (93.2)304 (6.8)0.0040.6160.0380.7110.015
Hemoglobin < 12 g/dL174 (79.2)45 (20.8)< 0.0010.9870.0041.225< 0.001
LVEF ≤ 30%167 (86.1)27 (8.5)< 0.0010.948< 0.0010.814< 0.001
Creatinine clearance ≤ 50 ml/min758 (88.3)100 (11.7)< 0.0010.2970.0430.3520.016
Left main disease478 (89.8)54 (10.2)< 0.0010.736< 0.0010.701< 0.001
TAPSE < 17 mm425 (89.5)50 (10.5)< 0.0010.3310.0470.4120.022
Moderate-severe tricuspid regurgitation1027 (91)102 (9)< 0.0010.3700.0230.4430.004
Anticoagulant337 (88.7)43 (11.3)< 0.0010.4420.025--
Organ failure521 (90.3)56 (9.7)0.0010.4820.007--
Coronary abnormalities< 0.001--
 None2331 (93.2)171 (6.8)Ref.
 1 VD108 (81.8)24 (18.2)-0.8750.023
 2 VD200 (92.2)17 (7.8)-1.044< 0.001
 3 VD1918 (94.8)106 5.2)-0.7460.008
Arrhythmia881 (90.7)90 (9.3)< 0.0010.3790.017--
Congestive heart failure4059 (94.7)228 (5.3)< 0.001-1.4020.000--
CARE score 3-5828 (90.5)87 (9.5)< 0.0010.2870.052--
Constant-4.6990.547-6.424< 0.001

Our final model had an AUC of 0.7564 for the ROC curve (Figure 2), indicating good discriminative performance (AUC > 0.7). Goodness-of-fit test with the Hosmer-Lemeshow (HL) test was used to evaluate the calibration of the model. The p-value for the HL test of the new scoring model in this study was p = 0.9510, indicating good calibration (Table 3).

6731bfb2-fb08-4a2a-abef-6422219e5d0e_figure2.gif

Figure 2. ROC curve of the final model.

Table 3. Goodness-of-fit of final model.

GroupsNActual (%)Predicted (%)Min %Max %HL
14945 (1.0)6.2 (1.2)0.41.70.22
251113 (2.5)10.1 (2.0)1.72.20.83
346814 (3.0)11.7 (2.5)2.22.70.48
447811 (2.3)14.3 (3.0)2.73.30.80
551020 (3.9)18.5 (3.6)3.34.00.13
646421 (4.5)20.8 (4.5)4.05.00.00
748824 (4.9)27.9 (5.7)5.06.50.58
848739 (8.0)36.3 (7.5)6.58.70.22
948848 (9.8)52.8 (10.8)8.713.60.50
10487123 (25.3)119.4 (24.5)13.680.10.14
Total4875318 (6.5)318 (6.5)0.480.13.92

Discussion

The purpose of this study was to develop a model that can be widely applied across various hospitals in Indonesia and integrated into local data collection and management systems. From this study, we proposed the following logistic regression formula intended for predicting in-hospital mortality for cardiac surgery patients:

Predicted mortality=e(β0+β1x1)1+e(β0+β1x1)

Where e = 2.718, β0 (constant of the logistic regression equation) = -6.424, βi is the coefficient of variable Xi, and so forth: 1.501 (history of PCI), 1.473 (history of cardiac surgery), 1.286 (severe musculoskeletal and neurological disorders), 1.225 (Hemoglobin < 12 mg/dL), 0.987 (BMI < 18.5 kg/m2), 0.928 (two or more procedures), 0.814 (LVEF ≤ 30%), 0.711 (CPB), 0.701 (left main disease), 0.443 (moderate to severe tricuspid regurgitation), 0.412 (TAPSE < 17 mm), 0.352 (creatinine clearance < 50 ml/min), 0.038 (age). Several variables in the new scoring model were not present in EuroSCORE II, these include history of PCI, hemoglobin < 12 g/dL, BMI < 18.5 kg/m2, left main disease, TAPSE < 17 mm, and moderate to severe tricuspid regurgitation.

In a retrospective study of 6,504 CABG patients, PCI was an independent predictor for in-hospital mortality (3.6% in patients with PCI vs 2.3% in patients without PCI).12 Another study also found PCI as predictor of in-hospital mortality and major adverse cardiac events (MACEs), with the negative impact being more pronounced in certain groups, such as patients with diabetes mellitus and triple-vessel coronary artery disease (CAD 3 VD).13 In contrast, a multi-center study of 34,316 CABG patients from 16 institutions found no significant association between history of previous PCI and mortality rates, though it did observe higher rates of major complications and longer hospital stays in the PCI group.14 The history of percutaneous coronary intervention (PCI) is associated with a poor prognosis in cardiac surgery patients due to various factors. The presence of a stent necessitates the surgeon to perform anastomosis on more distal and thinner branches of the coronary arteries, which increases the likelihood of graft occlusion. Another possibility is that multiple coronary interventions affect the natural blood flow in patients with collateral vessels, leading to microinfarctions in the myocardium. Patients who receive PCI with stenting typically have significant atherosclerosis, though not as advanced as those requiring surgical treatment. However, in the event of PCI failure, these patients require surgical revascularization, by which point their atherosclerosis may be more advanced and widespread. Additionally, stents have been linked to endothelial irregularities, which may lead to adverse cardiovascular outcomes, complicating coronary anastomosis and potentially impacting graft patency.15

Over the past two decades, several observational studies have shown a relationship between preoperative anemia and postoperative adverse outcomes.16 Anemia is clinically significant, and its prevalence increases in patients undergoing heart surgery. Patients undergoing cardiac surgery may be more susceptible to the effects of preoperative anemia, since these patients have limited cardiac reserves, while surgery itself and CPB has been associated with significant blood loss and hemodilution.17 Low hemoglobin levels has been linked to higher mortality and morbidity in older adults, as well as in patients with congestive heart failure and coronary artery disease, especially during acute coronary syndrome. These groups of patients have high perioperative risk due to their existing comorbidities, and the presence of severe coronary artery disease may exacerbate id. CABG patients are particularly sensitive to the effect of low hemoglobin levels due to their limited coronary reserves.1820

Obesity is well-known as a risk factor for developing cardiovascular diseases, and patients with a high BMI are frequently seen among those undergoing heart surgery (77% of the total patients).21 However, it is suggested that overweight and obese patients undergoing heart surgery had a better survival advantage compared to patients with a BMI < 18.5 kg/m2, normal weight patients, and morbidly obese patients.22 The term “obesity paradox” reflects the relationship between obesity, compared to normal weight, and decreased mortality rates. This paradoxical relationship has been demonstrated in diabetes, end-stage renal disease, hypertension, congestive heart failure, coronary artery disease (CAD), and peripheral artery disease.23,24 Excess adipose tissue provides benefits during illness and stress. Individuals with low BMI lack the reserves to tolerate the effects of weight loss, as seen in obese patients, contributing significantly to mortality.2527 A BMI < 18.5 kg/m2 may result from malnutrition and cachexia or comorbid conditions. It may also indicate underlying serious illnesses. Moderate obesity provides a protective effect in patients undergoing heart surgery and in the general population.28

Left main coronary artery disease (LMCAD) has been associated with poor outcomes of cardiac surgery procedures. This is largely because the left main coronary artery supplies a major portion of the heart muscle. Blockage in this artery may severely reduce blood flow to a large area of the heart, especially the left ventricle, which is responsible for pumping oxygen-rich blood throughout the body.29 The critical nature of this artery’s function in supplying a large portion of the heart underscores why LMCAD is linked to increased morbidity and mortality in cardiac surgery patients.30

It is suggested that the mortality rate in patients with right ventricular dysfunction is 3.5 to 4.2 times higher compared to patients normal right ventricle.31 The tricuspid annular plane systolic excursion (TAPSE) is a crucial echocardiographic parameter that plays a significant role in assessing cardiac function. TAPSE is a measure of the longitudinal motion of the tricuspid annulus towards the apex of the heart during systole, reflecting right ventricular systolic function.32 In the context of cardiac surgery, TAPSE serves as an essential indicator of right ventricular performance and can provide valuable insights into postoperative outcomes and potential complications. TAPSE assessment can offer insights into the impact of surgical interventions on right ventricular function and overall cardiac performance.33

Functional tricuspid regurgitation is often secondary to pulmonary hypertension, which causes right ventricular remodeling, tricuspid annulus dilation, papillary muscle displacement, and tricuspid valve leaflet tethering. The progression of tricuspid regurgitation is linked to a specific pattern of right ventricular remodeling, most prominently at the mid-ventricular level, with minimal changes in ventricular length accompanied by increased right ventricular sphericity. Persistent progression of tricuspid regurgitation is a sign of poor outcomes. In patients with pulmonary arterial hypertension hospitalized for right-sided congestive heart failure, the degree of tricuspid regurgitation is an independent predictor of poor outcomes.34

This study utilized a multicenter setting from four tertiary hospitals in Indonesia that serve as tertiary referral centers for cardiac surgery and have relatively adequate postoperative intensive care. This resulted in an adequate sample size while minimizing the influence of non-patient factors during and after surgery that could affect postoperative outcomes. However, limitations of this study should be considered. It only analyzed variables or conditions with a high incidence in the sampled population, potentially missing other variables that may impact mortality outcome but were not included in the new scoring model. The study also excludes patients with aortic dissection. This exclusion was necessary because the number of patients with aortic dissection was very small, and including them would obscure the influence of other candidate variables with higher incidence rates. Thus, the results of this study cannot be extrapolated to patients with aortic dissection. The study uses retrospective data, which complicates data control. Prospective data collection was not feasible due to the time and effort required. Additionally, the study included data from only four hospitals, raising the question of whether the results can be applied to other hospitals. The included hospitals have different characteristics and are expected to represent other hospitals in Indonesia. Nevertheless, further studies with prospective data samples are needed to validate the results of this study.

Conclusion

The new scoring model, consisting of 13 variables, has relatively accurate discrimination and calibration performances, making it potentially applicable to open-heart surgery patients in Indonesia. External validation of the new scoring model is necessary before it can be applied as a prediction score for cardiac surgery patients in Indonesia. Further study using prospective data and evaluating mortality rates over a longer period, such as 30-day, 90-day, and 1-5 year mortality, is recommended.

Declaration

Ethics and consent

Ethical approval was obtained from the Medical and Health Research Ethics Committee of the Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia on October 30, 2019 (No: KE/FK/1277/EC/2019). The ethics committee waived the informed consent requirement for this study due to its retrospective design based on patient medical records.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 10 Oct 2024
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Kurniawaty J, Setianto BY, Supomo S et al. The development of novel score model as a predictor of in-hospital mortality of adult cardiac surgery patients in Indonesia [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:1204 (https://doi.org/10.12688/f1000research.155017.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 10 Oct 2024
Views
5
Cite
Reviewer Report 27 Nov 2024
Amr Arafat, Tanta University, Tanta, Egypt;  Ministry of Defense Health Services, Riyadh, Saudi Arabia 
Approved with Reservations
VIEWS 5
The article “The Development of Novel Score Model as a Predictor of In-hospital Mortality of Adult Cardiac Surgery Patients in Indonesia" presents a study aimed at developing a novel scoring model to predict in-hospital mortality for adult cardiac surgery patients ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Arafat A. Reviewer Report For: The development of novel score model as a predictor of in-hospital mortality of adult cardiac surgery patients in Indonesia [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:1204 (https://doi.org/10.5256/f1000research.170132.r340392)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 10 Oct 2024
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.