The development of novel score model as a predictor of in-hospital mortality of adult cardiac surgery patients in Indonesia

Introduction

The trend of cardiac surgery is showing an increase in Indonesia. Cardiac surgery procedures have shown an increase from year to year in Indonesia.¹ Cardiac surgery has been regarded as one of the most expensive surgical procedures in a hospital and is associated with high mortality and morbidity rates.² To minimize perioperative mortality, patient selection has been advocated, using risk assessment systems developed over the last decades. Decision-making for surgery requires more than just clinical experience and intuition. This decision-making process is highly complex, and scoring systems were developed as a way to objectively assess patient conditions, determine patient mortality risk profiles, and improve patient outcomes.³

Several risk stratification models have been developed for cardiac surgery patients. The three most commonly used scoring systems are the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II,⁴ Society of Thoracic Surgeons (STS) score,² and Parsonnet score.⁵ EuroSCORE II has been validated in various population settings and remains the standard for many institutions.⁶ Among the various scoring systems, EuroSCORE II is the most frequently studied in Indonesia. Results from several studies in Indonesia have found that EuroSCORE II shows good discriminatory but weak calibration performance.^7–9

Existing risk models shows suboptimal performance in Indonesia population. These models generally predict outcomes more accurately in the settings where they were initially developed. Socio-economic conditions, living standards, health resources, and geographic and ethnic characteristics may affect the applicability of these models in different regions.¹⁰ In clinical practice, most predictive scores show lower accuracy when validated on new population. The two common possible explanations for this are that the predictive scores were not adequately developed as well as major differences between the populations in which they were developed and validated.¹¹

To this date, there is no known risk stratification model that has been developed from a local database or with characteristics that align with the Indonesia population. The main issue that we faced in Indonesia is the lack of a widely applicable scoring system for assessing in-hospital mortality risk in cardiac surgery patients. We propose to develop a mortality risk model for cardiac surgery patients from local database derived from the Indonesian population.

Methods

We conducted a retrospective study using medical record data from adult cardiac surgery patients who underwent procedures at four tertiary referral hospitals in Indonesia (Dr. Sardjito General Hospital in Yogyakarta, Harapan Kita National Heart Center Hospital in Jakarta, Dr. Kariadi General Hospital in Semarang, and Abdoel Wahab Sjahranie Regional Hospital in Samarinda). The subjects were adult patients (aged 18 to 85 years) who underwent cardiac surgery procedures between January 1, 2016, and December 31, 2020. Heart transplant patients, aortic dissection patients, and patients with incomplete primary medical records were excluded from the study. It was estimated that between 75 and 3500 subjects were required for each potential predictor. Thus, a minimum of 3500 subjects were required for this study. This study complies with the Declaration of Helsinki and received ethical approval from the Medical and Health Research Ethics Committee of the Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia on October 30, 2019 (No: KE/FK/1277/EC/2019)

The selection of potential predictors and definitions for each variable was conducted through a focused group discussion. Selected candidate variables were based on variables present in EuroSCORE II, CARE, Parsonnet, STS scores. We also considered other risk factors that not included in existing scores but proven or considered strongly associated with post-cardiac surgery mortality risk, and potentially suitable as candidate variables in model development.

The statistical analysis began with bivariate analysis using t-tests for normally distributed numeric variables; otherwise, Mann-Whitney tests were performed. Chi-square tests were performed for categorical variables. Candidate variables with p < 0.25 or clinically significant associations that cannot be ignored, such as age and gender, and occurrence rates exceeding 1% of the total sample size, were included in multivariate analysis. Multivariate analysis was performed using logistic regression with backward elimination method. Models with significant variables underwent bootstrapping. Odds ratios (OR) and 95% confidence intervals (CI) were calculated from bootstrapped coefficients. ROC (receiver operating characteristic) curves were constructed for each model, and the AUC (area under the curve) values were assessed and compared to determine the discriminative performance of each model. The final model is the best fit model for in-hospital mortality. Model calibration was tested using the Hosmer-Lemeshow test, comparing the model’s predicted mortality with actual mortality. A risk prediction model with a p-value > 0.05 is considered well-calibrated. The analysis was performed using STATA 15 software.

Results

A total of 4,940 samples from 4 participating hospitals met the initial inclusion criteria. Of these, 65 subjects were excluded (53 patients with aortic dissection and 12 patients with incomplete primary outcome data), resulting in a final sample size of 4,875 patients (Figure 1).

Figure 1. Study subjects flow diagram.

Compared to the EuroSCORE II database, our population was younger, with median age of 50.4 years, with higher proportion of female patients (36.9 %), and more patients had NYHA class I-II,. The in-hospital mortality rates for cardiac surgery patients at Dr. Sardjito General Hospital, Dr. Kariadi General Hospital, Abdoel Wahab Regional Hospital, and Harapan Kita National Heart Center were 8.6% (74 patients), 8.7% (113 patients), 9.3% (30 patients), and 4.18% (101 patients), respectively. These translated into overall mortality rate of 6.5%, which is higher than the mortality rate in the EuroSCORE II database. The demographic characteristics of the study subjects are shown in Table 1.

Of the 62 variables included in bivariate analysis, 36 variables had p < 0.25. The L-backward multivariable logistic regression analysis was performed and 19 variables were retained. These 19 variables were used as base model for our model building process.

After L-backward multivariable logistic regression analysis was performed, we further selected variables with p < 0.05 or with high β coefficient Model selection was based on AUCs of ROC curves and AIC values. The final model consisted of 13 variables (Table 2).

Our final model had an AUC of 0.7564 for the ROC curve (Figure 2), indicating good discriminative performance (AUC > 0.7). Goodness-of-fit test with the Hosmer-Lemeshow (HL) test was used to evaluate the calibration of the model. The p-value for the HL test of the new scoring model in this study was p = 0.9510, indicating good calibration (Table 3).

Figure 2. ROC curve of the final model.

Discussion

The purpose of this study was to develop a model that can be widely applied across various hospitals in Indonesia and integrated into local data collection and management systems. From this study, we proposed the following logistic regression formula intended for predicting in-hospital mortality for cardiac surgery patients:

Where e = 2.718, β₀ (constant of the logistic regression equation) = -6.424, β_i is the coefficient of variable X_i, and so forth: 1.501 (history of PCI), 1.473 (history of cardiac surgery), 1.286 (severe musculoskeletal and neurological disorders), 1.225 (Hemoglobin < 12 mg/dL), 0.987 (BMI < 18.5 kg/m²), 0.928 (two or more procedures), 0.814 (LVEF ≤ 30%), 0.711 (CPB), 0.701 (left main disease), 0.443 (moderate to severe tricuspid regurgitation), 0.412 (TAPSE < 17 mm), 0.352 (creatinine clearance < 50 ml/min), 0.038 (age). Several variables in the new scoring model were not present in EuroSCORE II, these include history of PCI, hemoglobin < 12 g/dL, BMI < 18.5 kg/m², left main disease, TAPSE < 17 mm, and moderate to severe tricuspid regurgitation.

In a retrospective study of 6,504 CABG patients, PCI was an independent predictor for in-hospital mortality (3.6% in patients with PCI vs 2.3% in patients without PCI).¹² Another study also found PCI as predictor of in-hospital mortality and major adverse cardiac events (MACEs), with the negative impact being more pronounced in certain groups, such as patients with diabetes mellitus and triple-vessel coronary artery disease (CAD 3 VD).¹³ In contrast, a multi-center study of 34,316 CABG patients from 16 institutions found no significant association between history of previous PCI and mortality rates, though it did observe higher rates of major complications and longer hospital stays in the PCI group.¹⁴ The history of percutaneous coronary intervention (PCI) is associated with a poor prognosis in cardiac surgery patients due to various factors. The presence of a stent necessitates the surgeon to perform anastomosis on more distal and thinner branches of the coronary arteries, which increases the likelihood of graft occlusion. Another possibility is that multiple coronary interventions affect the natural blood flow in patients with collateral vessels, leading to microinfarctions in the myocardium. Patients who receive PCI with stenting typically have significant atherosclerosis, though not as advanced as those requiring surgical treatment. However, in the event of PCI failure, these patients require surgical revascularization, by which point their atherosclerosis may be more advanced and widespread. Additionally, stents have been linked to endothelial irregularities, which may lead to adverse cardiovascular outcomes, complicating coronary anastomosis and potentially impacting graft patency.¹⁵

Over the past two decades, several observational studies have shown a relationship between preoperative anemia and postoperative adverse outcomes.¹⁶ Anemia is clinically significant, and its prevalence increases in patients undergoing heart surgery. Patients undergoing cardiac surgery may be more susceptible to the effects of preoperative anemia, since these patients have limited cardiac reserves, while surgery itself and CPB has been associated with significant blood loss and hemodilution.¹⁷ Low hemoglobin levels has been linked to higher mortality and morbidity in older adults, as well as in patients with congestive heart failure and coronary artery disease, especially during acute coronary syndrome. These groups of patients have high perioperative risk due to their existing comorbidities, and the presence of severe coronary artery disease may exacerbate id. CABG patients are particularly sensitive to the effect of low hemoglobin levels due to their limited coronary reserves.^18–20

Obesity is well-known as a risk factor for developing cardiovascular diseases, and patients with a high BMI are frequently seen among those undergoing heart surgery (77% of the total patients).²¹ However, it is suggested that overweight and obese patients undergoing heart surgery had a better survival advantage compared to patients with a BMI < 18.5 kg/m², normal weight patients, and morbidly obese patients.²² The term “obesity paradox” reflects the relationship between obesity, compared to normal weight, and decreased mortality rates. This paradoxical relationship has been demonstrated in diabetes, end-stage renal disease, hypertension, congestive heart failure, coronary artery disease (CAD), and peripheral artery disease.^23,24 Excess adipose tissue provides benefits during illness and stress. Individuals with low BMI lack the reserves to tolerate the effects of weight loss, as seen in obese patients, contributing significantly to mortality.^25–27 A BMI < 18.5 kg/m² may result from malnutrition and cachexia or comorbid conditions. It may also indicate underlying serious illnesses. Moderate obesity provides a protective effect in patients undergoing heart surgery and in the general population.²⁸

Left main coronary artery disease (LMCAD) has been associated with poor outcomes of cardiac surgery procedures. This is largely because the left main coronary artery supplies a major portion of the heart muscle. Blockage in this artery may severely reduce blood flow to a large area of the heart, especially the left ventricle, which is responsible for pumping oxygen-rich blood throughout the body.²⁹ The critical nature of this artery’s function in supplying a large portion of the heart underscores why LMCAD is linked to increased morbidity and mortality in cardiac surgery patients.³⁰

It is suggested that the mortality rate in patients with right ventricular dysfunction is 3.5 to 4.2 times higher compared to patients normal right ventricle.³¹ The tricuspid annular plane systolic excursion (TAPSE) is a crucial echocardiographic parameter that plays a significant role in assessing cardiac function. TAPSE is a measure of the longitudinal motion of the tricuspid annulus towards the apex of the heart during systole, reflecting right ventricular systolic function.³² In the context of cardiac surgery, TAPSE serves as an essential indicator of right ventricular performance and can provide valuable insights into postoperative outcomes and potential complications. TAPSE assessment can offer insights into the impact of surgical interventions on right ventricular function and overall cardiac performance.³³

Functional tricuspid regurgitation is often secondary to pulmonary hypertension, which causes right ventricular remodeling, tricuspid annulus dilation, papillary muscle displacement, and tricuspid valve leaflet tethering. The progression of tricuspid regurgitation is linked to a specific pattern of right ventricular remodeling, most prominently at the mid-ventricular level, with minimal changes in ventricular length accompanied by increased right ventricular sphericity. Persistent progression of tricuspid regurgitation is a sign of poor outcomes. In patients with pulmonary arterial hypertension hospitalized for right-sided congestive heart failure, the degree of tricuspid regurgitation is an independent predictor of poor outcomes.³⁴

This study utilized a multicenter setting from four tertiary hospitals in Indonesia that serve as tertiary referral centers for cardiac surgery and have relatively adequate postoperative intensive care. This resulted in an adequate sample size while minimizing the influence of non-patient factors during and after surgery that could affect postoperative outcomes. However, limitations of this study should be considered. It only analyzed variables or conditions with a high incidence in the sampled population, potentially missing other variables that may impact mortality outcome but were not included in the new scoring model. The study also excludes patients with aortic dissection. This exclusion was necessary because the number of patients with aortic dissection was very small, and including them would obscure the influence of other candidate variables with higher incidence rates. Thus, the results of this study cannot be extrapolated to patients with aortic dissection. The study uses retrospective data, which complicates data control. Prospective data collection was not feasible due to the time and effort required. Additionally, the study included data from only four hospitals, raising the question of whether the results can be applied to other hospitals. The included hospitals have different characteristics and are expected to represent other hospitals in Indonesia. Nevertheless, further studies with prospective data samples are needed to validate the results of this study.

Conclusion

The new scoring model, consisting of 13 variables, has relatively accurate discrimination and calibration performances, making it potentially applicable to open-heart surgery patients in Indonesia. External validation of the new scoring model is necessary before it can be applied as a prediction score for cardiac surgery patients in Indonesia. Further study using prospective data and evaluating mortality rates over a longer period, such as 30-day, 90-day, and 1-5 year mortality, is recommended.

Declaration