Age, neutrophil lymphocyte ratio, and radiographic assessment of the quantity of lung edema (RALE) score to predict in-hospital mortality in COVID-19 patients: a retrospective study

Background: Available data suggest that case fatality rate of COVID-19 patients in Surabaya is higher than global cases. Thus, it is important to identify risk factors to prevent the mortality. This study aimed to assess the factors associated with hospital mortality of COVID-19 patients, and develop a prediction score based on these findings. Methods: We analyzed 111 patients, who were diagnosed with COVID-19 based on reverse-transcriptase polymerase chain reaction. The following patient characteristics were obtained from records: age, gender, type of symptoms, onset of symptoms, neutrophil lymphocyte ratio (NLR), absolute lymphocyte count, chest x-ray abnormalities, lung involvement, type of lesion, radiographic assessment of the quantity of lung edema (RALE) score, and mortality. Data were analyzed using SPSS 25.0. Results Multivariate analysis showed that age >50 years ( p=0.043), NLR score >5.8 ( p=0.016) and RALE score >2 ( p=0.002) can predict the mortality of COVID-19 patients in the hospital. ROC curve analysis of the score ability to predict mortality showed an area under the curve of 0.794. The cut-off point is 4.5, with a sensitivity of 96.7% and specificity of 49.4% to predict the mortality of COVID-19 patient in the hospital. Conclusions Age, NLR score and RALE score were associated with mortality of COVID-19 patients in the hospital and might be used as a predictor for mortality of COVID-19 patients in health care centre where radiologists are available. The prediction score may be useful for frontline physicians to effectively manage patients with a higher score to prevent mortality.


Introduction
Cases of pneumonia of unknown etiology were first reported in the city of Wuhan, China, at the end of December 2019.After identification, the etiology of these cases was a new type of coronavirus (Wang et al., 2020).The World Health Organization officially called this coronavirus disease 2019   (WHO, 2020).COVID-19 is caused by severe acute respiratory syndrome coronavirus-2 (SARS-Co-V2) virus (Gorbalenya et al., 2020).
Since December 2019, more than 20 million people have been diagnosed with COVID-19, and more than 700,000 have died (Worldometers, 2020).In Indonesia, 128,776 people have been diagnosed with COVID-19, and 83,710 have died since March 2020 (Gugus Tugas Percepatan Penanganan COVID-19, 2020).The global case fatality rate (CFR) of COVID-19 in August was 3.7%.In Indonesia, the CFR is slightly higher (4.5%).Meanwhile, the CFR in Surabaya (the second biggest city in Indonesia) is almost twice that of Indonesia for the same period (8.9%).Several factors may associate with the higher CFR such as low testing, low contact tracing and poor COVID-19 register (Sorci et al., 2020).However, finding characteristics and risk factors to predict mortality is of utmost importance in order to prevent the mortality.It is also important to develop a prediction score to assess patients during early stages, when patients receive treatment at tertiary hospitals.
Several studies have found risk factors related to COVID-19 mortality (Galloway et al., 2020;� Gupta et al., 2020;� Jalili et al., 2020;� Lippi & Plebani, 2020;� Zhao et al., 2020).However, currently, there is no study that analyzes risk factors for COVID-19 mortality in hospitals for the Indonesian population.Existing prediction scores also based on complex laboratory findings, which are less feasible to be used in low resource health centers.Therefore, the purpose of this study was to assess the demographical characteristic, laboratory and radiological findings associated with COVID-19 mortality in hospital patients and develop a prediction score based on these findings.

Study design and setting
This was a retrospective study that was conducted in the Emergency Department of Airlangga University Teaching Hospital.We included all the patients who fulfilled the inclusion and exclusion criteria, who were admitted for COVID-19 from March 13, 2020 to May 15, 2020.A total of 111 patients met the criteria and included in this study.
We collected data regarding clinical symptoms and the date of onset of symptoms from emergency department medical records.

Patients
The inclusion criteria for this retrospective study were: (1) patients with a chief complaint of one COVID-19-related symptom, including: fever, dry cough, tiredness, aches and pains, nasal congestion, headache, conjunctivitis, sore throat, diarrhea, loss of taste or smell, a rash on skin, or discoloration of fingers or toes (WHO, 2020);� (2) confirmed SARS-CoV-2 infection by reverse-transcriptase polymerase chain reaction (RT PCR) using nasopharyngeal and oropharyngeal specimens;� (3) patients who underwent chest x-ray (CXR) on the day of admission to the hospital.
All patients underwent CXR and laboratory examination on the day of admission to the emergency department.

Subgrouping of patients
Patients were divided into two groups based on the outcome of the patient: group I (discharged with negative results for SARS-CoV-2 infection by RT-PCR test);� and group II (died).

Data collection
The following patient characteristics were obtained from the medical records: age, gender, type of symptoms, onset of symptoms, neutrophil lymphocyte ratio (NLR), absolute lymphocyte count (ALC), CRX abnormalities, lung involvement, type of lesion, radiographic assessment of the quantity of lung edema (RALE) score, and mortality.
Image acquisition and evaluation.Radiology data was collected from the radiology department.All the patients underwent an anteroposterior projection chest radiography at full inspiration where possible.The results were reviewed by two

Amendments from Version 1
Abstract: we have added the sentence "might be used as a predictor for mortality of COVID-19 patients in health care centre where radiologists are available" on the conclusions section.
Introduction: we have added the sentences "Several factors may associate with the higher CFR such as low testing, low contact tracing and poor COVID-19 register (Sorci et al., 2020).However, finding characteristics and risk factors to predict mortality is of utmost importance in order to prevent the mortality.It is also important to develop a prediction score to assess patients during early stages, when patients receive treatment at tertiary hospitals" on the second paragraph and "to assess the demographical characteristic, laboratory and radiological findings associated with COVID-19 mortality in hospital patients" on the last paragraph.
Discussion: we have added the sentences "Another study in Indonesia found a similar result where the average age of COVID-19 patients who died is 58.2 years old (Rozaliyani et al., 2020)" and "This study also showed that the mortality risk is three times higher in people age ≥50 years.This finding is similar to a study in the United States, which found that the mortality risk of COVID-19 was 8.1 times higher among those ≥55 years compared with individuals ages <54 years (Yanez et al., 2020)."On the second paragraph.We also added term "RALE score" on the sixth paragraph.
Conclusions: we have added the sentence "It may be used as a predictor for mortality of COVID-19 patients in the health care centre where radiologists are available".
Any further responses from the reviewers can be found at the end of the article radiologists (A.D.S, a radiologist with 25 years of experience and E.S, a radiologist with four years of experience) based on consensus.
CXRs were evaluated for the presence of pulmonary alterations, type of pulmonary alterations, and their distribution.CXR alterations that were found specifically in COVID-19 patients were defined according to the Fleischner Society's nomenclature, available in the Glossary of Terms for Thoracic Imaging (Hansell et al., 2008), as follows: -Reticular alteration: a reticular pattern is a collection of innumerable small linear opacities that, by summation, produce an appearance resembling a net -Consolidation: as a homogeneous increase in pulmonary parenchymal attenuation that obscures the margins of vessels and airway walls -Ground-glass opacity (GGO): an area of hazy increased lung opacity, usually extensive, within which margins of pulmonary vessels may be indistinct The distribution of pulmonary alterations was classified as lung involvement unilateral (right/left) or bilateral.Other features, such as pleural effusion, were also recorded.
The scores for each lung were summed to produce the final RALE score.

Statistical analysis
Statistical analysis was performed using IBM SPSS Statistics Version 25.0.Continuous variables were expressed as mean ± standard deviation values.The frequency of symptoms, laboratory findings and CXR findings was shown as the number of incidence and percentage per cluster of groups.
The correlation between the patient characteristic, symptoms, laboratory findings and CXR findings with the outcome was analyzed by logistic regression.We also conducted the Hosmer-Lemeshow test to evaluate the goodness of fit of the scoring model and conducted receiver operating characteristic (ROC) analysis to evaluate the sensitivity and specificity of the model.

Ethical considerations
This study received ethical approval from the Ethical Committee of Airlangga University Teaching Hospital (approval number 147/KEP/2020).Consent from the participants was waived by the committee.

Patient characteristics
A total of 111 COVID-19 patients were evaluated.In total, 72.9% (n=81) of patients were discharged from the hospital, while 27.1% (30) patients died during their hospitalization.There were 45 men (48.6%) and 47 women (51.4%).Average patient age was 51±14.2 years old.The mean age for the patients who died was higher (55±12.8years) compared to the patients who were discharged (48±14.8years).Fever (32.6%) and shortness of breath (26%) were the most frequent symptoms for all patients.However, among the patients who died, shortness of breath was the most frequent symptom (40.7%).The mean NLLR score is higher on the patient who died (10.1 ±10.5) compared to patients who were discharged (5.3 ±5.4).
Based on the odds ratio (OR) value of each variable, the mortality risk for patients who were of an older age is 2.787 times higher than patients who were of a younger age.A higher NLR score has a 3.246 times higher mortality risk than patients who have lower NLR score.Mortality risk for patients who have a higher RALE score is 6.826 times higher than patients who have lower RALE score.It should be noted that three patients died while showing no abnormalities on their CXR.

Prediction of mortality
We form a scoring model based on the OR of each variable, which can be seen in Table 4.If the variable were present, we gave the value of "1", and if absent we gave the value of "0".We also performed the Hosmer-Lemeshow test and concluded that the model is fit (p=0.802).Based on our scoring model, each score from 111 patients was calculated.The percentage of mortality was then calculated for each score.
Table 5 shows that high mortality (>60%) was seen for a total score of 13, while low mortality (<10%) was seen for a total score ≤3.
The total score was then analyzed with ROC analysis to predict the sensitivity and specificity for the probability of mortality of COVID-19 patients in the hospital.Figure 1 shows that based on the ROC curve, the area under the curve (AUC) of the score is 0.794.The score has a cut off point of 4.5, with a sensitivity of 96.7% and specificity of 49.4%, to predict the mortality of COVID-19 patients in the hospital.et al., 2020).The younger average age of the patients who died might be due to the co-morbidities that also develop in the relatively younger age of the Indonesian population, especially for type 2 diabetes (Arifin et al., 2019).This study also showed that the mortality risk is three times higher in people age ≥50 years.This finding is similar to a study in the United States, which found that the mortality risk of COVID-19 was 8.1 times higher among those ≥55 years old compared with individuals ages <54 years (Yanez et al., 2020).
Based on our findings, there is no correlation between gender and patient mortality.The result is different from studies in the United States and Iran, which conclude that men has a higher risk of death than women (Gupta et al., 2020;� Jalili et al., 2020).This could be due to the characteristic of the co-morbidities of the Indonesian population, which affect both men and women at an almost similar number (Kemenkes, 2019).We also found that there is no correlation between type and onset of symptoms to the outcome of the patients.
There is a correlation between NLR score and the patient outcome.Previously, it was concluded that higher NLR score related to the severe illness of the patients and could be an early identification for patient assessment in the early stage for intensive care unit (Liu et al., 2020;� Yang et al., 2020).Our study showed that the higher NLR score (>5.8) would increase the risk of mortality by three times.In contrast, we found no correlation between ALC score and patient mortality in the hospital.However, various other research shows that low ALC score is related to disease severity for patients admitted to the hospital for COVID-19 (Wagner et al., 2020) and a higher risk of mortality (Lippi & Plebani, 2020).
This study showed that there is no correlation for CXR abnormalities, lung involvement and type of lesion to COVID-19 mortality in the hospital.Interestingly, we also found that three patients (10%) who died showed normal CXR.This could be due to the lower sensitivity of CXR to detect lung lesions compare to CT scan.However, it also shows that COVID-19 could become severe rapidly even though patients have normal CXR on the day of the admission.
Interestingly, we found that RALE score is correlated with patient mortality, and a higher score (>2) will increase mortality risk by seven times.This finding is consistent with another study in the United Kingdom with a larger sample size, which showed that higher radiological severity score (>3) increased the incidence of critical admission or death (Galloway et al., 2020).RALE score in CXR could become a key predictor of mortality because of the simplicity of modality and is broadly available across health care provider.
The prediction score developed in this study demonstrated good accuracy (AUC of 79.4) to predict the discharged probability of COVID-19 patients in the hospital with only three  parameters.Another score showed good accuracy (AUC of 83) with more complicated parameters, such as health failure, procalcitonin, lactate dehydrogenase, chronic obstructive pulmonary disease, pulse oxygen saturation, heart rate, and age (Zhao et al., 2020).This suggests that our score can be used in a broader setting such as locations with low resource of health care service.

Limitations
This study only analyzes patient characteristics, and laboratory and radiology findings related to COVID-19 mortality in the hospital.We do not evaluate the co-morbidities of the patient and the hospital setting, such as the location of the treatment where the patient died (emergency room, intensive care, low care), or the resource (facilities, human resource, intervention).

Conclusions
Age, NLR score and RALE score were associated with mortality of COVID-19 patients in our hospital setting in an Indonesian population.It may be used as a predictor for mortality of COVID-19 patients in the health care centre where radiologists are available.The prediction score may be useful for physicians to determine the mortality risk of a patient with COVID-19.

Introduction:
My comment: To reduce the mortality rate in hospitals in Surabaya is not by finding characteristics and risk factors to predict mortality.As I know at that time tracing in Surabaya still low in subjects, so the CFR in this city is still higher than Jakarta and the mean CFR in Indonesia.
"the purpose of this study was to assess the factors associated with COVID-19 mortality in hospital patients and develop a prediction score based on these findings."My comment: To assess the factors associated with mortality even to develop prediction score should use all factors.But this study actually only uses some parameters not include commorbidities.

Discussion:
The statement: " …….this finding is similar to a study in the US, which found that an older age (> 60 years) is associated with a higher risk of death is not so accurate, because:" 1.

My comments:
The difference between mean ages of >50 and >60 is so significant.1.
Criteria for elder people in Indonesia is >60 years old.

2.
The younger average of the patients in this study is not representative of Indonesia Covid patients.

My comment:
This statement is over confident and is in contradiction with the statement: "This study showed that there is no correlation for CXR abnormalities……" 1.

Conclusion:
The statement: "It may be used as a predictor for mortality of COVID 19 patients in low health care resource setting." 1.
My comment : It is not so true.The RALE score should be made by radiologists who have many experiences, so it only applicable in a referral hospital setting. 1.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound?Yes Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?Are all the source data underlying the results available to ensure full reproducibility?Partly Are the conclusions drawn adequately supported by the results?Yes Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Pulmonary medicine I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Author Response 21 Dec 2020 Anggraini Dwi Sensusiati, Airlangga University Teaching Hospital, Surabaya, Indonesia The authors thank the Honorable Reviewer 2 for the excellent review of the paper and indepth suggestions made to improve the quality of the paper.Followings are the responses to the suggestions of the Honorable Reviewer.The valuable suggestions and corrections are incorporated cautiously, and the paper is made clearer while revising the manuscript.

Introduction:
Comment: To reduce the mortality rate in some hospitals in Surabaya is not by finding characteristics and risk factors to predict mortality.As I know at that time tracing in Surabaya still low in subjects, so the CFR in this city is still higher than Jakarta and the mean CFR in Indonesia.*We acknowledged the possible cause of high fatality rate and added the sentence "Several factors may associate with the higher CFR such as low testing, low contact tracing and poor COVID-19 register (Sorci et al., 2020).However, finding characteristics and risk factors to predict mortality is of utmost importance in order to prevent the mortality." "the purpose of this study was to assess the factors associated with COVID-19 mortality in hospital patients and develop a prediction score based on these findings."Comment: To assess the factors associated with mortality even to develop prediction score should use all factors.But this study actually only uses some parameters not include commorbidities.*We acknowledged the study limitation which did not evaluate the commorbidities in the limitations section.We focused the research on the demographical characteristic, laboratory and radiological findings associated with COVID-19 mortality in hospital patients and develop a prediction score based on these findings.

Discussion:
The statement: " …….this finding is similar to a in the US, which found that an older age (> 60 years) is associated with a higher risk of death is not so accurate, because:" Comments: The difference between mean ages of >50 and >60 is so significant.1.
Criteria for elder people in Indonesia is >60 years old.2.
The younger average of the patients in this study is not representative of Indonesia Covid patients.

3.
*We found similar finding in another research in Indonesia where the average age of COVID-19 patients who died is 58.2 years old (Rozaliyani et al., 2020).We also found that this finding is similar to a study in the United States, which found that the mortality risk of COVID-19 was 8.1 times higher among those ≥55 years compared with individuals ages <54 years (Yanez et al., 2020).
The statement : "CXR could become a key predictor of mortality because………."Comment: This statement is over confident and is in contradiction with the statement: "This study showed that there is no correlation for CXR abnormalities……" 1.
*We meant that the RALE score from CXR might become a key predictor

Conclusion:
The statement: "It may be used as a predictor for mortality of COVID 19 patients in low health care resource setting."Comment: It is not so true.The RALE score should be made by radiologists who have many experiences, so it only applicable in a referral hospital setting.

1.
*We changed the "low health care resource setting" into "health care centre where radiologists are available." Although in this paper there is no correlation for CXR abnormalities, lung involvement and type of lesion to COVID-19 mortality.However, I think all the above CXR changes would also contribute collectively to the RALE score.
Overall, well done and congratulation to the author.Just a minor error in spelling.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound?Yes

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate?Yes Are all the source data underlying the results available to ensure full reproducibility?Yes

Are the conclusions drawn adequately supported by the results? Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: General Radiology, Paediatric Radiology I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com

Figure 1 .
Figure 1.ROC curve analysis for the scoring model to predict the mortality in COVID-19 patients.

Table 1 .
Patient characteristics.Risk factors of mortalityFrom Table2, no significant relationship was observed between the outcome of the patients based on gender, type of symptoms, onset of symptoms, ALC score, x-ray abnormalities, and type of lesion.This suggested that those variables have no predictive value for mortality outcome for COVID-19 patients.There is a significant relationship between age, NLR score, lung involvement and RALE score towards the outcome of the patients (p-value <0.05).All of these variables were analyzed using logistic regression, which revealed there are three variables with p<0.05: age, NLR score and RALE score (Table

Table 2 . Risk factors of COVID-19 hospital mortality.
DiscussionThis is the first study analyze the radiologic and laboratory findings of COVID-19 patients in Indonesia.The main result of our study was that age, NLR score and RALE score have a strong relationship for mortality risk of COVID-19 patients in the hospital.This study found that the average age of COVID-19 patients who died is 55 ±12.8 years.Another study in Indonesia found a similar result where the average age of COVID-19 patients who died is 58.2 years old(Rozaliyani et al., 2020).This finding is younger than a study in Iraq, who found that the average age of COVID-19 patients who died in hospital is 67.49± 15.28 years (Jalili