Identification of Biomarkers for Severity in COVID-19 Through Comparative Analysis of Five Machine Learning Algoritms

1.1 Research question

What are the laboratory biomarkers to predict the severity of SARS-CoV-2 infection in patients from southeastern Mexico?

2.1 Programming language details

The Python programming language was used for all calculations. For the classification models, the sci-kit learn suite was used in all cases.

Auxiliary libraries:

2.2 Dataset details

This study includes patients diagnosed with COVID-19 according to the World Health Organization (WHO) guidelines for the clinical management of severe acute respiratory infections due to SARS-CoV-2,²⁴ who were admitted to the Intensive Care Unit (ICU) at the “Dr. Desiderio G. Rosado Carbajal” general hospital in the time period between April 1^st and July 31^st, 2021 in Comalcalco Tabasco, Mexico. The ethical and legal guidelines considered for sampling are described in De la Cruz-Cano et al.²⁵

The dataset has 138 entries that correspond to patients and 60 columns that represent different features. The features encompass a wide range of data, including demographic information, laboratory test results, vital signs, symptoms, pre-existing conditions, and the severity of the COVID-19 illness divided into three classes that are considered mild, moderate and severe.

2.3 Dataset characterizationc

Next, the study population contained in the dataset used is characterized. In the dataset, there is a greater number of male patients (84) compared to female patients (54). It is also observed that the average age of the patients is approximately 59.9 years, with an age range that goes from 41 to 82 years, and a median of 58.

The severity of COVID-19 presents a significant proportion of patients (44.93%) with moderate severity of the disease (count of 62 and proportion of 0.4493), while mild and severe cases constitute 20.29% (count of 28 and proportion of 0.2029) and 34.78% (count of 48 and proportion of 0.3478) of the total respectively. Based on these data, it can be seen that the dataset is moderately unbalanced. Regarding comorbidities, presented in Table 1, the most prevalent is Heart Disease (Heart disease) with 97.10%, followed by Chronic Kidney Disease (CKD) with 94.20%. On the other hand, the least prevalent is Obesity, presented in only 34.06% of patients.

Table 2 shows that the most common symptom among patients is Diarrhea, present in 94.93% of cases, while the least common symptom is Fever, reported only in 8.70% of patients.

2.4.1 Experimental design

The research strategy focused on two primary objectives. The first objective was to identify the most effective algorithm for analyzing the dataset, and to conduct a correlation analysis of the variables to provide a suitable context. This was illustrated in Figure X (referenced as Figure 1). The second objective was to conduct a thorough analysis of various optimization methods to identify the key variables for a simplified model. This model should maintain a minimum reduction in its predictive capabilities while providing insight on the importance of each feature. The methodology used to achieve these objectives is explained in detail below:

Figure 1. Experimental process and algorithms used to predict the severity of COVID-19.

2.4.2 Data processing

The dataset was processed to contain a subset of 40 feature columns. The subset used in this work was generated from the first set by removing several features not directly related to the biomarkers. All variables were normalized, putting them on the same scale to facilitate analysis. The exclusion of certain variables is justified due to the analysis's focus on biomarkers and their importance in detecting the severity of COVID-19, which are features that can be objectively measured and that provide an assessment of a health condition.

2.4.3 Evaluation of algorithms and metrics

Five classification algorithms were evaluated and compared: Random Forest, Support Vector Machine, Logistic Regression, Decision Tree and Neural Network (Multi Layer Perceptron or MLP). To compare the performance of the models, evaluation metrics such as precision, recall, F1 score, and accuracy were used. The test used to discern the robustness of the methods was stratified five-segment cross-validation. During subsequent variable reduction experiments, an average of these metrics was employed as objective functions for the metaheuristic search algorithms.

2.4.4 Model selection

The selection of the optimal model depends on the analysis of variables and the analysis of variable reduction that provide indications of the importance of the biomarkers. Given that there is an imbalance in the dataset, the models that will be selected for these analyzes will be those that are balanced by weights (or SMOTE in the case of the MLP). This ensures that uneven distribution of data is taken into account and more reliable and representative results are provided.

2.4.5 Correlation analysis

Through correlation analysis, the variables (biological markers) associated with the severity of COVID-19 (COVID19_Severity) were identified. The correlated variables or biological markers are represented in a Table and a heat map of correlations.

2.4.6 Variable reduction

Heuristic experiments were carried out to progressively reduce the variables and the model performance was evaluated for each set of selected variables to obtain a model with a reduced number of variables without significantly compromising accuracy. In addition, variable selection experiments were performed using the following algorithms: genetic, simulated annealing, and crow search. These methods introduced variety to the models, ensuring that parameters were selected equitably to achieve similar time durations.

2.4.7 Overfitting analysis

To evaluate the possibility of overfitting in the generated models, difference tests between the training and validation sets were implemented through learning curves. The curves, produced with the learning_curve() function from the sklearn library, illustrate the performance of the model when varying the size of the training set. A threshold of 0.05 (or 5%) was set for the difference between training and validation scores. If the difference exceeded this threshold, the model was considered overfitted. This methodology facilitated the identification of feature sets that provided an optimal balance between accuracy and generalization., illustrate the performance of the model when varying the size of the training set. A threshold of 0.05 (or 5%) was set for the difference between training and validation scores. If the difference exceeded this threshold, the model was considered overfitted. This methodology facilitated the identification of feature sets that provided an optimal balance between accuracy and generalization.

3.1 Algorithm performance

The results in Table 3 provide a comparative view of the performance of machine learning algorithms based on various metrics.

This Table shows the basic metrics of the algorithms. It can be seen that all algorithms exhibit solid performance, with the precision, recall, F1-score, and ROC AUC value of each algorithm all exceeding 0.9. The Random Forest stands out as the best algorithm, which reaches the maximum value of 1.0000 in all metrics, with the exception of the cross-validation score, which is 0.9727. It's worth noting that, natively, Gradient Boosting (including XGBoost) and k Nearest Neighbors (kNN) do not allow balancing of classes and therefore they have not been considered in the comparison with the purpose of evaluating the models under equal conditions, in the same way, all the models have been initialized in random_state=42).

Table 4 shows the accuracy, recall and F1 scores for each class of each algorithm. Random Forest achieves scores of 1,000 in all classes, but other algorithms, such as SVM and Logistic Regression, also show high performance in several classes, reaching scores of 1,000 in several metrics. The results reported for each metric with respect to the Random Forest algorithm select it as the best algorithm and it is subsequently used for the variable reduction experiments and other comparisons used.

3.2 Correlation analysis

The characterization analysis of the variables contained in the subset of data under study and the correlation analysis are presented below. The features are ordered by correlation. Table 5 contains the averages and range of all features, the correlation value and the Pearson P value with interpretation to provide context of the dataset.

Additionally, a bar graph of these correlations is shown in Figure 2.

Figure 2. Correlation Bar Chart (The first chart presents correlations and the second presents absolute correlations).

In the original heat map, correlations clusters are distinguished that are characterized by variables with magnitudes greater than or close to 0.5. These magnitudes suggest a potential impact on the subsequent phase of feature selection. In particular, in the upper left corner of the map, a grouping stands out where variables such as ferritin and oxygen saturation present an absolute correlation greater than 0.5 with each other. Closely, fibrinogen shows a correlation of 0.47. Likewise, in the lower corner of the map, a strong correlation is observed between the variables Na, Creatinine, Urea and Cl. In addition, in relation to these, the variables ALT and AST also stand out.

Figure 3, presents a refined version of this heatmap. In it, significant correlations with the target variable are evident (> 0.45). It can be deduced from this heat map that these correlations should, with high probability, be the ones selected in the variable reduction procedures.

Figure 3. Heat map of significant correlations (>0.45) between variables related to the target variable (correlations with magnitude greater than 0.5 are highlighted in red and bold).

3.3 Feature Selection

Four different approaches were used for feature selection: heuristic, genetic algorithm, simulated annealing algorithm and crow search algorithm. The two best models for each approach were identified (ranked based on cross-validation and number of variables). In the case of the heuristic approach, 3 and 4 variables were considered for the models. Once these models were found, it was decided that for metaheuristic approaches, the models would be encouraged to produce 3- and 4-feature solutions. The metaheuristic search algorithms were run ten times each and the two best models in each case were evaluated and compared at the end.

3.4 Heuristic Method

Several progressively generated models were trained by adding the features that were found to have statistical significance during the correlation analysis. Below is a brief description of them for context.

As mentioned before, a limited set of variables within such a list (see heatmap in Figure 3) are expected to be displayed during variable selection. The models were trained progressively and the graphical results can be seen in Figure 4.

Figure 4. Vertical curves present the first appearance of a model with the best F1-Score (Green) or the best cross-validation accuracy (Magenta).

The values of the models represented in said graph are found in Table 6.

During the training experiment with progressive addition of variables, 39 models were trained. The intermediate models, 20 to 38, have been omitted from Table 6 since their metrics were identical or inferior to subsequent models and the added variables did not show statistical significance during the correlation analysis. From this Table, procalcitonin and AST are highlighted as variables that increase the performance of the models; models were generated and analyzed with the progressive addition of these features. These models are referred to in this research simply as “Model 1” and “Model 2”.

In Table 7, a redistribution of the weights of the variables between Model 1 and Model 2 is observed. The inclusion of the variable AST in Model 2 entails a redistribution of the weight, especially with a notable reduction in the weight of procalcitonine from 0.2235 to 0.1140. However, despite the lower correlation of AST with severity (-0.2116), the inclusion of AST in Model 2 appears to contribute to a marginal improvement in the cross-validation accuracy of 0.9546 compared to 0.9636 for Model 1.

It is notable that both models achieve perfect accuracy and perfect F1-score, indicating their effectiveness in classification. The main difference between them lies in their performance in cross-validation, where Model 2 outperforms Model 1 with an increase of approximately 0.007 when including the variable LDH. This suggests that the inclusion of LDH may improve the robustness of the model and its generalization ability.

Despite the marginal improvement, the decrease in cross-validation metrics in Model 2 compared to Model 1 suggests that the inclusion of the AST variable could be adding additional complexity without necessarily improving the generalizability of the model. The apparently unequal distribution of weights in the second model compared to the first could be a sign of overfitting.

A joint superficial analysis of Tables 7 and 8, as well as the behavior of the metrics between models in Figure 4, suggests that although the Models 1 and 2 show excellence in their metrics, with a precision, F1-score and AUC of $1.0000$, this perfect performance could indicate a possible overfitting to the training data. To this end, subsequent evaluations are based on cross-validation as an indicator of performance and generalization (see Figure 6).

3.5 Metaheuristic method 1 (Genetic algorithm)

The algorithm used as an optimization function a combination of all the scores on average, so that it found the model that best satisfied all the metrics at the same time. The specifications of the genetic algorithm are as follows:

Individual Representation: Binary string of length equal to the number of features.

Aptitude Function: Average precision, recall, accuracy and F1 score.

Crossing: Two point crossing.

Mutation: Bit flip mutation with probability 0.05.

Selection: Tournament selection with size 3.

Population Size: 50.

Number of Generations: 20.

Hall of Fame: Best individual through all generations.

The performance of the best models found with the metaheuristic method is shown below in Table 9.

3.6 Metaheuristic method 2 (Simulated annealing algorithm)

This algorithm used a similar objective function as the previous algorithm. The specifications of the simulated annealing algorithm are as follows:

Individual Representation: Binary string of length equal to the number of features.

Power Function: Negative of the average of the F1 scores obtained through cross-validation.

Movement: Selects and deselects features randomly until the total of selected features equals 4.

State Initialization: Random binary string with exactly 4 selected features.

Initial temperature: 10.0.

Final Temperature: 0.01.

Number of Iterations: 150.

The performance of the best models found with metaheuristic method 2 is shown below in Table 10.

3.7 Metaheuristic method 3 (Crow search algorithm)

This algorithm used a similar objective function as the previous algorithms. The specifications of the crow search algorithm are as follows:

Individual Representation: Binary string of length equal to the number of features.

Aptitude Function: Average of cross-validation score and F1 score.

Crow Movement: If the probability of consciousness is less than 0.2, it moves towards the target, otherwise it moves randomly.

Mutation: Flip bit with probability 0.2 (probability of consciousness).

Penalty: Penalizes solutions with less than 3 features or 5 or more features.

Population Size: 50.

Number of Iterations: 30.

Hall of Fame: Better individual through all iterations.

The performance of the best models found with metaheuristic method 3 is shown below in Table 11.

3.8 Comparison of reduced models

Tables 12 and 13 show the effectiveness and weighting of the features of the models under study (AUC and Specificity values are omitted as they are identical in all cases and equal to 1.0000).

Certain clear patterns were observed in feature selection and its impact on model performance.

In Table 13, the models M3, M4, and M5, which have a cross-validation (CV) score of 0.9727, you can see the distribution of assigned weights of 0.5650 (M3), 0.4826 (M4) and 0.4920 (M5) respectively, for the Ferritin feature. This corroborates the importance of the variable in predicting the severity of COVID-19. In this table it can be seen that the ferritine feature is repeated in all models, and its weight is considerable in those with the best CV Scores (Models M3, M4, and M5), reinforcing its relevance in prediction. The variable sat. O2 is also selected consistently across almost all models, suggesting its predictive value, although its weight in the models varies, which could have implications for unbalancing the performance of each model.

Features such as urea, creatinine, LDH, ALT and procalcitonine are selected in certain models, with variations in their weights. This could suggest that its importance may fluctuate depending on the context of the other features in the model. Although all models achieve high accuracy, CV Scores vary, suggesting differences in their ability to generalize to new data. Models M3, M4, and M5 stand out and tie with the CV Score of 0.9727, an indicator of better generalization ability than the rest of the models generated in the list.

On the other hand, models M7 and M8 have the lowest CV Scores (0.9545 and 0.9455, respectively), which could indicate a lower robustness in generalization or even overfitting (see Figure 5 in the overfitting analysis). We can relate these results to the weights assigned to their features and the selection method used. In the model M4, the additional features creatinine and ALT are included with weights of 0.07377 and 0.1113 respectively. In contrast, model M3 uses urea and Cl with weights of 0.2660 and 0.1690. Despite these changes, the CV score remains the same at $0.9727$ for both models.

Figure 5. Cross-validation is performed for each model, along with the methods employed for variable selection.

Any model suspected of overfitting is represented with a grid.

Model M5, which also has a CV score of 0.9727, includes LDH and ALT with weights of 0.06770 and 0.1069, respectively, instead of creatinine which is found in the model M4. Despite this change, the weights assigned to ferritin and O2 saturation are very similar to those of the M4 model. A small difference of 0.006070 is shown when LDH is replaced by Creatinine. Models M4, M2 and M6 have CV scores of 0.9636. Despite sharing ferritin and O2 saturation with comparable weights with models M3, M4 and M5, these models also include procalcitonin, AST and fibrinogen with weights of 0.2235 (M1), 0.1191 (M2) and 0.1175 (M6) respectively.

Models M7 and M8 have the lowest CV scores (0.9545 and 0.9455) and include urea and Na with weights of 0.1019 and 0.09156. Although urea is in the best model and both urea and Na are highly correlated between the variables with statistical significance and with others, its performance is lower than that of other models. One possible reason is the interaction with the oxygen saturation variable, which unbalances the weight distribution, which could lead to overfitting.

A better qualitative comparison perspective and the importance of features can be appreciated in Table 14. In this table, the distribution of features in the models can be seen, where it can be observed that ferritine is present in all prediction models, due to its high correlation with the target variable and its high Pearson correlation. Following ferritine, oxygen saturation is present in almost all models, except the third one (which, according to Table 13, has a reasonable weight distribution and a high CV Score, so it could be possible to predict the disease severity without necessarily using oxygen saturation). Both features (ferritin and oxygen saturation) are considered as main features because they are present in all or almost all models, and this observation coincides with the calculated correlation for these features with the target feature. In addition, for the rest of the features, the complementary behavior of them with respect to the most frequent features can be seen. It is believed that those "secondary features" can be implicitly selected under diversity and cross-correlation criteria. It does not seem, in the first instance, to have any preferential distribution of secondary features, as the frequency values for these features fluctuate between 1 and 3 appearances. Being the secondary features most frequently appearing the LDH and ALT, whose target variable correlations are not necessarily the highest, but their interpretation with the p-value is deemed as "significant" (see Table 5).

3.9 Overfitting analysis

An overfitting analysis was performed to evaluate the ability of the Random Forest model to generalize to new data. The overfitting analysis was carried out using two complementary approaches.

Learning curve: Learning curves are generated using code:

train_sizes, train_scores, test_scores = learning_curve(estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_sizes)

In this code snippet, the learning_curve() function creates a set of training and validation scores for different training set sizes. An overfitted model will show a large discrepancy between training and validation scores, especially for larger training sets. In Figure 6 you can see these curves (the red one represents validation and the green one represents training).

Figure 6. The validation and training curves appear close and within the threshold in almost all cases except in model 8.

Difference between training and validation scores: The overfitting test is performed using the following code:

if (train_scores_mean[-1] - test_scores_mean[-1]) > 0.05:  overfitting = "Yes"else:  overfitting = "No"

This code snippet calculates the difference between the mean training and validation scores for the largest training set size. If this difference is greater than 0.05, the model is considered to be overfitting.

The training and cross-validation results that were used to determine if the model is overfitted are presented in Table 15.

3.10 Complementary analysis

The perplexity parameters of the best models are presented below in Table 16 and models 9, 10 and 11 have been added, generated progressively and reporting the biomarkers Ferritin + Oxygen saturation + Fibrinogen because they are the three with the highest correlation with the target variable. In addition, model 12 has been added, which contains all the variables. The specificity is the average of the three classes for this case.

A minimal difference between models can be noted in this table. One of the models from the heuristic algorithm presents the least perplexity of the set, closely followed by one of the best models according to the CV score. The one that is considered the best algorithm in Table 15 does not have the slightest measure of perplexity. Even so, in second place is one of the best models found in said table, which is also the only one of those at the top of the table that has a perfect Train Score. These results could indicate that by reducing model variables, there is a possibility of a slight degradation in performance, which could go unnoticed compared to classical accuracy tests.

This particular measure was adopted with a pragmatic consideration, since the probability distribution could be used experimentally as an indicator of the improvement or worsening of the condition in the absence of data of a temporal nature. This is because this distribution exhibits the proximity and similarity of a sample between different categories. It is suggested that perplexity can play a significant role as a metric to perform a differential analysis between the most prominent models.

The model highlighted in the table in question (determined by its CV score and its lower perplexity) stands out for a choice of biomarkers of a varied nature, which could suggest greater robustness of the model itself.

4.1 Importance of variables

This study supports the conclusions presented by Gharib et al.,⁷ which emphasizes the relevance of negative correlations with oxygen saturation and positive correlations with ferritin when investigating biomarkers of inflammation related to the severity of COVID-19. Furthermore, it highlights the importance of biomarkers such as IL-6, LDH, CRP and D-dimer. The correlations with the variables mentioned in the study by Khan et al8., are also corroborated during the analysis phase in this study, especially regarding ferritin (highly significant), IL-6 (highly significant) and CRP (slightly significant). Similarly, the study conducted in Alexandria by Abdelhalim et al10. found that 71.4% of non-hospitalized patients had elevated levels of serum ferritin, reaffirming its relevance as a biomarker in the diagnosis of COVID-19.

The results obtained could indicate that, beyond the individual impact that each variable may have, the interaction between various biomarkers could be of especially significant importance. Most of the models investigated integrate fundamental features, such as ferritin levels and oxygen saturation, along with other secondary features, such as fibrinogen, sodium (Na) levels, creatinine, urea, chloride (Cl), as well as the levels of the ALT and AST enzymes. In the differential analysis of perplexity, the three outstanding models were identified, and it was observed that the model that presented the best performance (determined by its highest score in the CV Score and its lowest perplexity) uses variables that reveal liver damage (represented by ALT), kidney damage (evidenced by creatinine), biomarkers that denote immunoinflammatory alterations (such as ferritine), as well as the balance in oxygenation (sat.02). These observations could suggest that feature diversity could play an important role in decreasing model perplexity and granularly increasing overall prediction performance. Within the focus of the study, it has not been found whether this importance of diversity has already been reported previously.

A point to highlight is the indication of Samprathi et al.¹³ on the non-recommendation of serum ferritin to monitor treatment response, despite its prominence in predictions. Additionally, a comprehensive review¹¹ identified lymphopenia, thrombocytopenia, interleukin-6, ferritin, among other biomarkers, as associated with severe and fatal cases of COVID-19. In particular, hyperferritinemia was highlighted as an indicator of systemic inflammation and a poor prognosis.

These findings, together with the identification of ferritin as an independent predictor of COVID-19 mortality in the study by Sibtain et al.,¹² highlight the need to continue investigating and evaluating the function and predictive value of these biomarkers in the disease using the interaction between variables as a framework.

4.2 Machine learning

This work is similar to the approach used in the work of Ahmed et al.¹⁵ However, there are fundamental differences in methodology. One of these is the deliberate exclusion of models that do not easily admit balance, as is the case of kNN. This choice was made despite the fact that many studies, including the recently mentioned,¹⁵ included it among their compared methods, along with Random Forest, Support Vector Machine, Logistic Regression and Decision Tree, such as the study by¹⁴ which used Random Forest and AdaBoost.

The current work has prioritized a preselection of biomarkers based on the Pearson p-value, facilitating the reduction of the search space for the metaheuristic algorithms that were used for subsequent feature selection. This preselection is consistent with the study by Elif et al.,¹⁷ which identified significant biomarkers with a similar method, agreeing with this study in biomarkers such as LDH, high leukocyte count and C-reactive protein. The models presented in said study achieved an accuracy of 0.9796 and an AUC value of 0.959, it can be suggested that the high prediction values may indeed be related to the selection of statistically significant features, given that this study reports exceptional metrics and AUC. A relevant aspect is that in the current work, unlike,²⁶ the data used during the correlation analysis have been used for subsequent variable reduction experiments. In other similar works^26,15 the 5-fold stratified cross-validation, considered fundamental in the analysis of this work to evaluate the performance of the models, has not been implemented. In the work of Patterson et al.¹⁸ A 10-fold CV approach is used which appears to be quite effective. This approach is feasible in this work given the magnitude of the dataset (n=225).

It is suggested that data balancing could present a significant improvement in model construction. Xiong et al.,²⁶ carried out an analysis that is similar to the approach of this study. Despite working with a more extensive dataset (n=287) and having a notable diversity in features, they do not mention a specific balancing process. From the analysis of its results, the Random Forest model stood out with an AUC of 0.970, while the SVM model recorded the highest precision, with 88.5%. However, despite its extension, our study, with a sample of n=138, delves deeper and expands on the features of said data, in addition to using balancing techniques. For their part, Gok et al.¹⁷ worked with a larger dataset (n=863), which was balanced using the SMOTE technique. Of the 8 trained models, Random Forest had the best performance, achieving an accuracy of 0.98, similar to the results of our study. A key difference between the work of Gok et al.¹⁷ and ours lies in the nature of the variables used for prediction. While Gok et al.¹⁷ they did not use laboratory results that contained biomarkers such as IL-6, LDH or Ferritin, in this study they have been essential in the analysis. The latter can make prediction difficult because the biomarkers are not very specific, which highlights the high precision of the models generated in said study. It is vital to highlight that the study by Gok et al.¹⁷ emphasized how data preprocessing and balancing positively influence the accuracy and robustness of the model.

It should be noted that although Xiong et al.²⁶ does not detail its balancing process, other studies, such as that of Wang et al.,²⁷ have adopted the SMOTE technique successfully. Specifically, Wang et al.²⁷ they achieved an exceptional precision of 0.9905 and an AUC value of 0.846. Given the performance results of the models, this study has recognized RandomForest as one of the best classifiers, aligning with observations in.^15,26 It is relevant to mention that this work has obtained an AUC value that slightly exceeds that reported by,¹⁷ but as the dataset is smaller, more tests must be performed before reaching definitive conclusions. When looking at other metrics, such as accuracy, this work has achieved comparable results to studies such as,¹⁴ which achieved an accuracy of 0.94 and an F1 Score of 0.86, and with an accuracy¹⁹ of 84.2 % and an AUC value of 0.874 for the first group (age ≤ 70) and 0.842 for the second (age ≥ 70).

Finally, while studies such as that of Iwendi et al.,¹⁴ which highlighted a correlation between gender and mortality, and that of Cui et al.,¹⁹ focused on the differences in biomarkers according to age, have provided unique perspectives, the present work has focused on validating and comparing techniques based on clinical analyzes with the aim of identifying the most appropriate approach to predict outcomes in patients with COVID-19. Future work could include the participation of these variables in the models.

4.3 Variable reduction

Feature selection has notably benefited from the use of metaheuristic algorithms, which have shown consistent improvements over traditional heuristic techniques, especially in high complexity areas such as COVID-19 severity prediction. In this work, we sought to use diverse algorithms inspired by evolution, physical mechanisms and collective intelligence, as mentioned in the work of Agrawa et al.²⁸ It should be noted that the algorithms applied were quite simple, trying to preserve the minimum version of each one to encourage equality of conditions, so they cannot be directly comparable with those of other works that present cumulative improvements said algorithms.^{20,29,21,22,23} Relevant differences with said works will be discussed below.

The genetic algorithm has proven effective in several contexts, including the work of Kabir et al.,²⁰ which introduced a redundancy reduction approach. In our study and in that of Hayet et al.,²⁹ the genetic algorithms provided remarkable results, although the choice of appropriate objective functions could have improved the performance. Focusing on the detection of COVID-19, the study by Hayet et al.,²⁹ highlights the importance of specific variables, such as CRP, Respiratory Rate, Oxygen Saturation, and LDH. The consistency in the selection of these variables between different investigations highlights their relevance, although differences have also been observed, such as the elimination of certain variables due to insufficient quality of the data.

Simulated Annealing, analyzed by Guha et al.,²¹ has proven to be useful for multidimensional problems. This work mentions a hybridization with a recently developed optimization algorithm called equilibrium optimization (Equilibrium Optimizer) which, according to said work, is capable of working during the exploitation phase to obtain better results than just simple simulated annealing, for this it is considered that simulated annealing works as a local preseeker. Additional mention is made of the work of Bandyopadhyay et al.,²³ where a two-stage pipeline was proposed for the detection of COVID-19 in radiological images. They used a DenseNet-based CNN for feature extraction and combined the Harris Hawks Optimization (HHO) algorithm with Simulated Annealing (SA) and chaotic initialization for selection. The study achieved an accuracy of 98.85% and reduced the number of features by 75%.

Despite efforts to use crow search, highlighted by Chen et al.,²² in our work, this algorithm showed limitations, including the potential for overfitting. In this study, the search is improved using a hierarchical approach that reports favorable results. In all previous works, an improvement in the results can be seen by hybridizing the algorithms with pre-optimization during the search phase that allows a better exploitation phase.