Keywords
Health care quality, effect, COVID-19. CPG, Clinical Practice Guidelines.
-
Views
-
Downloads
How to cite this article
Ramírez-Morera A, Salazar-Vargas J, Rivera-Chavarría AL and Urrútia G. Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review. [version 1; peer review: 1 approved]. F1000Research 2023, 12:1501 (https://doi.org/10.12688/f1000research.142098.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Systematic Review
Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review.
[version 1; peer review: 1 approved]
PUBLISHED 24 Nov 2023
OPEN PEER REVIEW
REVIEWER STATUS
Abstract
Corresponding authors:
Anggie Ramírez-Morera, Jordan Salazar-Vargas
Competing interests:
No competing interests were disclosed.
Grant information:
Access to databases, full-text articles, and other resources was provided by the IHCAI Foundation. It is important to note that the funders played no role in study design, data collection and analysis, publication decisions, or manuscript preparation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Copyright:
© 2023 Ramírez-Morera A et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Ramírez-Morera A, Salazar-Vargas J, Rivera-Chavarría AL and Urrútia G. Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review. [version 1; peer review: 1 approved]. F1000Research 2023, 12:1501 (https://doi.org/10.12688/f1000research.142098.1)
First published: 24 Nov 2023, 12:1501 (https://doi.org/10.12688/f1000research.142098.1)
Latest published: 24 Nov 2023, 12:1501 (https://doi.org/10.12688/f1000research.142098.1)
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Introduction
In the 1990s, the advent of Evidence-Based Clinical Practice Guidelines (EB-CPGs) signified an enhancement regarding patient care and influenced healthcare professionals’ decisions regarding interventions (Weisz et al., 2007; IOM, 1990).
As per the definition, EB-GPC involves declarations containing recommendations aimed at enhancing the uniformity and quality of healthcare (Mallonee et al., 2006; Woolf et al., 1999; Kwan, 2004). Moreover, they support informed choices for both patients and clinical professionals (IOM, 2011; Alonso-Coello et al., 2010).
Globally, substantial efforts have been directed towards developing and implementing EB-CPGs grounded in scientific research. The concept based on evidence denotes that the recommended interventions within the guidelines arise through rigorous, transparent, and impartial methodologies, extracting the highest level of scientific evidence to enhance clinical care (Linskey, 2010; Lim et al., 2008).
This research constitutes our third publication (Ramírez-Morera et al., 2019, 2022) and is part of a series of systematic reviews (SR) spanning diverse healthcare dimensions, all aimed at assessing the effectiveness of employing EB-CPGs. In this study, we will specifically focus on the management of COVID-19 and investigate the influence of using EB-CPGs to enhance clinical quality care across patient outcomes, structural, and process domains (Donabedian Model, Donabedian, 1988), specifically in COVID-19 management.
In healthcare, the term structure pertains to human resources, continuous education programmes, infrastructure, organisational setup (arrangement of healthcare personnel), and material inputs (budget and equipment) necessary to provide clinical care. This is related to ensuring access across levels of complexity (IOM, 1999; Donabedian, 1988).
It focuses on the manner and location in which clinical care is delivered, encompassing diagnostic tests, interventions such as surgeries, or other actions taken during patient treatment. The outcomes it measures are related to the clinician’s skill and expertise in addressing a condition, including prevention, detection, diagnosis, and monitoring (Dimick, 2010).
The patient’s health outcomes are the effects that the healthcare system produces through its interaction with the patient, or the progress achieved (Dimick, 2010). It also signifies the results generated in the patient’s evolution (Donabedian, 1988). Additionally, they encompass reports or assessments of the patient’s general condition or satisfaction (IOM, 1999).
The effect of EB-CPGs has been evaluated by other international studies, albeit focusing on the process domain, as they prioritise reviewing clinical practices and adherence to guidelines (Lugtenberg et al., 2009), reporting these as improvements in the care process. Nevertheless, there hasn’t been enough emphasis on studying how the use of EB-CPGs influences the dimensions of structure and patient outcomes.
In this systematic review, our focus is directed towards the COVID-19 disease due to its unexpected emergence, which has resulted in profound repercussions across various aspects (Enríquez & Sáenz, 2021). On the 30th day of January 2020, the WHO formally announced COVID-19 as a global public health urgency. This classification underlines the widespread diffusion of the outbreak across numerous nations, continents, or even globally and its notable impact on a substantial populace. Considering the increasing transmission and severity rates and the reality that a significant portion of the population lacks immunity to this novel virus, it was classified as a pandemic on the 11th of March 2020 (PAHO/WHO, 2020).
While the impact of COVID-19 was initially perceived as a health crisis, it has transcended beyond the realm of healthcare, extending to all aspects of social life and development. Its influence has reached a global scale, causing severe social, economic, and political repercussions. As the limitations of healthcare systems became evident, the world endeavoured to curb the contagion by first closing borders and subsequently shuttering economies. These measures incurred substantial economic and social costs (Dos Santos et al., 2023; Enríquez & Sáenz, 2021).
As of April 30, 2023, the global tally revealed, tragically, 7 million lives lost and in confirmed cases 766 million or more. However, prevailing trends in reported cases likely mask the true extent of infections and reinfections worldwide, a phenomenon corroborated by prevalence surveys. This discrepancy may partially arise from reduced diagnostic testing and reporting delays in several nations. Furthermore, the global response has been marked by the administration of an impressive 13.3 billion COVID-19 vaccine doses (PAHO/WHO, 2023).
This review holds importance due to the growing number of EB-CPGs developed in diverse subjects. The scientific production of studies and guidelines for COVID-19 management experienced an exponential increase compared to public health non-emergency periods, reflecting the global focus on seeking potential solutions (Albornoz et al., 2020; Guzmán-Useche & Rodríguez-Contreras, 2020). Evaluating the tangible impact on relevant outcomes of novel interventions or hypotheses remains imperative, avoiding being swayed solely by good intentions, hope, or the distress of finding something that demonstrates any benefit.
Few systematic reviews assess the influence of EB-CPGs on enhancing healthcare; these often concentrate solely on adherence, a specific domain, or clinical entity and typically pertain to a single country or region (Lugtenberg et al., 2009; Worrall et al., 1997; Grimshaw & Russell, 1993). This SR review fulfils the need for evidence synthesis on the utilisation of EB-CPGs, assessing the comprehensive quality of healthcare provision. It underscores the necessity for a systematic review that definitively illustrates the practical influence of evidence-based recommendations on COVID-19.
Methods
Our systematic review was developed by applying the Donabedian model, encompassing the domains of results, process, and structure to assess the impact on enhancing the quality of healthcare using EB-CPGs (Donabedian, 1988). The research was conducted in accordance with the Cochrane Handbook methodology guidelines, as outlined by Higgins et al. (2022a).
PROSPERO registration: CRD42022354708.
Study search
To guide the systematic search for studies and the selection process, we structured the research question using the PICO format (as detailed in Table 1). Our methodology combines the search of free-text words and controlled vocabulary, carefully chosen.
Table 1. Clinical question following PICO format.
We conducted a systematic search across numerous digital repositories to identify primary research articles. These databases comprised CINAHL, EMBASE, Academic Search Complete, Scopus, Biomedical Reference Collection: Psychology, Behavioral and Comprehensive Sciences Collection, Academic Edition of Nursing, Cochrane Methodology Register, Health Collection of Nursing and Allied: SPORTDiscus, & Comprehensive, Cochrane Database SR, APA PsycInfo, Sports Medicine & Rehabilitation Source, Health of Global, Alt HealthWatch, Consumer Health Complete, Biomedical Reference Collection: Basic, Pubmed, International Pharmaceutical Abstracts, MasterFILE Premier, AgeLine, AMED - The Allied and Complementary Medicine Database, HTA Database OVID and LILACS. Furthermore, inquiries were carried out within repositories of the Social Sciences Citation Index and Science Citation Index to identify documents referencing the studies included in this investigation.
We replicated the search strategy used in PubMed in other databases, adjusting the relevant controlled vocabulary as necessary. Additionally, we sought grey literature through manual scrutiny of impactful journals not previously reviewed, as well as from various specialized sources in this field.
We reached out to the authors of preselected articles for potential inclusion, contacting them to pose additional questions about their published work or any unpublished material. To identify potentially relevant studies, we engaged with researchers from other effective practice studies within the professional field. Our search covered studies published between January 2019 and May 2023 and was not restricted by language. Detailed information about our comprehensive, advanced search strategy and the findings can be found in Supplement N° 1 of extended data (Ramírez-Morera et al., 2023).
We utilized the web application Sciwheel Generator and Manager of Reference to handle the bibliographic references of the identified articles (Sciwheel, 2022).
Studies selection
The articles identified in the systematic search underwent evaluation by three reviewers (AR, ALR, JS). In cases of disagreement over selection, consensus was reached through discussion. Inclusion criteria encompassed: 1. Randomized Clinical Trials (RCTs) or cluster-type RCTs investigating the effects of implementing any form of EB-CPGs compared to non-use or alternative approaches. 2. RCTs analysing the impact on Donabedian model-defined domains (patient outcomes, process, and structure) when employing EB-CPGs for COVID-19 management. 3. Studies published between 2019 and 2023. 4. Language restrictions were not applied.
Data extraction
Data extraction was independently performed by three authors (ARL, JS, AR) using a modified electronic data sheet based on the data collection and verification list outlined authored from the EPOC Group of Cochrane (EPOC, 2017).
We appraised the risk of bias employing the approaches defined in the Guidebook for SR of Cochrane, specifically in chapters 8, 10, and 23 (Higgins et al., 2022a; Sterne et al., 2019). To assess bias arising from various systematic errors in the RCTs, we employed the RoB2 tool (Sterne et al., 2019) and adhered to the criteria outlined by Cochrane (Higgins et al., 2022b).
All RCTs meeting the eligibility criteria for this study underwent independent assessment by three authors (ALR, JS, AR), and any discrepancies were resolved through deliberation. No cluster-type RCTs were found; as a result, the Cochrane Handbook procedures in section 23 (Higgins et al., 2022c) or the RoB2 tool for cluster-type RCT assessment (Sterne et al., 2019) were not employed.
The evidence quality was assessed utilizing the GRADE method for all outcomes (Schünemann et al., 2013). The assessment ranges from very low, low, moderate, to high and was carried out using the GRADEpro digital platform (GRADEpro, 2021).
Version 5.4 of the software called Review Manager was employed for the analysis of the data (RevMan, 2020). We adapted the predefined templates provided by the system to extract information and present results in a more straightforward manner.
A meta-analysis was not conducted due to the significant variability within the effect measures found in the randomized controlled trials that were included. Consequently, statistical heterogeneity was not assessed.
Results
Identification and selection of studies
In PRISMA flowchart (Figure 1), the process for obtaining studies through the systematic search conducted is detailed (Page et al., 2021). We obtained 14,220 studies through database searches, and 26 studies were discovered through additional sources. After removing 8,774 duplicate records, we excluded 5,286 studies after reading the title and abstract. We conducted a complete text review of 160 articles, excluding 145 references that did not fulfil the criteria of the selection: one article was not a randomised controlled trial; 27 (19%) were clinical protocols that were analysed but did not assess an EB-CPG; 117 (81%) involved a single intervention appraised but did not evaluate an EB-CPG, and one involved assessing a CPG not related to Covid-19 disease.
Figure 1. Selection process for the included studies in PRISMA flowchart (PRISMA, Page et al., 2021).
* Supplement N° 1 displays the quantity of records retrieved from each searched database or registry.
** The authors excluded all records. After analysed by title and abstract, they were excluded.
*** The records were not RCTs. **** A clinical protocol was assessed. Did not evaluate an EB-CPG.
***** A single intervention was assessed. Did not evaluate an EB-CPG. ****** Assessment of a CPG not related to COVID-19 disease.
Supplement N° 2 provides supplementary information regarding the studies that were not included and the criteria of exclusion (Ramírez-Morera et al., 2023).
Characteristics of studies
We incorporated 15 randomized controlled trials examining EB-CPGs on COVID-19 (Barratt-Due et al., 2021; Ben Abdallah et al., 2023; Chen et al., 2023; Engelen et al., 2022; Gold et al., 2021; Gyselinck et al., 2022; Hamidi-Alamdari et al., 2021; Liesenborghs et al., 2021; Montejano et al., 2023; Rahman et al., 2022; Rauch et al., 2021; Sadeghi et al., 2023; Scholz et al., 2023; Suppan et al., 2020; Zurita-Cruz et al., 2022). Supplement N° 3 contains supplementary information following screening and assessment of the complete text (Ramírez-Morera et al., 2023).
The studies were primarily published in 2021 and 2023, each accounting for 33%. Articles predominantly originated from European countries (60%), Asia (20%), America (13.3%), and Africa (6.7%). Regarding the guideline scope and the outcome category of the included EB-CPGs in the selected studies, 11 (73%) focused on management and 9 (60%) on treatment, respectively (Table 2).
Table 2. Details for all the studies included in the systematic review (n=15).
We outline the included studies’ characteristics (n=15) in Table 2. For a more comprehensive exposition of these features, refer to Supplement N° 4, which contains detailed information provided as supplementary material (Ramírez-Morera et al., 2023). Supplement N° 5 displays the guidelines that were evaluated in the included articles (Ramírez-Morera et al., 2023).
In 14 studies, EB-CPGs interventions were used targeting adult populations (Barratt-Due et al., 2021; Ben Abdallah et al., 2023; Chen et al., 2023; Engelen et al., 2022; Gold et al., 2021; Gyselinck et al., 2022; Hamidi-Alamdari et al., 2021; Liesenborghs et al., 2021; Montejano et al., 2023; Rahman et al., 2022; Rauch et al., 2021; Sadeghi et al., 2023; Scholz et al., 2023; Suppan et al., 2020), and one study was directed towards pediatric population management (Zurita-Cruz et al., 2022).
Regarding the seriousness of COVID-19, one study was conducted in people with lower intensity COVID-19 cases (Gold et al., 2021), while ten studies focused on cases of moderate severity (Barratt-Due et al., 2021; Ben Abdallah et al., 2023; Chen et al., 2023; Engelen et al., 2022; Gyselinck et al., 2022; Hamidi-Alamdari et al., 2021; Liesenborghs et al., 2021; Rahman et al., 2022; Suppan et al., 2020, Zurita-Cruz et al., 2022). Furthermore, four studies specifically addressed individuals with more severe clinical conditions (Montejano et al., 2023; Sadeghi et al., 2023; Rauch et al., 2021; Scholz et al., 2023).
An investigation evaluated EB-CPG for its application in an outpatient setting (Gold et al., 2021), while 14 studies focused on its implementation within hospital environments (Barratt-Due et al., 2021; Ben Abdallah et al., 2023; Engelen et al., 2022; Gyselinck et al., 2022; Liesenborghs et al., 2021; Montejano et al., 2023; Rahman et al., 2022; Rauch et al., 2021; Sadeghi et al., 2023; Scholz et al., 2023; Suppan et al., 2020; Zurita-Cruz et al., 2022).
In 11 studies, the evaluation focused on EB-CPGs intended for use at the regional or international scope (Barratt-Due et al., 2021; Ben Abdallah et al., 2023; Gold et al., 2021; Gyselinck et al., 2022; Hamidi-Alamdari et al., 2021; Montejano et al., 2023; Rahman et al., 2022; Rauch et al., 2021; Sadeghi et al., 2023; Scholz et al., 2023; Zurita-Cruz et al., 2022), while four studies evaluated EB-CPGs within a national context (Chen et al., 2023; Engelen et al., 2022; Liesenborghs et al., 2021; Suppan et al., 2020).
Among the research analysing, a variety of interventions were examined, encompassing diverse strategies These strategies included the utilization of distinct methodologies, such as remdesivir and hydroxychloroquine (Barratt-Due et al., 2021; as per EB-CPG Lamontagne et al., 2020), the supplementation of zinc (Ben Abdallah et al., 2023; in accordance with EB-CPG COVID-19 TGP, 2023), the application of prospective audit and feedback mechanisms (Chen et al., 2023; based on EB-CPG COVID-19 TWG, 2020), the implementation of thromboprophylaxis including aprotinin, enoxaparin, or nadroparin (Engelen et al., 2022; following EB-CPG Vanassche et al., 2022), the enhancement of understanding guidelines (Gold et al., 2021; in line with EB-CPG UK HSA, 2020), the use of azithromycin (Gyselinck et al., 2022; according to EB-CPG Metlay et al., 2019), the administration of methylene blue (Hamidi-Alamdari et al., 2021; as outlined in EB-CPG Janssen et al., 2020), the implementation of itraconazole (Liesenborghs et al., 2021; based on EB-CPG Sciensano, 2020), the employment of TDF/FTC (tenofovir, disoproxil, fumarate/emtricitabine) subsequently, baricitinib was administered (Montejano et al., 2023; referencing EB-CPG Lamontagne et al., 2020), the introduction of colchicine (Rahman et al., 2022; per EB-CPG DCD, 2020), assessing the impact of equipment for personal protection on the effectiveness of chest compressions while performing CPR (Rauch et al., 2021; following EB-CPG Nolan et al., 2020), the inclusion of plasmapheresis (Sadeghi et al., 2023; as per EB-CPG Padmanabhan et al., 2019), the exploration of airway management strategies (Scholz et al., 2023; derived from EB-CPG Nolan et al., 2020), the advocacy for the appropriate utilization of equipment for personal protection (Suppan et al., 2020; according to EB-CPG BSC, 2020), and the vitamin D supplementation (Zurita-Cruz et al., 2022; outlined in EB-CPG Holick et al., 2011). Elaborated insights are available in Appendices N° 4 and N° 5.
Risk of bias evaluation
After applying the described methods, we evaluated bias risk utilizing the RoB2 tool for the fifteen RCTs included in this study.
We found that domain 1, which concerns the randomization process, exhibited a low level of bias risk in many of the studies, specifically in 10 out of 15 studies (66.7%). For domain 2, related to discrepancies in intended interventions, a low level of risk was observed in 6 studies (40%), while some concerns were noted in 5 studies (33.3%). In domain 3, focusing on missing outcome data, we identified a low risk of bias in 12 studies (80%). Measurement of the outcome (Domain 4) showed a high risk of bias in 13 studies (73%). In contrast, all studies displayed a low risk in domain 5, addressing the selection of the reported result (15; 100%).
Domain S evaluated the potential bias related to period and carryover effects. It is only applicable to crossover trials. In our assessment of Rauch et al. (2021), we identified a high risk within this domain.
Engelen et al. (2022) and Gyselinck et al. (2022) studies exhibited the most domains with a high risk of bias (D1, D3, and D4), along with some concerns noted for domain 2. The study exhibiting a low bias risk in all domains was Rahman et al. (2022). Concerning the overall risk of bias category, fourteen RCTs had high risk, while only Rahman et al. (2022) received a low-risk assessment. Figure 2 provides a summary of the results.
Figure 2. Risk of Bias analysis by using RoB2 tool (all included studies, n=15).
D: Domain D1: Randomisation process. DS: Risk of bias arising from period and carryover effects. This only applies to crossover trials. D2: Deviations from the intended interventions. D3: Missing outcome data. D4: Measurement of the outcome. D5: Selection of the reported result. Overall risk of bias.
: Low risk, 
: Some concerns, 
: High risk.
The primary source of bias risk identified in the included studies was associated with problems related to the absence of blinding or differing levels of blinding. Ten studies were conducted as per protocol analysis. Additionally, four studies were stopped due to futility (Ben Abdallah et al., 2023; Gyselinck et al., 2022; Liesenborghs et al., 2021; Montejano et al., 2023). Supplement N° 6 provides a more comprehensive explanation about this section (Ramírez-Morera et al., 2023).
Evaluation of evidence quality
Studies displaying the least biased risk (Rahman et al., 2022) and the highest biased risk (Gyselinck et al., 2022) underwent assessment using the GRADE method to construct a summary of findings (SoF). We assigned grades to a maximum of four outcomes for each study, including at least two with statistically significant results if such results were present. Creating a GRADE table of SoF enabled us to determine the potential ratings for the certainty of evidence across the 220 outcomes analysed in the 15 included studies.
According to GRADE classification, the evidence certainty ranged from low between high in the outcomes assessment. For the study with the least bias risk (Rahman et al., 2022), we identified variation in the certainty of evidence, from level high (diarrhoea and clinical deterioration) to moderate (death: all-cause mortality, and participants requiring ventilation support). This variation is attributed to imprecision, as detailed in Table 3a.
Table 3a. Summary of findings: Low risk of bias (Rahman et al., 2022).
| Patient or population: Moderate COVID-19 pneumonia. Setting: Treatment for COVID-19. Intervention: EB-CPGs for the management of COVID-19. Comparison: EB-CPGs for the management of COVID-19 plus Colchicine. | ||||||
|---|---|---|---|---|---|---|
| Outcomes | Anticipated absolute effects* | Relative effect (95% CI) | N° of participants (studies) | Certainty of the evidence (GRADE) | Importance | |
| Risk with EB-CPGs for the management of COVID-19 plus Colchicine | Risk with EB-CPGs for the management of COVID-19 | |||||
| Deterioration Follow-up: mean 28 days | 89 per 1 000 | 28 per 1 000 (11 to 72) | HR 0.31 (0.119 to 0.796) | 292 (1 RCT) | ⨁⨁⨁⨁ HIGH | CRITICAL |
| Diarrhoea Follow-up: mean 28 days | 185 per 1 000 | 46 per 1 000 (20 to 110) | RR 0.25 (0.107 to 0.596) | 292 (1 RCT) | ⨁⨁⨁⨁ HIGH | IMPORTANT |
| Participants requiring mechanical ventilation (both non-invasive and invasive) Follow-up: mean 28 days | 14 per 1 000 | 27 per 1 000 (6 to 128) | HR 2.0 (0.404 to 9.909) | 292 (1 RCT) | ⨁⨁⨁◯ MODERATEa | CRITICAL |
| Death (all-cause mortality) Follow-up: mean 28 days | 14 per 1 000 | 34 per 1 000 (8 to 141) | HR 2.5 (0.568 to 11.000) | 292 (1 RCT) | ⨁⨁⨁◯ MODERATEb | CRITICAL |
Regarding the Gyselinck et al. (2022) study with a high level of bias risk, we observed that the evidence certainty was consistently low across all four assessed outcomes (the occurrence long-term clinical enhancement or release from medical care; safety-related outcome involving a combination of cardiac events; adverse events and all-cause mortality). Mainly, this was attributable to factors associated with bias risk, particularly the absence of allocation concealment and blinding, as elaborated in Table 3b. Findings regarding the evidence certainty assessed for both studies aligned with the results obtained using the RoB2 tool.
Table 3b. Summary of findings: High risk of bias (Gyselinck et al., 2022).
| Patient or population: Hospitalised COVID-19 patients Setting: Treatment for COVID 19 Intervention: EB-CPGs for the management of COVID-19 Comparison: EB-CPGs for the management of COVID-19 plus Azithromycin | ||||||
|---|---|---|---|---|---|---|
| Outcomes | Anticipated absolute effects* | Relative effect (95% CI) | N° of participants (studies) | Certainty of the evidence (GRADE) | Importance | |
| Risk with EB-CPGs for the management of COVID-19 plus Azithromycin | Risk with EB-CPGs for the management of COVID-19 | |||||
| Incidence function of sustained clinical improvement or live discharge Follow-up: mean 29 days | 866 per 1 000 | 873 per 1 000 (775 to 942) | HR 1.030 (0.744 to 1.420) | 183 (1 RCT) | ⨁⨁◯◯ LOWa,b | CRITICAL |
| Safety outcome. Combined cardiac endpoint (hs-troponin >0.5 ng·mL−1 and/or ventricular arrhythmia requiring intervention and/or sudden cardiac death). Follow-up: mean 29 days | 202 per 1 000 | 189 per 1 000 (100 to 333) | OR 0.92 (0.44 to 1.98) | 183 (1 RCT) | ⨁⨁◯◯ LOWa,c | IMPORTANT |
| All-cause mortality Follow-up: mean 15 days | 42 per 1 000 | 19 per 1 000 (3 to 113) | HR 0.45 (0.073 to 2.830) | 149 (1 RCT) | ⨁⨁◯◯ LOWa,d | CRITICAL |
| Adverse Events Up to the End of Study Follow-up: mean 29 days | 244 per 1 000 | 210 per 1 000 (117 to 380) | RR 0.86 (0.48 to 1.56) | 183 (1 RCT) | ⨁⨁◯◯ LOWa,e | IMPORTANT |
* The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).
a Risk of bias: Lack of blinding. Lack of allocation concealment. The trial statistician remained blinded until the database lock. Downgraded -1 for risk of bias.
b Imprecision. 95%IC: 0.744 to 1.42. p: 0.86. Downgraded -1 for imprecision. Not statistically significant.
c Imprecision. 95%IC: 0.44 to 1.98. p: 0.43. Downgraded -1 for imprecision. Not statistically significant.
Analysis of the studies outcomes
The outcomes were categorized into both absolute and relative metrics. A comprehensive summary of effect measurements across the studies is not available due to the substantial variation in measurement units and clinical heterogeneity within the included articles.
The assessment of the effect measurements of the reported outcomes in the included articles exhibited substantial variability. Many of them were dichotomous, such as evaluating health progression, aligning with the patient outcome dimension. In the 15 evaluated RCTs, a total of 220 outcomes were assessed, with six (3%) relating to the healthcare structural aspect, 136 (62%) measures were attributed to the process dimension, and 78 (35%) interventions were linked to the patient outcome dimension (Table 4). Supplement N° 7 presents a more comprehensive overview of the primary findings across the assessed dominions in the included studies (Ramírez-Morera et al., 2023).
Table 4. Dimensions and effect summary of results (all included studies, n=15).
| N° | Citation | Intervention | ||||||
|---|---|---|---|---|---|---|---|---|
| Domain structure | Domain process | Domain patient outcome | In favour CPG | Equal | In favour control | Total | ||
| 1 | Barratt-Due et al., 2021 | 0 | 6 | 6 | 0 | 12 | 0 | 12 |
| 2 | Ben Abdallah et al., 2023 | 0 | 3 | 12 | 6 | 9 | 0 | 15 |
| 3 | Chen et al., 2023 | 0 | 4 | 8 | 0 | 10 | 2 | 12 |
| 4 | Engelen et al., 2022 | 0 | 4 | 10 | 0 | 14 | 0 | 14 |
| 5 | Gold et al., 2021 | 0 | 10 | 0 | 2 | 7 | 1 | 10 |
| 6 | Gyselinck et al., 2022 | 0 | 3 | 5 | 0 | 8 | 0 | 8 |
| 7 | Hamidi-Alamdari et al., 2021 | 0 | 1 | 19 | 2 | 7 | 11 | 20 |
| 8 | Liesenborghs et al., 2021 | 0 | 3 | 5 | 0 | 8 | 0 | 8 |
| 9 | Montejano et al., 2023 | 0 | 4 | 6 | 0 | 10 | 0 | 10 |
| 10 | Rahman et al., 2022 | 0 | 6 | 4 | 0 | 9 | 1 | 10 |
| 11 | Rauch et al., 2021 | 0 | 10 | 0 | 0 | 10 | 0 | 10 |
| 12 | Sadeghi et al., 2023 | 0 | 3 | 2 | 2 | 3 | 0 | 5 |
| 13 | Scholz et al., 2023 | 0 | 72 | 0 | 23 | 37 | 12 | 72 |
| 14 | Suppan et al., 2020 | 6 | 0 | 0 | 1 | 5 | 0 | 6 |
| 15 | Zurita-Cruz et al., 2022 | 0 | 7 | 1 | 2 | 6 | 0 | 8 |
| TOTAL n (%) | 6 (3%) | 136 (62%) | 78 (35%) | 38 (17%) | 155 (71%) | 27 (12%) | 220 | |
In terms of the effect of utilizing EB-CPGs, we identified 155 (71%) interventions in which the experimental and control groups demonstrated similar results, with no statistical significance. Six studies (40%), no outcomes with statistical significance were identified (Barratt-Due et al., 2021; Engelen et al., 2022; Gyselinck et al., 2022; Liesenborghs et al., 2021; Montejano et al., 2023; Rauch et al., 2021).
The outcome favoured the control group in 27 cases (12%). In three studies, more statistically significant results were observed favouring the control group (Chen et al., 2023; Hamidi-Alamdari et al., 2021; Rahman et al., 2022).
The intervention group was favoured in 38 cases (17%). More outcomes with statistical significance in favour of the experimental group were observed in 6 studies (Ben Abdallah et al., 2023; Gold et al., 2021; Sadeghi et al., 2023; Scholz et al., 2023; Suppan et al., 2020; Zurita-Cruz et al., 2022). A detailed description is available in Supplement N° 7 and Table 4 (Ramírez-Morera et al., 2023).
Discussion
Public and private institutions have invested in economic and methodological efforts for over two decades to generate high quality and more comprehensive EB-CPGs. These endeavours aim to address challenges faced by many healthcare organisations, such as population ageing, higher pressure for quality attention, higher cost of emerging health technologies, and variations in healthcare provision. These variations suggest that disparities could lead to deficient treatment or to misuse of resources. Furthermore, patients and clinicians aimed to receive and to provide the greatest viable care with assessable clinical outcomes. Conversely, despite these efforts, certain EB-CPGs seems to not contribute to effective, standardised clinical practices grounded in the best available evidence. This discrepancy may arise from potential deficiencies in their construction and implementation (Woolf et al., 1999; Alonso-Coello et al., 2010; IOM, 2011).
In three systematic reviews appraised the methodological quality of the development of COVID-19-related EB-CPGs, there is a consensus that the majority exhibit overall quality well below the recommended level to be considered suitable for use. According to the instrument AGREE II involvement of stakeholders, applicability, and rigour in the elaboration were deficient domains (Amer et al., 2022; Stamm et al., 2021; Wang et al., 2021).
Another review evaluated EB-CPGs for COVID-19 with the National Academy of Medicine criteria showing the following findings: Certainty and strength of evidence and recommendations were not graded or rated, no external review was conducted, patient or public perspectives were not included, and plans for updates were often absent. (Burns et al., 2021). On this occasion, it could result from the limited time available for their development and a scarcity of high-certainty evidence for their approach, owing to the prevailing health emergency (Subramanian et al., 2020).
We concur with Woolf et al. (1999) about EB-CPGs promoting established benefits and discouraging ineffective interventions have the potential to diminish morbidity and mortality while enhancing quality of life, particularly in precise conditions. Moreover, EB-CPGs contribute to improving care’s consistency.
Interventions in 15 RCTs examined showed effects on patient outcomes (35%), the care provision domain was the most extensively assessed (62%). They coincide with the findings of our first systematic review within this series (Ramírez-Morera et al., 2019) on EB-CPGs in cardiovascular health care. Notably, there remains an emphasis on evaluating adherence to EB-CPGs rather than assessing the effect of implementing the recommended interventions. We believe that researchers have persisted in their efforts to determine when EB-CPGs should or should not be used. Six interventions (3%) were identified for the domain of medical care structure, the same finding encountered in this dimension in previous studies (Ramírez-Morera et al., 2019) of EB-CPGs in cardiovascular healthcare and the second systematic review on EB-CPGs for breast cancer (Ramírez-Morera et al., 2022).
A systematic review by Grimshaw & Russell (1993) informed over 80% significant enhancements between the studies considered. In contrast to this third systematic review, 38 COVID-19 interventions favouring EB-CPGs were identified across all included studies (17% of all evaluated interventions). This result has remained consistent in our last two systematic reviews; when evaluating EB-CPGs for cardiovascular care, the proportion in favour of EB-CPGs was 30% (Ramírez-Morera et al., 2019), and for breast cancer EB-CPGs, it was 16% (Ramírez-Morera et al., 2022). We did not find results favouring the use of EB-CPGs to a greater extent within our three reviews. This suggests that there are still barriers to overcome for the true impact of their use to be more readily evident through randomised controlled trials (RCTs). One reason for this outcome could be the methodological shortcomings of constructing the evaluated EB-CPGs in the studies. Without ensuring the inclusion of interventions supported by high-certainty evidence and avoiding biases, achieving notable effects with their use becomes exceedingly challenging (Amer et al., 2022; Burns et al., 2021; Stamm et al., 2021; Wang et al., 2021; Subramanian et al., 2020). In alignment with the viewpoint of the review of Lugtenberg et al. (2009), they emphasised that focusing on the strength of recommendations is vital to comprehending the factors impacting guideline utilisation and enhancing patient outcomes.
Fourteen studies (93%) reported high risk of bias in overall, with only Rahman et al. (2022) writing low risk. This is another potential reason why the effect of using EB-CPGs isn’t as easily discernible. Observing this significant number of studies with bias risks concurs with the perspective of Ivers et al. (2012) about the compulsory evaluation of the EB-CPGs through studies of outstanding methodological quality. This approach promotes quality of care through feedback and audits.
We recognised that recommendations stratified by levels of evidence certainty are necessary. This implies prioritising the inclusion of recommendations based on higher evidence certainty in the EB-CPGs. This approach has the potential to create a significant beneficial effect supporting the utilisation of EB-CPGs. Furthermore, EB-CPGs enhance medical practice if they are considered alongside rigorous assessments of their adherence (Grimshaw & Russell, 1993; Ricci-Cabello et al., 2020), an approach that should be sensitised to be implemented across a more significant number of entities developing EB-CPGs.
Lugtenberg et al. (2009) noted a significant variation in the observed effects across recommendations within their systematic review. This finding has been consistently recurring throughout our three systematic reviews on this subject. We discovered that for a considerable number of assessed interventions (155, 71%), the utilisation of EB-CPGs showed no discernible impact in any dimension for COVID-19. This aligns with the trends observed in cardiovascular disease (55, 65%; Ramírez-Morera et al., 2019) and breast cancer (303, 83%; Ramírez-Morera et al., 2022). Woolf et al. (1999) considers that the methodology employed to evaluate the actual effectiveness of EB-CPGs continues unfinished, then a plan to enhance outcomes has yet to be identified.
EB-CPGs assumes a vital role when clinicians lack a precise comprehension of suitable practices and the scientific evidence required to inform decision-making (Woolf et al., 1999). As such, EB-CPGs creators should be vigilant in recognising health care professionals’ needs to effectively address all informational voids to cultivate a heightened eagerness towards their utilisation.
EB-CPGs hold promise in elevating healthcare quality. Nevertheless, integrating strategies that amplify their impact and implementation can further enhance their efficacy. Examples include the incorporation of visits with educational and academic purposes into current training programmes (O’Brien et al., 2007). Frequently, their proponents consider it like a panacea for healthcare challenges, often disregarding other practical actions that should accompany these guidelines (Woolf et al., 1999).
Conclusions
The role of EB-CPGs in high-quality healthcare is pivotal, especially when clinicians face uncertainty in discerning appropriate practices underpinned by scientific evidence. Establishing strategies to ensure standardised development and implementation of EB-CPGs is paramount. Integrated structured programmes within the education and healthcare systems can contribute significantly to this uniformity. Medical personnel should apply evidence-based recommendations to improve clinical practice and achieve desired outcomes.
A disparity exists among the number of EB-CPGs created to address COVID-19, those meeting a minimum standard of methodological quality for use, and the quantity of high-caliber studies assessing their efficacy. Given the few findings on the benefits of guidelines utilisation, the need for further investigation in this area persists. Gradually constructing a stronger hypothesis concerning the variables impacting this matter is a direction worth pursuing.
The variability in the outcomes associated with the recommendations contained in EB-CPGs implies that shifting the strategies until now employed would be beneficial. Focusing on an improved selection of evidence and recommendations described in EB-CPGs, alongside continuous scrutiny of adherence limitations through audits and constant training programmes, would be valuable. It is essential to develop implementation strategies customized for each recommendation.
In upcoming randomized controlled trials that are conducted, it would be helpful if researchers distinguish between the degrees of evidence certainty that supports the strength of their recommendations. They ought to prioritize the assessment of EB-CPGs containing recommendations likely to yield a greater impact (higher grades, certainty of the evidence: moderate or high).
Additional investigation is needed to address element and variables associated with the development and use of EB-CPGs that influence measurable beneficial effects. Increasing the number of EB-CPG assessed and prioritising evaluating the patient outcomes domain is imperative.
Due to the health emergency caused by COVID-19, many interventions were employed with limited or no scientific evidence supporting them. These same interventions were included in EB-CPGs that were rapidly developed and did not meet the minimum methodological quality criteria for recommending their use. We believe that the above circumstances could have influenced the findings of this systematic review.
Implications for the practice
This study is intended to assist EB-CPG developers in evaluating their impact on clinical quality and providing credible evidence regarding the advantages of their integration into healthcare practices. This encouragement promotes the continued development of EB-CPGs worldwide, particularly in countries with limited resources where resource optimization is paramount.
While few results from this systematic review were statistically significant, it still motivates us to continue supporting EB-CPG use to improve clinical practice and healthcare quality. The results of this study should be approached cautiously, and there is a need to enhance the implementation of EB-CPG, considering their active dissemination and inclusion in patients’ electronic records, with alerts to assist healthcare personnel in better adhering to them.
The efforts to develop EB-CPGs are substantial and should be complemented by psychosocial strategies that encourage healthcare personnel to adhere to the recommendations and assess their impact in the environment where they are implemented. Factors such as the time required for effective knowledge transfer, potential costs, risks, benefits, and anticipated effects must be considered for proper implementation.
Enhanced methodological training is necessary for EB-CPG developers. Those involved in their creation must have sufficient time, resources, and necessary methodological guidance to ensure compliance with the minimum criteria of the AGREE II Instrument for recommending their use. It’s plausible that if EB-CPGs only encompassed recommendations with high or moderate levels of evidence certainty, where their expected impact is substantial, this could result in increased motivation to utilize EB-CPGs and actively engage in assessing the impacts.
Implications for further research
This systematic review represents the third study in a series to evaluate the impact of EB-CPGs on improving healthcare standards.
Drawing from the latest studies in this topic and considering the inconclusive outcomes of the current study, additional research is required to determine the potential influence of EB-CPGs on healthcare quality. This implies placing greater emphasis on areas that have received less research in the past, such as the domain of patient outcomes and the domain of structure.
Lastly, there is a need for future RCTs to prioritize the maintenance of allocation concealment and blinding of the evaluated intervention while ensuring robust study design and methodological rigor. Researchers should consider the degree of evidence certainty that underpins the advice of EB-CPGs and stratify their evaluation accordingly.
Author contributions
Anggie Ramírez-Morera: Tasks include conceptualizing, curating data, analysing, acquiring funding, developing methodology, project administration, and writing original drafts, as well as reviewing and editing.
Jordan Salazar-Vargas: Tasks include curating data, analysing, acquiring funding, provision of resources, and writing original drafts, as well as reviewing and editing.
Ana Leonor Rivera-Chavarría: Tasks include curating data, analysing, acquiring funding, provision of resources, and writing original drafts, as well as reviewing and editing.
Gerard Urrútia Chuchí: Tasks include conceptualizing, and writing original drafts, as well as reviewing and editing.
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Ramírez-Morera A, Salazar-Vargas J, Rivera-Chavarría AL and Urrútia G. Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review. [version 1; peer review: 1 approved]. F1000Research 2023, 12:1501 (https://doi.org/10.12688/f1000research.142098.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 24 Nov 2023
Views
14
How to cite this report:
Marques Bernardo W. Reviewer Report For: Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review. [version 1; peer review: 1 approved]. F1000Research 2023, 12:1501 (https://doi.org/10.5256/f1000research.155598.r229775)
The direct URL for this report is:
https://f1000research.com/articles/12-1501/v1#referee-response-229775
https://f1000research.com/articles/12-1501/v1#referee-response-229775
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 23 Dec 2023
Approved
VIEWS 14
The authors made a systematic review including randomized controlled trials evaluating the impact of evidence based guidelines on the management of COVID. All the items involved in a consistent systematic review were present. I suggest to state clearly that there
... Continue reading
CITE
HOW TO CITE THIS REPORT
Marques Bernardo W. Reviewer Report For: Effects of evidence-based clinical practice guidelines for COVID-19 in health care quality improvements. A third systematic review. [version 1; peer review: 1 approved]. F1000Research 2023, 12:1501 (https://doi.org/10.5256/f1000research.155598.r229775)
The direct URL for this report is:
https://f1000research.com/articles/12-1501/v1#referee-response-229775
https://f1000research.com/articles/12-1501/v1#referee-response-229775
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
COMMENTS ON THIS REPORT
Comments on this article Comments (0)