Effects of evidence-based clinical practice guidelines for breast cancer in health care quality improvements. A second systematic review.

Background: Traditionally, EB-CPGs have been believed to mainly improve the quality and consistency of health care, but this claim must be conclusively proven. We used the Donabedian three-dimensional model (structure, process, and patient outcomes) to assess improvements in the quality of medical care derived from implementing EB-CPGs. This study corresponds to the second systematic review carried out as a series of studies on different clinical issues that aim to evaluate the effectiveness of the application of the EB-CPG for improving the quality of care. Methods: We followed the methods described by the Cochrane Handbook and presented a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of Controlled Trials, PubMed, and EBSCO Host databases, as well as the grey literature, between 1990 and April 2021. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Results: Of the total of 364 interventions included in the eleven RCTs evaluated, 11 (3%) were related to healthcare structure, 51 (14%) to the healthcare delivery process and 302 (83%) to patient outcomes. Regarding the impact of using the EB-CPGs, in 303 interventions (83%), there were no significant differences between the control and experimental groups. In 4 interventions (1%), the result favoured the control and intervention groups in 57 of the interventions (16%). Conclusions: Our study showed that EB-CPGs slightly enhanced the quality of health care in the three dimensions described by Donabedian. Future RCTs should improve their design and methodological rigour by considering the certainty of the evidence supporting the EB-CPGs recommendations. In that context, broader analyses could be performed, having more concise hypotheses for further research. Registration: PROSPERO CRD42020205594


Introduction
The emergence of the Evidence-Based Clinical Practice Guidelines (EB-CPGs) in the 1990s has improved decisionmaking for healthcare personnel and patients with different health conditions (IOM, 1990;Weisz et al., 2007).
Worldwide, significant efforts have been made to develop and implement EB-CPGs based on scientific evidence. The term "evidence-based" means that the recommendations described in the CPGs derive from the best scientific findings and the highest quality of evidence obtained from applying unbiased, transparent, and rigorous methods to support clinical care (Watters, 2008;Linskey, 2010).
EB-CPGs are statements that include recommendations intended to improve the quality and consistency of health care (Woolf et al., 1999;Kwan, 2004) and to assist clinical staff and patients in the decision-making process (Alonso-Coello et al., 2010;IOM, 2011).
This study corresponds to the second systematic review of a series of studies on different clinical subjects, which aim to assess the success of the use of EB-CPGs (Ramírez-Morera et al., 2019). Therefore, we evaluated the effectiveness of the application of EB-CPGs for the improvement of the quality of health care in three dimensions: structure, process, and patient outcome in the management of breast cancer (Donabedian Model, Donabedian, 1988).
Studies measuring the effects of EB-CPGs on the quality of health care have mainly focused on the effects on clinical practice (Lugtenberg et al., 2009), state that some international reviews have demonstrated that most of the studies have resulted in significant improvements to the process of care. However, few studies have focused on the effects of measures on patients' health outcomes.
We considered for this review breast cancer disease because this is the most common malignancy in women around the world (Ghoncheh et al., 2016). Although survival has improved in the last 30 years mainly because of the implementation of early detection programs and treatment improvement, it still registers 2.3 million new diagnostics in women during 2020 and 685 000 deaths within the same year (WHO, 2020).
Strategies to control and prevent this type of cancer must be a high priority for health policymakers (Ghoncheh et al., 2016). According to the above, hundreds of breast cancer guidelines have been published worldwide to reduce its negative impact on men's and especially on women's health.
This review is relevant because there is a growing number of EB-CPGs in different essential areas, and their actual impact on relevant outcomes needs to be assessed. Few systematic reviews evaluate the effect of EBGPC in improving health care; these focus solely on one clinical entity and cover a country or region (Grimshaw et al., 1993;Worrall et al., 1997;Lugtenberg et al., 2009;Ricci-Cabello et al., 2020). For this reason, we reviewed the evidence on the benefits of implementing EB-CPG to improve the quality of care. This review responds to the need for EB-CPG research synthesis on the overall quality of health care delivery.
This systematic review contributes to meeting the need for research synthesis about EB-CPG by assessing the overall quality of health care delivery. Besides, we visualise the need for a systematic review that conclusively demonstrates the pragmatic impact that evidence-based recommendations have on breast cancer patients.

Methods
We conducted a systematic review to identify and analyse the effect of EB-CPGs on health care quality improvement within the Donabedian Model dimensions: structure, process, and results (Donabedian,1988). We followed the Cochrane Handbook methodological recommendations described by Higgins et al. (2022a). This review is registered at PROSPERO (ID: CRD42020205594).

REVISED Amendments from Version 1
We modified the original article in response to the reviewers' comments: 1. We added breast cancer to the keywords. 2. In the sixth paragraph within the introduction section we added: "and treatment improvement".
Any further responses from the reviewers can be found at the end of the article

Study search
The research question was translated into the PICO framework for guiding the study search and the criteria selection (Table 1). We developed a method to incorporate the methodological component of the search strategy combined with selected index and free-text terms.
We explored the following electronic databases for primary studies: Cochrane Central Register of Controlled The PubMed search strategy was executed in the other databases using the appropriate controlled vocabulary. Searching for other resources included grey literature from different sources and hand searching of those high-yield journals and conference proceedings that have not already been hand searched on behalf of the Cochrane Collaboration.
Authors of relevant papers were contacted regarding any further published or unpublished work. Authors of other reviews in the field of effective professional practice were contacted regarding relevant studies of which they may be aware. We searched for studies published between January 1990 and April 2021. The search strategy was not restricted by language. An advanced search strategy and results are available as extended data, Appendix 1 (Ramírez-Morera et al., 2022).
For the management of bibliographic references of the articles found, the web application "Sciwheel Reference Manager & Generator" was used (Sciwheel, 2022).

Studies selection
The studies found through the search strategy were screened by two reviewers (AR, JS), and discrepancies about study selection were resolved by a third reviewer (MT). Inclusion criteria were: 1. Randomised Clinical Trial (RCT) or clustertype RCT measuring the impact of using any implementation model versus passive dissemination or no use of the EB-CPG. 2. The studies evaluated the impact on any of the three domains described in the Donabedian model (structure, process, and patient outcomes) for using EB-GPC in treating breast cancer. 3. No language restriction. 4. Published studies from 1990 to 2021.

Data extraction
Three authors (AR, JS, ALR) independently undertook data extraction. They used a modified version of the Cochrane Collaboration EPOC Group "Data Collection Checklist", employing an electronic datasheet (EPOC, 2019).
To assess the bias risk, we used standard Cochrane methods described in chapters 8, 10 and 23 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2022b;Sterne et al., 2019). In the case of RCTs, bias resulting from several types of systematic errors was assessed according to methods described by Cochrane (Higgins et al., 2022c) and following the RoB2 instrument (Sterne et al., 2019).
All studies deemed eligible for the review were assessed independently by the review authors (AR, MT), and discrepancies were resolved by discussion. A summary of the risk of bias assessment is presented as part of the characteristics of included studies table. We did not find cluster-type RCTs; then, we did not apply what is stated in chapter 23 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2022d) about assessing cluster RCT following the RoB2 instrument (Sterne et al., 2019).
An analysis of the quality of the evidence related to each of the outcomes was performed using the GRADE approach (Schünemann et al., 2013). We assessed the certainty of the body of evidence for each key outcome as "high", "moderate", "low", or "very low", using the GRADEpro GDT platform (GRADEpro, 2021).
Data were analysed using Review Manager software, version 5.4 (RevMan, 2020). Revman default templates for data extraction were modified to show the results in a simplified way.
We found very high variability between the measures of effect within the included studies in this review. Then, we decided not to perform a meta-analysis and, therefore, neither to measure statistical heterogeneity.

Study identification and selection
The process describing the analysis of studies retrieved through the systematic search is content in the PRISMA flowchart ( Figure 1, Page et al., 2021). We retrieved 25002 studies from database searching and 20 studies were found from additional sources identified. We excluded 15900 duplicated records, and 6072 studies were excluded after screening by title and abstract. We assessed 83 articles at the full-text level, excluding 72 references which did not meet the selection criteria: 51 (71%) were not randomised controlled trials, and 20 (28%) did not evaluate clinical practice guidelines.  . * The appendix 1 shows the number of records identified from each database or register searched. ** All records were excluded by the authors. Excluded after screening by Title/Abstract. *** The records were not randomised controlled trials. **** The records did not evaluate clinical practice guidelines. ***** They published the results included by Irene et al. (2019), whose study described a broader methodology and reported more detailed results
The trials were published between 2011 and 2020, most from 2014 to 2016 (5, 46%). Approximately 55% (6) were carried out in the United States of America. The clinical practice guidelines examined within the selected trials were follow-up (11, 100%) and treatment (4, 36%). When evaluating the outcome categories involving the clinical practice guidelines quoted, most referred to the quality of life (7, 63%). A summary of the characteristics of the included studies (n=11) is available ( Table 2). A broader description of these characteristics is available as extended data (   *Percentages exceed 100% because the categories are not mutually exclusive (i.e., some studies involved more than one type of guideline scope and more than one outcome category).

Risk of bias assessment
We assessed the risk of bias with the RoB2 instrument in the eleven included RCTs following the methods described.
We found that a low risk of bias prevailed (6; 55%) in domain 1: randomisation process. Some concerns occurred in domain 2: deviations from the intended interventions (7; 64%). A low risk of bias was found for domain 3: missing outcome data (10; 91%) and domain 4: measurement of the outcome (8; 73%). Finally, we found some concerns in domain 5: selecting the reported result (7; 64%).
Most of the risk of bias found in the included studies occurred due to the lack of existence or some level of blinding. In some cases, outcome data were not available entirely for all randomised participants. Four studies did not analyse the intention to treat, and some outcome measurement methods were not adequately described. A broader description of the risk of bias assessment is available as extended data in Appendix 6 (Ramírez-Morera et al., 2022).

Quality of evidence assessment
The studies showing the lowest risk of bias (Irene et al., 2019) and one resulting in the highest risk of bias (Park et al., 2019) were chosen to be evaluated with the GRADE methodology and to build a summary of the findings. We decided to grade only four outcomes per study described, including two reporting statistically significant results and two not statistically significant results. Performing a GRADE table provided the rank of possible grades for the certainty of the evidence found in the 362 interventions from the 11 studies included.
The results varied from high to very low certainty of the evidence, according to the GRADE classification. We found several types of systematic errors in the studies: random sequence generation (selection bias, in 4 studies), incomplete outcome data (attrition bias, in 3 studies), and imprecision (observed within all the studies, in 303 outcomes evaluated representing 83% of the total).    We found for the study with the lowest risk of bias (Irene et al., 2019) a range for the certainty of the evidence between high (for fertility-related concerns scale scores ≤3 and improvement with no or low fertility or pregnancy concerns) to moderate certainty of the evidence (for improvement in at least one women's health issue and 50% decrease in the hot flash score), as described in Table 3a.
In the case of Park et al. (2019) study reporting a high risk of bias, we found a range for the certainty of the evidence between low (moderate MET-min/wk and walk MET-min/wk at six months) and very low certainty of the evidence (Moderate MET-min/wk and Walk MET-min/wk at 12 months), as described in Table 3b. Our results after grading the certainty of the evidence for both studies were consistent with the findings from the RoB2 instrument.

Assessment of the studies outcomes
The outcomes were grouped in simple relative and absolute numbers. A global estimate of the measurements of the effects included in the studies is lacking because of the significant variability of measuring units combined with the clinical heterogeneity found among the studies included.
There was significant variability in the measurement of the outcomes reported in the studies. Most were continuous, e.g., to assess the quality of life, which corresponds to the dimension of patient outcome. A total of 362 were included in the 11 RCTs evaluated; 11 (3%) corresponded to the health care structure dimension, 51 (14%) interventions to the dimension process and 302 (83%) interventions to the dimension of patient outcomes. A broader description of the main findings by the dimensions evaluated in the included studies is available as extended data, Appendix 7 (Ramírez-Morera et al., 2022).
Regarding the impact of using EB-CPG, we found 303 (83%) interventions with no significant difference between the control and experimental groups. The outcome favoured the control group in 4 (1%). Three outcomes interfered with the patient's adherence (predictable variables: age, marital status, and hot flashes), as reported by Maly et al. (2017). Also, the fourth outcome informed by Klinkhammer-Schalke et al. (2012) reported rates of therapeutic options for Physiotherapy 16 (experimental group) vs 30 (control group), p: <0.02, at six months. The result favoured the intervention group for 57 interventions (16%) ( Table 4).

Discussion
For more than two decades, governmental and non-governmental institutions have been making economic and methodological efforts to develop more and better quality EB-CPGs, seeking to deal with different issues most healthcare systems face. Such as the ageing population, rising costs motivated by increased demand for quality care, increasingly expensive emerging health technologies, variability in the provision of health by presuming that part of this disparity could cause inadequate care (either overuse or underuse of supplies), and the desire for clinicians and patients to provide and to receive, respectively, the best possible care with measurable clinical effect. However, it still appears that some of these EB-CPGs are far from contributing to an effective, standardised clinical practice based on the best available evidence (Woolf et al., 1999;IOM, 2011;Alonso-Coello et al., 2010). We agree with Woolf et al. (1999) that EB-CPGs that promote proven benefits and discourage ineffective interventions could reduce morbidity and mortality, and improve quality of life, at least for some conditions. EB-CPGs can also improve the consistency of care.
The effects of the recommendations in the interventions included in the 11 RCTs considered in this project were in the structure of medical care (3%) and the care provided (14%); both were the least explored. Surprisingly, patient outcomes were the most evaluated domain (83%), with significant results in 43 of 302 (14%), representing 75% of all significant results.
This fact could lead us to suppose that researchers finally focused on the importance of evaluating the patient health, solely prioritising evaluating the adherence to the CPG (assess the process dimension). They are trying to glimpse more clearly what the use of the CPG represents for patients and not only for clinical staff. As this review considered breast cancer, we could not ignore that there is more social and economic pressure to know the patient outcome. Grimshaw and Russell (1993) findings described in their systematic review, "Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations", reported more than 80% significant improvements among the included studies. Contrasting their results, in this second systematic review, we found 57 interventions on breast cancer in favour of the use of EB-CPGs distributed in all the studies included (16% of all the interventions evaluated). Then, compared to our first systematic review, only half of the measures with statistically significant results favour using EB-CPGs (Ramírez-Morera et al., 2019). However, as 75% of these results were found for the dimension of results in patients, we keep optimistic about the finding on incoming reviews an increasingly more significant impact.
We also concur with Lugtenberg et al. (2009) about the need to focus on the strength of the recommendations for determining what factors influence the use of the guidelines and the improvement of the results of the patients.
Then we identified the need to perform recommendations distinguishing between the level of certainty of evidence (stratified analysis) as this could have a more significant impact on the results when the best available certainty of evidence recommendation is implemented. We keep in mind that explicit EB-CPG improves clinical practice when introduced within a context of rigorous evaluations (Grimshaw & Russell, 1993;Ricci-Cabello et al., 2020).
Some studies included in the review five reported outcomes in the process area (Boekhout et al., 2015;Grunfeld et al., 2011;Irene et al., 2019;Klinkhammer-Schalke et al., 2012;Maly et al., 2017). All of them described favourable results for the intervention in 14 of 51 measurements (27%). This fact reflects that researchers have continued to endeavour to measure when EB-CPGs should be used or not, but this time to a much lesser extent (14% vs 64%) when compared to the first review. Lugtenberg et al. (2009) reported that the size of the effects observed in their systematic review varied considerably between the recommendations within the guidelines. We repeated this finding in our study and the previous one (Ramírez-Morera et al., 2019). We found that in many evaluated interventions (303, 83%), the use of EB-CPG did not reflect any impact in any dimension. The approach followed to report the current effectiveness of EB-CPGs remains incomplete (Woolf et al., 1999), and a strategy that captures better results has not yet been found.
EB-CPGs may contribute to improving the quality of healthcare. However, it is still necessary to integrate them with strategies that enhance their use and effect, such as academic and educational visits as part of ongoing training programs (O'Brien et al., 2007). Repeatedly, advocates for EB-CPGs consider their only existence as a magic solution to solve health care problems; however, they ignore other practical actions that should be implemented along with the guidelines (Woolf et al., 1999).
EB-CPG has an essential role when clinicians do not clearly know the appropriate practice and which scientific evidence should support their decisions (Woolf et al., 1999). Then, EB-CPG developers should be vigilant in identifying these needs to help close this information gap and increase the enthusiasm for employing them.

Conclusions
Developing strategies for a more standardised implementation of the EB-CPGs through structured programs within health systems is essential. Improving the awareness of clinical staff about the possibility of enhancing clinical practice and patient outcomes by using evidence-based recommendations with an expected effect is a need.
There is an imbalance between the number of EB-CPGs developed for breast cancer and the number of high-quality studies evaluating their effectiveness. Due to the limited results found on the benefit of using EB-CPG, we must continue investigating the subject. We could gradually structure a more robust hypothesis about the variables influencing this issue.
The variation in the effects found for the recommendations included in the EB-CPGs suggests that it would be helpful to change strategies and focus on the analysis of the limitations of adherence and on designing implementation approaches adapting each recommendation.
In addition, future RCTs should distinguish the levels of certainty of the evidence supporting each recommendation in their evaluations. Researchers should focus on evaluating recommendations expected to have the most significant impact (superior levels of certainty of the evidence: High or Moderate).
More research is necessary to define which factors related to the implementation of EB-CPG and its specific recommendations are essential to predict the application of EB-CPG and, therefore, achieve better patient results.

Implications for the practice
This systematic review aimed to support the development of programs evaluating the effects of EB-CPG on the quality of health care. Also, to provide reliable evidence sustaining the decision-making process related to the production of EB-CPGs.
Even when some of the results of this systematic review were statistically significant, supporting the use of EB-CPG as a tool to improve clinical practice and quality of care, the results of this review need to be interpreted with caution.
The implementation of EB-CPGs must consider the differences in the measures of effect to define customised approaches and specific recommendations within the guideline to enhance health care.
For the adequate implementation of EB-CPGs, it is necessary to consider the possible costs, risks, and benefits and the expected effects derived from EB-CPGs recommendations. The efforts to build EB-CPGs must be complemented with psychosocial strategies promoting health personnel to follow the recommendations of the EB-CPG and evaluate their impact.
Greater methodological rigour in the development of CPGs is needed. It is also required to carry out this process within a standardised formal program in the health systems. Greater credibility could be achieved if recommendations are based on the best available evidence, improving the credibility of their positive effect among health personnel. It could also lead to a more willingness to implement the CPGs and to participate in evaluating their impact.

Implications for the research
This study corresponds to the second systematic review of a series of studies aiming to assess the effect of Evidence-Based Clinical Practice Guidelines (EB-CPGs) on improving health care quality. Our next and last project will investigate Covid-19 disease.
Due to recent research in this field, and the results of this study were not conclusive, more research is necessary to evaluate how EB-CPGs could impact the quality of health care, especially emphasising fewer investigated areas, such as the structure of healthcare services and patient outcomes.
Additionally, the design and methodological rigour applied to future RCTs should improve by considering the certainty of the evidence supporting the EB-CPGs recommendations. Besides, focusing on those with a greater level of evidence (high or moderate) which could lead to determining more clearly the effect that these recommendations have on the quality of health care.
Data (and software) availability Data availability Underlying data All data underlying the results are available as part of the article, and no additional source data are required. Open Peer Review reference to differences between control and experimental groups. But these differences were not associated with specific outcomes. The discussion and conclusions tell us about these limitations and reinforce the importance of considering and focusing on the 3 dimensions when EB guidelines are constructed. This systematic review brings a good model to be reproduced in other clinical subjects of guidelines, considering the adherence and the clinical impacts.
The first quality item of this study is the objective of not only critically evaluating EB guidelines, but mainly evaluating their usefulness measured by clinical outcomes in their implementation.
Analyzing patients with breast cancer as the focus of management, while bringing relevant interventions and outcomes, also brings great variability between conducts, increasing the heterogeneity of the analyses.
In the method, the critical use of three domains (structure, process and result) increases the ability to assess the implementation of the guidelines on the subject.
The search method, selection (eligibility criteria), data extraction, risk of bias assessment, and quality of evidence are fully reproducible and adequate.
There is a limitation in the expression of results due to the already expected heterogeneity between the randomized clinical trials included in the guidelines implementation environment (11 RCTs and more than 300 interventions involved).
The analysis, fundamentally supported by the "outcomes" domain, discourages the use of guidelines, since, in the majority, the results are similar when using or not using a standardized conduct. However, this analysis is not specific about the interventions adopted and measured outcomes, which in addition to not allowing a quantitative expression, does not allow for analyzing the reasons for the absence of differences in the implementation of the recommendations.
And the questions that remain are: is there still any doubt whether the homogenization of medical practice produces benefits and reduces harm to patients? Is this question still open to investigation?
Are the rationale for, and objectives of, the Systematic Review clearly stated? Yes

Is the statistical analysis and its interpretation appropriate?
Not applicable

Are the conclusions drawn adequately supported by the results presented in the review? Yes
Competing Interests: No competing interests were disclosed.

Ignacio Marín León
Internal Medicine, Virgen del Rocío University Hospital, Seville, Spain This is an original paper of substantial importance addressing CPG implementation effect. Mostly a very sensible, necessary and well-developed document. In general, the writing is clear and concise, however, needs some editing to avoid unnecessary repetition (i.e. the three domains of Donabedian proposal, and some others). The effect of the implementation of 11 guidelines identified after a rigorous and well-done literature search is presented, as detailed in Methods and Figure 1.
The reviewer has some concerns and suggestions regarding Methods (GRADE use) and Discussion that need to be addressed to improve the manuscript.
An article that is worthy of being indexed.

Methods:
Data extraction, page 4, last paragraph, "An analysis of the quality of the evidence…we assessed the certainty...": It is unclear how this was done. GRADE is a very labour-intensive and specific methodology, and to perform a GRADE evaluation, the authors have two options: a) simply seek the strength of evidence from the original implemented guideline included in the review, or b) evaluate and critically assess original articles cited in the guidelines reviewed, when they did not use GRADE methodology. This needs clarification on what the authors mean. In addition, readers do not need the authors to explain what GRADE method is.

Results:
Quality of evidence assessment, page 7: This section should be clarified in accordance with previous comments on the use of GRADE.

Discussion:
two not statistically significant results. Performing a GRADE table provided the rank of possible grades for the certainty of the evidence found in the 362 interventions from the 11 studies included.
Hence, we do not consider it necessary to modify the text in the section "Quality of evidence assessment". Then, we want to clarify that it is part of the objective and scope of this systematic review to evaluate the certainty of the evidence of the RCTs included and not of the EB-CPGs that they evaluated.

Answer to #B.1:
It is relevant to clarify that our article complies with F1000Research guidelines for preparing a Systematic Review article. The journal requests the main body section as follows: Main Body: The format of the main body of the article is flexible: it should be concise, making it easy to read and review, and presented in a format that is appropriate for the type of study presented. For most Systematic Reviews, the following standard format will be the most appropriate: Introduction, Methods, Results, Conclusions/Discussion.
Also, we consider that the aim of the discussion was achieved because we highlighted our findings and contrasted them with similar research. We must add that we have also complied with the PRISMA 2020 statement checklist (Page et al., 2021[1]). About the discussion, our study meets with the following items: 23a Provide a general interpretation of the results in the context of other evidence. 23b Discuss any limitations of the evidence included in the review. 23c Discuss any limitations of the review processes used. 23d Discuss the implications of the results for practice, policy, and future research.
Therefore, considering that neither of the indications describes an expected order of ideas as you have indicated, the authors alleged it convenient to include this first paragraph to provide a context about the world situation regarding the efforts to develop guidelines and the expectations placed by health caregivers on their effect. Also, to link this finding with the findings in our study.
Consequently, as the first paragraph of this section is congruent with the aim of this systematic review, we consider it appropriate to keep the paragraph unchanged.

B.2 Additionally, regarding your comment about the third paragraph of the discussion section:
The reviewer does not agree with the statements made in the current third paragraph, in the sense of contrasting the effect on the "process of care" with the effect on "patient outcomes". There are RCTs that relate the implementation of guidelines process of care with the appropriateness and necessity of care for patients' benefits (i.e. see  [2]).
We would like to explain the following: Answer #B.2: This statement refers to the findings we got from the first systematic review (Ramírez-Morera et al., 2019 [3]) that focused on the impact of CPGs in the cardiovascular approach; at that time, we included nine RCTs. Therefore, we decided to compare the results obtained in this second SR to our previous research regarding the number of interventions evaluated according to the dimensions described by Donabedian (1988) [4].
This report corresponds to the result of a Systematic Review that includes only Randomized Controlled Clinical Trials (RCTs) whose objective was to measure the impact that Clinical Practice Guidelines have produced on the quality of health care. This review's authors analyzed the RCT measurements' results according to the framework proposed by Avedis Donabedian [5] (7 January 1919-9 November 2000. Therefore, what has been understood by the implementation of clinical practice guidelines, as defined in the UK glossary published by NICE, adheres to the definition mentioned earlier: "implementation, The process of putting guideline recommendations into practice" (NICE 2014) [6] . Evaluation components of CPG use, such as clinical auditing, can be of great value. However, clinical auditing has not been disseminated to be available in other countries, predominantly lowincome countries.
In 2008, Dutch authors Lugtenberg M, Burgers JS and Westert GP published an excellent systematic review [7] using the quality of care model by Donabedian. This publication has been very inspiring for the authors of this series of systematic reviews. This review mainly includes RCT studies and some before and after studies done on CPG made in the Netherlands, a country with a long tradition in the design and use of CPG. Some of their results are encouraging, although other aspects were weak, such as the impact on patient's health, which should be considered a very relevant result.
Due to the nature of the data, it was impossible to have a quantitative synthesis, only qualitative. But it did not inhibit us from assessing the certainty of the evidence for the results of the included studies after searching and screening according to the inclusion criteria.
To determine the confidence of the evidence, we have followed Cochrane's recommendations using the tools developed by GRADE. GRADE is a tool that requires knowledge of the essential elements of effective measurement and its potential biases. The authors of this SR are trained to complete this task with adequate competency.

B.3 You also wrote: Some comment on study weaknesses is missing. Answer #B.3:
We did not write within a specific section of our article what the research's weaknesses are. The weaknesses of the study are described throughout different statements, for example, when describing the risk of bias assessment for the included studies. We also identified many EB-CPGs developed in contrast to the few numbers evaluated through formal RCTtype investigations.
We also wrote about the implications for research: Due to recent research in this field, and the results of this study were not conclusive, more research is necessary to evaluate how EB-CPGs could impact the quality of health care, especially emphasising fewer investigated areas, such as the structure of healthcare services and patient outcomes. Additionally, the design and methodological rigour applied to future RCTs should improve by considering the certainty of the evidence supporting the EB-CPGs recommendations. Besides, focusing on those with a greater level of evidence (high or moderate) which could lead to determining more clearly the effect that these recommendations have on the quality of health care.

C. About your minor comments: C.1 First, about the abstract section you wrote:
Abstract: It is required that the clinical problem addressed in the study (breast cancer) appears in the abstract and keywords.

Answer #C.1:
We agree to add "breast cancer" as a keyword.

C.2 Second, concerning the introduction section, you wrote: Introduction:
In the sixth paragraph: "Although survival has improved…..": There is a huge debate whether the improved survival depends on the early diagnosis or the improvement of knowledge of the natural history and new treatments of breast cancer  [8],  [9]). Thus, some sentences should be complemented with "...of early detection programs and treatment improvement".

Answer #C.2:
We agree to add the suggested text.

C.3 Third, referring to the discussion section, you wrote:
The comments on the usefulness of EB-CPG could be shortened.

Answer #C.3:
The authors alleged it is convenient to describe the usefulness of EB-CPG because it is congruent with the aim of this systematic review. Therefore, we disagree with the modification suggested.
C.4 Fourth and last, you wrote that there are 'partly' details of the methods and analysis provided to allow replication by others.

Answer #C.4:
Full details on methods and further analyses to allow replication by others can also be accessed through the research protocol, registered at PROSPERO: CRD42020205594.