Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review

Background: The development of evidence-based clinical practice guidelines (EB-CPGs) has increasing global growth; however, the certainty of impact on patients and health systems, as well as the magnitude of the impact, is not apparent. The objective of this systematic review was to assess the effectiveness of the application of EB-CPGs for the improvement of the quality of health care in three dimensions: structure, process and results in the patient for the management of cardiovascular disease. Methods: We followed the methods described by the Cochrane Handbook and present a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE databases, as well as the grey literature, between 1990 and June 2016. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Three authors independently carried out the data extraction, using a modified version of the Cochrane Effective Practice and Organization of Care form. Results: Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) were related to health care structure, 54 (64%) to the health care delivery process and 27 (32%) to patient outcomes. Regarding the impact of using the EB-CPGs, in 55 interventions (65%), there were no significant differences between control and experimental groups. In four interventions (5%), the result favoured the control group, and the result favoured the intervention group on 25 of the interventions (30%). Conclusions: This systematic review showed that EB-CPGs could be useful to improve the process and structure of health care and, to a lesser extent, to improve the patients’ outcomes. After analysing many studies, we could have one more hypothesis for further research, which could shed more light upon those undiscovered variables that might interfere with the use of the EB-CPGs. Registration: PROSPERO CRD42013003589


Introduction
This review refers to the changes in the quality of healthcare services that are direct consequences of the systematic use of evidence-based clinical practice guidelines (EB-CPGs). "Evidencebased" means that the recommendations are created using rigorous, unbiased and transparent methods of collation and appraisal, alsousing scientific findings of the highest quality and value to assist in providing optimal clinical care to the patient. (Guyatt, 1992;Sackett et al., 1996). EB-GPCs are evidence summaries and include systematically developed recommendations to assist physicians and patients in the process of making decisions (Alonso-Coello et al., 2010;Glasziou et al., 2011). EBCPGs are one of the most available and widely used tools for translating the knowledge generated by scientific research to practice. They should be based on the best scientific evidence available in order to improve the quality of patient care (Alonso-Coello et al., 2010;Grimshaw & Russell, 1993). Experts, institutions, and organizations worldwide have developed EBCPGs in all areas of medicine as a key tool to improve the quality of care Donabedian, 1988;Glasziou et al., 2011;Lugtenberg et al., 2009;Moore et al., 2015 states that before evaluating health care, we must first decide how to define "quality" and whether it depends only on the actions of physicians or if it also depends on contributions from the patients and the healthcare system.
Defining quality is challenging since it is not easy to characterise coherently and objectively. Health must be analysed from a holistic point of view, and guideline developers must determine the ideal amount of influence health should receive from individual preferences and social components. We must also understand the relationship between structural characteristics and healthcare processes, as well as their results in health services (Donabedian, 1988;Moore et al., 2015).
Since the 1990s, increasing numbers of EB-CPGs are being developed. However, it is unknown whether these high-quality recommendations have a beneficial impact on patient health. Despite the high number of recently published EB-CPGs, there are few studies on their effectiveness in improving clinical outcomes, the process and structure quality throughout the healthcare system.
After an exhaustive search, only two systematic reviews (SR) we found on this topic (Lugtenberg et al., 2009;Worrall et al., 1997). Worrall et al., 1997  This review sought to assess whether the quality of health care improves in patients with the cardiovascular disease when using EB-CPGs vs standard professional medical practice. The primary aim was to assess the impact of the EB-CPGs for the management of cardiovascular diseases on healthcare quality, in terms of patient outcomes, management process, and healthcare structure.

Methods
We designed a methodology aimed to find and analyse studies measuring the impact of EB-CPGs on the improvement of quality in health care services in the three areas proposed by Donabedian (Donabedian, 1988;Moore et al., 2015): structure, process, and patient outcomes. From the very beginning of the process when the authors wrote this review protocol, it was clear that these items would not be easy to measure and, as the search and data extraction moved forward, it became harder to synthesise the information delivered by the different studies included. The main obstacle was the inconsistency observed in the different outcome measures used by the studies, which included continuous as well as dichotomous values for different clinical conditions and interventions. The intervention definition read as any planned action taken to modify the clinical practice and use of the clinical guidelines for influencing in the clinical practice. The authors followed the methodological recommendations described in the Cochrane Handbook (Higgins & Green, 2012). This review is registered with PROSPERO (ID: CRD42013003589).

Study searches
We did a systematic search using the following electronic databases for primary studies (randomised controlled trials (RCTs)): Cochrane Central Register of Controlled Trials (CENTRAL). The Cochrane Library, including the Cochrane EPOC (Cochrane Effective Practice and Organization of Care) specialised database; MEDLINE; EMBASE; CINAHL; PsycINFO; LILACS; Health Technology Assessment Databases and Web of Science, Science Citation Index, and Social Sciences Citation Index.
The review authors combined search strategy for indexed terms and developed free text terms. We included searches of grey literature in different sources, such as reports of the world and regional conferences, academic theses and scientific reports not published in indexed journals. We searched for studies published between January 1990 and June 2016, without any language restriction. An advanced search strategy is available as Extended data, Appendix 1 (Ramirez et al., 2019).

Data extraction
Three authors independently performed data extraction using a modified shorter version of the Cochrane Collaboration EPOC Data Collection Checklist translated into Spanish. Apart from preparing the Spanish version, we eliminated several items not applicable to this review, given that we only included RCTs. We used a standardised digital form for data extraction and analysis. We used a standardised digital form for data extraction and analysis. We used Review Manager software (RevMan 5.3) for the data analysis.
We assessed the risk of bias (quality) according to the Cochrane Handbook (Higgins & Green, 2012). We found very high variability, so the study results were introduced as a narrative in the Results.
Because of the variability between the measurements of the effect of the impact of EB-CPGs on the change of quality in the studies included in this review, it was not possible or appropriate to perform a meta-analysis; therefore, it was not possible to measure the statistical heterogeneity.

Study identification and selection
When the first version of the protocol of this Systematic Review was initiated, consider the inclusion of several clinical topics in a single systematic review. We decided to create a series of three systematic reviews on different clinical topics: Cardiovascular Health, Breast Cancer and Child and Mother Health. The initial searches included all those topics with the same inclusion criteria described. After removing duplicates, the search produced 4279 potential studies. After screening by title and abstract, 4051 were excluded. After the full-text evaluation, 96 studies of all clinical subjects were selected, extended data, Appendix 2 (Ramirez et al., 2019).

Study characteristics
For the analysis, we organised the studies according to the topic or pathology being the core subject of the EB-CPG, and for this report, we only selected RCTs on cardiovascular diseases, because this theme accounts for the higher number of original articles. In total, we selected nine RCTs (Figure 1). Characteristics of the included studies are available as Extended data, Appendix 2 (Ramirez et al., 2019).
We included nine RCTs analysing cardiovascular diseases (Beaulieu et al., 2004;Berner et al., 2003;Ellis et al., 2000;Guadagnoli et al., 2004;Hand et al., 2014;Jäntti et al., 2007;Kiessling & Henriksson, 2002;Tierney et al., 2003;Tsuyuki et al., 2015). All trials were carried out between 2000 and 2015, five in the United States of America, two in Canada, one in Finland and one in Sweden. Eight articles addressed outpatient care and three inpatient care. The EB-CPGs included in the selected trial looked at the following clinical problems: management of stable angina pectoris (over 65 years), unstable angina, dyslipidaemia, acute infarction, heart failure and blood pressure. Besides, the trials included perioperative cardiac evaluations of patients with non-cardiac surgery, cardiopulmonary resuscitation and secondary prevention in patients having coronary artery disease.
A number of the 4,279 studies found in the initial search were not randomised clinical trials, as the authors described in their titles or abstracts. When assessing the full-text articles, we found that many were cluster trials and observational studies with a "before and after" design.

Quality of evidence assessment
As for the quality of the evidence, we observed the presence of a high or unclear risk of bias for allocation concealment (selection bias), blinding of participants and personnel (performance bias), and blinding of outcome assessment (detection bias), which can be explained by the nature of the interventions studied. We found several types of systematic errors: random sequence generation (selection bias), incomplete outcome data (attrition bias) and selective reporting (reporting bias). We found that the interventions measured yielded outcomes assessed with moderate to low evidence certainty according to the GRADE classification.

Risk of bias assessment
Analysing the risk of bias in the nine included RCTs studies found the random sequence generation (selection bias) assessment had a low risk of bias (between 50-75%) for allocation concealment (selection bias) 33% low, 33% unknown and 33% high risk of bias. The blinding of participants and personnel (performance bias) obtained almost 50% high risk of bias. The blinding of outcome assessments (detection bias) obtained a 50% low and 50% uncertain risk of bias. Incomplete outcome data (attrition bias) obtained between a 50 and 75% low risk of bias. For the selective reporting (reporting bias), there was a 100% low risk of bias. Finally, other types of bias had a low risk of between 50 and 75% ( Figure 2).
The studies with the lowest and highest risk of bias (Kiessling &Henriksson, 2002 andJäntti et al., 2007, respectively) were chosen to be evaluated with the GRADE methodology and to build a summary of findings table. The aim of doing this GRADE table was obtaining the rank of possible grades of the evidence certainty found in the 84 interventions from the nine studies included. The results of these studies produced moderate-to-low evidence certainty, according to the GRADE classification (see Table 1 and Table 2).

Assessment of study outcomes
The authors grouped the outcomes in simple relative and absolute numbers. There was not a global estimate of the measurements of the effects included in the studies because of the considerable variability of measuring units, as well as the clinical heterogeneity found among the studies included. The measurement of the outcomes reported in the studies was dichotomous, continuous or nominal; the majority were dichotomous and were used to measure the mastery of the process, e.g. the number of patients receiving an adequate treatment following a recommendation versus those who did not.
Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) corresponded to the health care structure dimension, 54 interventions (64%) to the dimension health care delivery and 27 interventions (32%) to the dimension of patient outcomes. Regarding the impact of EB-CPG use, we found that in 55 interventions (65%) there was no significant difference between the control and experimental groups.
In four interventions (5%) the outcome favoured the control group (comparison of the measure of compliance of the recommendations (the average adjusted by the patient characteristics, the setting, and the number of measures applied per patient)). The measure of effects (odds ratio (OR)) regarding the conditions acute myocardial infarction and heart failure, separated by care provided by a cardiologists or primary care physicians, were as follows: acute myocardial infarction, cared by a cardiologist OR 0.81 (95% CI: 0.79 to 0.83), and cared by a primary care physician OR 0.73 (95% CI: 0.71 a 0.76); heart failure, cared by a cardiologist OR 0.88 (95% CI: 0.86 to 0.90), and care by a primary care physician OR 0.79 (95% CI: 0.76 to 0.81).
The result favoured the intervention group for 25 interventions (30%). Some of the recommendations were: use of antiplatelet medication during the 24-hour hospital stay, a study of the left ventricle ejection fraction, total cholesterol and LDL measurements, cardiopulmonary resuscitation, and degree of compliance with EB-CPGs recommendations.

Discussion
Methodological efforts have been made to develop trustworthy EB-CPGs. However, it still seems that some of these EB-CPGs are far from the reality of the clinical practice (Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, 2011).
It is essential to emphasise the main findings from the outcome analysis. It is surprising that most of the studies did differentiate between the control and experimental groups regarding the improvements with the use of EB-CPGs. The effects of recommendations of the interventions included in the nine RCTs on the areas of health care structure (4%) and patient outcomes were the least studied (32%). This fact could lead us to assume that researchers have given more importance to  We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Low certainty: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect Very low certainty: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect Explanations 1. For the generation of the random sequence (selection bias) the risk is High: sealed, numbered and opaque envelopes were used for the randomisation of the cases to be treated. It is not clear why people with less work experience or no academic degree were assigned to the ERC 2000. For the allocation concealment (selection bias) the risk is High: It is likely that by the nature of the study the allocation concealment of the selection could not be made. For blinding of participants and staff (performance bias) the risk is High: It is likely that the nature of the study could not prevent participants from knowing which group they belonged to (which CPG they were using). For blinding of the outcome evaluation (detection bias) the risk is Low: A computer automatically collected it. For incomplete results data (attrition bias) the risk is Low: There was no loss of follow-up since it was a single session. For the particular report (notification bias) the risk is Low: A computer automatically collected it. In other risks of bias, the risk of bias is High: The description of the study design was not clear; therefore, we assumed that the study has a high risk of bias.
the evaluation of using (or not using) the recommendations in the area of the process, instead of their direct impact on the patient health.
The changes observed in the patient progression were generally modest. In the six studies evaluating patient outcomes (27 interventions), four of them reported a positive result for seven measurements (26%) of the interventions. For example, the study by Ellis et al. (2000) measured the difference in the average change of total cholesterol (mg/dl) and reported a decrease of 7.4 mg/dl in favour of the intervention. It is essential to mention that these measures were mostly surrogate variables. Only one study reported variables for clinical results (Tierney et al., 2003), with 13 interventions, but found no meaningful differences in the results between the groups.
Most of the studies (eight) reported results in the process area and showed favourable results for the intervention in six studies with 16 measurements. This fact reflects the reluctance of health care providers to use EB-CPGs and means that only 30% of these interventions followed the recommendations given by the EB-CPGs.
Our findings in this review coincide with some of those captured by Gagliardi et al. (2012) and Kastner et al. (2015) most of the studies included in this review used multiple strategies Is the active implementation more effective compared to passive implementation as it was used in the control group to improve secondary prevention of coronary artery disease?
Patient or population: person with coronary artery disease.
Intervention: Active implementation (general practitioners participated in learning dialogues using the CPG on secondary prevention of coronary artery disease, method of recurrent cases in their primary care centers) Control: Passive implementation (the CPG was mailed to general practitioners and presented at a conference) For the generation of the random sequence (selection bias) the risk of bias is Uncertain: Not described. For allocation concealment (selection bias) the risk is low: general practitioners and patients did not know in which research group they were assigned. For blinding of participants and staff (performance bias) the risk is low: General practitioners were not aware of being involved in the study and a blinded nurse on which group each patient belonged to, was the one who handled the paperwork, protocols of the investigation and had no contact with general practitioners. For blinding of the outcome assessment (detection bias), the risk is Low: The research codes and databases were not disclosed until the authors completed the statistical analysis. For incomplete results data (attrition bias), the risk is Low: The study used the intention-to-treat analysis and indicated that the follow-up for two years was 86%. For the particular report (notification bias) the risk is Low: The research codes and databases were not disclosed until the authors completed the statistical analysis. For Other biases the risk is Low: None known 2. p < 0.05 3. p > 0.05. Downgraded -1 for imprecision.
to implement the EB-CPGs (electronic notifications in a digital file, cell phone applications, letters, phone call memos, printed material), and the authors do not tailor to every recommendation (the implementation strategy is usually the same for every recommendation within a EB-CPG). Cosby (2006) formulates very clearly that although clinicians have the new evidence at hand, using it means a change in the behaviour and habits of the clinical management of patients that is complex to achieve unless appropriate strategies are applied. A more specific approach, based on the results of the analysis of the obstacles hindering the adoption of every recommendation separately, might improve the use and effect of the recommendations in practice.

Conclusions
There is an imbalance between the number of EB-CPGs developed and the number of high-quality studies evaluating their effectiveness. After analysing many studies, we can have one more hypothesis for further research for more light upon those undiscovered variables that might interfere with the use of the EB-CPGs. Therefore, more studies of good quality are still needed.
The variation in the effects of the recommendations of the EB-CPGs suggests that it would be useful to focus on the analysis of the adherence limitations, as well as on designing implementation strategies by adapting every recommendation, instead of considering the EB-CPGs as a whole. Further research is still needed to determine which factors related to the EB-CPGs and their specific recommendations are essential to predict the use of EB-CPGs, and thus achieve better patient outcomes.

Implications for practice
The initial objective of this review was to strengthen the development programs for EB-CPG by evaluating their effects on the quality of health care and to give reliable evidence to sustain the decision-making process related to the construction of EB-CPGs in medium-and low-income countries. Even the research evidence is not strong enough to support the EB-CPGs as a tool to improve the practice for a better quality of care, the results of this review need to be interpreted with caution. Definitely. It seems the standard application used so far must be reviewed and must incorporate new psychosocial strategies oriented toward driving change in clinical practice and the doctor-patient relationship We need reaching the right hands at the right time.
It is necessary to emphasise that the standard CPG implementation used so far must be reviewed, and must incorporate new psychosocial strategies oriented toward driving change in clinical practice and the doctor-patient relationships.
For an adequate implementation of a EB-CPG, it is necessary to take into account the possible costs, risks, and benefits that will be necessary to know the expected results precisely. The health systems that build EB-CPGs must maximise their efforts to get health care personnel to follow the recommendations of the EB-CPGs and make an effort to evaluate their impact.

Implications for research
This is the first of series of three systematic reviews on the effects of Evidence-Based Clinical Practice Guidelines (EB-CPGs) on health care quality improvements. The second RD will cover Breast Cancer and the third child and maternal health. We created an interactive https://lnkd.in/dwTxaD8 LIVE GUIDELINES program following the CDC https://lnkd.in/d5r4z6m Since research in this field is so new, and the results of this study were not conclusive, more research is needed to evaluate the change that EB-CPGs could make to the quality of health care, emphasising the less studied areas, such as the structure of health care services and patient outcomes.

Data availability
Underlying data All data underlying the results are available as part of the article and no additional source data are required.

Ignacio Marín León Fundación Enebro, Seville, Spain
Thank you for the invitation to review this well-conducted systematic review.
The study under review is very interesting and necessary for the Clinical Practice Guidelines (CPGs) community and for the implementation science and healthcare quality communities. I would like to congratulate the authors for this difficult challenge with some minor comments to contribute to improve the content and message of this necessary study to improve the content and message of this necessary study

ABSTRACT: Conclusion "There are probably still undiscovered variables that interfere with the use of the CPGs".
That could not be a conclusion from the study, given that the aim of the work was not to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, then no conclusion could be provided. A coherent conclusion could be the need to improve and systematize this kind of implementation studies.
pag 3, para 4, left column Introduction: "After an exhaustive search, only two systematic reviews (SR) we found on this topic," It is needed to provide both references

RESULTS: pag 3, 4 and figure 1. Study identification and selection. "Of these, we selected 96 studies after assessing the full texts".
It is needed, in order to strengthen the transparency and reproducibility of the work, to display in an extended data appendix, the full reference list of rejected originals and reasons to be rejected (96 found minus 9 selected = 87 rejected). It should be clarified what is meant in the figure 1 tab "# 87 of studies to include in other analyzes".

Pag 4, left column, para 3, rows 4-6:
"When assessing the full-text articles, we found that many were cluster trials" There is no clear reasons to reject RCT clusters studies, when the intervention could be randomized to cluster offices, and the outcomes effect measured at individual patient level. It is a well-accepted methodology in implementation science, given that can control for bias and non-measured variables. (Please see 2 references provided: Eccles and Romero). ,

Pag 5, Table 1, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)"
It is difficult to understand why the certainty of evidence for the outcomes "time without flow" and "delay to start CPR" are classified differently when, as displayed in the explanations, the bias and systematic errors are the same, and there is not a loss of patients to follow up. It is true that both effects measure score different statistical significance, but with data provided the readers feel with the same certainty (either moderate or low) that one effect is significantly different from the control group, and for the other outcomes (delay) there are not differences with the control group.

Pag 6, Table 2, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)"
The same reasoning for the difficulty to understand why are classified differently the certainty of evidence for the outcomes "LDL cholesterol" and "Total cholesterol".

"This fact could lead us to assume that researchers have given more importance to the evaluation of using (or not using) the recommendations in the area of the process, instead of their direct impact on the patient health".
Another explanation could be to emphasize that it use to be easier to measure process of care than patient health outcomes. These health outcomes take a longer period of observations than to measure how professionals adopt CPG recommendations. That can explain the "preferences" of researchers.

Conclusions
The same considerations exposed in the abstract regarding the sentences "There are probably still . undiscovered variables that interfere with the use of the CPGs" That could not be a conclusion from the study, given that was not the aim of the work to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, them no conclusion could be provided.
A coherent conclusion could be the needs to improve and systematize these kinds of implementation studies.

Are the conclusions drawn adequately supported by the results presented in the review? Partly
No competing interests were disclosed.

Competing Interests:
Reviewer Expertise: Internal Medicine. APPROVED WITH RESERVATIONS Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting and necessary for the Clinical Practice Guidelines (CPGs) community and for the implementation of science and healthcare quality communities. I would like to congratulate the authors for this difficult challenge with some minor comments to contribute to improving the content and message of this necessary study

#1 ABSTRACT: Conclusion
"There are probably still undiscovered variables that interfere with the use of the CPGs". That could not be a conclusion from the study, given that the aim of the work was not to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, then no conclusion could be provided. A coherent conclusion could be the need to improve and systematize this kind of implementation studies. We change the text for a better understanding of our meaning. After analyzing many Answer: studies, we can have one more hypothesis for further research for more light upon those undiscovered variables that might interfere with the use of the CPGs.

#2 Introduction: pag 3, para 4, left column "After an exhaustive search, only two systematic reviews (SR) we found on this topic,"
It is needed to provide both references We agree and change for: After an exhaustive search, only two systematic reviews (SR) Answer: we found on this topic (Worrall, 1997;Lugtenberg, 2009). Worral (1997  "Of these, we selected 96 studies after assessing the full texts". It is needed, in order to strengthen the transparency and reproducibility of the work, to display in an extended data appendix, the full reference list of rejected originals and reasons to be rejected (96 found minus 9 selected = 87 rejected). It should be clarified what is meant in the figure 1 tab "# 87 of studies to include in other analyses". We agree . The Appendix 2: List of studies selected after screening and assessing the full Answer: text is added at OSF Home, the link is: https://osf.io/9a5fm/?view_only=0ec7b68ed1004bf6b1aa74a19ed1912d We did not reject the 87 publications, not analyzed. This SR is the first of series that has three https://osf.io/9a5fm/?view_only=0ec7b68ed1004bf6b1aa74a19ed1912d We did not reject the 87 publications, not analyzed. This SR is the first of series that has three systematic reviews. This two new ones will include the topics breast cancer, type 2 diabetes and others.

#4 Pag 4, left column, para 3, rows 4-6:
"When assessing the full-text articles, we found that many were cluster trials" There is no clear reasons to reject RCT clusters studies, when the intervention could be randomized to cluster offices, and the outcomes effect measured at individual patient level. It is a well-accepted methodology in implementation science, given that can control for bias and non-measured variables. (Please see 2 references provided: Eccles and Romero). , The authors decide when preparing the protocol to include only RCT. Answer: Table 1, Rows 2-4 and Column 5: "Certainty of the evidence (GRADE)" It is difficult to understand why the certainty of evidence for the outcomes "time without flow" and "delay to start CPR" are classified differently when, as displayed in the explanations, the bias and systematic errors are the same, and there is not a loss of patients to follow up. It is true that both effects measure score different statistical significance, but with data provided the readers feel with the same certainty (either moderate or low) that one effect is significantly different from the control group, and for the other outcomes (delay) there are not differences with the control group.
Please check the note explanations both have different statistical significance due to this Answer: fact the certainty of evidence is downgraded for "Delay to start CPR (s)".

#6 Pag 6, Table 2, Rows 2-4 and Column 5:
"Certainty of the evidence (GRADE)" The same reasoning for the difficulty to understand why are classified differently the certainty of evidence for the outcomes "LDL cholesterol" and "Total cholesterol".
Please check the note explanations both have different statistical significance due to this Answer: fact the certainty of evidence is downgraded for "Difference in percent change in total cholesterol at 2 years [Mean (mmol / L)]". #7 Discussion: Pag 7, para 2, rows 7-13 "This fact could lead us to assume that researchers have given more importance to the evaluation of using (or not using) the recommendations in the area of the process, instead of their direct impact on the patient health". Another explanation could be to emphasize that it use to be easier to measure process of care than patient health outcomes. These health outcomes take a longer period of observations than to measure how professionals adopt CPG recommendations. That can explain the "preferences" of researchers. You are right. Answer:

#10 Conclusions
The same considerations exposed in the abstract regarding the sentences "There are . probably still undiscovered variables that interfere with the use of the CPGs" That could not be a conclusion from the study, given that was not the aim of the work to study factors and conditions that facilitates the implementation of CPG neither the conditions to change professional behavior. No data on these issues are reported, them no conclusion could be provided. A coherent conclusion could be the needs to improve and systematize these kinds of implementation studies. implementation studies.
We change the text for a better understanding of our meaning. After analyzing many Answer: studies, we can have one more hypothesis for further research for more light upon those undiscovered variables that might interfere with the use of the CPGs.
We are the author of this SR. ' ' these SRs Please add citations for these two references (SRs).
'...and international perspective together.' What do you mean by an 'international perspective'; of what; (i.e. of CPGs, of healthcare quality, or of cardiovascular healthcare?), please clarify or re-write.
page 3, right column, para 4: Regarding the data extraction modified and translated from the EPOC checklist, was it validated and if not, why the authors felt they did not need to? Additionally, please provide the readers who would be interested in the replication of this research with this tool as an online supplementary here. It was indeed mentioned with a link in the PROSPERO protocol ( , attached on Appendix 3) but the link did not https://fs20.formsite.com/mtristan/form41/index.html work, giving an error! Moreover, I checked the (Extended data) links in the OSF Home but could not retrieve it.
Please delete the duplicated sentence; 'We used a standardized digital ..........and analysis'. page 3, right column, para 7: The #87 studies that were planned to be included in another analysis were not mentioned here since the final included were 9 rather than 96 studies. Please elaborate on this in this location. Tables 1 and 2.: the GRADE quality assessments and the explanations for the certainty of evidence were very well conducted and written. '. patient outcomes page 7, left column, para4: Discussion: '..However, it still seems that some of CPGs are...' these page 7, right column, para 2: As part of discussing 'multiple strategies to implement the CPGs', I would recommend adding insights and citing these relevant keynote articles that discuss multi-faceted CPGI strategies and their importance:  Implement Sci. 2011;6:26. doi: 10.1186/1748 page 7, right column, para 4: '...... as well as on designing implementation strategies adapt every recommendation...' Do you mean "designing implementation strategies AND adapt every recommendation....if so add a comma ',' " or "designing implementation strategies (by) adapting every recommendation". Additionally, I suggest to add: instead of considering '....initiate a pilot implementation of specific recommendation(s) the CPGs as a whole'. implementation of page 7, right column, para 5: Implications for practice The initial objective of this review was to strengthen the development programs for CPGs (plural) The results of this review must need to be interpreted with caution. It reads better either (the results of this review interpreted with caution.) or (the results of must be this review interpreted with caution.). need to be 'It is necessary to emphasise that the standard implementation used so...' CPG page 7, right column, para 6: 'For an adequate implementation of a CPG, it is necessary to take into account the possible costs, benefits risks, and Finally, I would like to congratulate the authors once more on this work and invite all interested researchers to conduct similar research using this robust methodology to study the impact of EB-CPG implementation on all different healthcare specialties in addition to their effects on the interdisciplinary workflows and transitions of care between different specialties and services in healthcare. Thank you for the invitation to review this well-conducted systematic review. The study under review is very interesting for the evidence-based clinical practice guidelines (CPGs)-and for the healthcare quality and safety-research communities. Moreover, I would like to congratulate the authors for this work with some minor comments with the aim of improving the content and message of this distinguished study.

Abstract:
I recommend replacing the phrase 'results in patients' with as it is more ' ' patient outcomes commonly used in the field of quality and safety in healthcare. Additionally, the authors have used it for the rest of the MS.
We agree and we will make the change. Answer: 1.

1.
The #87 studies that were planned to be included in another analysis were not mentioned here since the final included were 9 rather than 96 studies. Please elaborate on this in this location.
We agree, and we will make the change. And we add the following: Answer: When the first version of the protocol of this Systematic Review was initiated, consider the inclusion of several clinical topics in a single systematic review. We decided to create a series of three systematic reviews on different clinical topics: Cardiovascular Health, Breast Cancer and Child and Mother Health. The initial searches included all those topics with the same inclusion criteria described. After removing duplicates, the search produced 4279 potential studies. After screening by title and abstract, 4051 were excluded. After the full-text evaluation, 96 studies of all clinical subjects were selected. For this SR, only nine studies covering the cardiovascular issue were included. This is the first of series of three systematic reviews on the effects of Evidence-Based Clinical Practice Guidelines (EB-CPGs) on health care quality improvements. The second RD will cover Breast Cancer and the third child and maternal health. We created an interactive https://lnkd.in/dwTxaD8 LIVE GUIDELINES program following the CDC https://lnkd.in/d5r4z6m '. analyses We agree, and we will make the change Answer: Tables 1 and 2.: the GRADE quality assessments and the explanations for the certainty of evidence were very well conducted and written. We agree, and we will make the change Answer: page 7, left column para 1: Once more, please replace 'patient results' with ' '. patient outcomes We agree, and we will make the change Answer: page 7, left column, para4:Discussion: '..However, it still seems that some of CPGs are...' these We agree, and we will make the change Answer: page 7, right column, para 2: As part of discussing 'multiple strategies to implement the CPGs', I would recommend adding insights and citing these relevant keynote articles that discuss multi-faceted CPGI strategies and their importance:  1.

1.
Thank you very much for the recommended references. We agree that they will give Answer: strength to the implementation aspect, one of the most sensitive components for the use of CPG. In relation to the reference Gagliardi 2011, We will not take it in the article because the Research Implementation Guide and Application Network (GiRAnet), is not active and in its link http://www.cebhc.co.za/guideline-implementability-research -y-application-network-giranet-2 /, one link does not work and the other is restricted access. We had access to another article by the same author of 2012, which we share ideas that we have already implemented and this study is in protocol: Gagliardi, A. R., Brouwers, M. C., & Bhattacharyya, O. K. (2012). The guideline implementability research and application network (GIRAnet): an international collaborative to support knowledge exchange: study protocol. Implementation Science, 7, 26. https://doi.org/10.1186Science, 7, 26. https://doi.org/10. /1748 The new paragraph reads as follows: Our findings in this review coincide with some of those captured by Gagliardi (2012) and Kastner (2015) most of the studies included in this review used multiple strategies to implement the EB-CPGs (electronic notifications in a digital file, cell phone applications, letters, phone call memos, printed material), and the authors do not tailor to every recommendation (the implementation strategy is usually the same for every recommendation within a EB-CPG) Cosby (2006) formulates very clearly that although clinicians have the new evidence at hand, using it means a change in the behaviour and habits of the clinical management of patients that is complex to achieve unless appropriate strategies are applied. A more specific approach, based on the results of the analysis of the obstacles hindering the adoption of every recommendation separately, might improve the use and effect of the recommendations in practice.
page 7, right column, para 4: '...... as well as on designing implementation strategies adapt every recommendation...'Do you mean "designing implementation strategies AND adapt every recommendation....if so add a comma ',' " or "designing implementation strategies (by) adapting every recommendation". Additionally, I suggest to add:'....initiate a pilot implementation of specific instead of considering the CPGs as a whole'. recommendation(s) implementation of We partially agree, and we will make the change. We will not mention or suggest of pilot Answer: program implementation because most of them majority yield very little or nothing. page 7, right column, para 5: Implications for practice The initial objective of this review was to strengthen the development programs for CPGs (plural) We agree, and we will make the change Answer: The results of this review must need to be interpreted with caution. It reads better either (the results of this review interpreted with caution.) or (the results of this review must be need to interpreted with caution.). be We agree, and we will make the change Answer: 'It is necessary to emphasise that the standard implementation used so...' We CPG Answer: agree, and we will make the change page 7, right column, para 6: 'For an adequate implementation of a CPG, it is necessary to take into account the possible costs, benefits risks, and We agree, and we will make the change Answer: We agree, and we will make the change Answer: We are the author of this SR.

Competing Interests:
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com