Comments about the appraisal of systematic reviews in restorative dentistry [version 2; peer review: 2 approved, 1 approved with reservations]

Adequate adoption of evidence-based practice is deeply rooted in accessing methodological quality and completeness of systematic reviews and meta-analyses reporting. Nonetheless, this assumption might be flawed if the methodological quality assessment has not been properly conducted. Taking the former statement into consideration, this correspondence article encourages the improvement of future tertiary manuscripts, especially in the field of restorative dentistry. Thus, this article addresses an overview of reviews in restorative dentistry by Sarkis-Onofre et al. in the May 2019 issue of the Journal of Esthetic and Restorative Dentistry as an example of evaluating tertiary evidence for increasing the awareness of reviewers, authors, and readers.


Amendments from Version 1
Following the suggestions of the reviewers, I have used an "overview of reviews" or a "tertiary study" instead of a methodological quality assessment.

REVISED
Comments about the appraisal of systematic reviews in restorative dentistry Dear respectable Advisory Editors and readers: I read with great interest the publication by Sarkis-Onofre et al. 1 "Systematic reviews in restorative dentistry: discussing relevant aspects," in the Journal of Esthetic and Restorative Dentistry in the May 2019 issue. This well-written overview of reviews or systematic review of systematic reviews stated that "This study was not registered in PROSPERO" since PROS-PERO indicates that "Reviews of methodological issues need to contain at least one outcome of direct patient or clinical relevance in order to be included in PROSPERO." Interestingly, despite the fact that the above referenced tertiary study falls in the PROSPERO's review of reviews category, it was neglected to being classified as such. Additionally, if PROS-PERO was not an option, the a priori protocol could have been easily registered in another repository. Therefore, the authors' arguments not to register their protocol in PROSPERO are not valid.
Moreover, the authors 1 mentioned that the previous version of their review 2 has a protocol available upon request. However, their first paper 2 , which is in a Brazilian University Magazine printed in Portuguese language, does not support the updated version of the review properly since their first version does not consider any protocol in the text.
Their critical appraisal using AMSTAR-2 appears in Table 2. Five out of the 16 included review studies in their review of reviews, had between one to four AMSTAR-2 items referred as "Authors reported different information by e-mail however, it was not found in the article." This reporting method may not be the most scholarly or safest to present their findings, especially when the authors of their included review studies kindly accepted to provide further clarification about their methodology.
Particularly, I am an author of Afrashtehfar et al. 3 "Failure rate of single-unit restorations on posterior vital teeth: a systematic review," and I regretted the online communication with their corresponding author when I was requested to provide further information. Perhaps Sarkis-Onofre et al. should have dedicated more time to conduct an adequate assessment 4 . For example, their negative categorization of the AMSTAR-2 items 4, 7, and 16 for Afrashtehfar et al. 3 may be mistaken. A comprehensive literature search (item 4) can be considered in the former paper 3 since it searched for published papers for over 20 years with no language restriction, using four databases and displaying each search strategy in the Appendix section. Additionally, the review hand searched eight journals and also screened manually in the reference list of all identified relevant primary studies and related secondary studies 3 . Next, the list of excluded primary studies and justifications (item 7) were provided in Supplemental Table 6 (i.e., full-text excluded articles and reasons for exclusion). Regarding any potential sources of conflict (item 16), it is well-stated on the first page of the review that this study was "Supported in part by a Knowledge Transfer Grant from the Network for Oral and Bone Health Research." Additionally, there is a section at the end of the paper for Acknowledgements where librarians and statisticians were thanked for their consultancy 3 .
Moreover, the search and eligibility criteria for Sarkis-Onofre et al. 1 were systematic reviews that met PRISMA-P, including adults over 18 years of age with direct composite resin restoration in posterior teeth compared with other materials or techniques used in posterior teeth regardless of the outcome up to October 15 th , 2017. However, some articles that fully suited their inclusion criteria were not included. For example, Afrashtehfar et al. 5 "Failure of single-unit restorations on root filled posterior teeth: a systematic review" was not included despite being available from November 21 st , 2016. Therefore, their search strategy and their search conduct (including the elimination of duplicates) 6 , as well as screening 7 , raise some serious methodological concerns 8 .
Their overview of reviews has a collaboration with well-known evidence-based medicine experts from Canada, Tricco and Moher 9 , which usually rely on the talent from their research team for screening and assessing the literature.
After a brief analysis, this letter encourages the improvement of future synthesis of studies including their quality assessment to: ▪ Address clarification with authors of potentially included studies safely and respectfully to avoid accusation, especially if there is no consensus on the matter from different experts (i.e., two experts as a minimum).
▪ Take the time and effort necessary to assess the review paper of interventions according to AMSTAR-2 4 . At least two experts in the field should also determine this instead of two research students.
▪ Spend sufficient time with expert librarians to develop an adequate search strategy in multiple databases.
▪ Use a reference manager and do not rely on removing the duplicates by selecting only one category (i.e., authors' names). Thus, the available categories should be combined to avoid removing records that may share the same publication journal, year, or authors.
▪ A PRISMA checklist 10,11 should be submitted, reporting compliance with each item by indicating the paragraph and page where they can be identified in the review/ appraisal. All the required reporting should be included in the quality assessment to ensure transparency and validity.

Data availability
Underlying data No data are associated with this article. Open Peer Review of a JERD-published overview of reviews that also included one of his SRs. In essence, this piece is not thought of as a standalone paper but is rather aimed at the already published overview like a correspondence with points to be addressed by the authors of the original overview in JERD. Therefore, I believe this would be more appropriate to be submitted as such in JERD and not here (not to mention that it is significantly more probable that it will be read and will be replied upon by the overview authors). Thanks for letting me see this.

Authorship contribution based on the ICMJE criteria
Specific comments Abstract. "accessing" is probably "assessing". 1.
Bear in mind that methodological quality is different from the risk of bias. I'm not sure which of the two you are implying here, but it makes sense that it is the latter.

2.
"The improvement of future methodological quality assessment manuscripts": this needs to be reformulated more succinctly.

3.
"Thus, this article addresses a methodological quality assessment about systematic reviews in restorative dentistry by Sarkis-Onofre et al. in the May 2019 issue of the Journal of Esthetic and Restorative Dentistry as an example of evaluating appraisals of reviews for increasing the awareness of reviewers, authors, and readers." Instead of relaying what this piece does, better relay its' scope/aim and its final recommendations.

4.
Text. The cited paper is neither "methodological quality assessment of systematic reviews" (methodological quality assessment is a procedure) nor a "systematic review of systematic reviews"-a systematic review addresses a clinically-based research question, ideally based on PICO. The cited paper is an overview of systematic reviews.

5.
The point about lack of registration in PROSPERO is a valid one, according to Sarkis-Onofre et al., since it complies with the instructions of PROSPERO. However, all studies that have an a priori protocol can register it (and benefit from this) and many different appropriate repositories exist.

6.
I'm not sure what is the point about the protocol being mentioned but not given. It doesn't matter what the authors say: if there is no publicly available registered/time-stamped protocol, there is none.

7.
Likewise, I'm not sure what is meant with the 3 rd paragraph. Information provided through e-mails might not necessarily be regarding the same way as published information, but can still be used in a scientific publication, if deemed appropriate.

8.
The paragraph about the Sarkis-Onofre et al. authors wrongly judging some AMSTAR 2 fields of your SR seem to be valid, however.

9.
Likewise, the comment about potentially missed SRs to include seems to be valid. 10.
The last bullet point might not necessarily be correct, since PRISMA is meant for systematic review-it is not a reporting guideline for overviews of review (though some fields of it might be adopted with modifications).

11.
Is the rationale for commenting on the previous publication clearly described? version 6.2 (updated February 2021). Cochrane, 2021. Available from www.training.cochrane.org/handbook.
The correspondence author does understand that PROSPERO is probably the most popular repository among many other available options. Indeed, the correspondence author did refer to PROSPERO as the cited paper did. In fact, the correspondence author quoted their statement. The correspondence author appreciates that Dr. Papageorgiou found valid the point concerning the lack of PROSPERO registration.
The point of mentioning PROSPERO is supported by AMSTAR-2, which considers "protocol registered before start of review" as a critical domain. Please refer to Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or nonrandomised studies of healthcare interventions, or both. BMJ 2017; 358: j4008. Thus, one would expect that an overview of reviews (or systematic review of systematic reviews) should avoid missing such critical domain that they have assessed in their study where they used systematic reviews as the analytic unit.
The correspondence author thanks Dr. Papageorgiou for all the other comments that he found valid. For instance, the overview is missing to include SRs, review authors wrongly judging some AMSTAR-2 fields of a previous SR by the correspondence author, and lack of repository registry.
Lastly, the author appreciates Dr. Papageorgiou's comments regarding this correspondence comment. The points that could be perceived as a disagreement have been addressed by the correspondence author here. the literature and of the analysis of articles present in this article thanks to the reporting items and quality assessments.
The author underlines in this commentary his criticisms of the review under consideration and makes suggestions for improving the methodology and reporting of this kind of research. The author appears to be involved in Sarkis-Onofre's research, and this limits his field of action in contesting the execution of the study, underlining how another research of his has been excluded. However, we believe that the criticisms raised go beyond the conflict of interest and can be considered valid.
As described by the author, we agree with the need for more transparency on the methodology of these studies.
Regarding the registration on PROSPERO, we believe that this kind of revision remains valid regardless of this. The contestation of the protocol used by the author, rather than leading to the use of standardized protocols, we believe is necessary that it be well described in the article and that it be available upon request by the authors of the study.
The criticisms that are posed in the article, we believe are valid and well structured. The bibliography that underlies them is thick and published in high-level journals. We, therefore, consider this article suitable for indexing, sure of the fact that it can be an added value for the quality of this type of research. on perceived errors in the AMSTAR-2 assessments made to his own review included in Sarkis-Onofre et al., and on the failure to identify and include in the overview another work he authored (Afrashtehfar et al., 2016 2 ). While these are undeniable facts and the methods applied in the overview were subpar, it's not clear whether the errors identified are the exception or the norm, nor their impact on the overview conclusions.
He also points out the need for more transparency in the reporting of AMSTAR-2 assessments whenever these challenge personal communications with the original study authors. I fully support the idea to present in an appendix the clarifications about methodology provided in personal communications and the reasons for Sarkis-Onofre et al. to accept or disregard them in each case.
However, there are two instances of unjustified statements made in the comment. First, the author challenges the justification of Sarkis-Onofre et al. for not registering their methodological overview in PROSPERO, arguing that as an overview it could have been registered. However, this fails to acknowledge that methodological overviews that provide no data on patient-relevant outcomes don't verify PROSPERO registration criteria for methodological reviews.
Second, the author assumes that the overview, which updates a previously published overview from different authors, should follow the original overview protocol. But, while any review must have a pre-specified protocol, it's not necessary for an update to follow the protocol of the original review, as that protocol may be inexistent or outdated. What is important is that Sarkis-Onofre et al. do have a protocol available at request.

Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: I am a statistician, author of several systematic reviews, and an editor in two Cochrane Review Groups, conducting peer reviews frequently.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.