Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement

Carles Fernández-Jané; Mireia Solà-Madurell; Mingkun Yu; Changhao Liang; Yutong Fei; Mercè Sitjà-Rabert; Gerard Úrrutia

doi:10.12688/f1000research.22843.3

Home Browse Completeness of reporting acupuncture interventions for chronic obstructive...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement

[version 3; peer review: 2 approved]

Previously titled: Quality of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement

Carles Fernández-Jané ^1,2, Mireia Solà-Madurell¹, Mingkun Yu³, [...] Changhao Liang³, Yutong Fei³, Mercè Sitjà-Rabert^1,2, Gerard Úrrutia⁴

Carles Fernández-Jané ^1,2, Mireia Solà-Madurell¹, [...] Mingkun Yu³, Changhao Liang³, Yutong Fei³, Mercè Sitjà-Rabert^1,2, Gerard Úrrutia⁴

PUBLISHED 20 Nov 2020

Author details Author details

¹ School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain
² Global Research on Wellbeing (GRoW) Research Group, Ramon Llull University, Barcelona, Spain
³ Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
⁴ Institut d'Investigació Biomèdica Sant Pau (IIB Sant Pau). CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain

Carles Fernández-Jané
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Mireia Solà-Madurell
Roles: Conceptualization, Investigation, Writing – Review & Editing

Mingkun Yu
Roles: Investigation, Writing – Review & Editing

Changhao Liang
Roles: Investigation, Writing – Review & Editing

Yutong Fei
Roles: Conceptualization, Methodology, Writing – Review & Editing

Mercè Sitjà-Rabert
Roles: Methodology, Writing – Review & Editing

Gerard Úrrutia
Roles: Methodology, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: The completeness of reporting of acupuncture interventions is critical to ensure the applicability and reproducibility of acupuncture clinical trials. In the past, different publications have evaluated the completeness of reporting of acupuncture interventions for different clinical situations, such as knee osteoarthritis, neurological diseases or cancer. However, this has not been done for acupuncture trials for chronic obstructive pulmonary disease (COPD).

Objective: To assess the completeness of reporting of acupuncture interventions in trials for COPD.

Methods: A total of 11 English and Chinese databases were screened up until May 2019 for randomised or quasi-randomised control trials of acupuncture for COPD. The STRICTA checklist was used to determine the quality of the reporting of acupuncture interventions.

Results: A total of 28 trials were included in our review. Out of the 16 STRICTA checklist subitems analysed, only 4 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 30%.

Conclusion: The adherence to STRICTA guidelines of acupuncture trials for COPD is suboptimal, and future efforts need to be addressed to improve the completeness of reporting.

Keywords

Quality of Reporting, Acupuncture, COPD

Corresponding author: Carles Fernández-Jané

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2020 Fernández-Jané C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Fernández-Jané C, Solà-Madurell M, Yu M et al. Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement [version 3; peer review: 2 approved]. F1000Research 2020, 9:226 (https://doi.org/10.12688/f1000research.22843.3) First published: 02 Apr 2020, 9:226 (https://doi.org/10.12688/f1000research.22843.1) Latest published: 20 Nov 2020, 9:226 (https://doi.org/10.12688/f1000research.22843.3)

Revised Amendments from Version 2

The term “quality of reporting”, has been changed to “completeness of reporting” as “quality” is an ambiguous and problematic word, and to use.

A mistake in the result section in the abstract has been corrected.

Also, several clarifications have been made in the introduction and methods sections.

In the results section, a new paragraph has been added to give an "overall summary" of the results.

Finally, a new paragraph has been added in the discussion section to address possible interventions to improve the completeness of reporting.

See the authors' detailed response to the review by Erik Cobo
See the authors' detailed response to the review by David Blanco

Introduction

Recent systematic reviews have assessed acupuncture’s efficacy for chronic obstructive pulmonary disease (COPD)^1–3. These reviews have concluded that, even though acupuncture could have some benefits, the risk of bias of the included trials is too high to draw any strong conclusion. Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.

Complete and clear information regarding clinical trial methodology is not only essential to adequately assess health research but also for its applicability and reproducibility. This is especially important in complex interventions, such as acupuncture, that can be practiced in many different styles and variations. Aspects such as point selection, depth of the insertion, stimulation method and response sought, which may be very different between practitioners, could have an impact on the therapeutic effect⁴. Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.

The STandards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) guidelines were created to improve the completeness of reporting in acupuncture trials and facilitate transparency in published reports⁵. These guidelines were updated in 2010 as an extension of the CONsolidated Standards Of Reporting Trials (CONSORT) guidelines⁶ and consist of 6 key items and 17 subitems addressing aspects such as acupuncture rationale, needling details, treatment regime, other components of the treatment, practitioner background and details about the control or comparator.

Although there is evidence that STRICTA guidelines have helped to improve acupuncture’s reporting, there is still a lot of room for improvement⁷. This has also been seen in recent publications for some specific conditions such as neurological diseases^8–10 or cancer¹¹, concluding that reporting is still suboptimal in these conditions. However, to our knowledge, there is currently no publications regarding the completeness of reporting of acupuncture interventions in COPD trials.

Evaluating the adherence of acupuncture clinical trials for COPD to SRICTA guidelines is crucial to detect underreported subitems and therefore highlight current deficiencies. This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.

The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture.

Methods

Study selection

In this study, we used the results of our previous systematic review, which included randomised or quasi-randomised trials using filiform needle acupuncture for COPD². Published studies were comprehensively searched in the following databases from their inception to May 2019: Cochrane Central Register of Controlled Trials (CENTRAL), Medline, Embase, CINAHL, AMED (Ovid), PEDro, PsycINFO, CNKI, VIP, Wanfang, and Sino-Med. Detailed descriptions of the inclusion criteria, information sources, search strategies and study selections are published elsewhere² and the protocol is available at http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014015074.

Data collection process

Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.

For data collection and STRICTA assessment, a specific extraction table with instructions of how to assess each of the 16 STRICTA subitems was created by the authors (Extended data¹²). This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability. The data of the pilot test was included in the final analysis.

Our aim was to assess the acupuncture interventions, and therefore, the 17th STRICTA subitem “precise description of the control or comparator” was not considered. However, we did include the 16th STRICTA subitem “rationale of the chosen comparator”, since it is critical to justify which component of the acupuncture treatment is being assessed.

Since some STRICTA subitems refer to multiple aspects (e.g., “names of points used” subitem refers not only to the name or location of points but also to if they are used unilaterally or bilaterally), besides considering items just as reported or not reported, we also considered partially reported items and recorded the reasons for there being. This was done to provide more detailed information regarding the aspects that should be improved in the reporting of future trials. Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion. Although some subitems were considered that could potentially be “not applicable” (NA) for some pragmatic designs, none of the trials required this.

Data analysis

A descriptive analysis was used to summarise the results using percentages and absolute numbers.

Results

Number of publications and characteristics

In our systematic review, we screened all 5030 unduplicated titles and abstracts retrieved, and obtained 166 full text articles. Finally, we included 28 trials using filiform needles for COPD (Figure 1). Of those, only 6 were published in English-language journals, 1 in a German-language journal and 21 in Chinese-language journals. Details regarding the study characteristics and inclusion process have been published elsewhere².

Figure 1. Flow diagram.

Completeness of reporting

Out of the 16 STRICTA checklist subitems analysed, only 4 were considered appropriately reported in more than 70% of the trials; style of acupuncture, variation extent, retention time and frequency and duration of treatment sessions. We also found that 7 other subitems were correctly reported in less than 30% of the trials; depth of insertion, needle type, number of treatment sessions, details of other interventions administered, setting and context of treatment, description of participating acupuncturists and rationale for the control.

Ratings for STRICTA domains are summarized in Figure 2. Details for each trial are shown in Table 1, including reasons for considering partial reporting.

Table 1. Completeness of reporting according to STRICTA guidelines.

STRICTA Items		Reported, % (n)	Not reported, % (n)	Partially reported, % (n) Reason, % (n)
1) Acupuncture rationale	1a) Style of acupuncture	71.4 (20)	14.2 (4)	14.2 (4) Not reported in the correct section, 14.2 (4)
	1b) Reasoning for treatment provided, based on historical context, literature sources, and/or consensus methods, with references where appropriate	64.2 (18)	21.4 (6)	14.2 (4) Not reported in the correct section, 14.2 (4)
	1c) Extent to which treatment was varied	82.1 (23)	17.8 (5)	0 (0)
2) Details of needling	2a) Number of needle insertions per subject per session	46.4 (13)	28.5 (8)	17.8 (5) Do not mention the number of needles, 17.8 (5)
	2b) Names (or location if no standard name) of points used (uni/bilateral)	35.7 (10)	7.14 (2)	57.14 (16) Do not mention uni or bilateral insertion, 53 (15) Not all point locations are described, 3.5 (1)
	2c) Depth of insertion, based on a specified unit of measurement or on a particular tissue level	28.5 (8)	71.4 (20)	0 (0)
	2d) Responses sought	53.5 (15)	46.4 (13)	0 (0)
	2e) Needle stimulation	67.8 (19)	17.8 (5)	14.2 (4) Manual stimulation is mentioned but the specify the method is not, 10.7 (3) Electrical stimulation is mentioned but not parameters used, 3.5 (1)
	2f) Needle retention time	75 (21)	25 (7)	0 (0)
	2g) Needle type	21.4 (6)	28.5 (8)	50 (14) Material is not reported, 25 (7) Manufacturer is not reported, 32.1 (9) Diameter and length are not reported, 10.7 (3)
3) Treatment regime	3a) Number of treatment sessions	0 (0)	0 (0)	100 (28) The actual number of treatments received is not reported or not clear, 100 (28)
3) Treatment regime	3b) Frequency and duration of treatment sessions	100 (28)	0 (0)	0 (0)
4) Other components of treatment	4a) Details of other interventions administered to the acupuncture group	28.5 (8)	57.1 (16)	14.2 (4)
4) Other components of treatment	4b) Setting and context of treatment, including instructions to practitioners, and information and explanations to patients	7.1 (2)	89.2 (25)	3.5 (1)
5) Practitioner background	5) Description of participating acupuncturists	3.5 (1)	89.2 (25)	7.1 (2) Years of experience not mentioned, 7.1 (2)
6) Control or comparator interventions	6a) Rationale for the control or comparator	7.1 (2)	85.7 (24)	7.1 (2) Not reported in the correct section: 7.1 (2)

STRICTA, STandards for Reporting Interventions in Clinical Trials of Acupuncture.

Acupuncture rationale. Acupuncture rationale was considered adequately reported in 20 trials (71%) regarding acupuncture style, 18 trials (64%) regarding reasoning of the treatment and 24 trials (85%) regarding treatment variation. Trials classified with partial reporting mentioned acupuncture style (3 trials) and reasoning (4 trials) in the introduction section but not in the methods section.

Figure 2. Summary of completeness of reporting according to STRICTA guidelines.

Details of needling. Adequate reporting of needling details was very heterogenous along all 4 subdomains. Best reported subdomains were “needle retention time” (21 trials, 75%) and “needle stimulation” (19 trials, 67%). Worse reported items were “name of the points” (10 trials, 35%), “depth of insertion” (8 trials, 28%) and “needle type” (6 trials, 21%). Partial reporting was observed in great proportion in “name of acupuncture points” (16 trials, 57%) being the main reason not describing if points were used unilaterally or bilaterally. Regarding the 46% of the trials classified with a partial reporting in the “needle type” item, there was missing information about the needle manufacturer (32.1%, 9 trials), material (25%, 7 trials) and length and diameter (10.7%, 3 trials).

Treatment regime. While “frequency and duration of the treatment sessions” was considered adequately reported for all trials (100%), the “number of treatment sessions” was considered completely reported in none of them (0%). Although this might seem a contradiction, since the number of treatment sessions can be calculated from the treatment regime, in STRICTA, the number of treatment sessions does not only include the planned number of sessions but also the actual number of treatments received. This information was missing or considered not clear in all trials.

Other components of treatment. Other components of treatment were one of the poorer described items. “Details of other interventions administrated” was only reported in 8 trials (28%) and “settings and context of treatments”, which refers to “instructions to practitioners that might modify their normal practice, for example, prescribing or proscribing explanations to patients about their diagnosis”, were only reported in 2 trials (7.1%).

Practitioner background. This item was only addressed completely in 1 trial (3.5%), and only 2 more trials (7.1%) partially reported it without stating practitioner’s years of experience.

Control or comparator. The “rational of the chosen comparator” was only correctly described in 2 trials (7.1%), while in 2 more trials (7.1%), this was mentioned in the introduction but not in the methods section.

Discussion

We found important limitations in the completeness of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.

Recently, several similar studies have been published. Lu et al.¹¹ and Hughes et al.¹³ used STRICTA to evaluate trials with cancer patients, and Wei et al. used STRICTA to evaluate trials with stroke patients⁸. Although they all concluded that reporting should be improved, there were also some differences. While Hughes et al. found better reporting regarding details of needling and treatment regimen, other reviews found lower reporting on these subitems, especially regarding number of needles per session and depth of insertion. Poor reporting on “details of other interventions administered”, “description of acupuncturist” and “rationale for the control or comparator” subitems was found in all studies.

The differences mentioned above could be due to several reasons. First, Lu’s and Weis’s reviews, as well as our own, included Chinese-language trials, while Hugs’ study included only English-language trials. Trials published in English-language journals seem to have greater completeness of reporting than those in Chinese-language journals, according to Lu’s review. However, Wei et al. found better reporting of the subitems “treatment reasoning” and “response sought” in Chinese journals and better reporting on “practitioner’s background” in English journals.

Second, since STRICTA does not clearly specify how items should be judged, authors might have used different criteria. For example, regarding the subitem “number of treatment sessions”, the STRICTA statement says that “the actual number of treatments received by participants should be reported in the Results section” not only the planned ones. Whereas in our review, we did not consider that this subitem was fully reported unless this information was explicitly stated; others might have been more permissive. Also, the criteria to consider proper reporting on “other components of treatment” might vary widely between reviewers.

Third, sometimes information might have been reported in sections such as the introduction and the discussion. While some authors might not have given much importance to this, we decided to take it into consideration.

To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown¹⁴.

Strengths and limitations

To our knowledge, this is the first study to assess the completeness of reporting of acupuncture interventions for COPD. We included all acupuncture trials for COPD published until May 2019 with no language restriction, which is important since we only found 6 acupuncture trials published in English. Also, we did not only assess if STRICTA subitems were adequately reported or not but also analysed partial reporting and its reasons, which might be more helpful for authors to realise what specific information is currently missing.

Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper completeness of reporting assessment tool for acupuncture interventions. To minimise this problem, each item was assessed by two reviewers independently, and a standardised extraction form was developed to unify reviewers’ criteria. Also, it would have been interesting to compare the completeness of reporting depending on the language of publication, so we could explore differences in journal standards. However, due to the low number of non-Chinese-language publications found (1 in German and 6 in English), we decided not to do so. Finally, STRICTA guidelines are specific for the filiform needle acupuncture technique and are not suitable to assess other interventions. Therefore, we did not include trials using only moxibustion, acupressure or transcutaneous electrical nerve stimulation.

Conclusion

The completeness of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.

Data availability

Underlying data

Figshare: Raw data file, https://doi.org/10.6084/m9.figshare.11999970.v1¹⁵.

Extended data

Figshare: Extended data 1 Extraction form, https://doi.org/10.6084/m9.figshare.11999994.v1¹².

Data is available under a Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Acknowledgements

We would like to thank the students from the Beijing University of Chinese Medicine Liu Kexin, Zhang Zixuan, Ming Yang, Gao Jiaqi, Li Yilin, Zhang Zhijia, Luo Lingxiao, Shi Yixin, Zhao Luming, Zhang Bingrui, Zhao Leying, Liu Jinjun, Ding Maoyu, Fan Shixuan and Ren Yiming for participating in the project and helping to define the suitability of the data extraction form. The corresponding author is a PhD candidate at the Universitat Autònoma de Barcelona, Spain.

Faculty Opinions recommended

References

1. Coyle ME, Shergis JL, Huang ET, et al.: Acupuncture therapies for chronic obstructive pulmonary disease: a systematic review of randomized, controlled trials. Altern Ther Health Med. 2014; 20(6): 10–23. PubMed Abstract
2. Fernández-Jané C, Vilaró J, Fei Y, et al.: Filiform needle acupuncture for copd: A systematic review and meta-analysis. Complement Ther Med. 2019; 47: 102182. PubMed Abstract | Publisher Full Text
3. Hsieh PC, Yang MC, Wu YK, et al.: Acupuncture therapy improves health-related quality of life in patients with chronic obstructive pulmonary disease: A systematic review and meta-analysis. Complement Ther Clin Pract. 2019; 35: 208–218. PubMed Abstract | Publisher Full Text
4. Shi GX, Yang XM, Liu CZ, et al.: Factors contributing to therapeutic effects evaluated in acupuncture clinical trials. Trials. 2012; 13: 42. PubMed Abstract | Publisher Full Text | Free Full Text
5. MacPherson H, Altman DG, Hammerschlag R, et al.: Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT statement. J Evid Based Med. 2010; 3(3): 140–55. PubMed Abstract | Publisher Full Text
6. Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340: c332. PubMed Abstract | Publisher Full Text | Free Full Text
7. Svenkerud S, MacPherson H: The impact of STRICTA and CONSORT on reporting of randomised control trials of acupuncture: a systematic methodological evaluation. Acupunct Med. 2018; 36(6): 349–357. PubMed Abstract | Publisher Full Text
8. Wei JJ, Yang WT, Yin SB, et al.: The quality of reporting of randomized controlled trials of electroacupuncture for stroke. BMC Complement Altern Med. 2016; 16(1): 512. PubMed Abstract | Publisher Full Text | Free Full Text
9. Luo G, Lu L, Zeng J: Quality of reporting of randomised controlled trials of acupuncture for neurological diseases conducted in China. Acupunct Med. 2014; 32(5): 411–7. PubMed Abstract | Publisher Full Text
10. Zeng J, Lin G, Li L, et al.: Assessment of reporting quality in randomised controlled trials of acupuncture for post-stroke rehabilitation using the CONSORT statement and STRICTA guidelines. Acupunct Med. 2017; 35(2): 100–106. PubMed Abstract | Publisher Full Text
11. Lu L, Liao M, Zeng J, et al.: Quality of reporting and its correlates among randomized controlled trials on acupuncture for cancer pain: application of the CONSORT 2010 Statement and STRICTA. Expert Rev Anticancer Ther. 2013; 13(4): 489–98. PubMed Abstract | Publisher Full Text
12. Fernández C: Extended data 1 Extraction form. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.11999994.v1
13. Hughes JG, Lewith G, MacPherson H, et al.: Assessment of the quality of reporting in studies of acupuncture for patients with cancer using the STRICTA guidelines. Acupunct Med. 2019; 37(4): 223–227. PubMed Abstract | Publisher Full Text
14. Blanco D, Altman D, Moher D, et al.: Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open 2019; 9(5): e026589. PubMed Abstract | Publisher Full Text | Free Full Text
15. Fernández C: Raw data file. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.11999970.v1

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 02 Apr 2020

Author details Author details

Carles Fernández-Jané
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Mireia Solà-Madurell
Roles: Conceptualization, Investigation, Writing – Review & Editing

Mingkun Yu
Roles: Investigation, Writing – Review & Editing

Changhao Liang
Roles: Investigation, Writing – Review & Editing

Yutong Fei
Roles: Conceptualization, Methodology, Writing – Review & Editing

Mercè Sitjà-Rabert
Roles: Methodology, Writing – Review & Editing

Gerard Úrrutia
Roles: Methodology, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (3)

version 3

Revised

Published: 20 Nov 2020, 9:226

https://doi.org/10.12688/f1000research.22843.3

version 2

Revised

Published: 23 Oct 2020, 9:226

https://doi.org/10.12688/f1000research.22843.2

version 1

Published: 02 Apr 2020, 9:226

https://doi.org/10.12688/f1000research.22843.1

© 2020 Fernández-Jané C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Fernández-Jané C, Solà-Madurell M, Yu M et al. Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement [version 3; peer review: 2 approved]. F1000Research 2020, 9:226 (https://doi.org/10.12688/f1000research.22843.3)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 3

VERSION 3

PUBLISHED 20 Nov 2020

Revised

Views

Reviewer Report 01 Dec 2020

Erik Cobo, Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain

Approved

https://doi.org/10.5256/f1000research.30829.r75127

Thanks for considering most of my suggestions. I think ... Continue reading

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 23 Nov 2020

David Blanco, Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain

Approved

https://doi.org/10.5256/f1000research.30829.r75126

I am happy with this new version of the manuscript. It is clear and ... Continue reading

CITE

Report a concern

Respond or Comment

Version 2

VERSION 2

PUBLISHED 23 Oct 2020

Revised

Views

Reviewer Report 13 Nov 2020

David Blanco, Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain

Approved with Reservations

https://doi.org/10.5256/f1000research.30190.r73827

Thank you for allowing me to review this manuscript. The subject is the evaluation of the degree of adherence to STRICTA guideline items in the context of acupuncture trials. Overall, I find the manuscript attractive and well-written.
I hope ... Continue reading

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.
Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.
Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)
The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”
Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.
I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.
For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Stevens A, Shamseer L, Weinstein E, Yazdi F, et al.: Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review.BMJ. 2014; 348: g3804 PubMed Abstract | Publisher Full Text
2. Blanco D, Altman D, Moher D, Boutron I, et al.: Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open. 2019; 9 (5). Publisher Full Text
3. Logullo P, MacCarthy A, Kirtley S, Collins GS: Reporting guideline checklists are not quality evaluation forms: they are guidance for writing.Health Sci Rep. 2020; 3 (2): e165 PubMed Abstract | Publisher Full Text
4. Hair K, Macleod MR, Sena ES, IICARus Collaboration: A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus).Res Integr Peer Rev. 2019; 4: 12 PubMed Abstract | Publisher Full Text
5. Barnes C, Boutron I, Giraudeau B, Porcher R, et al.: Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial.BMC Med. 2015; 13: 221 PubMed Abstract | Publisher Full Text
6. Blanco D, Schroter S, Aldcroft A, Moher D, et al.: Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial. BMJ Open. 2020; 10 (5). Publisher Full Text
7. Cobo E, Cortés J, Ribera JM, Cardellach F, et al.: Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial.BMJ. 2011; 343: d6783 PubMed Abstract | Publisher Full Text
8. Hopewell S, Boutron I, Altman DG, Barbour G, et al.: Impact of a web-based tool (WebCONSORT) to improve the reporting of randomised trials: results of a randomised controlled trial.BMC Med. 2016; 14 (1): 199 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Journal policies, peer review, reporting guidelines

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 24 Nov 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

24 Nov 2020

Author Response
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments
- Despite being a
... Continue reading
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”.

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Response: We have now added the information at the beginning of the results section.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.

Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected
“Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.”

Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.

Response: The structure of the standardized data extraction table is fully available as extended data

Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Response: Yes, he pilot data extraction was included in the analysis. This has been clarified in the data collection section

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)

Response: This has been now described
“We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.”

The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”

Response: This has been corrected

Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.

Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.”

I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.

Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.”

For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Response: A new paragraph in the discussion section about the topic has been added
“To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown ¹⁴”
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”.

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Response: We have now added the information at the beginning of the results section.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.

Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected
“Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.”

Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.

Response: The structure of the standardized data extraction table is fully available as extended data

Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Response: Yes, he pilot data extraction was included in the analysis. This has been clarified in the data collection section

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)

Response: This has been now described
“We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.”

The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”

Response: This has been corrected

Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.

Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.”

I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.

Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.”

For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Response: A new paragraph in the discussion section about the topic has been added
“To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown ¹⁴”
Competing Interests: No competing interests are disclosed Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 24 Nov 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

24 Nov 2020

Author Response
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments
- Despite being a
... Continue reading
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”.

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Response: We have now added the information at the beginning of the results section.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.

Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected
“Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.”

Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.

Response: The structure of the standardized data extraction table is fully available as extended data

Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Response: Yes, he pilot data extraction was included in the analysis. This has been clarified in the data collection section

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)

Response: This has been now described
“We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.”

The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”

Response: This has been corrected

Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.

Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.”

I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.

Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.”

For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Response: A new paragraph in the discussion section about the topic has been added
“To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown ¹⁴”
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”.

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Response: We have now added the information at the beginning of the results section.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.

Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected
“Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.”

Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.

Response: The structure of the standardized data extraction table is fully available as extended data

Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Response: Yes, he pilot data extraction was included in the analysis. This has been clarified in the data collection section

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)

Response: This has been now described
“We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.”

The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”

Response: This has been corrected

Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.

Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.”

I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.

Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.”

For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Response: A new paragraph in the discussion section about the topic has been added
“To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown ¹⁴”
Competing Interests: No competing interests are disclosed Close
Report a concern

Version 1

VERSION 1

PUBLISHED 02 Apr 2020

Views

Reviewer Report 06 Oct 2020

Erik Cobo, Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain

Approved with Reservations

https://doi.org/10.5256/f1000research.25221.r71710

I read this paper and I think it could be accepted provided authors correct my three major suggestions and address my minor suggestions.

Major suggestions:

Please revise the numbers in your flow diagram

I read this paper and I think it could be accepted provided authors correct my three major suggestions and address my minor suggestions.

Major suggestions:

Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.
Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.
Please, allow access to your data to other scientists. Please, consider a public repository.

Minor suggestions:

As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.
In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.
Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹
Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?
Please, consider to explain the pilot process and how did you “ensure” its usability.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Goodman S, Fanelli D, Ioannidis J: What does research reproducibility mean?. Science Translational Medicine. 2016; 8 (341). Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: RCTs, reporting guidelines, causal modeling, meta-researcch

CITE

Report a concern

Author Response 21 Oct 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

21 Oct 2020

Author Response

We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and ... Continue reading We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.

As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63.
Studies has been changed for articles in the diagram

Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.

Conclusions have been re-written to include the mentioned recommendations
“The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.”

Please, allow access to your data to other scientists. Please, consider a public repository.
The data is already public and available in the “data availability” section.

Minor suggestions:
As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.

We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section.

We would prefer not to change the nomenclature as the real the objective of the study is to assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader.

In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.

This has been clarified in the introduction section
“Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.”

Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹

Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section:

“Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.”

We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader.

Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?

Every paper was analysed by two reviewers, this has been now clarified in the methods section:

“Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.”

Please, consider to explain the pilot process and how did you “ensure” its usability.

This has been now clarified in the “data collection process” section.

“This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.”
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.

As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63.
Studies has been changed for articles in the diagram

Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.

Conclusions have been re-written to include the mentioned recommendations
“The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.”

Please, allow access to your data to other scientists. Please, consider a public repository.
The data is already public and available in the “data availability” section.

Minor suggestions:
As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.

We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section.

We would prefer not to change the nomenclature as the real the objective of the study is to assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader.

In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.

This has been clarified in the introduction section
“Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.”

Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹

Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section:

“Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.”

We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader.

Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?

Every paper was analysed by two reviewers, this has been now clarified in the methods section:

“Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.”

Please, consider to explain the pilot process and how did you “ensure” its usability.

This has been now clarified in the “data collection process” section.

“This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.”
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 21 Oct 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

21 Oct 2020

Author Response

We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and ... Continue reading We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.

As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63.
Studies has been changed for articles in the diagram

Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.

Conclusions have been re-written to include the mentioned recommendations
“The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.”

Please, allow access to your data to other scientists. Please, consider a public repository.
The data is already public and available in the “data availability” section.

Minor suggestions:
As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.

We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section.

We would prefer not to change the nomenclature as the real the objective of the study is to assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader.

In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.

This has been clarified in the introduction section
“Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.”

Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹

Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section:

“Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.”

We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader.

Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?

Every paper was analysed by two reviewers, this has been now clarified in the methods section:

“Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.”

Please, consider to explain the pilot process and how did you “ensure” its usability.

This has been now clarified in the “data collection process” section.

“This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.”
We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.

As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63.
Studies has been changed for articles in the diagram

Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.

Conclusions have been re-written to include the mentioned recommendations
“The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.”

Please, allow access to your data to other scientists. Please, consider a public repository.
The data is already public and available in the “data availability” section.

Minor suggestions:
As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.

We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section.

We would prefer not to change the nomenclature as the real the objective of the study is to assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader.

In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.

This has been clarified in the introduction section
“Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.”

Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹

Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section:

“Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.”

We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader.

Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?

Every paper was analysed by two reviewers, this has been now clarified in the methods section:

“Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.”

Please, consider to explain the pilot process and how did you “ensure” its usability.

This has been now clarified in the “data collection process” section.

“This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.”
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 02 Apr 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 3 (revision) 20 Nov 20	read	read
Version 2 (revision) 23 Oct 20		read
Version 1 02 Apr 20	read

Erik Cobo, Polytechnic University of Catalonia, Barcelona, Spain
David Blanco, Universitat Internacional de Catalunya, Barcelona, Spain

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

6 Views

01 Dec 2020 | for Version 3

Erik Cobo, Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain

6 Views Cite this report Responses(0)

Approved

Thanks for considering most of my suggestions. I think the paper is now much clearer and probably more reprodicible.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Clinical trials; Reporting Guidelines; meta-research; research waste

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

8 Views

23 Nov 2020 | for Version 3

David Blanco, Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain

8 Views Cite this report Responses(0)

Approved

I am happy with this new version of the manuscript. It is clear and well-reported. Congratulations on your work and thank you for the opportunity to review it!

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Journal policies, peer review, reporting guidelines

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

21 Views

13 Nov 2020 | for Version 2

David Blanco, Physiotherapy Department, Universitat Internacional de Catalunya, Barcelona, Spain

21 Views Cite this report Responses(1)

Approved With Reservations

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.
Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.
Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)
The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”
Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.
I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.
For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Journal policies, peer review, reporting guidelines

Respond to this report

Responses (1)

Author Response

24 Nov 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

General comments

Despite being a term commonly used in the literature, I would suggest authors to avoid using “quality of reporting”, as “quality” is an ambiguous and problematic word, and to use “completeness of reporting”, which is clearer and more connected to reporting guidelines¹ ².

Response: We agree with the suggestion and have changed “quality of reporting” for “completeness of reporting”.

Abstract

If you include the sentence: “Out of the 16 STRICTA checklist subitems analysed, only 3 were considered appropriately reported in more than 70% of the trials, while 7 were correctly reported in less than 40%.” you would also need to include this information in the Results section (perhaps as a first paragraph of the “Quality of reporting” subsection). As a reader, I would appreciate to have in the main body of the article some sort of “overall summary” of the completeness of the included papers and not just the results for each separate item.

Response: We have now added the information at the beginning of the results section.

Introduction

In the last sentence of the first paragraph, you seem to imply that the risk of bias of a certain study and its quality of reporting have no connection. In fact, complete and transparent reporting is what makes it possible for a reviewer to assess the risk of bias of a study. I believe this point deserves to be reflected in the paper.

Response: We agree with the reviewers comments and therefore, we modified our last sentence to show that risk of bias and its quality of reporting are connected
“Risk of bias is certainly a critical aspect in randomised control trials; however, to be able to adequately assess sources of bias, complete information needs to be reported.”

Importantly, the fact that you state, “The aim of this study is to comprehensibly evaluate the adherence to STRICTA guidelines of trials using acupuncture for COPD”, could make the reader think that you are going to use the percentage of completeness per trial as the unit of interest. However, you are analysing the percentage of completeness per STRICTA item. I would therefore suggest to reformulate the study aim to avoid confusion: “The aim of this study is to comprehensibly evaluate the degree of completeness of reporting for each STRICTA item in randomised trials using acupuncture” or something similar.

Response: We have reformulated the aim of the study to avoid confisions as the reviewer suggested.

Methods

It would be helpful that you explain in a few words what was the structure of the standardized data extraction form and what you were interested in looking at.

Response: The structure of the standardized data extraction table is fully available as extended data

Were the three papers you used for the pilot also included in the final analysis? This should be mentioned.

Response: Yes, he pilot data extraction was included in the analysis. This has been clarified in the data collection section

Discussion

In the first paragraph of the discussion, it would be good to mention in brackets the criteria that you chose to highlight these underreported items (% of adequate reporting less than X%?)

Response: This has been now described
“We found important limitations in the quality of reporting of acupuncture interventions in trials for COPD, especially regarding “depth of insertion”, “needle type”, “number of treatment sessions”, “details of other interventions administered”, “setting and context of treatments”, “description of acupuncturist” and “rationale for the control or comparator” with less than 30% of the trials reporting them completely.”

The sentence “The differences between studies could be due to several reasons” should be placed at the beginning of the next paragraph as it is heavily connected with it. For example, “The differences between the studies we mentioned above could be due to several reasons.”

Response: This has been corrected

Concerning the (very appropriate) point you make in the fourth paragraph, you could also mention that RGs are originally just guidance for writing and they are not intended to be evaluation forms despite everyone uses them as such³.

Response: This is explained in the limitations section “Limitations of this study include that the STRICTA guidelines are not a rating scale; therefore, there are no clear indications of how to judge each subitem and when to consider it fully reported. This issuer must be addressed in the future by developing a proper quality of reporting assessment tool for acupuncture interventions.”

I would also better argue why you took into consideration (and considered as “partially reported” instead of fully reported) those items that were reported in a different section. I guess that when you say section you refer to the main 4 sections (IMRaD) – perhaps you could make it clearer in the methods.

Response: It was already explained in the methods section “Partial reporting was also considered when the authors reported information in other sections, such as the introduction or the discussion.”. However we have now clarified also in the discussion section “Third, sometimes information might have been reported in sections such as the introduction and the discussion.”

For me, it is not enough to say “Strategies for improving the understanding of the guides for authors, reviewers, and journal editors are needed, as well as to improve its implementation”. You should definitely include a paragraph in the Discussion section reflecting on what kind of strategies targeting authors, peer reviewers, or journal editors could work in the particular context of acupuncture journals (or more generally, if you prefer so). In the introduction, you mentioned, “This will help to improve the reporting of future trials and facilitate their applicability in clinical practice and the reproducibility of future research.” However, I believe that it is not enough to point out that there are reporting issues without reflecting on what kind of strategies should journals follow to turn things around. As you probably know, some recent experiments are testing out different editorial strategies⁴ ⁵ ⁶ ⁷ ⁸ as well as different scoping and systematic reviews/surveys exploring this issue.

Response: A new paragraph in the discussion section about the topic has been added
“To try to improve the adherence to reporting guidelines several strategies have been proposed including training on the use of the guidelines, improving understanding, encouraging adherence, checking adherence and providing feedback, and the involvement of experts. Unfortunately, the effectiveness of many of those interventions is still unknown ¹⁴”

View more View less

Competing Interests

No competing interests are disclosed

Back to all reports

Reviewer Report

27 Views

06 Oct 2020 | for Version 1

Erik Cobo, Statistics and Operations Research Department, Polytechnic University of Catalonia, Barcelona, Spain

27 Views Cite this report Responses(1)

Approved With Reservations

I read this paper and I think it could be accepted provided authors correct my three major suggestions and address my minor suggestions.

Major suggestions:

Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.
Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.
Please, allow access to your data to other scientists. Please, consider a public repository.

Minor suggestions:

As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.
In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.
Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹
Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?
Please, consider to explain the pilot process and how did you “ensure” its usability.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Goodman S, Fanelli D, Ioannidis J: What does research reproducibility mean?. Science Translational Medicine. 2016; 8 (341). Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

RCTs, reporting guidelines, causal modeling, meta-researcch

Respond to this report

Responses (1)

Author Response

21 Oct 2020

Carles Fernández, School of Health Science Blanquerna, Ramon Llull University, Barcelona, Spain

We thank you for your time and comments on the manuscript. Please find below a point by point response to them:

Major suggestions:
Please revise the numbers in your flow diagram and add explanations (maybe in a legend) about your more important labels, such us ineligible population/comparison/… Regarding the numbers, please note that the box “records screened” has more records than the previous one. Or than 62 >< 28+7+13+7+6+1+1=63. Please also note that studies are not the same unit as articles. Please, unify.

As it is already explained in the figure, one trial included two interventions, that is why the total number of studies included is 62 but the summation of all the interventions is 63.
Studies has been changed for articles in the diagram

Please consider to re-write your conclusions clarifying which recommendations could potentially improve quality in the future. I mean, recommendations to STRICTA authors; recommendations to journals; recommendations to reviewers, recommendations to authors; and so on.

Conclusions have been re-written to include the mentioned recommendations
“The quality of reporting of acupuncture interventions in COPD trials according to STRICTA guidelines is suboptimal. Strategies for improving the understanding of the guides for authors, reviewers and journal editors are needed, as well as to improve its implementation.”

Please, allow access to your data to other scientists. Please, consider a public repository.
The data is already public and available in the “data availability” section.

Minor suggestions:
As you stated, “the STRICTA guidelines are not a rating scale”. However, you are repeating through the text that you are attempting to “assess the quality of reporting”. I wonder if you should also highlight the need to further develop reporting guidelines to provide such a tool to assess the quality of reporting. Should you speak about the completeness of reporting? I definitively like much more your statement “evaluate the adherence to Stricta guidelines”.

We agree on highlighting the need of developing a reliable tool for assessing the quality of reporting and this has been added in the discussion section.

We would prefer not to change the nomenclature as the real the objective of the study is to assess the quality of reporting, and using the adherence to current reporting guidelines is the method to do so. We believe that introducing the term “completeness of the reporting” may confuse reader.

In the introduction, I wonder if you should clarify that “risk of bias” applies to the study; and “quality of reporting” to the paper.

This has been clarified in the introduction section
“Risk of bias of bias of the studies is certainly a critical aspect in randomised control trials; however, the quality of reporting of the published papers is also a key point.”

Also in the introduction, you use the terms "replication" and "reproducibility". I would like you to clarify your meanings, perhaps with some references. For example, are you following the definition from Goodman, Fannelly & Ioanidis in “What does research reproducibility mean?”¹

Here replication and reproducibility are used in a common language sense and the meaning is clarified at the end of the introduction section:

“Therefore, to be able to replicate an acupuncture intervention in clinical practice or reproduce it in another trial, it is necessary to fully describe how it is applied.”

We believe that using the terms “methods reproducibility”, “Results reproducibility” or “inferential reproducibility” might not clarify but confuse the reader.

Please, further specify how reviewers “worked in pairs”? Did they split the sample? Did both pairs of reviewers analyze some papers?

Every paper was analysed by two reviewers, this has been now clarified in the methods section:

“Data from each trail was extracted independently by two reviewers (CFJ, MSM, MY and CHL) using a standardised data extraction form, and disagreements were solved by consensus.”

Please, consider to explain the pilot process and how did you “ensure” its usability.

This has been now clarified in the “data collection process” section.

“This extraction table was tested with pilot data of 3 papers to solve disagreements on its understanding and ensure its usability.”

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Coyle ME, Shergis JL, Huang ET, et al.: Acupuncture therapies for chronic obstructive pulmonary disease: a systematic review of randomized, controlled trials. Altern Ther Health Med. 2014; 20(6): 10–23. PubMed Abstract

[2] 2. Fernández-Jané C, Vilaró J, Fei Y, et al.: Filiform needle acupuncture for copd: A systematic review and meta-analysis. Complement Ther Med. 2019; 47: 102182. PubMed Abstract | Publisher Full Text

[3] 3. Hsieh PC, Yang MC, Wu YK, et al.: Acupuncture therapy improves health-related quality of life in patients with chronic obstructive pulmonary disease: A systematic review and meta-analysis. Complement Ther Clin Pract. 2019; 35: 208–218. PubMed Abstract | Publisher Full Text

[4] 4. Shi GX, Yang XM, Liu CZ, et al.: Factors contributing to therapeutic effects evaluated in acupuncture clinical trials. Trials. 2012; 13: 42. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. MacPherson H, Altman DG, Hammerschlag R, et al.: Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT statement. J Evid Based Med. 2010; 3(3): 140–55. PubMed Abstract | Publisher Full Text

[6] 6. Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340: c332. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Svenkerud S, MacPherson H: The impact of STRICTA and CONSORT on reporting of randomised control trials of acupuncture: a systematic methodological evaluation. Acupunct Med. 2018; 36(6): 349–357. PubMed Abstract | Publisher Full Text

[8] 8. Wei JJ, Yang WT, Yin SB, et al.: The quality of reporting of randomized controlled trials of electroacupuncture for stroke. BMC Complement Altern Med. 2016; 16(1): 512. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Luo G, Lu L, Zeng J: Quality of reporting of randomised controlled trials of acupuncture for neurological diseases conducted in China. Acupunct Med. 2014; 32(5): 411–7. PubMed Abstract | Publisher Full Text

[10] 10. Zeng J, Lin G, Li L, et al.: Assessment of reporting quality in randomised controlled trials of acupuncture for post-stroke rehabilitation using the CONSORT statement and STRICTA guidelines. Acupunct Med. 2017; 35(2): 100–106. PubMed Abstract | Publisher Full Text

[11] 11. Lu L, Liao M, Zeng J, et al.: Quality of reporting and its correlates among randomized controlled trials on acupuncture for cancer pain: application of the CONSORT 2010 Statement and STRICTA. Expert Rev Anticancer Ther. 2013; 13(4): 489–98. PubMed Abstract | Publisher Full Text

[12] 12. Fernández C: Extended data 1 Extraction form. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.11999994.v1

[13] 13. Hughes JG, Lewith G, MacPherson H, et al.: Assessment of the quality of reporting in studies of acupuncture for patients with cancer using the STRICTA guidelines. Acupunct Med. 2019; 37(4): 223–227. PubMed Abstract | Publisher Full Text

[14] 14. Blanco D, Altman D, Moher D, et al.: Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open 2019; 9(5): e026589. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Fernández C: Raw data file. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.11999970.v1

Completeness of reporting acupuncture interventions for chronic obstructive pulmonary disease: Review of adherence to the STRICTA statement

Abstract

Keywords

Revised Amendments from Version 2

Introduction

Methods

Study selection

Data collection process

Data analysis

Results

Number of publications and characteristics

Figure 1. Flow diagram.

Completeness of reporting

Table 1. Completeness of reporting according to STRICTA guidelines.

Figure 2. Summary of completeness of reporting according to STRICTA guidelines.

Discussion

Strengths and limitations

Conclusion

Data availability

Underlying data

Extended data

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated