Professional medical writing support and the reporting quality of randomized controlled trial abstracts among high-impact general medical journals

Background: In articles reporting randomized controlled trials, professional medical writing support is associated with increased adherence to Consolidated Standards of Reporting Trials (CONSORT). We set out to determine whether professional medical writing support was also associated with improved adherence to CONSORT for Abstracts. Methods: Using data from a previously published cross-sectional study of 463 articles reporting randomized controlled trials published between 2011 and 2014 in five top medical journals, we determined the association between professional medical writing support and CONSORT for Abstracts items using a Wilcoxon rank-sum test. Results: The mean proportion of adherence to CONSORT for Abstracts items reported was similar with and without professional medical writing support (64.3% vs 66.5%, respectively; p=0.30). Professional medical writing support was associated with lower adherence to reporting study setting (relative risk [RR]; 0.40; 95% confidence interval [CI], 0.23–0.70), and higher adherence to disclosing harms/side effects (RR 2.04; 95% CI, 1.37–3.03) and funding source (RR 1.75; 95% CI, 1.18–2.60). Conclusions: Although professional medical writing support was not associated with increased overall adherence to CONSORT for Abstracts, important aspects were improved with professional medical writing support, including reporting of adverse events and funding source. This study identifies areas to consider for improvement.


Introduction
Prior studies demonstrate low levels of adherence to Consolidated Standards of Reporting Trials (CONSORT) guidelines 1,2 , as well as CONSORT for Abstracts 3,4 in reporting randomized controlled trials (RCTs). Professional medical writing support, correctly acknowledged, is endorsed by Good Publication Practice (GPP3) 5 , and its prevalence increased between 2001/2002 and 2009/2010, with a reported doubling to nearly 35% of industry-sponsored studies 6 . Professional medical writing support is associated with increased adherence to CONSORT in articles reporting RCTs; in a sample of open-access journals, the number of articles that completely reported ≥50% of the studied CONSORT items was significantly higher with professional medical writing support (39%) than without professional medical writing support (21%; p<0.05) 7,8 .
The purpose of this study was to determine whether professional medical writing support was also associated with improved adherence to CONSORT for Abstracts by analyzing a published dataset from five high-impact general medical journals with overall variable and incomplete adherence 9 .

Methods
We examined data from a published cross-sectional study of 463 articles reporting RCTs 9 . The RCTs were published between 2011 and 2014 in five top medical journals: The New England Journal of Medicine, Annals of Internal Medicine, The Lancet, The BMJ, and JAMA. We determined the association between professional medical writing support and the reporting of CONSORT for Abstracts items 10 (Table 1) using a Wilcoxon rank-sum test. One author (CS), who was blinded to the CONSORT for Abstracts scores using de-identified original dataset outputs, identified articles as being prepared with professional medical writing support using automated searching of the full text article followed by manual review if they acknowledged the involvement of one of the following: medical writer, medical writing, writing services, writing assistance, editorial assistance, or editorial support. The context of these terms was also examined.
Mean proportions of CONSORT for Abstract adherence with and without professional medical writing support was compared using a Wilcoxon rank-sum test, and then tested with additional effect of variable journal adherence using an analysis of variance (ANOVA). The relative risk (RR) and 95% confidence interval (CI) for each item's adherence with and without professional medical writing support were calculated using the command "oddsratio" in the R package fmsb 0.5.2 11 . All statistical analyses were performed in R 3.2.2 12 .

Results
From the original published dataset of 463 abstracts from RCTs reported in five journals, acknowledged professional medical writing support was observed in 66 articles (14.3%). Two articles identified in the automated search were excluded on manual review, one of which stated 13 , "there was no writing assistance from anyone who is not listed as an author," and the other 14 , "the Writing committee drafted the report… without editorial assistance." The mean proportion of CONSORT for Abstracts items reported in articles with (n=66) and without (n=397) professional medical writing support was 64.3% versus 66.5%, respectively; p=0.3044 (Wilcoxon rank-sum test). This difference remained nonsignificant when journal variation in CONSORT for Abstracts adherence was considered (ANOVA, p=0.1347). Overall, reporting of the individual CONSORT for Abstracts items was similar with and without professional medical writing support (RR 0.97; 95% CI, 0.88-1.07) ( Figure 1). However, a lower rate of reporting the study setting (item 4) was observed in articles with professional medical writing support (RR 0.40; 95% CI, 0.23-0.70). Conversely, professional medical writing support was associated with higher adherence to reporting both harms and side effects (item 16) (RR 2.04; 95% CI, 1.37-3.03) and source of funding (item 19) (RR 1.75; 95% CI, 1.18-2.60).

Discussion
Although professional medical writing support was not associated with increased overall adherence to CONSORT for Abstracts, important aspects were improved with professional medical writing support, including reporting of adverse events and funding source. These data confirm prior evidence showing that professional medical writing support is associated with improved safety reporting 15 , and serve to support the important role of professional medical writers in promoting adherence to Medical Publishing Insights & Practices recommendations to improve adverse event reporting in clinical trial publications 16 . Reporting of funding source was also improved with professional medical writing support, most likely reflecting the emphasis placed on transparency about funding by GPP3 guidelines 5,17 . In articles with professional medical writing support, we observed 100% adherence for other important CON-SORT for Abstract items, including reporting of the clinical trials registration number (item 18), vital for transparency and study tracking, which has recently been automated using technology such as the TrialsTracker 18 .
In this post hoc analysis, there was substantial risk of false positives given that separate confidence intervals were computed for 19 different CONSORT items. However, the p value for

Amendments from Version 1
We have revised our publication to further clarify the methodology and robustness of the observed differences in CONSORT for Abstracts adherence with and without professional medical writing support, and in the context of the published literature. Outcome 7. Do the authors explicitly state the primary or main outcome for the trial and when it was assessed (e.g. the time frame over which it was measured)?

REVISED
Randomization 8. Do the authors clearly describe the method for random sequence generation in assigning participants to interventions?
9. Do the authors clearly describe the method of allocation concealment?
Blinding 10. Is the study "blinded" or "masked" to group assignment?

12.
Is the number of participants randomized to each group stated?
13. Is the number of participants analyzed for each group stated?
14. Is the primary outcome result for each group stated?
15. Is there an estimated effect size and precision (i.e. are confidence intervals given)?
16. Is there an explicit statement of harms or side effects, or an explicit statement of their absence? Harms and side effects do not have to be specifically labeled as such. Abstracts can satisfy this criterion by stating specific outcomes that are not the primary objective of the study and could reasonably be considered a harm or side effect of the intervention or treatment. This is typically stated in the last sentence of the results. setting (item 4) was 0.0010, for harms or side effects (item 16) was 0.0004, and for source of funding (item 19) was 0.0055; the first two survive a Bonferroni correction, which would conservatively demand that p<0.0026, and all three pass the less stringent Benjamini-Hochberg procedure. Nevertheless, we were disappointed to see that professional medical writing support was not associated with improvements in reporting of other CONSORT for Abstract items, including specification in the title of the design of the study (item 2) and that it was randomized (item 1), and reporting of the numbers randomized and analyzed (items 12 and 13). Indeed, professional medical writing support was actually associated with worse reporting of one item, study setting (item 4) which may be explained by a lack of prioritization by professional medical writers within the constraints of abstract word count limitations. These areas represent clear areas in which professional medical writers can help further improve the reporting of clinical trials in the abstracts of journal articles. The relatively small number of studies with professional medical writing support is this dataset (n=66) did not allow for further exploration of these data (e.g. type of funding, trial phase, institutions where performed). It would be of interest to determine if similar trends are observed in separate studies of CONSORT for Abstracts adherence.
Although this was a post-hoc analysis, it has the advantage that CONSORT for Abstracts adherence was assessed before our study question was posed. Additionally, the presence of professional medical writing support was assigned by an assessor who was blinded to the CONSORT for Abstracts score. However, in the original study, inter-rater agreement for scoring was 84%, which is suboptimal 9 . Furthermore, the original dataset was limited to a sample of high-impact journals, and so may not be generalizable to the biomedical literature as a whole. Indeed, in this dataset of high-impact journals, adherence to CONSORT for Abstracts is likely to have been influenced by the journal's in-house scientific editing; consequently, the impact of professional medical writing support on adherence to CONSORT for Abstracts may be greater in journals without professional in-house editing. These data were potentially confounded by funding source; because professional medical writing support is typically restricted to industry-funded studies, it is possible that the review processes followed by industry 19 , rather than professional medical writing support per se, caused the improvements in reporting that we observed. However, in industry-funded articles, professional medical writing support was associated with a greater than two-fold increase in ≥50% adherence to CONSORT items studied compared with industry-funded articles prepared without this support (38% vs 18%, p<0.05) 7,8 . In addition, industry funding alone had no impact on the quality of CONSORT reporting in the absence of professional medical writing support 7,8 . Nevertheless, it can be difficult to correctly ascribe the role of the funder from the details provided in manuscripts as, for example, investigator-led studies typically undergo a different review process to those conducted with full industry support 20 . Finally, although Good Publication Practice guidelines (GPP3) encourage transparency of professional medical writing support 21 , it remains possible that it was not consistently acknowledged in the studied dataset. In a systematic review of the medical literature, the prevalence of ghostwriting was found to be approximately 5% 22 . Any unacknowledged professional medical writing support would tend towards the null hypothesis enhancing the confidence in the observed statistical differences.
In summary, although professional medical writing support was not associated with increased overall adherence to CONSORT for Abstracts, important aspects were improved with professional medical writing support, including reporting of adverse events and funding source. Ensuring adherence to reporting guidelines is a complex task, so we believe that there is a role for reporting professionals such as professional medical writers to work with authors and journals, to provide training, writing and reviewing, and thereby improve the quality of reporting of clinical trials.

Dataset 1: Dataset for reporting of CONSORT for Abstracts items in articles with and without acknowledged professional medical writing support. Dataset used per Hays et al. 9
with the addition of column C for this post hoc analysis by professional medical writing support (yes: 1; no: 0). doi, 10.5256/ f1000research.12268.d172437 23

Competing interests
Mills I is an employee of PAREXEL International. Sheard C, Hays M, and Douglas K have no disclosures to report. Gattrell WT is an employee of Ipsen Biopharma. Winchester CC is an employee and Director of Oxford PharmaGenesis Ltd, and a Director and shareholder of Oxford PharmaGenesis Holdings Ltd.

Grant information
The author(s) declared that no grants were involved in supporting this work. reporting checklist. The study is thus quite methodologically biased, and it is not clear whether the observed differences are real, or random. For example, the finding that the acknowledgment of professional writing assistance decreases the frequency of adequately reporting the setting of the study is difficult to explain. The potential randomness of the significant findings should be addressed in the Discussion section.

Open Peer Review
Making a publication out of the original publication by looking at a single characteristic does not really warrant the publication of the submitted manuscript. It would be preferable if other characteristics were explored in relation to reporting completeness in abstracts, such as the type of funding, phase of the trial, institutions where trials were performed.
The manuscript is well written and structured, and there are no major comments or suggestions for improvement except the conceptual one described above. In the Methods section, the extraction of data for this study is not fully explained -how many experts extracted the data, how were different terms in the statements on writing assistance interpreted and categorized, etc.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes We thank for her insight and providing suggestions to improve our paper.

Ana Marušić
We feel that the data described herein fulfill the criteria for publication as a "Research Note" in F1000 Research, designed to report "single-finding papers that can be reported with one or two illustrations." Our intention was to conduct a post hoc analysis to specifically examine the role of F1000 Research, designed to report "single-finding papers that can be reported with one or two illustrations." Our intention was to conduct a post hoc analysis to specifically examine the role of professional medical writing support on CONSORT adherence as previously described by two of the authors.
In the discussion, we acknowledge that there was substantial risk of false positives given that separate confidence intervals were computed for 19 different CONSORT items. However, the p value for setting (item 4) was 0.0010, for harms or side effects (item 16) was 0.0004, and for source of funding (item 19) was 0.0055; the first two survive a Bonferroni correction, which would conservatively demand that p<0.0026, and all three pass the less stringent Benjamini-Hochberg procedure.
We now include a note in the discussion that the limited number of articles with acknowledged professional medical writing support did not allow further exploration of the results (e.g. type of funding, trial phase, institutions where performed) and also mention that it would be of interest to examine whether similar trends are observed in separate studies of CONSORT for Abstracts.
As described in the original submission, one author performed the data extraction conducting a blinded automated search per the defined terms followed by reexamining for context. We note two articles that were excluded from the analysis by manual review and provide rationale. In one case there was no writing assistance from anyone not listed as an author and in the second case the "Writing committee drafted the report…without editorial assistance." We have made it clear that this was performed using the full text articles.
No competing interests were disclosed. Competing Interests: