Reporting characteristics of allergic rhinitis trials registered on ClinicalTrials.gov and in publications: an observational study

Ethics policies

All methods were performed in accordance with the relevant guidelines and regulations and there was no need for approval from an institutional review board (Ethics Committee of the University of Split School of Medicine), as this study is cross-sectional and historical cohort database study. We neither collected patient data nor performed experimental procedures.

Furthermore, all data used in this study are publicly available in the ClinicalTrials.gov registry and therefore no permission was required to access the data.

This study neither studied nor analyzed the raw data of other studies. We analyzed their data which were publicly registered in ClinicalTrials.gov and published in publicly available scientific journals. So no permission was needed here either.

Data from other studies registered in the ClinicalTrials.gov registry, which we analyzed, were anonymized by their authors, so no anonymization was required on our study.

Study period and data sources

Figure 1 shows our retrospective analysis. The start date (9/27/2009) of the search allowed at least two years for the study to post in ClinicalTrials.gov and publish results in journals. The last search date (10/4/2019) allowed more than 10 years from first to last posting and publishing of results.

Figure 1. Study flowchart.

Flow diagram of retrospective cross-sectional study with inclusion and exclusion criteria. FDAAA: The Food and Drug Administration Amendment Act, FDA: The Food and Drug Administration.

We accessed the history of all changes in registration elements through the ClinicalTrials.gov Archive site. We searched ClinicalTrials.gov for completed RCTs using the following keywords: “allergic rhinitis”, “nasal allergies”, “rhinoconjunctivitis”, “hay fever”, and “atopic rhinitis”. We did not use Medical Subject Headings (MeSH) terms.

For trials with publications provided on ClinicalTrials.gov, we selected only those that reported the results of the current trial to find corresponding publications to our sample of AR RCTs. For trials without publications provided, we searched PubMed, Web of Science, Scopus, and Google Scholar with the National Clinical Trial (NCT) identifier provided by ClinicalTrials.gov in the trial record that is usually listed in the abstract or main text of published articles.²¹ If the initial search failed, we searched using the principal investigator's name and study title. Only full publications were compared with registered data. Publication dates were determined from online publication dates for articles made available ahead of print.²²

Sample

Inclusion criteria are shown in Figure 1. Requirements for an applicable clinical trial according to the FDAAA 801 were verified according to the “Elaboration of definitions of responsible party and applicable clinical trial (ACT)” from 3/9/2009 for trials initiated after 9/27/2007 and according to the “Checklist for evaluating whether a clinical trial or study is an ACT” for those initiated after 1/18/2017.²³

Data extraction and comparisons

Differences between first and last registration were assessed from 20 out of 24 extracted WHO TRDS items, while we assessed differences between last registration and publications from 15 out of 24 items. The “Contact for Public Queries” and “Contact for Scientific Queries” items were excluded from both analyzes due to their unavailability in ClinicalTrials.gov while “Recruitment Status” item was excluded due to it's transient nature. Furthermore, the “Date of Registration in Primary Registry”, “Secondary Identifying Numbers”, “Secondary Sponsor(s)” and “Public Title” items were excluded from the analysis of the differences between last registration and publications due to their exclusion from journals. The “IPD Sharing Statement” and “Ethics review” items added to the WHO TRDS in November 2017 were not included in the analysis for trials started before the WHO TRDS extension. Also the “IPD Sharing Statement” was not included in the analysis for publications before the mandatory data sharing statement from July 1, 2018.²⁴ In contrast, the “Completion Date” and “Basic Results” (comprises the current “Summary Results” item) items were required by the FDAAA 801 in ClinicalTrials.gov¹⁵ and were therefore analyzed. Additionally, the “All-cause Mortality” item in the Adverse Events (AEs) section was required by the Final Rule of the FDAAA, so we did not analyze this item for trials with a primary completion date before its implementation in 2017. We studied the reporting of deaths of such trials from other elements of the outcome data, primarily from Serious AEs (SAEs).

We determined the completeness and informativeness, i.e., missing information or uninformative terminology (unspecified or unclear information for the relevant registry item such as a code instead of a generic name of a drug), respectively, at first and last registration and in publications of WHO TRDS items.^8,25 We used previous methods²⁵ to evaluate the history of changes in WHO TRDS items from first to last registration and changes between last registration and corresponding publications. Changes were defined as qualitative (difference in the meaning of the provided information) or quantitative (difference in a numerical entry).²⁵ We modified criteria developed by Chan et al.⁸ to describe discrepancies between registered and published outcomes as: a new primary outcome introduced in the article, an omitted registered primary or secondary outcome in the article, a registered primary or secondary outcome switched in the article or vice versa, a new secondary outcome introduced in the article, a registered primary or secondary outcome changed in the article, a new outcome methodology introduced or the registered one changed in the article, the timing of assessment changed in the article, and a new primary or secondary outcome introduced in the article as a combination of registered outcomes.

The completeness of results reporting from first to last registration and from last registration to the publication were determined by the presence of Participant Flow elements, Baseline Characteristics elements, Outcome Measures elements, and AEs.

Additionally, we analyzed these discrepancies in data reporting between trials with prospective and retrospective trial registration in ClinicalTrials.gov as well as between industry vs. non-industry sponsored trials. Trials with prospective registration started during or after first registration, and trials with retrospective registration started before it. We also analyzed the time differences between several points in the registration and publication.

Two investigators (IP and SP) independently extracted data in parallel from the entire cohort of trials for completeness, informativeness and changes to avoid potential data collector bias from possible subjective interpretation. Inter-rater reliability was high for changes in the WHO TRDS items (kappa range 0.83 to 1.00). We resolved through consensus discussion the differences in our interpretation of the secondary outcome changes, which had the lowest kappa in any single category (0.83, 95% confidence interval (CI) 0.73 to 0.94). We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting observational studies.²⁶ There was no need for approval from an institutional review board (Ethics Committee of the University of Split School of Medicine), as this study is a cross-sectional and historical cohort database study. We neither collected patient data nor performed experimental procedures.

Statistical analysis

Data extracted from ClinicalTrials.gov were coded and then entered into a Microsoft Excel spreadsheet. Dichotomous variables were registration period (retrospective or prospective), sponsorship (industry or non-industry) and completeness of registration (trial with or without missing registration items). The number of major changes and trial duration in months were treated as nonparametric, and were categorized based on a median split, where values were dichotomized as 0, when less than or equal to their respective median, and as 1 when greater than their respective medians. Differences in the trials, completeness, and major changes to the WHO TRDS items were compared using the Chi-square test. Frequencies, medians with the 95% CI were presented. Frequencies were compared using Chi-squared tests. We used IBM SPSS Statistics 21 (SPSS, Inc., an IBM company, Chicago, IL, USA, RRID:SCR_002865) for the analyses. Differences were considered significant at a P<.05.

General characteristics

A total of 76 trials were analyzed and 42 (55.3%) were published, two of which were in Japanese and one was only available as a Supplement. Thus, 39 (51.3%) publications were available for analysis. All were published in journals that follow ICMJE recommendations and only one (2.6%) was an ICMJE member.²⁴ Further, there were 16 (41.0%) out of the 39 published trials where the corresponding publication was published before the results were posted in ClinicalTrials.gov^[1]. Most trials in ClinicalTrials.gov were phase 3 (n=41 [53.9%]), quadruple-blind (n=33 [43.4%]), parallel group (n=64 [84.2%]) and placebo controlled (n=57 [75.0%]). In contrast, most published trials were double-blind (n=32 [82.1%]) (Table 1).

All 76 trials started before the implementation of the Final Rule and there were 22 (28.9%) trials with retrospective registration in ClinicalTrials.gov. Table 2 shows the median elapsed time between several points in the registration and publication of the trials. Most trials (n=64 [84.2%]) as well as most publications (n=36 [92.3%]) were industry-sponsored. We found that industry-sponsored compared to non-industry funded trials had a statistically significant shorter duration (χ²=6.848, P=.009).

Completeness of the WHO TRDS items

Table 3 shows which WHO TRDS items are most often missing at first and last registration and in publications. At least 1 WHO TRDS item was missing in 35 (46.1%) trials at first registration, in 15 (19.7%) at last registration and in 22 (56.4%) publications. There were statistically significantly more incomplete first registrations in trials with prospective registration than in those with retrospective (n=32 [91.4%] vs. n=3 [8.6%]); χ²=13.096, P≤.001.

Major WHO TRDS item changes

All registered trials had major changes in WHO TRDS items in the period between first and last registration as well as all publications (Tables 4-5 and Supporting Tables 1-3 in Extended data). The median major changes per trial between first and last registration was 31 (95% CI 35.73 to 65.29), and per publication it was 28 (95% CI 24.6 to 32.2) (Table 4). There was no difference in the number of major changes to WHO TRDS items per trial between first and last registration. as well as between last registration and publication according to prospective or retrospective trial registration; χ²=3.003, P=.083 and χ²=2.820, P=.093, respectively.

Primary and Key Secondary Outcome changes

Predominantly, methodological details were changed in both primary and secondary outcomes between first and last registration as well as in publications (Table 5). Furthermore, primary outcome changes between first and last registration and in publications included the addition of new outcomes and changes to existing ones. According to prospective or retrospective trial registration, we found no difference in the number of major changes to primary and secondary outcomes between first and last registration as well as between last registration and publication; χ²=0.616, P=.433; χ²=2.758, P=.097; χ²=2.201, P=.138 and χ²=2.820, P=.093, respectively.

Informativeness of the WHO TRDS items

We found uninformative reporting of WHO TRDS items in ClinicalTrials.gov and publications (Table 6). The median uninformative WHO TRDS items per trial at first registration was 3 (95% CI 3.1 to 3.8), 1 (95% CI 1.3 to 1.9) at last registration, while in publications it was 0 (95% CI 0.08 to 0.43). At first registration, most trials had uninformative “Primary and Key Secondary Outcomes” and “Intervention(s)” items. The outcomes were uninformative mainly due to their unclear methodology that was clarified by last registration, and the interventions mainly due to a missing time frame (Table 6). Table 6 also shows that registered and published interventions from RCTs on AR had uninformative descriptions. 28 (71.8%) publications had a less informative “Study Type” than in ClinicalTrials.gov, mostly due to omitted study phase (Supporting Table 3 in Extended data).

Completeness of the reporting of results

Regarding participant characteristics, all trials in ClinicalTrials.gov reported Participant overall flow and Baseline population descriptions while only 1 (2.6%) out of 39 publications did not. All 76 trials reported SAEs and Other AEs (OAEs) in ClinicalTrials.gov. In contrast, 30 (76.9%) publications reported SAEs and 38 (97.4%) OAEs. Due to primary completion dates after 2017, the reporting of the “All-cause Mortality” item was analyzed for only nine (11.8%) trials, all of which have a reported item in ClinicalTrials.gov. Of these, four were published and two of them reported deaths. There were only two (2.6%) trials that reported deaths, both as an SAE. Of these, both were published, and only one publication reported it. Overall, deaths were reported in 20 (51.3%) publications.