Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study

Background: An important principle in the good conduct of clinical trials is that a summary of the trial protocol, with a pre-defined primary outcome, should be freely available before the study commences. The clinical trials registry ClinicalTrials.gov provides one method of doing this, and once the trial is registered, any changes made to the primary outcome are documented. The objectives of this study were: to assess the proportion of registered trials on ClinicalTrials.gov that had the primary outcome changed; to assess when the primary outcome was changed in relation to the listed study start and end dates and to assess whether the primary outcome change had any relation to the study sponsor. Methods: A cross-sectional analysis of all interventional clinical trials registered on ClinicalTrials.gov as of 25 October 2012 was performed. The main outcome was any change made to the initially listed primary outcome and the time of the change in relation to the trial start and end date. Findings: Our analysis showed that 28229 of 89204 (31.7%) registered studies had their primary outcome changed. Industry funding was associated with all primary outcome changes, odds ratio (OR)= 1.36, 95% confidence interval (CI)=1.31-1.41, p<0.001; with primary outcome changes after study start date OR=1.37, 95% CI=1.32-1.42, p<0.001; with primary outcome changes after primary completion date OR=1.84, 95% CI=1.75-1.94, p<0.001 and with primary outcome changes after study completion date OR=1.82, 95% CI=1.73-1.91, p<0.001. Conclusions A significant proportion of interventional trials registered on ClinicalTrials.gov have their primary outcomes altered after the listed study start and completion dates. These changes are associated with funding source.

Clinical trials provide the principal method with which to assess the effectiveness of therapeutic strategies 1 . An important principle in the good conduct of clinical trials is that a summary of the trial protocol, with a pre-defined primary outcome, should be freely available before the study commences 1 . In February 2000, the United States (US) Food and Drug Administration (FDA) created an online clinical trials registry named ClinicalTrials.gov 2 . From 2005, the International Committee of Medical Journal Editors (ICMJE) required that clinical trials should be listed in a clinical trial registry to qualify for publication 3 . The registration of a clinical trial usually involves reporting informationon 20 items proposed by the World Health Organization (WHO) registration advisory group, including the primary outcome of the study 4 . One reason for the creation of this registry was to help to reduce the risk of selective reporting of outcomes that had been previously identified. For example, a cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen andFrederiksberg in Denmark in 1994-1995 found that 62% of trials had at least one primary outcome that was changed, introduced, or omitted 5 . A more recent study looking at trials that were registered on trial websites (such as ClinicalTrials.gov), found that 31% of trials displayed some evidence of discrepancies between the outcomes registered and the outcomes published 6 . Therefore, even trial registration may not be a complete barrier to selective outcome reporting.
ClinicalTrials.gov tracks all changes made to registered protocols. The objectives of this study were: to assess the proportion of registered trials on ClinicalTrials.gov that had the primary outcome changed; to assess when the primary outcome was changed in relation to the listed study start and end dates and to assess whether the primary outcome change had any relation to the study sponsor.

Data source
ClinicalTrials.gov is a publicly available trial registry and results database developed and maintained by the US National Library of Medicine on behalf of the US National Institutes of Health. The following information was collected from each study: study registration date (the date the study is registered with ClinicalTrials.gov); the study start date (defined as the date that enrollment to the protocol begins); the primary completion date (defined as the anticipated or actual date the final subject was examined or received an intervention for the purposes of final collection of data for the primary outcome, whether the clinical trial concluded according to the pre-specified protocol or was terminated); study completion date (defined as the final date on which data was collected); original primary outcome (defined as a specific key measurement(s) or observation(s) used to measure the effect of experimental variables in a study) and date submitted; current primary outcome and date submitted; study phase (phase of investigation as defined by the US FDA), data monitoring committee (whether an independent group of scientists has been appointed to monitor the safety and the scientific integrity of a human research intervention, and to make recommendations to the sponsor regarding the termination of the trial for efficacy, for harm or for futility); study sponsor (defined as the primary organization that oversees the implementation of the study and is responsible for data analysis) and collaborators (defined as other organizations (if any) providing support, including funding, design, implementation, data analysis and reporting).

Study sample
For studies to be included in this analysis a primary outcome had to be registered. A study was classified as not having a primary outcome changed if the original primary outcome was listed as 'same as current'. For a subset of interventional trials, specifically those that had 'multiple sclerosis' or 'diabetes' in the title, we looked for primary outcomes that had a significant change, as the authors have experience in these fields. We defined a discrepancy between primary outcomes if the two were clearly different (e.g. in one study the primary outcome was changed from 'expanded disability status scores assessed every 12 weeks' to 'annualized relapse rate'). Two authors (SVR and JP) independently assessed whether or not the studies had significant changes; then they met to compare results. For any discrepancies they met with another author (LH) and a consensus was reached. The kappa statistic for agreement between the first two observers was 0.87.
ClinicalTrials.gov stores funding organization information using two data elements: lead sponsor (defined as the organization or person who oversees the clinical study and is responsible for analyzing the study data) and collaborator (defined as an organization other than the sponsor that provides support for a clinical study). We derived probable funding source from the lead sponsor and collaborator fields using the following algorithm: if the lead sponsor and any collaborators were from industry and no non-industry sponsors or collaborators were listed, then the study was categorized as industry funded; if the lead sponsor and/or collaborator included industry and non-industry sources, then the study was categorized as mixed; finally if the lead sponsor and any collaborators were not from industry and no industry sponsors or collaborators were listed, then the study was categorized as non-industry funded. Two authors (SVR and JP) independently appraised the funding status of all studies; then they met to compare results. For any discrepancies they consulted another author (LH) and a final consensus was reached. The kappa statistic for agreement between the first two observers was 0.76.

Statistical analysis.
We used logistic regression to calculate odds ratios (ORs) and 95% confidence intervals (95% CI) for comparisons between outcome changed and non-outcome changed groups, using registration date and funding source as explanatory variables.
had their primary outcome changed after the study had supposedly started. Funding source and registration year were associated with trials that had their primary outcome changed after the study start date ( ,748 trials that also had a study start date, 23,080 that also had a primary completion date and 21,747 that also had a study completion date. Funding source was associated to all primary outcome changes, primary outcome changes after listed study start date and primary outcome changes after listed primary completion date following adjustment for study phase and presence of a DMC (Table 3). Study phase and use of a DMC were also associated to primary outcome changes. All associations remained similar when restricting to completed trials, although the associations of mixed funding with all primary outcome changes and primary outcome changes after study start date became non-significant.

Significant primary outcome changes in multiple sclerosis interventional trials
A subset of interventional trials in multiple sclerosis was investigated to assess the actual change in primary outcome. Fifty out of 422 (11.9%) registered trials were deemed to have a significant change in the primary outcome, 49 out of 416 (11.8%) were changed after the study start date, 32 out of 187 (17.1%) were changed after the primary completion date and 26 out of 191 (13.6%) were changed after the study completion date. After adjusting for study registration year, industry funding was associated with all primary outcome changes (OR=4.92, 95% CI 2.12-11.41, p<0.001), primary Table 2

Summary
We assessed the proportion of primary outcome changes for interventional trials registered on ClinicalTrials.gov from 1999 to 2012, when these changes occurred and whether these changes related to funding source. Industry funding was associated with primary outcome changes. These associations remained even after adjusting for the phase of the study and the presence of a DMC. The changes appeared to peak in the period between 2004 and 2007. This may suggest that a lower number of trials were registered before this time period and perhaps that there was not enough time to accrue for changes to be made in more recently registered trials (or that the changes made to the protocol decrease over time). When looking at significant primary outcome changes, the proportion of trials with their outcome changed was much less than all changes; nevertheless, industry funding was still associated with significant primary outcome changes.
There are many reasons for departures from the initial study protocol. Authors should identify and explain any such changes, however Table 3

. Association of funding status, registration year, use of a data monitoring committee and study phase with a) all primary outcome changes, b) primary outcome changes after listed study start date, c) primary outcome changes after listed primary completion date on ClinicalTrials.gov and d) primary outcome changes after listed study completion date on ClinicalTrials.gov.
OR=odds ratio.   in the protocol 7 . The proportion of primary outcome changes that we found was perhaps lower than that found previously, but this may be due to the different methodology used in this study. The relationship between protocol changes and funding has not been thoroughly investigated. Chan and colleagues found that 61% of the 51 trials with major discrepancies between the study protocol and publication were funded solely by industry sources compared to 49% of the 51 trials without discrepancies 5 .

Conclusions
Primary outcome changes are made to study protocols registered on ClinicalTrials.gov and these changes are associated with funding source. Author contributions SVR and BG conceived and designed the study. SVR, JP, LH, APS, MK and DM analysed the data. SVR and BG interpreted the data. SVR drafted the article. All authors revised the article and gave final approval for publication.

Competing interests
The authors declare no competing interests.

Grant information
The author(s) declared that no grants were involved in supporting this work.
no such information is given on ClinicalTrials.gov and thus we have no explanation for our results. The results obtained here may suggest that industry funded trials are more diligent in reporting changes to protocols. Indeed, the primary outcome measure data element was not available in ClinicalTrials.gov until late 2004 and it was considered an optional data element until December 2012. There are implications of the data reported here for medical journals, reviewers and drug approval agencies.

Limitations of this study
A limitation for our study is the scope of the data. We used data from only one trial website, and thus the generalisability to other registration sites is unclear. We have also assumed the data entered regarding study start and completion dates were accurate. We have no information on the reasons for the changes being made. We also did not assess the significance of the primary outcome change for the vast majority of changes; the sub-studies in multiple sclerosis and diabetes suggested that significant changes may only occur in approximately 8% of all registered trials, but this may or may not be applicable to trials for other disorders. Some primary outcome changes may be typographical/semantic and may not reflect actual changes to the nature of the outcome (although one would expect these to occur equally regardless of funding source).
Background from other studies Previous reports have not investigated changes to a primary outcome as entered on a registration website, but they have compared trial protocols to published studies. Four previous studies have shown that in 47-74% of studies the primary outcome stated in a protocol was the same as in a subsequent publication; between 13 and 31% of primary outcomes specified in the protocol were omitted in the publication and between 10 and 18% of reports introduced a primary outcome in the publication that was not specified This is an interesting report with some interesting results. Upon reading the report it did lead to the thinking of possible confounding factors which could explain some of the differences if not all.
Some of the confounding factors could not be easily investigated in the current data set and I will consider these first.
The first is whether the treatment being investigated in the trial is a licensed or unlicensed therapy. Though in theory "do-able" the authors can be forgiven for not investigating this. However, could it be proposed that a trial of a new unlicensed treatment where knowledge is emerging -and being published as the trial is ongoing -is more likely to have an endpoint changed than a licensed treatment with known properties?
The other consideration is therapeutic area. The authors to a degree looked into this by investigating two sub populations of MS and diabetes trials but it could be that certain therapeutic areas predominate in certain sectors?
One final consideration is if the endpoint change was pre-planned in some way in some form of adaptive design.
With respect to confounders which could be investigated the main consideration is study duration. Might a study that has gone on longer be more likely to have an endpoint change? This could be legitimate if, as mentioned previously this change is due to published work becoming available. It could also be (maybe) legitimate if, for completed studies, it has dragged on for an age with no chance to reach the target sample size. So to a degree there is a case of fitting tailoring the study to what can be answered with the sample size. The interaction between study duration and funder would also be interesting (as well as just allowing in the model) There are two markers in the analysis which do suggest study duration may impact on endpoint change. The first is study registration year and the other is phase of development. In the case of phase of development the effect seems to be the reverse, it could be argued, to what one would expect. Early phase trials are learning trials not confirming trials. A consequence is the endpoints may change to reflect the properties of a treatment an investigator is learning about in an ongoing manner. It would be interesting with study duration in the model how it would effect the comparison of phase of development Following on from the point in the previous paragraph therefore an analysis of just trials which are Phase III would be of interest. These trials should have everything set and pre-specified in terms of analysis protocols. This is a possible area for change which has been provided in this article.

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
No competing interests were disclosed. Competing Interests: