Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study

Sreeram V Ramagopalan; Andrew P. Skingsley; Lahiru Handunnetthi; Daniel Magnus; Michelle Klingel; Julia Pakpoor; Ben Goldacre

doi:10.12688/f1000research.6312.2

Home Browse Funding source and primary outcome changes in clinical trials registered...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Note

Revised

Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study

[version 2; peer review: 2 approved]

Sreeram V Ramagopalan ¹, Andrew P. Skingsley², Lahiru Handunnetthi³, [...] Daniel Magnus⁴, Michelle Klingel⁵, Julia Pakpoor³, Ben Goldacre⁵

Sreeram V Ramagopalan ¹, Andrew P. Skingsley², [...] Lahiru Handunnetthi³, Daniel Magnus⁴, Michelle Klingel⁵, Julia Pakpoor³, Ben Goldacre⁵

PUBLISHED 24 Apr 2015

Author details Author details

¹ SVR Consulting, London, UK
² Imperial College Medical School, London, UK
³ Medical Research Council Functional Genomics Unit and Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK
⁴ School of Social and Community Medicine, University of Bristol, Bristol, UK
⁵ Department of Epidemiology, London School of Hygiene and Tropical Medicine, London, UK

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the All trials matter collection.

Abstract

Background: We and others have shown a significant proportion of interventional trials registered on ClinicalTrials.gov have their primary outcomes altered after the listed study start and completion dates. The objectives of this study were to investigate whether changes made to primary outcomes are associated with the likelihood of reporting a statistically significant primary outcome on ClinicalTrials.gov.
Methods: A cross-sectional analysis of all interventional clinical trials registered on ClinicalTrials.gov as of 20 November 2014 was performed. The main outcome was any change made to the initially listed primary outcome and the time of the change in relation to the trial start and end date.
Findings: 13,238 completed interventional trials were registered with ClinicalTrials.gov that also had study results posted on the website. 2555 (19.3%) had one or more statistically significant primary outcomes. Statistical analysis showed that registration year, funding source and primary outcome change after trial completion were associated with reporting a statistically significant primary outcome.
Conclusions: Funding source and primary outcome change after trial completion are associated with a statistically significant primary outcome report on clinicaltrials.gov.

Keywords

clinical trials, funding, primary outcome,

Associated Short Research Article Sreeram V. Ramagopalan, Andrew P. Skingsley, Lahiru Handunnetthi, Michelle Klingel, Daniel Magnus, Julia Pakpoor, Ben Goldacre » Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study, F1000Research 2014, 3:77 (https://doi.org/10.12688/f1000research.3784.1)

Corresponding authors: Sreeram V Ramagopalan, Ben Goldacre

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2015 Ramagopalan SV et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Ramagopalan SV, Skingsley AP, Handunnetthi L et al. Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study [version 2; peer review: 2 approved]. F1000Research 2015, 4:80 (https://doi.org/10.12688/f1000research.6312.2) First published: 26 Mar 2015, 4:80 (https://doi.org/10.12688/f1000research.6312.1) Latest published: 24 Apr 2015, 4:80 (https://doi.org/10.12688/f1000research.6312.2)

Revised Amendments from Version 1

We thank the reviewers for their comments. We have updated the manuscript with the following:

1) Defining how a primary outcome change was defined.
2) Highlighting the differences in effect direction for associated variables.
3) Providing an analysis controlling for study phase (results materially unchanged)
4) Performing the analysis after excluding potential non-inferiority trials (results materially unchanged)

To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.

Introduction

Clinical trials provide the principal method with which to assess the effectiveness of therapeutic strategies¹. An important principle in the good conduct of clinical trials is that a summary of the trial protocol, with a pre-defined primary outcome, should be freely available before the study commences¹. In February 2000, the United States (US) Food and Drug Administration (FDA) created an online clinical trials registry named ClinicalTrials.gov². We and others have shown a significant proportion of interventional trials registered on ClinicalTrials.gov have their primary outcomes altered after the listed study start and completion dates^3,4. In this extended analysis, we sought to investigate whether changes made to primary outcomes are associated with the likelihood of reporting a statistically significant primary outcome on ClinicalTrials.gov.

Methods

We used R (http://cran.r-project.org/web/packages/rclinicaltrials/vignettes/basics.html) to download data from all completed interventional clinical studies registered with ClinicalTrials.gov as of 20th November 2014, as previously described³. New to this study, we also downloaded data concerning study results for these trials; specifically the ‘p value’ fields from the ‘study results’ tab for primary outcomes.

We searched for potential non-inferiority studies by text searching for the word inferiority in the title.

Changes in primary outcomes were defined as previously described³. A study was classified as not having a primary outcome changed if the original primary outcome was listed as ‘same as current’. Probable funding source was derived using the algorithm previously described³.

A trial having a statistically significant primary outcome was defined as a trial having a P value less than 0.05 in the p value field in the study results tab for any primary outcome.

We used logistic regression to calculate odds ratios (ORs) and 95% confidence intervals (95% CI) for comparisons between significant primary outcome and non-significant primary outcome groups, using registration date, primary outcome change after study completion and funding source as explanatory variables. P-values <0.05 were interpreted as significant. Statistical analyses were conducted using the STATA 12.0.0 software.

Results

studyid	studyphase	hasdatacommittee	firstreceived	sponsortype	primaryoutcomemeasuresredefined	changedafterstudystart	changedafterprimarycompletion	changedaftercompletion	pubmedrefs	clingovrefs	had_results	delta_pom	sigprimary	sigsecondary	pomnumberchange	pomaftercompletion
NCT00000371	3		1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00000479	3	1	1999	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00000575	3	1	1999	1	1	1	1	1	1	1	1	0	0	0	0	0
NCT00000620	3	1	1999	0	1	1	1	1	1	1	1	0	0	1	0	0
NCT00001151	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001213	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001566	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001586	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001656	4	0	1999	0	1	1	1	1	0	1	1	0	1	0	0	0
NCT00001703	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001723	2	1	1999	1	1	1	1	1	0	1	1	0	1	0	0	0
NCT00001832	2	1	1999	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00001941	1	0	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00001959	2	0	2000	0	1	1	1	1	0	1	1	0	1	0	0	0
NCT00001984	2	1	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00002975	2	1	1999	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00003377	1	1	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00003590	2	0	1999	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00003659	2		1999	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00003782	3	1	1999	0	1	1	1	0	1	1	1	0	0	0	0	0
NCT00003869	3		1999	0	1	1	1	1	0	0	1	0	0	1	0	0
NCT00003896	2	0	1999	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00003907	2	0	1999	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00003910	2	0	1999	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00004143	2	1	1999	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00004500	3	1	1999	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00004547	2	0	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00004562	3	1	2000	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00004635	3	0	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00004859	3	1	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00004888	2		2000	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00004978	3	1	2000	1	1	1	1	1	1	1	1	0	0	1	0	0
NCT00004980	2	1	2000	0	1	1	1	1	0	1	1	0	1	1	0	0
NCT00005669	2	1	2000	1	1	1	1	0	1	1	1	0	0	0	0	0
NCT00005908	2	0	2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00005937	2	1	2000	2	1	1	1	1	0	1	1	0	0	0	0	0
NCT00005947	3		2000	0	1	1	1	1	1	1	1	0	0	1	0	0
NCT00006151		0	2000	1	1	1	1	1	0	1	1	0	0	0	0	0
NCT00006156			2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00006164	3	1	2000	1	1	1	1	0	1	1	1	0	0	0	0	0
NCT00006170	4	0	2000	0	1	1	1	1	1	1	1	0	1	0	0	0
NCT00006178	2		2000	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00006184	2	0	2000	0	1	1	1	0	0	1	1	0	0	0	0	0
NCT00006237	3	1	2000	0	1	1	1	1	0	0	1	0	1	0	0	0
NCT00006305	3	1	2000	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00006389	2		2000	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00006409	2	1	2000	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00006478	2		2000	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00007644	3	1	2000	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00008138	2	0	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00010257	2	0	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00010439	2	1	2001	1	1	1	1	1	0	0	1	0	0	0	0	0
NCT00010803	3	1	2001	0	1	1	1	0	1	1	1	0	0	0	0	0
NCT00013611	3	1	2001	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00014911	2		2001	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00015847	2	1	2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00016354	1	1	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00018031	2		2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00019604	2	0	2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00019747	2	1	2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00021229	1	1	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00021541	2	1	2006	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00022490	2	1	2001	0	1	1	0	0	0	0	1	0	0	0	0	0
NCT00022672	3		2001	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00023309	2	0	2001	0	1	1	1	0	1	1	1	0	1	1	0	0
NCT00023322	2	0	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00023673	1		2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00023712	2		2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00023764	2		2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00024102	3	1	2001	0	1	1	1	1	1	1	1	0	1	1	0	0
NCT00025662	2	1	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00027378	2	0	2001	0	1	1	1	1	1	1	1	0	1	0	0	0
NCT00027560	2		2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00028002	2		2001	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00028093	4	0	2001	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00028769	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00029107	2	0	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00029146	3	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00029172	4	1	2002	0	1	1	0	1	0	1	1	0	0	0	0	0
NCT00030901	3	1	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00030992	2	0	2002	0	1	1	1		0	1	1	0	0	0	0	0
NCT00031447	3		2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00031460	3		2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00031486	3		2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00032487	3	1	2002	1	1	1	1	1	1	1	1	0	1	0	0	0
NCT00032591	4	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00032630	3	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00033540	2	1	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00033657	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00033917	3		2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00035555	2		2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00035815	3	1	2002	1	1	1	1	1	0	1	1	0	0	0	0	0
NCT00035932	3	0	2002	2	1	1	1	1	0	0	1	0	0	1	0	0
NCT00036270	3	1	2002	2	1	1	1	0	1	1	1	0	0	0	0	0
NCT00036569	2	0	2002	0	1	1	1		0	1	1	0	0	0	0	0
NCT00037830	2	0	2002	0	1	1	1	1	0	0	1	0	1	1	0	0
NCT00038103	2	0	2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00038467	3	1	2002	1	1	1	1	1	0	0	1	0	1	1	0	0
NCT00038857	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00038948	3	1	2002	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00039130	2	0	2002	0	1	1	1		0	1	1	0	0	0	0	0
NCT00039377	2		2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00039741	2		2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00039871	3	1	2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00040365	2	0	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00040664	2		2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00041067	2	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00041080	3		2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00041132	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00041392	2		2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00041717	3	1	2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00041756	2	0	2002	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00041938	3	1	2002	0	1	1	1	0	1	1	1	0	0	1	0	0
NCT00042432	2		2002	2	1	1	1	1	0	1	1	0	1	1	0	0
NCT00042939	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00042991	1		2002	0	1	1	1	1	0	0	1	0	0	1	0	0
NCT00043186	2		2002	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00043979	2	0	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00044005	2	0	2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00044044	2	0	2002	2	1	1	1	1	0	0	1	0	1	1	0	0
NCT00044213	3	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00044512	2	0	2002	2	1	1	1	1	0	1	1	0	0	0	0	0
NCT00044655	4	1	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00045162	3	1	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00045487	2	0	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00045630	2	0	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00046228	3		2002	2	1	1	1	1	1	1	1	0	0	1	0	0
NCT00047385	3	1	2002	0	1	1	1	1	1	1	1	0	1	1	0	0
NCT00047463	2	1	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00047879	2	0	2006	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00048347	2	0	2002	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00048542	3	1	2002	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00048568	3		2002	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00048581	3		2002	2	1	1	1	1	0	1	1	0	1	1	0	0
NCT00048724	3	1	2002	2	1	1	1	1	1	1	1	0	0	1	0	0
NCT00048737	1	0	2002	1	1	1	1	1	1	1	1	0	0	0	0	0
NCT00048893	1	1	2002	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00048932	3		2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00049036	2		2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00049257	2	1	2002	1	1	1	1	1	0	0	1	0	0	0	0	0
NCT00049842	3	1	2002	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00050011	3	0	2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00050089		0	2002	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00050167	1	0	2002	1	1	1	1	0	0	0	1	0	0	0	0	0
NCT00050960	3		2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00050986	1	0	2002	1	1	1	1	1	0	0	1	0	0	0	0	0
NCT00051025	2		2002	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00051558	3	0	2003	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00051636	3		2003	2	1	1	1	1	0	0	1	-3	0	0	1	1
NCT00052962	3	1	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00053014	2	1	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00053365	2		2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00053417	2		2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00053482	2	1	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00053495	2	1	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00053703	4	1	2003	0	1	1	1	1	1	1	1	4	1	0	1	1
NCT00054275	2	1	2003	1	1	1	1	0	0	0	1	0	0	0	0	0
NCT00054327	2	1	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00054639	2		2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00054665	2	0	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00054847		1	2003	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00055237	2	0	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00055471	2		2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00055497	3	0	2003	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00056160	3	1	2003	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00056316	1	1	2003	0	1	1	1	1	0	1	1	0	1	1	0	0
NCT00056407	3		2003	2	1	1	1	1	1	1	1	0	1	1	0	0
NCT00056472	3	1	2003	0	1	1	1	1	1	1	1	0	1	1	0	0
NCT00056498	4	1	2003	1	1	1	1	1	1	1	1	0	1	0	0	0
NCT00056550	3	0	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00056563	3	1	2003	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00056862	4	0	2003	0	1	1	1	1	0	1	1	0	0	1	0	0
NCT00057330	3		2003	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00057681	3	1	2003	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00057811	2	1	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00057837	2	0	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00057876	3	1	2003	0	1	1	1	1	0	0	1	0	1	0	0	0
NCT00057941	2		2003	0	1	1	0	0	1	1	1	0	0	0	0	0
NCT00057954	2	0	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00058019	2	1	2003	0	1	1	1	1	0	0	1	0	1	0	0	0
NCT00059215	2	0	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00059475	2	0	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00059839	3	1	2003	0	1	1	1	1	1	0	1	0	0	0	0	0
NCT00060008		0	2003	0	1	1	1	1	0	0	1	0	1	0	0	0
NCT00060346	2	0	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00060528	1	0	2003	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00060840	2	1	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00060944	2	0	2003	2	1	1	1	1	0	0	1	0	1	1	0	0
NCT00061048	2	0	2003	0	1	1	1	0	1	1	1	0	0	0	0	0
NCT00061373	2	1	2003	0	1	1	1	1	0	1	1	0	0	0	0	0
NCT00061633	2	1	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00061893	2	1	2003	0	1	1	1	1	1	1	1	0	0	0	0	0
NCT00061945	1		2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00062010	2	0	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00062439	2	1	2003	0	1	1	1	1	0	0	1	0	0	0	0	0
NCT00062647	2	0	2003	2	1	1	1	1	1	1	1	0	0	0	0	0
NCT00062738	2	1	2003	1	1	1	1	1	1	1	1	0	0	1	0	0
NCT00062751	2	0	2003	2	1	1	1	1	0	0	1	0	0	0	0	0
NCT00062764	2	0	2003	0	1	1	1	1	0	1	1	0	0	1	0	0
This is a portion of the data; to view all the data, please download the file.

Dataset 1.Dataset of funding source, primary outcome changes and statistical significance of clinical trials registered on ClinicalTrials.org.

All clinical studies classified as ‘interventional studies’ registered with ClinicalTrials.gov as of 20^th November 2014 are shown. Probable funding source was derived using the algorithm previously described³. A statistically significant primary outcome was defined as a trial having a P value less than 0.05. 1=yes; 0=no; blank=no info in all columns except “studyphase” and “sponsortype (0=public; 1=industry; 2=mixed). pom: primary outcome measure; sig: significance.

As of 20 November 2014, 13,238 completed interventional trials were registered with ClinicalTrials.gov that also had study results posted on the website. The trials were registered between 1999 and 2014 and 2555 (19.3%) had one or more statistically significant primary outcomes. There were 3934 (29.7%) trials classed as non-industry funded, 1569 (11.9%) as mixed and 7735 (58.4%) as industry funded. 12632 (95.4%) trials had a change in the primary outcome reported at initial registration; 12243 (92.5%) of these occurred after the trial completion date.

Statistical analysis showed that registration year, funding source and primary outcome change after trial completion were associated with the reporting a statistically significant primary outcome (Table 1). Mixed funding and increased year of registration (i.e. more recent calendar time) were associated with a decreased odds of reporting a statistically significant primary outcome. A primary outcome change and industry funding were associated with an increased odds of reporting a statistically significant primary outcome. We identified 123 trials that had inferiority in the title. Removing these studies from the analysis did not materially change the results.

When including study phase in the analysis (10633 studies with study phase data available), mixed funding and registration year became non-significant.

Table 1. Association of funding status and primary outcome change after trial completion with reporting a statistically significant primary outcome.

	Odds Ratio (95% confidence interval)	P value
Public funding	1
Mixed funding	0.79 (0.67–0.94)	0.008
Industry funding	1.39 (1.25–1.54)	<0.001

No primary outcome change	1
Primary outcome change after completion date	1.53 (1.12–2.10)	0.008

Registration year (per additional year)	0.97 (0.95–0.99)	0.006

Conclusions

We found that the reporting of statistically significant outcomes on ClinicalTrials.gov was more likely for trials with primary outcomes that had been changed and also those funded by industry. Previous studies have documented these associations^5,6, and we confirm these using ClinicalTrials.gov data. There are limitations to our analyses- we have not investigated in any detail the nature of the primary outcome change and the potential effect this would have on the statistical analysis/outcomes. As discussed previously³, some primary outcome changes that we have identified may be typographical/semantic and may not reflect actual changes to the nature of the outcome. We also did not look specifically to see whether a changed primary outcome was the one with a statistically significant finding, just whether a statistically significant finding was found for any primary outcome for the study. The vast majority of studies with results reported on ClinicalTrials.gov had a primary outcome change. This suggests that these trials are ones where the registrations have more diligent data updating. Nevertheless, this should be seen in equal measure for trials with and without statistically significant primary outcomes. In summary, funding source and primary outcome changes are associated with the reporting of statistically significant primary outcomes on ClinicalTrials.gov.

Data availability

F1000Research: Dataset 1. Dataset of funding source, primary outcome changes and statistical significance of clinical trials registered on ClinicalTrials.org, 10.5256/f1000research.6312.d45056⁷

Author contributions

SVR and BG conceived and designed the study. SVR, JP, LH, APS, MK and DM analysed the data. SVR and BG interpreted the data. SVR drafted the article. All authors revised the article and gave final approval for publication.

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

F1000 recommended

References

1. Pocock SJ: Clinical trials: a practical approach. Wiley; 1983; 286. Reference Source
2. Zarin DA, Tse T, Williams RJ, et al.: The ClinicalTrials.gov results database -- update and key issues. N Engl J Med. 2011; 364(9): 852–60. PubMed Abstract | Publisher Full Text | Free Full Text
3. Ramagopalan S, Skingsley AP, Handunnetthi L, et al.: Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study [v1; ref status: indexed, http://f1000r.es/34l]. F1000Res. 2014; 3: 77. PubMed Abstract | Publisher Full Text | Free Full Text
4. Huić M, Marušić M, Marušić A: Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One. 2011; 6(9): e25258. PubMed Abstract | Publisher Full Text | Free Full Text
5. Bhandari M, Busse JW, Jackowski D, et al.: Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004; 170(4): 477–80. PubMed Abstract | Free Full Text
6. Mathieu S, Boutron I, Moher D, et al.: Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009; 302(9): 977–84. PubMed Abstract | Publisher Full Text
7. Ramagopalan SV, Skingsley AP, Handunnetthi L, et al.: Dataset 1 in: Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study. F1000Research. 2015. Data Source

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 26 Mar 2015

Author details Author details

¹ SVR Consulting, London, UK
² Imperial College Medical School, London, UK
³ Medical Research Council Functional Genomics Unit and Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK
⁴ School of Social and Community Medicine, University of Bristol, Bristol, UK
⁵ Department of Epidemiology, London School of Hygiene and Tropical Medicine, London, UK

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 24 Apr 2015, 4:80

https://doi.org/10.12688/f1000research.6312.2

version 1

Published: 26 Mar 2015, 4:80

https://doi.org/10.12688/f1000research.6312.1

Copyright

© 2015 Ramagopalan SV et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Ramagopalan SV, Skingsley AP, Handunnetthi L et al. Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study [version 2; peer review: 2 approved]. F1000Research 2015, 4:80 (https://doi.org/10.12688/f1000research.6312.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 24 Apr 2015

Revised

Views

26

Reviewer Report 05 May 2015

Deborah Korenstein, Division of General Medicine, Mount Sinai School of Medicine, New York, NY, USA

Approved

https://doi.org/10.5256/f1000research.6895.r8561

The authors have ... Continue reading

CITE

Report a concern

Respond or Comment

Views

22

Reviewer Report 27 Apr 2015

Janet Wale, Cochrane Collaboration Consumer Network, Brunswick, VIC, Australia

Approved

https://doi.org/10.5256/f1000research.6895.r8472

The authors have clarified a ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 26 Mar 2015

Views

34

Reviewer Report 02 Apr 2015

Janet Wale, Cochrane Collaboration Consumer Network, Brunswick, VIC, Australia

Approved with Reservations

https://doi.org/10.5256/f1000research.6769.r8116

This Research Note is an extension of the authors' 2014 article (reference 3). It has a very clearly defined question, whether changes made to the primary outcomes are associated with statistically significant primary outcomes. The present data therefore includes only ... Continue reading

This Research Note is an extension of the authors' 2014 article (reference 3). It has a very clearly defined question, whether changes made to the primary outcomes are associated with statistically significant primary outcomes. The present data therefore includes only completed interventional studies on clinicaltrials.gov.
The second paragraph of the Methods section refers to the 2014 article. This is unhelpful, particularly as it is not clear from the 2014 article how 'changes in primary outcomes' are defined. The final paragraph of the Results section states that registration year, funding source and primary outcome change after trial completion were associated with a significant primary outcome - yet these are in opposite directions; and registration year is complex (looking at the data and 2014 article). That is brevity has taken over from clarity.

Some of the limitations are included in the Conclusions: what exactly the changes were ('semantics' versus actual change; whether the changed outcome was the statistically significant outcome reported). The authors have not gone on to analyse their results by phase of trial; if the trials are randomised controlled trials, or otherwise. Another important question is how many of the completed trials have reported their results within a set timeframe (one year/two years), that is what about the trials that have not reported their results?
Has the number of industry funded trials increased over time compared with mixed and public funded trials? In plain language, what is the extent of the problem?

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Views

33

Reviewer Report 30 Mar 2015

Deborah Korenstein, Division of General Medicine, Mount Sinai School of Medicine, New York, NY, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.6769.r8112

Ramagopalan and colleagues have expanded on their previous work to assess the relationship between changes in the primary endpoint on clinicaltrials.gov and both funding source and a “positive” trial result. The authors found that changes to the primary listed endpoint ... Continue reading

Ramagopalan and colleagues have expanded on their previous work to assess the relationship between changes in the primary endpoint on clinicaltrials.gov and both funding source and a “positive” trial result. The authors found that changes to the primary listed endpoint were associated with both industry funding and with a positive outcome.

Their sample included completed interventional clinical trials listed on clinicaltrials.gov. They defined trials as having a positive result if they had a listed p-value <0.05. This may be problematic since it appears that their sample included non-inferiority trials (though it is not clear how many) and for these trials a non-significant p-value may indicate a “positive” (or at least non-inferior) result. Since there have been growing numbers of non-inferiority trials published in recent years, this may be a substantial issue. The authors may want to consider identifying non-inferiority trials and considering their results differently, or at least reporting the prevalence of non-inferiority trials if possible.

Aside from this methodologic weakness the other methods are rather straightforward and clear. However, the authors found that 95.4% of trials had changed the primary outcome at some point during the registration period. In contrast, in their previous work the same authors found that 32% of trials registered with clinicaltrials.gov had changed the primary endpoint. The reason for this dramatic difference is not clear, and the authors do acknowledge that the vast majority of studies changed their primary endpoint and that many of the changes may have been trivial. Further, in spite of this surprising finding the authors still found significant associations. However, the near-total prevalence of changes to the primary endpoint certainly suggests that changing a primary endpoint in the registry is highly routine and likely does not reflect fundamental change to the study. This weakens the relevance of the findings.

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 26 Mar 2015

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 24 Apr 15	read	read
Version 1 26 Mar 15	read	read

Deborah Korenstein, Mount Sinai School of Medicine, New York, NY, USA
Janet Wale, Cochrane Collaboration Consumer Network, Brunswick, VIC, Australia

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

Back to all reports

Reviewer Report

26 Views

05 May 2015 | for Version 2

Deborah Korenstein, Division of General Medicine, Mount Sinai School of Medicine, New York, NY, USA

26 Views Cite this report Responses(0)

Approved

The authors have adequately addressed my concerns.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

22 Views

27 Apr 2015 | for Version 2

Janet Wale, Cochrane Collaboration Consumer Network, Brunswick, VIC, Australia

22 Views Cite this report Responses(0)

Approved

The authors have clarified a number of the issues raised.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

34 Views

02 Apr 2015 | for Version 1

Janet Wale, Cochrane Collaboration Consumer Network, Brunswick, VIC, Australia

34 Views Cite this report Responses(0)

Approved With Reservations

This Research Note is an extension of the authors' 2014 article (reference 3). It has a very clearly defined question, whether changes made to the primary outcomes are associated with statistically significant primary outcomes. The present data therefore includes only completed interventional studies on clinicaltrials.gov.
The second paragraph of the Methods section refers to the 2014 article. This is unhelpful, particularly as it is not clear from the 2014 article how 'changes in primary outcomes' are defined. The final paragraph of the Results section states that registration year, funding source and primary outcome change after trial completion were associated with a significant primary outcome - yet these are in opposite directions; and registration year is complex (looking at the data and 2014 article). That is brevity has taken over from clarity.

Some of the limitations are included in the Conclusions: what exactly the changes were ('semantics' versus actual change; whether the changed outcome was the statistically significant outcome reported). The authors have not gone on to analyse their results by phase of trial; if the trials are randomised controlled trials, or otherwise. Another important question is how many of the completed trials have reported their results within a set timeframe (one year/two years), that is what about the trials that have not reported their results?
Has the number of industry funded trials increased over time compared with mixed and public funded trials? In plain language, what is the extent of the problem?

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

33 Views

30 Mar 2015 | for Version 1

Deborah Korenstein, Division of General Medicine, Mount Sinai School of Medicine, New York, NY, USA

33 Views Cite this report Responses(0)

Approved With Reservations

Ramagopalan and colleagues have expanded on their previous work to assess the relationship between changes in the primary endpoint on clinicaltrials.gov and both funding source and a “positive” trial result. The authors found that changes to the primary listed endpoint were associated with both industry funding and with a positive outcome.

Their sample included completed interventional clinical trials listed on clinicaltrials.gov. They defined trials as having a positive result if they had a listed p-value <0.05. This may be problematic since it appears that their sample included non-inferiority trials (though it is not clear how many) and for these trials a non-significant p-value may indicate a “positive” (or at least non-inferior) result. Since there have been growing numbers of non-inferiority trials published in recent years, this may be a substantial issue. The authors may want to consider identifying non-inferiority trials and considering their results differently, or at least reporting the prevalence of non-inferiority trials if possible.

Aside from this methodologic weakness the other methods are rather straightforward and clear. However, the authors found that 95.4% of trials had changed the primary outcome at some point during the registration period. In contrast, in their previous work the same authors found that 32% of trials registered with clinicaltrials.gov had changed the primary endpoint. The reason for this dramatic difference is not clear, and the authors do acknowledge that the vast majority of studies changed their primary endpoint and that many of the changes may have been trivial. Further, in spite of this surprising finding the authors still found significant associations. However, the near-total prevalence of changes to the primary endpoint certainly suggests that changing a primary endpoint in the registry is highly routine and likely does not reflect fundamental change to the study. This weakens the relevance of the findings.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

[1] 1. Pocock SJ: Clinical trials: a practical approach. Wiley; 1983; 286. Reference Source

[2] 2. Zarin DA, Tse T, Williams RJ, et al.: The ClinicalTrials.gov results database -- update and key issues. N Engl J Med. 2011; 364(9): 852–60. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Ramagopalan S, Skingsley AP, Handunnetthi L, et al.: Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study [v1; ref status: indexed, http://f1000r.es/34l]. F1000Res. 2014; 3: 77. PubMed Abstract | Publisher Full Text | Free Full Text

[4] 4. Huić M, Marušić M, Marušić A: Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One. 2011; 6(9): e25258. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Bhandari M, Busse JW, Jackowski D, et al.: Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004; 170(4): 477–80. PubMed Abstract | Free Full Text

[6] 6. Mathieu S, Boutron I, Moher D, et al.: Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009; 302(9): 977–84. PubMed Abstract | Publisher Full Text

[7] 7. Ramagopalan SV, Skingsley AP, Handunnetthi L, et al.: Dataset 1 in: Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study. F1000Research. 2015. Data Source

Funding source and primary outcome changes in clinical trials registered on ClinicalTrials.gov are associated with the reporting of a statistically significant primary outcome: a cross-sectional study

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Results

Table 1. Association of funding status and primary outcome change after trial completion with reporting a statistically significant primary outcome.

Conclusions

Data availability

Author contributions

Competing interests

Grant information

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

The problem

How to fix it

Competing Interests Policy

Stay Updated