Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials

Routinely collected data about health in medical records, registries and hospital activity statistics is now routinely collected in an electronic form. The extent to which such sources of data are now being routinely accessed to deliver efficient clinical trials, is unclear. The aim of this study was to ascertain current practice amongst a United Kingdom (UK) cohort of recently funded and ongoing randomised controlled trials (RCTs) in relation to sources and use of routinely collected outcome data. Recently funded and ongoing RCTs were identified for inclusion by searching the National Institute for Health Research journals library. Trials that have a protocol available were assessed for inclusion and those that use or plan to use routinely collected health data (RCHD) for at least one outcome were included. RCHD sources and outcome information were extracted. Of 216 RCTs, 102 (47%) planned to use RCHD. A RCHD source was the sole source of outcome data for at least one outcome in 46 (45%) of those 102 trials. The most frequent sources are Hospital Episode Statistics (HES) and Office for National Statistics (ONS), with the most common outcome data to be extracted being on mortality, hospital admission, and health service resource use. Our study has found that around half of publicly funded trials in a UK cohort (NIHR HTA funded trials that had a protocol available) plan to collect outcome data from routinely collected data sources.


Introduction
Routinely collected data about health in medical records, registries and hospital activity statistics is now routinely collected in an electronic form. Progress in achieving connectivity, data linkage and security now offers the possibility of better use of this data for research purposes. For example, recent evidence shows the utility of long-term follow-up of trial patients by linkage to routinely collected health data (RCHD) sources (Fitzpatrick et al., 2018). Innovative data-enabled study designs can answer pressing knowledge gaps in research evidence. However, the extent to which such sources of data are now being routinely employed in research to deliver efficient clinical trials, potentially at a wide scale, is unclear.
The aim of this study was to ascertain current practice amongst a United Kingdom (UK) cohort of recently funded and ongoing randomised controlled trials (RCTs) in relation to sources and use of routinely collected outcome data. We chose NIHR HTA because they are a major source of funding for investigatorled publicly funded clinical trials within the UK in an NHS setting. We define RCHD to be data collected without specific a priori research questions developed prior to using the data for research.

Inclusion criteria
The following inclusion criteria were used: 1. Ongoing RCT of any type including feasibility or pilot work, funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme; 2. availability of a protocol; and 3. use of RCHD for at least one study outcome. If the final published report was shown alongside the protocol this was taken to mean that the RCT was not ongoing but the status had not been updated to 'Published', and the study was excluded.

Search methods
In the absence of a protocol, the study was excluded. For studies with multiple protocol versions, the most recently available version was used.

Data extraction
One person (AM) extracted the information and categorised each RCHD source, with a second person (PW) checking classifications and explanations. The information extracted was as follows: Lead Investigator surname, year started, ISRCTN, project title, study type, use of RCHD for at least one study outcome, availability of a protocol, any details of data quality assessment of RCHD source prior to use, RCHD source name, reasons for wanting outcome data from RCHD source, specific outcomes and outcome type where clear data to be used will come from named RCHD sources. Figure 1 shows the study flow diagram. 279 records were identified through database searching and screened for inclusion. 22 were non-RCTs, 1 was a completed RCT, 30 were RCTs but no protocol was available and 10 were unclear. Of the remaining 216 NIHR HTA trials with a protocol available for further study, 102 (47%) planned to use RCHD for at least one outcome. Table 1 shows the reasons for collecting trial outcome data from routine sources from the 102 eligible trials. The RCHD source was the sole source of outcome data for at least one outcome in 46 (45%) of those 102 trials (categories 3, 4 and 6 in Table 1). In five of these 46 protocols there was reference to prior feasibility work confirming aspects of the quality of the data to be sufficient for the main trial. Of the 102 trials, 14 (categories 7a-7d in Table 1) planned to assess the feasibility of using the RCHD sources during the trial, although details of the assessment were often lacking. Raw data for Figure 1 and Table 1 and

Discussion
Our study has found that around half of publicly funded trials in a UK cohort (NIHR HTA funded trials that had protocol available) plan to collect outcome data from RCHD sources. A cohort of 189 RCTs published since 2000, the majority of which were carried out in North America (McCord et al., 2019), found this figure to be higher at 84%, however, they identified their cohort as those mentioning 'EHR' in some way,

Amendments from Version 2
This article has been updated in response to peer reviewer 2, and includes additions to the text in both the Introduction and Discussion sections.
Any further responses from the reviewers can be found at the end of the article Table 1. Reasons for sourcing outcome data from RCHD sources in 102 studies. Multiple categories can apply to a single study.

Categories Total
(1) (1a) 'Supplementing data collection for withdrawn patients (consent asked for at time of withdrawal)' 7 (1b) 'Supplementing data collection for lost-to-follow-up patients' 8 (1c) 'Supplementing data collection for withdrawn patients (consent NOT ASKED FOR at time of withdrawal)' 2 (1e) 'Continued data collection for withdrawn patients (consent asked for at time of withdrawal)' 1 (2) (2) 'Supplementing data collection for unobtainable/missing data' 3 (3) (3a) ' As the sole source of all outcome data' 0 (3b) ' As the sole source of all outcome data except for data related to protocol adherence and adverse event reporting being collected using CRFs' 0 (4) (4) ' As the sole source of some outcome data' 43

Categories Total
(5) (5a) ' As a source of some outcome data, alongside other sources for the same outcome data (e.g. CRF)' 51 (5b) ' As a source of some outcome data, but collected by CRF if unable to access data' 3 (6) (6a) 'Registry trial*: As the sole source of outcome data with purpose-built Module to collect remaining outcome data' 1 (6b) 'Registry trial*: All outcome data collected through multiple RCHD sources except for questionnaire data' 1 (6c) 'Registry trial*: All outcome data collected through multiple RCHD sources except for some baseline data, questionnaire data and other patient-reported data' 1 (7) (7a) 'RCHD compared to trial collected data as part of feasibility assessment criteria' 11 (7b) 'RCHD compared to trial collected data as a main trial secondary outcome' 1 (7c) 'RCHD compared to trial collected data and then collect long-term follow-up data as part of trial' 1 (7d) 'RCHD compared to trial collected data and then collect long-term follow-up data after trial has been completed' 1 (10) 'Patients asked to provide written consent for continuation in the study once have regained capacity. Those who prefer not to be actively involved in the study follow-up, then asked to provide consent to using their routinely collected NHS data' 1 Total 155 * A registry trial is a RCT conducted using clinical observational registries as the main source of outcome data collection trial publication (Kwakkenbos et al., 2018). An extension to the SPIRIT guidelines for trials using RCHD is soon to be initiated, and will determine whether this information should be included in the trial protocol. As a minimum, it is recommended that trialists provide evidence in any funding application about the quality of the data from the RCHD source.

Open Peer Review
A sentence could be included to explain your rationale for selecting trials from the Health Technology Assessment Programme at NIHR as against some other source of publicly funded trials. For example your study excludes UKMRC funded trials. The implications of excluding these mechanism-of-action and therapeutic intervention trials could be explained in the discussion.
Your flow diagram indicates that you identified articles through database searching (NIHR library) (n=279) and other sources (n=0). If NIHR HTA trials were of interest, what "other sources" were searched, and why? Table 2 includes an "other" category for sources of outcome data. It only includes 5 studies, so perhaps you could list these "other" sources, in an appendix if necessary.
You mention that McCord et al. found that 8% of trials published since 2000 used outcome data from RCHD sources. The McCord paper states, "In most (84%) of the trials in our sample, outcomes were measured with the use of EHRs". It's not entirely clear where the 8% is from? ○ I think it's important to note that they searched up to 2017; thus, since your study included trials up to almost the end of 2019, the higher percentage you found may represent a shift over time, especially as the appreciation for pragmatic trials and routinely collected data for these trials has increased in recent years.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound?

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes The text already references this re We have removed the text regarding this from the abstract and changed the text within the Discussion to say: "A cohort of 189 RCTs published since 2000, the majority of which were carried out in North America (McCord et al., 2019), found this figure to be higher at 84%, however, they identified their cohort as those mentioning 'EHR' in some way, i.e. a selected cohort, whereas ours is an unselected cohort so they are not comparable due to the selectivity of the samples." EHR.

Major Comments
I have only one major comment and it is the reason for both the 'partly' options below. The sample was selected to be using routinely collected data for at least one study outcome. Therefore I think the main result should contain this information. I consider that "in a UK cohort" is not enough of a description of the cohort. The fact that the sample was selected based on using routine data for an outcome is crucial in the interpretation.
The main result is that of 102 protocols using routinely collected data for an outcome, 46 were using routinely collected data as their sole source for at least one outcome. 46/102=45%. Around a half of NIHR HTA funded trials that had an uploaded protocol and used routinely collected health data for at least one study outcome, used solely routinely collected data for at least one trial outcome. I think this is an important result.

Minor comments
Abstract -last part of the last sentence has a word missing "The majority of which were carried out in North America". 1.
If you have space in the text, it would be useful to add the information that 30 were omitted due to not having a protocol. 2.
The flow chart shows you selected the papers by selecting RCT, those that had a protocol and then those using routinely collected data for at least one outcome. I would be tempted to list the inclusion criteria in the paper in the same order.

3.
The second inclusion criteria is "use of routinely collected health data". Elsewhere you use the term EHR. I would be tempted to be consistent.

4.
Table 1: category 10 description appears incomplete. 5. Table 1: could you add a footnote of the definition of a registry trial? 6.

If applicable, is the statistical analysis and its interpretation appropriate? Partly
Are all the source data underlying the results available to ensure full reproducibility?
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com