Why are feasibility studies accessing routinely collected health data? A systematic review

Eligibility

Our systematic review⁵ (PROSPERO CRD42019123088), published in 2020, considered successful applications for routinely collected health data (such as NHS Digital) from publicly-accessible release registers. This identified 160 successful applications for RCHD relating to RCTs between 2013 and 2018. Eligibility for that review included RCHD being used for any of the following: (i) replacing conventionally collected trial data; (ii) cross-checking against existing trial data (including participant-reported data); (iii) cross-checking RCHD from different sources; (iv) prompt the trial team to further investigate a possible outcome measure or event; (v) cost-effectiveness analysis; and (vi) solely methodological purposes. Use of RCHD to evaluate the number of potential patients without using the RCHD for the purposes of recruitment was excluded. This was achieved by initially selecting every entry containing the following terms rand*, trial, RCT, study, placebo, phase. The resultant list was then screened independently by two people to achieve the final list of all RCTs. Any disagreements were resolved by discussion and re-checking.

The publicly available application summary for these 160 trials was hand-searched by two people and considered eligible for this review of feasibility studies if any of the following criteria were met: (i) included a description of a feasibility or pilot RCT in the study design, (ii) aimed to develop a full scale RCT following the conclusion of the feasibility trial, or (iii) included feasibility as a trial outcome. Again, at this stage, any disagreements were resolved by discussion and re-checking.

Data collection and analysis

The underpinning data extraction has been detailed previously.⁵ In summary, information was extracted from the publicly-accessible data release registers and any publicly available sources (trial websites, publications, trial registrations and protocols, last searched April 2020). Two authors independently extracted data onto a piloted data extraction form and any disagreements were resolved by discussion and re-checking. Trial teams were not contacted for information or clarification. Data collection included information about the RCT, the source of RCHD, and the way in which the data were used (e.g. linkage identifiers used, category of data use).

For each included feasibility trial, any previous access of RCHD prior to 2013 was also included. We extracted data about the trial, the RCHD source and the planned use of the RCHD.

For these feasibility trials we also collected the wording used to describe the use of RCHD in publicly available trial protocols, websites, publications and registrations (Last checked April 2020).

The dataset was stored and analysed descriptively in Microsoft Excel (2016).

Patient and public involvement

There was no patient and public involvement in this review.

RCT characteristics

The search and extraction of data for feasibility trials was conducted between November 2019 and April 2020 and the data set is available from the UCL Research Data repository.⁸ The characteristics of the 15/160 (9%) feasibility trials that accessed RCHD are shown in Table 1. Feasibility studies were generally small (median sample size 100, range 41-4061 participants). All 15 (100%) had, by the data freeze, completed recruitment and follow-up. The majority (9/15, 60%) were individually randomised trials; the remainder (6/15, 40%) were cluster randomised. The main disease category was mental health (4/15, 27%) followed by cardiovascular, infection and stroke (each 3/15, 20%). RCT intervention was often compared to standard-of-care (13/15, 87%), with only 2/15 (13%) to be assessed against a placebo-based control. The majority of interventions in these trials were a change to patient management (10/15, 67%). For example, direct transport to specialist centres rather than Accident & Emergency services or providing paramedics with additional training. 12/15 (80%) trials had an administrative, non-clinical primary outcome measure (e.g. can the trial team recruit 60 patients in one year); the other 3/15 (20%) had a clinical primary outcome measure. All trials recruited only in the UK. Most were coordinated through a UK Clinical Research Collaboration (UKCRC) Registered Clinical Trials Unit (9/15, 60%) and were predominantly open label (9/15, 60%). More trials recruited in secondary care (10/15, 67%) than primary care (5/15, 33%).

RCHD access and use

The most commonly accessed data sources were NHS Digital (5/15,33%), SAIL databank (3/15, 20%), electronic Data Research and Innovation Service (eDRIS, formerly called ISD) (2/15, 13%) and Paediatric Intensive Care Audit Network (PICANET) (2/15, 13%) (Table 2), with each trial only accessing one source. Data collected from RCHD application logs indicated that the most common reasons for requesting RCHD were for data collection during the feasibility trial period (7/15, 47%), follow-up after the feasibility trial period (4/15, 27%), and recruitment (2/15, 13%) with 2/15 (13%) not clearly stating the reason (Table 2).

In some instances, the planned use of RCHD in the trial release register was inconsistent with information gathered from publicly available sources, such as protocols, publications and trial websites. Of the 15 trials, 40% (6/15) had the same RCHD use stated in publicly-available literature and the RCHD release register, for 3/15 (20%) the uses in both sources had some overlap (i.e. the use was similar but not the same) and for 6/15 (40%) the uses for the RCHD were disparate (Table 3). For 2/6 trials with consistent stated use, RCHD use was unclear in both the application and the publicly available literature, with the latter not mentioning the use of RCHD. In those with overlap, the trial documentation was more explicit than the release register. One trial for example had stated in the release register that RCHD would be used for ‘data collection’. In the trial protocol however, RCHD use was described as: “Primary outcome measure: (…) this will be measured using routinely collected NHS data. The Hospital Episode Statistics (HES) system provides a central record of an individual’s use of all emergency departments in England and data will be extracted from this system (using participants’ NHS numbers) to provide information on individual participants’ use of emergency departments at baseline and over the 12 months of follow-up”. This is more specific than claimed in the release register. Out of the six trials for which the stated uses in the RCHD release register and publicly available literature were disparate, one trial had stated in the release register that RCHD would be accessed for adverse events whereas RCHD was accessed specifically for “Hospital attendances, cancer diagnoses and deaths”.

Underlying data

UCL Research Data: Underlying data for ‘Why are feasibility studies accessing routinely collected health data? A systematic review’, https://doi.org/10.5522/04/14743836.v1.⁸

This project contains the following underlying data:

Reporting guidelines

UCL Research data: PRISMA checklist and flow diagram for ‘Why are feasibility studies accessing routinely collected health data? A systematic review’, https://doi.org/10.5522/04/14743836.v1.⁸

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).