Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials

Andrew J. McKay; Ashley P. Jones; Carrol L. Gamble; Andrew J. Farmer; Paula R. Williamson

doi:10.12688/f1000research.23316.1

Home Browse Use of routinely collected data in a UK cohort of publicly funded...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Brief Report

Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials

[version 1; peer review: 1 approved with reservations]

Andrew J. McKay¹, Ashley P. Jones¹, Carrol L. Gamble^1,2, Andrew J. Farmer³, Paula R. Williamson²

Andrew J. McKay¹, Ashley P. Jones¹, [...] Carrol L. Gamble^1,2, Andrew J. Farmer³, Paula R. Williamson²

PUBLISHED 04 May 2020

Author details Author details

¹ Liverpool Clinical Trials Centre, University of Liverpool, a member of Liverpool Health Partners, Liverpool, UK
² MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Liverpool, UK
³ Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Andrew J. McKay
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Resources, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Ashley P. Jones
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Carrol L. Gamble
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Andrew J. Farmer
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Paula R. Williamson
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Resources, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

Abstract

Routinely collected data about health in medical records, registries and hospital activity statistics is now routinely collected in an electronic form. The extent to which such sources of data are now being routinely accessed to deliver efficient clinical trials, is unclear. The aim of this study was to ascertain current practice amongst a United Kingdom (UK) cohort of recently funded and ongoing randomised controlled trials (RCTs) in relation to sources and use of routinely collected outcome data.
Recently funded and ongoing RCTs were identified for inclusion by searching the National Institute for Health Research journals library. Trials that have a protocol available were assessed for inclusion and those that use or plan to use routinely collected health data for at least one outcome were included. Routinely collected data sources and outcome information were extracted.
A total of 279 studies were identified with 102 eligible for data extraction. An Electronic Health Record (EHR) was the sole source of outcome data for at least one outcome in 46 trials. The most frequent sources are Hospital Episode Statistics (HES) and Office for National Statistics (ONS), with the most common outcome data to be extracted being on mortality, hospital admission, and health service resource use.
Our study has found that around half of publicly funded trials in a UK cohort plan to collect outcome data from routinely collected data sources. This is much higher than the figure of 8% found in a cohort of 189 RCTs published since 2000, the majority of were carried out in North America (McCord et al., 2019).

Keywords

Electronic Health Records, Data linkage, EHR, NIHR HTA, Randomised Clinical Trial, Randomised Controlled Trial, RCT, Registry, Routinely collected data

Corresponding author: Paula R. Williamson

Competing interests: AJF is Chair of an NIHR HTA Funding Committee. CLG is Director of the Liverpool Clinical Trials Centre which receives NIHR Clinical Trials Unit support funding.

Grant information: AJF is an NIHR Senior Investigator and receives support from NIHR Oxford Biomedical Research Centre. PRW is an NIHR Senior Investigator and lead for the MRC/NIHR Trials Methodology Research Partnership (Grant reference: MR/S014357/1).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 McKay AJ et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: McKay AJ, Jones AP, Gamble CL et al. Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials [version 1; peer review: 1 approved with reservations]. F1000Research 2020, 9:323 (https://doi.org/10.12688/f1000research.23316.1) First published: 04 May 2020, 9:323 (https://doi.org/10.12688/f1000research.23316.1) Latest published: 12 Mar 2021, 9:323 (https://doi.org/10.12688/f1000research.23316.3)

Introduction

Routinely collected data about health in medical records, registries and hospital activity statistics is now routinely collected in an electronic form. Progress in achieving connectivity, data linkage and security now offers the possibility of better use of this data for research purposes. For example, recent evidence shows the utility of long-term follow-up of trial patients through the electronic health record (EHR) (Fitzpatrick et al., 2018). Innovative data-enabled study designs can answer pressing knowledge gaps in research evidence. However, the extent to which such sources of data are now being routinely employed in research to deliver efficient clinical trials, potentially at a wide scale, is unclear.

The aim of this study was to ascertain current practice amongst a United Kingdom (UK) cohort of recently funded and ongoing randomised controlled trials (RCTs) in relation to sources and use of routinely collected outcome data. We define routinely collected health data to be data collected without specific a priori research questions developed prior to using the data for research.

Methods

Inclusion criteria

The following inclusion criteria were used:

1. Ongoing RCT of any type including feasibility or pilot work, funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme;
2. use of routinely collected health data for at least one study outcome; and
3. availability of a protocol.

Search methods

A search of the NIHR Journals Library was undertaken to find protocols registered as of 25/10/2019. The search fields and terms used to select were:

1. Search term: ‘Random’
2. Research type: ‘Primary research’
3. Programme: ‘HTA’
4. Status: ‘Research in progress’

If the final published report was shown alongside the protocol this was taken to mean that the RCT was not ongoing but the status had not been updated to ‘Published’, and the study was excluded.

In the absence of a protocol, the study was excluded. For studies with multiple protocol versions, the most recently available version was used.

Data extraction

One person (AM) extracted the information and categorised each EHR, with a second person (PW) checking classifications and explanations. The information extracted was as follows: Lead Investigator surname, year started, ISRCTN, project title, study type, use of routinely collected health data for at least one study outcome, availability of a protocol, any details of EHR data quality assessment prior to use, EHR name, reasons for sourcing outcome data from EHR, specific outcomes and outcome type where clear data to be used will come from named EHRs.

Results

Figure 1 shows the study flow diagram. There were 102 eligible trials available for further study.

Figure 1. PRISMA flow diagram.

Table 1 shows the reasons for collecting trial outcome data from routine sources. The EHR was the sole source of outcome data for at least one outcome in 46 trials (categories 3, 4 and 6 in Table 1). In five of these 46 protocols there was reference to prior feasibility work confirming aspects of the quality of the data to be sufficient for the main trial. Of the 102 trials, 14 (categories 7a-7d in Table 1) planned to assess the feasibility of using the EHR data sources during the trial, although details of the assessment were often lacking. Raw data for Figure 1 and Table 1 and Table 2 are available (see Underlying data, McKay et al. (2020)).

Table 1. Reasons for sourcing outcome data from EHRs in 102 studies.

Multiple categories can apply to a single study.

	Categories	Total
(1)	(1a) 'Supplementing data collection for withdrawn patients (consent asked for at time of withdrawal)'	7
	(1b) 'Supplementing data collection for lost-to-follow-up patients'	8
	(1c) 'Supplementing data collection for withdrawn patients (consent NOT ASKED FOR at time of withdrawal)'	2
	(1e) 'Continued data collection for withdrawn patients (consent asked for at time of withdrawal)'	1
(2)	(2) 'Supplementing data collection for unobtainable/missing data'	3
(3)	(3a) 'As the sole source of all outcome data'	0
	(3b) 'As the sole source of all outcome data except for data related to protocol adherence and adverse event reporting being collected using CRFs'	0
(4)	(4) 'As the sole source of some outcome data'	43
(5)	(5a) 'As a source of some outcome data, alongside other sources for the same outcome data (e.g. CRF)'	51
	(5b) 'As a source of some outcome data, but collected by CRF if unable to access data'	3
(6)	(6a) 'Registry trial: As the sole source of outcome data with purpose-built Module to collect remaining outcome data'	1
	(6b) 'Registry trial: All outcome data collected through multiple EHRs except for questionnaire data'	1
	(6c) 'Registry trial: All outcome data collected through multiple EHRs except for some baseline data, questionnaire data and other patient-reported data'	1
(7)	(7a) 'EHR data compared to trial collected data as part of feasibility assessment criteria'	11
	(7b) 'EHR data compared to trial collected data as a main trial secondary outcome'	1
	(7c) 'EHR data compared to trial collected data and then collect long-term follow-up data as part of trial'	1
	(7d) 'EHR data compared to trial collected data and then collect long-term follow-up data after trial has been completed'	1
	(7e) 'Representativeness of randomised patients compared with all eligible patients using EHR data as part of feasibility assessment criteria'	1
(8)	(8a) 'Participants flagged with NHS Digital/other: Check health status of patient prior to contacting in case patient has died'	2
	(8b) 'Participants flagged with NHS Digital/other: Check health status/notification of any deaths, causes'	12
(9)	(9) 'Set up mechanisms for long-term follow-up'	4
(10)	(10) 'Patients asked to provide written consent for continuation in the study once have regained capacity. Those who prefer not to be actively involved in the study follow-up, then asked to provide co	1
	Total	155

Table 2. Categories of EHR sources of outcome data in 46 studies where this was the sole source for at least one outcome.

Source	Number (%)
(i) Primary care data (all regional equivalents)	8 (17%)
(ii) HES (and/or regional equivalents)	27 (59%)
(iii) ONS (and/or regional equivalents)	27 (59%)
(iv) Data collected specifically for patient group or healthcare intervention (to include patient registries, ICNARC, ambulance, etc)	26 (57%)
(v) Other	5 (11%)

Table 2 shows the sources of outcome data to be used in these 46 studies. The most frequent sources are Hospital Episode Statistics (HES) and Office for National Statistics (ONS), with the most common outcome data to be extracted being on mortality, hospital admission, and health service resource use (see Underlying data, Data Set 5; McKay et al. (2020)). The full list of data sources is given in Extended data, Supplementary Table 1 (McKay et al., 2020).

Discussion

Our study has found that around half of publicly funded trials in a UK cohort plan to collect outcome data from routinely collected data sources. This is much higher than the figure of 8% found in a cohort of 189 RCTs published since 2000, the majority of which were carried out in North America (McCord et al., 2019).

Very few trial teams described any assessments of data quality from EHRs in the protocol. Work is ongoing that should determine whether such information should be reported in the trial publication (Kwakkenbos et al., 2018). An extension to the SPIRIT guidelines for EHR-supported trials is soon to be initiated, and will determine whether this information should be included in the trial protocol. As a minimum, it is recommended that trialists provide evidence in any funding application about the quality of the data from the EHR.

Data availability

Underlying data

Figshare: Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. https://doi.org/10.6084/m9.figshare.12185193 (McKay et al., 2020).

This project contains the following underlying data:

Data_Set_1_Details_and_Figure_1_v1.0.csv. (Study identifiers and raw data used for Figure 1.)
Data_Set_2_Table_1_v1.0.csv. (Raw data used for Table 1.)
Data_set_3_Supp_Table_1_v1.0.csv. (Raw data used for Supplementary Table 1.)
Data_set_4_Table_2_v1.0.csv. (Raw data used for Table 2.)
Data_set_5_Outcomes_using_EHR_data_v1.0.csv. (Raw data showing details of outcomes using EHR data.)

Extended data

Figshare: Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. https://doi.org/10.6084/m9.figshare.12185193 (McKay et al., 2020).

This project contains the following extended data:

Supplementary Table 1 - EHR sources of outcome data v1.0.pdf. (Supplementary Table 1.)

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Faculty Opinions recommended

References

Fitzpatrick T, Perrier L, Shakik S, et al.: Assessment of Long-term Follow-up of Randomized Trial Participants by Linkage to Routinely Collected Data: A Scoping Review and Analysis. JAMA Netw Open. 2018; 1(8): e186019. PubMed Abstract | Publisher Full Text | Free Full Text
Kwakkenbos L, Juszczak E, Hemkens LG, et al.: Protocol for the development of a CONSORT extension for RCTs using cohorts and routinely collected health data. Res Integr Peer Rev. 2018; 3: 9. PubMed Abstract | Publisher Full Text | Free Full Text
Mc Cord KA, Ewald H, Ladanie A, et al.: Current use and costs of electronic health records for clinical trial research: a descriptive study. CMAJ open. 2019; 7(1): E23–E32. PubMed Abstract | Publisher Full Text | Free Full Text
McKay A, Jones A, Gamble C, et al.: Data sets used and Supplementary Table 1. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.12185193.v1

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 04 May 2020

Author details Author details

Ashley P. Jones
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Carrol L. Gamble
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Andrew J. Farmer
Roles: Conceptualization, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Review & Editing

Competing interests

AJF is Chair of an NIHR HTA Funding Committee. CLG is Director of the Liverpool Clinical Trials Centre which receives NIHR Clinical Trials Unit support funding.

Grant information

AJF is an NIHR Senior Investigator and receives support from NIHR Oxford Biomedical Research Centre. PRW is an NIHR Senior Investigator and lead for the MRC/NIHR Trials Methodology Research Partnership (Grant reference: MR/S014357/1).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (3)

version 3

Revised

Published: 12 Mar 2021, 9:323

https://doi.org/10.12688/f1000research.23316.3

version 2

Revised

Published: 01 Jun 2020, 9:323

https://doi.org/10.12688/f1000research.23316.2

version 1

Published: 04 May 2020, 9:323

https://doi.org/10.12688/f1000research.23316.1

© 2020 McKay AJ et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

McKay AJ, Jones AP, Gamble CL et al. Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials [version 1; peer review: 1 approved with reservations]. F1000Research 2020, 9:323 (https://doi.org/10.12688/f1000research.23316.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 04 May 2020

Views

Reviewer Report 13 May 2020

Sharon Love, MRC Clinical Trials Unit, University College London, London, UK

Approved with Reservations

https://doi.org/10.5256/f1000research.25738.r63052

This is a review of the protocols of RCTs, currently in progress, funded by NIHR, UK. RCTs were selected from NIHR HTA funding stream list if they claimed to be using routinely collected data for at least one study outcome. The authors found 102 trial protocols matching this criteria and from data extraction that 46 of these were using routinely collected data solely for at least one outcome. The research also found that a handful referenced previous feasibility work confirming the quality of the EHR and also gives a useful table categorising for the 102 trials how they used EHR.

Major Comments
I have only one major comment and it is the reason for both the ‘partly’ options below. The sample was selected to be using routinely collected data for at least one study outcome. Therefore I think the main result should contain this information. I consider that “in a UK cohort” is not enough of a description of the cohort. The fact that the sample was selected based on using routine data for an outcome is crucial in the interpretation.

The main result is that of 102 protocols using routinely collected data for an outcome, 46 were using routinely collected data as their sole source for at least one outcome. 46/102=45%. Around a half of NIHR HTA funded trials that had an uploaded protocol and used routinely collected health data for at least one study outcome, used solely routinely collected data for at least one trial outcome.
I think this is an important result.

Minor comments

Abstract – last part of the last sentence has a word missing “The majority of which were carried out in North America”.
If you have space in the text, it would be useful to add the information that 30 were omitted due to not having a protocol.
The flow chart shows you selected the papers by selecting RCT, those that had a protocol and then those using routinely collected data for at least one outcome. I would be tempted to list the inclusion criteria in the paper in the same order.
The second inclusion criteria is “use of routinely collected health data”. Elsewhere you use the term EHR. I would be tempted to be consistent.
Table 1: category 10 description appears incomplete.
Table 1: could you add a footnote of the definition of a registry trial?

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: With others, I have conducted a review of the use of routinely collected health data by using release lists from registries which has been accepted for publication but is not yet published.

Reviewer Expertise: Trial conduct, particularly monitoring and the use of routinely collected health data.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 01 Jun 2020

Andrew McKay, Liverpool Clinical Trials Centre, University of Liverpool, a member of Liverpool Health Partners, Liverpool, UK

01 Jun 2020

Author Response

Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the ... Continue reading Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the “UK cohort” is “NIHR HTA trials with a protocol” ongoing at the stated data extraction date.

Major comments part 2: We have now made this clearer.

Minor comments: Thank you for your comments. We have addressed them all within the article update to version 2. In relation to one specific comment, we have chosen to use ‘routinely collected health data (RCHD)’ throughout rather than ‘Electronic Health Record (EHR)’ for consistency.
Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the “UK cohort” is “NIHR HTA trials with a protocol” ongoing at the stated data extraction date.

Major comments part 2: We have now made this clearer.

Minor comments: Thank you for your comments. We have addressed them all within the article update to version 2. In relation to one specific comment, we have chosen to use ‘routinely collected health data (RCHD)’ throughout rather than ‘Electronic Health Record (EHR)’ for consistency.
Competing Interests: None. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 01 Jun 2020

Andrew McKay, Liverpool Clinical Trials Centre, University of Liverpool, a member of Liverpool Health Partners, Liverpool, UK

01 Jun 2020

Author Response

Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the ... Continue reading Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the “UK cohort” is “NIHR HTA trials with a protocol” ongoing at the stated data extraction date.

Major comments part 2: We have now made this clearer.

Minor comments: Thank you for your comments. We have addressed them all within the article update to version 2. In relation to one specific comment, we have chosen to use ‘routinely collected health data (RCHD)’ throughout rather than ‘Electronic Health Record (EHR)’ for consistency.
Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the “UK cohort” is “NIHR HTA trials with a protocol” ongoing at the stated data extraction date.

Major comments part 2: We have now made this clearer.

Minor comments: Thank you for your comments. We have addressed them all within the article update to version 2. In relation to one specific comment, we have chosen to use ‘routinely collected health data (RCHD)’ throughout rather than ‘Electronic Health Record (EHR)’ for consistency.
Competing Interests: None. Close
Report a concern

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 04 May 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 3 (revision) 12 Mar 21
Version 2 (revision) 01 Jun 20	read	read
Version 1 04 May 20	read

Sharon Love, University College London, London, UK
Alison H Howie, Western University, London, Canada

Merrick Zwarenstein, Western University, London, Canada

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

22 Views

02 Nov 2020 | for Version 2

Alison H Howie, Western University, London, ON, Canada

Merrick Zwarenstein, Centre for Studies in Family Medicine, Department of Family Medicine, Western University, London, Canada

22 Views Cite this report Responses(1)

Approved

A sentence could be included to explain your rationale for selecting trials from the Health Technology Assessment Programme at NIHR as against some other source of publicly funded trials. For example your study excludes UKMRC funded trials. The implications of excluding these mechanism-of-action and therapeutic intervention trials could be explained in the discussion.

Your flow diagram indicates that you identified articles through database searching (NIHR library) (n=279) and other sources (n=0). If NIHR HTA trials were of interest, what “other sources” were searched, and why?

Table 2 includes an “other” category for sources of outcome data. It only includes 5 studies, so perhaps you could list these “other” sources, in an appendix if necessary.

You mention that McCord et al. found that 8% of trials published since 2000 used outcome data from RCHD sources.

The McCord paper states, “In most (84%) of the trials in our sample, outcomes were measured with the use of EHRs”. It’s not entirely clear where the 8% is from?

I think it’s important to note that they searched up to 2017; thus, since your study included trials up to almost the end of 2019, the higher percentage you found may represent a shift over time, especially as the appreciation for pragmatic trials and routinely collected data for these trials has increased in recent years.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Randomized trials, esp pragmatic trials; Epidemiology.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

12 Mar 2021

Andrew McKay, Liverpool Clinical Trials Centre, University of Liverpool, a member of Liverpool Health Partners, Liverpool, UK

Thanks for your important comments. We have made some changes within the article update to version 3.

Re Comment 1:
We have added a sentence added to introduction:
“We chose NIHR HTA because they are a major source of funding for investigator-led publicly funded clinical trials within the UK in an NHS setting.”

Re Comment 2:
No other sources were searched. The ‘other sources’ box is part of the ‘PRISMA 2009 Flow Diagram’ template.

Re Comment 3:
These “other” sources are already listed in Supplementary Table 1 as follows:

(1) - Costa (2016a)
“Linkage to routine NHS datasets”

(2) - Morris (2018)
“Central UK NHS bodies for long-term outcomes”

(3) - Ramnarayan (2019)
“Data linkage with routine sources (e.g. NHS Digital or equivalent)”

(4) - Tickle (2017)
“Routinely collected data held by the Information Services Division (ISD) of NHS Scotland, Business Services Authority (BSA, England) and Business Services Organisation (BSO, Northern Ireland)”

(5) - Toff (2013)
“Central NHS databases administered by the Health and Social Care Information Centre and its counterparts in the devolved nations”

The text already references this re Table 2 in the Results section: “…The full list of RCHD sources is given in Extended data, Supplementary Table 1 (McKay et al., 2020).”

Re Comment 4a:
Thank you for spotting this error – the 8% should read 84%. On reflection, we don’t think the McCord 2019 ‘cohort’ is a similar type of cohort to ours so have amended the text.

We have removed the text regarding this from the abstract and changed the text within the Discussion to say:
“A cohort of 189 RCTs published since 2000, the majority of which were carried out in North America (McCord et al., 2019), found this figure to be higher at 84%, however, they identified their cohort as those mentioning ‘EHR’ in some way, i.e. a selected cohort, whereas ours is an unselected cohort so they are not comparable due to the selectivity of the samples.”

Re Comment 4b:
We have checked the data and can confirm that there has been no shift over time re trials that use RCHD for at least one study outcome:
Trials started <=2017: 79/166=(47.6%)
Trials started 2018-2019: 23/50=(46.0%).

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

21 Views

02 Jun 2020 | for Version 2

Sharon Love, MRC Clinical Trials Unit, University College London, London, UK

21 Views Cite this report Responses(0)

Approved

Thank you for making the suggested changes. I have no further comments.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Trial conduct, particularly monitoring and the use of routinely collected health data.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

32 Views

13 May 2020 | for Version 1

Sharon Love, MRC Clinical Trials Unit, University College London, London, UK

32 Views Cite this report Responses(1)

Approved With Reservations

Abstract – last part of the last sentence has a word missing “The majority of which were carried out in North America”.
If you have space in the text, it would be useful to add the information that 30 were omitted due to not having a protocol.
The flow chart shows you selected the papers by selecting RCT, those that had a protocol and then those using routinely collected data for at least one outcome. I would be tempted to list the inclusion criteria in the paper in the same order.
The second inclusion criteria is “use of routinely collected health data”. Elsewhere you use the term EHR. I would be tempted to be consistent.
Table 1: category 10 description appears incomplete.
Table 1: could you add a footnote of the definition of a registry trial?

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

With others, I have conducted a review of the use of routinely collected health data by using release lists from registries which has been accepted for publication but is not yet published.

Reviewer Expertise

Trial conduct, particularly monitoring and the use of routinely collected health data.

Respond to this report

Responses (1)

Author Response

01 Jun 2020

Andrew McKay, Liverpool Clinical Trials Centre, University of Liverpool, a member of Liverpool Health Partners, Liverpool, UK

Major comments: Thanks for your important comments. We have made these clearer within the article update to version 2.

Major comments part 1: We have now made it clear that the “UK cohort” is “NIHR HTA trials with a protocol” ongoing at the stated data extraction date.

Major comments part 2: We have now made this clearer.

Minor comments: Thank you for your comments. We have addressed them all within the article update to version 2. In relation to one specific comment, we have chosen to use ‘routinely collected health data (RCHD)’ throughout rather than ‘Electronic Health Record (EHR)’ for consistency.

View more View less

Competing Interests

None.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] Fitzpatrick T, Perrier L, Shakik S, et al.: Assessment of Long-term Follow-up of Randomized Trial Participants by Linkage to Routinely Collected Data: A Scoping Review and Analysis. JAMA Netw Open. 2018; 1(8): e186019. PubMed Abstract | Publisher Full Text | Free Full Text

[2] Kwakkenbos L, Juszczak E, Hemkens LG, et al.: Protocol for the development of a CONSORT extension for RCTs using cohorts and routinely collected health data. Res Integr Peer Rev. 2018; 3: 9. PubMed Abstract | Publisher Full Text | Free Full Text

[3] Mc Cord KA, Ewald H, Ladanie A, et al.: Current use and costs of electronic health records for clinical trial research: a descriptive study. CMAJ open. 2019; 7(1): E23–E32. PubMed Abstract | Publisher Full Text | Free Full Text

[4] McKay A, Jones A, Gamble C, et al.: Data sets used and Supplementary Table 1. figshare. Dataset. 2020. http://www.doi.org/10.6084/m9.figshare.12185193.v1

Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials

Abstract

Keywords

Introduction

Methods

Inclusion criteria

Search methods

Data extraction

Results

Figure 1. PRISMA flow diagram.

Table 1. Reasons for sourcing outcome data from EHRs in 102 studies.

Table 2. Categories of EHR sources of outcome data in 46 studies where this was the sole source for at least one outcome.

Discussion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated