The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (study within a trial)

Peter Knapp; Rachael Heys; Lucy Dabner; Karen Sheehan; Helena Smartt; Terrie Walker-Smith; Chris A Rogers; Serban Stoica; Jenny Roche; Thirimon Moe-Byrne; Jacqueline M Martin-Kerry; Rebecca Sheridan; Elizabeth Coleman; Steven Higgins

doi:10.12688/f1000research.110083.1

Home Browse The effects of multimedia information on recruitment and retention...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (study within a trial)

[version 1; peer review: 1 approved with reservations, 1 not approved]

Peter Knapp ¹, Rachael Heys², Lucy Dabner², [...] Karen Sheehan³, Helena Smartt², Terrie Walker-Smith², Chris A Rogers², Serban Stoica³, Jenny Roche¹, Thirimon Moe-Byrne¹, Jacqueline M Martin-Kerry¹, Rebecca Sheridan¹, Elizabeth Coleman¹, Steven Higgins⁴

Peter Knapp ¹, Rachael Heys², [...] Lucy Dabner², Karen Sheehan³, Helena Smartt², Terrie Walker-Smith², Chris A Rogers², Serban Stoica³, Jenny Roche¹, Thirimon Moe-Byrne¹, Jacqueline M Martin-Kerry¹, Rebecca Sheridan¹, Elizabeth Coleman¹, Steven Higgins⁴

PUBLISHED 21 Mar 2022

Author details Author details

¹ Department of Health Sciences, University of York, UK, York, UK
² Bristol Trials Centre, Bristol Medical School,, University of Bristol, Bristol, UK
³ University Hospitals Bristol and Weston NHS Foundation Trust,, Bristol, UK
⁴ School of Education, Durham University, Durham, UK

Peter Knapp
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Rachael Heys
Roles: Data Curation, Project Administration, Writing – Review & Editing

Lucy Dabner
Roles: Data Curation, Project Administration, Writing – Review & Editing

Karen Sheehan
Roles: Data Curation, Project Administration, Writing – Review & Editing

Helena Smartt
Roles: Data Curation, Formal Analysis, Investigation, Supervision, Validation, Writing – Review & Editing

Terrie Walker-Smith
Roles: Data Curation, Project Administration, Writing – Review & Editing

Chris A Rogers
Roles: Funding Acquisition, Investigation, Methodology, Supervision, Validation, Writing – Review & Editing

Serban Stoica
Roles: Funding Acquisition, Investigation, Supervision, Writing – Review & Editing

Jenny Roche
Roles: Data Curation, Formal Analysis, Methodology, Validation, Writing – Review & Editing

Thirimon Moe-Byrne
Roles: Investigation, Project Administration, Validation, Writing – Review & Editing

Jacqueline M Martin-Kerry
Roles: Data Curation, Investigation, Methodology, Project Administration, Writing – Review & Editing

Rebecca Sheridan
Roles: Investigation, Methodology, Writing – Review & Editing

Elizabeth Coleman
Roles: Formal Analysis, Methodology, Validation, Writing – Review & Editing

Steven Higgins
Roles: Funding Acquisition, Investigation, Methodology, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background:
Digital multimedia information (MMI) has potential for use in trial recruitment but there is little formal evaluation. The objectives were to evaluate digital MMI about a trial for its effects on recruitment, retention, participation decisions, and patients’ acceptability, compared with printed information (PIS) alone and when combined with PIS.
Methods:
SWAT (study within a trial) using random parallel-group individual allocation within the Thermic-3 trial evaluating warm versus cold cardioplegia solution during cardiac surgery.
Set in one UK hospital, participants were 147 children (0-16 years) awaiting surgery for congenital heart defects; 38% were female.
Participants and their parents/guardian received trial information via multimedia (including text, animated videos and talking-head videos) for viewing at home (MMI group; n=49), or PIS (PIS group; n=47), or both (PIS&MMI group; n=51).
Primary outcome was recruitment rate to the Thermic-3 trial comparing PIS-alone and MMI-alone. Secondary outcomes were recruitment rate comparing PIS-alone and combined PIS&MMI; Decision-Making Questionnaire; 3 ‘free-text’ questions (deriving subjective evaluations); trial retention.
Results:
MMI produced a 14.2% absolute increase in recruitment, which was not statistically significant: 32 (65.3%) participants were recruited from the MMI group; 24 (51.1%) from the PIS group (OR 1.80; 95% CI 0.79 to 4.10, p = 0.16); and 22 from the PIS&MMI group. There was no difference in recruitment through combined PIS&MMI (43.1% vs 51.1%; OR 0.73; 95% CI 0.33 to 1.61; p= 0.43). Questionnaires were returned by 17 (12%) participants and analysed descriptively. Trial retention (at 3 months) was high in all groups (72/77; 93.5% overall) and there was no difference due to information format received before participating.
Conclusions:
MMI increased recruitment to the Thermic-3 trial but the difference was not statistically significant, and the SWAT was small.
Trial registration: TRECA ISRCTN73136092 and NI Hub for Trials Methodology Research SWAT Repository (SWAT 97). Thermic-3: ISRCTN13467772.

Keywords

SWAT, trial, recruitment, information, multimedia, cardioplegia, children

Corresponding author: Peter Knapp

Competing interests: No competing interests were disclosed.

Grant information: TRECA was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research Programme (NIHR HS&DR 14/21/21).
Thermic-3 was funded by the British Heart Foundation (CH/1992027/7163 and CH/17/1/32804) and supported by the NIHR Bristol Biomedical Research Centre. Thermic-3 was designed and delivered in collaboration with the Bristol Trials Centre, a UKCRC registered clinical trials unit (CTU), which is in receipt of NIHR CTU support funding.
The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Copyright: © 2022 Knapp P et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Knapp P, Heys R, Dabner L et al. The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (study within a trial) [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2022, 11:340 (https://doi.org/10.12688/f1000research.110083.1) First published: 21 Mar 2022, 11:340 (https://doi.org/10.12688/f1000research.110083.1) Latest published: 21 Mar 2022, 11:340 (https://doi.org/10.12688/f1000research.110083.1)

Introduction

Around half of randomised controlled trials experience significant recruitment shortfall or delays, which can increase costs, delay the production of findings, and potentially result in underpowered, inconclusive trials.¹ When people are approached about trial participation they must receive sufficient information to make a valid consent decision and the information should be both thorough and understandable. However, much printed trial participant information has been criticised as unengaging, hard to navigate, long and technical.²^,³ Despite these concerns, participant information can facilitate recruitment, as confirmed in a recent ‘review of reviews’.⁴

When children or young people are being recruited to trials they should have the opportunity to take part in decisions on participation, depending on their age and maturity. In order to do this, they need to understand what the research entails.⁵ However, research terms and concepts may be more difficult for children to understand, in particular any procedures and risks,⁶ and the implications of taking part.⁷^–⁹

In the UK the age at which young people can consent to research also depends on the type of research and the child’s assessed capacity; for Clinical Trials of an Investigational Medicinal Product (CTIMPs), which are mostly pharmaceutical trials, the minimum age of consent is 18. This detail means that decision making about trial participation in children and young people is more complex than it is for adults.

Recent research has highlighted the importance of direct provision of research information to children and young people, rather than via parents/guardians, with a focus on the ease of understanding and appeal of the information.¹⁰ However, appeal should not be achieved through superficiality or lack of balance.

Appeal and understanding may be improved through the delivery of multimedia information (MMI), including video, animations, audio and infographics, as well as conventional text. The UK Health Research Authority recommends the exploration of non-print media for research consent decisions.¹¹ The increased choice and flexibility offered by MMIs may result in increased engagement, with dual channel theory suggesting that it could increase understanding when compared to single channel presentation, such as text alone.¹²^,¹³ Evidence suggests that multimedia can increase information understanding and recruitment of research participants¹⁴ although this has so far only been tested in adults. Increasingly people access many forms of information digitally, and so there is potential for multimedia to provide mandated health information,¹⁵^,¹⁶ although not everyone prefers digital or online provision. Furthermore, good access to the internet is not universal, potentially compounding income-related health inequalities.¹⁷ Online provision of information also has some challenges and potential barriers. For example, some children and young people with health conditions have concerns about privacy and trustworthiness.¹⁸

The relative effectiveness of multimedia and traditional printed information on recruitment of children and young people to trials, was evaluated in the TRECA (TRials Engagement in Children and Young people) study.¹⁹^,²⁰ This was undertaken through six linked SWATs (study within a trial), comparing the effects of information format on patient recruitment, retention, decision-making and information acceptability.²¹^,²² We report the SWAT embedded within the Thermic-3 trial, which tested the effects of cold or warm cardioplegia solution used during surgical repair of congenital heart defects.²³

Methods

Ethical approval

The TRECA study received approval from the NHS Yorkshire & the Humber – Bradford Leeds Research Ethics Committee (17/YH/0082) and the Health Research Authority (IRAS ID 212761) on 14^th July 2017. It is also registered on the Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT 97) (Martin-Kerry et al., 2017). Thermic-3 received approval from London - Central Research Ethics Committee (18/LO/0205) on 30^th April 2018.

Registration

Trial registration: TRECA ISRCTN73136092 and NI Hub for Trials Methodology Research SWAT Repository (SWAT 97). Thermic-3: ISRCTN13467772.

Study design

The SWAT used a three-arm, parallel-group, individual allocation randomised controlled trial (RCT) design.²⁴

Following simple randomisation, according to allocation, participants received either a printed participant information sheet (PIS), or an MMI, or both (PIS&MMI). The study used the computer generated randomisation service provided by sealed envelope.²⁵

The host trial (Thermic-3) was a NIHR Bristol Biomedical Research Centre-funded, single-centre (Bristol Royal Hospital for Children, UK) RCT seeking to improve the outcomes of children undergoing surgical repair of congenital heart defects, involving cardiopulmonary bypass. Cardioplegia is used to stop and protect the heart during surgery and there is uncertainty about the relative effectiveness of warm or cold cardioplegia.²³ In the Thermic-3 trial children were randomly allocated to warm or cold cardioplegia.

Study participants

All children and young people (aged 0-17 years) identified as potentially eligible for Thermic-3 were eligible for TRECA. There were no additional eligibility criteria.

The TRECA priority was to evaluate information in children old enough to have some understanding of the host trial, assessed as age 6 and above. However, it was not practical for Thermic-3 to set different age entry criteria for TRECA and for the host trial (recruiting children of any age).

Intervention

The PIS was the standard written participant information sheet developed by the Thermic-3 trial team, with feedback from the Generation R Young People’s Advisory Group (YPAG) and then approved by the Trial Sponsor (University Hospitals Bristol and Weston NHS Foundation Trust), the NHS Research Ethics Committee and the UK Health Research Authority. Several versions were developed, including a version for parents (with the information directed to the parent) and three versions for children and young people (directed to the child or young person), designed to be appropriate for defined age groups (7–10, 11–15 and 16–17 years). The length of the PIS ranged from 2 to 7 pages depending on the age group.²⁶

The MMIs for the Thermic-3 trial were developed by the TRECA team at the University of York and a website and video creation company (Morph). Three MMI versions were developed, corresponding to the age groups of the PIS provision; all had the child as subject. Parents could choose to view any version.

The multimedia resources could be viewed on PC, phone or tablet computer. The resource included five short video animations, each lasting 45-60 seconds (one specific to Thermic-3: ‘Summary of the key aspects of the Thermic-3 trial’; and four that were trial-generic: ‘Why do we do trials?’; ‘What are trials?’; ‘Who’s in a trial team’; ‘Assent and consent’), and 34 short ‘talking head’ videos. The number of talking head videos varied by age of intended user: 34 video clips for over 16-year-olds, 30 video clips for those aged 10 to 15, and 22 video clips for those aged under 10. The ‘talking head’ video clips were recorded with four individuals (with consent): a trial investigator, a research nurse, plus a parent and a patient involved in a similar cardiac surgery study. Each clip lasted 10-80 seconds, describing different aspects of the trial and procedures. The MMI content was organised on six main webpages with the following headings: ‘Home page (including summary animation)’; ‘About the trial’; ‘Taking part’; ‘After the trial’; ‘Questions’; ‘Contacts’. (A summary of the MMI can be viewed here²⁷: https://www.york.ac.uk/healthsciences/research/health-policy/research/thermic-summary/). The TRECA MMIs were developed through extensive qualitative research and user testing. Principles of participatory design were used to develop their style and format,²⁸^–³⁰ informed by information design. When required, text on the MMI was revised according to Plain English principles,³¹ to maximise readability and age-appropriateness.

Procedure

Potential participants for the Thermic-3 trial were identified from operating theatre and clinic lists. Eligible patients for Thermic-3 were randomised to one of the three TRECA SWAT groups. Patients and/or parents were given at least 24 hours to make a decision on whether or not to participate in the Thermic-3 trial. Written informed consent/assent for the Thermic-3 trial was obtained from parents or patients. (Parents consented for patients aged up to 15 years; from 16 years, patients could consent to the trial themselves).

According to allocation they received information about Thermic-3 via printed PIS or MMI or both. The printed PIS was provided at the hospital and participants could take it home. Those allocated to receive the MMI were given a laminated card with the URL for the MMI, which they could access at home. All patients and their families approached for participation in Thermic-3, regardless of their decision to take part, were given a printed Decision-Making Questionnaire (DMQ)²⁶ (and Freepost envelope to the TRECA study team) for completion. Demographic information was collected from participants (including age; gender; and Index of Multiple Deprivation national decile, based on the patient’s home address).

Masking

The recruitment staff and participants could not be masked to information allocation due to the nature of the intervention. However, participants were not aware that they were being randomised within the TRECA SWAT, as approved by NHS Research Ethics Committee and UK Health Research Authority, and they were not aware that other participants were being given a different format of information.

Outcome measures

The primary outcome of the SWAT was the proportion of eligible patients who were randomised to participate in Thermic-3, from the total eligible patients. The secondary outcomes were retention in the trial (assessed 3 months after recruitment); quality of participation decision-making, assessed through the 9-item decision-making Likert questionnaire (DMQ); and information evaluation and acceptability assessed through three ‘free text’ questions.

Each item of the DMQ was scored 0-4, deriving a total possible score range of 0-36. A higher DMQ score indicates better quality of decision-making. The DMQ comprised items evaluating aspects of trial participation decision-making indicated as important in the underpinning empirical work,²⁸ including items on: information content; the experience of participation; participation advantages and disadvantages; the process of decision-making; uncertainty in trials; and decisional confidence. The three ‘free text’ questions asked respondents to: (i) suggest any further information they would have wanted; (ii) identify aspects explained well; and, (iii) make any other comments.

Sample size, statistical and ‘free text’ analyses

No sample size was calculated for individual SWATs in TRECA; the overall sample size for TRECA was based on a prospective meta-analysis of the six SWATs (10% relative risk increase from 80% to 88% recruitment rate; 80% power, alpha 0.05; assumed heterogeneity 50%; overall n = 1,794).

All analyses were conducted in STATA v16.³² Participant baseline data have been summarised descriptively overall and by TRECA trial arm (PIS only, MMI only or PIS&MMI). Participant baseline characteristics have also been reported by Thermic-3 recruitment status (consented, not consented).

For the primary outcome analysis recruitment rates were compared between the PIS-only and MMI-only arms using logistic regression following modified intention-to-treat principles, where all patients eligible for TRECA were included in the group they were randomised to and excluding participants who were not eligible to join Thermic-3, to assess the effect of replacing printed information with multimedia. Recruitment status was the dependent variable with TRECA allocation included as an independent variable. The results have been presented as an odds ratio (OR), with associated 95% confidence interval (CI) and p-value. Due to several participants not receiving their allocation, the logistic regression was repeated under a per-protocol approach, where only those eligible patients who received their allocated arm were included (and excluding those not receiving the allocated information). The secondary analysis compared the PIS-only arm and the combined PIS&MMI arm in an analogous manner to the primary analysis, assessing the effect of providing multimedia information in addition to printed information.

The same approach was adopted for the secondary outcome, retention, with Risk Adjustment for Congenital Heart Surgery (RACHs) score also included as an independent variable (because it was a stratification variable from the Thermic-3 trial randomisation). It was intended that Thermic-3 trial allocation would also be included as an independent variable however this data was not available at the time of analysis. For the DMQ secondary outcome the responses to each question (including the amount of missing responses) were summarised descriptively and by TRECA arm. When two adjacent scores for a questionnaire item were given, the lower score was taken. A total score was generated by summing the values (0-4) for all nine questions. Up to three missing values were allowed, with the total score calculated by replacing the missing values with the mean score from the completed responses. The calculated total scores of the DMQ scale were summarised descriptively by TRECA arm.

Patient and Public Involvement (PPI)

The TRECA Patient and Public Involvement Group commented on the design and content of the MMIs.³² They provided guidance on information priorities in the MMIs during the development of the TRECA MMI template, and also provided SWAT-specific feedback on video storyboards, and on the wording of text and animation voiceover content.

Results

A total of 153 potential Thermic-3 participants were randomised into the TRECA SWAT (September 2018- March 2020) to receive a PIS (n = 49) or MMI (n = 51) or both (n = 53). Six (3.9%) of the randomised TRECA participants were subsequently not eligible for randomisation in Thermic-3 (PIS only: n = 2; MMI only: n = 2; PIS&MMI: n = 2).²⁶

Hence, 147 (96.1%) participants were included in the analyses.³³ Participants were between 0 and 16 years old. The median age of the participants was 0.67 years (IQR 0.33, 3.75) and a higher proportion were male (62%). The baseline characteristics of these 147 participants, broken down by TRECA allocation, are given in Table 1.

Table 1. Participant baseline characteristics.

	PIS only (n = 47)	MMI only (n = 49)	PIS & MMI (n = 51)	Overall (n = 147)
Age
n (missing)	47 (0)	49 (0)	51 (0)	147 (0)
Median years (IQR)	0.75 (0.25, 4.33)	0.5 (0.33, 3.33)	0.92 (0.33, 3.75)	0.67 (0.33, 3.75)
Gender, n (%)
Male	30 (64)	28 (57)	33 (65)	91 (62)
Female	17 (36)	21 (43)	18 (35)	56 (38)
Deprivation index for home address decile*
n (missing)	47 (0)	49 (0)	51 (0)	147 (0)
Median (IQR)	4 (1, 8)	5 (2, 6)	2 (1, 6)	4 (1, 7)

* 1 = most deprived. PIS = participant information sheet. MMI = multimedia information.

Table 2 presents the TRECA allocation that was received when it was different to the randomised allocation, and the reasons for the change.

Table 2. Reasons for differing allocations randomised and received.

Allocation randomised	Allocation received	Reasons for change
MMI only	PIS only	- Parents had no means of accessing MMI, so paper copies were given (n = 1) - Wrong information given in error (n = 1) - Not known (n = 1)
MMI only	PIS&MMI	- MMI given initially, but PIS also had to be given when final decision was made about entering Thermic-3 (due to WiFi issues at the hospital) (n = 1) - Not known (n = 3)
PIS&MMI	PIS only	- Time constraints (n = 1) - WiFi issues at the hospital (n = 1) - Not known (n = 2)
Any	None	- Participants too stressed/anxious/overwhelmed (n = 8) - Not interested in research (n = 3) - Parents declined information (n = 2) - Wanted surgeon to decide (n = 1) - Not known (n = 7)

Primary analysis

Recruitment

Of the 147 eligible participants who were randomised in TRECA, 78 (53.1%) went on to consent to the Thermic-3 trial. Broken down by randomised TRECA allocation we have: PIS: 24 (51.1%); MMI 32 (65.3%); PIS&MMI 22 (43.1%). Participant baseline characteristics, separated into those who consented to Thermic-3 and those who did not, are given in Table 3.

Table 3. Baseline characteristics broken down by Thermic-3 recruitment status.

	PIS		MMI		PIS&MMI
	Consented	Not consented	Consented	Not consented	Consented	Not consented
	(n=24)	(n = 23)	(n = 32)	(n = 17)	(n = 22)	(n = 29)
Age
n (missing)	24 (0)	23 (0)	32 (0)	17 (0)	22 (0)	29 (0)
Median years (IQR)	2.0 (0.33, 5.67)	0.33 (1.67, 3.50)	0.54 (0.33, 4.42)	0.50 (0.25, 1.83)	0.79 (0.33, 3.08)	1.5 (0.33, 5.58)
Gender, n (%)
Male	16 (67)	14 (61)	18 (56)	10 (59)	16 (73)	17 (59)
Female	8 (33)	9 (39)	14 (44)	7 (41)	6 (27)	12 (41)
Deprivation index for home address decile*
n (missing)	24 (0)	23 (0)	32 (0)	17 (0)	22 (0)	29 (0)
Median (IQR)	4.5 (1.5, 8.0)	3.0 (1.0, 8.0)	5.0 (3.0, 7.0)	4.0 (1.0, 6.0)	4.0 (1.0, 6.0)	1.0 (1.0, 5.0)

* 1 is most deprived. PIS = participant information sheet. MMI = multimedia information. IQR = inter-quartile range.

Primary analysis (MMI only, PIS only)

The modified intention-to-treat primary analysis gave an OR of 1.80 (95% CI 0.79 to 4.10, p = 0.16), indicating a higher recruitment rate in the MMI arm of the SWAT (absolute rate increase 14.2%; relative rate increase 27.8%), although the difference is not statistically significant. Repeating this analysis using a per-protocol approach the results gave an OR of 2.89 (95% CI 1.03 to 8.11, p = 0.04), indicating that, when analysing only those patients who received their information allocation, there is a higher recruitment rate in the MMI arm, which is statistically significant.

Secondary analyses

Recruitment (PIS&MMI, PIS only)

The modified intention-to-treat secondary analysis yielded an OR of 0.73 (95% CI 0.33 to 1.61, p = 0.43) and the per-protocol analysis had an OR of 0.88 (95% CI 0.36 to 2.10, p = 0.77). Table 4 summarises the results from all the recruitment analyses, both indicating no differences in recruitment rates between the combined MMI & PIS arm and the PIS-only arm.

Table 4. Summary of recruitment analyses.

Analysis	TRECA allocations included	N	OR	95% CI	p-value
Primary	MMI only, PIS only	96 (49 MMI, 47 PIS)	1.80	0.79, 4.10	0.16
Primary PP	MMI only, PIS only	76 (34 MMI, 42 PIS)	2.89	1.03, 8.11	0.04
Secondary	PIS&MMI, PIS only	98 (51 PIS&MMI, 49 MMI)	0.73	0.33, 1.61	0.43
Secondary PP	PIS&MMI, PIS only	81 (39 PIS&MMI, 42 PIS)	0.88	0.36, 2.10	0.77

Retention

Of the 78 participants who consented to the Thermic-3 trial, 77 went on to be randomised (trial recruitment finished due to Covid-19 before one patient – in the PIS arm - could be randomised) and are included in the retention analysis. Of the 77 randomised participants in Thermic-3, 72 (93.5%) completed the 3 months follow-up: PIS: 21/23 (91.3%); MMI: 30/32 (93.8%); PIS&MMI: 21/22 (95.4%).

MMI only, PIS only

The logistic regression gave an OR of 1.62 (95% CI 0.20 to 12.98, p = 0.65), suggesting no difference in retention rate when comparing the MMI and PIS arms.

MMI & PIS, PIS only

The logistic regression gave an OR of 2.05 (95% CI 0.17 to 24.6, p = 0.57), suggesting no difference in retention rate when comparing the combined PIS&MMI arm with the PIS-only arm.

DMQs

A total of 17 (12.2%) questionnaires were received and analysed. A summary of the responses to each of the nine questions is given in Table 5. The overall mean total score was 30.1 (n = 17, SD 5.7). The mean total score was 29.5 in the PIS only arm (n = 8, SD 6.9), 31.8 in the MMI only arm (n = 6, SD 4.4) and 28.3 in the PIS&MMI arm (n = 3, SD 4.9).

Table 5. Questionnaire item responses.

		Hard	OK	Easy	Very easy
1) The information I saw about the THERMIC-3 trial was easy to understand.	PIS, n (%)	1 (13)	5 (63)	2 (25)	0 (0)
	MMI, n (%)	0 (0)	1 (17)	2 (33)	3 (50)
	Both, n (%)	0 (0)	1 (33)	2 (67)	0 (0)

		Not at all	Not really	Not sure	Yes, mostly	Yes, completely	Missing
2) The information helped me understand what it would be like for my son or daughter to take part in the THERMIC-3 study.	PIS, n (%)	0 (0)	1 (13)	0 (0)	4 (50)	3 (38)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	2 (33)	4 (67)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	2 (67)	1 (33)	0 (0)
3) The information helped me understand how my son’s or daughter’s treatment or care might change if s/he took part in the THERMIC-3 study.	PIS, n (%)	0 (0)	1 (13)	1 (13)	1 (13)	5 (63)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	1 (17)	5 (83)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	2 (67)	1 (33)	0 (0)
4) The possible benefits of taking part in the THERMIC-3 trial were made clear in the information.	PIS, n (%)	0 (0)	1 (13)	0 (0)	2 (25)	5 (63)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	3 (50)	3 (50)	0 (0)
	Both, n (%)	0 (0)	0 (0)	1 (33)	1 (33)	1 (33)	0 (0)
5) The possible disadvantages of taking part in the THERMIC-3 trial were made clear in the information.	PIS, n (%)	0 (0)	1 (13)	0 (0)	3 (38)	4 (50)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	3 (50)	3 (50)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	3 (100)	0 (0)	0 (0)
6) The information about the THERMIC-3 trial helped me discuss the trial with the person who asked my son or daughter to take part (usually a doctor, nurse or researcher).	PIS, n (%)	0 (0)	0 (0)	0 (0)	3 (38)	5 (63)	0 (0)
	MMI, n (%)	0 (0)	1 (17)	0 (0)	2 (33)	3 (50)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	2 (67)	1 (33)	0 (0)
7) The information about the THERMIC-3 study helped me discuss taking part with my son or daughter.	PIS, n (%)	0 (0)	0 (0)	0 (0)	2 (25)	4 (50)	2 (25)
	MMI, n (%)	0 (0)	1 (17)	0 (0)	3 (50)	2 (33)	0 (0)
	Both, n (%)	1 (33)	0 (0)	0 (0)	1 (33)	1 (33)	0 (0)
8) I am confident that I have made the right decision about whether or not my son or daughter should take part in the THERMIC-3 study.	PIS, n (%)	0 (0)	1 (13)	0 (0)	1 (13)	6 (75)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	0 (0)	6 (100)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	1 (33)	2 (67)	0 (0)
9) In all, the information about the THERMIC-3 trial helped me make my decision about whether or not my son or daughter should take part.	PIS, n (%)	0 (0)	1 (13)	0 (0)	0 (0)	7 (88)	0 (0)
	MMI, n (%)	0 (0)	0 (0)	0 (0)	1 (17)	5 (83)	0 (0)
	Both, n (%)	0 (0)	0 (0)	0 (0)	1 (33)	2 (67)	0 (0)

DMQ ‘free text’ comments

All participants’ responses are available³⁴; they have been categorised according to the information format actually received.

There were three responses to Question 10 (‘any additional information they would have wanted’), requesting the inclusion of an internet link to the completed study in the adult population, and the standard approach. 11 participants provided comments for Question 11 (‘which aspects were explained well?’): many stating that everything was explained well (n=5). In response to Question 12 (‘do you have any other comments?’) three participants made comments, including one referring to the risk of harm in the trial (from a participant who did consent to the Thermic-3 trial).

Discussion

Almost 55% of eligible patients were recruited to the Thermic-3 trial during the SWAT. The absolute rate of recruitment was 14% higher in the MMI group than the PIS group, but the difference was not statistically significant. The per protocol analysis did show a higher recruitment rate in the MMI, which was statistically significant. There was no difference in recruitment rates between the PIS-only and the combined PIS&MMI groups. DMQs were returned by only a small proportion of those randomised, reducing their generalisability, and there were insufficient data to analyse the DMQ total score or individual item scores. Rates of retention were high throughout the Thermic-3 trial and there was no difference in rates among the three information groups.

The SWAT used random allocation to assess the impact of information format on trial recruitment, retention and decision-making. Recruiting nurses were not masked to allocation, although there was concealment of allocation. Participants were not aware of the information SWAT (and so did not consent to it), as approved by the Research Ethics Committee, so their masking was maintained.

The animations and multimedia information were produced by expert, commercial developers. Furthermore, the content and appearance of the MMI were informed by detailed empirical research and Patient and Public Involvement; consequently, the design and content of the MMI resources were carefully considered and of high quality. The SWAT was small and single-centre, and questionnaires were returned by a minority of participants, preventing the analysis of questionnaire scores. The rate of questionnaire return was lower than in other TRECA SWATs and may be partly attributable to the time gap between initial trial invitation and decision making.

Unfortunately, in the case of 11 participants the provided information format was different to the allocated format. In three cases the MMI was not accessible; while the rate of this problem is low, it remains an important concern about the provision of digital trial information in some hospital sites, particularly in a time-pressured situation.²⁰ The TRECA multimedia resources were developed primarily for children or young people, with parents also accessing a version that had been written for young people. However, in the Thermic-3 trial, the majority of patients were very young children, and so in most cases consent decisions were taken by parents only. However, the higher recruitment rate in the MMI-only arm does suggest that the MMI was suitable for parents.

Multimedia information for trial recruitment remains innovative and rarely used, although recently it is being used more often. However, it has not been subject to much evaluation, particularly in children or young people. In two other reported TRECA studies, one in a hypothetical trial setting³⁵ and the other a SWAT,³⁶ children were more likely to rate multimedia information as ‘easy to understand’ than printed information, and the MMIs produced a small but not statistically significant increase in trial recruitment. The MMI also resulted in greater confidence in decision-making in one of the studies.³⁵ Both studies generated higher rates of positive evaluative comments about the MMI than the printed trial information. Two systematic reviews of trials of multimedia information to inform consent in adults reported that they may increase understanding of consent and the research, and increase information retention.³⁷^,³⁸ There has been more evaluation of multimedia information in healthcare delivery, showing some patient benefits (e.g. Refs. 39,40). However, most of these studies involved adults. In child and adolescent populations video animations alone have had more evaluation in controlled studies, showing benefits on children’s knowledge, skills and satisfaction in a range of healthcare settings (e.g. Refs. 41–44).

This small SWAT within the Thermic-3 trial showed that trial recruitment was not improved through digital provision of multimedia recruitment information, although the per protocol analysis did produce an increase in recruitment rate. Furthermore, the combined provision of digital and print information did not increase recruitment. Subsequent TRECA analysis will examine the overall effects of printed and multimedia information across all six SWATs, and the patterns of participant use of the various pages and videos on the MMIs. However, there remains a need for further evaluation of the preferred design of digital, multimedia information in paediatric trials, its impact on recruitment and retention outcomes and patient and parent acceptability, and on trial recruiters’ communication with patients.

Data availability

Underlying data

figshare: Thermic3_TRECA_Data. https://doi.org/10.6084/m9.figshare.19196144.v1³⁴

This project contains the following files:

- Thermic3_Sharable_Data.xls
- Thermic3_Sharable_Data_NoKey.xls
- Thermic3_Data_Key.xls
- Underlying Data DMQ Free text responses.docx

Extended data

figshare. Thermic3_TRECA_CONSORT_PIL_DMQ. https://doi.org/10.6084/m9.figshare.19298861.v1²⁶

This project contains the following files:

- Extended data DMQ 5-11 years old.docx
- Extended data DMQ 12-18 years old.docx
- Extended data DMQ parents-family.docx
- Thermic-3 ParentGuardian Information Leaflet v3.0.pdf
- Thermic-3 Patient Information Leaflet (16-17 years) v3.0.pdf

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

The intellectual property for the animations and the MMIs is owned jointly by The University of York and Morph. As agreed with the funder (NIHR), the animations can be used free of charge in any publicly- or charity-funded research. Anyone who would like to request to use the animations can do so by contacting peter.knapp@york.ac.uk.

Reporting guidelines

figshare: CONSORT checklist and flow diagram for ‘The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (study within a trial)’. https://doi.org/10.6084/m9.figshare.19298861.v1

Author contributions

PK obtained funding for TRECA and led the study. PK, JM-K, RS and SH developed the TRECA multimedia with Morph. RS and JM-K liaised with the TRECA PPI group. JM-K, LD, RH, KS, HS, TW-S, CAR, SS and TMB set up the SWAT and obtained data. JR and EC analysed the data. PK and TMB drafted the manuscript and all authors contributed to its revision.

Acknowledgments

We wish to acknowledge the significant contributions made by the TRECA PPI group, the TRECA advisory group and the TRECA NIHR Study Steering Committee. We thank Professor Ian Watt for his guidance. The multimedia resources were developed in collaboration with Morph (https://morph.co.uk/)

References

1. Treweek S, Pitkethly M, Cook J, et al.: Strategies to improve recruitment to randomised trials. Cochrane Database Syst. Rev. 2018a; 2018: Mr000013. Publisher Full Text
2. Caldwell PH, Dans L, de Vries MC , et al.: Standard 1: consent and recruitment. Pediatrics. 2012; 129 Suppl 3: S118–S123. PubMed Abstract | Publisher Full Text
3. Eder ML, Yamokoski AD, Wittmann PW, et al.: Improving informed consent: suggestions from parents of children with leukemia. Pediatrics. 2007; 119(4): e849–e859. PubMed Abstract | Publisher Full Text
4. Sheridan R, Martin-Kerry JM, Hudson J, et al.: Why do patients take part in research? An overview of systematic reviews of psychosocial barriers and facilitators. Trials. 2020; 21(1): 259. PubMed Abstract | Publisher Full Text
5. Nuffield Council on Bioethics: Children and clinical research: ethical issues. London.2015.
6. Hunfeld JAM, Passchier J: Participation in medical research; a systematic review of the understanding and experience of children and young people. Patient Educ. Couns. 2012; 87(3): 268–276. PubMed Abstract | Publisher Full Text
7. Barfield RC, Church C: Informed consent in pediatric clinical trials. Curr. Opin. Pediatr. 2005; 17(1): 20–24. Publisher Full Text
8. Simon CM, Siminoff LA, Kodish ED, et al.: Comparison of the informed consent process for randomized clinical trials in pediatric and adult oncology. J. Clin. Oncol. 2004; 22(13): 2708–2717. PubMed Abstract | Publisher Full Text
9. Stryker JE, Wray RJ, Emmons KM, et al.: Understanding the decisions of cancer clinical trial participants to enter research studies: factors associated with informed consent, patient satisfaction, and decisional regret. Patient Educ. Couns. 2006; 63(1): 104–109. Publisher Full Text
10. Crane S, Broome ME: Understanding ethical issues of research participation from the perspective of participating children and young people: a systematic review. Worldviews Evid.-Based Nurs. 2017; 14: 200–209. Publisher Full Text
11. Health Research Authority. Applying a proportionate approach to the process of seeking consent.HRA Guidance.2016.
12. Hermann M: 3-dimensional computer animation--a new medium for supporting patient education before surgery. Acceptance and assessment of patients based on a prospective randomized study--picture versus text. Chirurg. 2002; 73(5): 500–507. Publisher Full Text
13. Tait AR, Voepel-Lewis T, Moscucci M, et al.: Patient comprehension of an interactive, computer-based information program for cardiac catheterization: a comparison with standard information. Arch. Intern. Med. 2009; 169(20): 1907–1914. PubMed Abstract | Publisher Full Text
14. Hutchison C, Cowan C, McMahon T, et al.: A randomised controlled study of an audiovisual patient information intervention on informed consent and recruitment to cancer clinical trials. Br. J. Cancer. 2007; 97(6): 705–711. PubMed Abstract | Publisher Full Text
15. Antoniou EE, Draper H, Reed K, et al.: An empirical study on the preferred size of the participant information sheet in research. J. Med. Ethics. 2011; 37(9): 557–562. PubMed Abstract | Publisher Full Text
16. Shneerson C, Windle R, Cox K: Innovating information-delivery for potential clinical trials participants. What do patients want from multi-media resources?. Patient Educ. Couns. 2013; 90(1): 111–117. PubMed Abstract | Publisher Full Text
17. Office for National Statistics: Exploring the UK’s digital divide 2019.Reference Source
18. Blower S, Swallow V, Maturana C, et al.: Children and young people’s concerns and needs relating to their use of health technology to self-manage long-term conditions: a scoping review. Arch. Dis. Child. 2020; 105(11): 1093–1104. PubMed Abstract | Publisher Full Text
19. Martin-Kerry J, Bower P, Young B, et al.: Developing and evaluating multimedia information resources to improve engagement of children, young people, and their parents with trials (TRECA study): Study protocol for a series of linked randomised controlled trials. Trials. 2017; 18(1): 265. PubMed Abstract | Publisher Full Text
20. Martin-Kerry J, Parker A, Bower P, et al.: SWATted away: the challenging experience of setting up a programme of SWATs in paediatric trials. Trials. 2019; 20(1): 141. PubMed Abstract | Publisher Full Text
21. Rick J, Graffy J, Knapp P, et al.: Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials. Trials. 2014; 15: 407. PubMed Abstract | Publisher Full Text
22. Treweek S, Bevan S, Bower P, et al.: Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?. Trials. 2018; 19(1): 139. PubMed Abstract | Publisher Full Text
23. Heys R, Stoica S, Angelini G, et al.: Intermittent antegrade warm-blood versus cold-blood cardioplegia in children undergoing open heart surgery: a protocol for a randomised controlled study (Thermic-3). BMJ Open. 2020; 10(10): e036974. PubMed Abstract | Publisher Full Text
24. Madurasinghe VW; Sandra Eldridge on behalf of MRCSG, Gordon Forbes on behalf of the SECG: Guidelines for reporting embedded recruitment trials. Trials. 2016; 17(1): 27. PubMed Abstract | Publisher Full Text
25. Sealed Envelope: Thermic-3: 2019. Reference Source
26. Coleman E: Thermic3_TRECA_CONSORT_PIL_DMQ. figshare. Journal Contribution. 2022. Reference Source
27. Morph: 2021. Reference Source
28. Martin-Kerry JM, Knapp P, Atkin K, et al.: Supporting children and young people when making decisions about joining clinical trials: qualitative study to inform multimedia website development. BMJ Open. 2019; 9: e023984. PubMed Abstract | Publisher Full Text
29. Martin-Kerry JM, Higgins S, Knapp P, et al.: Engaging children, young people, parents and health professionals in interviews about abstract topics: using a ranking exercise to elicit views on health research.Under review.
30. Sheridan R, Martin-Kerry J, Watt I, et al.: User testing digital, multimedia information to inform children, young people and their parents about healthcare trials. J. Child Health Care. 2018; 23: 468–482. Publisher Full Text
31. Knapp P, Raynor DK, Silcock J, et al.: Can user testing of a clinical trial patient information sheet make it fit-for-purpose?--a randomized controlled trial. BMC Med. 2011; 9: 89. PubMed Abstract | Publisher Full Text
32. StataCorp: Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC; 2019.
33. Sheridan R, Preston J, Stones S, et al.: Patient and public involvement in a study of multimedia clinical trial information for children, young people and families. Research for All. 2020; 4(1): 47–65. Publisher Full Text
34. Coleman E: Thermic-3 dataset.2022. Publisher Full Text
35. Knapp P, Mandall N, Hulse W, et al.: Evaluating the use of multimedia information when recruiting young people to orthodontics research: A randomised controlled trial. J. Orthod. 2021; 48: 343–351. Publisher Full Text
36. Knapp P, Perry DC, Achten J, et al.: Does digital, multimedia information increase recruitment to a children’s wrist fracture treatment trial, and what do people think of it? A SWAT (Study Within A Trial).Under Review.
37. Palmer BW, Lanouette NM, Jeste DV: Effectiveness of multimedia aids to enhance comprehension of research consent information: a systematic review.2011.
38. Nishimura A, Carey J, Erwin PJ, et al.: Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med. Ethics. 2013; 14: 28. PubMed Abstract | Publisher Full Text
39. Ciciriello S, Johnston RV, Osborne RH, et al.: Multimedia educational interventions for consumers about prescribed and over-the-counter medications. Cochrane Database Syst. Rev. 2013; 4: Cd008416. Publisher Full Text
40. Dekkers T, Melles M, Groeneveld BS, et al.: Web-Based Patient Education in Orthopedics: Systematic Review. J. Med. Internet Res. 2018; 20(4): e143. PubMed Abstract | Publisher Full Text
41. Knox ECL, Quirk H, Glazebrook C, et al.: Impact of technology-based interventions for children and young people with type 1 diabetes on key diabetes self-management behaviours and prerequisites: a systematic review. BMC Endocr. Disord. 2019; 19(1): 7. PubMed Abstract | Publisher Full Text
42. Saengow VE, Chancharoenchai P, Saartying W, et al.: Epilepsy video animation: Impact on knowledge and drug adherence in pediatric epilepsy patients and caregivers. Clin. Neurol. Neurosurg. 2018; 172: 59–61. Publisher Full Text
43. Düzkaya DS, Bozkurt G, Ulupınar S, et al.: The Effect of a Cartoon and an Information Video About Intravenous Insertion on Pain and Fear in Children Aged 6 to 12 Years in the Pediatric Emergency Unit: A Randomized Controlled Trial. J. Emerg. Nurs. 2021; 47(1): 76–87. Publisher Full Text
44. Sahebalam R, Rafieinezhad R, Boskabad M: Comparison of the Efficacy of Jilo Animation Approach versus Conventional Tell-Show-Do (TSD) Technique on Cooperation and Anxiety Levels of Children during Dental Practice: A Randomized Controlled Clinical Trials. J. Dent. (Shiraz). 2020; 21(4): 284–291. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 21 Mar 2022