A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment rates in a randomised controlled trial of yoga for older adults with multimorbidity

Introduction

Efficient recruitment to randomised controlled trials (RCTs) is important to achieve the target sample size and statistical power within the planned budget and time frame. Incentives, monetary or otherwise, are sometimes used to increase trial recruitment.¹ We conducted a ‘study within a trial’ (SWAT) to evaluate the effects of including a small, unconditional financial incentive and/or a pen in the postal recruitment pack on the rate of randomisation into the host trial.

Methods

Results

In total, 852 allocations were generated but, due to one of the participating GP practices having a shorter mailing list than anticipated, only 818 (96.0%) were used (Table 1; Figure 1). In these analyses, the potential participants who were sent a pen (n = 409) consist of the group who received a pen and a £5 note (n = 203), and the group who received a pen only (n = 206). These are compared with those who were not sent a pen (n = 409), consisting of the group who were only sent a £5 note (n = 206), and the group who were sent neither a pen nor £5 (n = 203). Similarly, those who were sent £5 (n = 409) consist of the group sent both a pen and £5 (n = 203), and the group sent £5 only (n = 206). These are compared with the potential participants who were not sent £5 (n = 409), consisting of the group who were sent a pen only (n = 206), and the group who were sent neither a pen nor £5 (n = 203).

Figure 1. Participant flow diagram.

26 (3.2%) SWAT participants were randomised into the host trial (Table 2). There was no evidence that randomisation rates were improved by including a pen (pen: 15/409, 3.7%; no pen: 11/409, 2.7%; OR 1.38, 95% CI 0.63–3.04, p = 0.43) or £5 (£5: 14/409, 3.4%; no £5: 12/409, 2.9%; OR 1.18, 95% CI 0.54–2.57, p = 0.69) in the recruitment packs. The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 0.98, 95% CI 0.20–4.79, p = 0.98). In a meta-analysis with the OTIS SWAT, the pooled OR associated with receipt of a pen was 1.12 (95% CI 0.75–1.67, p = 0.58) (Figure 2).

Figure 2. Meta-analysis of inclusion of a pen in postal recruitment packs on randomisation into host trial.

There was no evidence that including a pen increased the likelihood of returning a screening form (pen: 66/409, 16.1%; no pen: 61/409, 14.9%; OR 1.10; 95% CI 0.75–1.61, p = 0.61), but there was strong evidence for including £5 (£5: 77/409, 18.8%; no £5: 50/409, 12.2%; OR 1.67; 95% CI 1.13–2.45, p = 0.01). The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 1.66, 95% CI 0.76–3.60, p = 0.20).

There was no evidence of a difference in time to return a screening form associated with inclusion of a pen (HR 1.09; 95% CI 0.77–1.55, p = 0.61), but including £5 decreased the time to return a screening form (HR 1.56; 95% CI 1.09–2.22, p=0.02). See Kaplan–Meier plots (Figure 3). The Grambsch and Therneau test did not indicate deviation from the proportional hazards assumption.³ The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 1.56, 95% CI 0.76–3.19, p = 0.22).

Figure 3. Kaplan–Meier curves for time to return screening form.

The additional cost of including a pen in the postal mailout was £0.32; the inclusion of £5 additionally cost only the value of the note itself. Given the 1% increase in participants randomised when sent a pen, 100 (1/0.01) potential participants would need to be sent a pen to recruit one additional participant at a cost of £32 (100×£0.32). We would need to send 200 participants £5, at a cost of £1000, to recruit one extra participant.

Discussion

Participants sent a pen or £5 were marginally more likely to be randomised into the GYY trial than those who did not receive the incentive; however, the differences were not statistically significant. The randomisation rates in the ‘no intervention’ arms (2.7% and 2.9%) were similar to the 3% assumed in the sample size calculation but the observed group differences were smaller than the 4% difference we were powered for; therefore, this SWAT was underpowered to detect the differences observed.

There was little or no evidence that sending a pen increased the likelihood of returning a screening form or decreasing time to return the form.

Although no statistically significant interactions between the pen and £5 were observed, this cannot be ruled out as the sample size of this trial was likely insufficient to be powered to detect an interaction.

A small, monetary incentive was effective at prompting return of the screening form, and of a swifter return, but it did not result in more participants being randomised into the host trial. Some anecdotal evidence from the GYY trial’s process evaluation suggested participants felt it unnecessary to receive £5 with their recruitment pack as they would willingly have joined the trial without this purely to help themselves, others and the research. In addition, this may have caused potential confusion if participants discussed receiving £5 during their yoga sessions as to why some received it and some did not. Such sentiments were not expressed in relation to being sent a pen, potentially suggesting that people view non-monetary incentives differently (more like a gift) than monetary incentives. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in older adults with multimorbidity.

Pens are cheaper but provided little evidence of benefit. If the observed effect of a 1% difference was true then we would need sufficient SWATs to provide an overall sample size of around 11,000 participants to confirm this. Because the extra cost of recruiting an additional participant is relatively small, more SWATs are required to assess whether this difference is a true effect, since sending pens could be a cost-effective intervention for recruitment.

Data availability