Using pens as an incentive for trial recruitment of older adults: An embedded randomised controlled trial

Introduction

Randomised controlled trials (RCTs) are vital in establishing the effectiveness of interventions. However, recruitment into RCTs remains a substantial challenge¹, with only around 55% of healthcare RCTs achieving their recruitment target and about 32% having to extend their recruitment period^2,3. This can lead to underpowered trials that fail to find relevant group differences as statistically significant, as well as delayed results and increased costs due to recruitment extensions¹. Despite this, there is currently a lack of evidence to inform researchers of how recruitment into RCTs might best be improved¹, though this has been identified as a high priority to address⁴. Therefore, it is crucial that potential strategies to improve recruitment are robustly evaluated by embedding RCTs evaluating such strategies into real ‘host’ RCTs⁵.

One strategy to improve recruitment is the use of incentives. Based on the principle of reciprocity, receiving incentives is hypothesised to encourage individuals to respond to the positive behaviour in a positive way^6,7. Both monetary and non-monetary incentives are frequently used by clinical trials units in the UK to support recruitment, despite a lack of evidence of their impact⁸. Some evidence suggests that monetary incentives improve RCT recruitment rates¹; however, this strategy is expensive and ethically controversial⁹. In contrast, non-monetary incentives, such as providing pens, are cheaper and more ethically sound¹⁰.

The use of pens as an incentive is an especially appealing strategy for RCTs that utilise large-scale database recruitment. This method involves distributing invitation packs to individuals identified as potentially eligible for a trial from database searches (e.g. General Practitioner [GP] records) and is particularly suitable for recruiting participants with chronic conditions and for recruitment into RCTs that evaluate public health interventions¹¹. Database recruitment is minimally labour intensive, inexpensive, and associated with improved recruitment rates compared to opportunistic recruitment¹¹. Nevertheless, it would be valuable to explore how this strategy could be more efficient, as recruitment yield can still be low. This can especially be the case when recruiting older adults, a population faced with numerous barriers to trial participation, such as reduced mobility, a lack of trust and understanding of trials, and the belief that participation would be too burdensome^12,13. For example, an RCT evaluating a podiatry intervention for falls prevention in older adults randomised just 2.7% of those approached via database recruitment into the trial¹⁴.

The inclusion of pens within trial invitation packs may not only improve recruitment rates through encouraging reciprocal positive behaviour, but the convenience of a pen being readily available may prompt rapid completion and return of trial documentation¹⁵. Despite this, no previous RCTs have evaluated the impact of distributing pens within invitation packs on recruitment into RCTs. However, a trial evaluating recruitment into a questionnaire survey reported that providing a study-branded pen within the invitation pack, to individuals who had previously not responded, improved response rates¹⁶.

Some relevant trials have explored whether providing pens improves response rates to postal questionnaires, though these have yielded mixed findings. A previous trial found that sending a pen with a postal questionnaire to consultants did not improve response rates¹⁵. However, other trials have reported that providing a study-branded pen or pencil was a cost-effective strategy which improved follow-up questionnaire response rates^16,17. Similarly, a UK-based embedded RCT, evaluating an osteoporosis screening programme, found that including pens with postal questionnaires led to a marginal increase in response rates, a reduction in the number of reminders required, and a reduction in time to return the questionnaire¹⁰. While the effects reported in this trial were all very small, the provision of pens was considered cost-effective due to their low cost. Given these promising results, it would be valuable for further embedded RCTs to evaluate whether these findings generalise to improvements in trial recruitment when a pen is included within the invitation pack.

In this paper we describe an embedded RCT (or ‘study within a trial’ [SWAT]) designed to evaluate the effectiveness of including a pen within the trial invitation pack on the recruitment of older adults, identified from GP database searches, into the Occupational Therapist Intervention Study (OTIS)¹⁸. Specifically, this RCT evaluated the impact of providing pens on subsequent recruitment rates into the OTIS trial, as well as return rates of recruitment documentation, eligibility of respondents, and the retention of participants. This trial not only helped to address the lack of RCTs on the use of pens as an incentive for trial recruitment, but also explored how to further improve the efficiency of database recruitment, focusing specifically on older adults, who can be particularly challenging to recruit.

Methods

Results

We randomised 1943 participants into this embedded trial (648 to receive a pen with their invitation pack; 1295 to not receive a pen); however, 81 of these invitation packs were not sent out (pen arm = 28; no pen arm = 53) and were excluded from this analysis (Figure 1). Therefore, we included 1862 participants in this analysis (pen arm = 620, 33.3%; no pen arm = 1242, 66.7%). Of these participants, 919 (49.4%) were posted an invitation pack from GP practice 1 and 943 (50.6%) were posted a pack from GP practice 2. Raw data are available on Open Science Framework²¹.

Figure 1. Flow diagram depicting the recruitment and retention of participants in this embedded trial.

Discussion

This embedded RCT evaluated the effectiveness of including a non-monetary incentive, in the form of a York Trials Unit branded pen, within invitation packs mailed out from GP practices on the recruitment of older adults into the OTIS trial. The absolute difference in the percentage of embedded trial participants randomised to the OTIS trial was 0.17% (4.5% in the pen arm, 4.3% in the no pen arm) and was not statistically significant, which suggests that providing a pen within trial invitation packs was not an effective incentive to improve recruitment of older adults into the host RCT.

Whether providing pens as a recruitment incentive is cost-effective remains uncertain. Based on the randomisation rate of 4.3% (or 43 per 1,000) achieved in this embedded trial using standard invitation packs, and given that the printing, packaging, and postage costs for each standard pack was £2.53, it costs £2,530 to send 1,000 standard packs to recruit 43 participants into the host trial, or £58.84 per participant. The pens cost £0.32 each, so it costs an additional £320 per 1,000 packs distributed with a pen. For this price, approximately five participants could be recruited using standard packs; therefore, including a pen would need to increase the percentage of eligible participants randomised by 0.5% (or 5 per 1,000) to be cost-effective. If the point estimate reported here (0.17%) is the true difference, providing pens would not be cost-effective. However, if the upper 95% confidence limit of the difference (2.16%) is the true difference, providing pens would likely be cost-effective. Consequently, additional trials are needed to evaluate this recruitment strategy. Furthermore, as the cost of pens could be reduced if a non-branded style were used, further trials could additionally evaluate the effectiveness of the branding.

Within this embedded RCT, the provision of pens was not associated with a significant difference in any of the secondary outcomes, though all results favoured the pen arm. Providing a pen in the invitation pack resulted in a small increase in screening form return rates (absolute difference of 2.5%). There were also trends for those who received a pen to return their screening form more quickly and to be more likely to remain in the OTIS trial for at least 3 months after being randomised (96.4% vs. 90.7%). These results suggest that including a pen in trial invitation packs may marginally boost the return of trial recruitment documentation among older adults, a population that can be particularly difficult to recruit^12,13, and may have benefits on trial retention. While improvements in screening form response rates did not translate to improvements in randomisation rates in this trial, it is possible that it may do in other trials, particularly those with broader eligibility criteria. This further highlights the need for future trials to evaluate pens as a recruitment incentive.

This embedded RCT adds to the limited and mixed literature on the provision of pens on response rates to trial documentation, with some previous trials showing an effect^10,16,17 and others showing no effect¹⁵. While previous trials have considered the impact of providing pens on questionnaire return rates, this trial was the first to evaluate the inclusion of a pen within the trial invitation pack on RCT recruitment.

This embedded trial was limited by only involving two GP practice mail-outs and focusing on older adults. It would be beneficial for future embedded trials, within large-scale RCTs utilising database recruitment, to involve a greater number of mail-outs to further evaluate this question. Further research should also explore the impact of providing pens on the recruitment of different participant populations (e.g. different age groups). Meta-analysis could then be used to explore the effectiveness of including pens within trial invitation packs and whether this varies depending on participant demographics.

Conclusions

Providing a pen within trial invitation packs had marginally beneficial effects on screening forms return rates and retention within this embedded trial, though did not improve randomisation rates of older adults into the host RCT. Further embedded trials are necessary to evaluate whether providing pens in invitation packs is a cost-effective incentive for trial recruitment.

Data availability