F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.52164.1 Research Article Articles A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment rates in a randomised controlled trial of yoga for older adults with multimorbidity

[version 1; peer review: 1 approved, 1 approved with reservations]

 Fairhurst Caroline Conceptualization Data Curation Formal Analysis Funding Acquisition Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-0547-462X a 1 Roche Jenny Data Curation Formal Analysis Visualization Writing – Original Draft Preparation Writing – Review & Editing 1 Bissell Laura Conceptualization Funding Acquisition Writing – Review & Editing 2 Hewitt Catherine Conceptualization Funding Acquisition Methodology Project Administration Supervision Writing – Review & Editing 1 Hugill-Jones Jess Project Administration Writing – Review & Editing https://orcid.org/0000-0002-9915-3941 1 Howsam Jenny Conceptualization Funding Acquisition Writing – Review & Editing 2 Maturana Camila S Project Administration Writing – Review & Editing https://orcid.org/0000-0002-4946-8515 1 Corbacho Martin Belen Conceptualization Funding Acquisition Writing – Review & Editing https://orcid.org/0000-0002-2359-0379 1 Paul Shirley-Anne S Project Administration Writing – Review & Editing 1 Rose Fi Project Administration Writing – Review & Editing https://orcid.org/0000-0003-0587-683X 1 Torgerson David J Conceptualization Funding Acquisition Methodology Project Administration Supervision Writing – Review & Editing https://orcid.org/0000-0002-1667-4275 1 Ward Lesley Investigation Writing – Review & Editing 3 Wiley Laura Project Administration Writing – Review & Editing https://orcid.org/0000-0001-9619-4484 1 Tew Garry A Conceptualization Funding Acquisition Investigation Methodology Project Administration Supervision Writing – Review & Editing 3 York Trials Unit, ARRC Building, Department of Health Sciences, University of York, UK, Heslington, York, YO10 5DD, UK British Wheel of Yoga Qualifications, 25 Jermyn Street, Sleaford, NG34 7RU, UK Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, NE1 8ST, UK caroline.fairhurst@york.ac.uk

No competing interests were disclosed.

 27 4 2021 2021 10 326 19 4 2021 Copyright: © 2021 Fairhurst C et al. 2021 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: Monetary and other incentives may increase recruitment to randomised controlled trials.

Methods: This was a 2x2 factorial ‘study within a trial’ of including a pen and/or £5 with a postal recruitment pack to improve randomisation rate (primary outcome) into the host Gentle Years Yoga trial in older adults with multimorbidity. Secondary outcomes: return, and time to return, of screening form, and the cost per additional participant recruited. Binary data were analysed using logistic regression and time to return data using Cox proportional hazards regression.

Results: 818 potential host trial participants included. Between those sent a pen (n=409) and not sent a pen (n=409), there was no evidence of a difference in the likelihood of being randomised (15 (3.7%) versus 11 (2.7%); OR 1.38, 95% CI 0.63–3.04), in returning a screening form (66 (16.1%) versus 61 (14.9%); OR 1.10, 95% CI 0.75–1.61) nor in time to return the screening form (HR 1.09, 95% CI 0.77–1.55). There was evidence of improved screening return rates (77 (18.8%) versus 50 (12.2%); OR 1.67, 95% CI 1.13–2.45) and time to return screening form (HR 1.56, 95% CI 1.09–2.22) but not randomisation (14 (3.4%) versus 12 (2.9%); OR 1.18, 95% CI 0.54–2.57) in those sent £5 (n=409) compared with those not sent £5 (n=409). No significant interaction effects between the interventions were observed. The cost per additional participant recruited was £32 for the pen and £1000 for the £5 incentive.

Conclusion: Including a small, monetary incentive encouraged increased and faster response to the recruitment invitation but did not result in more participants being randomised into the host trial. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in this population. Pens are cheaper but did not provide evidence of benefit. Further studies may be required.

 study within a trial pen financial incentive recruitment factorial randomised controlled trial older people multimorbidity National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme 17/94/36 The GYY trial and SWATs were funded by the United Kingdom National Health Service (NHS) through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme, open call project number 17/94/36. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Efficient recruitment to randomised controlled trials (RCTs) is important to achieve the target sample size and statistical power within the planned budget and time frame. Incentives, monetary or otherwise, are sometimes used to increase trial recruitment. 1 We conducted a ‘study within a trial’ (SWAT) to evaluate the effects of including a small, unconditional financial incentive and/or a pen in the postal recruitment pack on the rate of randomisation into the host trial.

 Methods Design

This 2x2 randomised factorial SWAT was embedded in the Gentle Years Yoga (GYY) trial, which is a multi-centre RCT of the clinical and cost-effectiveness of a yoga programme for older adults with multimorbidity (currently recruiting; ISRCTN13567538, registered 18/03/2019 https://www.isrctn.com/ISRCTN13567538). The SWAT was registered with the Northern Ireland Network for Trials Methodology Research SWAT repository on 01/04/2018 ( SWAT94; https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/). The GYY trial, and its embedded sub-studies, has approval from the North East–York Research Ethics Committee, obtained on 24/04/2019 (19/NE/0072), and the Health Research Authority.

 Participants

Patients who appeared to meet the GYY trial eligibility criteria and were to be mailed a recruitment pack, as identified by four participating GP practices, were eligible for inclusion in this methodological sub-study. The recruitment pack included an invitation letter, participant information sheet, consent form, screening form, and prepaid envelopes to return documentation to the York Trials Unit, University of York. A random sample of packs also included a £5 note and/or a pen (branded with the trial logo) as part of this SWAT. The packs were sent out in August 2019.

 Interventions

Financial incentives have been found to increase recruitment by 4% (95% CI -1–8%) in a meta-analysis. 1 However, most of the included studies used an incentive of £100, which is larger than publicly funded trials can usually afford. There remains, therefore, uncertainty as to whether financial incentives should be used and, if so, what amount.

Offering a potential participant a gift such as a pen may make them more likely to take up the invitation to enrol in a trial. It is also possible that the convenience of having a pen to hand upon receipt of the invitation may help facilitate a swifter response. However, a previous SWAT conducted by the York Trials Unit evaluated the use of pens as an incentive for recruitment into the OTIS trial of older adults and showed no difference on randomisation rate (pen 4.5%; no pen 4.3%, odds ratio (OR) 1.04, 95% CI 0.65–1.67, p = 0.86), screening rate (pen 14.2%, no pen 11.7%, OR 1.25, 95% CI 0.94–1.67, p = 0.12), or time to return screening form (hazard ratio (HR) 1.23, 95% CI 0.94–1.60, p = 0.13). 2 To our knowledge, this is the only previous RCT to evaluate pens to increase trial recruitment, so more evidence is needed; hence, we conducted this SWAT.

 Outcome measures

The primary outcome was randomisation into the host GYY trial. Secondary outcomes were return, and time to return, of a screening form to the York Trials Unit. The cost per additional participant recruited was calculated for each intervention.

 Sample size and randomisation

Due to financial restrictions, we could afford to involve a sample of 850 recruitment packs in this SWAT. This would give 80% power (two-sided α=0.05) to detect a difference in recruitment rate of 4% (from 3% to 7%) for either of the interventions, relative to not receiving that intervention.

Block randomisation of size 4 was used to allocate recruitment packs 1:1:1:1 to: no pen or £5; £5 only; pen only; or pen and £5. A trial statistician, not involved in the production of recruitment packs or recruitment of participants, generated the sequence using Stata v15 (RRID: SCR_012763). Stata is a proprietary software but an open-access alternative in which the sequence could have been generated is Microsoft Excel (RRID: SCR_016137).

 Blinding and consent

The statisticians were not blinded to allocation. Similarly, due to the nature of the interventions, participants could not be blinded to their allocation; however, they were not specifically informed about the SWAT nor that the incentive they received had been chosen through a process of randomisation. Specific consent for the trial was not required by the Research Ethics Committee, as it was considered low risk. Written informed consent for the main trial was obtained from all participants who agreed to take part in GYY.

 Statistical analysis

The primary comparisons in this trial are the main effects of being sent a pen, and of being sent a £5 note. Returning a screening form and being randomised into the GYY trial were both analysed using multivariable logistic regression, including the two interventions (pen and £5). Time to return the screening form (in days from the date the recruitment pack was sent out to the date it was returned) was analysed using Cox proportional hazards regression. Screening forms that were not returned were censored at eight weeks after they were sent out. These analyses provide an estimate of the average effect of each intervention, assuming there is no interaction between them. In secondary analyses, the interaction between the two interventions was tested by extending the original models to include the interaction term. Analyses were conducted in Stata v16 (RRID: SCR_012763). An open-access alternative that can perform an equivalent function to Stata for analysis is R, a free software environment for statistical computing and graphics (RRID: SCR_001905).

 Results

In total, 852 allocations were generated but, due to one of the participating GP practices having a shorter mailing list than anticipated, only 818 (96.0%) were used ( Table 1; Figure 1). In these analyses, the potential participants who were sent a pen (n = 409) consist of the group who received a pen and a £5 note (n = 203), and the group who received a pen only (n = 206). These are compared with those who were not sent a pen (n = 409), consisting of the group who were only sent a £5 note (n = 206), and the group who were sent neither a pen nor £5 (n = 203). Similarly, those who were sent £5 (n = 409) consist of the group sent both a pen and £5 (n = 203), and the group sent £5 only (n = 206). These are compared with the potential participants who were not sent £5 (n = 409), consisting of the group who were sent a pen only (n = 206), and the group who were sent neither a pen nor £5 (n = 203). Number of participants randomised to each group.
Pen No pen Total
£5 203 206 409
No £5 206 203 409
Total 409 409 818
Participant flow diagram.

26 (3.2%) SWAT participants were randomised into the host trial ( Table 2). There was no evidence that randomisation rates were improved by including a pen (pen: 15/409, 3.7%; no pen: 11/409, 2.7%; OR 1.38, 95% CI 0.63–3.04, p = 0.43) or £5 (£5: 14/409, 3.4%; no £5: 12/409, 2.9%; OR 1.18, 95% CI 0.54–2.57, p = 0.69) in the recruitment packs. The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 0.98, 95% CI 0.20–4.79, p = 0.98). In a meta-analysis with the OTIS SWAT, the pooled OR associated with receipt of a pen was 1.12 (95% CI 0.75–1.67, p = 0.58) ( Figure 2). SWAT results.
Pen (n = 409) No pen (n = 409) £5 (n = 409) No £5 (n = 409) Interaction coefficient (95% CI), p-value c
Returned screening form, n (%) 66 (16.1) 61 (14.9) 77 (18.8) 50 (12.2) 1.66 (0.76–3.60), 0.20
Adjusted odds ratio a (95% CI), p-value 1.10 (0.75–1.61), 0.61 1.67 (1.13–2.45), 0.01
Time to return (days) b , median (IQR) 11 (9–14) 11 (7–18) 11 (10–18) 8.5 (7–14) 1.56 (0.76–3.19), 0.22
Adjusted hazards ratio a (95% CI), p-value 1.09 (0.77–1.55), 0.61 1.56 (1.09–2.22), 0.02
Randomised, n (%) 15 (3.7) 11 (2.7) 14 (3.4) 12 (2.9) 0.98 (0.20–4.79), 0.98
Adjusted odds ratio a (95% CI), p-value 1.38 (0.63–3.04), 0.43 1.18 (0.54–2.57), 0.69

All comparisons are between the intervention compared with its respective control; treatment effect estimates >1 represent a favourable outcome for the relevant intervention.

Median and inter-quartile range (IQR) calculated for returned forms only.

Interaction between the two interventions.

 Meta-analysis of inclusion of a pen in postal recruitment packs on randomisation into host trial.

There was no evidence that including a pen increased the likelihood of returning a screening form (pen: 66/409, 16.1%; no pen: 61/409, 14.9%; OR 1.10; 95% CI 0.75–1.61, p = 0.61), but there was strong evidence for including £5 (£5: 77/409, 18.8%; no £5: 50/409, 12.2%; OR 1.67; 95% CI 1.13–2.45, p = 0.01). The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 1.66, 95% CI 0.76–3.60, p = 0.20).

There was no evidence of a difference in time to return a screening form associated with inclusion of a pen (HR 1.09; 95% CI 0.77–1.55, p = 0.61), but including £5 decreased the time to return a screening form (HR 1.56; 95% CI 1.09–2.22, p=0.02). See Kaplan–Meier plots ( Figure 3). The Grambsch and Therneau test did not indicate deviation from the proportional hazards assumption. 3 The interaction between the interventions was investigated as a secondary analysis and was not found to be statistically significant (interaction coefficient 1.56, 95% CI 0.76–3.19, p = 0.22). Kaplan–Meier curves for time to return screening form.

The additional cost of including a pen in the postal mailout was £0.32; the inclusion of £5 additionally cost only the value of the note itself. Given the 1% increase in participants randomised when sent a pen, 100 (1/0.01) potential participants would need to be sent a pen to recruit one additional participant at a cost of £32 (100×£0.32). We would need to send 200 participants £5, at a cost of £1000, to recruit one extra participant.

 Discussion

Participants sent a pen or £5 were marginally more likely to be randomised into the GYY trial than those who did not receive the incentive; however, the differences were not statistically significant. The randomisation rates in the ‘no intervention’ arms (2.7% and 2.9%) were similar to the 3% assumed in the sample size calculation but the observed group differences were smaller than the 4% difference we were powered for; therefore, this SWAT was underpowered to detect the differences observed.

There was little or no evidence that sending a pen increased the likelihood of returning a screening form or decreasing time to return the form.

Although no statistically significant interactions between the pen and £5 were observed, this cannot be ruled out as the sample size of this trial was likely insufficient to be powered to detect an interaction.

A small, monetary incentive was effective at prompting return of the screening form, and of a swifter return, but it did not result in more participants being randomised into the host trial. Some anecdotal evidence from the GYY trial’s process evaluation suggested participants felt it unnecessary to receive £5 with their recruitment pack as they would willingly have joined the trial without this purely to help themselves, others and the research. In addition, this may have caused potential confusion if participants discussed receiving £5 during their yoga sessions as to why some received it and some did not. Such sentiments were not expressed in relation to being sent a pen, potentially suggesting that people view non-monetary incentives differently (more like a gift) than monetary incentives. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in older adults with multimorbidity.

Pens are cheaper but provided little evidence of benefit. If the observed effect of a 1% difference was true then we would need sufficient SWATs to provide an overall sample size of around 11,000 participants to confirm this. Because the extra cost of recruiting an additional participant is relatively small, more SWATs are required to assess whether this difference is a true effect, since sending pens could be a cost-effective intervention for recruitment.

 Data availability Underlying data

OSF: Underlying data for ‘A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment rates in a randomised controlled trial of yoga for older adults with multimorbidity’. https://doi.org/10.17605/OSF.IO/2CJZH. 4

This project contains the following underlying data:

Data file 1. GYY recruitment SWAT csv data.csv

Data file 2. GYY recruitment SWAT Stata data.dta

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

 Reporting guidelines

OSF: CONSORT checklist for ‘A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment rates in a randomised controlled trial of yoga for older adults with multimorbidity’. https://doi.org/10.17605/OSF.IO/EU68F. 5

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

 Acknowledgements

We would like to acknowledge the contributions of Professor Tim Rapley (Professor of Applied Health Care Research, Department of Social Work, Education and Community Wellbeing, Northumbria University) and Helen Tilbrook (York Trials Unit, University of York), who were co-applicants on the initial GYY trial grant proposal.

 References Treweek S Pitkethly M Cook J : Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2:MR000013. 29468635 10.1002/14651858.MR000013.pub6 PMC7078793 Whiteside K Flett L Mitchell A : Using pens as an incentive for trial recruitment of older adults: An embedded randomised controlled trial. F1000Res. 2019;8:315. Published 2019 Mar 21. 30984388 10.12688/f1000research.18300.1 PMC6446499 Grambsch PM Therneau TM : Proportional hazards tests and diagnostics based on weighted residuals. Biometrika. 1994;81:515–526. 10.1093/biomet/81.3.515 Fairhurst C Torgerson DJ Roche J : GYY recruitment factorial pen/£5 SWAT. 2021, April 16. 10.17605/OSF.IO/2CJZH Fairhurst C Torgerson DJ Roche J : GYY recruitment factorial pen/£5 SWAT. 2021, April 16. 10.17605/OSF.IO/EU68F 10.5256/f1000research.55404.r122638 Reviewer response for version 1 Shiely Frances 1 Referee https://orcid.org/0000-0003-0969-8321 Trials Research and Methodologies Unit, HRB Clinical Research Facility, Mercy University Hospital, University College Cork, Cork, Ireland

Competing interests: No competing interests were disclosed.

 24 2 2022 Copyright: © 2022 Shiely F 2022 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Article Summary

This was a 2x2 factorial ‘study within a trial’ of including a pen and/or £5 with a postal recruitment pack to improve randomisation rate (primary outcome) into the host Gentle Years Yoga trial in older adults with multimorbidity. The authors conclude that including a small, monetary incentive encouraged an increased and faster response to the recruitment invitation but did not result in more participants being randomised into the host trial.

Major

The major issue is the phrasing of the primary outcome as the rate of randomisation. You didn’t analyse the time to randomisation in your analysis, you analysed the proportion randomised. I believe a better description is the proportion randomised to the host trial or just simply number of participants randomised to the host trial.

Given my comments, I would remove the term 'rates' from the title also.

The presentation of secondary outcomes prior to presenting the primary outcome could be described as selective outcome reporting. Please adjust this throughout and present your primary outcome first, including in your tables.

Minor

General Comments

Check your tenses throughout. Sometime you say 'was' and sometimes you say 'has'. 'Consist of' should be 'consisted of'. Check the English language also as sometime some sentences are phrased a little unusually or are missing the indefinite article or definite article.

Abstract

'818 potential host trial participants included'. Add 'were included'.

In your results you say there was no evidence of a difference in the likelihood of being randomised. This is nitpicking a little but I think it is clear to stick to the number or proportion randomised.

Conclusion of abstract: Again modify your response here and put your primary outcome first.

Methods Section

Design

I think you should include information in this section on the consent. I can see it further down. Perhaps reconsider all the smaller headings to bring the information together. This is a very small issue.

Participants

Remove the word 'appeared' from the first sentence as it introduces unnecessary uncertainty around your eligibility criteria.

Intervention

The information you have currently given in the intervention section belongs in the introduction section. In this section, I would expect to see what the intervention was. I would also expect to see what the control group was in this SWAT. I assume it is no incentive, but it should be clearly stated.

Outcomes

Again, consider the primary outcome here. Also, when you state the secondary outcomes, list them all together. The cost per additional participant recruited looks like it was not part of the secondary outcomes as it is in a separate sentence.

Results

Change 'potential participants' to 'participants' as the SWAT is completed. Delete the following text as it is all evident in Table 1 and is actually easier to follow in the table.

"In these analyses, the potential participants who were sent a pen (n = 409) consist of the group who received a pen and a £5 note (n = 203), and the group who received a pen only (n = 206). These are compared with those who were not sent a pen (n = 409), consisting of the group who were only sent a £5 note (n = 206), and the group who were sent neither a pen nor £5 (n = 203). Similarly, those who were sent £5 (n = 409) consist of the group sent both a pen and £5 (n = 203), and the group sent £5 only (n = 206). These are compared with the potential participants who were not sent £5 (n = 409), consisting of the group who were sent a pen only (n = 206), and the group who were sent neither a pen nor £5 (n = 203)."

Conclusion

Your primary outcome should be reported first.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

Trial Methodology; Epidemiology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Fairhurst Caroline York Trials Unit, UK

Competing interests: N/A

 1 3 2022

Author response:

Thank you for your helpful review. We have amended the text and title to remove ‘rate(s)’ accordingly. Throughout the Abstract, main body of the text, and the tables we have made sure that the primary outcome is presented before the secondary outcomes. The reason we sometimes presented the outcome of 'returned screening form' first was not to be selective around the reporting of outcomes, but rather to reflect the chronological order in which the outcomes occurred (participants first returned their screening forms and, if eligible, were randomised). However, we can see how this might be construed and so have amended the text and tables accordingly so the primary outcome is presented first. We have changed the Abstract according to all your other suggestions too, plus a few others to keep it within the word limit of 300. 

We agreed that information on consent was better placed in the Participants section, and so moved this detail up from its original place further down the paper. We have moved text from the Intervention section to the Introduction, and have provided a clear description of the four groups of the SWAT in the Intervention section.

As suggested, we have deleted text in the Results that was also depicted in Table 1. The use of the terminology ‘potential participant’ refers to the fact that the people sent the recruitment pack were only ‘potential’ participants in relation to the host trial, but they were participants in the SWAT, and so we can see the confusion. We have reconsidered the wording throughout.

In general, the paper has been proof-read and edits made to improve clarity and readability.

We really appreciate your comments and believe the changes we have made in response to them have really improved the paper. If you have any other suggestions for improvement, we would be happy to receive them.

 10.5256/f1000research.55404.r97206 Reviewer response for version 1 Dombrowski Stephan 1 Referee University of New Brunswick, Fredericton, NB, Canada

Competing interests: No competing interests were disclosed.

 11 2 2022 Copyright: © 2022 Dombrowski S 2022 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

Thank you for the opportunity to review this very interesting 2x2 experimental SWAT study which I enjoyed reading.  The paper is well-written and interesting. Below are some comments, the authors might want to consider for a potential revision of their manuscript.

Abstract:

Minor suggestion is to re-phrase “into the host Gentle Years Yoga trial in older adults with multimorbidity” to “into the host trial Gentle Years Yoga in older adults with multimorbidity”. Separating ‘host’ and ‘trial’ was confusing on the first reading. There are a couple more instances in the paper which might benefit from keeping host trial as one phrase. Maybe ‘in’ should be ‘for’ in this sentence?

Results first sentence (“818 potential host trial participants included.”) seems ungrammatical.  ‘were included’ perhaps?

Conclusions: “ Including a small, monetary incentive encouraged increased and faster response to the recruitment invitation but did not result in more participants being randomised into the host trial. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in this population.”. I think this is accurate, but wonder if you want to consider putting the main message first for your primary outcome (numbers randomised), rather than the secondary outcomes of response rate and speed?

Conclusions: “Pens are cheaper but did not provide evidence of benefit.”. I suppose that depends on the pen that is chosen.  Either phrase in the past (“were cheaper”) or consider omitting. Not sure it adds much to the conclusions.

Conclusions: the last sentence seems to include very little study-specific information that wouldn’t be true for almost every research study (“Further studies may be required.”) and could be omitted without losing any critical content.

Introduction:

Overall I do like a focused and succinct introduction but wondered whether a little more information was required to contextualise the study some more.  I found a lot of relevant contextual information in the methods section under ‘Interventions’ which I would have expected in the introduction. I wonder if that can be moved up within the manuscript.  Below I suggest additional information for the methods section which outlines the intervention in somewhat more detail.

Did the authors have any study aims or a priori hypotheses they were testing within the study? If yes then it might be useful to explicitly state these at the end of the introduction.

Methods

Very minor point: Did participants receive multiple ‘prepaid envelopes’ or just one (all intervention material is described in singular except the envelopes)? The reason I am asking is because the intervention was included as part of the whole package.  The more content in the package, and the more complex the response to the invite (e.g. through having to navigate multiple envelopes), the less likely people are to respond and miss the intervention.

Under intervention, it would be useful to move the current text to the introduction as it provides a rationale for the study, rather than describing the intervention. It would be useful to describe intervention details, including:

How was the value of 5 pounds decided?

Was the money provided in cash?

Why would the pen and the money components interact? Are there any theoretical reason why such an interaction might occur?

What did patients get told about the money, if anything? This is important as having seemingly randomly a 5er in a research recruitment package might be confusing to people, unless there is some kind of explanation why it was sent. Incentives and rewards typically have an element of contingency attached to them.

More details on the pen would be useful, as there is a difference between a cheap throw-away pen and a more substantial and durable one. A picture of the pen would be useful. In fact, a picture of the entire online package with all its content would be even better.

Could the authors explain how they calculated the following “The cost per additional participant recruited was calculated for each intervention.”?

All the Tables and figures have one group designated as pen/no pen, and the other as £5/ No £5. Would it be better to talk about ‘money’ or ‘cash’ for international audiences (i.e. intervention groups = ‘pen/no pen’ and ‘money/ no money), rather than use the label which designates the exact value in British currency?

Results

The meta-analysis in the results section is somewhat surprising given that it has not been mentioned in the methods section and does not appear to be related to a research question. It might be more useful to compare the results of this study with other similar studies in the discussion section.

Discussion

In line with the conclusions in the abstract, and for consistency of message, I wonder if the authors might want to consider to state no difference in randomisation between the groups, rather than using more ambiguous phrasing of ‘marginally more significant.

The discussion of the ‘anecdotal evidence’ is interesting, adding qualitative accounts of participants would have added context and additional information useful to interpret the findings. It would have been particularly informative to ask those who returned but declined, as well as those who never returned. Perhaps add a sentence for future SWATs to consider?

The sentence “Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in older adults with multimorbidity.” could be increased in clarity. The previous paragraph discusses both pen and cash, so it is not clear if the authors mean either or only cash with ‘this intervention’. I assume the money, given the paragraph structure, but a quick read of the paper might lead to some confusion. Replacing ‘this intervention’ with ‘a 5 pounds incentive’ or something might be useful.

I think that the interpretation of the findings in relation to the incentive depends on how the money was presented in the context of the recruitment package – see methods comments on additional information which would add value to the paper. The conclusion might benefit from taking into consideration that the form of delivery of the incentive could have affected how people saw it, which subsequently impacted their decision making around participation.

I wonder if a concluding sentence might be beneficial to add to finish the discussion.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Health psychology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Fairhurst Caroline York Trials Unit, UK

Competing interests: N/A

 1 3 2022

Author response:

Thank you for your thorough and constructive review. We have changed the Abstract according to all your suggestions, plus a few others to keep it within the word limit of 300. We have moved text from the Intervention section to the Introduction, where we agree it is better placed, and have added in our hypotheses to the end of the Introduction. In relation to the content of the recruitment packs, all potential participants of the host trial, GYY, received the same standard recruitment pack in the post. This included an invitation letter, participant information sheet (PIS), consent and contact details form, screening form and two prepaid envelopes. The reason that two envelopes were included is that, if the person wished to take part in the trial, we asked them to return their consent form and contact details in one envelope, and their completed screening form in the other (all documents were linked by a unique, prepopulated trial ID number). This is so that, should the post be intercepted, an individual’s contact details are separated from the confidential information they provide in the screening form. In the SWAT, for those randomised to receive a pen and/or £5, the allocated intervention was enclosed within the standard recruitment pack for the host trial. The only other difference was some wording on the PIS, to include:

“Please find enclosed a complimentary £5 AND/OR pen given as a thank you for considering taking part. If you choose not to take part you can still keep this.”

Therefore, participants in all four groups of the SWAT received the exact same contents in the recruitment pack APART FROM the inclusion of a pen and/or £5 if allocated, and the additional wording in the PIS. 

In the Intervention section, we have elaborated on the incentives, providing detail about why the value of £5 was chosen and a description of the pen. More detail has also been included on how the 'cost per additional participant recruited' was calculated. We have heeded your suggestion to change £5 to 'cash' in some of the text and tables for international audiences, and have moved the pen meta-analysis to the Discussion.

We have made changes to the Discussion as per all your suggestions, and added in the following concluding sentence:

“In conclusion, we did not find evidence that the inclusion of a pen and/or £5 was particularly effective or represented good value for money for improving recruitment into a trial of Gentle Years Yoga for older adults with multimorbidity.”

We greatly appreciate your comments and believe the changes we have made in response to them have really improved the paper. If you have any other suggestions for improvement, we would be happy to receive them.