An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting

Alex S. Mitchell; Liz Cook; Alexandra Dean; Caroline Fairhurst; Matthew Northgraves; David J. Torgerson; Mike Reed

doi:10.12688/f1000research.24244.2

Home Browse An embedded randomised controlled retention trial of personalised...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting

[version 2; peer review: 2 approved]

Alex S. Mitchell ¹, Liz Cook¹, Alexandra Dean¹, [...] Caroline Fairhurst¹, Matthew Northgraves², David J. Torgerson¹, Mike Reed^1,3

Alex S. Mitchell ¹, Liz Cook¹, [...] Alexandra Dean¹, Caroline Fairhurst¹, Matthew Northgraves², David J. Torgerson¹, Mike Reed^1,3

PUBLISHED 15 Sep 2021

Author details Author details

¹ Department of Health Sciences, University of York, UK, York, Y010 5DD, UK
² Hull Health Trials Unit, University of Hull, Hull, HU6 7RU, UK
³ North Tyneside General Hospital, Northumbria Healthcare NHS Foundation Trust, North Shields, Tyne and Wear, NE29 8NH, UK

Alex S. Mitchell
Roles: Data Curation, Formal Analysis, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Liz Cook
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

Alexandra Dean
Roles: Investigation, Project Administration, Resources, Writing – Review & Editing

Caroline Fairhurst
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – Review & Editing

Matthew Northgraves
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

David J. Torgerson
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Supervision, Writing – Review & Editing

Mike Reed
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Studies Within A Trial (SWAT) collection.

Abstract

Background: Several studies have investigated whether personalising trial documentation can aid recruitment and retention. We did a ‘study within a trial’ (SWAT) evaluating the effectiveness of a personalised text message compared to a non-personalised text message, on the retention rate in a large orthopaedic trial.
Methods: The SWAT was embedded in the Knee Replacement Bandaging Study (KReBS) trial. The primary outcome was the proportion of 12-month questionnaires returned. Secondary outcomes were the proportion of questionnaires completed and time to questionnaire return. Binary data were analysed using logistic regression and time to return using Cox proportional hazards regression. Odds ratios (OR) and hazard ratios (HR) are presented, with associated 95% confidence intervals (CI) and p-values.
Results: In total, 1465 participants were included in the SWAT. In the personalised group, 644/723 (89.1%) of participants returned a questionnaire, compared to 654/742 (88.1%) in the non-personalised group. The absolute difference in return rate was 0.9% (95% CI: -2.3% to 4.2%; p=0.57). There was no evidence of a difference between the groups in the likelihood of returning a questionnaire (OR 1.09; 95% CI: 0.79 to 1.51; p=0.61), the likelihood of returning a complete questionnaire (OR 1.11; 95% CI: 0.82 to 1.51; p=0.50) nor in time to return (HR 1.05; 95% CI: 0.94 to 1.17; p=0.40).
Conclusion: This SWAT adds to the growing evidence base for whether personalised text messages are effective.

Registration: ISRCTN87127065 (20/02/2017); SWAT 35 (01/12/2015)

Keywords

SWAT, Study Within A Trial, attrition, SMS, text messages

Corresponding author: Alex S. Mitchell

Competing interests: No competing interests were disclosed.

Grant information: The KReBS trial was funded by 3M, while the SWAT was supported by the PROMETHEUS programme funded by the Medical Research Council [MR/R013748/1].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2021 Mitchell AS et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Mitchell AS, Cook L, Dean A et al. An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting [version 2; peer review: 2 approved]. F1000Research 2021, 9:591 (https://doi.org/10.12688/f1000research.24244.2) First published: 11 Jun 2020, 9:591 (https://doi.org/10.12688/f1000research.24244.1) Latest published: 15 Sep 2021, 9:591 (https://doi.org/10.12688/f1000research.24244.2)

Revised Amendments from Version 1

We have added additional detail to the article in response to the issues raised by the reviewer. We have outlined how the validity of the phone numbers was checked at two different stages, and also provided detail on the pre-planned retention strategies.

We have also made a minor amendment to the results in light of a duplicate randomisation that was found in the host trial, which means that 2334 rather than 2335 participants were randomised to the host trial. This duplicate randomisation was not randomised to the SWAT and therefore has no other impact on the results. We have updated the figure to take this change into account.

See the authors' detailed response to the review by Michelle E Kho

Introduction

Clinical trialists have identified the recruitment and retention of participants as key issues for randomised controlled trials (RCT)^1,2.

Several studies have investigated whether personalising trial documentation can aid recruitment and retention^3,4. Recently, Cochrane et al. looked at the effect of personalised text messages compared to standard text messages in improving retention rates⁵. This study was carried out in response to a number of embedded trials evaluating the effectiveness of SMS messages in improving retention rates^6–11, alongside a study suggesting personalised messages increased the payment of delinquent fines¹².

To further add to the evidence on the effectiveness of personalised text messages, we did a ‘study within a trial’ (SWAT) evaluating the effectiveness of a personalised text message compared to a non-personalised text message on postal questionnaire response rates in a large orthopaedic trial.

Methods

Design

This paper details the methods and results of a SWAT embedded within the prospectively registered Knee Replacement Bandaging Study (KReBS) RCT (ISRCTN87127065, registered on 20 February 2017). KReBS evaluated the effectiveness of a two-layer compression bandage compared with a standard wool and crepe bandage applied post-operatively on patient-reported outcomes in total knee replacement patients¹³.

Participants

The SWAT was conducted in 26 NHS hospital trust sites and was implemented at the start of the study. All KReBS participants were eligible for this SWAT provided they had opted in to receiving SMS messages and were not deceased or withdrawn from follow-up before being due to be sent their 12-month postal questionnaire.

Intervention

Participants in the SWAT were sent either a personalised or non-personalised text message (Table 1) four days after their 12-month questionnaire was sent.

Table 1. Description of the contents of the personalised and non-personalised text messages.

Text message type	Text message content
Personalised	“KReBS Trial: [Title] [Surname] you should have received a questionnaire in the post by now. Your answers are important; so please help by returning it as soon as you can. Thanks”
Non-personalised	“KReBS Trial: You should have received a questionnaire in the post by now. Your answers are important; so please help by returning it as soon as you can. Thanks”

KReBS - Knee Replacement Bandaging Study

The validity of the phone number given by participants was checked at two different stages. At the first stage, after randomisation, invalid numbers not beginning with “07”, which are the first two digits of all UK mobile phone numbers were removed from the database, participants providing numbers such as these were not sent a text message. The second stage occurred at the point of the text messages being sent out. Some participants provided phone numbers that began with “07”, but were actually invalid and therefore not picked up at the first stage. An attempt to send a text message to the phone number would have resulted in a failure in the message being delivered.

All SWAT participants received pre-planned retention strategies within KReBS. This consisted of a reminder letter and additional copy of the questionnaire if the participant had not returned a completed copy 4 weeks after sending out the original copy. If there was still no response following the postal reminder, participants were contacted by telephone to obtain the patient-reported outcomes.

Outcomes

The primary outcome was the proportion of participants who returned a 12-month questionnaire. Secondary outcomes were the proportion of participants who completed the questionnaire and time to questionnaire return. A questionnaire was considered complete if the participant had answered 11 or more questions of the 12-item host trial primary outcome, the Oxford Knee Score¹⁴.

Sample size

Since this was an embedded trial, the sample size was determined by the number of participants in the main KReBS trial¹³, which aimed to recruit 2600 participants.

Randomisation

Participants were randomised into the SWAT using simple randomisation in a 1:1 allocation ratio. The allocation schedule was generated by a researcher at the York Trials Unit not involved in the recruitment or follow-up of participants.

Blinding

Participants were not informed of their explicit participation in the SWAT, but due to the nature of the intervention could not be blinded to whether the text was personalised or non-personalised. Similarly, it was not possible to blind research staff to SWAT allocation.

Approvals

The SWAT was approved by the Research Ethics Committee North East – Newcastle and North Tyneside on 13/04/2018 (REC Number 16/NE/0400; Amendment Number 16/NE/0400/AM14). As the SWAT was deemed to be low risk, explicit informed consent was not obtained for participation.

Statistical analysis

Analyses were carried out using Stata v16.0¹⁵. A diagram detailing the flow of participants through the SWAT is provided, and baseline characteristics are presented by SWAT allocation. Outcomes are summarised descriptively. Statistical tests were two-sided using a 5% significance level, and were done on an intention to treat basis. All analyses (except the calculation of the absolute difference in return rate which was estimated using the two-sample test of proportions) used mixed effects regression, adjusting for SWAT allocation and host trial allocation as fixed effects and trial site as a random effect. Relevant parameter estimates are presented with associated 95% confidence intervals and p-values.

The proportion of participants who returned a 12-month questionnaire, and proportion complete, was analysed using logistic regression. A second SWAT evaluating receipt of a pen on response rates was also embedded in KReBS at 12 months¹⁶. In a sensitivity analysis, we additionally adjusted the primary model for pen SWAT allocation.

Time to questionnaire return was analysed using a Cox proportional hazards shared frailty model. Participants who did not return a questionnaire were censored at 90 days.

Results

In total, 2334 participants were recruited into the KReBS trial and 1470 were randomised to the SWAT (Figure 1). The average age was 66.8 years and 54.0% were female (Table 2¹⁷). Five participants died or withdrew following randomisation and as a result 723 participants in the personalised group, and 742 in the non-personalised group, were sent a 12-month questionnaire and were included in the analysis. Of these, 680 (94.1%) of the 723 participants in the personalised group, and 701 (94.5%) of the 742 in the non-personalised group, were sent a text.

Figure 1. Study flow diagram.

Table 2. Baseline characteristics of the study within a trial (SWAT) participants.

	Personalised (n=726)	Non-personalised (n=744)	Total (n=1470)
Gender, n (%) Male Female Missing	335 (46.1) 391 (53.9) 0 (0)	340 (45.7) 403 (54.2) 1 (0.1)	675 (45.9) 794 (54.0) 1 (0.1)
Age n (%) Mean (SD) Median (IQR)	726 (100) 66.9 (8.5) 67.2 (60.7, 72.9)	743 (99.9) 66.8 (8.5) 67.0 (60.8, 72.4)	1469 (99.9) 66.8 (8.5) 67.1 (60.8, 72.7)
Oxford Knee Score n (%) Mean (SD) Median (IQR)	576 (79.3) 20.4 (8.0) 20 (14, 26)	582 (78.2) 20.5 (8.0) 20 (15, 26)	1158 (78.8) 20.4 (8.0) 20 (15, 26)

In the personalised group, 644/723 (89.1%) participants returned a questionnaire, compared to 654/742 (88.1%) in the non-personalised group (Table 3¹⁷). The absolute difference in return rate was 0.9% (95% CI: -2.3% to 4.2%; p=0.57). There was no evidence of a difference between the groups in the likelihood of returning a questionnaire (OR 1.09; 95% CI: 0.79 to 1.51; p=0.61), the likelihood of returning a complete questionnaire (OR 1.11; 95% CI: 0.82 to 1.51; p=0.50) nor in time to return (HR 1.05; 95% CI: 0.94 to 1.17; p=0.40). In total, 1465 participants were also randomised to the pen SWAT. When the primary model was repeated with the addition of pen SWAT allocation, the results remained the same.

Table 3. Descriptive summaries of primary and secondary outcomes.

	Personalised (n=723)	Non-personalised (n=742)	Total (n=1465)
Returned questionnaire, n (%) Yes No	644 (89.1) 79 (10.9)	654 (88.1) 88 (11.9)	1298 (88.6) 167 (11.4)
Completed questionnaire, n (%) Yes No	634 (87.7) 89 (12.3)	641 (86.4) 101 (13.6)	1275 (87.0) 190 (13.0)
Time to return, days n (%) Mean (SD) Median (IQR)	644 (100) 15.9 (15.0) 10 (8, 16)	654 (100) 17.0 (20.4) 10 (8, 16)	1298 (100) 16.5 (17.9) 10 (8, 16)

Discussion

This embedded trial found little evidence to suggest personalised text messages are more effective than non-personalised text messages in encouraging return and completion of questionnaires. The trial did not find evidence of a statistically significant difference between groups in any of the outcomes, although effect size estimates favoured the personalised group. On the other hand, while Cochrane and colleagues also did not find evidence of a statistically significant difference between groups, estimates of effect mostly favoured the non-personalised group⁵.

The SWAT had a large sample size, which means the results can be generalised to other orthopaedic studies. However, completion rate was calculated as a proportion of all SWAT participants rather than all SWAT participants who returned a questionnaire, and as a result questionnaire completion was highly correlated with questionnaire return. In addition, some participants included in the analysis did not receive a text message.

Conclusion

This SWAT adds to the growing evidence base for whether personalised trial documentation, in particular text messages, are effective.

Data availability

Underlying data

Open Science Framework: Underlying data and CONSORT diagram for an embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting. https://doi.org/10.17605/OSF.IO/KHJ8E¹⁷

This project contains the following underlying data:

- KReBS_Text_SWAT_Clean.sas (Study data in SAS compatible format)
- KReBS_Text_SWAT_Clean.csv (Study data in .csv format)
- KReBS_Text_SWAT_Clean_Key.xlsx (Key for datasets)

Reporting guidelines

Open Science Framework: CONSORT checklist for ‘An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting’ https://doi.org/10.17605/OSF.IO/KHJ8E¹⁷

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Faculty Opinions recommended

References

1. Bower P, Brueton V, Gamble C, et al.: Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials. 2014; 15: 399. PubMed Abstract | Publisher Full Text | Free Full Text
2. Hewitt CE, Kumaravel B, Dumville JC, et al.: Assessing the impact of attrition in randomized controlled trials. J Clin Epidemiol. 2010; 63(11): 1264–1270. PubMed Abstract | Publisher Full Text
3. Edwards PJ, Roberts I, Clarke MJ, et al.: Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009; (3): MR000008. PubMed Abstract | Publisher Full Text
4. McCaffery J, Mitchell A, Fairhurst C, et al.: Does handwriting the name of a potential trial participant on an invitation letter improve recruitment rates? A randomised controlled study within a trial [version 1; peer review: 2 approved]. F1000Res. 2019; 8: 659. PubMed Abstract | Publisher Full Text | Free Full Text
5. Cochrane A, Welch C, Fairhurst C, et al.: An evaluation of a personalised text message reminder compared to a standard text message on postal questionnaire response rates: an embedded randomised controlled trial [version 1; peer review: 2 approved]. F1000Res. 2020; 9: 154. PubMed Abstract | Publisher Full Text | Free Full Text
6. Ashby R, Turner G, Cross B, et al.: A randomized trial of electronic reminders showed a reduction in the time to respond to postal questionnaires. J Clin Epidemiol. 2011; 64(2): 208–212. PubMed Abstract | Publisher Full Text
7. Brabyn S, Adamson J, MacPherson H, et al.: Short message service text messaging was feasible as a tool for data collection in a trial of treatment for irritable bowel syndrome. J Clin Epidemiol. 2014; 67(9): 993–1000. PubMed Abstract | Publisher Full Text
8. Clark L, Ronaldson S, Dyson L, et al.: Electronic prompts significantly increase response rates to postal questionnaires: a randomized trial within a randomized trial and meta-analysis. J Clin Epidemiol. 2015; 68(12): 1446–1450. PubMed Abstract | Publisher Full Text
9. Keding A, Brabyn S, MacPherson H, et al.: Text message reminders to improve questionnaire response rates. J Clin Epidemiol. 2016; 79: 90–95. PubMed Abstract | Publisher Full Text
10. Man MS, Tilbrook HE, Jayakody S, et al.: Electronic reminders did not improve postal questionnaire response rates or response times: a randomized controlled trial. J Clin Epidemiol. 2011; 64(9): 1001–1004. PubMed Abstract | Publisher Full Text
11. Richmond SJ, Keding A, Hover M, et al.: Feasibility, acceptability and validity of SMS text messaging for measuring change in depression during a randomised controlled trial. BMC Psychiatry. 2015; 15: 68. PubMed Abstract | Publisher Full Text | Free Full Text
12. Haynes LC, Green DP, Gallagher R, et al.: Collection of delinquent fines: An adaptive randomized trial to assess the effectiveness of alternative text messages. J Policy Anal Manag. 2013; 32(4): 718–730. Publisher Full Text
13. Cook L, Northgraves MJ, Fairhurst C, et al.: Knee Replacement Bandaging Study (KReBS) evaluating the effect of a two-layer compression bandage system on knee function following total knee arthroplasty: study protocol for a randomised controlled trial. Trials. 2019; 20(1): 261. PubMed Abstract | Publisher Full Text | Free Full Text
14. Dawson J, Fitzpatrick R, Carr A, et al.: Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br. 1996; 78(2): 185–190. PubMed Abstract | Publisher Full Text
15. StataCorp: Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC, 2019.
16. Mitchell AS, Cook L, Dean A, et al.: Using pens as an incentive for questionnaire return in an orthopaedic trial: an embedded randomised controlled retention trial [version 1; peer review: 1 approved with reservations]. F1000Res. 2020; 9: 321. Publisher Full Text
17. Mitchell A, Cook L, Dean A, et al.: Underlying data and CONSORT diagram for an embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting. 2020. http://www.doi.org/10.17605/OSF.IO/KHJ8E

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 11 Jun 2020

Author details Author details

Alex S. Mitchell
Roles: Data Curation, Formal Analysis, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Liz Cook
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

Alexandra Dean
Roles: Investigation, Project Administration, Resources, Writing – Review & Editing

Caroline Fairhurst
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – Review & Editing

Matthew Northgraves
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

David J. Torgerson
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Supervision, Writing – Review & Editing

Mike Reed
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The KReBS trial was funded by 3M, while the SWAT was supported by the PROMETHEUS programme funded by the Medical Research Council [MR/R013748/1].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 15 Sep 2021, 9:591

https://doi.org/10.12688/f1000research.24244.2

version 1

Published: 11 Jun 2020, 9:591

https://doi.org/10.12688/f1000research.24244.1

© 2021 Mitchell AS et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Mitchell AS, Cook L, Dean A et al. An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting [version 2; peer review: 2 approved]. F1000Research 2021, 9:591 (https://doi.org/10.12688/f1000research.24244.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 15 Sep 2021

Revised

Views

Reviewer Report 10 May 2022

David J Keene, University of Oxford, Oxford, UK

Approved

https://doi.org/10.5256/f1000research.77319.r128647

Thank you for the invitation to review this interesting SWAT report. I note the other peer reviewers comments and responses. Overall this concise summary documents a well-designed and conducted SWAT. The authors are to be commended given the delivery was within a large and complex clinical trial.

My recommendations to improve the manuscript are as follows:

Abstract – the context of the intervention, a ‘personalised text message’, was not clear in isolation in the abstract, i.e. it was sent 4 day after sending a 12-month postal questionnaire to prompt completion.

Methods

The details of the KReBS trial are signposted but it would be helpful to have more background for the reader e.g. country, dates of conduct, main eligibility criteria for the main trial.

For the intervention, stating the aim of the personalised message would aid clarity, it appears to be a prompt rather than a reminder but would be good to be explicit about this.

Randomisation – how was the allocation schedule generated, computer programme?

The statistical approaches seem reasonable but this is not my area of expertise.

Discussion

Some interpretation of the results in the context of other trials that have used personalised vs non-personalised text messages for first time postal questionnaire completion would be informative. Other SWATs have looked at personalisation as a reminder SMS after no postal return rather than a proactive prompt. Differences in host trials may also be important, there were good response levels in this host trial overall.

The 95% CI includes a small negative to positive effect on response rate (-2.3% to 4.2%). Would the authors see value of further SWATs to enable meta-analysis and improve precision in the estimates? With a low cost and simple intervention such as this, a marginal difference could be considered valuable.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Trauma, Orthopaedic and Rehabilitation Clinical Trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 11 Jun 2020

Views

Reviewer Report 03 Jul 2020

Michelle E Kho, School of Rehabilitation Science, McMaster University, Hamilton, ON, Canada

Approved

https://doi.org/10.5256/f1000research.26744.r64613

Mitchell et al. elegantly describe a SWAT of personalized vs. non-personalized reminder text messages embedded in a randomized orthopaedic trial. The primary outcome was questionnaire return at 12 months, and the secondary outcomes were proportion of completed questionnaires, and time ... Continue reading

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires¹. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?
The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?
In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, et al.: Methods to increase response to postal and electronic questionnaires.Cochrane Database Syst Rev. 2009. MR000008 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Research methodology, trial design and reporting

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 15 Sep 2021

Alex Mitchell, Department of Health Sciences, University of York, UK, York, Y010 5DD, UK

15 Sep 2021

Author Response

We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment ... Continue reading We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment to the results of the paper, as since publication a duplicate randomisation was found, and therefore 2334 participants were randomised to the host trial, not 2335. This does not impact the results of the SWAT, as the duplicate randomisation was not given an allocation in the SWAT.

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires1. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?

Thank you, we have added text under the Intervention section of the paper, describing the strategies used in the host trial to optimise questionnaire return.

The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?

In total, 864 participants were not randomised to the SWAT. This number consists of the 806 who did not opt-in to receive SMS, and the 58 who had either withdrawn, did not provide a valid mobile phone number or were missed. Of these 864, 827 had not withdrawn by the 12-month time point and were sent a questionnaire. 704 returned the questionnaire, which corresponds to a response rate of 85.1%. This is slightly lower than the response rate observed in the SWAT, which may be due to participants who didn’t opt-in to receive SMS or didn’t provide a valid mobile phone number being less engaged with the study. Due to this being a post-hoc analysis, we have decided not to include this result in the publication.

In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

We have added text describing the two stages of mobile phone number verification in the Intervention section of the paper.
We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment to the results of the paper, as since publication a duplicate randomisation was found, and therefore 2334 participants were randomised to the host trial, not 2335. This does not impact the results of the SWAT, as the duplicate randomisation was not given an allocation in the SWAT.

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires1. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?

Thank you, we have added text under the Intervention section of the paper, describing the strategies used in the host trial to optimise questionnaire return.

The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?

In total, 864 participants were not randomised to the SWAT. This number consists of the 806 who did not opt-in to receive SMS, and the 58 who had either withdrawn, did not provide a valid mobile phone number or were missed. Of these 864, 827 had not withdrawn by the 12-month time point and were sent a questionnaire. 704 returned the questionnaire, which corresponds to a response rate of 85.1%. This is slightly lower than the response rate observed in the SWAT, which may be due to participants who didn’t opt-in to receive SMS or didn’t provide a valid mobile phone number being less engaged with the study. Due to this being a post-hoc analysis, we have decided not to include this result in the publication.

In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

We have added text describing the two stages of mobile phone number verification in the Intervention section of the paper.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 15 Sep 2021

Alex Mitchell, Department of Health Sciences, University of York, UK, York, Y010 5DD, UK

15 Sep 2021

Author Response

We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment ... Continue reading We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment to the results of the paper, as since publication a duplicate randomisation was found, and therefore 2334 participants were randomised to the host trial, not 2335. This does not impact the results of the SWAT, as the duplicate randomisation was not given an allocation in the SWAT.

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires1. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?

Thank you, we have added text under the Intervention section of the paper, describing the strategies used in the host trial to optimise questionnaire return.

The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?

In total, 864 participants were not randomised to the SWAT. This number consists of the 806 who did not opt-in to receive SMS, and the 58 who had either withdrawn, did not provide a valid mobile phone number or were missed. Of these 864, 827 had not withdrawn by the 12-month time point and were sent a questionnaire. 704 returned the questionnaire, which corresponds to a response rate of 85.1%. This is slightly lower than the response rate observed in the SWAT, which may be due to participants who didn’t opt-in to receive SMS or didn’t provide a valid mobile phone number being less engaged with the study. Due to this being a post-hoc analysis, we have decided not to include this result in the publication.

In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

We have added text describing the two stages of mobile phone number verification in the Intervention section of the paper.
We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment to the results of the paper, as since publication a duplicate randomisation was found, and therefore 2334 participants were randomised to the host trial, not 2335. This does not impact the results of the SWAT, as the duplicate randomisation was not given an allocation in the SWAT.

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires1. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?

Thank you, we have added text under the Intervention section of the paper, describing the strategies used in the host trial to optimise questionnaire return.

The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?

In total, 864 participants were not randomised to the SWAT. This number consists of the 806 who did not opt-in to receive SMS, and the 58 who had either withdrawn, did not provide a valid mobile phone number or were missed. Of these 864, 827 had not withdrawn by the 12-month time point and were sent a questionnaire. 704 returned the questionnaire, which corresponds to a response rate of 85.1%. This is slightly lower than the response rate observed in the SWAT, which may be due to participants who didn’t opt-in to receive SMS or didn’t provide a valid mobile phone number being less engaged with the study. Due to this being a post-hoc analysis, we have decided not to include this result in the publication.

In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

We have added text describing the two stages of mobile phone number verification in the Intervention section of the paper.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 11 Jun 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 15 Sep 21		read
Version 1 11 Jun 20	read

Michelle E Kho, McMaster University, Hamilton, Canada
David J Keene, University of Oxford, Oxford, UK

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

8 Views

10 May 2022 | for Version 2

David J Keene, University of Oxford, Oxford, UK

8 Views Cite this report Responses(0)

Approved

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Trauma, Orthopaedic and Rehabilitation Clinical Trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

34 Views

03 Jul 2020 | for Version 1

Michelle E Kho, School of Rehabilitation Science, McMaster University, Hamilton, ON, Canada

34 Views Cite this report Responses(1)

Approved

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires¹. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?
The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?
In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Research methodology, trial design and reporting

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

15 Sep 2021

Alex Mitchell, Department of Health Sciences, University of York, UK, York, Y010 5DD, UK

We thank the reviewer for her positive feedback and constructive comments, to which we have responded below. In addition to responding to the comments, we have made a minor amendment to the results of the paper, as since publication a duplicate randomisation was found, and therefore 2334 participants were randomised to the host trial, not 2335. This does not impact the results of the SWAT, as the duplicate randomisation was not given an allocation in the SWAT.

There is a large body of evidence outlining strategies to improve return rates of postal questionnaires1. The primary outcome in this study, questionnaire return at 12 months, could be confounded by the host trial efforts to improve postal questionnaire return. Can the authors report more information on the host trial efforts to optimize postal questionnaire return?

Thank you, we have added text under the Intervention section of the paper, describing the strategies used in the host trial to optimise questionnaire return.

The questionnaire return rates for this subgroup of host trial participants was very high (~89%). Can the authors report the questionnaire return rates for the 809 people in the host trial who were not included in the SWAT, or if this is not available, the overall questionnaire return rate for the host trial?

In total, 864 participants were not randomised to the SWAT. This number consists of the 806 who did not opt-in to receive SMS, and the 58 who had either withdrawn, did not provide a valid mobile phone number or were missed. Of these 864, 827 had not withdrawn by the 12-month time point and were sent a questionnaire. 704 returned the questionnaire, which corresponds to a response rate of 85.1%. This is slightly lower than the response rate observed in the SWAT, which may be due to participants who didn’t opt-in to receive SMS or didn’t provide a valid mobile phone number being less engaged with the study. Due to this being a post-hoc analysis, we have decided not to include this result in the publication.

In Figure 1, can you please clarify the difference between those excluded in the first box for “did not provide a valid mobile phone number” and those who were sent a questionnaire and “did not provide a valid phone number”?

We have added text describing the two stages of mobile phone number verification in the Intervention section of the paper.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Bower P, Brueton V, Gamble C, et al.: Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials. 2014; 15: 399. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Hewitt CE, Kumaravel B, Dumville JC, et al.: Assessing the impact of attrition in randomized controlled trials. J Clin Epidemiol. 2010; 63(11): 1264–1270. PubMed Abstract | Publisher Full Text

[3] 3. Edwards PJ, Roberts I, Clarke MJ, et al.: Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009; (3): MR000008. PubMed Abstract | Publisher Full Text

[4] 4. McCaffery J, Mitchell A, Fairhurst C, et al.: Does handwriting the name of a potential trial participant on an invitation letter improve recruitment rates? A randomised controlled study within a trial [version 1; peer review: 2 approved]. F1000Res. 2019; 8: 659. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Cochrane A, Welch C, Fairhurst C, et al.: An evaluation of a personalised text message reminder compared to a standard text message on postal questionnaire response rates: an embedded randomised controlled trial [version 1; peer review: 2 approved]. F1000Res. 2020; 9: 154. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Ashby R, Turner G, Cross B, et al.: A randomized trial of electronic reminders showed a reduction in the time to respond to postal questionnaires. J Clin Epidemiol. 2011; 64(2): 208–212. PubMed Abstract | Publisher Full Text

[7] 7. Brabyn S, Adamson J, MacPherson H, et al.: Short message service text messaging was feasible as a tool for data collection in a trial of treatment for irritable bowel syndrome. J Clin Epidemiol. 2014; 67(9): 993–1000. PubMed Abstract | Publisher Full Text

[8] 8. Clark L, Ronaldson S, Dyson L, et al.: Electronic prompts significantly increase response rates to postal questionnaires: a randomized trial within a randomized trial and meta-analysis. J Clin Epidemiol. 2015; 68(12): 1446–1450. PubMed Abstract | Publisher Full Text

[9] 9. Keding A, Brabyn S, MacPherson H, et al.: Text message reminders to improve questionnaire response rates. J Clin Epidemiol. 2016; 79: 90–95. PubMed Abstract | Publisher Full Text

[10] 10. Man MS, Tilbrook HE, Jayakody S, et al.: Electronic reminders did not improve postal questionnaire response rates or response times: a randomized controlled trial. J Clin Epidemiol. 2011; 64(9): 1001–1004. PubMed Abstract | Publisher Full Text

[11] 11. Richmond SJ, Keding A, Hover M, et al.: Feasibility, acceptability and validity of SMS text messaging for measuring change in depression during a randomised controlled trial. BMC Psychiatry. 2015; 15: 68. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Haynes LC, Green DP, Gallagher R, et al.: Collection of delinquent fines: An adaptive randomized trial to assess the effectiveness of alternative text messages. J Policy Anal Manag. 2013; 32(4): 718–730. Publisher Full Text

[13] 13. Cook L, Northgraves MJ, Fairhurst C, et al.: Knee Replacement Bandaging Study (KReBS) evaluating the effect of a two-layer compression bandage system on knee function following total knee arthroplasty: study protocol for a randomised controlled trial. Trials. 2019; 20(1): 261. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Dawson J, Fitzpatrick R, Carr A, et al.: Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br. 1996; 78(2): 185–190. PubMed Abstract | Publisher Full Text

[15] 15. StataCorp: Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC, 2019.

[16] 16. Mitchell AS, Cook L, Dean A, et al.: Using pens as an incentive for questionnaire return in an orthopaedic trial: an embedded randomised controlled retention trial [version 1; peer review: 1 approved with reservations]. F1000Res. 2020; 9: 321. Publisher Full Text

[17] 17. Mitchell A, Cook L, Dean A, et al.: Underlying data and CONSORT diagram for an embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting. 2020. http://www.doi.org/10.17605/OSF.IO/KHJ8E

An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Design

Participants

Intervention

Table 1. Description of the contents of the personalised and non-personalised text messages.

Outcomes

Sample size

Randomisation

Blinding

Approvals

Statistical analysis

Results

Figure 1. Study flow diagram.

Table 2. Baseline characteristics of the study within a trial (SWAT) participants.

Table 3. Descriptive summaries of primary and secondary outcomes.

Discussion

Conclusion

Data availability

Underlying data

Reporting guidelines

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated