Keywords
Cluster randomised controlled trial; feasibility study; recruitment; SWAT; social media; testicular neoplasms; men’s health.
This article is included in the Studies Within A Trial (SWAT) collection.
Eight out of ten adults use social media, yet its efficacy in recruitment in clinical trials remains under-explored. The purpose of this SWAT was to determine which recruitment method, X, Facebook or QR code via posters, was more efficient and cost effective for recruiting participants to the host trial.
A cluster randomised cross-over design evaluated three recruitment strategies, X, Facebook and QR code. Seven Gaelic Athletic Association (GAA) clubs were randomised to receive either X, Facebook, or QR code. The seven clubs were re-randomised twice more, two weeks apart. There were two primary outcomes: 1. proportion of participants who consent to participate, relative to the number of players contacted; and 2. proportion of participants who consent to participate, relative to the number of players who clicked the link to register their interest.
Fifty participants were randomised to three recruitment methods, and 47 were retained in the host trial. Participants mainly heard about the study through friends, with some engagement via social media platforms Facebook and X, and little to no engagement with the QR code. Primary outcomes were hindered by the inability to disaggregate data by GAA club. Economic outcomes revealed QR code as the costliest strategy, and while X was cheaper than Facebook in terms of the number of clicks, Facebook demonstrated better recruitment and retention and thus reduced the costs per participant.
While the inability to disaggregate data by club was a limitation, the study revealed that Facebook outperformed X and QR codes in terms of recruitment and participant retention and was thus considered to be more cost effective. The findings emphasise the importance of considering engagement patterns and cost-effectiveness in designing recruitment strategies for clinical trials, especially within the dynamic landscape of social media use.
Cluster randomised controlled trial; feasibility study; recruitment; SWAT; social media; testicular neoplasms; men’s health.
Successful trial recruitment is challenging, with fewer than 50% of trials meeting their targets.1 The evidence available to trialists to support decisions on design, conduct and reporting of randomised trials is sparse, suggesting further trial methodological work is needed.2 PRioRiTy I3 & II4 studies identified the most important unanswered research questions in recruitment and retention, respectively, and one of the top ten recruitment priorities was the use of technology in the recruitment process. Eight out of ten adults use social media, yet its efficacy in recruitment in clinical trials remains under-explored.5 The purpose of this study within a trial (SWAT) was to determine which recruitment method (X, formerly Twitter), Facebook, or quick response [QR] codes via posters) was more efficient and cost effective for recruiting participants to the primary host trial. We also aimed to determine if the recruitment method influenced retention in the host trial. The SWAT protocol is registered in the Northern Ireland SWAT repository as SWAT 162.
We used a cluster randomised cross-over design to evaluate the three recruitment strategies (X, Facebook and QR code). Seven Gaelic Athletic Association [1] (GAA – Irish football and hurling) clubs agreed to participate in the SWAT and were randomised using Research Randomizer (https://www.randomizer.org/) to either Facebook (n = 2), X (n = 3) or QR code (n = 2) for a period of two weeks. After two weeks, the seven clubs were re-randomised to either one of the other recruitment methods, i.e., the two Facebook clubs were then randomised to get either X or QR. After a further two weeks, the GAA clubs changed to the final remaining recruitment method for one further two-week period. Clubs were asked to post the Facebook and X posts twice per week between 12pm and 1pm. We were dependent on the clubs to do the posting when we directed them to but had no control over this occurrence. In relation to the QR code, 10 posters were displayed in each club for the two-week period of their randomisation.
Participants were eligible to participate in the SWAT if they were assigned male at birth, members (i.e., players and coaches) of the target GAA clubs, residing in the Republic of Ireland, and aged 18 to 50 years (age group at risk for testicular diseases).
Two PPI collaborators on the host trial are members of the participating GAA clubs and were involved in the development of the grant application which included the SWAT. They provided background information on player demographics, determining the potential recruitment strategies for the SWAT, and identified the appropriate time in the GAA season to start the recruitment. Both collaborators were compensated for their time and are included as co-authors (AOC and MOR).
1. Proportion of participants who consent to participate in the host trial, relative to the number of players contacted via each recruitment method.
2. Proportion of participants who registered their interest to participate, relative to those consented to the host trial via each social recruitment method.
Title
Enhancing Men’s Awareness of Testicular Diseases (E-MAT): A Feasibility Study (ClinicalTrials.gov Identifier: NCT05146466).
Aim of host trial
To examine the feasibility of conducting a definitive trial to test the effect of E-MATVR (Virtual Reality) (intervention) compared to E-MATE (Electronic information only) (control) on testicular knowledge and testicular self-examination behaviours among male GAA players over three months. We developed a three-level Virtual Reality (VR) educational game called “Enhancing Men’s Awareness of Testicular diseases” or E-MATVR, delivered using a VR headset, handheld controllers, and voiceover.6,7 E-MATVR uses light humour to educate men about testicular diseases and encourages them to examine their testicles and seek medical help for symptoms of concern. The feasibility study compares E-MATVR to a traditional electronic intervention that uses plain text and images via a Tablet or E-MATE. The primary outcomes measured in the feasibility trial were testicular knowledge and testicular self-examination behaviours. The feasibility trial took place in nine geographically distributed GAA sports clubs in County Cork, a southern region in Ireland, over a three-month period. For each of the nine participating GAA clubs, individual participants were randomised to either the intervention or control arm. Risk of contamination was low due to the geographical dispersion of the GAA clubs.
Outcomes were measured at baseline (T0, pre-engagement with E-MATVR or E-MATE), immediately post-test (T1, post engagement with E-MATVR or E-MATE), and three months post-test (T2) using electronic surveys. We expected that players would pass the study link to other players. To capture this, at T0 we asked the participants if they learned about either of the three recruitment methods directly, link through WhatsApp, or if they were referred by a friend.
The GAA club has a record of self-identified males on the club Facebook page, which gives us the total number targeted (denominator). We were dependent on the clubs to make the Facebook posts.
X
On X there is an Engagement Application Programming Interface (API) which measures engagements - a count of the number of time a user has interacted with the X post, how often it has been favourited, liked, reposted, replied, and shared. We were dependent on the clubs to make the X posts.
QR code
The QR code was displayed by means of a professionally designed poster at each club. Ten posters were displayed in each club for the 2-week period of their randomisation. We were able to monitor the number of people who used the QR code.
Micro-costing techniques were used to estimate cost-per-strategy. Costs include direct design and distribution for each strategy; print (received from the company who created the posters and Facebook and X posts), collection costs (travel as per the University’s travel allowance) and personnel costs (valued using national salary scales for staff grade level as per national guidelines8). Trial data were used to identify and measure the quantity of each resource and market values were employed to value them in Euros, for the year 2022.
Exploratory data analysis was conducted, and outliers reconciled. Analysis was conducted on an intention-to-treat basis using the cluster randomisation scheme. Analysis was conducted using R (version 4.0.3; R Project for Statistical Computing, Vienna, Austria) and, in all cases, a 2-sided type I error rate of 0.05 was taken as statistically significant.
Nine GAA clubs were recruited to the host trial. Seven agreed to participate in the SWAT. Fifty participants, from seven GAA clubs, were randomised to the three recruitment methods. Table 1 shows the number of male members in each club, i.e., target population. In total, 47 out of 50 SWAT participants were retained in the host trial.
GAA Club | No. male members >18 years | GAA Club Facebook (Likes) | GAA Facebook Clicks | GAA Club X (Followers) | X Clicks | QR Code posters | QR code poster clicks | No. recruited from each club | No. retained from each club |
---|---|---|---|---|---|---|---|---|---|
1 | 53 | 987 | NA* | 2,804 | NA* | 10 | NA* | 11 | 11 |
2 | 451 | 2,600 | NA* | 3,241 | NA* | 10 | NA* | 15 | 15 |
3 | 166 | 2,092 | NA* | 5,295 | NA* | 10 | NA* | 1 | 0 |
4 | 350 | 4,000 | NA* | 15,800 | NA* | NA* | 7 | 7 | |
5 | 110 | 3,085 | NA* | 4,920 | NA* | 10 | NA* | 6 | 5 |
6 | Unknown | 2,179 | NA* | 3,640 | NA* | 10 | NA* | 7 | 7 |
7 | 50 | 3,859 | NA* | 2,709 | NA* | 10 | NA* | 3 | 2 |
Total | 177^ | 234^ | 13^ | 50 | 47 |
At T0 we asked participants where they heard about the feasibility trial (Table 2). The majority heard about it exclusively from a friend (n = 20). Facebook and X were the next most common. Only one participant heard about the trial through the QR code.
We had two primary outcomes: 1) the proportion of participants who consent to participate, relative to the number of players contacted via each recruitment method; and 2) the proportion of participants who consent to participate, relative to the number of players who clicked the link to register their interest via each recruitment method. We were unable to disaggregate our data by GAA club thus we were unable to present data for the primary outcomes. We worked with a design company and used one Facebook link, one X link and one QR code link for ALL of the GAA clubs.
However, we are able to present some useful data, to assist others who use these recruitment methods in the future. The data presented in Table 3 are for the 16 people who entered the host trial, but not coming exclusively from the SWAT. Though X had the most clicks, it had fewer people who consented to the trial when compared to Facebook. Engagement via the QR code was poor.
X | QR Code | ||
---|---|---|---|
No. Clicks | 234 | 177 | 13 |
No. Consented | 6 | 9 | 1 |
No. Retained | 5 | 7 | 1 |
We were able to identify the peak hours of use for X and Facebook, using the Engagement API which provides access to post impression and engagement metrics. X peaks in use at 12pm, 6pm and 9pm. Facebook peaks in use at 8am and 8pm. In terms of the days of the week that showed most usage, X peaked on Tuesday and Thursday while Facebook peaked on Friday. This may have been the days the X and Facebook posts were made but we do not have that information from each club to clarify this.
We had four secondary outcomes. Of the 50 participants randomised to the SWAT, 47 were retained. As so few were lost to follow-up (n = 3), we could not assess primary retention across the three recruitment methods. The remaining three secondary outcomes were economic outcomes related to cost-per-strategy. 1) unit cost per person who registered their interest by clicking the link/scanning QR code; 2) unit cost per person consented to the host trial; 3) unit cost per person retained at end point of the host trial. We were unable to disaggregate the SWAT information (n = 50) from the host feasibility trial (n = 74). Thus, the information presented below, and in Tables 4 and 5, is for the host trial. We first determined the total costs per recruitment strategy. As can be seen in Table 4, the QR code was the most expensive, while X and Facebook had equivalent total costs. As per Tables 4 and 5, in all cases, the QR code was the most expensive. While X was cheaper than Facebook in terms of the number of clicks, the engagement was better with Facebook resulting in more being consented and retained through Facebook, thereby reducing costs per participant.
X | QR code | ||
---|---|---|---|
Digital design and distribution1 | €50.33 | €50.33 | € 50.33 |
Poster design, printing and distribution2 | - | - | €380.56 |
Communication and follow-up: personnel time 3 | €3,689.97 | €3,689.97 | €3,689.97 |
Total | €3,740.30 | €3,740.30 | €4,120.86 |
The results of our study provide insights into the effectiveness and cost-effectiveness of different recruitment methods in the context of a community sports club setting. Our study involved nine GAA clubs (n = 74), with seven participating in the SWAT (n = 50) who were randomised to three recruitment methods. Most participants heard about the study exclusively from a friend, highlighting the importance of interpersonal networks in disseminating information within the GAA community; replicated in other studies showing the centrality of social networks on and offline in transmitting health care information.9 This is likely a feature of communication in all sports clubs in the community. Facebook and X were also prominent sources of information, with only one participant discovering the study through the poster with QR code. This indicates the limited effectiveness of QR codes as a standalone recruitment method in this context. When compared to other social media recruitment methods Facebook and X, the QR code was also relatively expensive. We do not recommend using it in a trial in the community. X and Facebook set-up costs were equal, but initial engagement, i.e., numbers of clicks, was greater in X, thereby reducing the cost per participant. However, the proportion who consented from the engagement was greater with Facebook, thus Facebook can be considered cost effective compared to X when considering those consented and retained.
1. The inability to disaggregate data by GAA club due to the use of a single Facebook link, one X link and one QR code link for ALL of the GAA clubs is a limitation. Understanding the aggregate is useful, but there may be diversity in recruitment that is under-explored, such as impact on recruitment by individual or group characteristic (e.g., ethnicity10). Creating separate links (Facebook, X, QR code) for each of the GAA clubs would have allowed for a more granular analysis of the recruitment process. This would have provided valuable insights into the effectiveness of each method within individual clubs and could have informed targeted recruitment strategies.
2. Monitoring X and Facebook interactions in a sport club setting is challenging, as it was not possible to disentangle the SWAT participant data from general GAA club data. The reason for this was because we did not have direct administrative access to the club accounts. To conduct recruitment in the way we did, and to be able to evaluate it, researchers would need to be given user/administrator rights to each club’s Facebook and X accounts.
3. In addition to members of the GAA club, there were other individuals that follow the GAA Club’s X and Facebook pages, so relying on club membership to determine the denominator was not meaningful.
4. Many of the participants heard about the study from friends, therefore, we were unable to determine which recruitment method they were recruited through.
5. While we were focused on three recruitment methods, as we can see in Table 1, some people accessed the Facebook and X links via Text Message or WhatsApp, making it difficult to track the reach of the recruitment method.
6. We were overambitious when defining our outcomes and for a small SWAT, there were too many. The ‘keep it simple’ approach – fewer outcomes – would have been better.
7. It is evident that the hours of use for X and Facebook which would help identify suitable times to post information on the study in future trials. For this community of users, X spikes in use at 12pm, 6pm, and 9pm. Facebook peaks in use at 8am and 8pm. In terms of the days of the week that showed most usage, X peaked on Tuesday and Thursday while Facebook peaked on Friday. This may have been the days the X and Facebook posts were made but we do not have the required information from each GAA club to determine that.
In conclusion, we have tentative evidence on the ability to recruit males to healthcare trials which demonstrated better value for money through Facebook, followed by X, with little to no-engagement with QR codes. Engaging with social networks is essential to support recruitment. The limitations of data tracking and customisation to evaluate recruitment methods in research studies are highlighted, especially when dealing with multiple groups or recruitment methods in a pragmatic community setting. We encourage others to evaluate their recruitment methods in SWATs to support or refute our conclusions and support trial engagement. These insights can inform future research, particularly future definitive trials, allowing for more robust recruitment of participants, data collection and analysis, ultimately enhancing the validity and reliability of research findings.
Ethical approval for this SWAT was obtained from the Clinical Research Ethics Committee (CREC), University College Cork in December 2021 (CREC Review Reference Number: ECM 06/2023 PUB). We confirm that we obtained written, informed consent from all participants enrolled in this study.
The data underlying the results are available as part of the article and no additional source data are required, except for social media data and the data from the QR code posters. The social media data came from the Facebook page and X pages of the GAA clubs. This data is in the form of posts, likes, followers and clicks. The QR code posters contained data on clicks only. The social media data cannot be shared due to the ethical and copyright restrictions surrounding social media data. The Methods section contains detailed information to allow replication of the study. Also, the study is small due to its nature and it could potentially identify the GAA clubs if we release it publicly. Any queries about the methodology should be directed to the corresponding author. If someone requests the data or have questions about the methodology, we are happy to discuss this with them.
We wish to express our appreciation to the GAA clubs who facilitated the data collection for this SWAT, and to their members for partaking.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Partly
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Partly
Are all the source data underlying the results available to ensure full reproducibility?
No
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Digital health, addiction, randomised controlled trials, systematic reviews and meta-analyses.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |
---|---|
1 | |
Version 1 28 Mar 24 |
read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)