ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Using X, Facebook QR codes to optimise recruitment to a feasibility trial Enhancing Men’s Awareness of Testicular Diseases (E-MAT) in a cluster randomised Study Within A Trial (SWAT): Lessons learned.

[version 1; peer review: 1 approved with reservations]
PUBLISHED 28 Mar 2024
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

This article is included in the Studies Within A Trial (SWAT) collection.

Abstract

Background

Eight out of ten adults use social media, yet its efficacy in recruitment in clinical trials remains under-explored. The purpose of this SWAT was to determine which recruitment method, X, Facebook or QR code via posters, was more efficient and cost effective for recruiting participants to the host trial.

Methods

A cluster randomised cross-over design evaluated three recruitment strategies, X, Facebook and QR code. Seven Gaelic Athletic Association (GAA) clubs were randomised to receive either X, Facebook, or QR code. The seven clubs were re-randomised twice more, two weeks apart. There were two primary outcomes: 1. proportion of participants who consent to participate, relative to the number of players contacted; and 2. proportion of participants who consent to participate, relative to the number of players who clicked the link to register their interest.

Results

Fifty participants were randomised to three recruitment methods, and 47 were retained in the host trial. Participants mainly heard about the study through friends, with some engagement via social media platforms Facebook and X, and little to no engagement with the QR code. Primary outcomes were hindered by the inability to disaggregate data by GAA club. Economic outcomes revealed QR code as the costliest strategy, and while X was cheaper than Facebook in terms of the number of clicks, Facebook demonstrated better recruitment and retention and thus reduced the costs per participant.

Conclusions

While the inability to disaggregate data by club was a limitation, the study revealed that Facebook outperformed X and QR codes in terms of recruitment and participant retention and was thus considered to be more cost effective. The findings emphasise the importance of considering engagement patterns and cost-effectiveness in designing recruitment strategies for clinical trials, especially within the dynamic landscape of social media use.

Keywords

Cluster randomised controlled trial; feasibility study; recruitment; SWAT; social media; testicular neoplasms; men’s health.

Introduction

Successful trial recruitment is challenging, with fewer than 50% of trials meeting their targets.1 The evidence available to trialists to support decisions on design, conduct and reporting of randomised trials is sparse, suggesting further trial methodological work is needed.2 PRioRiTy I3 & II4 studies identified the most important unanswered research questions in recruitment and retention, respectively, and one of the top ten recruitment priorities was the use of technology in the recruitment process. Eight out of ten adults use social media, yet its efficacy in recruitment in clinical trials remains under-explored.5 The purpose of this study within a trial (SWAT) was to determine which recruitment method (X, formerly Twitter), Facebook, or quick response [QR] codes via posters) was more efficient and cost effective for recruiting participants to the primary host trial. We also aimed to determine if the recruitment method influenced retention in the host trial. The SWAT protocol is registered in the Northern Ireland SWAT repository as SWAT 162.

Methods

Study design and randomisation

We used a cluster randomised cross-over design to evaluate the three recruitment strategies (X, Facebook and QR code). Seven Gaelic Athletic Association [1] (GAA – Irish football and hurling) clubs agreed to participate in the SWAT and were randomised using Research Randomizer (https://www.randomizer.org/) to either Facebook (n = 2), X (n = 3) or QR code (n = 2) for a period of two weeks. After two weeks, the seven clubs were re-randomised to either one of the other recruitment methods, i.e., the two Facebook clubs were then randomised to get either X or QR. After a further two weeks, the GAA clubs changed to the final remaining recruitment method for one further two-week period. Clubs were asked to post the Facebook and X posts twice per week between 12pm and 1pm. We were dependent on the clubs to do the posting when we directed them to but had no control over this occurrence. In relation to the QR code, 10 posters were displayed in each club for the two-week period of their randomisation.

Population

Participants were eligible to participate in the SWAT if they were assigned male at birth, members (i.e., players and coaches) of the target GAA clubs, residing in the Republic of Ireland, and aged 18 to 50 years (age group at risk for testicular diseases).

Patient and Public Involvement (PPI)

Two PPI collaborators on the host trial are members of the participating GAA clubs and were involved in the development of the grant application which included the SWAT. They provided background information on player demographics, determining the potential recruitment strategies for the SWAT, and identified the appropriate time in the GAA season to start the recruitment. Both collaborators were compensated for their time and are included as co-authors (AOC and MOR).

Primary outcomes: Primary recruitment

  • 1. Proportion of participants who consent to participate in the host trial, relative to the number of players contacted via each recruitment method.

  • 2. Proportion of participants who registered their interest to participate, relative to those consented to the host trial via each social recruitment method.

Secondary outcomes: Primary retention and cost-per-strategy

  • 1. Proportion of participants randomised who remain to the conclusion of the study.

  • 2. Unit cost per person who registered their interest by clicking the link/scanning QR code.

  • 3. Unit cost per person consented to the host trial.

  • 4. Unit cost per person retained at end point of the host trial.

Host trial description

Title

Enhancing Men’s Awareness of Testicular Diseases (E-MAT): A Feasibility Study (ClinicalTrials.gov Identifier: NCT05146466).

Aim of host trial

To examine the feasibility of conducting a definitive trial to test the effect of E-MATVR (Virtual Reality) (intervention) compared to E-MATE (Electronic information only) (control) on testicular knowledge and testicular self-examination behaviours among male GAA players over three months. We developed a three-level Virtual Reality (VR) educational game called “Enhancing Men’s Awareness of Testicular diseases” or E-MATVR, delivered using a VR headset, handheld controllers, and voiceover.6,7 E-MATVR uses light humour to educate men about testicular diseases and encourages them to examine their testicles and seek medical help for symptoms of concern. The feasibility study compares E-MATVR to a traditional electronic intervention that uses plain text and images via a Tablet or E-MATE. The primary outcomes measured in the feasibility trial were testicular knowledge and testicular self-examination behaviours. The feasibility trial took place in nine geographically distributed GAA sports clubs in County Cork, a southern region in Ireland, over a three-month period. For each of the nine participating GAA clubs, individual participants were randomised to either the intervention or control arm. Risk of contamination was low due to the geographical dispersion of the GAA clubs.

Data collection

Outcomes were measured at baseline (T0, pre-engagement with E-MATVR or E-MATE), immediately post-test (T1, post engagement with E-MATVR or E-MATE), and three months post-test (T2) using electronic surveys. We expected that players would pass the study link to other players. To capture this, at T0 we asked the participants if they learned about either of the three recruitment methods directly, link through WhatsApp, or if they were referred by a friend.

Facebook

The GAA club has a record of self-identified males on the club Facebook page, which gives us the total number targeted (denominator). We were dependent on the clubs to make the Facebook posts.

X

On X there is an Engagement Application Programming Interface (API) which measures engagements - a count of the number of time a user has interacted with the X post, how often it has been favourited, liked, reposted, replied, and shared. We were dependent on the clubs to make the X posts.

QR code

The QR code was displayed by means of a professionally designed poster at each club. Ten posters were displayed in each club for the 2-week period of their randomisation. We were able to monitor the number of people who used the QR code.

Costing methodology

Micro-costing techniques were used to estimate cost-per-strategy. Costs include direct design and distribution for each strategy; print (received from the company who created the posters and Facebook and X posts), collection costs (travel as per the University’s travel allowance) and personnel costs (valued using national salary scales for staff grade level as per national guidelines8). Trial data were used to identify and measure the quantity of each resource and market values were employed to value them in Euros, for the year 2022.

Data analysis

Exploratory data analysis was conducted, and outliers reconciled. Analysis was conducted on an intention-to-treat basis using the cluster randomisation scheme. Analysis was conducted using R (version 4.0.3; R Project for Statistical Computing, Vienna, Austria) and, in all cases, a 2-sided type I error rate of 0.05 was taken as statistically significant.

Results

Nine GAA clubs were recruited to the host trial. Seven agreed to participate in the SWAT. Fifty participants, from seven GAA clubs, were randomised to the three recruitment methods. Table 1 shows the number of male members in each club, i.e., target population. In total, 47 out of 50 SWAT participants were retained in the host trial.

Table 1. Characteristics of the SWAT GAA clubs.

GAA ClubNo. male members >18 yearsGAA Club Facebook (Likes)GAA Facebook ClicksGAA Club X (Followers)X ClicksQR Code postersQR code poster clicksNo. recruited from each clubNo. retained from each club
153987NA*2,804NA*10NA*1111
24512,600NA*3,241NA*10NA*1515
31662,092NA*5,295NA*10NA*10
43504,000NA*15,800NA*NA*77
51103,085NA*4,920NA*10NA*65
6Unknown2,179NA*3,640NA*10NA*77
7503,859NA*2,709NA*10NA*32
Total177^234^13^5047

* NA = Not available.

^ Unable to ascertain which of the clubs the clicks came from.

At T0 we asked participants where they heard about the feasibility trial (Table 2). The majority heard about it exclusively from a friend (n = 20). Facebook and X were the next most common. Only one participant heard about the trial through the QR code.

Table 2. Where participants heard about the feasibility trial.

Where participants heard about the trialNumber of participants
Friend20
Facebook9
X6
Other4
Facebook link by text message (e.g., WhatsApp)3
X link by text message (e.g., WhatsApp)3
QR code1
Facebook and X1
Facebook, X and Friend1
Facebook and Facebook link by WhatsApp1
Facebook, Facebook link by WhatsApp and X1
Total50

We had two primary outcomes: 1) the proportion of participants who consent to participate, relative to the number of players contacted via each recruitment method; and 2) the proportion of participants who consent to participate, relative to the number of players who clicked the link to register their interest via each recruitment method. We were unable to disaggregate our data by GAA club thus we were unable to present data for the primary outcomes. We worked with a design company and used one Facebook link, one X link and one QR code link for ALL of the GAA clubs.

However, we are able to present some useful data, to assist others who use these recruitment methods in the future. The data presented in Table 3 are for the 16 people who entered the host trial, but not coming exclusively from the SWAT. Though X had the most clicks, it had fewer people who consented to the trial when compared to Facebook. Engagement via the QR code was poor.

Table 3. Numbers of clicks, consented and retained per recruitment method.

XFacebookQR Code
No. Clicks23417713
No. Consented691
No. Retained571

We were able to identify the peak hours of use for X and Facebook, using the Engagement API which provides access to post impression and engagement metrics. X peaks in use at 12pm, 6pm and 9pm. Facebook peaks in use at 8am and 8pm. In terms of the days of the week that showed most usage, X peaked on Tuesday and Thursday while Facebook peaked on Friday. This may have been the days the X and Facebook posts were made but we do not have that information from each club to clarify this.

Secondary outcomes

We had four secondary outcomes. Of the 50 participants randomised to the SWAT, 47 were retained. As so few were lost to follow-up (n = 3), we could not assess primary retention across the three recruitment methods. The remaining three secondary outcomes were economic outcomes related to cost-per-strategy. 1) unit cost per person who registered their interest by clicking the link/scanning QR code; 2) unit cost per person consented to the host trial; 3) unit cost per person retained at end point of the host trial. We were unable to disaggregate the SWAT information (n = 50) from the host feasibility trial (n = 74). Thus, the information presented below, and in Tables 4 and 5, is for the host trial. We first determined the total costs per recruitment strategy. As can be seen in Table 4, the QR code was the most expensive, while X and Facebook had equivalent total costs. As per Tables 4 and 5, in all cases, the QR code was the most expensive. While X was cheaper than Facebook in terms of the number of clicks, the engagement was better with Facebook resulting in more being consented and retained through Facebook, thereby reducing costs per participant.

Table 4. Total cost per strategy (€, 2022).

XFacebookQR code
Digital design and distribution1€50.33€50.33€ 50.33
Poster design, printing and distribution2--€380.56
Communication and follow-up: personnel time 3€3,689.97€3,689.97€3,689.97
Total€3,740.30€3,740.30€4,120.86

1 Design company cost €151 equally split between 3 strategies (Source: Trial documentation).

2 Poster design (€160), printing (€113.65) and delivery (€106.91) (Source: Trial documentation).

3 Research Assistant salary €31,617 pa (Source: Trial documentation).

Table 5. Economic outcomes.

XFacebookQR Code
Outcome 1€ /Clicks15.9821.13316.99
XFacebookQR Code
Outcome 2€ /Consent623.38415.594,120.86
XFacebookQR Code
Outcome 3€ /Retained748.06534.334,120.86

Discussion

The results of our study provide insights into the effectiveness and cost-effectiveness of different recruitment methods in the context of a community sports club setting. Our study involved nine GAA clubs (n = 74), with seven participating in the SWAT (n = 50) who were randomised to three recruitment methods. Most participants heard about the study exclusively from a friend, highlighting the importance of interpersonal networks in disseminating information within the GAA community; replicated in other studies showing the centrality of social networks on and offline in transmitting health care information.9 This is likely a feature of communication in all sports clubs in the community. Facebook and X were also prominent sources of information, with only one participant discovering the study through the poster with QR code. This indicates the limited effectiveness of QR codes as a standalone recruitment method in this context. When compared to other social media recruitment methods Facebook and X, the QR code was also relatively expensive. We do not recommend using it in a trial in the community. X and Facebook set-up costs were equal, but initial engagement, i.e., numbers of clicks, was greater in X, thereby reducing the cost per participant. However, the proportion who consented from the engagement was greater with Facebook, thus Facebook can be considered cost effective compared to X when considering those consented and retained.

Lessons learned

  • 1. The inability to disaggregate data by GAA club due to the use of a single Facebook link, one X link and one QR code link for ALL of the GAA clubs is a limitation. Understanding the aggregate is useful, but there may be diversity in recruitment that is under-explored, such as impact on recruitment by individual or group characteristic (e.g., ethnicity10). Creating separate links (Facebook, X, QR code) for each of the GAA clubs would have allowed for a more granular analysis of the recruitment process. This would have provided valuable insights into the effectiveness of each method within individual clubs and could have informed targeted recruitment strategies.

  • 2. Monitoring X and Facebook interactions in a sport club setting is challenging, as it was not possible to disentangle the SWAT participant data from general GAA club data. The reason for this was because we did not have direct administrative access to the club accounts. To conduct recruitment in the way we did, and to be able to evaluate it, researchers would need to be given user/administrator rights to each club’s Facebook and X accounts.

  • 3. In addition to members of the GAA club, there were other individuals that follow the GAA Club’s X and Facebook pages, so relying on club membership to determine the denominator was not meaningful.

  • 4. Many of the participants heard about the study from friends, therefore, we were unable to determine which recruitment method they were recruited through.

  • 5. While we were focused on three recruitment methods, as we can see in Table 1, some people accessed the Facebook and X links via Text Message or WhatsApp, making it difficult to track the reach of the recruitment method.

  • 6. We were overambitious when defining our outcomes and for a small SWAT, there were too many. The ‘keep it simple’ approach – fewer outcomes – would have been better.

  • 7. It is evident that the hours of use for X and Facebook which would help identify suitable times to post information on the study in future trials. For this community of users, X spikes in use at 12pm, 6pm, and 9pm. Facebook peaks in use at 8am and 8pm. In terms of the days of the week that showed most usage, X peaked on Tuesday and Thursday while Facebook peaked on Friday. This may have been the days the X and Facebook posts were made but we do not have the required information from each GAA club to determine that.

Conclusion

In conclusion, we have tentative evidence on the ability to recruit males to healthcare trials which demonstrated better value for money through Facebook, followed by X, with little to no-engagement with QR codes. Engaging with social networks is essential to support recruitment. The limitations of data tracking and customisation to evaluate recruitment methods in research studies are highlighted, especially when dealing with multiple groups or recruitment methods in a pragmatic community setting. We encourage others to evaluate their recruitment methods in SWATs to support or refute our conclusions and support trial engagement. These insights can inform future research, particularly future definitive trials, allowing for more robust recruitment of participants, data collection and analysis, ultimately enhancing the validity and reliability of research findings.

Ethical considerations

Ethical approval for this SWAT was obtained from the Clinical Research Ethics Committee (CREC), University College Cork in December 2021 (CREC Review Reference Number: ECM 06/2023 PUB). We confirm that we obtained written, informed consent from all participants enrolled in this study.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 28 Mar 2024
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Shiely F, Cooke E, McCarthy M et al. Using X, Facebook QR codes to optimise recruitment to a feasibility trial Enhancing Men’s Awareness of Testicular Diseases (E-MAT) in a cluster randomised Study Within A Trial (SWAT): Lessons learned. [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:235 (https://doi.org/10.12688/f1000research.145823.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 28 Mar 2024
Views
13
Cite
Reviewer Report 19 Jun 2024
Nikolaos Boumparis, Swiss Research Institute for Public Health and Addiction, Zurich, Switzerland 
Approved with Reservations
VIEWS 13
This study investigates the effectiveness and cost-effectiveness of three recruitment methods (X, Facebook and QR codes) for a feasibility trial aimed at increasing male awareness of testicular disease. 

- The manuscript is well written and generally easy ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Boumparis N. Reviewer Report For: Using X, Facebook QR codes to optimise recruitment to a feasibility trial Enhancing Men’s Awareness of Testicular Diseases (E-MAT) in a cluster randomised Study Within A Trial (SWAT): Lessons learned. [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:235 (https://doi.org/10.5256/f1000research.159827.r289144)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 28 Mar 2024
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.