Detection of illicit online sales of fentanyls via Twitter

A counterfeit fentanyl crisis is currently underway in the United States. Counterfeit versions of commonly abused prescription drugs laced with fentanyl are being manufactured, distributed, and sold globally, leading to an increase in overdose and death in countries like the United States and Canada. Despite concerns from the U.S. Drug Enforcement Agency regarding covert and overt sale of fentanyls online, no study has examined the role of the Internet and social media on fentanyl illegal marketing and direct-to-consumer access. In response, this study collected and analyzed five months of Twitter data (from June-November 2015) filtered for the keyword “fentanyl” using Amazon Web Services. We then analyzed 28,711 fentanyl-related tweets using text filtering and a machine learning approach called a Biterm Topic Model (BTM) to detect underlying latent patterns or “topics” present in the corpus of tweets. Using this approach we detected a subset of 771 tweets marketing the sale of fentanyls online and then filtered this down to nine unique tweets containing hyperlinks to external websites. Six hyperlinks were associated with online fentanyl classified ads, 2 with illicit online pharmacies, and 1 could not be classified due to traffic redirection. Importantly, the one illicit online pharmacy detected was still accessible and offered the sale of fentanyls and other controlled substances direct-to-consumers with no prescription required at the time of publication of this study. Overall, we detected a relatively small sample of Tweets promoting illegal online sale of fentanyls. However, the detection of even a few online sellers represents a public health danger and a direct violation of law that demands further study.

TM is a non-compensated member of the academic advisory panel of the Alliance for Safe Online Pharmacies (ASOP), a Competing interests: 501(c)(4) social welfare organization engaged on the issue of illicit online pharmacies. There was no involvement of anyone other than the authors in the conception, design, collection, planning, conduct, analysis, interpretation, writing, and discussion to submit this work. Authors report no other financial relationships with any organizations that might have an interest in the submitted work. Creative Commons Attribution , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Licence TM received funding for the data collection phase of this study from an ASOP pilot research grant exploring prescription drug Grant information: abuse risks online. Though this pilot research grant supported the overall process of capturing data, the aims of the grant were not related to this study.

Introduction
A fentanyl crisis is currently underway in the United States, characterized by an influx of counterfeit fentanyl-laced prescription drugs (e.g. Xanax ® , Norco ® , OxyContin ® , and Oxycodone) now being advertised, sold and consumed by the public 1 . The result of this counterfeit infiltration into the U.S. drug supply chain has been an alarming increase in fentanyl-related overdose, deaths and seizures, due to illicitly produced products containing undeclared fentanyls and related fentanyl analogues/variants (e.g. acetyl fentanyl, butyrfentanyl, and furanylfentanyl.) [1][2][3] Fueling this public health emergency is a global network of fentanyl producers and traffickers including countries such as China (where powdered fentanyls and other synthetic opiates are sold with pill presses or sold as precursors used to manufacture fentanyl), Mexico (where fentanyls are manufactured in clandestine laboratories and smuggled into the USA) and Canada (where fentanyls are also manufactured and sold locally) 1 . Unfortunately, criminals see an opportunity to sell cheap and deadly versions of fentanyl-laced pills to satiate demand driven by a national prescription opioid and heroin epidemic, with the U.S. Drug Enforcement Agency (DEA) estimating that a kilogram of fentanyl could generate $5-20 million in retail counterfeit sales 1 .
Other studies have explored the public health consequences of counterfeit fentanyl and have argued for better surveillance and harm reduction approaches to curb consumption and demand 2,3 . However, the potential impact and easy accessibility of fentanyls sold direct-to-consumer via the Internet and social media has not been examined. Other studies have identified illegal marketing and direct sales of other prescription controlled substances by illicit online pharmacies and the use of popular social media sites to promote these services 4-10 .
No study has assessed the role of Internet in fentanyl product promotion, despite concerns from the DEA about documented covert and overt sale of fentanyls online 1 . To better characterize these risks, we conducted a social media surveillance study using big data approaches to identify fentanyl promotion and sale via the popular microblogging platform Twitter.

Methods
Our data collection was part of a larger study exploring user behavior characteristics of illicit prescription drug abuse mediated by Twitter, with tweets collected over a five-month period (June-November 2015) filtered for the prescription drug abuserelated keywords, including the term "fentanyl." 5,6,10 Data collection and analysis was carried out using a methodology combining Amazon Web Services (AWS) cloud computing virtual instances for data collection and machine learning algorithms to analyze tweets using both assisted and unassisted protocols used in previously published studies 5,6,10 .
Specifically, we used a Biterm Topic Model (BTM) that can identify underlying latent patterns or "topics" present in a large corpus of tweets 6,11 . Given the input corpus of text, and a predefined number k, the model outputs a set of k latent topics present in the corpus. Each topic represents an underlying pattern associated with fentanyl twitter conversations that were then isolated and coded using human annotation.
The number of topics to be detected was set to 40 and the alpha and beta parameters of BTM were set to 1 and 0.01 respectively. The output of the BTM was set to display the top 10 words with the highest weight for each topic. Each topic was manually reviewed based on these words and classified into "news" and "online pharmacy" themes (results discussed below). We also used a keyword filtering process using common terms included in the text of tweets associated with illegal online drug promotion, also known as "selling arguments," (e.g. "buy", "discount", "price") in conjunction with the BTM outputs. Code used in this study is available from https://github.com/kjanani/ health_topicmodeling.

Results
Our study collected 28,711 fentanyl-related tweets during 2015, a period when the fentanyl crisis was escalating 6,11 . The majority of topics detected in the whole corpus of fentanyl-related tweets were news-related (97.3%, n=27,940), detailing counterfeit fentanyl dangers reported in national and local media outlets.
After isolating Tweets related to news reports, we used a keyword filtering process in conjunction with machine learning to detect the subset of tweets associated with illegal online drug promotion. Using this approach, we detected 771 (<1% of total) tweets promoting the marketing and sale of fentanyls and other controlled substances online. These tweets were then manually annotated by the authors (inter-coder reliability kappa=0.98) to assess if they included a hyperlink enabling direct-to-consumer sale and purchase of illegal fentanyls. Nine unique tweets (not duplicates or retweets) and their associated hyperlinks were then identified for further analysis.
When examining website content of hyperlinks, 6 were associated with online classified ads, 2 with illicit online pharmacies, and 1 could not be classified due to traffic redirection. These results indicate some interesting trends. First, it appears that individual drug dealers use online classifieds ads to digitally advertise "street buys" of controlled substances (Example A, Figure 1). Additionally, one illicit online pharmacy that we detected and which is currently was still accessible at the time of initial publication of this study. offers the sale of fentanyls and other controlled substances direct-to-consumers with no prescription required (Example B). Its website purportedly offers Abstral® 800mg (fentanyl brand name) for $3.00 per tablet, and based on further inspection of WHOIS data, has its internet domain registration identity and location masked by an Internet privacy service company.
We note that there are certain limitations to this study. First the study only examined online promotion and availability at a single point-of-time, as websites of this nature are often removed or become inactive. Additionally, because of the illegal nature of the websites identified, we did not purchase fentanyls and test them for authenticity and potency. Buying controlled substances and making payments to an illicit online pharmacy raises serious legal concerns and is generally illegal.

Discussion
Our results indicate that in our entire corpus of fentanyl-related tweets, direct-to-consumer sale of fentanyls occurred infrequently. This is not surprising, given the deadly nature and high potency of fentanyl, and that many victims may not actively seek its purchase 1,2 . However, the presence of even a few online sellers is concerning, as these actions represent a clear violation of Federal law (e.g. Controlled Substances Act and the Ryan Haight Online Pharmacy Consumer Protection Act) and directly endanger the public. Importantly, these sites may only be the tip of the iceberg, as hyperlinks in one result directed us to online user forums (Google Groups) actively selling fentanyl online (Example C).
Our data collection occurred in late 2015, a period arguably at an early stage of the fentanyl crisis epidemiological curve, which may limit the generalizability of the results. Hence, this indicates that more research is needed to accurately characterize the dangers of the online environment in relation to counterfeit fentanyls. This includes longer term follow-up studies examining online promotion and access to fentanyl and interrogating other online platforms (including other social media sites, chat rooms/user forums, the deep web, and private sales facilitated by online and mobile technology).

Data availability
Raw datasets have not been made available per concerns regarding user information and confidentiality of publicly available processed data. This data is stored at the Global Health Policy Institute and is available upon request in a de-identified and aggregated dataset. Please contact the institute's general contact email (ghpolicyinstitute@gmail.com) for further information.
The code for BTM can be found at https://github.com/kjanani/ health_topicmodeling. Competing interests TM is a non-compensated member of the academic advisory panel of the Alliance for Safe Online Pharmacies (ASOP), a 501(c)(4) social welfare organization engaged on the issue of illicit online pharmacies. There was no involvement of anyone other than the authors in the conception, design, collection, planning, conduct, analysis, interpretation, writing, and discussion to submit this work. Authors report no other financial relationships with any organizations that might have an interest in the submitted work.

Grant information
TM received funding for the data collection phase of this study from an ASOP pilot research grant exploring prescription drug abuse risks online. Though this pilot research grant supported the overall process of capturing data, the aims of the grant were not related to this study.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes My main concern is the discontinuity between the introduction and the research conducted. The introduction focused on problems of pharmaceutical purchased which were laced with counterfeit fentanyl. This is true and is a major health hazard as pointed out by the authors. Fentanyls are a deadly pharmaceuticals and again this was emphasized by the authors. However, the paper's research focus is on tweets from people who are interested in purchasing fentanyl illegally. It wasn't on the counterfeit fentanyl laced products. I suggest that the intro be rewritten and I mention the legitimate uses of fentanyl. Please mention that one of fentanyl's major uses is as a pain reliever in hospital surgical suites and must be carefully controlled by medical practitioners. To reiterate, the introduction doesn't match well the research content.
The methodology approach was was appropriate and innovative. I know they provided a link for a full description of the search terms, but in reading it I suggest that they provided at least a partial list of the names used i.e. different spellings of fentanyl, different names for the same product (foreign names). Searching tweets was original, but it didn't produce huge number of tweets. This is worth reporting in . The first thing I thought of was why tweet when you can that few people tweet to purchase fentanyl do a google search and get right to the web sites? This needs to be put in the discussion as to why I am sure there are other methods to find the product. the results were so low.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound? Yes