<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.5263.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Research Note</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                    <subj-group>
                        <subject>Multiple Sclerosis &amp; Related Disorders</subject>
                    </subj-group>
                    <subj-group>
                        <subject>Statistical Methodologies &amp; Health Informatics</subject>
                    </subj-group>
                    <subj-group>
                        <subject>Web and Social Media</subject>
                    </subj-group>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Using Twitter to investigate opinions about multiple sclerosis treatments: a descriptive, exploratory study</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Ramagopalan</surname>
                        <given-names>Sreeram V.</given-names>
                    </name>
                    <uri content-type="orcid">https://orcid.org/0000-0002-4766-5160</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Wasiak</surname>
                        <given-names>Radek</given-names>
                    </name>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Cox</surname>
                        <given-names>Andrew P.</given-names>
                    </name>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Evidera, London, W6 8DL, UK</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:sreeram@ramagopalan.net">sreeram@ramagopalan.net</email>
                </corresp>
                <fn fn-type="con">
                    <p>SVR is the guarantor of the study. SVR performed the analysis. APC contributed to the analysis and interpretation of the data. SVR wrote the first draft and all authors contributed to subsequent drafts and the final paper.</p>
                </fn>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>SVR, RW and APC are employees of Evidera.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>10</day>
                <month>9</month>
                <year>2014</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2014</year>
            </pub-date>
            <volume>3</volume>
            <elocation-id>216</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>9</day>
                    <month>9</month>
                    <year>2014</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Ramagopalan SV et al.</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/3-216/pdf"/>
            <abstract>
                <p>
                    <bold>Background:</bold> Multiple sclerosis (MS) is a common complex disorder, with new treatment options emerging each year. Social media is being increasingly used to investigate opinions about drugs, diseases and procedures. In this descriptive exploratory study, we sought to investigate opinions about currently available MS treatments.</p>
                <p>
                    <bold>Methods:</bold> The Twitter resource Topsy was searched for tweets mentioning the following MS treatments: Aubagio, Avonex, Betaferon or Betaseron, Copaxone, Extavia, Gilenya, Lemtrada, Novantrone, Rebif, Tysabri and Tecfidera between 1 Jan 2006 to 31 Jul 2014. Tweets were normalised and sentiment analysis performed.</p>
                <p>
                    <bold>Results:</bold> In total, there were 60037 unique tweets mentioning an MS treatment. About half of the tweets contained non-neutral sentiment. Mean sentiment scores were different for treatments ranging from -0.191to 0.282 when investigating all tweets. These differences in sentiment scores between treatments were statistically significant (P&lt;0.001). Sentiment scores tended to be higher for oral MS treatments than injectable treatments.</p>
                <p>
                    <bold>Conclusions:</bold> Many tweets about MS treatments have a non-neutral sentiment. The analysis of social media appears to be a potential avenue for exploring patient opinion about MS treatments.</p>
            </abstract>
            <funding-group>
                <funding-statement>The author(s) declared that no grants were involved in supporting this work.</funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>The analysis of social media is becoming a powerful tool that is being used increasingly to answer research questions across numerous areas including disease spatio-temporal epidemiology and drug adverse events
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>&#x2013;
                    <xref ref-type="bibr" rid="ref-3">3</xref>
                </sup>. Key stake-holders in the pharmaceutical industry, including patients, physicians, regulatory authorities and pharmaceutical companies, are increasingly using web technologies such as social media, blogs and forums to generate and access opinions and real-world evidence of potentially medically important issues. This content serves as an important source of on-line medical opinions, information and sentiments relating to particular drugs and events. The underlying assumption is that with access to such information, a patient will be able to make more informed decisions about drugs, diseases, procedures and health-care providers.</p>
            <p>Multiple sclerosis (MS) is a chronic, neurodegenerative autoimmune disorder of the central nervous system (CNS). With a prevalence of one per 800 in North America and Northern Europe, MS is the most common acquired neurological disorder in young adults
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup>. About 85% of patients present initially with relapsing-remitting MS (RRMS), characterized by recurrent episodes of neurological dysfunction interspersed with periods of lack of apparent disease activity
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup>.</p>
            <p>At present, there are nine disease modifying therapies (DMTs) approved by the US Food and Drug Administration (FDA) and 10 DMTs approved by the European Medicines Agency (EMA) for the treatment of RRMS, with new treatment options emerging each year. Approved treatments include interferons (Avonex, Betaferon, Betaseron, Extavia, Rebif), glatiramer acetate (Copaxone), natalizumab (Tysabri), and, more recently, the oral treatments teriflunomide (Aubagio), fingolimod (Gilenya), and dimethyl fumarate (Tecfidera). In this study, we explored whether we could analyse social media to help gauge patient sentiment about treatments using MS as an example. We used the popular social media site Twitter (
                <ext-link ext-link-type="uri" xlink:href="http://twitter.com">http://twitter.com</ext-link>) to explore the reporting of patient sentiment and emotions about MS treatments.</p>
        </sec>
        <sec sec-type="methods">
            <title>Methods</title>
            <sec>
                <title>Data</title>
                <p>The Twitter resource, Topsy (
                    <ext-link ext-link-type="uri" xlink:href="http://topsy.com/">http://topsy.com/</ext-link>), which houses all tweets made since 2006, was searched for the following brand names of MS treatments: Aubagio, Avonex, Betaferon or Betaseron, Copaxone, Extavia, Gilenya, Lemtrada, Novantrone, Rebif, Tysabri and Tecfidera using a daily search-time window (i.e. searching for tweets made every day), and specifying the English language. Brand names were used as we thought this would be more likely to reflect patient tweets and further the generic name for some MS treatments are not specific MS treatments. All dates from 1 Jan 2006 to 31 Jul 2014 were searched. For days in which there were more than 1000 tweets satisfying the search criteria, an hourly search-time window was applied for that day, to enable all available tweets to be found (the resource limits searches to 1000 results).</p>
                <p>Tweets were downloaded in Extensible Markup Language (XML) format from 
                    <ext-link ext-link-type="uri" xlink:href="http://topsy.com/">topsy.com</ext-link> using the application program interface, otterapi (
                    <ext-link ext-link-type="uri" xlink:href="https://code.google.com/p/otterapi/">https://code.google.com/p/otterapi/</ext-link>).</p>
            </sec>
            <sec>
                <title>Data filtering</title>
                <p>Tweets were subsequently filtered to generate datasets for analysis:</p>
                <p>1. A unique dataset was generated from the &#x201c;highlight&#x201d; data class; thus, removing all directly copied retweets. This was performed so that sentiment analysis could be performed on unique tweets and not bias analyses by having several copies of the same tweet.</p>
                <p>All subsequent filtering was case-insensitive.</p>
                <p>2. The unique dataset from (1) was filtered to remove items relating to company share prices/stockmarket news.</p>
                <p>This was achieved by removing all tweets that contained:</p>
                <p>a) &#x201c;market_jp&#x201d;, &#x201c;thestreet&#x201d;, &#x201c;rtebusiness&#x201d;, &#x201c;pharma&#x201d;,or &#x201c;pharmsales&#x201d; in the &#x201c;permalink&#x201d; dataclass,</p>
                <p>or</p>
                <p>b) &#x201c;bloomberg&#x201d;, &#x201c;forbes&#x201d;, &#x201c;dow jones&#x201d;, &#x201c;financial times&#x201d;, &#x201c;stockpickr&#x201d;, &#x201c;marketwatch&#x201d;, &#x201c;business:&#x201d;, &#x201c;profit&#x201d;, &#x201c;shares&#x201d;, or &#x201c;sec&#x201d; in the &#x201c;highlight&#x201d; data class. This filter was performed as we wanted to identify patient opinion about MS treatments and not stock market related tweets. This filter did retain tweets containing company names, some of which were stock/share price related but some tweets containing company names were from patients.</p>
                <p>3. The dataset from (2) was further filtered to remove items that mentioned the manufacturing companies by name: tweets were removed if they contained any of the following:</p>
                <p>&#x201c;novartis&#x201d;, &#x201c;elan&#x201d;, &#x201c;biogen&#x201d;, &#x201c;merck&#x201d;, &#x201c;bayer&#x201d;, &#x201c;genzyme&#x201d;, &#x201c;sanofi&#x201d;, &#x201c;teva&#x201d;, or &#x201c;serono&#x201d;. This filter was stringent and removed the majority of stock/share related tweets, but also removed some patient tweets.</p>
            </sec>
            <sec>
                <title>Normalisation</title>
                <p>Because of the short nature of tweets, typographical errors, ad-hoc abbreviations, phonetic substitutions, ungrammatical structures and emoticons are common, causing problems for text processing tools. Tokenisation and normalisation to make better sense of the tweet texts was achieved using TwitIE (
                    <ext-link ext-link-type="uri" xlink:href="http://gate.ac.uk/sale/ranlp2013/twitie/twitie-ranlp2013.pdf?m=1">http://gate.ac.uk/sale/ranlp2013/twitie/twitie-ranlp2013.pdf?m=1</ext-link>). Normalisation did not remove or alter any of the drug names.</p>
            </sec>
            <sec>
                <title>Sentiment and word frequency analysis</title>
                <p>Tweets were grouped into sequential monthly time periods for sentiment analysis using the twitteR R package (
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/geoffjentry/twitteR/">https://github.com/geoffjentry/twitteR/</ext-link>) and Jeffrey Breen&#x2019;s sentiment analysis code (
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/jeffreybreen/twitter-sentiment-analysis-tutorial-201107">https://github.com/jeffreybreen/twitter-sentiment-analysis-tutorial-201107</ext-link>; a tutorial can be found at: 
                    <ext-link ext-link-type="uri" xlink:href="http://www.inside-r.org/howto/mining-twitter-airline-consumer-sentiment">http://www.inside-r.org/howto/mining-twitter-airline-consumer-sentiment</ext-link>). Word frequency analysis in tweets was performed using TagCrowd (
                    <ext-link ext-link-type="uri" xlink:href="http://tagcrowd.com">http://tagcrowd.com</ext-link>). TagCrowd uses language-specific lists of common words which are removed from analysis.</p>
            </sec>
            <sec>
                <title>Statistical analysis</title>
                <p>Using lists of 2006 positive and 4783 negative words (
                    <ext-link ext-link-type="uri" xlink:href="http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html#lexicon">http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html#lexicon</ext-link>), the sentiment score for any tweet is calculated as follows:</p>
                <p>
                    <italic toggle="yes">Sentiment score = number of positive words - number of negative words</italic>
                </p>
                <p>If the sentiment score &gt; 0, this means that the sentence has an overall 'positive opinion', if the sentiment score &lt; 0, this means that the sentence has an overall 'negative opinion', if the sentiment score=0, then the sentence is considered to be a 'neutral opinion'. Sentiment scores were summed for all tweets for each MS treatment, and means calculated. Mean sentiment scores were compared across treatments using the Kruskal-Wallis test. Statistical analysis was performed using R version 3.1.1 and p values less 0.05 were considered significant.</p>
            </sec>
        </sec>
        <sec sec-type="results">
            <title>Results</title>
            <p>In total, there were 60037 unique tweets mentioning an MS treatment. The number of tweets by month is shown in 
                <xref ref-type="fig" rid="f1">Figure 1</xref>. Tweets for Tysabri started the earliest (January 2008) and Aubagio the latest (February 2009). When removing tweets that included share/stock information there were 56708 unique tweets and when removing tweets that included share/stock information or company names there were 41690 unique tweets.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>Number of tweets by month for each treatment.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure1.gif"/>
            </fig>
            <p>The number of tweets by treatment, overall and when removing tweets that included share/stock information and/or company names is shown in 
                <xref ref-type="table" rid="T1">Table 1</xref>. Tysabri had the largest number of tweets (n=14542, all tweets; n=10984 after filtering for company names and stock/share tweets) and Novantrone had the lowest (n=110, all tweets; n=109 after filtering for company names and stock/share tweets), both before and after filtering.</p>
            <table-wrap id="T1" orientation="portrait" position="anchor">
                <label>Table 1. </label>
                <caption>
                    <title>Number of tweets by treatment.</title>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Treatment</th>
                            <th align="center" colspan="1" rowspan="1">Aubagio</th>
                            <th align="center" colspan="1" rowspan="1">Avonex</th>
                            <th align="center" colspan="1" rowspan="1">Betaferon</th>
                            <th align="center" colspan="1" rowspan="1">Copaxone</th>
                            <th align="center" colspan="1" rowspan="1">Extavia</th>
                            <th align="center" colspan="1" rowspan="1">Gilenya</th>
                            <th align="center" colspan="1" rowspan="1">Lemtrada</th>
                            <th align="center" colspan="1" rowspan="1">Novantrone</th>
                            <th align="center" colspan="1" rowspan="1">Rebif</th>
                            <th align="center" colspan="1" rowspan="1">Tecfidera</th>
                            <th align="center" colspan="1" rowspan="1">Tysabri</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>First Tweet Month</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">Feb 2009</td>
                            <td align="center" colspan="1" rowspan="1">Apr 2008</td>
                            <td align="center" colspan="1" rowspan="1">May 2008</td>
                            <td align="center" colspan="1" rowspan="1">Apr 2008</td>
                            <td align="center" colspan="1" rowspan="1">Sep 2008</td>
                            <td align="center" colspan="1" rowspan="1">Apr 2008</td>
                            <td align="center" colspan="1" rowspan="1">Oct 2008</td>
                            <td align="center" colspan="1" rowspan="1">Dec 2008</td>
                            <td align="center" colspan="1" rowspan="1">Apr 2008</td>
                            <td align="center" colspan="1" rowspan="1">Oct 2008</td>
                            <td align="center" colspan="1" rowspan="1">Jan 2008</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Total Number of Tweets</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">2814</td>
                            <td align="center" colspan="1" rowspan="1">4643</td>
                            <td align="center" colspan="1" rowspan="1">1615</td>
                            <td align="center" colspan="1" rowspan="1">11634</td>
                            <td align="center" colspan="1" rowspan="1">511</td>
                            <td align="center" colspan="1" rowspan="1">9376</td>
                            <td align="center" colspan="1" rowspan="1">5634</td>
                            <td align="center" colspan="1" rowspan="1">110</td>
                            <td align="center" colspan="1" rowspan="1">4236</td>
                            <td align="center" colspan="1" rowspan="1">4922</td>
                            <td align="center" colspan="1" rowspan="1">14542</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Total Number of Tweets with</bold>
                                <break/>
                                <bold>Stock Tweets Removed</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">2632</td>
                            <td align="center" colspan="1" rowspan="1">4540</td>
                            <td align="center" colspan="1" rowspan="1">1577</td>
                            <td align="center" colspan="1" rowspan="1">10676</td>
                            <td align="center" colspan="1" rowspan="1">501</td>
                            <td align="center" colspan="1" rowspan="1">8902</td>
                            <td align="center" colspan="1" rowspan="1">5302</td>
                            <td align="center" colspan="1" rowspan="1">110</td>
                            <td align="center" colspan="1" rowspan="1">4162</td>
                            <td align="center" colspan="1" rowspan="1">4637</td>
                            <td align="center" colspan="1" rowspan="1">13669</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Total Number of Tweets with</bold>
                                <break/>
                                <bold>Stock and Company Names</bold>
                                <break/>
                                <bold>Removed</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">1601</td>
                            <td align="center" colspan="1" rowspan="1">4214</td>
                            <td align="center" colspan="1" rowspan="1">1392</td>
                            <td align="center" colspan="1" rowspan="1">6818</td>
                            <td align="center" colspan="1" rowspan="1">432</td>
                            <td align="center" colspan="1" rowspan="1">6999</td>
                            <td align="center" colspan="1" rowspan="1">2444</td>
                            <td align="center" colspan="1" rowspan="1">109</td>
                            <td align="center" colspan="1" rowspan="1">3784</td>
                            <td align="center" colspan="1" rowspan="1">2913</td>
                            <td align="center" colspan="1" rowspan="1">10984</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <p>The sentiment score analysis of all normalised tweets, normalised tweets excluding those that contained share/stock information and normalised tweets excluding those that contained share/stock information and company names are shown in 
                <xref ref-type="table" rid="T2">Table 2</xref>, 
                <xref ref-type="table" rid="T3">Table 3</xref> and 
                <xref ref-type="table" rid="T4">Table 4</xref>. About half of all tweets in all analyses had a neutral sentiment (43&#x2013;61%, all tweet data; 45&#x2013;57% after filtering for company names and stock/share tweet data). Tweets for drugs that contained sentiment were more likely to be positive sentiment, apart from tweets for Novantrone and Tysabri (23&#x2013;33% for drugs apart from Novantrone (16%), all tweet data; 24&#x2013;31% for drugs apart from Novantrone (17%) and Tysabri (28%), after filtering for company names and stock/share tweet data).</p>
            <table-wrap id="T2" orientation="portrait" position="anchor">
                <label>Table 2. </label>
                <caption>
                    <title>Sentiment analysis of all normalised tweets.</title>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Treatment</th>
                            <th align="center" colspan="1" rowspan="1">Aubagio</th>
                            <th align="center" colspan="1" rowspan="1">Avonex</th>
                            <th align="center" colspan="1" rowspan="1">Betaferon</th>
                            <th align="center" colspan="1" rowspan="1">Copaxone</th>
                            <th align="center" colspan="1" rowspan="1">Extavia</th>
                            <th align="center" colspan="1" rowspan="1">Gilenya</th>
                            <th align="center" colspan="1" rowspan="1">Lemtrada</th>
                            <th align="center" colspan="1" rowspan="1">Novantrone</th>
                            <th align="center" colspan="1" rowspan="1">Rebif</th>
                            <th align="center" colspan="1" rowspan="1">Tecfidera</th>
                            <th align="center" colspan="1" rowspan="1">Tysabri</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Positive Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.303</td>
                            <td align="center" colspan="1" rowspan="1">0.277</td>
                            <td align="center" colspan="1" rowspan="1">0.241</td>
                            <td align="center" colspan="1" rowspan="1">0.293</td>
                            <td align="center" colspan="1" rowspan="1">0.233</td>
                            <td align="center" colspan="1" rowspan="1">0.318</td>
                            <td align="center" colspan="1" rowspan="1">0.299</td>
                            <td align="center" colspan="1" rowspan="1">0.164</td>
                            <td align="center" colspan="1" rowspan="1">0.265</td>
                            <td align="center" colspan="1" rowspan="1">0.33</td>
                            <td align="center" colspan="1" rowspan="1">0.323</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Negative Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.118</td>
                            <td align="center" colspan="1" rowspan="1">0.224</td>
                            <td align="center" colspan="1" rowspan="1">0.218</td>
                            <td align="center" colspan="1" rowspan="1">0.216</td>
                            <td align="center" colspan="1" rowspan="1">0.162</td>
                            <td align="center" colspan="1" rowspan="1">0.155</td>
                            <td align="center" colspan="1" rowspan="1">0.18</td>
                            <td align="center" colspan="1" rowspan="1">0.3</td>
                            <td align="center" colspan="1" rowspan="1">0.248</td>
                            <td align="center" colspan="1" rowspan="1">0.123</td>
                            <td align="center" colspan="1" rowspan="1">0.252</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>No Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.579</td>
                            <td align="center" colspan="1" rowspan="1">0.499</td>
                            <td align="center" colspan="1" rowspan="1">0.541</td>
                            <td align="center" colspan="1" rowspan="1">0.491</td>
                            <td align="center" colspan="1" rowspan="1">0.605</td>
                            <td align="center" colspan="1" rowspan="1">0.527</td>
                            <td align="center" colspan="1" rowspan="1">0.521</td>
                            <td align="center" colspan="1" rowspan="1">0.536</td>
                            <td align="center" colspan="1" rowspan="1">0.487</td>
                            <td align="center" colspan="1" rowspan="1">0.548</td>
                            <td align="center" colspan="1" rowspan="1">0.425</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Summed Sentiment Score</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">777</td>
                            <td align="center" colspan="1" rowspan="1">263</td>
                            <td align="center" colspan="1" rowspan="1">32</td>
                            <td align="center" colspan="1" rowspan="1">1081</td>
                            <td align="center" colspan="1" rowspan="1">25</td>
                            <td align="center" colspan="1" rowspan="1">2111</td>
                            <td align="center" colspan="1" rowspan="1">920</td>
                            <td align="center" colspan="1" rowspan="1">-21</td>
                            <td align="center" colspan="1" rowspan="1">74</td>
                            <td align="center" colspan="1" rowspan="1">1388</td>
                            <td align="center" colspan="1" rowspan="1">1513</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Mean Sentiment Score</bold>
                                <break/>
                                <bold>(Standard Deviation)</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.276
                                <break/>(0.91)</td>
                            <td align="center" colspan="1" rowspan="1">0.057
                                <break/>(1.095)</td>
                            <td align="center" colspan="1" rowspan="1">0.02
                                <break/>(0.984)</td>
                            <td align="center" colspan="1" rowspan="1">0.093
                                <break/>(1.031)</td>
                            <td align="center" colspan="1" rowspan="1">0.049
                                <break/>(0.854)</td>
                            <td align="center" colspan="1" rowspan="1">0.225
                                <break/>(0.949)</td>
                            <td align="center" colspan="1" rowspan="1">0.163
                                <break/>(0.959)</td>
                            <td align="center" colspan="1" rowspan="1">-0.191
                                <break/>(1.054)</td>
                            <td align="center" colspan="1" rowspan="1">0.017
                                <break/>(1.101)</td>
                            <td align="center" colspan="1" rowspan="1">0.282
                                <break/>(0.932)</td>
                            <td align="center" colspan="1" rowspan="1">0.104
                                <break/>(1.149)</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <table-wrap id="T3" orientation="portrait" position="anchor">
                <label>Table 3. </label>
                <caption>
                    <title>Sentiment analysis of all normalised tweets with tweets containing share/stock information excluded.</title>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Treatment</th>
                            <th align="center" colspan="1" rowspan="1">Aubagio</th>
                            <th align="center" colspan="1" rowspan="1">Avonex</th>
                            <th align="center" colspan="1" rowspan="1">Betaferon</th>
                            <th align="center" colspan="1" rowspan="1">Copaxone</th>
                            <th align="center" colspan="1" rowspan="1">Extavia</th>
                            <th align="center" colspan="1" rowspan="1">Gilenya</th>
                            <th align="center" colspan="1" rowspan="1">Lemtrada</th>
                            <th align="center" colspan="1" rowspan="1">Novantrone</th>
                            <th align="center" colspan="1" rowspan="1">Rebif</th>
                            <th align="center" colspan="1" rowspan="1">Tecfidera</th>
                            <th align="center" colspan="1" rowspan="1">Tysabri</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Positive Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.299</td>
                            <td align="center" colspan="1" rowspan="1">0.275</td>
                            <td align="center" colspan="1" rowspan="1">0.236</td>
                            <td align="center" colspan="1" rowspan="1">0.293</td>
                            <td align="center" colspan="1" rowspan="1">0.236</td>
                            <td align="center" colspan="1" rowspan="1">0.315</td>
                            <td align="center" colspan="1" rowspan="1">0.296</td>
                            <td align="center" colspan="1" rowspan="1">0.164</td>
                            <td align="center" colspan="1" rowspan="1">0.267</td>
                            <td align="center" colspan="1" rowspan="1">0.328</td>
                            <td align="center" colspan="1" rowspan="1">0.319</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Negative Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.119</td>
                            <td align="center" colspan="1" rowspan="1">0.227</td>
                            <td align="center" colspan="1" rowspan="1">0.221</td>
                            <td align="center" colspan="1" rowspan="1">0.214</td>
                            <td align="center" colspan="1" rowspan="1">0.162</td>
                            <td align="center" colspan="1" rowspan="1">0.156</td>
                            <td align="center" colspan="1" rowspan="1">0.183</td>
                            <td align="center" colspan="1" rowspan="1">0.3</td>
                            <td align="center" colspan="1" rowspan="1">0.25</td>
                            <td align="center" colspan="1" rowspan="1">0.123</td>
                            <td align="center" colspan="1" rowspan="1">0.252</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>No Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.582</td>
                            <td align="center" colspan="1" rowspan="1">0.498</td>
                            <td align="center" colspan="1" rowspan="1">0.543</td>
                            <td align="center" colspan="1" rowspan="1">0.493</td>
                            <td align="center" colspan="1" rowspan="1">0.603</td>
                            <td align="center" colspan="1" rowspan="1">0.53</td>
                            <td align="center" colspan="1" rowspan="1">0.521</td>
                            <td align="center" colspan="1" rowspan="1">0.536</td>
                            <td align="center" colspan="1" rowspan="1">0.483</td>
                            <td align="center" colspan="1" rowspan="1">0.549</td>
                            <td align="center" colspan="1" rowspan="1">0.43</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Summed Sentiment Score</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">712</td>
                            <td align="center" colspan="1" rowspan="1">231</td>
                            <td align="center" colspan="1" rowspan="1">4</td>
                            <td align="center" colspan="1" rowspan="1">1091</td>
                            <td align="center" colspan="1" rowspan="1">26</td>
                            <td align="center" colspan="1" rowspan="1">1977</td>
                            <td align="center" colspan="1" rowspan="1">818</td>
                            <td align="center" colspan="1" rowspan="1">-21</td>
                            <td align="center" colspan="1" rowspan="1">69</td>
                            <td align="center" colspan="1" rowspan="1">1313</td>
                            <td align="center" colspan="1" rowspan="1">1278</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Mean Sentiment Score</bold>
                                <break/>
                                <bold>(Standard Deviation)</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.271
                                <break/>(0.911)</td>
                            <td align="center" colspan="1" rowspan="1">0.051
                                <break/>(1.099)</td>
                            <td align="center" colspan="1" rowspan="1">0.003
                                <break/>(0.973)</td>
                            <td align="center" colspan="1" rowspan="1">0.102
                                <break/>(1.035)</td>
                            <td align="center" colspan="1" rowspan="1">0.052
                                <break/>(0.859)</td>
                            <td align="center" colspan="1" rowspan="1">0.222
                                <break/>(0.951)</td>
                            <td align="center" colspan="1" rowspan="1">0.154
                                <break/>(0.962)</td>
                            <td align="center" colspan="1" rowspan="1">-0.191
                                <break/>(1.054)</td>
                            <td align="center" colspan="1" rowspan="1">0.017
                                <break/>(1.107)</td>
                            <td align="center" colspan="1" rowspan="1">0.283
                                <break/>(0.934)</td>
                            <td align="center" colspan="1" rowspan="1">0.093
                                <break/>(1.149)</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <table-wrap id="T4" orientation="portrait" position="anchor">
                <label>Table 4. </label>
                <caption>
                    <title>Sentiment analysis of all normalised tweets with tweets containing share/stock information and company names excluded.</title>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Treatment</th>
                            <th align="center" colspan="1" rowspan="1">Aubagio</th>
                            <th align="center" colspan="1" rowspan="1">Avonex</th>
                            <th align="center" colspan="1" rowspan="1">Betaferon</th>
                            <th align="center" colspan="1" rowspan="1">Copaxone</th>
                            <th align="center" colspan="1" rowspan="1">Extavia</th>
                            <th align="center" colspan="1" rowspan="1">Gilenya</th>
                            <th align="center" colspan="1" rowspan="1">Lemtrada</th>
                            <th align="center" colspan="1" rowspan="1">Novantrone</th>
                            <th align="center" colspan="1" rowspan="1">Rebif</th>
                            <th align="center" colspan="1" rowspan="1">Tecfidera</th>
                            <th align="center" colspan="1" rowspan="1">Tysabri</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Positive Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.299</td>
                            <td align="center" colspan="1" rowspan="1">0.264</td>
                            <td align="center" colspan="1" rowspan="1">0.239</td>
                            <td align="center" colspan="1" rowspan="1">0.29</td>
                            <td align="center" colspan="1" rowspan="1">0.255</td>
                            <td align="center" colspan="1" rowspan="1">0.284</td>
                            <td align="center" colspan="1" rowspan="1">0.311</td>
                            <td align="center" colspan="1" rowspan="1">0.165</td>
                            <td align="center" colspan="1" rowspan="1">0.267</td>
                            <td align="center" colspan="1" rowspan="1">0.283</td>
                            <td align="center" colspan="1" rowspan="1">0.275</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>Negative Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.128</td>
                            <td align="center" colspan="1" rowspan="1">0.237</td>
                            <td align="center" colspan="1" rowspan="1">0.224</td>
                            <td align="center" colspan="1" rowspan="1">0.221</td>
                            <td align="center" colspan="1" rowspan="1">0.185</td>
                            <td align="center" colspan="1" rowspan="1">0.164</td>
                            <td align="center" colspan="1" rowspan="1">0.195</td>
                            <td align="center" colspan="1" rowspan="1">0.303</td>
                            <td align="center" colspan="1" rowspan="1">0.26</td>
                            <td align="center" colspan="1" rowspan="1">0.143</td>
                            <td align="center" colspan="1" rowspan="1">0.277</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Proportion of Tweets with</bold>
                                <break/>
                                <bold>No Sentiment</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.573</td>
                            <td align="center" colspan="1" rowspan="1">0.499</td>
                            <td align="center" colspan="1" rowspan="1">0.537</td>
                            <td align="center" colspan="1" rowspan="1">0.489</td>
                            <td align="center" colspan="1" rowspan="1">0.56</td>
                            <td align="center" colspan="1" rowspan="1">0.553</td>
                            <td align="center" colspan="1" rowspan="1">0.493</td>
                            <td align="center" colspan="1" rowspan="1">0.532</td>
                            <td align="center" colspan="1" rowspan="1">0.473</td>
                            <td align="center" colspan="1" rowspan="1">0.574</td>
                            <td align="center" colspan="1" rowspan="1">0.448</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Summed Sentiment Score</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">395</td>
                            <td align="center" colspan="1" rowspan="1">120</td>
                            <td align="center" colspan="1" rowspan="1">4</td>
                            <td align="center" colspan="1" rowspan="1">677</td>
                            <td align="center" colspan="1" rowspan="1">20</td>
                            <td align="center" colspan="1" rowspan="1">1149</td>
                            <td align="center" colspan="1" rowspan="1">378</td>
                            <td align="center" colspan="1" rowspan="1">-21</td>
                            <td align="center" colspan="1" rowspan="1">12</td>
                            <td align="center" colspan="1" rowspan="1">523</td>
                            <td align="center" colspan="1" rowspan="1">-115</td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">
								
                                <bold>Mean Sentiment Score</bold>
                                <break/>
                                <bold>(Standard Deviation)</bold>
							</td>
                            <td align="center" colspan="1" rowspan="1">0.247
                                <break/>(0.923)</td>
                            <td align="center" colspan="1" rowspan="1">0.028
                                <break/>(1.11)</td>
                            <td align="center" colspan="1" rowspan="1">0.003
                                <break/>(0.999)</td>
                            <td align="center" colspan="1" rowspan="1">0.099
                                <break/>(1.077)</td>
                            <td align="center" colspan="1" rowspan="1">0.046
                                <break/>(0.91)</td>
                            <td align="center" colspan="1" rowspan="1">0.164
                                <break/>(0.933)</td>
                            <td align="center" colspan="1" rowspan="1">0.155
                                <break/>(1.034)</td>
                            <td align="center" colspan="1" rowspan="1">-0.193
                                <break/>(1.058)</td>
                            <td align="center" colspan="1" rowspan="1">0.003
                                <break/>(1.136)</td>
                            <td align="center" colspan="1" rowspan="1">0.18
                                <break/>(0.914)</td>
                            <td align="center" colspan="1" rowspan="1">-0.01
                                <break/>(1.129)</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <p>Summing sentiment scores for all tweets showed positive overall sentiment scores for all drugs apart from Novantrone (all analyses) and Tysabri (only after filtering for company names and stock/share tweet data). Gilenya had the highest summed sentiment score in all analyses. Boxplots of sentiment scores of all normalised tweets, normalised tweets excluding those that contained share/stock information and normalised tweets excluding those that contained share/stock information and company names are shown in 
                <xref ref-type="fig" rid="f2">Figure 2</xref>, 
                <xref ref-type="fig" rid="f3">Figure 3</xref> and 
                <xref ref-type="fig" rid="f4">Figure 4</xref>. The mean sentiment score ranged from -0.191 to 0.282 (all tweet data); and -0.193 to 0.247 (after filtering for company names and stock/share tweet data). Novantrone always had the lowest mean sentiment score. Tecfidera had the highest mean score in the all tweet data, and Aubagio had the highest mean score in the filtered for company names and stock/share tweet data. The mean sentiment scores were different in all analyses (P&lt;0.001 in the all tweet data, filtered for stock/share tweet data and filtered for company names and stock/share tweet data).</p>
            <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                <label>Figure 2. </label>
                <caption>
                    <title>Boxplots of sentiment scores for all normalised tweets.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure2.gif"/>
            </fig>
            <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                <label>Figure 3. </label>
                <caption>
                    <title>Boxplots of sentiment scores of all normalised tweets with tweets containing share/stock information excluded.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure3.gif"/>
            </fig>
            <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                <label>Figure 4. </label>
                <caption>
                    <title>Boxplots of sentiment scores of all normalised tweets with tweets containing share/stock information and company names excluded.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure4.gif"/>
            </fig>
            <p>Most common words in tweets for treatments were investigated. Example word clouds for the 50 most common words (excluding commonly used English words and drug names) in all normalised tweets for Avonex, Rebif and Tysabri are shown in 
                <xref ref-type="fig" rid="f5">Figure 5</xref>, 
                <xref ref-type="fig" rid="f6">Figure 6</xref> and 
                <xref ref-type="fig" rid="f7">Figure 7</xref>. Of note is the frequency of &#x2018;flu&#x2019; and &#x2018;injection&#x2019; in Avonex and Rebif tweets and &#x2018;infusion&#x2019; and &#x2018;pml&#x2019; in Tysabri tweets.</p>
            <fig fig-type="figure" id="f5" orientation="portrait" position="float">
                <label>Figure 5. </label>
                <caption>
                    <title>Word cloud for all normalised tweets for Avonex.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure5.gif"/>
            </fig>
            <fig fig-type="figure" id="f6" orientation="portrait" position="float">
                <label>Figure 6. </label>
                <caption>
                    <title>Word cloud for all normalised tweets for Rebif.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure6.gif"/>
            </fig>
            <fig fig-type="figure" id="f7" orientation="portrait" position="float">
                <label>Figure 7. </label>
                <caption>
                    <title>Word cloud for all normalised tweets for Tysabri.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/5610/f62a6576-dd02-4050-8cc9-84b8102e8e41_figure7.gif"/>
            </fig>
        </sec>
        <sec sec-type="discussion">
            <title>Discussion</title>
            <p>We present here, to the best of our knowledge, the first analysis of social media for MS treatments. A significant proportion of tweets did contain non-neutral sentiment about MS treatments, and the distribution of sentiment score was different between treatments. Thus it appears that Twitter can be a potential resource to understand patient opinion about MS treatments. When looking at frequency of words, notably &#x2018;flu&#x2019; and &#x2018;injection&#x2019; were in the 50 most common words in tweets about Rebif and Avonex and &#x2018;infusion&#x2019; and &#x2018;pml&#x2019; in the 50 most common words in tweets about Tysabri. Flu-like symptoms are well known side-effects of the injectible treatments Rebif and Avonex and progressive multifocal leukoencephalopathy or &#x2018;pml&#x2019;, is a well-known risk for patients taking the intravenously infused Tysabri
                <sup>
                    <xref ref-type="bibr" rid="ref-5">5</xref>
                </sup>. This provides some sort of face validity for our results reflecting real specific tweets about MS treatments.</p>
            <p>Interestingly, the oral MS treatments- Gilenya, Aubagio and Tecfidera had the highest mean sentiment scores and Gilenya had the highest summed sentiment score in all analysis. This may reflect a well known patient preference for oral therapies as compared to injectible treatments
                <sup>
                    <xref ref-type="bibr" rid="ref-6">6</xref>,
                    <xref ref-type="bibr" rid="ref-7">7</xref>
                </sup>. Further work is needed to explore tweets in detail to see if the higher mean sentiment scores are related to positive tweets about the fact that these drugs are to be taken orally.</p>
            <p>There are a number of limitations to this study. We are using automated tools to assign sentiment to tweet content- these tools will not recognise the intricacies of human language e.g. the context of the tweet and sarcasm for example. Further, whilst we tried to normalise tweets, the diversity of twitter slang will mean that abbreviations may not be recognised. We may have underestimated the number of tweets as we used brand names to identify drugs. Any tweets using the generic name or shortened versions will be missed. Whilst we tried to focus on tweets from patients, it is inevitable that business related tweets will have been included in our analysis and some patient tweets lost during filtering. It is also possible that not all tweets were delivered to us by the Twitter interface, although that is not possible to verify.</p>
            <p>Our findings and any interpretation should be regarded as speculative and exploratory. The results represent what can be potentially done relatively quickly and easily using data from Twitter. More rigorous analytical methods can be applied for more specific questions (e.g. the analysis of adverse events). It is clear from this study that tweets are written about MS treatments and many of these have a non-neutral sentiment. Further work is needed to look at these tweets in detail to further understand patient opinion about MS treatments.</p>
        </sec>
    </body>
    <back>
        <ref-list>
            <ref id="ref-1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Broniatowski</surname>
                            <given-names>DA</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Paul</surname>
                            <given-names>MJ</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Dredze</surname>
                            <given-names>M</given-names>
                        </name>
					</person-group>:
                    <article-title>National and local influenza surveillance through Twitter: an analysis of the 2012-2013 influenza epidemic.</article-title>
                    <source>
						
                        <italic toggle="yes">PLoS One.</italic>
					</source>
                    <year>2013</year>;<volume>8</volume>(<issue>12</issue>):<fpage>e83672</fpage>.
                    <pub-id pub-id-type="pmid">24349542</pub-id>
                    <pub-id pub-id-type="doi">10.1371/journal.pone.0083672</pub-id>
                    <pub-id pub-id-type="pmcid">3857320</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <label>2</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Tsuya</surname>
                            <given-names>A</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Sugawara</surname>
                            <given-names>Y</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Tanaka</surname>
                            <given-names>A</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Do cancer patients tweet? Examining the twitter use of cancer patients in Japan.</article-title>
                    <source>
						
                        <italic toggle="yes">J Med Internet Res.</italic>
					</source>
                    <year>2014</year>;<volume>16</volume>(<issue>5</issue>):<fpage>e137</fpage>.
                    <pub-id pub-id-type="pmid">24867458</pub-id>
                    <pub-id pub-id-type="doi">10.2196/jmir.3298</pub-id>
                    <pub-id pub-id-type="pmcid">4060148</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <label>3</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Freifeld</surname>
                            <given-names>CC</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Brownstein</surname>
                            <given-names>JS</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Menone</surname>
                            <given-names>CM</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Digital drug safety surveillance: monitoring pharmaceutical products in twitter.</article-title>
                    <source>
						
                        <italic toggle="yes">Drug Saf.</italic>
					</source>
                    <year>2014</year>;<volume>37</volume>(<issue>5</issue>):<fpage>343</fpage>&#x2013;<lpage>350</lpage>.
                    <pub-id pub-id-type="pmid">24777653</pub-id>
                    <pub-id pub-id-type="doi">10.1007/s40264-014-0155-x</pub-id>
                    <pub-id pub-id-type="pmcid">4013443</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Ramagopalan</surname>
                            <given-names>SV</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Dobson</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Meier</surname>
                            <given-names>UC</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Multiple sclerosis: risk factors, prodromes, and potential causal pathways.</article-title>
                    <source>
						
                        <italic toggle="yes">Lancet Neurol.</italic>
					</source>
                    <year>2010</year>;<volume>9</volume>(<issue>7</issue>):<fpage>727</fpage>&#x2013;<lpage>739</lpage>.
                    <pub-id pub-id-type="pmid">20610348</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S1474-4422(10)70094-6</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-5">
                <label>5</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Tanasescu</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Ionete</surname>
                            <given-names>C</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Chou</surname>
                            <given-names>IJ</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Advances in the treatment of relapsing-remitting multiple sclerosis.</article-title>
                    <source>
						
                        <italic toggle="yes">Biomed J.</italic>
					</source>
                    <year>2014</year>;<volume>37</volume>(<issue>2</issue>):<fpage>41</fpage>&#x2013;<lpage>49</lpage>.
                    <pub-id pub-id-type="pmid">24732658</pub-id>
                    <pub-id pub-id-type="doi">10.4103/2319-4170.130440</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <label>6</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Dibonaventura</surname>
                            <given-names>MD</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Wagner</surname>
                            <given-names>JS</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Girman</surname>
                            <given-names>GJ</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Multinational Internet-based survey of patient preference for newer oral or injectable Type 2 diabetes medication.</article-title>
                    <source>
						
                        <italic toggle="yes">Patient Prefer Adherence.</italic>
					</source>
                    <year>2010</year>;<volume>4</volume>:<fpage>397</fpage>&#x2013;<lpage>406</lpage>.
                    <pub-id pub-id-type="pmid">21206515</pub-id>
                    <pub-id pub-id-type="doi">10.2147/PPA.S14477</pub-id>
                    <pub-id pub-id-type="pmcid">3003606</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <label>7</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Fallowfield</surname>
                            <given-names>L</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Atkins</surname>
                            <given-names>L</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Catt</surname>
                            <given-names>S</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Patients&#x2019; preference for administration of endocrine treatments by injection or tablets: results from a study of women with breast cancer.</article-title>
                    <source>
						
                        <italic toggle="yes">Ann Oncol.</italic>
					</source>
                    <year>2006</year>;<volume>17</volume>(<issue>2</issue>):<fpage>205</fpage>&#x2013;<lpage>210</lpage>.
                    <pub-id pub-id-type="pmid">16239231</pub-id>
                    <pub-id pub-id-type="doi">10.1093/annonc/mdj044</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report6809">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.5610.r6809</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Hutchinson</surname>
                        <given-names>Michael</given-names>
                    </name>
                    <xref ref-type="aff" rid="r6809a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r6809a1">
                    <label>1</label>St Vincent's University Hospital, University College Dublin, Dublin, Ireland</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>Disclosures: Michael Hutchinson served on a medical advisory board for the CONFIRM study [BG00012] for Biogen Idec, serves on the editorial board of the Multiple Sclerosis Journal, has received speaker&#x2019;s honoraria from Merck- Serono, Novartis, Biogen Idec and Bayer-Schering and receives research support from Dystonia Ireland, the Health Research Board of Ireland and the Foundation for Dystonia Research.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>9</day>
                <month>12</month>
                <year>2014</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Hutchinson M</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport6809" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.5263.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>An interesting paper and an introduction to the reviewer of the new methods of analysis of social media. The authors have emphasised the pains that they took to exclude tweets from company and financial sources. The difficulty is that one cannot be certain that what they are analysing is not contaminated by tweets stimulated in some way from particular pharmaceutical companies. There is, of course, a "newness" affect and this explains the frequency of the tweets for Aubagio and Tecfidera. If one excludes Novantrone and Tysabri, then the lowest sentiment scores are for the interferons. What I find remarkable in this report is that the highest mean sentiment score is for Aubagio, a drug which many neurologists would have less enthusiasm for, given problems in its use in patients of childbearing age. Interesting findings and nicely presented paper, but I am still concerned about the source data; are all these tweets from genuine patients?</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report6808">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.5610.r6808</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Taylor</surname>
                        <given-names>Bruce V</given-names>
                    </name>
                    <xref ref-type="aff" rid="r6808a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r6808a1">
                    <label>1</label>Menzies Research Institute Tasmania, University of Tasmania, Hobart, Australia</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>28</day>
                <month>11</month>
                <year>2014</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Taylor BV</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport6808" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.5263.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>I found this a very interesting report and feel that as the authors do that this method of analysis may provide a real world window into patient perceptions of treatment, efficacy, and side effects. It would be interesting to assess churn in the twitter responses, for example PML in Tysabri is a major issue to patients and it would be interesting to see how this has trended over time, as the period for which tweets were collected includes the period of time when PML was a major issue (newly described) and would have been a major source of negative sentiment. Similarly for the newer agents it would be interesting to track these findings over time from release date and see whether positivity or negativity changes with patient experience.</p>
            <p>This type of research will undoubtedly provide significant and important patient focused outcomes for future medication usage and tolerance studies. at considerable cost efficiency.</p>
            <p>Can the authors look at trending words in these tweets, for example PML in Tecfidera or hair loss for Aubagio?</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
</article>
