Influenza Altmetric Attention Score and its association with the influenza season in the USA

Introduction

In the last few years, a new way to measure the attention brought by journal articles, termed altmetrics (a shortening of “alternative metrics” or “article-level metrics”), was adopted. It was also considered an “alternative” to the conventional citation-based measures. Altmetrics measure the impact and attention of an individual article¹. Altmetrics are increasingly recognized tools with an aim to measure the real-time influence of an academic article². Altmetrics measure the impact of journal articles by tracking social media, Wikipedia, public policy documents, blogs and mainstream news activity, after which an overall Altmetric attention score (AAS) is calculated for every journal article³. Altmetrics have been used to measure the impact of articles on a disease⁴, or even the impact of article on a whole field⁵. Altmetric.com is one of the providers of altmetrics and was found to have the best coverage of blog posts, news, and tweets. It pulls data from social media (e.g. Twitter and Facebook), traditional media (e.g. The Guardian and New York Times), blogs for individuals and organizations (e.g. Cancer Research UK), and online reference managers (e.g. Mendeley and CiteULike), policy documents published by official websites (e.g., .gov). The AAS is a quantitative measure of the quality and quantity of attention an output has received, it provides an indicator of the amount of attention a research has received. It weights the amount of attention received by each source based on an algorithm.

Each country has its own influenza detection center; the U.S has the Centers for Disease Control and Prevention (CDC), Europe has the European Influenza Surveillance Scheme (EISS), and Japan has the Infectious Disease Surveillance Center (IDSC)⁶. The problem of influenza detection and prediction can be tracked back to Serfling’s work in 1963 in epidemiology, which tried to find a threshold for influenza breakout⁷. Since then, various approaches have been proposed for flu detection and prediction in multiple situations^7–9. A previous project by Google in cooperation with the CDC was able to track in a population based on influenza-related web form queries on the Google search engine¹⁰. This approach has paved the way for many new approaches designed using the same concept of using search engines for flu detection in the USA¹¹. In this study, we aim to assess the AAS for influenza related articles. Moreover, we will assess the top articles and journals publishing about influenza in terms of attention they brought.

Methods

Results

A total of 24,964 PubMed documents were extracted. Among them, 12,395 documents had at least one Altmetric point. The total number of mentions for the included documents was 185,744, of which 152,899 were from social media, 20,499 were from news and blogs, 10,608 were from policy and patents, 1,309 were from other sources and 479 were from academic sources. The USA contributed to 28,001 (20.4%) of the total mentions, followed by UK 12,007 (8.8%), and Japan 8,684 (6.3%). The average US mentions for the influenza related articles each month from 2012-2018, and their total mentions are shown in Table 1.

On one-way ANOVA, we found a significant difference between the months (p< 0.001). Following post-hoc analysis, we found a significant difference in mean AAS between February and each of January (p< 0.001, mean difference of 117.4 with 95% CI: 89.7 to 145.2) and March (p< 0.001, mean difference of 460.7 with 95% CI: 430.2 to 491.1). We also found a significant difference between June and each of May (p< 0.001, mean difference of 1221.4 with 95% CI: 87.0 to 155.8) and July (p< 0.001, mean difference of 162.7 with 95% CI: 126.1 to 199.2). We also found a significant difference between October and each of September (p< 0.001, mean difference of 88.8 with 95% CI: 59.6 to 118.0) and November (p< 0.001, mean difference of 154.8 with 95% CI: 125.8 to 183.9). As shown in Figure 1, there are three peaks for the AAS; the highest is observed in February with a mean AAS of 1076.5 (±614.6), the second highest peak is in October with a mean AAS of 831.4 (±441.9), and the third peak is in June with a mean AAS of 586.2 (±271.1).

Figure 1. Average Altmetric Attention Score (AAS) for each month from years 2012 to 2018.

There are three peaks for the AAS; the highest is observed in February with a mean AAS of 1076.5 (±614.6), the second peak is in October with a mean AAS of 831.4 (±441.9), and the third is in June with a mean AAS of 586.2 (±271.1).

The journals publishing articles with highest AAS scores were PLOS ONE with a total AAS of 872 for 979 research outputs, followed by Vaccine with 842 for 1015 research outputs, and Influenza & Other Respiratory Viruses with 465 for 465 research outputs. Table 2 shows the journals that published more than 100 influenza-related research during 2019.

The top research article in terms of AAS is entitled “Infectious virus in exhaled breath of symptomatic seasonal influenza cases from a college community” published in “Proceedings of the National Academy of Sciences of the United States of America” in January 2018, with an AAS of 2927. Table 3 shows research outputs discussing influenza by AAS with an AAS of more than 1000.

We found around 49 articles discussing the use of websites to detect influenza in USA (Figure 2).

Figure 2. Article discussing the use of websites to detect influenza in USA.

Discussion

The research on influenza attracted considerable attention, as measured by the AAS, with the USA the source of the greatest attention. For influenza research from the USA, we observed three peaks for the AAS. The highest peak is observed in February, with a mean AAS of 1076.5 (±614.6), which corresponds to the peak of influenza season as reported by CDC; the second peak is in October with a mean AAS of 831.4 (±441.9), which corresponds to the beginning of the influenza vaccination season; and the third is in June with a mean AAS of 586.2 (±271.1), which corresponds to the end of the influenza season. Almost 10,608 were from policy and patents, representing 5.7% of total attention score, which reflect influenza mentions in official and policy websites (e.g., websites ending with .gov).

Previous studies have used several analytic methods to correlate with influenza season. One of the first studies that brought significant public attention was the one that based its influenza surveillance on Google search engine query data⁹. in a study co-authored by Google Inc. and CDC researchers. The idea behind this surveillance system was detecting health-seeking behavior in the form of queries to online search engine, where this system managed to estimate weekly influenza activity with only a one-day lag from the CDC actual data. Other studies that used similar estimation techniques followed, where a study by Dugas et al. correlated queries to Google search engine with ILI cases reported by emergency departments¹³. This approach of estimating influenza infection trends based on search engine query was also found to be accurate in other countries, for instance, Europe¹⁴, China¹⁵, and South Korea¹⁶. Other authors also used the Yahoo search engine query to yield similar estimations¹⁷. Several studies also used Twitter massages and tweets to detect trends that may correlate with ILI trends as detected by CDC^11,18–21. Other studies used text mining to extract influenza-related blogs from several web and social media sources²². In another approach, several authors used Wikipedia access logs to achieve accurate, real time estimation of influenza cases^23,24. In a study by Santillana et al., the authors combined data from search engines, social media and hospital visits to estimate influenza activity in USA²⁵.

During our literature review, we found around 49 articles discussing the use of websites to detect influenza in USA (Figure 2). Using search engines as a source of data (e.g. Google and Yahoo) has limited the data provided^17,19, compared to micro-blogging websites (e.g. twitter), which contain more semi-structured metadata enabling a more detailed statistical analysis (e.g. cities, gender, age)²⁶. Several papers proposed different models for detecting flu using Twitter-based methods. Ritterman et al. showed that twitter can improve the accuracy of market forecasting by detecting early external events like H1N1²⁷, followed by another study which used twitter, multiple regression, and document filtering to detect relationship between tweets and national data statistics²⁶. In another study, Broniatowski et al. created a new supervised classification model that separates tweets indicating influenza infection from those indicating influenza awareness or concern²⁰.

In general, the interest in publishing about influenza has increased in the recent years²⁸, with USA being the top country in terms of influenza research production^29,30. From the overall influenza research output, influenza vaccine was one of the main topics researched and Journal of Virology and Vaccine journal published the highest number of research articles since 1900²⁹. We also found that PLOS ONE was the top journal in terms of AAS followed by Vaccine.

With the recent emergence of the coronavirus (COVID-19), severe studies used the altmetric analysis to gain insight about its related publications and public’s response to such new publications³¹. An altmetric analysis of COVID-19 articles found several factors affecting article’s attention, including the title and how positive were the results³². Another article also showed a higher attention for articles published in high quality journals³³. Such higher attention and impact of articles published in higher quality journals might be related to the dedicated social media centers in these journals to publicly promote published articles³⁴.

Some limitations to the present study need to be taken into account. The search queries in these models are not exclusively submitted by users experiencing influenza-like symptoms, thus the correlations observed might be only meaningful across large populations. In addition, despite strong historical correlations, these systems remain susceptible to false alerts caused by a sudden increase in ILI-related queries. An unusual event, such as a drug recall for a popular cold or flu remedy, announcing a new flu strain, etc., could cause such a false alert¹⁹. Disease mentions sometimes depend on social events, which might not be related to disease spread, like holding a conference about flu pandemic. Another limitation to using web-based tools is coverage. Additionally, much of the world is currently excluded from the current systems, which can only process English-language tweets²⁰. Future studies should further assess the validity of our descriptive results by performing sensitivity analysis using the number of articles published each year or proceeding years and correlate AAS score with weekly flu activity data. Moreover, other confirmatory studies may assess the validity of our results by assessing data for influenza in other countries such as Australia and New Zealand, where the influenza seasons are different in timing to those to the USA.

The use of social media interaction to describe epidemiological studies has been evolving. We observed a seasonal trend in the attention toward influenza-related research, with three annual peaks that correlated with the beginning, peak, and end of influenza seasons in USA, according to CDC data. We believe that analyzing the attention of influenza related research may aid in detecting influenza season’s peaks, which may be a useful tool in areas with limited on-site detection centers. While this study is a descriptive data and its results provide preliminary data, its results should be cautiously interpreted due to the descriptive nature of the study.

Data availability