Ethical research standards in a world of big data

Introduction

The growing popularity of social media sites presents a unique opportunity to study human interactions and experiences. Twitter, one of the most popular social media sites, allows users to ‘microblog’ by sharing 140 character messages with their social network. Twitter reports that there are 255 million monthly active users sending 500 million tweets each day¹. Researchers have begun to use this data to answer questions in a variety of fields^2–8. Recent reflections on the data collection practices of the US National Security Administration have spurred similar meditations on the ethics of digital research⁹. The concern is that Twitter data could conceivably be used in a way that violates the privacy and rights of the tweet authors. The discussions in the ethics fields have not been made easily accessible to researchers conducting this kind of research, who may be unfamiliar with the ethics literature. To protect the rights of Twitter users, we propose the following guidelines (see Figure 1), which we hope will become the standard among researchers, journal editors, and Institutional Review Boards.

Figure 1. Proposed guidelines for the ethical use of Twitter for research.

Twitter data have already been used in a number of studies to detect influenza-like illness (ILI)^3–6, risky behaviors associated with the transmission of HIV⁷, sentiments about childhood vaccination programs⁸, and political sentiments⁹. These study designs generally feature count data, rather than user-specific data. For example, there are multiple studies that compare the proportion of tweets about flu-related symptoms to public health data on influenza-like incidence (ILI). An increase in syndromic flu tweets might indicate that an outbreak is occurring. These data are usually reported either at the national level or without any geographic parameters. Another common study design aims to determine public sentiments by counting words, phrases, and emoticons that co-occur with keywords like ‘Obama’. These sentiment indicators can be used to infer public opinions about political elections, mental health, or consumer products.

Unlike Facebook, Pinterest and other competitors, Twitter provides several application programming interfaces (APIs) that allow real-time access to vast amounts of content. Data streamed through the APIs include metadata about the authors, including the text location from their profile (e.g. ‘Baltimore’), their time zone, the time they sent the tweet, the number of friends and followers they have, the number of tweets they have ever sent, and more (Figure 2). Approximately 1% of tweets have a geolocation, which uses GPS to append the author’s precise geographic coordinates to the tweet. Geolocations are sufficiently detailed to determine from which wing of a building a tweet was sent. The default privacy settings do not enable geolocation, but do make a user’s tweets and metadata available through the API. Users can modify their setting to make their profile private, which shields their account from public view online and from the API¹⁰.

Figure 2. An excerpt of a single tweet returned through the Twitter API¹¹.

There are numerous ways to access the data through these APIs: one method is through the ‘garden hose’, which is a random sample of approximately 1% of all live-streamed tweets. Other access methods include the search API which enables searching for particular users, hashtags, or locations, and author-specific queries which can retrospectively gather up to 3,200 tweets from a single user¹⁰. Furthermore, in 2010 Twitter donated its entire historical record of tweets to the US Library of Congress. Detailed plans for these data are not yet available, but the Library of Congress has indicated that it intends to collaborate with academic institutions to make the data available to researchers^11,12.

The strength of tweets as a data source is in the volume; collection through the garden hose API brings in approximately 60,000–100,000 tweets per day. However because tweets are short and often lack context, it is difficult for computers to determine tweet content automatically. For this reason, researchers primarily use tweet data to conduct population-level research concerned with trends and patterns. Study designs rely on large volumes of data to accommodate false positives and negatives. A typical data set contains millions of tweets and many thousands of tweet authors. However, a user-centric use case involving Twitter is not inconceivable. Researchers interested in social network analysis, qualitative research, and rare-event topics may eventually turn to Twitter as a data source. Potential methodologies include building a social network out of @mentions (the @ is Twitter lexicon for referencing another user); mining qualitative data from specific user’s accounts; or conducting prospective research by following a person or small group of people over time. These user-centric approaches are fundamentally different from population-level studies, and may require different ethical considerations than aggregated study designs. Additional methodologies might also involve interacting with Twitter users, which will not be addressed here.

Under non-digital circumstances, ethics guidelines suggest that collecting information from a public space where people could ‘reasonably expect to be observed by strangers’ is considered appropriate even without informed consent¹³. According to these guidelines, tweets are text that users publish for the purpose of sharing with others. The weakness of this argument is that it fails to distinguish between population-level research and research focused on selected individuals. It would be clearly unethical for a researcher to follow one specific shopper around the mall and gather data exclusively about them without their consent. However, simply counting or observing behavior in aggregate without consent at a mall is an acceptable research practice. The difference is that the latter example adheres to a level of privacy that the observed individual might expect from being in public, whereas the former violates those natural privacy boundaries. A similar distinction is needed in digital research.

As an example of the potential privacy pitfalls of digital research, suppose investigators were interested in the social networks of adolescents suffering from depression. A research plan might look like this: the investigators gather geocoded tweets that contain words relevant to the topic of interest, as shown in Figure 3. They filter for geocodes that correspond to school locations in order to identify adolescent users. From there, a simple query to the Twitter API returns a list of followers for each of those presumably depressed adolescents. They now have a social network. For each member of the network, they mine the user’s tweet histories to find identifying details such as their real names. The researchers then use the gathered information to ‘snowball’ data collection by curating from a variety of different sources like Facebook, tumblr and the White Pages. They can collect birth dates, cell phone numbers, home addresses, favorite hangout spots, “likes” and “dislikes”, etc. The final result would be detailed demographic information for potentially thousands of people who exhibit symptoms of depression or are connected to a depressed adolescent. Current guidelines do not prohibit this kind of research activity. However, if the same information were collected through surveys or other traditional means, Institutional Review Board (IRB) approval would be needed.

Figure 3. Each dot is a geolocated tweet collected through the Twitter API.

The example tweet displayed is fabricated.

According to the US Department of Health and Human Services Policy for Protection of Human Research Subjects, data that are publicly available are exempt from requiring IRB approval¹⁴. Because Twitter data are public, they technically fall under this exemption. Furthermore, Twitter’s privacy policy makes no secret of the fact that user data are indexed by search engines, archived within the US Library of Congress, and are available through an API¹⁵. However, it is unlikely that many users follow the link to read the lengthy and complex document. One study found that it would take 244 hours a year for an average internet user to read every privacy policy of the unique sites they visit¹⁶.

To help researchers navigate the landscape of ethics in online research, several principles for ethical conduct have been proposed. The Menlo Report¹⁷ was conceived as a complement of the Belmont Report, tailored to information and communication technology research¹⁸. The two reports share three basic principles: respect for people, beneficence, and justice. The Menlo Report adds an additional principle, respect for law and public interest. These principles are meant to inform and guide researchers and ethicists ‘in ethical analyses and self-regulation’¹⁷. The Association of Internet Research published a similar document in 2012 outlining questions and considerations for researchers to weigh¹⁹. Like the Menlo Report, the Association of Internet Research report urges a case by case approach. For more in depth discussions about the role of IRB in digital research, see for example the works of Solberg²⁰ and Buchanon et al.²¹.

Elsewhere in the ethics literature, several themes have emerged. Respecting context, particularly in circumstances when the digital content creators might desire proper attribution, has been central^22,23. The definition and accompanying expectations of ‘public data’ has been discussed variously by others^24–28. The necessity and feasibility of obtaining informed consent, particularly when minors might be in the digital study sample, has also been identified as an obstacle^29–31.

Though useful as foundational documents, efforts to make these ideas accessible to digital researchers have been few. The US Consumer Privacy Bill of Rights (CPBR)³² may serve as a useful framework for outlining best practices for researchers conducting research with Twitter. The CPBR was issued by the Obama administration in February 2012 in order to “give consumers clear guidance on what they should expect from those who handle their personal information, and set expectations for companies that use personal data”. There are seven principles enumerated by CPBR in Table 1. The guidelines proposed here echo many of the concepts previously identified as important, but do so in a way that is actionable (see Figure 1).

Proposed guidelines for the ethical use of Twitter data

Conclusions

Research involving Twitter is growing in popularity, but the issues surrounding the ethics of using it as a data source have not yet been closely examined. There are hypothetical study designs that could use Twitter data in a way that violates the privacy and ethical treatment of participants. In order to avoid those misuses, six guidelines derived from the US Consumer Privacy Bill of Rights are proposed. We welcome discourse in the research community on this topic, and encourage further discussion.

Please use the #TACTICS hashtag on Twitter to participate in this discussion online.