Connecting Altmetric: Integrating with Institutional Publications Systems

Tracking attention to research outputs at Altmetric

Altmetrics, or alternative metrics, are increasingly recognised as an emerging signal of research engagement beyond traditional citation metrics (Bornmann, 2014; Dinsmore et al., 2014; Priem, 2013; Thelwall et al., 2013). Altmetrics enable researchers, institutions (Liu & Adie, 2014), publishers and funders (Dinsmore et al., 2014) to measure and report on broader societal engagement with research outputs.

Altmetric, founded by Euan Adie in 2011, monitors attention to research outputs across a range of non-traditional sources including news, policy documents, blogs and social media. Altmetric’s tools, including the Altmetric Explorer, Altmetric for Institutions and Altmetric Badges, support researchers, institutions and organisations in tracking attention to research outputs beyond traditional citations.

In order for altmetrics providers to build reliable and robust tools based upon high quality data, it is important we work closely with the scholarly community and use standardised metadata to describe and connect research outputs. So how does this process work at Altmetric? We are a data science company, and track attention to research outputs with unique identifiers such as Digital Object Identifiers (DOIs), PubMed IDs (PMIDs), Research Papers in Economics (RePEc) IDs and more. In addition, we text mine policy documents and news sources in order to identify mentions of research papers without a persistent identifier or URL in the body of the text, matching the item up with the corresponding record in the Altmetric database. We also ensure mentions of different versions of the same paper are disambiguated in order to present a single unified record for all versions of an item, such as the publisher, PubMed and institutional repository version, presenting all mentions in a single Altmetric Details Page in the database.

Tracking outputs via persistent identifiers

Altmetric’s process of tracking attention to research outputs is further enabled by standardised metadata formats in order to connect mentions for different versions of papers. We track attention to research outputs with persistent identifiers including DOIs, RePEc IDs, arXiv IDs, PubMed IDs, Astrophysics Data System (ADS) Bibcodes, Handle identifiers and unique resource identifiers (URIs). When a link to a research paper is shared across a source of attention that we track, e.g. in a news story or shared on Twitter, we follow the link to the page and search the publication page for item metadata, particularly searching for the persistent identifier such as a DOI. For example, a news story discusses a research paper, and links to the journal article hosted on the publisher homepage. As Altmetric recognises the publisher domain, we follow that link to the content hosting platform, and scrape the persistent identifier and key item metadata; adding that item as a record to our database. The Altmetric technical infrastructure is embedded with an identifier mapping to ensure we recognise and disambiguate across multiple versions. A key challenge is to track a wide range of sources where research is being discussed, while being able to make sense of the disparate mentions of over 3.8 million research outputs as of May 2015.

In order to collect the basic descriptive information about each item we track, Altmetric automatically harvest metadata in Dublin Core meta tags from platforms such as institutional repositories, e.g. DC.title, DC.identifier and DC.creator. For publisher-hosted content, we harvest the HTML “<meta>” tags such as citation_doi and citation_title from the metadata of the item page (see Table 1).

It is important that content creators and publishers assign persistent identifiers and standardised metadata as listed above, exposing this in a machine-readable format across platforms, in order to allow altmetrics providers to recognise and disambiguate mentions of research outputs. Many publishers and institutional repositories already include high quality metadata to enable Google Scholar visibility, as described further in Google Scholar’s Inclusion Guidelines for Webmasters. In the future, we’d like to harvest complex, standardised metadata from publisher platforms to describe research outputs at a more granular level, such as author identifiers, author affiliations, contributor roles and funder information.

Enhancing altmetrics data

Altmetric also work with a number of larger publishers to enhance item metadata mined from content platforms. For example, when we track a mentioned item published by Springer, we query the Springer Application Program Interface (API) with the item DOI and pull the subject classification from their database in order to add subject level metadata at the Altmetric record level. In addition, Altmetric collect journal International Standard Serial Numbers (ISSNs) for each item we track and attach Excellence in Research Australia (ERA) Field of Research (FoR) subject codes. This enables end users to slice Altmetric data by subject at the item level. We would, however, like to see wider adoption of a lightweight standardised subject classification system across publishers and repositories.

Altmetric-ORCID integration

In 2015, Altmetric will be introducing support for ORCID, building on the app (http://altmetric-orcid-profiles.herokuapp.com/) launched during the 2014 1:AM altmetrics conference hackathon. We’ll be adding support to search for ORCID IDs in the Explorer and mine for ORCID IDs as standard during automated metadata harvesting.

Connecting to publications systems and institutional repositories

Collecting, structuring and enhancing altmetrics data is a core part of the work we do behind the scenes at Altmetric. We launched Altmetric for Institutions in 2014 to enable universities and institutions to search, browse and report on attention to their research outputs at the author, department, custom group and item level. As Altmetric already tracks attention to publications across our sources, all we need from the institution is the details of institutional published outputs, associated authors and department groups to structure the altmetrics data according to institutional requirements. In order to build Altmetric for Institutions at the institutional level, we developed a number of connectors to harvest publications information directly from research information management systems, as described in more detail below. This enables us to work with existing technical systems and publications metadata curated by the institution.

Integrating with Symplectic Elements: University of Melbourne

An early Altmetric for Institutions integration was with the research information management system Symplectic Elements. Created by our Digital Science sister company, Symplectic, and based in the same office as the Altmetric team, we built a connector to the Symplectic Elements API in order to pull author, publication and organisational hierarchy metadata through to populate Altmetric for Institutions. The University of Melbourne are an Altmetric early adopter and Symplectic Elements customer. This enabled us to develop the Symplectic Elements connector with the established University of Melbourne Elements instance, which is populated with over 150,000 publications and 26,000 authors. Following an initial import of the above data, we now run regular updates from the University of Melbourne instance of Symplectic Elements to ensure Altmetric for Institutions remains up-to-date with modified or recently claimed publications for all authors. We make use of the persistent identifier metadata fields in Symplectic Elements, which allows us to match publications with the corresponding record and associated attention in the Altmetric database. In addition, the Altmetric badges feature alongside each item record in the Symplectic Elements interface, helping raise the visibility of altmetrics and research engagement with university administrators and researchers in their daily workflows.

Building a VIVO connector: Stony Brook University

Our next major Altmetric for Institutions connector enabled us to harvest publications, author and department metadata from VIVO profiles. Working with Stony Brook University School of Medicine, we built a connector that could pull in the key metadata including relevant authors, associated papers and medical faculties in order to create a searchable, browsable and reportable structure within Altmetric for Institutions. In addition, Stony Brook used the Altmetric API in order to integrate the Altmetric badges in their internal publications system and raise awareness of public attention to research with practitioners and researchers across the organisation.

Harvesting via OAI-PMH: World Bank Group

An important Altmetric for Institutions integration is working with open source repositories to harvest publications information via the Open Archives Initiative – Protocol for Metadata Harvesting (OAI-PMH). In Spring 2015, we worked with the World Bank Group to develop our tools for harvesting via OAI-PMH and mapping across multiple platforms. Firstly, we connected to the World Bank’s Open Knowledge Repository (OKR), which is based on DSpace, via OAI-PMH and this enabled us to populate Altmetric for Institutions with the relevant publications information and connect to the associated altmetrics. In addition, we set up publication tracking for all World Bank published content hosted across their four dissemination platforms: OKR, World Bank eLibrary, Documents and Reports Archive, and RePEc. As World Bank Group publications are often made available across all of these platforms, it was key that we were able to recognise different versions of the same item.

In order to set up a process of disambiguation for the World Bank collection, we worked with the World Bank team to harvest relevant identifier mappings added to their repository feed and create this mapping within the Altmetric database. As a result, when a World Bank Group paper from any of their platforms is shared across the sources we track, we will disambiguate within the database and ensure all posts are merged for that item. The World Bank Group identifier mapping infrastructure includes Handle IDs, RePEc IDs, DOIs and internal World Bank IDs, demonstrating how important it is that persistent identifiers are used across publications systems in open formats to enable mapping and reliable altmetrics tracking. In addition, the World Bank embedded the Altmetric badges in both OKR and World Bank eLibrary in order to showcase attention to their published research.

Practical applications of altmetrics

Having discussed several integrated aspects of the Altmetric technical infrastructure, it is important to consider the practical applications of altmetrics in institutions. Altmetrics are used in practice across the scholarly communications lifecycle (Madjarevic & Davies, 2015) – from librarians, research offices and communications teams, to individual researchers and faculty heads. By analysing the underlying qualitative data – the mentions and discussions – we can further understand successful research dissemination, support researchers making the case for grant funding or promotion and provide a broader, more coherent understanding of the potential impact of research on society.

A key institutional use case is surfacing conversations surrounding research outputs beyond traditional citation analysis, conversations that may have previously been unknown to the institution or researcher, in order to collect evidence of indicators of impact for funders or to present during promotion reviews. By surfacing this qualitative data, Altmetric helps organisations identify potential impact in sources such as policy documents. For example, an intervention review paper produced by University of Melbourne researchers and published in Cochrane Database of Systematic Reviews (Boyle et al., 2008), studies the use of probiotics for treating eczema. It found that probiotics were not an effective treatment. The paper has a relatively low Altmetric score of 13, but received mentions in policy documents from the European Food Safety Authority and the Royal College of Paediatrics and Child Health and is cited on the main Wikipedia page for Dermatitis. This demonstrates the importance of analysing the underlying qualitative mentions beyond score in order to identify indicators of research impact. Furthermore, the World Bank Group used Altmetric for Institutions to track attention to research in developing nations, a key target community for their publication outputs. In a recent World Bank Group report, Addressing Inequality in South Asia (Rama et al., 2014), the publications team were able to observe geographical trends in sharing activity across South East Asia in the Twitter demographics, as visualised on the Altmetric Details Page (Figure 1).

Figure 1. Altmetric Details Page for the World Bank Group report, addressing inequality in South Asia.

Additional key use cases include: identifying successful author impact activities to recognise institutional best practice in promoting research impact; encouraging researchers to include in grant applications, CVs and author profiles; enhancing library liaison services; encouraging staff to deposit in research information management systems; integrating data in performance reports; and identifying popular research to further promote.

Improvements to the scholarly metadata across platforms

Here at Altmetric, there are a number of improvements we’d like to see across the scholarly publications and repository communities to drive altmetrics future developments and enable robust, interoperable data. This includes the broad adoption of the following metadata as standard across publisher platforms and repositories:

The addition of the above metadata fields to research outputs across publisher platforms and repositories, opens opportunities for the altmetrics community to further develop tools and data analysis in robust and innovative ways.