Connecting Altmetric: Integrating with Institutional Publications Systems [version 1; peer review: 1 approved, 1 approved with reservations]

This paper discusses Altmetric tools for institutions and how they are supported by accurate, up-to-date and re-usable research information. We examine the importance of standardised metadata formats across research information management systems in enabling altmetric providers to deliver data in robust, reliable and meaningful ways. We share our experience of collaborating with a range of institutions to report and analyse the attention to their collection of research outputs and surfacing altmetrics data at the author, department and institutional level. This includes working with institutions to ensure we can harvest from or integrate with existing technical infrastructure in order to match outputs with the corresponding altmetrics data in the Altmetric database. We discuss integrations with institutional repositories and publications systems including Symplectic Elements, VIVO and DSpace. Finally, we study motivations for incorporating metrics into workflows and systems across institutions, and how altmetrics can be integrated with existing research support and bibliometrics services

Altmetric, founded by Euan Adie in 2011, monitors attention to research outputs across a range of non-traditional sources including news, policy documents, blogs and social media.Altmetric's tools, including the Altmetric Explorer, Altmetric for Institutions and Altmetric Badges, support researchers, institutions and organisations in tracking attention to research outputs beyond traditional citations.
In order for altmetrics providers to build reliable and robust tools based upon high quality data, it is important we work closely with the scholarly community and use standardised metadata to describe and connect research outputs.So how does this process work at Altmetric?We are a data science company, and track attention to research outputs with unique identifiers such as Digital Object Identifiers (DOIs), PubMed IDs (PMIDs), Research Papers in Economics (RePEc) IDs and more.In addition, we text mine policy documents and news sources in order to identify mentions of research papers without a persistent identifier or URL in the body of the text, matching the item up with the corresponding record in the Altmetric database.We also ensure mentions of different versions of the same paper are disambiguated in order to present a single unified record for all versions of an item, such as the publisher, PubMed and institutional repository version, presenting all mentions in a single Altmetric Details Page in the database.
Tracking outputs via persistent identifiers Altmetric's process of tracking attention to research outputs is further enabled by standardised metadata formats in order to connect mentions for different versions of papers.We track attention to research outputs with persistent identifiers including DOIs, RePEc IDs, arXiv IDs, PubMed IDs, Astrophysics Data System (ADS) Bibcodes, Handle identifiers and unique resource identifiers (URIs).When a link to a research paper is shared across a source of attention that we track, e.g. in a news story or shared on Twitter, we follow the link to the page and search the publication page for item metadata, particularly searching for the persistent identifier such as a DOI.For example, a news story discusses a research paper, and links to the journal article hosted on the publisher homepage.As Altmetric recognises the publisher domain, we follow that link to the content hosting platform, and scrape the persistent identifier and key item metadata; adding that item as a record to our database.The Altmetric technical infrastructure is embedded with an identifier mapping to ensure we recognise and disambiguate across multiple versions.
A key challenge is to track a wide range of sources where research is being discussed, while being able to make sense of the disparate mentions of over 3.8 million research outputs as of May 2015.
In order to collect the basic descriptive information about each item we track, Altmetric automatically harvest metadata in Dublin Core meta tags from platforms such as institutional repositories, e.g.DC.title, DC.identifier and DC.creator.For publisher-hosted content, we harvest the HTML "<meta>" tags such as citation_doi and citation_title from the metadata of the item page (see Table 1).
It is important that content creators and publishers assign persistent identifiers and standardised metadata as listed above, exposing this in a machine-readable format across platforms, in order to allow altmetrics providers to recognise and disambiguate mentions of research outputs.Many publishers and institutional repositories already include high quality metadata to enable Google Scholar visibility, as described further in Google Scholar's Inclusion Guidelines for Webmasters.In the future, we'd like to harvest complex, standardised metadata from publisher platforms to describe research outputs at a more granular level, such as author identifiers, author affiliations, contributor roles and funder information.

Enhancing altmetrics data
Altmetric also work with a number of larger publishers to enhance item metadata mined from content platforms.For example, when we track a mentioned item published by Springer, we query the Springer Application Program Interface (API) with the item DOI and pull the subject classification from their database in order to add subject level metadata at the Altmetric record level.In addition, Altmetric collect journal International Standard Serial Numbers (ISSNs) for each item we track and attach Excellence in Research Australia (ERA) Field of Research (FoR) subject codes.This enables end users to slice Altmetric data by subject at the item level.We would, however, like to see wider adoption of a lightweight standardised subject classification system across publishers and repositories.

Altmetric-ORCID integration
In 2015, Altmetric will be introducing support for ORCID, building on the app (http://altmetric-orcid-profiles.herokuapp.com/)launched during the 2014 1:AM altmetrics conference hackathon.We'll be adding support to search for ORCID IDs in the Explorer and mine for ORCID IDs as standard during automated metadata harvesting.
Connecting to publications systems and institutional repositories Collecting, structuring and enhancing altmetrics data is a core part of the work we do behind the scenes at Altmetric.We launched Altmetric for Institutions in 2014 to enable universities and institutions to search, browse and report on attention to their research outputs at the author, department, custom group and item level.
As Altmetric already tracks attention to publications across our sources, all we need from the institution is the details of institutional published outputs, associated authors and department groups to structure the altmetrics data according to institutional requirements.In order to build Altmetric for Institutions at the institutional level, we developed a number of connectors to harvest publications information directly from research information management systems, as described in more detail below.This enables us to work with existing technical systems and publications metadata curated by the institution.
Integrating with Symplectic Elements: University of Melbourne An early Altmetric for Institutions integration was with the research information management system Symplectic Elements.Created by our Digital Science sister company, Symplectic, and based in the same office as the Altmetric team, we built a connector to the Symplectic Elements API in order to pull author, publication and organisational hierarchy metadata through to populate Altmetric for Institutions.The University of Melbourne are an Altmetric early adopter and Symplectic Elements customer.This enabled us to develop the Symplectic Elements connector with the established University of Melbourne Elements instance, which is populated with over 150,000 publications and 26,000 authors.Following an initial import of the above data, we now run regular updates from the University of Melbourne instance of Symplectic Elements to ensure Altmetric for Institutions remains up-to-date with modified or recently claimed publications for all authors.We make use of the persistent identifier metadata fields in Symplectic Elements, which allows us to match publications with the corresponding record and associated attention in the Altmetric database.In addition, the Altmetric badges feature alongside each item record in the Symplectic Elements interface, helping raise the visibility of altmetrics and research engagement with university administrators and researchers in their daily workflows.
Building a VIVO connector: Stony Brook University Our next major Altmetric for Institutions connector enabled us to harvest publications, author and department metadata from VIVO profiles.Working with Stony Brook University School of Medicine, we built a connector that could pull in the key metadata including relevant authors, associated papers and medical faculties in order to create a searchable, browsable and reportable structure within Altmetric for Institutions.In addition, Stony Brook used the Altmetric API in order to integrate the Altmetric badges in their internal publications system and raise awareness of public attention to research with practitioners and researchers across the organisation.
Harvesting via OAI-PMH: World Bank Group An important Altmetric for Institutions integration is working with open source repositories to harvest publications information via the Open Archives Initiative -Protocol for Metadata Harvesting (OAI-PMH).In Spring 2015, we worked with the World Bank Group to develop our tools for harvesting via OAI-PMH and mapping across multiple platforms.Firstly, we connected to the World Bank's Open Knowledge Repository (OKR), which is based on DSpace, via OAI-PMH and this enabled us to populate Altmetric for Institutions with the relevant publications information and connect to the associated altmetrics.In addition, we set up publication tracking for all World Bank published content hosted across their four dissemination platforms: OKR, World Bank eLibrary, Documents and Reports Archive, and RePEc.As World Bank Group publications are often made available across all of these platforms, it was key that we were able to recognise different versions of the same item.
In order to set up a process of disambiguation for the World Bank collection, we worked with the World Bank team to harvest relevant identifier mappings added to their repository feed and create this mapping within the Altmetric database.As a result, when a World Bank Group paper from any of their platforms is shared across the sources we track, we will disambiguate within the database and ensure all posts are merged for that item.The World Bank Group identifier mapping infrastructure includes Handle IDs, RePEc IDs, DOIs and internal World Bank IDs, demonstrating how important it is that persistent identifiers are used across publications systems in open formats to enable mapping and reliable altmetrics tracking.
In addition, the World Bank embedded the Altmetric badges in both OKR and World Bank eLibrary in order to showcase attention to their published research.

Practical applications of altmetrics
Having discussed several integrated aspects of the Altmetric technical infrastructure, it is important to consider the practical applications of altmetrics in institutions.Altmetrics are used in practice across the scholarly communications lifecycle (Madjarevic & Davies, 2015) -from librarians, research offices and communications teams, to individual researchers and faculty heads.By analysing the underlying qualitative data -the mentions and discussions -we can further understand successful research dissemination, support researchers making the case for grant funding or promotion and provide a broader, more coherent understanding of the potential impact of research on society.
A key institutional use case is surfacing conversations surrounding research outputs beyond traditional citation analysis, conversations that may have previously been unknown to the institution or researcher, in order to collect evidence of indicators of impact for funders or to present during promotion reviews.By surfacing this qualitative data, Altmetric helps organisations identify potential impact in sources such as policy documents.For example, an intervention review paper produced by University of Melbourne researchers and published in Cochrane Database of Systematic Reviews (Boyle et al., 2008), studies the use of probiotics for treating eczema.It found that probiotics were not an effective treatment.
The paper has a relatively low Altmetric score of 13, but received mentions in policy documents from the European Food Safety Authority and the Royal College of Paediatrics and Child Health and is cited on the main Wikipedia page for Dermatitis.This demonstrates the importance of analysing the underlying qualitative mentions beyond score in order to identify indicators of research impact.Furthermore, the World Bank Group used Altmetric for Institutions to track attention to research in developing nations, a key target community for their publication outputs.In a recent World Bank Group report, Addressing Inequality in South Asia (Rama et al., 2014), the publications team were able to observe geographical trends in sharing activity across South East Asia in the Twitter demographics, as visualised on the Altmetric Details Page (Figure 1).
Additional key use cases include: identifying successful author impact activities to recognise institutional best practice in promoting research impact; encouraging researchers to include in grant applications, CVs and author profiles; enhancing library liaison services; encouraging staff to deposit in research information management systems; integrating data in performance reports; and identifying popular research to further promote.

Improvements to the scholarly metadata across platforms
Here at Altmetric, there are a number of improvements we'd like to see across the scholarly publications and repository communities to drive altmetrics future developments and enable robust, interoperable data.This includes the broad adoption of the following metadata as standard across publisher platforms and repositories: • Author identifiers, e.g.ORCID IDs attached to all authors and records.
• Persistent identifiers as standard for all research outputs, including grey literature and data sets.
• Departmental groupings and sets to enable organisational analysis.
• Funder metadata to identify research papers produced as a result of external funding, e.g.FundRef.
• Subject taxonomy, namely the adoption of an agreed subject classification system.
• Standardised final date of publication available and distinct in each item record.The addition of the above metadata fields to research outputs across publisher platforms and repositories, opens opportunities for the altmetrics community to further develop tools and data analysis in robust and innovative ways.

Conclusion
Altmetrics providers are keen to work with scholarly publishing and institutional repository communities to develop the quality and breadth of persistent identifiers and standard metadata fields.During Altmetric's work to develop automated item tracking processes, version disambiguation techniques and integrating with a range of publication systems, we identified a number of benefits and potential improvements to the existing practices as described above.
However, an overwhelming finding is by connecting Altmetric to existing institutional publication systems, builders of altmetric tools are able to offer low barrier entry to altmetrics at the institutional level to enable deeper impact analysis and uncover conversations about research across broader society.

Philip Young
Newman Library, Virginia Tech, Blacksburg, VA, USA This article is informative and well written overview of how the Altmetric product integrates with institutional platforms.The sketch of how Altmetric tracks research in the Introduction I found particularly useful.It is worth noting that the author is an employee of Altmetric, but I feel this is acceptable for an opinion article.I agree with reviewer 1 that the article suffers somewhat from a focus on the company and could be strengthened in some areas by a broader perspective.hese do not need to be significant changes, but can be brief mentions outlined below.
First, on the topic of standardizing metadata formats, areas of consensus among altmetrics providers might be noted wherever possible, rather than focusing solely on what Altmetric would like to see.Also, NISO's current efforts might be acknowledged here.
Second, under the heading "Practical applications of altmetrics," the statement beginning "Altmetrics are used in practice aross the scholarly communications lifecycle..." might be qualified, as it seems to overstate the extent to which altmetrics are currently in use (which is avoided in the cited article).Granted, this information may be proprietary, but it would be useful to know how widespread altmetrics use is.
Third, an significant issue of institutional interest that might be touched upon is the relationship between altmetrics and open access (this came to mind in the discussion of the World Bank reports).This seems to be a missed opportunity given the discussion of repositories and altmetrics (though there is brief mention of the use case "encouraging staff to deposit in research information management systems").As with numerous studies on the OA citation advantage, do OA articles receive more mentions?Can Altmetric data tell us which articles (or which versions) are OA and which are not?
Fourth, I agree with reviewer 1 that some criticisms or limitations of altmetrics might be admitted.It seems unlikely, for example, that altmetrics would provide useful data in all disciplines.
I think these four issues could be addressed without significant revisions and would improve this paper.
A problem (which I do not expect the author to address here) with Altmetric is the prominent display of scores in the "donut" with a simultaneous downplaying of that number.For example, the author says "The paper has a relatively low Altmetric score of 13…." and then emphasizes "the importance of analysing the underlying qualitative mentions beyond score…" If it is more important to look at the sources, then the score should be done away with so it does not become subject to abuse like the journal impact factor (one reason that I prefer the displays of other altmetrics providers).As with the JIF, quantitative measures are too tempting to our short-cutting, convenience-minded human nature.
Competing Interests: I am a member of the Altmetric ambassador program.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.My review suggestions for this article all center around strengthening that separation for this article and making sure it provides more than just advertisement for a particular company and gives at least some sense of a broader product category as well as a bit more on that product category's disadvantages as well as its advantages.I don't believe that this article should "be about something else" just that it needs to provide a bit more diversity of ideas and opinions to make it a little less lopsided.
One way to address this issue earlier in the process would have been to collaborate on authorship of the article (and presumably the conference presentation it is based on) with someone at one or more of the institutions that participated as case studies.
Introduction, the author goes through some of the strengths of altmetrics to give a more balanced sense of whether or not altmetrics (and Altmetrics) are worth the investment for institutions.The author should also address some of the criticisms of altmetrics that are quite common in the literature.The Bornmann article in the references also addresses disadvantages and they could have been explored.
As well in the introduction, some mention could have been made of other companies that offer personal or institutional altmetrics solutions, such as Impact Story.
The criticisms brought up in the introduction could have also been addressed in the Practical applications of altmetrics section of the article.This would make the whole article stronger, especially as opinion rather than advertisement.There is some mention of taking the Altmetrics score in context of other measures in the section, but these could be expanded.
Finally, one small suggestion for wording.In the conclusion the word "finding" is used in relation to the conclusions drawn from the case studies."Findings" usually relate to research rather than opinion.
One thing I would also like to see is a few more illustrations/screenshots from the various systems that Altmetric built with their partner institutions.
Competing Interests: No competing interests were disclosed.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com

Figure 1 .
Figure 1.Altmetric Details Page for the World Bank Group report, addressing inequality in South Asia.

Reviewer
Report 21 July 2015 https://doi.org/10.5256/f1000research.6995.r9375© 2015 Dupuis J.This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.John Dupuis Steacie Science & Engineering Library, York University, Toronto, ON, Canada Overall, this article very strong, and is certainly publishable.The nature of the Opinion-style article gives the author considerable leeway compared to a research-based article.However, when the article is by an employee of a company and it describes that company's products, the line between Opinion and Advertorial can be thin.This article certainly does a good job of describing what altmetrics are, what the positives aspects of their use are as well as how Altmetric's offerings demonstrate those advantages.