ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

The impact of the open-access status on journal indices: a review of medical journals

[version 1; peer review: 3 approved, 1 approved with reservations]
PUBLISHED 07 Mar 2019
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

This article is included in the Research on Research, Policy & Culture gateway.

Abstract

Background: Over the past few decades, there has been an increase in the number of open access (OA) journals in almost all disciplines. This increase in OA journals was accompanied an increase in funding to support such movements. Medical fields are among the highest funded fields, which further promoted its journals to move toward OA publishing. Here, we aim to compare OA and non-OA journals in terms of citation metrics and other indices.
Methods: We collected data on the included journals from Scopus Source List on 1st November 2018.  We filtered the list for medical journals only. For each journal, we extracted data regarding citation metrics, scholarly output, and wither the journal is OA or non-OA.
Results: On the 2017 Scopus list of journals, there was 5835 medical journals. Upon analyzing the difference between medical OA and non-OA journals, we found that OA journals had a significantly higher CiteScore (p< 0.001), percent cited (p< 0.001), and source normalized impact per paper (SNIP) (p< 0.001), whereas non-OA journals had higher scholarly output (p< 0.001). Among the five largest journal publishers, Springer Nature published the highest frequency of OA articles (31.5%), while Wiley-Blackwell had the lowest frequency among its medical journals (4.4%).
Conclusion: Among medical journals, although non-OA journals still have higher output in terms of articles per year, OA journals have higher citation metrics.

Keywords

Open access, Journal, Medicine, Bibliometrics, Citation

Introduction

Open access (OA) journals allow free (access to/availability of) academic articles, they enable any user to read, search, download, share, use them for indexing, print the full texts, or utilize them as data for software without being charged1. Over the past 20 years, there has been an increase in the number of OA medical journals. According to Web of Science, published OA articles as a proportion of total publications increased from 9.5% to 24% from 1998 to 2018. These OA journals provide an easily accessed source of information, a source that is accessible even for developing and low income countries2.

Bibliometric analysis are methods or applications used to measure the influence of authors or scientific papers, of which, citation analysis is the most commonly used methods3. Now several citation databases have become available, with the three largest being Web of Science, Scopus and PubMed. These databases record the number of times that a journal article has been cited by other papers4. The use of bibliometric analysis is becoming more popular to assess the performance of different aspects of the scholarly and scientific fields. Analysis can be at the level of the researchers themselves, journals, departments, universities, national organizations, and even entire nations58. There are several databases that can be used to perform the bibliometric analysis, with each database having its own characteristics; these include Google Scholar, Pubmed (Only biomedical citations), Scopus, and Web of Science4. According to the number, coverage, and quality of citations covered by the databases, Scopus has wide coverage of high quality journals, compared to high number of citations at the expense of quality for Google Scholar, and high quality at the expense of number of citations for Web of Science911.

It is claimed that the emergence of OA journals has led to better dissemination of knowledge with the additional benefit of more citations for the authors, although this is still a matter of debate12. In this study, we aim to study the OA status of medical journals and the impact of the open-access status on journal indices using the Scopus database.

Methods

Data collection

We collected data on the included journals from Scopus Source List on 1st November 2018 (see Underlying data13). We filtered the list for medical journals (which include all specialties in medicine, as per Scopus categorization).

Variables

For each journal, we extracted the following citation metrics: Citation count, Percent Cited, CiteScore, CiteScore Percentile, SCImago Journal Rank, Source Normalized Impact per Paper (SNIP), and SCImago Quartiles. Details about these metrics and how they are calculated can be found on Scopus website.

Moreover, scholarly output is defined as sum of documents published in the serial title (e.g. 2017) in the 3 years prior to the year of the metric (e.g. 2014 – 16). Open access Journals covered by Scopus are indicated as Open Access if the journal is listed in the Directory of Open Access Journals (DOAJ) and/or the Directory of Open Access Scholarly Resources (ROAD).

Statistical analysis

We used SPSS version 22.0 (Chicago, USA) in our analysis. We used means (± standard deviation) to describe continuous variables (i.e. journal indices). We used counts (frequency) to describe other nominal variables (i.e. publishers and OA journals). We performed Mann-Whitney tests to analyze the difference between measurements and OA status, and we presented data as medians (25% to 75% quartiles). To analyze open access journals between radiology and medicine, we used the weighting cases function in SPSS and a Chi-square test. All underlying assumptions were met, unless otherwise indicated. A p value of 0.05 was considered as significant.

Results

In the 2017 Scopus list of journals, there was 5835 medical journals. Regarding the 5 most common publishers, 890 (15.3%) journals were from Elsevier, 653 (11.2%) Springer Nature, 196 (6.8%) Taylor & Francis, 360 (6.2%) Wiley-Blackwell, and 304 (5.2%) Wolters Kluwer. 1293 (22.2%) journals were OA journals. Table 1 indicates the minimum, maximum, mean, and standard deviation of medical journal indices.

Table 1. Descriptive statistics for medical journals.

NMinimumMaximumMeanStd. Deviation
CiteScore583501301.582.588
Percentile583509948.0328.666
Citation count5835077809761.872221.841
Scholarly output5835111270346.14505.062
Percent cited5835010045.6726.800
SNIP58350.00088.1640.752601.450990
SJR58350.00061.7860.826741.572423
Rank58351.00785.00162.7102167.95247

SNIP: Source Normalized Impact per Paper; SJR: SCImago Journal Rank.

Upon analyzing the difference between medical OA and non-OA journals, we found significant differences in the following indices:

  • CiteScore (p< 0.001): with a median of 1.19 (25–75%: 0.53–2.21) for OA journals, and a median of 1.06 (25–75%: 0.26–2.18) for non-OA journals.

  • Scholarly output (p< 0.001): with a median of 157 (25–75%: 76–319.5) for OA, and a median of 205 (25–75%: 107–423) for non-OA journals.

  • Percent cited (p< 0.001): with a median of 52% (25–75%: 32%-70%) for OA, and a median of 48% (25–75%: 19%–68%) for non-OA journals.

  • SNIP (p< 0.001): with a median of 0.706 (25–75%: 0.370–1.023) for OA, and a median of 0.617 (25–75%: 0.176–1.013) for non-OA journals.

Upon comparing open access journals between the 5 most common publishers, we found a significant difference (p< 0.001). Post-hoc analysis showed that Wiley-Blackwell has significantly lower number of open access journals 16 (4.4%) open access journals compared to others. Table 2 shows the open access status for the most common publishers.

Table 2. A comparison in the percentage of open access (OA) journals between the top five publishers of medical journals.

Open accessTotal
NoYes
publishersElsevierCount756134890
84.9%15.1%100.0%
Springer NatureCount447206653
68.5%31.5%100.0%
Taylor & FrancisCount32472396
81.8%18.2%100.0%
Wiley-BlackwellCount34416360
95.6%4.4%100.0%
Wolters Kluwer HealthCount22975304
75.3%24.7%100.0%
OthersCount24427903232
75.6%24.4%100.0%
TotalCount454212935835
77.8%22.2%100.0%

Discussion

Our study found that OA medical journals had significantly higher CiteScores, Percent cited and SNIP; which is consistent with a number of previous studies made across a variety of disciplines including philosophy, political science, engineering, mathematics, physics, computer science and agriculture; all of which concluded that open access publications have a greater research impact (higher citation rate) than non-open access publications12,1416. On the other hand, in a randomized controlled trial conducted on 11 biological and medical journals, it was found that only 2 of these journals showed positive and significant OA effects. In addition, it was found that OA advantage is declining by about 7% per year, from 32% in 2004 to 11% in 200717. Chua et al. found that there was significantly more citations in OA articles than in non-OA articles within almost identical journals’ impact factor18. Moreover, comparing citations in OA and non-OA articles in the same journal showed significant citation privilege for OA publications in several studies. For example, for the Journal of Postgraduate Medicine a comparison of citations per 100 articles per year before and after the journal became open access showed an increase between 3 and 4.5 times in citations19. In a longitudinal study of a cohort of OA and non-OA articles, it was shown that OA articles are cited earlier, and almost 2 times more frequently than non-OA articles in the first 4–16 months after publication in the same journal20. Regardless of all the aforementioned findings, our study found that non-OA medical journals have significantly higher Scholarly Output which can be strongly linked to the fact that most non-OA medical journals have been established years before OA journals, which have only recently emerged21.

We found that the number of OA journals varied among publishers, with Whiley-Blackwell having the least, with only 16 journals (4.4%), and the most with Springer Nature (206, 31.5%). In a previous study that analyzed OA articles published by different publishers, regardless of the discipline, they found that Elsevier had the highest number of OA articles, followed by Springer Nature and Whiley-Blackwell22. A longitudinal study comparing hybrid open access articles between publishers found great variation depending on the discipline23. For instance, medicine is the discipline which most frequently publishes in hybrid OA23.

Our study has potential limitations. In this study, we didn’t account for the effect of publishing OA articles in non-OA journals (hybrid journals), as “Gold” OA publishing (i.e. fully OA journals) relates to publication of articles that are freely available to view and these may occur in OA or hybrid journals. Moreover, future studies should consider analyzing specialties within medicine (e.g. oncology), where we believe there will be variations in the effect of OA publishing within these specialties.

Data availability

Underlying data

Harvard Dataverse: Medical journals. https://doi.org/10.7910/DVN/YYUTGG13.

This project contains the following underlying data:

  • Medical journals 2017 dataset.tab (Scopus search results from the 1st November 2018)

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 07 Mar 2019
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
AlRyalat SA, Saleh M, Alaqraa M et al. The impact of the open-access status on journal indices: a review of medical journals [version 1; peer review: 3 approved, 1 approved with reservations]. F1000Research 2019, 8:266 (https://doi.org/10.12688/f1000research.17979.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 07 Mar 2019
Views
8
Cite
Reviewer Report 03 Apr 2019
Sung-Tae Hong, Seoul National University College of Medicine, Seoul, South Korea 
Approved
VIEWS 8
General comment: This article describes difference of journal citation metrics between OA and non-OA journals in medical scope. The subjected journals are 5835 medical journals which are indexed in the Scopus and compared indices are metrics data supplied by Scopus ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Hong ST. Reviewer Report For: The impact of the open-access status on journal indices: a review of medical journals [version 1; peer review: 3 approved, 1 approved with reservations]. F1000Research 2019, 8:266 (https://doi.org/10.5256/f1000research.19663.r45999)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
19
Cite
Reviewer Report 03 Apr 2019
Deeb N. Salem, Department of Medicine, Tufts Medical Center, Boston, MA, USA 
Approved
VIEWS 19
  1. The article is generally well done with respect to its method and purpose. My major concern is its purpose: assuming adequate methods, statistical analyses and result, what is the significance?  If there are implications of the research
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Salem DN. Reviewer Report For: The impact of the open-access status on journal indices: a review of medical journals [version 1; peer review: 3 approved, 1 approved with reservations]. F1000Research 2019, 8:266 (https://doi.org/10.5256/f1000research.19663.r46001)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
17
Cite
Reviewer Report 19 Mar 2019
Suneet Sood, Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Johor Bahru, Malaysia 
Approved
VIEWS 17
This paper intends to measure journal indices for OA journals, and compare them with the indices of Non-OA journals. The authors used the Scopus database.
Although the theme is not new, I believe that the large number of indices ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Sood S. Reviewer Report For: The impact of the open-access status on journal indices: a review of medical journals [version 1; peer review: 3 approved, 1 approved with reservations]. F1000Research 2019, 8:266 (https://doi.org/10.5256/f1000research.19663.r45458)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
24
Cite
Reviewer Report 12 Mar 2019
Ernesto Roldan-Valadez, Directorate of Research, Hospital General de Mexico “Dr Eduardo Liceaga”, Mexico City, Mexico 
Approved with Reservations
VIEWS 24
The authors have performed an interesting study comparing differences in selected metrics (CiteScore, percent cited, SNIP, scholarly output) between medical OA and non-OA journals.
This study, however, presents some limitations that require significant revisions:
 
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Roldan-Valadez E. Reviewer Report For: The impact of the open-access status on journal indices: a review of medical journals [version 1; peer review: 3 approved, 1 approved with reservations]. F1000Research 2019, 8:266 (https://doi.org/10.5256/f1000research.19663.r45461)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 07 Mar 2019
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.