A bibliometric study on the research outcome of Brazil, Russia, India, China, and South Africa

Background: Publication is one of the quantitative measures of countries' contribution to research and innovation. This paper attempts to understand the publication related information of the BRICS countries (Brazil, Russia, India, China, and South Africa). Methods: Detailed analysis of publications on the basis of collaboration, research area, number of publications, percentage of Gross Domestic Product (GDP) spent on research, and citation is presented in the paper. An attempt is also made to understand the relations between each of the parameters and the overall performance of the country. Results: Times Higher Education global ranking is considered as a measure to validate the claims of this paper. This study shows that among the BRICS nations, China with the highest percentage of GDP spent on research has also the highest number of researchers and publication output whereas South Africa excels in terms of number of international collaborative publications and publications in high impact journals. This article has highlighted the distribution of publications as per the subject area with India leading in the area of Computer Science. Discussion: Results showed a strong relationship between each of the parameters discussed on the research performance of a country.


Introduction
The BRICS (Brazil, Russia, India, China, and South Africa) countries are emerging as major influencers of geopolitical changes. BRICS was referred to for the first time in 2001 in a report titled Building Better Global Economic BRICS by Goldman Sachs economist Jim O'Neill. With the success of BRICS over the years, scientific literature has tried to have an improved understanding of these economies. 1 In 2011 a total of 17.3% of research papers in science and engineering were published by BRICS countries compared to just 7.6% in 1995. This accounted for a 233.4% increase in publication output in 2011 compared to 1995. 2 Globally, Brazil ranked 13th in terms of peer-reviewed papers produced between 2011 and 2016. Expenditure on research and development (R&D) as a percentage of Gross Domestic Product (GDP) was 1.17% in 2014 whereas citation impact has remained below average during the same period but improved between 2011 (0.73) and 2016 (0.86), an increase of 18%. 3 Russia had set aside a budget of 70 billion roubles (approx. US $3 billion) in 2018 while doubling its scientific paper output between 2006 and 2016. 4 A total of 74,697 papers (research, review articles, and conference papers) were published by the Russian Federation in Scopus-indexed journals with a compound annual growth rate (CAGR) of 8% between 2006 and 2016, more significantly between 2012 and 2016 with a CAGR of 15.2%, while 54,669 papers were published in Web of Science (WoS) journals with CAGR of 5.9% during 2006-2016 and 12.9% between 2012-2016. 5 Russia and China are predominantly single funding agencies, whereas Brazil and India have diversified funding sources. It was also found that research publications from Russia with foreign funding received more visibility and are published in journals with an average mean impact factor 1.9 times greater than Russian-funded publications in WoS. 6 Russia spends 1% of national income on R&D, whereas public institutions conduct about 75% of research. 7 In recent years, China is second only to the United States (US) in terms of R&D investment. It spent around 2.08% of the national GDP on R&D in 2015. Chinese Academy of Sciences (CAS), Ministry of Science and Technology (MOST), and National Natural Science Foundation of China (NSFC) are the three major funding agencies in the country. 8 Based on recent data, South Africa spent about 0.8% of GDP on R&D.
The government of South Africa also aims to increase this spending to 1.5% of GDP in the ensuing decade. 9 In the fiveyear block of 2013-2017, researchers from South Africa increased their average growth rate in WoS publications by 13.12%, almost double that of the world average of 6.36%. 10 South Africa as a country is ranked 18th in the world in social sciences and arts and humanities; 33rd in life sciences; 38th in physical sciences and 40th in technology out of 198 countries studied. However, the total number of papers published by South Africa during 1995-2016 is approximately 31,651 for social sciences and 37,847 for technology. Though by sheer number the technology domain has produced more papers, the ranking for this domain for SA is way below 18 for social sciences with a much lesser number of papers. Similarly, for the arts and humanities field, with 8,842 papers South Africa's global rank was 18. 11 The journal publication output in 2016 increased by 4.5% compared to 2015, which was still lower than 6.4% during 2014-15. The majority of papers from researchers were in the fields of science, engineering, and technology, which accounted for 58.44% of total publications. 12 R&D expenditure for India tripled in the last decade; from Rs 24,117 crores in 2004-05 to Rs 85,326 crores in 2014-15. However, as a percentage of GDP, the R&D expenditure remained stagnant at 0.7% during the same period. Of total investment in R&D, the public sector has contributed between 0.4-0.5%, whereas 0.2-0.3% came from the private sector. Strikingly for India, the government is also a user of a research fund; apart from playing the role of the largest funder, the major chunk of R&D expenditure is used by the Central government. In addition, to increase nominal R&D expenditure, India's share of total publication output also increased during 2009-2014 with a growth of 14% during the said period; an increase in the share of total global publications rose from 3.1% in 2009 to 4.4% in 2014. India was also ranked 6th in terms of total scientific publication output globally with a growth rate of 13.9% (Scopus) and 7.1% (Science Citation Index -SCI) compared to the world growth rate of 4.4% and 4.1%, respectively. 13,14 The number of researchers per million population increased from 110 in 2000 to 218 in 2015. 14 BRICS represents about 42% of the world's total population. 31.2% of science and engineering doctorates globally are produced by BRICS. 15 BRICS countries together accounted for an impressive 233.4% growth in 2011 over 1995 in publishing science and engineering research papers, whereas the growth rate for other countries was 31.3%. 16 Other than Russia, the output of BRICS countries increased for most frequently cited papers at a higher rate than top-cited countries globally. 17 The co-authored papers among the BRICS countries are found to be countries with which they have "geographical" "cultural" and/or "historical" proximity. 18 The intensity of intra-BRICS countries' collaboration is also low. 19 Further, the strength of each BRICS country is in the different scientific domain in terms of their research output according to the WoS database for the period 2008-2011. Research output from Brazil was primarily in agricultural, plant and animal sciences; for Russia it was physics, chemistry, mathematics and geosciences; India's research outputs were in the fields of pharmacology and toxicology, agricultural sciences, material sciences, and chemistry; while China's publications were in the fields of material sciences, chemistry, and physics. South Africa's research output focused more on the fields of plant and animal sciences, immunology, environment/ecology as well as multidisciplinary research areas. Quantitatively, China is the largest in terms of publication output, whereas South Africa is the smallest among the five BRICS countries. Alternatively, based on Scopus data for the year 2015, China took the lead in the subject areas of mathematics, physics and astronomy, multidisciplinary science, and earth and planetary science. For Brazil, the strongest subject area according to Scopus is dentistry; for Russia, it is physics and astronomy; India is strong in the fields of pharmacology, toxicology, and pharmaceutics; and for South Africa, the strongest subject area is agricultural and biological sciences. 19 Based on Scopus data, the total publication output of BRICS countries in 2015 was 29% with China contributing 18%, India 5%, Russia and Brazil 2.6%, and South Africa 0.72%. 19 From the above discussion, it is clear that existing literature has discussed the need for analysis of research outcomes of a country. From the literature study, it can be seen that BRICS nations are the emerging economies where research publications are gaining over the years. With a motive to understand the research scenarios of BRICS nations this study compares number of researchers and the research outcomes of these nations in terms of research spent, citations, collaborative publications, distribution of publications across different subject areas and percentage of high impact publications.

Background
The ranking of universities has always been a topic of debate, but it is a necessary mechanism to understand the performance of the universities across the globe. There are multiple ranking agencies that rate higher education institutions across the globe such as the QS world ranking, Times Higher Education (THE) world university ranking, Shanghai university ranking, Leiden ranking, etc. THE is one of the widely accepted ranking agencies that ranks the higher education institutions based on four basic parameters, which are teaching, research, international outlook, and industry interaction, out of which research contributes almost 62.5% of the overall scores, thus making it important for the proposed work to analyze the outcomes concerning research. Considering the emerging economies (BRICS), classification is made across the ranking to project the distribution of the number of universities across the entire scale. For this study we have classified the ranking window into five different ranges, which are 1 to 250, 251 to 500, 500 to 800, 801 to 1000, and more than 1000. Figure 1 shows the distribution of universities of BRICS nation across these five ranges for the 2018 ranking and Figure 2 shows the distribution for 2019 ranking. It can be seen that from Brazil, 32 universities have been featured in the 2018 ranking and the number has increased to 36 for the year 2019 ranking. From Russia 27 universities feature in the year 2018, and the number increased to 35 for the 2019 ranking. India had 42 universities ranked in 2018 and the numbers rose to 49 for the year 2019 ranking. There were 63 Chinese universities ranked in the year 2018 and 72 universities ranked for 2019 and finally, from South Africa, eight universities feature in 2018 ranking and nine universities for the year 2019. From the trends, it can be understood that in all the emerging countries the number of universities ranked has increased by a minimum of 10% percent or more as compared to its previous years. From the figures, it can be seen that for the year 2018, universities from South Africa, China, and Russia were featured in the bracket of 1-250 with China leading the group by seven universities, with no universities from India and Brazil in that bracket. In the bracket of 251-500, all the nations have representation, with Russia as the leading country with seven universities followed by China with five universities, then by South Africa with three universities and both Brazil and India with two universities each in this range. The 501-800 bracket is the range in which the maximum number of universities are placed; China leads the list with 32 universities, followed by India with 15 universities, Brazil with eight universities, Russia with five universities, and South Africa with three universities. In the next bracket of 801-1000, China leads by 16 universities followed by India with 13 universities, Brazil with 11 universities, Russia with five universities, and South Africa with one university. In the last bracket of universities ranked more than 1001+ India leads by 12 universities followed by Brazil with 11 universities, Russia with nine universities, China with three universities, and no university from South Africa is ranked in this bracket.
The 2019 ranking also showed a similar trend with some slight modification in the behavior, which is due to an increase in the number of universities. Again, in the bracket of 1-250 only universities from China, Russia, and South Africa are featured with seven, two, and one universities, respectively. The 251-500 bracket has Chinese universities in the lead with seven universities, followed by India with five universities, Russia with three universities, Brazil and South Africa with two universities each. Similarly to 2018, the year 2019 also shows a big number of universities in the 501-800 range with China having 28 universities, followed by India with 16 universities, Russia with nine universities, Brazil with five universities, and South Africa with three universities. The next bracket of 801-1000 also had an increase in number as compared to the previous year, with China having 22 universities, followed by India with 12 universities, Brazil with eight universities, Russia with four universities, and South Africa with one university. The last bracket of 1001+ ranking shows Brazil leading the list with 21 universities, followed by Russia with 18, India with eight universities, and China with eight universities, and lastly South Africa with one. From the comparison of the two years, it can be seen that India and China have improved in numbers and also quality with not much change from Brazil, Russia, and South Africa.
To further understand the ranking of universities we attempt to analyze the research outcomes of the countries, as research constitutes 62.5% weightage in the computation of scores for ranking the universities. Several parameters contribute to research outcomes; a detailed analysis is presented in the following section considering parameters such as the number of researchers, percentage of GDP spend on research, number of publications, citations, collaboration, and area of research.

Methods
To further analyze the reasons for the same we have first looked at the comparison of researchers' population across the countries (Source: https://data.worldbank.org/indicator/SP.POP.SCIE.RD.P6?view=chart), Research and Development spend by % GDP (Source: https://data.worldbank.org/indicator/GB.XPD.RSDV.GD.ZS), and publication data from Scopus/ SciVal. Data related to researcher population for the years 2013 to 2017 is extracted from the world bank data, for computation the average value over the year is taken. Similarly for computation of research spend by percentage GDP is also extracted from the world bank data for the years 2013 to 2017 by taking the average over the period, all the data were extracted from the source site on 18 th December 2019. Publication related data corresponding to country output is extracted from the Scopus website on 18 th December 2019. Parameters related to the citation outcome, subject wise breakup and international coauthored publication details are analyzed from the extracted data. SciVal tool from the Elsevier is used to obtained score related to citation impact, international collaboration as obtained on 18 th Dec 2019. We also analyzed research output on three important parameters, namely researcher per million populations, total scholarly output, and expenditure on R&D as a percentage of GDP. The data on these three parameters were obtained from World Bank indicators. We compared these three parameters against the Field Weighted Citation Impact (FWCI) and its impact on collaborative publications with international authors. Based on three basic parameters, subject-wise analysis for BRICS countries' research output was carried out.

Results
The researcher population is one of the important factors which determines the research strength of the institution or country. To understand the outcome in terms of publication in Scopus indexed publications for five years (2013-17) for the researcher population across the BRICS nations, a graph is plotted in Figure 3. One more important factor which constitutes the growth of research is the percentage of GDP spent on research. Figure 4 shows the comparison of the percentage of GDP spent on research by countries to the citation index of the country (here we consider the FWCI as a citation metric as it is the normalized value of citation across the various subject areas). China has around 2.1% of its GDP spent on research, followed by Brazil with 1.27%, Russia with 1.07%, South Africa  To have a picture of the quantity and quality of research, a graph was plotted in Figure 5 comparing the population of the researchers to the country average FWCI. It can be seen that China produces the highest number of publications, whereas South Africa with the least number of publications has the best quality citation of 1.27. Brazil, India, and Russia's research outcomes and quantity of publications lie in this range.
From the above discussion, it can be seen that the quantity of research output, quality of research, researcher population, and research investment of the country are interrelated. To have a deeper understanding of the rest of the variables, the next section focuses on other parameters like the collaboration and subject area of the research.  Yearly scholarly output in Scopus-indexed publications of BRICS countries is plotted in Figure 6. It is seen that China produces the highest number of publications, which is much higher than the summation of the output of all other BRICS nations. India produces the next highest number of publications, which is almost 20% in comparison to China. Brazil and Russia produce almost the same number of publications. It can be observed that Brazil produced higher numbers of publications in comparison to Russia prior to 2015, but post 2015 the publication output of Russia is higher as compared to Brazil. South Africa produces the least publications as compared to all the other countries, a trend which is almost consistent over the period studied (2012-17). The quantity of publications produced by India constantly increases over the  period 2012-17. As expected for the time period in 2015, the publication output of China was mostly moving in a positive direction. This increase in number may be relative to the increased number of researchers and also the higher research investment.

Collaboration
Collaboration in the publication would reflect the multicenter activity carried out by researchers. It is expected that research executed with multiple partners would have a relatively higher outreach and would lead to higher readership and citation. To understand this, Figure 7 shows the FWCI of BRICS nations in comparison to the percentage of publications with international collaboration for the time period 2012-17. From the figures, it is seen that South Africa has the highest percentage of international collaborative publication (as high as 45%) and also has the highest FWCI as compared to all BRICS nations. On the other hand, India has the lowest percentage of international collaborative publications and FWCI. China, Brazil, and Russia have a similar percentage of papers with international collaboration: 19%, 25%, and 28%, respectively. The FWCI of these nations is 0.9, 0.2, and 0.89, respectively. From Figure 7 it can be observed that India, Brazil, and South Africa have a similar trend. The behavior of Chinese and Russian characteristics is a little deviated, which may be due to other factors.
An attempt is made to understand the distribution of the international collaborative papers published by BRICS among different regions across the globe as shown in Figure 8. For the analysis, the world is divided into six regions, namely Africa, Asia Pacific, Europe, Middle East, North America, and South America. Figure 8 shows the distribution of BRICS collaborative publications among these regions. It is observed that the highest percentage of collaborative publications for Brazil, India, South Africa, and Russia happens from the European region with 58%, 40%, 70%, and 58% of international collaborative publications, respectively. China has the highest percentage of collaborative publications from North America (50%). For the rest of the countries the North American collaborative publication share among collaborative papers is second; Brazil with 43%, South Africa with 38%, Russia with 29%, and India with 38%. With China again a different characteristic is observed; the Asia Pacific collaborative publication share is second with a share of approximately 36%. Based on percentage share of collaborative publications, the Asia Pacific region is in third position for Brazil, South Africa, Russia, and India with 8%, 30%, 26%, and 35%, respectively. European collaborative publication share is seen to be the third largest share of publications for China with 20%. For Brazil, the rest of the publications are shared with South America, the Middle East and the African region, with a percentage share of 15%, 1%, and 0.8%, respectively. In the case of South Africa, the rest of the collaborative publications are from the African region with 23%, followed by South America and the Middle East with almost 8% each. Russia's collaborative publications are shared 9% with the Middle East, followed by 8% with South America and 7.2% with Africa. India's collaborative publications with countries in the Middle East is around 16%, followed by 9% with African countries, and 7.5% with the South American region. Lastly, China has around 2% of papers in collaboration with the Middle East, South America, and Africa. Except for Russia where a large skew is present for European collaborative publication (70%), for the rest of the countries it can be seen that they have a similar sharing percentage.

Relation of BRICS nations
Once we have understood the collaborative publications with the other regions, it is also necessary to understand the significance of collaboration mutually inside the BRICS nations. To understand the same, collaborative publications and their impact in terms of FWCI are plotted in Figures 9 to 13. Figure 9 shows the collaboration of Brazil with other BRICS nations and its impact. Similarly, Figure 10 shows the relation of Russia with other BRICS nations, followed by India in Figure 11, China in Figure 12, and South Africa in Figure 13.     respectively. Russia has around 0.48% of its publications in collaboration with South Africa; these papers yield a FWCI of 7.2. It is seen that Russia has the highest collaboration with China and the papers with the highest FWCI are from South African collaborations, with a FWCI of 7.2. It can be understood that from the above graphs in Figure 9 to Figure 13 that South Africa shares a larger percentage of its publication with other BRICS nations, which is around 9.4% of its total publications. On the other hand, China has the lowest percentage of collaborative publications with other BRICS nations of 0.85%. Collaborative publication with South Africa has a higher FWCI as compared to other BRICS nations.
Publication output in high impact journals Along with collaboration, one more important parameter that also affects the quality of the publication is the choice of journal. For the analysis, the publication of BRICS nations in the journals that are in the top 10th percentile by citation score is compared with the quality in terms of FWCI. Figure 14 shows the outcome of BRICS nations in the top 10th percentile journals. It was found that 24.8% of South African publications are in the top 10th percentile journals with the  highest FWCI of 1.39. Russia has the lowest percentage of publications in the top 10th percentile journals and its FWCI is also lowest at 0.73. India has around 15.5% of its publications in top 10th percentile journals with a FWCI of 0.79. Brazil, with a FWCI of 0.89, publishes 19.1% of its publications in the top 10th percentile journals. China produces around 22.8% of its publications in the top 10th percentile journals with a FWCI of 0.9. From the above discussions, it is clear that collaborative publication and journal quality are factors that would affect the outcome of the publication. Along with these two parameters, one more factor affecting the outcome is the subject area of the journals where papers are published.

Subject area
The choice of journal for publishing a research article is becoming very important as it relates to the outcome of the publication in terms of research visibility and citation. The selection of journals based on subject area is also an important parameter that needs to be considered. As metrics like FWCI are normalized parameters across a given subject area, the outcome of the publication would depend on the subject area. Based on the present-day scenario, journals are categorized into 300+ sub-subject areas by Scopus. For the present analysis, we are considering the broad classification as followed by THE, where all journals are divided into 11 main subject topics, which are 'arts and humanities', 'business and economics', 'clinical, pre-clinical and health', 'computer science', 'education', 'engineering and technology', 'law', 'life sciences', 'physical sciences', 'psychology' and 'social sciences'. In this classification, the available 300+ sub-subject area journals are mapped to these 11 areas. It is to be noted that the journals can be overlapping with one or more classified areas as many journals publish work on multi disciplinarily or interdisciplinarity research areas. Table 1  Arts and humanities: Figure 15 shows the output of BRICS nations in the subject area of 'arts and humanities' for the period 2012-17. China produces a large number of papers in 'arts and humanities', whereas the percentage share of South African papers is the highest. The FWCI of South African publications is the highest with a value of 1.1, followed Business and economics: China produces a large number of papers in the subject area of 'business and economics'. South Africa has a large share of its publications in 'business and economics'. China also produces 28.4% of its publications in the top 10th percentile journals and has the highest FWCI of 1.13. Brazil has around 12% of its publications in the top 10th percentile journals with a FWCI of 0.64. South Africa produces around 10.2% of its publications in the top 10th percentile journals with a FWCI of 0.82. The FWCI of Russian publications is 0.83 with around 6.2% in the top 10th percentile journals. Around 7% of publications from India are in the top 10th percentile journals with a FWCI of 0.75, as evident in Figure 16.
Clinical, pre-clinical, and health: China produces the highest number of papers in the subject area of 'clinical, preclinical and health' which is around 470,000, but Brazilian publications have a higher percentage share in this subject area as compared to other countries as shown in Table 1. South African publications have the highest FWCI quality parameter      top 10th percentile journals, with a FWCI of 0.72. India, with 5.45% of its publications in the top 10th percentile journals, has a FWCI of 0.56. Russia, with 2.5% of its publications in the top 10th percentile journals, has a FWCI of 0.54, and Brazil also has around 2.55% of its publications in journals of the top 10th percentile as indicated in Figure 19.     0.65, which is lowest among all the BRICS nations. It can be seen that in all these nations the quality of the publication is proportional to the number of publications in the top 10th percentile journals as displayed in Figure 22.
Physical science: In the area of physical science, as shown in Figure 23 From the above discussion, it is clear that the quality of publication depends on multiple parameters like collaboration, publication in quality journals, and subject area of publication.

Discussion
Research and innovation are the parameters that often decide the growth of an institute or organization or country. Publication output is one of the measures by which a country's research can be quantified. An attempt was made in this paper to study the behavior of publications of the developing countries Brazil, Russia, India, China, and South Africa (BRICS). To validate the analysis of the work reported, the number and position of universities ranked in THE were treated as a unit of measure. It is evident from the publication that China which has the higher number of publications, research spend, researcher population, has a greater number of universities ranked. On the other hand, SouthAfrica with lesser publication outcome, research spend, etc. has lower number of universities ranked. But it can also be seen that the percentage of universities ranked higher due to higher quality parameters like higher FWCI, international collaboration, etc. Though the number of universities being ranked is higher due to higher researcher volume and publication the quality parameters reflect on the percentage of universities ranked higher. To understand the behavior of publication output, only Scopus-indexed sources were considered as THE considers only Scopus-indexed publications. The first parameter to be evaluated was the population of researchers and it was found that China has the largest population also produces the highest number of publications; the relationship between researcher population and quantity of publications was evident.
The second parameter to be analyzed was the percentage of GDP spent on research and though it was expected that it would be proportional to the quality and number of publications, in reality, it was not so, leading to further analysis to find the other parameters contributing to the outcome. One such measure was collaborative publications, and from this analysis it was evident that South Africa with the most collaborative research had a higher impact than the rest of the BRICS nations. The pattern of collaboration was also studied to understand the geography of collaboration. Results show that South Africa, Russia, and Brazil have more collaboration with Europe, China with North America, and India showed broader collaboration.
The focus area of each of the countries is unique to its culture, geography, and relevance to society. An attempt was made to understand the impact of focus area in terms of the 11 basic subjects on the quantity and quality of research. The nations showed an inclination to a few research areas; for example, Brazil had a large focus toward clinical, preclinical, and life sciences. China and India had a large focus on engineering and physical sciences. Russia showed large publication numbers (more than 50% of its publications) in the areas of physical sciences, followed by engineering with 34%. South Africa showed equal priorities to clinical, life science, and physical science. It was also found that the fewest publications were from law. The paper tried to report the behavioral analysis of research in the BRICS nations.

Data availability
All data underlying the results are available as part of the article and no additional source data are required. I found this article interesting and insightful and particularly well written. The authors have done a noteworthy study to gauge the scientific input and its impact on development in BRICS countries. As such, for this manuscript, the authors have used multiple sources of data namely World Bank, Scopus, and also from Elsevier which is praiseworthy and is quite elaborate. This kind of bibliometric study is needed to bring the focus on core areas of development that will help in 21stcentury nation-building. It also helps to show gaps, and improve research thrust and intensify research pursuits in those areas which lack focus which need some special stimulus to kick start research. These will help governments and policymakers to take note of the direction in which future research should be shaped.

Open Peer Review
In my opinion, if the authors would have correlated this data with some more relevant info on commercialization efforts (patents), the number of millionaires, HNWI, thrown in with data on AI, 5G/6G internet connectivity for good measure it would have been a comprehensive study although quite complicated which would need more time and detailed statistical analysis to pool the results and come to a meaningful conclusion.
Anyway, I congratulate the authors for choosing this study impacting the BRICS.

Is the study design appropriate and is the work technically sound? Yes
Are sufficient details of methods and analysis provided to allow replication by others? Yes