National culture as a correlate of research performance and impact [ version 2 ; peer review : 2 approved with reservations ] Previously titled : National culture as a correlate of research impact and productivity

National culture has been overlooked in discussions related to research performance and impact owing to individual, socio-political structure, and economic factors. This study shows the relationships between the dimensions of cultural value orientation of the nation and research performance & impact. More than 60 countries were included and Spearman correlation analysis was employed. The variables were taken from Geert Hofstede and Scimago Journal & Country Rank worksheets. This study found that (1) Power distance the positive inclination of the culture toward power disparities among people is negatively correlated with research impact; (2) Individualism the level of independence a society keeps up among its individuals  are positively correlated with research performance and research impact; (3) Indulgence the degree to which society members do not attempt to control their urges is positively correlated with research impact; and (4) after controlling the Log GDP per capita, uncertainty avoidance the  manner in which that a society seeks to manage the actuality that the future can never be controlled is negatively correlated with research impact.


Introduction
Makri (2018) recently released a report on the increasing number of publications in various countries. She stated that it's unclear what has triggered and driven the strong gains in Egypt and Pakistan. Throughout the report, various variables believed to be responsible for the increasing number of publications, such as indexation duration, funding, global engagement, international collaboration, and political policies on science and higher education, are explained.
Several predictors of research performance and impact had been identified, i.e. author characteristics, co-authorship networks, citation history, journal impact factors, tweets (Xiaomei et al., 2017), cohort effects (in terms of scientific discipline), age, career stages, gender, the country of origin of the PhD holders, and reward structure of the research enactment (Claudia & Francisco, 2007). They are mostly at the individual and institutional level. At the country level, the predictors are the number of universities, GDP per capita, control of corruption, civil liberties (Mueller et al., 2016), country's wealth and population size, country's value of research tradition, tenure and promotion criterion, experimental costs, IRB (Institutional Review Boards) review flexibility, language barrier, and the training of new young researchers (Demaria, 2009).
However, national cultural orientation (in this paper, the term is used interchangeably with: national culture, national cultural value, national culture dimension) is yet to be analyzed, with the present study assuming that individual, institutional, and structural factors are also influenced by the cultural values of a nation. Hofstede Insights (2019) defined culture as the collective mental programming of the human mind which distinguishes one group of people from another, consisting of six dimensions, i.e. (1) power distance (PDI) -acceptance on the unequal power distribution in a society; (2) uncertainty avoidance (UAI) -intolerance of ambiguity and uncustomary thoughts and practices; (3) individualism (IDV) -projection of individuals' "I" in a society rather than "we" (collectivism); (4) masculinity (MAS) -the toughness and competitiveness rather than the tenderness and cooperativeness (femininity) orientation; (5) long term orientation (LTOWVS) -the society's preference of time-honored rather than pragmatic approaches (short term normative orientation); and (6) indulgence (IVR) -the society facilitation towards a fun and enjoyable life rather than restraint (suppression of needs gratification by strict social norms).
National culture is relatively stable (Maseland & van Hoorn, 2017) and is widely used to explain various performances at the country level, such as learning and academic performance (Signorini et al., 2009). The present study hypothesized that there are correlations between the national culture dimensions and research performance indicators. The research performance is assumed to be mediated by research culture, and the culture experiences stimulations and challenges from the national culture.

Methods
All following data were retrieved on August 18, 2019, and compiled into a worksheet (see Underlying data (Abraham, 2019) as the material of this present analysis. Countries' region, total documents/DOC, citable documents/CITA, citations/CIT, self-citations/SELF, H-index/HINDEX, and citations per document/CPD (1996- Principal component analysis (PCA) and Independentsamples Kruskal-Wallis H Test were done using IBM SPSS Statistics version 25 for Windows to get two major components from dimensions reduction of DOC, CITA, CIT, SELF, HINDEX, and CPD, as well as comparison between countries' regions in terms of the reduced dimensions. Correlation analysis was conducted using JASP version 0.10.2 for Windows, and Partial correlation analysis was conducted using IBM SPSS Statistics.

Results
The data analysis results are presented in Table 1.

Discussion
The purpose of this study is to show whether there are correlations between national cultural values and research performance and impact. Because correlation is not causation, the following analysis and interpretation do not attempt to state definitively that there is a causal effect from one variable to another. Even though in this discussion cultural value orientation is often used as an explanation of research performance and impact, this is more due to the chronological flow that culture comes and envelops, engulfs a country first than the SCIMAGOJR measures. The argument is in line with the proposition of Sen (2004) that culture is a constituent of development and economic behavior, as expressed as follows:

Type Display Results
Descriptive statistics of SCIMAGOJR measures Table 2 The DOC, CITA, CIT, SELF, CPD, HINDEX data are not normally distributed.
Principal component analysis of SCIMAGOJR measures Table 3 & Figure 1 There are 2 components extracted with a total variance explained 92.073%, namely: • Component 1: "Research Performance" (a synthesis of DOC, CITA, CIT, SELF, HINDEX) • Component 2: "Research Impact" (based on CPD alone) The correlation between Components is weak (< 0.2). It might be that CPD is more difficult to manipulate or be an object of the author's engineering.
Comparison between regions (Kruskal-Wallis H Test result)  Table 5 The data is not normally distributed; therefore, correlation analysis was done with Spearman's correlation.
Plots of national cultures, research performance, and Log GDP per capita (1) Individualism, long term orientation; (2) Long term orientation, individualism; (3) Individualism, uncertainty avoidance; (4) Long term orientation, power distance; (5) Masculinity, indulgence. For countries with the lowest research performance, there is no data available on their national cultural orientation.
Plots of national cultures, research impact, and Log GDP per capita The complete data (six cultural orientations) available are from (1) Belgium, and (2) United Kingdom. Descriptively, in each of these countries, the cultural orientations that play roles the most and the least are, respectively: (1) Uncertainty avoidance, power distance; (2) Individualism, uncertainty avoidance (as well as power distance). For countries with the lowest research impact, there is no data available on national cultural orientation.
Spearman's correlation Table 6 The results, with q (adjusted p) < 0.00714, are: • PDI is negatively correlated with Research Impact • IDV is positively correlated with Research Performance and Research Impact • IVR is positively correlated with Research Impact • LGDP is positively correlated with Research Performance Partial correlation (controlling LGDP) Table 7 The directions of correlation between variables are the same as the results of Spearman's correlation above, but there is an additional new result: • UAI is negatively correlated with Research Impact   "The furtherance of well-being and freedoms that we seek in development cannot but include the enrichment of human lives through … forms of cultural expression and practice, which we have reason to value …. Cultural influence can make a major difference to work ethics, responsible conduct, spirited motivation, dynamic management, entrepreneurial initiatives, willingness to take risks, and a variety of other aspects of human behavior which can be critical to economic success." (pp. 39-40).
In other words, culture can influence public policy which regulates human capital; whereas, research performance and impact depends on human capital, in addition to the fact that research is a contributor to economic growth and development (Blanco et al., 2015). However, this study is cautious for not trapping itself in cultural determinism.

The limitation of SCIMAGOJR data
There are a number of things that need to be stated from the beginning of this discussion, namely: Firstly, the SCIMAGOJR data ( Secondly, in a number of dimensions of research performance and impact measurement (Table 3, Figure 1), Scopus, which supplies the data of SCIMAGOJR, has a number of limitations; for example (1) Scopus has poor coverage of articles, conference papers, and book chapters compared to Crossref, Dimensions, Google Scholar, and Microsoft Academic; (2) Scopus is somewhat late in indexing in-press articles compared to all four; (  Research performance across regions Based on comparisons between regions (Table 4), over the past 23 years (1996-2019) (1) both Eastern Europe and Middle East have better research performance than Latin America, Pacific Region, and Africa; (2) Asiatic region has better research performance than Pacific Region. However, from the aspect of research impact, Latin America outperforms the Middle East, Asiatic Region, and Eastern Europe. Those findings show that research performance and research impact (Table 5) are not always directly proportional, they can even be inversely correlated ( Figure 2, Figure 3).
Eastern Europe's superiority in terms of research performance may be due to the rise of democracy, the emergence of the need for research excellence standards, the promotion of international research collaboration, and cooperation with international bodies (such as the World Bank) that enable these countries to enjoy large research grants ( "Though not a uniform phenomenon across all disciplines or countries, some participants noted that in CEE (Central and Eastern Europe) research tends to be more dependent on political power. This can relate both to the partisan provision of financial resources and to researchers' ambitions to convince political actors." It appears that political activities are melting pots of the interests of academics, politicians, and research funders, which provide work opportunities that has implication in improving research performance in the region's countries. Those interests are given "energy" by the belief of the people that "Our people are not perfect, but our culture is superior to others." (Kim, 2018, para. 6).
Makri's finding (2018) regarding the progressive research achievement of Egypt confirmed the finding that the Middle East has been able to surpass Latin America, the Pacific Region, and Africa in terms of research performance. Different from Eastern Europe, the performance in Middle East has drivers centripetalized on the publishing business. Although some of the Middle East countries are plagued with protracted conflict (Gul et al., 2015), Habibzadeh (2019) noted that there is a "meeting point" between the career interests of faculty members in universities and the business interests of publishing in the countries. This is exacerbated by the relaxation of the promotion standard of faculty members, so that a surge in publication occurs in Scopus indexed journals-that grow rapidly quantitatively in those countries, but of which many have transformed into predatory ones. Habibzadeh (2019, p. 4) conveyed more about the phenomenon: "Recently, some indexing systems, like Scopus, have also pursued the same strategy and delisted some of the low-quality journals published in the Middle East and Iran. Although some of the editors and publishers of the delisted journals have     attributed these events to political issues, to be honest, I, for one, believe that in most instances, they, themselves, should bear the brunt of the situations they have for their poor work quality." (p. 4) Noteworthy is the fact mentioned by Plackett (2015), that: "The predatory journal industry exists on a spectrum-at one end, some such journals maintain they are conducting valid peer review. At the other end of the spectrum, predatory journals sometimes blackmail academics who eventually realize they've published in a journal with a negative reputation." (para. 21) That is, the issue of predatory journals in the Middle East is not an easy problem to evaluate. This argument is reinforced by Jones' (2015) argument, that the flourish of predatory journals is not the real problem. Only with slow, careful, detailed analysis, concern, and empathy even can be liberated from the old ways of seeing" (p. xxiii). These qualities may attract citations repeatedly. This explanation, nevertheless, is still speculative and requires testing in subsequent empirical studies.
National culture and research performance and impact Power distance (PDI) has a negative correlation with research impact ( The positive correlations between individualism (IDV) and research performance and impact could be explained using the findings of Deschacht & Maes (2017). They found that in countries with more individualistic cultures: (1) the scientists prioritize their self-development, (2) the records of scientific work are historically longer (usually Western countries), and (2) self-citations flourish more. This does not necessarily mean that there have been citation abuses, but that self-citation is used to refer to their prior works, thereby, preventing unnecessary repetitions of ideas in newer works (Deschacht, 2017). Although IDV and collaboration are often contested (e.g. Kemp, 2013), a "collaborative individualism" (Limerick & Cunnington, 1993) -stressing both working together and self-emancipation -is possible, explaining the positive correlation.
Indulgence (IVR) is positively correlated with research impact; this may be because IVR -the warranted one -facilitates academic freedom (Ohmann, 2011), as stated by Jefferson (2011) regarding psychological gratification, "Difference of opinion is advantageous … [F]ree inquiry must be indulged, and how can we wish others to indulge it while we refuse it ourselves" (p. 26). Conversely, a restraint (as opposed to indulgence) will facilitate the destruction of goal pursuit, e.g. designing and executing impactful studies, through psychological reactance and unwarranted indulgence (Buzinski & Price, 2015).
Sabbatical leave is a representative example of warranted IVR that faculty members could increase research impact through  IVR may also manifest itself in a "lovely" academic writing style (Kiriakos & Tienari, 2018). This style is not dry and cold, but rather dialogical, humanistic, more reflexive, and capable of showing authors' courage and vulnerability. Compelling insights are more easily born from the writings that embody those qualities; as mentioned, "a thin line exists between interesting insights and self-indulgence" (Nadin & Cassell, 2006, p. 214). Scientific authors who read such works would be attracted to cite them, leading to an increase in the works' impact. In addition, "strategic indulgence" is possible and known to be a creative process that enables one to balance academic activity (such as writing) with non-academic ones (Jia et al., 2018) -fostering insight.
Uncertainty avoidance (UAI) is found negatively correlated with research impact (Table 7). This is understandable considering that impactful research requires innovation. The characteristics of UAI -which are intolerant of ideas and practices that are ambiguous and not conventional -do not support innovation (Bauer & Suerdem, 2016). Uncertainty avoidance cultural orientation is difficult to challenge and scrape unfunctional attitudes and values that are already stable. Therefore, it will also be hard to produce breakthroughs in research and publication, reducing the potential for citations per document.
One premise advocated by Leiden Manifesto for Research Metrics is "Science and technology indicators are prone to conceptual ambiguity and uncertainty and require strong assumptions that are not universally accepted" (Hicks et al., 2015, para. 21). Higher UAI national culture would adhere to the invariance assumption that is detrimental to the development of science and publication real impact. Un-openness to the pluralistic approach in the impact measurement will invite citation cartels. Citations per document (CPD) will be seen reductionistically as the destination of scientific works, so that CPD will be easy to become a target of manipulation. In fact, we have been reminded that the production of knowledge and its memories must not forget the relevance of knowledge to diverse publics. What is needed is a "careful and conscientious citation ... [citation as] a form of engagement", in which "citation as a crude measure of impact" is only the byproduct of the reflexive action (Mott & Cockayne, 2017, p. 2, 11). It will need lower UAI.

Conclusion
National culture dimensions, especially power distance, individualism, indulgence, and uncertainty avoidance are pivotal variables that are to be considered in justifying research impact. In addition, the only variable that correlates with research performance is individualism.
Owing to the fact that the national culture is relatively enduring, countries need to measure their elasticity of hopes and action plans in an effort to boost research performance and impact, by integrating the national culture in the estimate. National culture can be integrated as a moderating variable in the predictive relationship between GDP per capita and research performance and impact. Diversification of this study -based on the document and authors' collaboration types, the indexing databases, the disciplines, as well as the history and development of the research in a country -is a future opportunity for further study. The discussion has been almost entirely rewritten based on this, including a section on limitations of the data and analyses ○ I note here that Reviewer 2 has also provided some additional comments in their second review. I agree with both of these points, that the labelling of the PCA plots should be checked and made more appropriate, and that the results section would still benefit from a combined text and table presentation. I think this would entail just re-adding some of the deleted text from the previous version. I think that keep the results and discussion sections distinct though would be beneficial, as there is a lot to digest here. The present version is a bit weird with all of the results condensed into one table, referring to further results which are only then first present in the discussion.
Additional comments Looking at the data again, I am still a bit worried that this involves time series data, but they are not being treated as such. Thus, the results are most likely suffering from autocorrelation or long-term impacts of background trends, and require further statistical analysis to be sure of this. I am really only familiar with time series analysis for palaeontological data though, see for example here: https://www.nature.com/articles/ncomms12737#Sec11 1 -I am a bit suspicious that because of this problem, many of the correlation coefficients reported might be artificially higher than what is realistic. I think that this needs to be very carefully considered here. My apologies for not indicating this in the first report, but I could not see the data to check this.

○
The limitations section might work better after the rest of the discussion ○ My expertise on the intersection between politics and research is quite limited, and I will refrain from commenting on those elements of the discussion. Although they are, at least to me, very interesting! ○ As much of the discussion again is based on the results, which I suspect might change given my recommendations for the methods above, I will refrain from commenting on them too much at the present. From what I can gauge though, they seem to be well thought out, contain relevant literature, and do not oversell the results too much (in their present state).

○
My apologies for asking for potentially more analytical work to be done at this stage. I feel that it is necessary to look at the data through the lens of time though to better understand some of the results being obtained here. Keep up the great work for now! Second, in the principal component analysis, I don't think the first component should be labeled 'research performance'. The term 'research performance' is very general and could mean many different things. Therefore, 'research performance' is not a very helpful label for the first component. The essential difference between the two components is that the first one consists of size-dependent variables (i.e., variables that increase with the number of publications of a country) while the second component consists of a size-independent variable (i.e., a variable that is independent of the number of publications of a country). The first component could be labeled 'research output' (since all variables depend on the size of the research output of a country), while the second one could be labeled 'research impact'.
this is the case, the use of Pearson correlation analysis could easily lead to misleading results. The use of for instance the Spearman correlation may then be more appropriate.
The author relies strongly on statistical significance testing. My recommendation is to leave out all significance tests and instead to present confidence intervals for the correlation coefficients.
Significance testing leads to problematic dichotomous thinking, as has for instance been pointed out in a recent contribution in Nature (Amrhein et al. 1 ). Following the so-called estimation statistics approach, reporting confidence intervals is preferable over reporting significance tests ( https://en.wikipedia.org/wiki/Estimation_statistics). I am aware that another reviewer (Tennant) recommends performing even more significance tests. I disagree with this recommendation. I don't consider this to be good statistical practice.
It would be nice if the author could deepen the analysis a bit more. This can for instance be done by showing scatter plots for the most interesting relationships between variables. In these scatter plots, the names of countries could be shown, especially for those countries that seem to display interesting behavior (e.g., outliers). This would lead to a more in-depth analysis that probably offers richer insights.
The paper uses lots of abbreviations. This makes the paper more difficult to read. My recommendation is to reduce the number of abbreviations that are used. It may also be helpful to include a table listing all abbreviations and the corresponding full terms.
The author interprets the analysis in terms of correlation instead of causation. This is very good. There is a risk, however, that some readers may give causal interpretations to the findings of the author. My suggestion is to add a sentence at the end of the paper emphasizing that causal interpretations are not warranted.
Yes present report well within that. Not sure if the comment about China at the end of the Introduction adds too much here.

Materials and methods
So the methods are pretty simple, which is nice. But also, I think perhaps a bit too simple here given that you're performing a lot of bivariate analyses, and a couple of extra steps are recommended. First, you want to perform an assessment of normality for data series prior to any correlation analyses, using the Shapiro-Wilk test (e.g.,shapiro.test function in R). From the output, if the p-values are greater than the pre-defined alpha level (traditionally, 0.05) this implies that the distribution of the data are not significantly different from a normal distribution, and therefore you can assume normality and use Pearson's test (Pearson's product moment correlation coefficient [r]). If p > 0.05, you should instead perform a non-parametric Spearman's rank correlation (ρ). ○ Secondly, once you've done this, for each test, report both the raw and adjusted pvalues. The latter can be calculated using the p.adjust() function, and using the 'BH' model (Benjamini & Hochberg, 1995 1 ). This method accounts for the false-discovery test when performing multiple hypothesis tests with the same data set, which can inflate type-1 error (i.e. in order to avoid falsely rejecting a true null hypothesis; a false positive). What this will probably do is reduce the 'significance' of some of your results too (which is why it's best to report both the raw and adjusted values).
○ ○ In addition to this, it seems like you have multivariate data, so multivariate analyses might be more informative here. I would strongly recommend performing a Principal Components Analysis on your data (perhaps just only with the variables with more complete data), and inspecting that as a compliment to the bivariate ones. This is fairly easy to do and display using in built functions in R.

Results
I expect that the results will change a bit given my above recommendations to the methods, so won't comment too much on them at this stage. The nice thing about PCA though is that it produces good summary plots, which might be useful here.

○
In the text, can the country abbreviations be given to make reading a bit easier? ○ M, SD, and N I think need explaining here too. Lots of acronyms can get a bit confusing! ○ Discussion and conclusions As above, I don't want to comment too much on the Discussion and Conclusions at the present, as I think the above recommended methods will change some of the interpretations. However, at the present there seems to be a logical progression between reported results and conclusions.

○
Congratulations to the author on a great and interesting piece of work. I would be happy to see a revised version of this too if needed.

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate? Partly Are all the source data underlying the results available to ensure full reproducibility? Yes

Are the conclusions drawn adequately supported by the results? Yes
Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Palaeontology, Open Scholarly Communication
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com