Academia ’ s Big Five : a normative taxonomy for the epistemic responsibilities of universities

This paper proposes a normative taxonomy by which universities can express the extent to which they meet five core epistemic responsibilities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge and understanding. The core epistemic responsibilities, which we call the Big Five, are to (1) foster research integrity, (2) teach for intellectual virtue, (3) address the big questions of life, (4) give humanistic inquiry and education a proper place, and (5) serve society. The paper characterizes the Big Five in some detail and explains why they are core epistemic responsibilities of universities. The paper concludes by describing the steps that should be taken in order to test, amend, and implement the taxonomy.


Introduction
In this paper, we propose a normative taxonomy of what we call the 'Big Five' in academia: five core epistemic responsibilities of universities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge, understanding, insight, rationality, and explanation. Thus, they are to be distinguished from moral responsibilities -that concern the wellbeing of humans and animals -and practical responsibilities, such as the responsibility to develop societally useful technologies and effective medical interventions 1 . For each epistemic responsibility, we distinguish five levels describing the extent to which a university meets that responsibility or strives to do so. Our taxonomy is meant as a tool to assess the degree to which a university meets its core epistemic responsibilities. The format we use is inspired by the Transparency and Openness Promotion (TOP) guidelines 2 that journals can use to describe the extent to which they meet the goals of Open Science.
The taxonomy proposed here is a product of two research projects funded by the Templeton World Charity Foundation: Science beyond Scientism (2013)(2014)(2015)(2016) and The Epistemic Responsibilities of the University (2016-2019) 3 . The first project explored which questions science can and which ones it cannot address, as well as whether the natural sciences are the only reliable source of knowledge. The second project explored what the core epistemic responsibilities of universities are, given various contemporary challenges, such as hypercompetition, publication pressure, the marginalization of the humanities, and the commercialization of the university. In both projects, philosophers worked in close cooperation with biomedical and social scientists. We present this as work in progress, as a starting point for gaining experience with using the taxonomy and consensus building for a more mature version.
This paper is structured as follows. First, we provide the taxonomy by specifying academia's Big Five epistemic responsibilities and detailing five levels of meeting them (Table 1). After that, we argue that these are indeed five core epistemic responsibilities of universities (section 2). Finally, we lay out which future steps we aim to take to test, amend, and implement the taxonomy (section 3).

A normative taxonomy
Our proposal distinguishes five epistemic responsibilities. We consider each to be of equal importance, so the order in which we present them is not hierarchical. The attainment levels I through V for each responsibility, however, are meant hierarchically: each level presents a more advanced stage of meeting the responsibility at issue.
We think of these responsibilities as attaching primarily to entire universities. So, in order to meet them, each responsibility should have systematic consequences, throughout the university and its faculties, departments, institutes, or other organizational parts. If a university strives to teach for intellectual virtue at the highest level, for example, students throughout the university should be instructed in what these virtues are and stimulated to cultivate them through virtue-building teaching activities.
This doesn't mean that there cannot be an internal division of labour when it comes to meeting epistemic responsibilities: giving humanistic inquiry a proper place will primarily fall on humanities departments, although humanities scholars will teach in other departments and collaborate with scientists in other departments as well when a university strives for level V of this responsibility. Similarly, not all departments and research teams will have to serve society in the same way or to an equal degree. There will be significant differences between, say, the theoretical physicists and the nutrition scientists.
We will now clarify each responsibility briefly and motivate why it belongs on the list of epistemic responsibilities of universities.

To foster research integrity.
Research integrity is fostered by getting rid of perverse incentives, stimulating good mentoring, having an open research climate, and so on. Detrimental research practices include both rare major research misbehaviours like fabrication of data and highly prevalent minor misbehaviours like selective reporting 4 . By 'responsible conduct of research' we mean behavior that meets the principles and standards for good research, as laid out in major codes of conduct for research integrity 5 . Such behavior can be stimulated at the level of individual scholars, but also that of groups, such as research teams or departments. Ideally, research integrity is promoted for both individuals and groups.
The results of scientific and scholarly research play a crucial role in modern society. Universities carry out a substantial part of this research and educate and train researchers who perform the studies and apply the results. To ensure the validity and trustworthiness of findings research needs to be performed according to the principles and standards for research integrity. In recent years it has become painfully evident that there is substantial room for improvement in the level of compliance to these principles and standards 6 . Surveys indicate that 2% of researchers admit to having fabricated or falsified data themselves, while one third says that they have been engaged in less serious questionable research practices 7 . Only 10 -40% of study results turn out to be reproducible when the study is repeated 8 . This is due to small sample sizes, selective reporting and other questionable research practices. Although these phenomena are understudied, it is clear that more transparency 9 and specifically preregistration 10 of study protocols and data analysis plans will lead to considerable improvements. This epistemic responsibility entails not only the adoption of transparency and educating staff and students in responsible conduct of research, but also removing perverse incentives from the ways in which researchers are assessed 11 and performing research on research to strengthen the evidence base for optimizing research integrity.

To teach for intellectual virtue.
Among the intellectual virtues are: open-mindedness, attentiveness, charitableness, intellectual courage, creativity, curiosity, discernment, honesty, intellectual humility, objectivity, parsimony, perseverance, studiousness and wisdom 12 . Educating for intellectual virtues is part of the traditional ideal of Bildung. Moral virtues, such as generosity, kindness, or benevolence, may well also be important and universities might have a responsibility to cultivate them, but, if so, this isn't an epistemic responsibility. When universities choose to teach for intellectual virtues, they can do so, minimally, by merely bringing up these virtues in educational settings; or they can explore them through case studies of intellectually virtuous scientists and scholars and their contributions to epistemic progress, or, most advanced, they can actively cultivate intellectual virtues in their students by having them mimic and practise intellectually virtuous behaviour. Developing intellectual virtues requires not merely instruction about virtues or reflection on them, but training and exercise 13 .
Teaching for intellectual virtues is a responsibility of universities almost by definition. Intellectual virtues are broadly understood as qualities that make someone a well-trained thinker or inquirer 14 . Hence, the two main tasks of universities, research and teaching, are both served by teaching for intellectual virtue. By aiming to train students to become skilled thinkers (perhaps among other things), universities set an ambitious goal for education. Similarly, academic research also needs skilled thinkers. Moreover, the intellectual virtues are widely relevant outside the university: in politics, journalism, medicine, law, law enforcement, social work -it's hard to think of any sphere of life where good, analytical, critical, clearheaded, creative, or otherwise high quality thinking wouldn't matter.

To address the big questions of life. By 'the big questions'
we mean such questions as: What is the origin and ultimate destination of all that exists? What is the future of the earth's ecosystem? What is consciousness? Do humans have free will? Is there (objective) good and evil? Can the human mind understand the world and, if so, how? Does life have meaning? Does God exist? How does science relate to religion? These are universal questions in the sense that they have been asked in most cultures and societies throughout history, up to the present; they are not restricted to local concerns or specific academic disciplines.
Addressing the big questions of life is an epistemic responsibility of the university for a number of reasons. First, these questions are too important to be left entirely to non-academics. Second, many academics themselves are greatly interested in these questions, even if their specialized science and scholarship does not or even cannot answer them. Third, clarifying and attempting to answer these questions affects how we look at ourselves and what we deem important. Fourth, several big questions are factual questions -highly abstract and large scale, but factual nonetheless. They are not questions about tastes or preferences. They seem to admit of true or false answers. So, at least in principle, they fall within the purview of scientific and humanistic inquiry. God's perfection and omnipotence led Galileo to assume that mathematics is our best guide to understanding the orbits of the planets and hence that heliocentrism rather than geocentrism is correct 16 . Darwin asked whether humans are unique or whether all life on earth is monogenetic, which led him to develop evolutionary theory 17 . Einstein opposed the Copenhagen interpretation of quantum mechanics on metaphysical grounds, as he didn't believe the universe could be fundamentally probabilistic 18 . For these reasons universities cannot and should not operate as if the big questions don't exist or cannot be taken seriously. Rather, they should take them seriously and mobilize their intelligence as well as state of the art science and scholarship to address these questions in an intellectually responsible way.

To give humanistic inquiry and education a proper place.
Universities ought to have room for the full range of academic disciplines, in both the sciences and the humanities. Many of today's most urgent challenges in society cannot be solved by purely scientific or technological means; successful solutions require compelling communication, consideration of moral values and norms, and in-depth understanding of cultures and religions. All of these things, and much more, are studied in the humanities. Hence, universities ought to facilitate and embrace humanistic inquiry and teaching, and stimulate interdisciplinary collaborations between scientists and humanities scholars. Of course, some universities -technological universities, for instance -do not include humanities departments. For them, this responsibility may be either left aside, or be interpreted as follows: the responsibility to give due weight to knowledge and understanding produced by humanities departments in other universities.
Giving the humanities a proper place is a responsibility of the university for one basic reason: the humanities can deliver truth, knowledge and insight in areas where the sciences cannot 19 . The humanities have their own objects of study: they study objects that have "meaning" in a special sense, viz. meaning that derives from human conventions, from human intentions, and/or from human purposive behaviour 20 . The knowledge and understanding the humanities provide differs from the knowledge and understanding that the sciences offer, in that the former is often 'indexical' (that is, related to human interests and concerns), 'perspectival' (it specifies how things look from, say, a romantic perspective), and value-related (that is, related to social, political, moral, aesthetic, or religious values) 21 . In addition to this, the humanities are particularly suitable for educating students to become well-informed, critical citizens who can reflect on socially urgent questions about life, health, education, justice, equality, liberty, etc. and participate fully in society and politics 22 .

To serve society.
Universities can serve society at a number of levels: local (a city or region), national, or international, humanity worldwide. What we have in mind here is serving society epistemically, that is, to help society acquire true belief, knowledge, and understanding about important issues. So, what we have in mind are such things as press releases, expert advice, popular articles, opinion pieces, public lectures, interviews, and so on. Of course, universities sometimes also serve society in a more practical manner, e.g., by way of proposing effective policies, producing medical interventions and other technologies. Such practical interventions are often based on scientific evidence, so the epistemic and the practical are not entirely separate, but they can be distinguished for analytical purposes. We focus on the former here, as our taxonomy concerns the epistemic rather than the moral, practical, or social responsibilities of universities. This is a responsibility for at least two reasons. First, many scientific and scholarly discoveries are so complex that if academics do not disseminate their knowledge, those discoveries will remain unknown among the larger audience. Second, it often requires extensive academic knowledge to understand the importance and ramifications of various discoveries. Knowledge and understanding are of intrinsic epistemic value. If the university does not serve society by sharing academic knowledge and understanding, then, for much academic knowledge and understanding, that value will be attained only by a very small group of academics in the relevant field. If the university takes its epistemic responsibility of knowledge dissemination seriously, then much larger groups -academics in other fields, society as a whole -will attain those epistemic values.

Future steps
As indicated, we propose our normative taxonomy as a tool to assess the degree to which a university meets the Big Five epistemic responsibilities. Our proposal is a first attempt; in future work we aim to validate, test, amend, and implement the taxonomy. We envision doing this in four consecutive steps.
First, we want to fine-tune our taxonomy in a Delphi Study 23,24 with international experts that aims in its first round at adding, 16  replacing, and reformulating various epistemic responsibilities. The second and third Delphi rounds will seek consensus on the corresponding levels of meeting the responsibility at issue and explore what the best practices are for reaching higher levels of specific responsibilities.
Next, we will organize co-creation workshops 25 with representatives of relevant stakeholders in order to discuss a penultimate version of the taxonomy. The focus of the workshop will be on the operationalization of the levels of meeting the different epistemic responsibilities in a way which makes application of the taxonomy feasible, transparent, and as objective as possible.
Then, we will test and qualitatively evaluate the taxonomy in a number of universities, resulting in a definitive description of the responsibilities, the levels, and a toolkit of best practices.
Finally, we will publish and disseminate the results on a dedicated website and explore whether the taxonomy is a suitable alternative for, or addition to, the currently dominant Academic Ranking of World Universities 26 and the Times Higher Education World University Rankings 27 .

Data availability
No data is associated with this article.

Grant information
Work on this paper was funded by the Templeton World Charity Foundation and was carried out within the project The Epistemic Responsibilities of the University (2016-2019). The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Templeton World Charity Foundation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
My main comments pertain to the taxonomy proposed in Table 1. I would not argue with the five epistemic responsibilities put forward; however, I did wonder whether five levels of taxonomy are required for any or all of the five proposed responsibilities. Some of the descriptions provided for levels I and II seem unlikely to occur in practice, and I could not always see clear distinctions between some levels proposed, most frequently for levels I and II.
For example, for "1. To foster research integrity", I could not see a clear difference between "Issues of detrimental research practices and responsible conduct of research are neglected" (level I) and "Issues…. are occasionally addressed" (level II). There may not be much difference in practice between something being "neglected" or "occasionally addressed", and it is also not clear how "occasionally" would be reliably measured. Similarly, I could not see a clear difference between level III "Detrimental research practices are addressed… but there is no academic climate that actively stimulates responsible conduct of research" and level IV "There is an academic climate that detects and acts upon detrimental research practices and actively stimulates responsible conduct of research, but only at the level of individual researchers".
For "2. To teach for intellectual virtue", I couldn't see clear differences between level I "the university…. pays no attention to intellectual virtues" and level II "intellectual virtues are considered important, but not taught". I also could not imagine any University that would fall into either of these categories.
Again, I couldn't see clear differences for the proposed levels I and II for "3. To address the big questions of life", ie these questions being "neglected" (level I), versus "mentioned but discarded" (level II).

2.
For "4. To give humanistic inquiry and education a proper place", I also saw no real distinction between the humanities being "marginalized" (level I) and "not marginalized but considered and treated as being inferior" (level II). Levels III, IV, and V commonly refer to humanities being given a "proper place" in the university, but this definition will naturally vary between universities according to their type, i.e. technical universities versus universities with a strong and stated focus upon the humanities. Some wording within levels III-V also lacked definition, i.e. "in isolation" (level III), "some other disciplines" (level IV), and "across disciplines" (level V). "Disciplines" can be defined quite narrowly by some universities, so working with other disciplines or across disciplines does not always imply working between the humanities and the sciences.
Finally, for "5. To serve society", universities where "research and teaching are confined to …purely academic challenges" (level I) could argue that they are serving society in this way, possibly by pursuing academic challenges that can indirectly linked to the big questions of life. It also seems unlikely that any university would be defined by level II "Research and teaching identify societal challenges, but the university leaves it to others to confer knowledge … relevant to those challenges".
Are all factual statements correct and adequately supported by citations?
Contemporary challenges such as hypercompetition, publication pressures, marginalisation of the humanities and commercialization of universities (page 3, paragraph 2) should be supported by references.
The status of reference 1 (footnote 1) which is a manuscript by Peels et al, could be updated (page 3).
Are arguments sufficiently supported by evidence from the published literature?
Although the manuscript states that "more transparency and specifically pre-registration of study protocols and data analysis plans will lead to considerable improvements (in the use of small sample sizes, selective reporting and other questionable research practices, from the preceding sentence), this seems to be somewhat of an overstatement. Firstly, more transparency and pre-registration may not impact on the use of small sample sizes, as these might represent feasibility constraints that pre-registration may not overcome. Secondly, I understand that there is not yet a substantial body of literature that describes the possible benefits and/or drawbacks of study pre-registration, by comparing the results and outcomes of pre-registered studies versus comparable studies that lacked pre-registration.
Finally, the manuscript states that (the big questions of life) "are too important to be left entirely to non-academics". This statement could be written in a more inclusive way, for example by indicating that as these questions are so important, they are everyone's concern, including the concern of academics.

Are the conclusions drawn balanced and justified on the basis of the presented arguments? Yes
No competing interests were disclosed.

Competing Interests:
Reviewer Expertise: Molecular biology, molecular genetics, human tissue biobanking, research integrity I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Author Response 02 Jul 2020 , Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Rik Peels
Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled that was recently funded by the Templeton World Epistemic Progress in the University (2020-2023) Charity Foundation. JB's remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.
Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.
Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: "Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements." We didn't consider this to be a particularly strong claim, but we have now changed it into: "Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements." Finally, we have adopted JB's suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: "The big questions are everyone's concern, academics included." No competing interests were disclosed. This article proposes what Peels call a "normative taxonomy" of the "Big Five" epistemic et al. responsibilities universities ought to meet: (1) fostering research integrity, (2) teaching (for) intellectual virtues, (3) addressing the big questions, (4) valuing the humanities, and (5) serving society. This normative taxonomy essentially serves as an designed to assess the performance of analytical rubric universities. One could easily imagine -and in fact, Peels propose as much in §3 -refining the et al. rubric presented in Table 1, using it to assess the performance of various universities, and then comparing their "Big Five" scores to their world university rankings. We agree that undertaking these tasks presents many enticing possibilities, especially to those of us inclined against bean counting approaches to accountability. Nevertheless, since they position themselves as aspiring university rankers, they should address the range of critiques targeting existing rankings, their methodologies, and the performativity of these rankings. They should then situate their own approach within that context, arguing especially that it is an improvement on -and escapes the criticisms levelled at -existing rankings. This is our global concern.
Below, we offer a number of other substantial concerns, followed by a shorter list of minor issues to consider: Why these five? Peels propose five epistemic responsibilities of universities as Big Five. The authors point out et al. the that they consider each of the Big Five equally important. They also defend each of the five as indeed responsibilities that universities ought to meet. However, they fail to argue that these five epistemic responsibilities are the most important epistemic responsibilities for universities, that these five are in fact equally important, that these five are sufficient to capture the epistemic responsibilities universities ought to meet, or that universities have no other equally worthy responsibilities that ought to be included.

Why five?
We agree that the proposed Big Five are epistemic responsibilities that universities ought to meet. Using the proposed rubric to evaluate universities might provide valuable information about particular areas of strength and weakness at a given university. However, designing the rubric around five separate epistemic responsibilities suggests that universities could meet each of their epistemic responsibilities to varying degrees. One university might be particularly strong in terms of fostering research integrity but fail miserably at teaching for intellectual virtue. Another might excel at addressing the big questions but do a lousy job of valuing the humanities. Although these possibilities sound plausible, using an prima facie analytic approach to epistemic responsibilities also generates some less than desirable outcomes. Could a university that fails to teach for intellectual virtue really excel at fostering research integrity? That seems plausible only if we have a very limited procedural view of what counts as fostering research integrity. Without valuing the humanities, how could a university excel at addressing the big questions? Unless we adopt scientism, the answer seems to be that it could not. Similar problems arise when we consider the relations between the other proposed epistemic responsibilities. The use of a taxonomical vocabulary relations between the other proposed epistemic responsibilities. The use of a taxonomical vocabulary further strengthens this worry. Taxonomies should display not only differences, but also relationships and similarities. The current form of representation is not, in that sense, a taxonomy, but (for the most part) a normative . Only when the ties binding the elements together are conceptualised convincingly can we list convincingly speak of a taxonomy. For this reason, we suggest that the authors consider enriching their list to raise it to the level of a taxonomy or abandon talk of a 'taxonomy' altogether. The authors could adopt the vocabulary of an analytical rubric, instead.
Even if the authors decide to use the language of rubrics rather than taxonomies, we suggest that the authors consider a more holistic approach along with an analytic one, just as virtue ethics appeals not only to the individual virtues, but also to the question of character. Rather than describing teaching for intellectual virtue as part of , perhaps the authors could consider fostering the ability to pursue Bildung while cultivating as analogous to a university's character. Call it the rubric, Wissenschaft Bildung W-B under which different (now, no longer separate) epistemic responsibilities could be grouped together to indicate different levels of overall achievement. In order for a university to score best on the rubric, it W-B would have to excel at meeting of its epistemic responsibilities (for a brief discussion of the difference all between analytic and holistic rubrics, as well as examples and further references, see Rubrics: useful ). assessment tools. Centre for Teaching Excellence, University of Waterloo

Why insist on separating epistemic from other responsibilities universities have?
This question follows on the previous one, in that it aims to nudge the authors away from using only an analytic approach. By insisting on distinguishing epistemic responsibilities from moral, practical, and social responsibilities, the authors impoverish the idea of what it means for universities to serve society. If the goal is to work towards alternative forms of assessment and accountability (thereby politicizing them further), excluding other responsibilities and artificially fragmenting the landscape accordingly, one would expect [a] the epistemic responsibility taxonomy to be situated within a larger responsibility taxonomy and/or [b] an argument that leaving other responsibilities aside is preferable. More in line with the suggestion to pursue a more holistic approach, we must ask: could we really say that a university is a good university if it meets only its epistemic responsibilities? We think not. Perhaps the rubric we W-B suggest, or another holistic approach, could help address this issue. Note that one could also include analytical rubrics for moral, practical, and social responsibilities of universities and incorporate the responsibilities included in each of these separate rubrics into a holistic approach.
Our point is not that analysis is always bad. Indeed, analysis can be quite informative and useful as part of a formative evaluation. We are simply suggesting that the authors consider using their analytic approach to complement a more holistic approach.

Who counts, and why do they count?
In §3, the authors suggest optimising their rubric by means of a Delphi Study "with international experts", which could lead to fine-tuning it, but possibly also to radical revisions. If the latter is the case, then what is the status of the current five epistemic responsibilities? When it comes to the actual Delphi study, whom they choose to participate in the study will likely make a very big difference to the final design of the rubric. Presumably, the authors focus on "experts" because they assume experts know more about epistemic responsibilities than non-experts do. Yet, the way in which the authors present it suggests a retreat to the deficit/diffusion model of public understanding of science. Not only has the deficit model been shown to be factually incorrect, it also presupposes a social contract for science and scholarship that imagines universities as ivory towers. In their discussion of a university's responsibility to serve society epistemically, the authors focus on the supply side of knowledge production, suggesting that knowledge dissemination is how best to serve society. Arguably, however, serving society -even if we limit this to an epistemic responsibility -means something other than telling society what we academics think they need 1 1.
epistemic responsibility -means something other than telling society what we academics think they need to know. Here again, the separation between epistemic and other responsibilities creates a lot of friction, since a departure from the deficit model requires a high degree of interaction and participation beyond universities. In fact, one could argue that interaction and participation are epistemic requirements (co production of knowledge).
Along these lines, we suggest that non-experts may have valuable feedback to offer, even if the authors ultimately decide only to pursue the development of an analytic rubric for epistemic responsibilities of universities. One way to include the demand side of knowledge production in the design of their rubric would be to recruit non-expert stakeholders to participate in the proposed Delphi Study and workshops or organise parallel expert (Delphi) and citizen (Citizen Summit) consultations.
We also have a few more minor points that nevertheless warrant attention: Despite the argument that the rubric is to be applied to entire universities, the responsibilities focus very much on individuals and groups (which seem to be multiple individuals in the ways in which they are discussed) and less on the level of structures and collectives.
Peels frame irreproducibility solely as the result of sub-par science and thus as a research et al. integrity issue. Their previous work, as well current scholarly debates on the characters and qualities or irreproducibiluty and irreplicability, takes up a much more nuanced position. Perhaps the authors will consider adding some of that nuance here.
Excusing a few universities (technical universities or polytechnics, in this case) of taking responsibility for one of the epistemic responsibilities (#4) suggests that giving humanistic inquiry and education a proper place is optional. This seems to conflict with the idea that all responsibilities are equally important. It also opens up possibilities for policies that deprioritise humanities research and teaching (cf. the current report in NL). Van Rijn References 1. Rubrics: Useful Assessment Tools.

Centre for Teaching Excellence, University of Waterloo Reference Source
Is the topic of the opinion article discussed accurately in the context of the current literature? Partly

Are the conclusions drawn balanced and justified on the basis of the presented arguments? Yes
The reviewers are currently engaged in an ongoing debate with the authors of the Competing Interests: paper on whether the humanities need a replication drive like that currently ongoing for the sciences. Since the reviewers take the negative side of that debate -we do not think the humanities need a replication drive -and the authors defend the affirmative side, someone might believe we are incapable of offering a fair review. The debate is cordial, however, and one of the authors (Peels) has been included as replication drive -and the authors defend the affirmative side, someone might believe we are incapable of offering a fair review. The debate is cordial, however, and one of the authors (Peels) has been included as a presenter in a session conference session on the topic organized by the reviewers. We expect that (fruitful) debate to continue, although it is possible that we could reach consensus on the matter. Although we offer a critical review here, we do so in the spirit of helping the authors strengthen their arguments and not from any ill will.
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy-for a taxonomy "should display not only differences but also relationships and similarities". Instead of offering a taxonomy, they urge, we could also offer rubrics .
We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn't mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities' epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.
A More Holistic Rubric? H&P add that just as the virtues of an individual person are expressions of that person's , character so the university has (or should have) a , and their friendly suggestion is that the character character of the university is .

To pursue Wissenschaft while cultivating Bildung
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university's character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of . Bildung Deficit/diffusion Model of Public Understanding H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
We certainly didn't mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard. 1.

Smaller Points
H&P raise three smaller points: We focus, they say, more on individuals than on structures and collectives. We're not sure what gave rise to this impression -as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.
In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We've done this in the revised version of the paper (note 9). We make it seem as if the humanities are . This may be a misunderstanding. All we optional wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.
No competing interests were disclosed.

Competing Interests:
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com