Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities

This paper proposes a normative taxonomy by which universities can express the extent to which they meet five core epistemic responsibilities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge and understanding. The core epistemic responsibilities, which we call the Big Five, are to (1) foster research integrity, (2) teach for intellectual virtue, (3) address the big questions of life, (4) give humanistic inquiry and education a proper place, and (5) serve society. The paper characterizes the Big Five in some detail and explains why they are core epistemic responsibilities of universities. The paper concludes by describing the steps that should be taken in order to test, amend, and implement the taxonomy.


Introduction
In this paper, we propose a normative taxonomy of what we call the 'Big Five' in academia: five core epistemic responsibilities of universities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge, understanding, insight, rationality, and explanation. This is not to deny that universities have other important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating native people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more 1 . However, all institutions and companies have various responsibilities of these kinds, not only universities. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: to produce knowledge, to understand, to gain insight into phenomena, and so on. That is why we focus on epistemic responsibilities in this paper.
For each epistemic responsibility, we distinguish five levels describing the extent to which a university meets that responsibility or strives to do so. Our proposed taxonomy is meant as a tool to assess the degree to which a university meets its core epistemic responsibilities. The format we use is inspired by the Transparency and Openness Promotion (TOP) guidelines 2 that journals can use to describe the extent to which they meet the goals of Open Science.
The taxonomy proposed here is a product of two research projects funded by the Templeton World Charity Foundation: Science beyond Scientism (2013)(2014)(2015)(2016) and The Epistemic Responsibilities of the University (2016-2019) 3 . The first project explored which questions science can and which ones it cannot address, as well as whether natural sciences are the only reliable source of knowledge. The second project explored what the core epistemic responsibilities of universities are, given various contemporary challenges, such as hypercompetition, publication pressure, the marginalization of the humanities, and the commercialization of the university 4 . In both projects, philosophers worked in close cooperation with biomedical and social scientists. We present this as work in progress, as a starting point for gaining experience with using the taxonomy and consensus building for a more mature version. We will do so in a third project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. This paper is structured as follows. First, we provide our proposed taxonomy by specifying academia's Big Five epistemic responsibilities and detailing five levels of meeting them ( §2). After that, we argue that these are indeed five core epistemic responsibilities of universities ( §3). Finally, we lay out which future steps we aim to take to test, amend, and implement the taxonomy ( §4).

A normative taxonomy
Our proposal distinguishes five epistemic responsibilities (see Table 1). We assume each to be of more or less equal importance, so the order in which we present them is not hierarchical. The attainment levels I through V for each responsibility, however, 1   This doesn't mean that there cannot be an internal division of labor when it comes to meeting epistemic responsibilities: giving humanistic inquiry a proper place will primarily fall on humanities departments, although humanities scholars will teach in other departments and collaborate with scientists in other departments as well when a university strives for level V of this responsibility. Similarly, not all departments and research teams will have to serve society in the same way or to an equal degree. There will be significant differences between, say, the theoretical physicists and the nutrition scientists.
We will now clarify each responsibility briefly and motivate why it belongs on the list of epistemic responsibilities of universities.

To foster research integrity.
Research integrity is fostered by getting rid of perverse incentives, stimulating good mentoring, having an open research climate, and so on. Detrimental research practices include both rare major research misbehaviors like fabrication of data and highly prevalent minor misbehaviors like selective reporting 5 . By 'responsible conduct of research' we mean behavior that meets the principles and standards for good research, as laid out in major codes of conduct for research integrity 6 . This can be done on the level of individual scholars, but also that of groups, such as research teams or departments. Ideally, research integrity is promoted for both individuals and groups.
The results of scientific and scholarly research play a crucial role in modern society. Universities carry out a substantial part of this research and educate and train researchers who perform the studies and apply the results. To ensure the validity and trustworthiness of findings research needs to be performed according to the principles and standards for research integrity.
In recent years it has become painfully evident that there is substantial room for improvement in the level of compliance to these principles and standards 7 . Surveys indicate that 2% of researchers admit to having fabricated or falsified data themselves, while one third says that they have been engaged in less serious questionable research practices 8 . Only 10 -40% of study results turn out to be reproducible when the study is repeated 9 . This is often due to small sample sizes, selective reporting, and other questionable research practices 10 . Although these phenomena are understudied, it is to be expected that more transparency 11 and specifically preregistration 12 of study protocols and data analysis plans will lead to improvements. This epistemic responsibility entails not only the adoption of transparency and educating staff and students in responsible conduct of research, but also removing perverse incentives from the ways in which researchers are assessed 13 and performing research on research to strengthen the evidence base for optimizing research integrity.  8 Fanelli -How many scientists fabricate and falsify research PLoS ONE 2009; 4(5); e5738. Note that these numbers concern scientists' self-reporting; they go up considerably when researchers are asked whether they're aware of colleagues having committed scientific fraud or engaged in questionable research practices. studiousness, wisdom 14 . Educating for intellectual virtues is part of the traditional ideal of Bildung. Moral virtues, such as generosity, kindness, or benevolence, may well also be important and universities might have a responsibility to cultivate them, but, if so, this isn't an epistemic responsibility. When universities choose to teach for intellectual virtues, they can do so, minimally, by merely bringing up these virtues in educational settings; or they can explore them through case studies of intellectually virtuous scientists and scholars and their contributions to epistemic progress, or, most advanced, they can actively cultivate intellectual virtues in their students by having them mimick and practice intellectually virtuous behavior. Developing intellectual virtues requires not merely instruction about virtues or reflection on them, but also training and exercise 15 .

To teach for intellectual virtue.
Teaching for intellectual virtues is a responsibility of universities almost by definition. Intellectual virtues are broadly understood as qualities that make someone a well-trained thinker or inquirer 16 . Hence, the two main tasks of universities, research and teaching, are both served by teaching for intellectual virtue. By aiming to train students to become skilled thinkers (perhaps among other things), universities set an ambitious goal for education. Similarly, academic research also needs skilled thinkers. Moreover, the intellectual virtues are widely relevant outside the university: in politics, journalism, medicine, law, law enforcement, social work -it's hard to think of any sphere of life were good, analytical, critical, clearheaded, creative, or otherwise high quality thinking wouldn't matter.

To address the big questions of life.
By 'the big questions' we mean such questions as: What is the origin and ultimate destination of all that exists? What is the future of the earth's ecosystem? What is consciousness? Do humans have free will? Is there (objective) good and evil? Can the human mind understand the world and, if so, how? Does life have meaning? Does God exist? How does science relate to religion? These are universal questions in the sense that they have been asked in most cultures and societies throughout history, up to the present; they are not restricted to local concerns or specific academic disciplines.
Addressing the big questions of life is an epistemic responsibility of the university for a number of reasons. First, these questions are too important to be left entirely to non-academics. The big questions are everyone's concern, academics included. Second, many academics themselves are greatly interested in these questions, even if their specialized science and scholarship do not or even cannot answer them. Third, clarifying and attempting to answer these questions affects how we look at ourselves and what we deem important. Fourth, several big questions are factual questions -highly abstract and large scale, but factual nonetheless. They are not questions about tastes or preferences. It seems like they admit of true or false answers. So, at least in principle, they fall within the purview of scientific and humanistic inquiry. Fifth, big questions can inspire smaller and more manageable research questions. For example, asking about the fundamental nature of reality led to the hypothesis of atomism in Ancient Greece 17 . Ideas about God's perfection and omnipotence led Galileo to assume that mathematics is our best guide to understanding the orbits of the planets and hence that heliocentrism rather than geocentrism is correct 18 . Darwin asked whether humans are unique or whether all life on earth is monogenetic, which led him to develop evolutionary theory 19 . Einstein opposed the Copenhagen interpretation of quantum mechanics on metaphysical grounds, as he didn't believe the universe could be fundamentally probabilistic 20 . For these reasons universities cannot and should not operate as if the big questions don't exist or cannot be taken seriously. Rather, they should take them seriously and mobilize their intelligence as well as their state of the art science and scholarship to address these questions in an intellectually responsible way.

To give humanistic inquiry and education a proper place.
Universities ought to have room for the full range of academic disciplines, in both the sciences and the humanities. Many of today's most urgent challenges in society cannot be solved by purely scientific or technological means; successful solutions require compelling communication, consideration of moral values and norms, in-depth understanding of cultures and religions. All of these things, and much more, are studied in the humanities. Hence, universities ought to facilitate and embrace humanistic inquiry and teaching, and stimulate interdisciplinary collaborations between scientists and humanities scholars. Of course, some universities -technological universities, for instance -do not include humanities departments. For them, this responsibility may be interpreted as follows: the responsibility to give due weight to knowledge and understanding produced by humanities departments in other universities. Of course, this does not remove other epistemic responsibilities that are met primarily in the humanities, such as the epistemic responsibility to address the big questions of life.
Giving the humanities a proper place is a responsibility of the university for one basic reason: the humanities can deliver truth, knowledge and insight in areas where the sciences cannot 21 . The humanities have their own objects of study: they study objects that have "meaning" in a special sense, viz. meaning that derives from human conventions, from human intentions, and/or from human purposive behavior 22 . The knowledge and understanding the humanities provide differs from the knowledge and understanding that the sciences offer, in that the former is often 'indexical' (that is, related to human interests and concerns), 'perspectival' (it specifies how things look from, say, a romantic perspective), and value-related (that is, related to social, political, moral, aesthetic, or religious values) 23 . In addition to this, the humanities are particularly suitable for educating students to become well-informed, critical citizens who can reflect on socially urgent questions about life, health, education, justice, equality, liberty, etc. and participate fully in society and politics 24 .

5.
To serve society. Universities can serve society at a number of levels: local (a city or region), national, or international, humanity worldwide. What we have in mind here is serving society epistemically, that is, to help society acquire true belief, knowledge, and understanding about important issues. Of course, universities sometimes also serve society in a more practical manner, e.g., by way of proposing effective policies, producing medical interventions and other technologies. Such practical interventions are often based on scientific evidence, so the epistemic and the practical are not entirely separate, but they can be distinguished for analytical purposes. We focus on the former here, as our taxonomy concerns the epistemic rather than the moral, practical, or social responsibilities of universities.
Universities have the epistemic responsibility to serve society by, among other things, disseminating knowledge and understanding about issues that academics have investigated. Let us stress that we have knowledge dissemination rather than knowledge utilization in mind here, since we are concerned with epistemic rather than practical responsibilities of the university. So, what we have in mind are such things as press releases, expert advice, popular articles, opinion pieces, public lectures, interviews, and so on.
Serving society, however, is a two-way street. It's not only about disseminating knowledge that has been produced by scientists, but also requires taking onboard insights from society and the general public. Universities can thus also serve society by taking in urgent questions, analyzing social concerns, and incorporating the perspectives and criticisms of non-scientists 25 . Going even further, initiatives under the banner of 'citizen science' and 'science with and for society' promote the co-production of scientific knowledge by scientists and citizens 26 . Such initiatives shouldn't be thought of as replacing traditional science, but rather as complementing it in order to let science serve society better. This is a responsibility for at least two reasons. First, many scientific and scholarly discoveries are so complex that if academics do not disseminate their knowledge, those discoveries will remain unknown among the larger audience. Second, it often requires extensive academic knowledge to understand the importance and ramifications of various discoveries. Knowledge and understanding are of intrinsic epistemic value. If the university does not serve society by sharing academic knowledge and understanding, then, for much academic knowledge and understanding, that value will be attained only by a very small group of academics in the relevant field. If the university takes its epistemic responsibility of knowledge dissemination seriously, then much larger groups -academics in other fields, society as a whole -will attain those epistemic values.
Finally, we should be clear that we don't want to deny that the epistemic responsibilities we distinguish are related to each other in various ways. For instance, responsibility 1 (to foster research integrity) has to do with responsibility 2 (to teach for intellectual virtue): a university cannot be serious about research integrity but without teching for for curiosity, open-mindedness, thoroughness, and intellectual perseverance. The exact relations between these responsibilities is up for debate; we wanted to leave room for different opinions about this while working towards a potential consensus that these are indeed key epistemic responsibilities of universities. Even so, it is useful to draw analytic distinctions between the responsibilities. The way we formulated them makes it possible to check their presence relatively independently from one another and we consider that an advantage of our approach.

Future steps
As indicated, we propose our normative taxonomy as a tool to assess the degree to which a university meets the Big Five epistemic responsibilities. Our proposal is a first attempt; in future work we aim to validate, test, amend, and implement the taxonomy. Particularly, we will work on distinguishing the five levels postulated for each epistemic responsibility more clearly and we will explore in more detail how our taxonomy differs from others and whether it can serve as a tool to rank universities. We envision doing this in four consecutive steps.
First, we want to fine-tune our taxonomy in a Delphi Study 27,28 with international experts that aims in its first round at adding, replacing, and reformulating various epistemic responsibilities. The second and third Delphi rounds will seek consensus on the corresponding levels of meeting the responsibility at issue and explore what the best practices are in reaching higher levels of specific responsibilities.
Next, we will organize co-creation workshops 29 with representatives of relevant stakeholders in order to discuss a penultimate version of the taxonomy. The focus of the workshop will be on the operationalization of the levels of meeting the different epistemic responsibilities in a way which makes application of the taxonomy feasible, transparent, and as objective as possible.
Then, we will test and qualitatively evaluate the taxonomy in a number of universities, resulting in a definitive description of the responsibilities, the levels, and a tool-kit of best practices.
Finally, we will publish and disseminate the results on a dedicated website and explore whether the taxonomy is a suitable alternative for, or addition to, the currently dominant Academic Ranking of World Universities 30 and the Times Higher Education World University Rankings 31 .

Author contributions
Each author has significantly contributed to this paper on each of the items. The order of names represents the amount of time they have put into it.

Bart Penders
Maastricht University, Maastricht, The Netherlands

Britt Holbrook
Department of Humanities, New Jersey Institute of Technology, Newark, NJ, USA We appreciate the authors' responses to our comments. Their changes to the manuscript manage to alleviate many of the concerns we have. However, one key concern -the Big One -remains: the authors' aspiration to act as rankers themselves. Even if the ranking they propose were conceptually and methodologically flawless-and we trust that the authors would not be willing to make such a bold claim to perfection, even after the extensive testing and validation they propose -the ranking would still result in moral and political problems associated with ranking.
In a wonderfully incisive Twitter thread (https://twitter.com/jelena3121/status/1356210115406397443), Jelena Brankovic, a long-time researcher on rankings, suggests that "Even if you made [rankings] super transparent, accountable, and well-governed, that could even aggravate the adverse effects." No ranking merely documents performance. The act of ranking necessarily inflates minor differences into different ranks. Every ranking is a zero sum game. One university can rise in the rankings only if another goes down. Insofar as every ranking is a zero sum game, every ranking generates reputation. Every ranking produces competition for that reputation. Indeed, reputational ranking becomes the One Big Goal for universities. The authors' aspiration to turn the Big Five into a ranking will insure that, rather than working to improve their attention to their epistemic responsibilities, universities will treat those responsibilities merely as means to achieving a higher ranking.
We would like to invite the authors to frame their ambitions more cautiously. Used as a rubric for relatively low stakes formative assessment, the Big Five might well improve the performance of some universities. Deployed as a ranking for all universities, however, the Big Five will become all about the Big One. This is an interesting manuscript which is somewhat outside my field of expertise, which is centred around molecular biology and genetics. There is no doubt that this will have limited my ability to provide a detailed critique of the arguments presented, and this may also limit the value of some of my suggestions. Nonetheless I advance these in the hope that my comments will indicate how this article was received by someone outside the field, given that this paper will interest a broad range of academics and researchers.
My main comments pertain to the taxonomy proposed in Table 1. I would not argue with the five epistemic responsibilities put forward; however, I did wonder whether five levels of taxonomy are required for any or all of the five proposed responsibilities. Some of the descriptions provided for levels I and II seem unlikely to occur in practice, and I could not always see clear distinctions between some levels proposed, most frequently for levels I and II.
For example, for "1. To foster research integrity", I could not see a clear difference between "Issues of detrimental research practices and responsible conduct of research are neglected" (level I) and "Issues…. are occasionally addressed" (level II). There may not be much difference in practice between something being "neglected" or "occasionally addressed", and it is also not clear how "occasionally" would be reliably measured. Similarly, I could not see a clear difference between level III "Detrimental research practices are addressed… but there is no academic climate that actively stimulates responsible conduct of research" and level IV "There is an academic climate that detects and acts upon detrimental research practices and actively stimulates responsible conduct of research, but only at the level of individual researchers".
For "2. To teach for intellectual virtue", I couldn't see clear differences between level I "the university…. pays no attention to intellectual virtues" and level II "intellectual virtues are considered important, but not taught". I also could not imagine any University that would fall into either of these categories.
Again, I couldn't see clear differences for the proposed levels I and II for "3. To address the big questions of life", ie these questions being "neglected" (level I), versus "mentioned but discarded" (level II).
For "4. To give humanistic inquiry and education a proper place", I also saw no real distinction between the humanities being "marginalized" (level I) and "not marginalized but considered and treated as being inferior" (level II). Levels III, IV, and V commonly refer to humanities being given a "proper place" in the university, but this definition will naturally vary between universities according to their type, i.e. technical universities versus universities with a strong and stated focus upon the humanities. Some wording within levels III-V also lacked definition, i.e. "in isolation" (level III), "some other disciplines" (level IV), and "across disciplines" (level V). "Disciplines" can be defined quite narrowly by some universities, so working with other disciplines or across disciplines does not always imply working between the humanities and the sciences.
Finally, for "5. To serve society", universities where "research and teaching are confined to …purely academic challenges" (level I) could argue that they are serving society in this way, possibly by pursuing academic challenges that can indirectly linked to the big questions of life. It also seems unlikely that any university would be defined by level II "Research and teaching identify societal challenges, but the university leaves it to others to confer knowledge … relevant to those challenges".
Are all factual statements correct and adequately supported by citations?
Contemporary challenges such as hypercompetition, publication pressures, marginalisation of the humanities and commercialization of universities (page 3, paragraph 2) should be supported by references.

1.
The status of reference 1 (footnote 1) which is a manuscript by Peels et al, could be updated (page 3).

2.
Are arguments sufficiently supported by evidence from the published literature?
Although the manuscript states that "more transparency and specifically pre-registration of study protocols and data analysis plans will lead to considerable improvements (in the use of small sample sizes, selective reporting and other questionable research practices, from the preceding sentence), this seems to be somewhat of an overstatement. Firstly, more transparency and preregistration may not impact on the use of small sample sizes, as these might represent feasibility constraints that pre-registration may not overcome. Secondly, I understand that there is not yet a substantial body of literature that describes the possible benefits and/or drawbacks of study preregistration, by comparing the results and outcomes of pre-registered studies versus comparable studies that lacked pre-registration.
Finally, the manuscript states that (the big questions of life) "are too important to be left entirely to non-academics". This statement could be written in a more inclusive way, for example by indicating that as these questions are so important, they are everyone's concern, including the concern of academics.

Are the conclusions drawn balanced and justified on the basis of the presented arguments? Yes
Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Author Response 02 Jul 2020 Rik Peels, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB's remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.
Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.
Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: "Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements." We didn't consider this to be a particularly strong claim, but we have now changed it into: "Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements." Finally, we have adopted JB's suggestion for a more inclusive formulation about whose This article proposes what Peels et al. call a "normative taxonomy" of the "Big Five" epistemic responsibilities universities ought to meet: (1) fostering research integrity, (2) teaching (for) intellectual virtues, (3) addressing the big questions, (4) valuing the humanities, and (5) serving society. This normative taxonomy essentially serves as an analytical rubric designed to assess the performance of universities. One could easily imagine -and in fact, Peels et al. propose as much in §3 -refining the rubric presented in Table 1, using it to assess the performance of various universities, and then comparing their "Big Five" scores to their world university rankings. We agree that undertaking these tasks presents many enticing possibilities, especially to those of us inclined against bean counting approaches to accountability. Nevertheless, since they position themselves as aspiring university rankers, they should address the range of critiques targeting existing rankings, their methodologies, and the performativity of these rankings. They should then situate their own approach within that context, arguing especially that it is an improvement onand escapes the criticisms levelled at -existing rankings. This is our global concern.
Below, we offer a number of other substantial concerns, followed by a shorter list of minor issues to consider: Why these five? Peels et al. propose five epistemic responsibilities of universities as the Big Five. The authors point out that they consider each of the Big Five equally important. They also defend each of the five as indeed responsibilities that universities ought to meet. However, they fail to argue that these five epistemic responsibilities are the most important epistemic responsibilities for universities, that these five are in fact equally important, that these five are sufficient to capture the epistemic responsibilities universities ought to meet, or that universities have no other equally worthy responsibilities that ought to be included.

Why five?
We agree that the proposed Big Five are epistemic responsibilities that universities ought to meet. Using the proposed rubric to evaluate universities might provide valuable information about particular areas of strength and weakness at a given university. However, designing the rubric around five separate epistemic responsibilities suggests that universities could meet each of their epistemic responsibilities to varying degrees. One university might be particularly strong in terms of fostering research integrity but fail miserably at teaching for intellectual virtue. Another might excel at addressing the big questions but do a lousy job of valuing the humanities. Although these possibilities sound prima facie plausible, using an analytic approach to epistemic responsibilities also generates some less than desirable outcomes. Could a university that fails to teach for intellectual virtue really excel at fostering research integrity? That seems plausible only if we have a very limited procedural view of what counts as fostering research integrity. Without valuing the humanities, how could a university excel at addressing the big questions? Unless we adopt scientism, the answer seems to be that it could not. Similar problems arise when we consider the relations between the other proposed epistemic responsibilities. The use of a taxonomical vocabulary further strengthens this worry. Taxonomies should display not only differences, but also relationships and similarities. The current form of representation is not, in that sense, a taxonomy, but (for the most part) a normative list. Only when the ties binding the elements together are conceptualised convincingly can we convincingly speak of a taxonomy. For this reason, we suggest that the authors consider enriching their list to raise it to the level of a taxonomy or abandon talk of a 'taxonomy' altogether. The authors could adopt the vocabulary of an analytical rubric, instead.
Even if the authors decide to use the language of rubrics rather than taxonomies, we suggest that the authors consider a more holistic approach along with an analytic one, just as virtue ethics appeals not only to the individual virtues, but also to the question of character. Rather than describing teaching for intellectual virtue as part of Bildung, perhaps the authors could consider fostering the ability to pursue Wissenschaft while cultivating Bildung as analogous to a university's character. Call it the W-B rubric, under which different (now, no longer separate) epistemic responsibilities could be grouped together to indicate different levels of overall achievement. In order for a university to score best on the W-B rubric, it would have to excel at meeting all of its epistemic responsibilities (for a brief discussion of the difference between analytic and holistic rubrics, as well as examples and further references, see Rubrics: useful assessment tools. Centre for Teaching Excellence, University of Waterloo 1 ).

Why insist on separating epistemic from other responsibilities universities have?
This question follows on the previous one, in that it aims to nudge the authors away from using only an analytic approach. By insisting on distinguishing epistemic responsibilities from moral, practical, and social responsibilities, the authors impoverish the idea of what it means for universities to serve society. If the goal is to work towards alternative forms of assessment and accountability (thereby politicizing them further), excluding other responsibilities and artificially fragmenting the landscape accordingly, one would expect [a] the epistemic responsibility taxonomy to be situated within a larger responsibility taxonomy and/or [b] an argument that leaving other responsibilities aside is preferable. More in line with the suggestion to pursue a more holistic approach, we must ask: could we really say that a university is a good university if it meets only its epistemic responsibilities? We think not. Perhaps the W-B rubric we suggest, or another holistic approach, could help address this issue. Note that one could also include analytical rubrics for moral, practical, and social responsibilities of universities and incorporate the responsibilities included in each of these separate rubrics into a holistic approach.
Our point is not that analysis is always bad. Indeed, analysis can be quite informative and useful as part of a formative evaluation. We are simply suggesting that the authors consider using their analytic approach to complement a more holistic approach.

Who counts, and why do they count?
In §3, the authors suggest optimising their rubric by means of a Delphi Study "with international experts", which could lead to fine-tuning it, but possibly also to radical revisions. If the latter is the case, then what is the status of the current five epistemic responsibilities? When it comes to the actual Delphi study, whom they choose to participate in the study will likely make a very big difference to the final design of the rubric. Presumably, the authors focus on "experts" because they assume experts know more about epistemic responsibilities than non-experts do. Yet, the way in which the authors present it suggests a retreat to the deficit/diffusion model of public understanding of science. Not only has the deficit model been shown to be factually incorrect, it also presupposes a social contract for science and scholarship that imagines universities as ivory towers. In their discussion of a university's responsibility to serve society epistemically, the authors focus on the supply side of knowledge production, suggesting that knowledge dissemination is how best to serve society. Arguably, however, serving society -even if we limit this to an epistemic responsibility -means something other than telling society what we academics think they need to know. Here again, the separation between epistemic and other responsibilities creates a lot of friction, since a departure from the deficit model requires a high degree of interaction and participation beyond universities. In fact, one could argue that interaction and participation are epistemic requirements (co production of knowledge).
Along these lines, we suggest that non-experts may have valuable feedback to offer, even if the authors ultimately decide only to pursue the development of an analytic rubric for epistemic responsibilities of universities. One way to include the demand side of knowledge production in the design of their rubric would be to recruit non-expert stakeholders to participate in the proposed Delphi Study and workshops or organise parallel expert (Delphi) and citizen (Citizen Summit) consultations.

We also have a few more minor points that nevertheless warrant attention:
Despite the argument that the rubric is to be applied to entire universities, the responsibilities focus very much on individuals and groups (which seem to be multiple individuals in the ways in which they are discussed) and less on the level of structures and collectives.

1.
Peels et al. frame irreproducibility solely as the result of sub-par science and thus as a research integrity issue. Their previous work, as well current scholarly debates on the characters and qualities or irreproducibiluty and irreplicability, takes up a much more nuanced position. Perhaps the authors will consider adding some of that nuance here.

2.
Excusing a few universities (technical universities or polytechnics, in this case) of taking responsibility for one of the epistemic responsibilities (#4) suggests that giving humanistic inquiry and education a proper place is optional. This seems to conflict with the idea that all responsibilities are equally important. It also opens up possibilities for policies that 3.
deprioritise humanities research and teaching (cf. the current Van Rijn report in NL).
that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn't immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we'll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.
Why These Five? H&P note that we don't argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)-(c), H&P are right that more support is needed to back up these claims -the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that's part of the abovementioned new project. As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding.
That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy-for a taxonomy "should display not only differences but also relationships and similarities". Instead of offering a taxonomy, they urge, we could also offer rubrics. We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn't mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities' epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens.
We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.
A More Holistic Rubric? H&P add that just as the virtues of an individual person are expressions of that person's character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung. We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university's character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
We certainly didn't mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points: We focus, they say, more on individuals than on structures and collectives. We're not sure what gave rise to this impression -as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.

1.
In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P 2. say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We've done this in the revised version of the paper (note 9). We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.

3.
Competing Interests: No competing interests were disclosed.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com