Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities

Rik Peels; René van Woudenberg; Jeroen de Ridder; Lex Bouter

doi:10.12688/f1000research.19459.1

Home Browse Academia’s Big Five: a normative taxonomy for the epistemic responsibilities...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Opinion Article

Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities

[version 1; peer review: 1 approved, 1 approved with reservations]

Rik Peels ¹, René van Woudenberg¹, Jeroen de Ridder¹, Lex Bouter²

PUBLISHED 13 Jun 2019

Author details Author details

¹ Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
² Amsterdam Medical Centers, Amsterdam, The Netherlands

Rik Peels
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation

René van Woudenberg
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jeroen de Ridder
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Lex Bouter
Roles: Conceptualization, Funding Acquisition, Investigation, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Research on Research, Policy & Culture gateway.

Abstract

This paper proposes a normative taxonomy by which universities can express the extent to which they meet five core epistemic responsibilities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge and understanding. The core epistemic responsibilities, which we call the Big Five, are to (1) foster research integrity, (2) teach for intellectual virtue, (3) address the big questions of life, (4) give humanistic inquiry and education a proper place, and (5) serve society. The paper characterizes the Big Five in some detail and explains why they are core epistemic responsibilities of universities. The paper concludes by describing the steps that should be taken in order to test, amend, and implement the taxonomy.

Keywords

Big question, epistemic responsibilities, humanities, research integrity, societal relevance, teaching, university, virtue

Corresponding author: Rik Peels

Competing interests: No competing interests were disclosed.

Grant information: Work on this paper was funded by the Templeton World Charity Foundation and was carried out within the project The Epistemic Responsibilities of the University (2016-2019).
The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Templeton World Charity Foundation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2019 Peels R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Peels R, van Woudenberg R, de Ridder J and Bouter L. Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2019, 8:862 (https://doi.org/10.12688/f1000research.19459.1) First published: 13 Jun 2019, 8:862 (https://doi.org/10.12688/f1000research.19459.1) Latest published: 06 Jul 2020, 8:862 (https://doi.org/10.12688/f1000research.19459.2)

1. Introduction

In this paper, we propose a normative taxonomy of what we call the ‘Big Five’ in academia: five core epistemic responsibilities of universities. Epistemic responsibilities are responsibilities that have to do with the attainment of knowledge, understanding, insight, rationality, and explanation. Thus, they are to be distinguished from moral responsibilities – that concern the well-being of humans and animals – and practical responsibilities, such as the responsibility to develop societally useful technologies and effective medical interventions¹. For each epistemic responsibility, we distinguish five levels describing the extent to which a university meets that responsibility or strives to do so. Our taxonomy is meant as a tool to assess the degree to which a university meets its core epistemic responsibilities. The format we use is inspired by the Transparency and Openness Promotion (TOP) guidelines² that journals can use to describe the extent to which they meet the goals of Open Science.

The taxonomy proposed here is a product of two research projects funded by the Templeton World Charity Foundation: Science beyond Scientism (2013–2016) and The Epistemic Responsibilities of the University (2016–2019)³. The first project explored which questions science can and which ones it cannot address, as well as whether the natural sciences are the only reliable source of knowledge. The second project explored what the core epistemic responsibilities of universities are, given various contemporary challenges, such as hypercompetition, publication pressure, the marginalization of the humanities, and the commercialization of the university. In both projects, philosophers worked in close cooperation with biomedical and social scientists. We present this as work in progress, as a starting point for gaining experience with using the taxonomy and consensus building for a more mature version.

This paper is structured as follows. First, we provide the taxonomy by specifying academia’s Big Five epistemic responsibilities and detailing five levels of meeting them (Table 1). After that, we argue that these are indeed five core epistemic responsibilities of universities (section 2). Finally, we lay out which future steps we aim to take to test, amend, and implement the taxonomy (section 3).

Table 1. Normative taxonomy of epistemic responsibilities of universities.

Big Five epistemic responsibilities	Level I	Level II	Level III	Level IV	Level V
1. To foster research integrity	Issues of detrimental research practices and responsible conduct of research are neglected	Issues of detrimental research practices and responsible conduct of research are occasionally addressed	Detrimental research practices are addressed in response to breaches and problems, but there is no academic climate that actively stimulates responsible conduct of research	There is an academic climate that detects and acts upon detrimental research practices and actively stimulates responsible conduct of research, but only at the level of individual researchers	There is an academic climate that detects and acts upon detrimental research practices and actively stimulates responsible conduct of research, at both the level of individual researchers and that of teams and departments
2. To teach for intellectual virtue	The university teaches for knowledge and understanding, but pays no attention to intellectual virtues	The university teaches for knowledge and understanding, and intellectual virtues are considered important but not taught	The university teaches for knowledge and understanding and includes education about what the intellectual virtues are	The university teaches for knowledge and understanding, explicitly includes intellectual virtues in education, and explores cases that show the relevance of intellectual virtues	The university teaches for knowledge and understanding, includes intellectual virtues in education, and practically engages in developing intellectual virtues
3. To address the big questions of life	The big questions are neglected; there is no attention for them	The big questions are mentioned, but discarded as meaningless or outside the scope of universities	The big questions are taken seriously and addressed in teaching, but in isolation, within relatively isolated departments or courses	The big questions are taken seriously and addressed in (interdisciplinary) research and teaching in some departments and courses	The big questions are taken seriously and addressed throughout the university’s (interdisciplinary) research and teaching in all programs and departments
4. To give humanistic inquiry and education a proper place	The humanities are acknowledged as legitimate academic disciplines, but marginalized	The humanities are not marginalized, but considered and treated as inferior to the sciences	The humanities have a proper place in the university, but operate in isolation	The humanities have a proper place in the university, and there is cooperation with some other disciplines in teaching and research	The humanities have a proper place in the university and there is intensive and broad cooperation across disciplines in teaching and research
5. To serve society	Research and teaching are confined to what is considered relevant for purely academic challenges	Research and teaching identify societal challenges, but the university leaves it to others to confer knowledge and understanding relevant to those challenges	Research and teaching that addresses societal challenges is carried out upon request, but deemed substantially less important than fundamental research and teaching	Research and teaching that addresses societal challenges has a fixed place in the university, but is deemed less important than fundamental research and teaching	Research and teaching that addresses societal challenges has a fixed place in many departments and programs, and is seen as equally important as fundamental research and teaching

2. A normative taxonomy

Our proposal distinguishes five epistemic responsibilities. We consider each to be of equal importance, so the order in which we present them is not hierarchical. The attainment levels I through V for each responsibility, however, are meant hierarchically: each level presents a more advanced stage of meeting the responsibility at issue.

We think of these responsibilities as attaching primarily to entire universities. So, in order to meet them, each responsibility should have systematic consequences, throughout the university and its faculties, departments, institutes, or other organizational parts. If a university strives to teach for intellectual virtue at the highest level, for example, students throughout the university should be instructed in what these virtues are and stimulated to cultivate them through virtue-building teaching activities.

This doesn’t mean that there cannot be an internal division of labour when it comes to meeting epistemic responsibilities: giving humanistic inquiry a proper place will primarily fall on humanities departments, although humanities scholars will teach in other departments and collaborate with scientists in other departments as well when a university strives for level V of this responsibility. Similarly, not all departments and research teams will have to serve society in the same way or to an equal degree. There will be significant differences between, say, the theoretical physicists and the nutrition scientists.

We will now clarify each responsibility briefly and motivate why it belongs on the list of epistemic responsibilities of universities.

1. To foster research integrity. Research integrity is fostered by getting rid of perverse incentives, stimulating good mentoring, having an open research climate, and so on. Detrimental research practices include both rare major research misbehaviours like fabrication of data and highly prevalent minor misbehaviours like selective reporting⁴. By ‘responsible conduct of research’ we mean behavior that meets the principles and standards for good research, as laid out in major codes of conduct for research integrity⁵. Such behavior can be stimulated at the level of individual scholars, but also that of groups, such as research teams or departments. Ideally, research integrity is promoted for both individuals and groups.

The results of scientific and scholarly research play a crucial role in modern society. Universities carry out a substantial part of this research and educate and train researchers who perform the studies and apply the results. To ensure the validity and trustworthiness of findings research needs to be performed according to the principles and standards for research integrity. In recent years it has become painfully evident that there is substantial room for improvement in the level of compliance to these principles and standards⁶. Surveys indicate that 2% of researchers admit to having fabricated or falsified data themselves, while one third says that they have been engaged in less serious questionable research practices⁷. Only 10 – 40% of study results turn out to be reproducible when the study is repeated⁸. This is due to small sample sizes, selective reporting and other questionable research practices. Although these phenomena are understudied, it is clear that more transparency⁹ and specifically preregistration¹⁰ of study protocols and data analysis plans will lead to considerable improvements. This epistemic responsibility entails not only the adoption of transparency and educating staff and students in responsible conduct of research, but also removing perverse incentives from the ways in which researchers are assessed¹¹ and performing research on research to strengthen the evidence base for optimizing research integrity.

2. To teach for intellectual virtue. Among the intellectual virtues are: open-mindedness, attentiveness, charitableness, intellectual courage, creativity, curiosity, discernment, honesty, intellectual humility, objectivity, parsimony, perseverance, studiousness and wisdom¹². Educating for intellectual virtues is part of the traditional ideal of Bildung. Moral virtues, such as generosity, kindness, or benevolence, may well also be important and universities might have a responsibility to cultivate them, but, if so, this isn’t an epistemic responsibility. When universities choose to teach for intellectual virtues, they can do so, minimally, by merely bringing up these virtues in educational settings; or they can explore them through case studies of intellectually virtuous scientists and scholars and their contributions to epistemic progress, or, most advanced, they can actively cultivate intellectual virtues in their students by having them mimic and practise intellectually virtuous behaviour. Developing intellectual virtues requires not merely instruction about virtues or reflection on them, but training and exercise¹³.

Teaching for intellectual virtues is a responsibility of universities almost by definition. Intellectual virtues are broadly understood as qualities that make someone a well-trained thinker or inquirer¹⁴. Hence, the two main tasks of universities, research and teaching, are both served by teaching for intellectual virtue. By aiming to train students to become skilled thinkers (perhaps among other things), universities set an ambitious goal for education. Similarly, academic research also needs skilled thinkers. Moreover, the intellectual virtues are widely relevant outside the university: in politics, journalism, medicine, law, law enforcement, social work – it’s hard to think of any sphere of life where good, analytical, critical, clearheaded, creative, or otherwise high quality thinking wouldn’t matter.

3. To address the big questions of life. By ‘the big questions’ we mean such questions as: What is the origin and ultimate destination of all that exists? What is the future of the earth’s ecosystem? What is consciousness? Do humans have free will? Is there (objective) good and evil? Can the human mind understand the world and, if so, how? Does life have meaning? Does God exist? How does science relate to religion? These are universal questions in the sense that they have been asked in most cultures and societies throughout history, up to the present; they are not restricted to local concerns or specific academic disciplines.

Addressing the big questions of life is an epistemic responsibility of the university for a number of reasons. First, these questions are too important to be left entirely to non-academics. Second, many academics themselves are greatly interested in these questions, even if their specialized science and scholarship does not or even cannot answer them. Third, clarifying and attempting to answer these questions affects how we look at ourselves and what we deem important. Fourth, several big questions are factual questions – highly abstract and large scale, but factual nonetheless. They are not questions about tastes or preferences. They seem to admit of true or false answers. So, at least in principle, they fall within the purview of scientific and humanistic inquiry. Fifth, big questions can inspire smaller and more manageable research questions. For example, asking about the fundamental nature of reality led to the hypothesis of atomism in Ancient Greece¹⁵. Ideas about God’s perfection and omnipotence led Galileo to assume that mathematics is our best guide to understanding the orbits of the planets and hence that heliocentrism rather than geocentrism is correct¹⁶. Darwin asked whether humans are unique or whether all life on earth is monogenetic, which led him to develop evolutionary theory¹⁷. Einstein opposed the Copenhagen interpretation of quantum mechanics on metaphysical grounds, as he didn’t believe the universe could be fundamentally probabilistic¹⁸. For these reasons universities cannot and should not operate as if the big questions don’t exist or cannot be taken seriously. Rather, they should take them seriously and mobilize their intelligence as well as state of the art science and scholarship to address these questions in an intellectually responsible way.

4. To give humanistic inquiry and education a proper place. Universities ought to have room for the full range of academic disciplines, in both the sciences and the humanities. Many of today’s most urgent challenges in society cannot be solved by purely scientific or technological means; successful solutions require compelling communication, consideration of moral values and norms, and in-depth understanding of cultures and religions. All of these things, and much more, are studied in the humanities. Hence, universities ought to facilitate and embrace humanistic inquiry and teaching, and stimulate interdisciplinary collaborations between scientists and humanities scholars. Of course, some universities – technological universities, for instance – do not include humanities departments. For them, this responsibility may be either left aside, or be interpreted as follows: the responsibility to give due weight to knowledge and understanding produced by humanities departments in other universities.

Giving the humanities a proper place is a responsibility of the university for one basic reason: the humanities can deliver truth, knowledge and insight in areas where the sciences cannot¹⁹. The humanities have their own objects of study: they study objects that have “meaning” in a special sense, viz. meaning that derives from human conventions, from human intentions, and/or from human purposive behaviour²⁰. The knowledge and understanding the humanities provide differs from the knowledge and understanding that the sciences offer, in that the former is often ‘indexical’ (that is, related to human interests and concerns), ‘perspectival’ (it specifies how things look from, say, a romantic perspective), and value-related (that is, related to social, political, moral, aesthetic, or religious values)²¹. In addition to this, the humanities are particularly suitable for educating students to become well-informed, critical citizens who can reflect on socially urgent questions about life, health, education, justice, equality, liberty, etc. and participate fully in society and politics²².

5. To serve society. Universities can serve society at a number of levels: local (a city or region), national, or international, humanity worldwide. What we have in mind here is serving society epistemically, that is, to help society acquire true belief, knowledge, and understanding about important issues. So, what we have in mind are such things as press releases, expert advice, popular articles, opinion pieces, public lectures, interviews, and so on. Of course, universities sometimes also serve society in a more practical manner, e.g., by way of proposing effective policies, producing medical interventions and other technologies. Such practical interventions are often based on scientific evidence, so the epistemic and the practical are not entirely separate, but they can be distinguished for analytical purposes. We focus on the former here, as our taxonomy concerns the epistemic rather than the moral, practical, or social responsibilities of universities.

This is a responsibility for at least two reasons. First, many scientific and scholarly discoveries are so complex that if academics do not disseminate their knowledge, those discoveries will remain unknown among the larger audience. Second, it often requires extensive academic knowledge to understand the importance and ramifications of various discoveries. Knowledge and understanding are of intrinsic epistemic value. If the university does not serve society by sharing academic knowledge and understanding, then, for much academic knowledge and understanding, that value will be attained only by a very small group of academics in the relevant field. If the university takes its epistemic responsibility of knowledge dissemination seriously, then much larger groups – academics in other fields, society as a whole – will attain those epistemic values.

3. Future steps

As indicated, we propose our normative taxonomy as a tool to assess the degree to which a university meets the Big Five epistemic responsibilities. Our proposal is a first attempt; in future work we aim to validate, test, amend, and implement the taxonomy. We envision doing this in four consecutive steps.

First, we want to fine-tune our taxonomy in a Delphi Study^23,24 with international experts that aims in its first round at adding, replacing, and reformulating various epistemic responsibilities. The second and third Delphi rounds will seek consensus on the corresponding levels of meeting the responsibility at issue and explore what the best practices are for reaching higher levels of specific responsibilities.

Next, we will organize co-creation workshops²⁵ with representatives of relevant stakeholders in order to discuss a penultimate version of the taxonomy. The focus of the workshop will be on the operationalization of the levels of meeting the different epistemic responsibilities in a way which makes application of the taxonomy feasible, transparent, and as objective as possible.

Then, we will test and qualitatively evaluate the taxonomy in a number of universities, resulting in a definitive description of the responsibilities, the levels, and a toolkit of best practices.

Finally, we will publish and disseminate the results on a dedicated website and explore whether the taxonomy is a suitable alternative for, or addition to, the currently dominant Academic Ranking of World Universities²⁶ and the Times Higher Education World University Rankings²⁷.

Data availability

No data is associated with this article.

Grant information

Work on this paper was funded by the Templeton World Charity Foundation and was carried out within the project The Epistemic Responsibilities of the University (2016–2019). The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Templeton World Charity Foundation.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Footnotes

¹Some of us have made this distinction before; see, for instance, Peels R, de Ridder J, Haven T, Bouter L (2019). Value Pluralism in Research Integrity. under review.

²Nosek B et al. (2015). Promoting an Open Research Culture. Science 348 1422–5, https://doi.org/10.1126/science.aab2374 and https://cos.io/our-services/top-guidelines/

³For some of the outcomes, see de Ridder J, Peels R, van Woudenberg R (2018). Scientism: Prospects and Problems (New York: Oxford University Press).

⁴See National Academies of Sciences, Engineering, and Medicine (2017). Fostering Integrity in Research (Washington, DC: The National Academies Press). https://doi.org/10.17226/21896

⁵See, e.g., the Singapore Statement (https://wcrif.org/guidance/singapore-statement); ALLEA (2017). The European Code of Conduct for Research Integrity Revised Edition, https://ec.europa.eu/research/participants/data/ref/h2020/other/hi/h2020-ethics_code-of-conduct_en.pdf, last visited May 14^th 2019; DORA (2018). San Francisco Declaration on Research Assessment, https://sfdora.org; VSNU (2018). Netherlands Code of Conduct for Research Integrity, Revised Edition, http://www.vsnu.nl/files/documents/Netherlands%20Code%20of%20Conduct%20for%20Research%20Integrity%202018.pdf, last visited May 14^th2019.

⁶See National Academies of Sciences, Engineering, and Medicine (2017). Fostering Integrity in Research (Washington, DC: The National Academies Press). https://doi.org/10.17226/21896; Hiney, M (2015). Research Integrity: What it Means, Why it Is important and How we Might Protect it. EC Briefing Paper - Research Integrity (https://www.scienceeurope.org/wp-content/uploads/2015/12/Briefing_Paper_Research_Integrity_web.pdf)

⁷Fanelli, D (2009). How Many Scientists Fabricate and Falsify Research?. PLoS ONE 4(5); e5738. https://doi.org/10.1371/journal.pone.0005738 Note that these numbers concern scientists’ self-reporting; they go up considerably when researchers are asked whether they’re aware of colleagues having committed scientific fraud or engaged in questionable research practices.

⁸KNAW Replication studies (https://www.knaw.nl/nl/actueel/publicaties/replication-studies); Baker, M (2016). Is There a Replicability Crisis? Nature 533(7604): 452–454. https://doi.org/10.1038/533452a Some of us have recently argued that replication is also a desideratum in the humanities. See Peels R and Bouter L (2018). The Possibility and Desirability for Replication in the Humanities. Palgrave Communications 4:95, https://doi.org/10.1057/s41599-018-0149-x.

⁹Wilkinson MD, Dumontier M, et al. (2016). The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific Data 3: 160018. https://doi.org/10.1038/sdata.2016.18; https://www.go-fair.org/fair-principles/

¹⁰Nosek B, Ebersole CR, DeHaven AC, Mellor DT (2018). The Preregistration Revolution. PNAS 115: 2600–2606. https://doi.org/10.1073/pnas.1708274114

¹¹Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, and Goodman SN (2018). Assessing Scientists for Hiring, Promotion, and Tenure. PLoS Biol 16(3): e2004089. https://doi.org/10.1371/journal.pbio.2004089

¹²See Battaly H, ed. (2010). Virtue and Vice, Moral and Epistemic (Oxford: Wiley-Blackwell); Baehr J, ed. (2016). Intellectual Virtues and Education: Essays in Applied Virtue Epistemology (New York: Routledge).

¹³See Ritchhart R (2002). Intellectual Character: What It Is, Why It Matters, and How to Get It (San Francisco: Jossey-Bass).

¹⁴See Roberts R and Woods J (2007). Intellectual Virtues (New York: Oxford University Press); Baehr J (2011). The Inquiring Mind (New York: Oxford University Press); Battaly H, ed. (2018). Routledge Handbook of Virtue Epistemology (London: Routledge).

¹⁵Berryman S (2016). Ancient Atomism, in: Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy https://plato.stanford.edu/archives/win2016/entries/atomism-ancient/.

¹⁶See Shea W and Artigas M (2003). Galileo in Rome: The Rise and Fall of a Troublesome Genius (Oxford: Oxford University Press); Remmert V (2005). Galileo, God, and Mathematics. In Koetsier T, Bergmans L (eds.), Mathematics and the Divine. A Historical Study (Amsterdam: Elsevier), pp. 347–360.

¹⁷See Desmond A and Moore J (1992). Darwin (New York: W.W. Norton).

¹⁸See Isaacson W (2017). Einstein. His Life and Universe (New York: Simon & Schuster).

¹⁹Thus also, Peels R (2018). Epistemic Values in the Humanities and the Sciences, History of Humanities 3.1, 89–111. https://doi.org/10.1086/696304

²⁰See van Woudenberg R (2018). The Nature of the Humanities, Philosophy 93: 109–140. https://doi.org/10.1017/S003181911700047X

²¹See chapter 2 of Foley R (2018). The Geography of Insight. The Sciences, the Humanities, How They Differ, Why They Matter (Oxford: Oxford University Press).

²²See Nussbaum M (2010). Not for Profit: Why Democracy Needs the Humanities (Princeton: Princeton University Press).

²³Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, Bouter LM, de Vet HCW, Mokkink LB (2018). COSMIN Methodology for Evaluating the Content Validity of Patient-Reported Outcome Measures: A Delphi Study. Quality of Life Research 27: 1159–70. https://doi.org/10.1007/s11136-018-1829-0

²⁴ https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf.

²⁵Ramaswamya V and Ozcan K (2018). What is Co-Creation? An Interactional Creation Framework and its Implications for Value Creation. Journal of Business Research 84: 196–05. https://doi.org/10.1016/j.jbusres.2017.11.027

²⁶http://www.shanghairanking.com/

²⁷https://www.timeshighereducation.com/world-university-rankings

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 13 Jun 2019

Author details Author details

¹ Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
² Amsterdam Medical Centers, Amsterdam, The Netherlands

Rik Peels
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation

René van Woudenberg
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jeroen de Ridder
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Lex Bouter
Roles: Conceptualization, Funding Acquisition, Investigation, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Work on this paper was funded by the Templeton World Charity Foundation and was carried out within the project The Epistemic Responsibilities of the University (2016-2019).
The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Templeton World Charity Foundation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 06 Jul 2020, 8:862

https://doi.org/10.12688/f1000research.19459.2

version 1

Published: 13 Jun 2019, 8:862

https://doi.org/10.12688/f1000research.19459.1

© 2019 Peels R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Peels R, van Woudenberg R, de Ridder J and Bouter L. Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2019, 8:862 (https://doi.org/10.12688/f1000research.19459.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 13 Jun 2019

Views

Reviewer Report 11 Dec 2019

Jennifer Byrne, NSW Health Statewide Biobank, School of Medical Sciences, Faculty of Medicine and Health, The Children's Hospital at Westmead, Sydney, NSW, Australia

Approved

https://doi.org/10.5256/f1000research.21335.r57012

This is an interesting manuscript which is somewhat outside my field of expertise, which is centred around molecular biology and genetics. There is no doubt that this will have limited my ability to provide a detailed critique of the arguments presented, and this may also limit the value of some of my suggestions. Nonetheless I advance these in the hope that my comments will indicate how this article was received by someone outside the field, given that this paper will interest a broad range of academics and researchers.

My main comments pertain to the taxonomy proposed in Table 1. I would not argue with the five epistemic responsibilities put forward; however, I did wonder whether five levels of taxonomy are required for any or all of the five proposed responsibilities. Some of the descriptions provided for levels I and II seem unlikely to occur in practice, and I could not always see clear distinctions between some levels proposed, most frequently for levels I and II.

For example, for “1. To foster research integrity”, I could not see a clear difference between “Issues of detrimental research practices and responsible conduct of research are neglected” (level I) and “Issues…. are occasionally addressed” (level II). There may not be much difference in practice between something being “neglected” or “occasionally addressed”, and it is also not clear how “occasionally” would be reliably measured. Similarly, I could not see a clear difference between level III “Detrimental research practices are addressed… but there is no academic climate that actively stimulates responsible conduct of research” and level IV “There is an academic climate that detects and acts upon detrimental research practices and actively stimulates responsible conduct of research, but only at the level of individual researchers”.

For “2. To teach for intellectual virtue”, I couldn’t see clear differences between level I “the university…. pays no attention to intellectual virtues” and level II “intellectual virtues are considered important, but not taught”. I also could not imagine any University that would fall into either of these categories.

Again, I couldn’t see clear differences for the proposed levels I and II for “3. To address the big questions of life”, ie these questions being “neglected” (level I), versus “mentioned but discarded” (level II).

For “4. To give humanistic inquiry and education a proper place”, I also saw no real distinction between the humanities being “marginalized” (level I) and “not marginalized but considered and treated as being inferior” (level II). Levels III, IV, and V commonly refer to humanities being given a “proper place” in the university, but this definition will naturally vary between universities according to their type, i.e. technical universities versus universities with a strong and stated focus upon the humanities. Some wording within levels III-V also lacked definition, i.e. “in isolation” (level III), “some other disciplines” (level IV), and “across disciplines” (level V). “Disciplines” can be defined quite narrowly by some universities, so working with other disciplines or across disciplines does not always imply working between the humanities and the sciences.

Finally, for “5. To serve society”, universities where “research and teaching are confined to …purely academic challenges” (level I) could argue that they are serving society in this way, possibly by pursuing academic challenges that can indirectly linked to the big questions of life. It also seems unlikely that any university would be defined by level II “Research and teaching identify societal challenges, but the university leaves it to others to confer knowledge … relevant to those challenges”.

Are all factual statements correct and adequately supported by citations?

Contemporary challenges such as hypercompetition, publication pressures, marginalisation of the humanities and commercialization of universities (page 3, paragraph 2) should be supported by references.
The status of reference 1 (footnote 1) which is a manuscript by Peels et al, could be updated (page 3).

Are arguments sufficiently supported by evidence from the published literature?

Although the manuscript states that “more transparency and specifically pre-registration of study protocols and data analysis plans will lead to considerable improvements (in the use of small sample sizes, selective reporting and other questionable research practices, from the preceding sentence), this seems to be somewhat of an overstatement. Firstly, more transparency and pre-registration may not impact on the use of small sample sizes, as these might represent feasibility constraints that pre-registration may not overcome. Secondly, I understand that there is not yet a substantial body of literature that describes the possible benefits and/or drawbacks of study pre-registration, by comparing the results and outcomes of pre-registered studies versus comparable studies that lacked pre-registration.

Finally, the manuscript states that (the big questions of life) “are too important to be left entirely to non-academics”. This statement could be written in a more inclusive way, for example by indicating that as these questions are so important, they are everyone’s concern, including the concern of academics.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Partly
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Molecular biology, molecular genetics, human tissue biobanking, research integrity

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

06 Jul 2020

Author Response

Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to ... Continue reading Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB’s remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.

Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.

Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: “Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements.” We didn’t consider this to be a particularly strong claim, but we have now changed it into: “Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements.”

Finally, we have adopted JB’s suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: “The big questions are everyone’s concern, academics included.”
Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB’s remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.

Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.

Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: “Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements.” We didn’t consider this to be a particularly strong claim, but we have now changed it into: “Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements.”

Finally, we have adopted JB’s suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: “The big questions are everyone’s concern, academics included.”
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

06 Jul 2020

Author Response

Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to ... Continue reading Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB’s remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.

Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.

Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: “Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements.” We didn’t consider this to be a particularly strong claim, but we have now changed it into: “Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements.”

Finally, we have adopted JB’s suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: “The big questions are everyone’s concern, academics included.”
Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB’s remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.

Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.

Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: “Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements.” We didn’t consider this to be a particularly strong claim, but we have now changed it into: “Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements.”

Finally, we have adopted JB’s suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: “The big questions are everyone’s concern, academics included.”
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 01 Jul 2019

Britt Holbrook, Department of Humanities, New Jersey Institute of Technology, Newark, NJ, USA

Bart Penders, Maastricht University, Maastricht, The Netherlands

Approved with Reservations

https://doi.org/10.5256/f1000research.21335.r49894

This article proposes what Peels et al. call a “normative taxonomy” of the “Big Five” epistemic responsibilities universities ought to meet: (1) fostering research integrity, (2) teaching (for) intellectual virtues, (3) addressing the big questions, (4) valuing the humanities, and (5) serving society. This normative taxonomy essentially serves as an analytical rubric designed to assess the performance of universities. One could easily imagine – and in fact, Peels et al. propose as much in §3 – refining the rubric presented in Table 1, using it to assess the performance of various universities, and then comparing their “Big Five” scores to their world university rankings. We agree that undertaking these tasks presents many enticing possibilities, especially to those of us inclined against bean counting approaches to accountability. Nevertheless, since they position themselves as aspiring university rankers, they should address the range of critiques targeting existing rankings, their methodologies, and the performativity of these rankings. They should then situate their own approach within that context, arguing especially that it is an improvement on – and escapes the criticisms levelled at – existing rankings. This is our global concern.

Below, we offer a number of other substantial concerns, followed by a shorter list of minor issues to consider:

Why these five?
Peels et al. propose five epistemic responsibilities of universities as the Big Five. The authors point out that they consider each of the Big Five equally important. They also defend each of the five as indeed responsibilities that universities ought to meet. However, they fail to argue that these five epistemic responsibilities are the most important epistemic responsibilities for universities, that these five are in fact equally important, that these five are sufficient to capture the epistemic responsibilities universities ought to meet, or that universities have no other equally worthy responsibilities that ought to be included.

Why five?
We agree that the proposed Big Five are epistemic responsibilities that universities ought to meet. Using the proposed rubric to evaluate universities might provide valuable information about particular areas of strength and weakness at a given university. However, designing the rubric around five separate epistemic responsibilities suggests that universities could meet each of their epistemic responsibilities to varying degrees. One university might be particularly strong in terms of fostering research integrity but fail miserably at teaching for intellectual virtue. Another might excel at addressing the big questions but do a lousy job of valuing the humanities. Although these possibilities sound prima facie plausible, using an analytic approach to epistemic responsibilities also generates some less than desirable outcomes. Could a university that fails to teach for intellectual virtue really excel at fostering research integrity? That seems plausible only if we have a very limited procedural view of what counts as fostering research integrity. Without valuing the humanities, how could a university excel at addressing the big questions? Unless we adopt scientism, the answer seems to be that it could not. Similar problems arise when we consider the relations between the other proposed epistemic responsibilities. The use of a taxonomical vocabulary further strengthens this worry. Taxonomies should display not only differences, but also relationships and similarities. The current form of representation is not, in that sense, a taxonomy, but (for the most part) a normative list. Only when the ties binding the elements together are conceptualised convincingly can we convincingly speak of a taxonomy. For this reason, we suggest that the authors consider enriching their list to raise it to the level of a taxonomy or abandon talk of a ‘taxonomy’ altogether. The authors could adopt the vocabulary of an analytical rubric, instead.

Even if the authors decide to use the language of rubrics rather than taxonomies, we suggest that the authors consider a more holistic approach along with an analytic one, just as virtue ethics appeals not only to the individual virtues, but also to the question of character. Rather than describing teaching for intellectual virtue as part of Bildung, perhaps the authors could consider fostering the ability to pursue Wissenschaft while cultivating Bildung as analogous to a university’s character. Call it the W-B rubric, under which different (now, no longer separate) epistemic responsibilities could be grouped together to indicate different levels of overall achievement. In order for a university to score best on the W-B rubric, it would have to excel at meeting all of its epistemic responsibilities (for a brief discussion of the difference between analytic and holistic rubrics, as well as examples and further references, see Rubrics: useful assessment tools. Centre for Teaching Excellence, University of Waterloo¹).

Why insist on separating epistemic from other responsibilities universities have?
This question follows on the previous one, in that it aims to nudge the authors away from using only an analytic approach. By insisting on distinguishing epistemic responsibilities from moral, practical, and social responsibilities, the authors impoverish the idea of what it means for universities to serve society. If the goal is to work towards alternative forms of assessment and accountability (thereby politicizing them further), excluding other responsibilities and artificially fragmenting the landscape accordingly, one would expect [a] the epistemic responsibility taxonomy to be situated within a larger responsibility taxonomy and/or [b] an argument that leaving other responsibilities aside is preferable. More in line with the suggestion to pursue a more holistic approach, we must ask: could we really say that a university is a good university if it meets only its epistemic responsibilities? We think not. Perhaps the W-B rubric we suggest, or another holistic approach, could help address this issue. Note that one could also include analytical rubrics for moral, practical, and social responsibilities of universities and incorporate the responsibilities included in each of these separate rubrics into a holistic approach.

Our point is not that analysis is always bad. Indeed, analysis can be quite informative and useful as part of a formative evaluation. We are simply suggesting that the authors consider using their analytic approach to complement a more holistic approach.

Who counts, and why do they count?
In §3, the authors suggest optimising their rubric by means of a Delphi Study “with international experts”, which could lead to fine-tuning it, but possibly also to radical revisions. If the latter is the case, then what is the status of the current five epistemic responsibilities? When it comes to the actual Delphi study, whom they choose to participate in the study will likely make a very big difference to the final design of the rubric. Presumably, the authors focus on “experts” because they assume experts know more about epistemic responsibilities than non-experts do. Yet, the way in which the authors present it suggests a retreat to the deficit/diffusion model of public understanding of science. Not only has the deficit model been shown to be factually incorrect, it also presupposes a social contract for science and scholarship that imagines universities as ivory towers. In their discussion of a university’s responsibility to serve society epistemically, the authors focus on the supply side of knowledge production, suggesting that knowledge dissemination is how best to serve society. Arguably, however, serving society – even if we limit this to an epistemic responsibility – means something other than telling society what we academics think they need to know. Here again, the separation between epistemic and other responsibilities creates a lot of friction, since a departure from the deficit model requires a high degree of interaction and participation beyond universities. In fact, one could argue that interaction and participation are epistemic requirements (co production of knowledge).

Along these lines, we suggest that non-experts may have valuable feedback to offer, even if the authors ultimately decide only to pursue the development of an analytic rubric for epistemic responsibilities of universities. One way to include the demand side of knowledge production in the design of their rubric would be to recruit non-expert stakeholders to participate in the proposed Delphi Study and workshops or organise parallel expert (Delphi) and citizen (Citizen Summit) consultations.

We also have a few more minor points that nevertheless warrant attention:

Despite the argument that the rubric is to be applied to entire universities, the responsibilities focus very much on individuals and groups (which seem to be multiple individuals in the ways in which they are discussed) and less on the level of structures and collectives.
Peels et al. frame irreproducibility solely as the result of sub-par science and thus as a research integrity issue. Their previous work, as well current scholarly debates on the characters and qualities or irreproducibiluty and irreplicability, takes up a much more nuanced position. Perhaps the authors will consider adding some of that nuance here.
Excusing a few universities (technical universities or polytechnics, in this case) of taking responsibility for one of the epistemic responsibilities (#4) suggests that giving humanistic inquiry and education a proper place is optional. This seems to conflict with the idea that all responsibilities are equally important. It also opens up possibilities for policies that deprioritise humanities research and teaching (cf. the current Van Rijn report in NL).

Is the topic of the opinion article discussed accurately in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Rubrics: Useful Assessment Tools. Centre for Teaching Excellence, University of Waterloo. 2019. Reference Source

Competing Interests: The reviewers are currently engaged in an ongoing debate with the authors of the paper on whether the humanities need a replication drive like that currently ongoing for the sciences. Since the reviewers take the negative side of that debate – we do not think the humanities need a replication drive – and the authors defend the affirmative side, someone might believe we are incapable of offering a fair review. The debate is cordial, however, and one of the authors (Peels) has been included as a presenter in a session conference session on the topic organized by the reviewers. We expect that (fruitful) debate to continue, although it is possible that we could reach consensus on the matter. Although we offer a critical review here, we do so in the spirit of helping the authors strengthen their arguments and not from any ill will.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

CITE

Report a concern

Author Response 06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

06 Jul 2020

Author Response
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. ... Continue reading
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. First, that we situate our taxonomy in the wider context of other rankings. Second, that we address criticisms of existing rankings, and explain why we think ours is an improvement.
H&P are right that there are many other rankings, even many other non-standard rankings that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn’t immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we’ll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.

Why These Five?
H&P note that we don’t argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)–(c), H&P are right that more support is needed to back up these claims – the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that’s part of the abovementioned new project.
As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding. That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy—for a taxonomy “should display not only differences but also relationships and similarities”. Instead of offering a taxonomy, they urge, we could also offer rubrics.
            We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn’t mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities’ epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
            As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.

A More Holistic Rubric?
H&P add that just as the virtues of an individual person are expressions of that person’s character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung.
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university’s character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
            We certainly didn’t mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
            The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points:

We focus, they say, more on individuals than on structures and collectives. We’re not sure what gave rise to this impression – as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.

In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We’ve done this in the revised version of the paper (note 9).

We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. First, that we situate our taxonomy in the wider context of other rankings. Second, that we address criticisms of existing rankings, and explain why we think ours is an improvement.
H&P are right that there are many other rankings, even many other non-standard rankings that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn’t immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we’ll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.

Why These Five?
H&P note that we don’t argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)–(c), H&P are right that more support is needed to back up these claims – the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that’s part of the abovementioned new project.
As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding. That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy—for a taxonomy “should display not only differences but also relationships and similarities”. Instead of offering a taxonomy, they urge, we could also offer rubrics.
            We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn’t mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities’ epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
            As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.

A More Holistic Rubric?
H&P add that just as the virtues of an individual person are expressions of that person’s character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung.
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university’s character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
            We certainly didn’t mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
            The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points:

We focus, they say, more on individuals than on structures and collectives. We’re not sure what gave rise to this impression – as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.

In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We’ve done this in the revised version of the paper (note 9).

We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

06 Jul 2020

Author Response
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. ... Continue reading
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. First, that we situate our taxonomy in the wider context of other rankings. Second, that we address criticisms of existing rankings, and explain why we think ours is an improvement.
H&P are right that there are many other rankings, even many other non-standard rankings that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn’t immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we’ll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.

Why These Five?
H&P note that we don’t argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)–(c), H&P are right that more support is needed to back up these claims – the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that’s part of the abovementioned new project.
As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding. That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy—for a taxonomy “should display not only differences but also relationships and similarities”. Instead of offering a taxonomy, they urge, we could also offer rubrics.
            We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn’t mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities’ epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
            As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.

A More Holistic Rubric?
H&P add that just as the virtues of an individual person are expressions of that person’s character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung.
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university’s character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
            We certainly didn’t mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
            The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points:

We focus, they say, more on individuals than on structures and collectives. We’re not sure what gave rise to this impression – as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.

In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We’ve done this in the revised version of the paper (note 9).

We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.
We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. First, that we situate our taxonomy in the wider context of other rankings. Second, that we address criticisms of existing rankings, and explain why we think ours is an improvement.
H&P are right that there are many other rankings, even many other non-standard rankings that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn’t immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we’ll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.

Why These Five?
H&P note that we don’t argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)–(c), H&P are right that more support is needed to back up these claims – the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that’s part of the abovementioned new project.
As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding. That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy—for a taxonomy “should display not only differences but also relationships and similarities”. Instead of offering a taxonomy, they urge, we could also offer rubrics.
            We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn’t mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities’ epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
            As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.

A More Holistic Rubric?
H&P add that just as the virtues of an individual person are expressions of that person’s character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung.
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university’s character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
            We certainly didn’t mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
            The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points:

We focus, they say, more on individuals than on structures and collectives. We’re not sure what gave rise to this impression – as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.

In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We’ve done this in the revised version of the paper (note 9).

We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 13 Jun 2019

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 06 Jul 20	read
Version 1 13 Jun 19	read	read

Bart Penders, Maastricht University, Maastricht, The Netherlands

Britt Holbrook, New Jersey Institute of Technology, Newark, USA
Jennifer Byrne, The Children's Hospital at Westmead, Sydney, Australia

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

36 Views

04 Mar 2021 | for Version 2

Bart Penders, Maastricht University, Maastricht, The Netherlands

Britt Holbrook, Department of Humanities, New Jersey Institute of Technology, Newark, NJ, USA

36 Views Cite this report Responses(1)

Approved

We appreciate the authors’ responses to our comments. Their changes to the manuscript manage to alleviate many of the concerns we have. However, one key concern – the Big One – remains: the authors' aspiration to act as rankers themselves. Even if the ranking they propose were conceptually and methodologically flawless– and we trust that the authors would not be willing to make such a bold claim to perfection, even after the extensive testing and validation they propose – the ranking would still result in moral and political problems associated with ranking.

In a wonderfully incisive Twitter thread (https://twitter.com/jelena3121/status/1356210115406397443), Jelena Brankovic, a long-time researcher on rankings, suggests that "Even if you made [rankings] super transparent, accountable, and well-governed, that could even aggravate the adverse effects." No ranking merely documents performance. The act of ranking necessarily inflates minor differences into different ranks. Every ranking is a zero sum game. One university can rise in the rankings only if another goes down. Insofar as every ranking is a zero sum game, every ranking generates reputation. Every ranking produces competition for that reputation. Indeed, reputational ranking becomes the One Big Goal for universities. The authors’ aspiration to turn the Big Five into a ranking will insure that, rather than working to improve their attention to their epistemic responsibilities, universities will treat those responsibilities merely as means to achieving a higher ranking.

We would like to invite the authors to frame their ambitions more cautiously. Used as a rubric for relatively low stakes formative assessment, the Big Five might well improve the performance of some universities. Deployed as a ranking for all universities, however, the Big Five will become all about the Big One.

Competing Interests

No competing interests were disclosed.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

10 Mar 2021

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Drs Penders and Holbrook worry that our Big Five would just add another specimen to the many rankings already in existence, thus reinforcing various undesirable moral and political downsides of rankings, such as competition and optimization to spurious indicators. After all, a university can go up in a ranking only if another goes down.

Our reply is pragmatic. We believe the development and use of rankings is pretty much unavoidable. Prospective students and job candidates will always want information shortcuts to sort through the hundreds or thousands of universities and colleges they might be interested in. If this is right, we would prefer that rankings are based on things that matter the most, such as meeting one’s foundational epistemic responsibilities concerning research, education, outreach, and service, rather than superficial indicators, such as citation counts and their derivatives or reputation based on surveys among peers.

The referees claim that universities will misuse our five responsibilities as just another option to achieve a higher ranking. But this is a rather gloomy prophecy. We prefer to take a slightly more optimistic view of universities and their academic staff and administrators. Why not trust that among the thousands of universities worldwide at least some will pursue these responsibilities not because they want to climb in the Big Five ranking per se, but because they are convinced that these are indeed core responsibilities that matter and that they therefore ought to be taken seriously? Moreover, even if they all did it for merely instrumental reasons, doing the right thing for bad reasons is surely to be preferred over doing the wrong thing for bad reasons?

Finally we would like to point out two things. First, it is practically impossible to select a university where you want to work or study on the basis of thousands of brochures and websites. Decision tools are essential and that makes academic ‘TripAdvisors’ indispensable. Second, nothing guarantees that we will be successful in designing a feasible ranking based on our Big Five responsibilities. But we feel it’s at least worth a try, so we will continue to explore the possibility.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

33 Views

11 Dec 2019 | for Version 1

Jennifer Byrne, NSW Health Statewide Biobank, School of Medical Sciences, Faculty of Medicine and Health, The Children's Hospital at Westmead, Sydney, NSW, Australia

33 Views Cite this report Responses(1)

Approved

Contemporary challenges such as hypercompetition, publication pressures, marginalisation of the humanities and commercialization of universities (page 3, paragraph 2) should be supported by references.
The status of reference 1 (footnote 1) which is a manuscript by Peels et al, could be updated (page 3).

Is the topic of the opinion article discussed accurately in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Partly
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Molecular biology, molecular genetics, human tissue biobanking, research integrity

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Jennifer Byrne (JB) first notes that the distinctions between the different levels of meeting our responsibilities remain somewhat unclear. This is a good and important point; these levels ought to be distinguished more clearly. In fact, this is one of the main goals of our new project entitled Epistemic Progress in the University (2020-2023) that was recently funded by the Templeton World Charity Foundation. JB’s remarks provide plenty of food for thought on the distinctiveness of the levels as we formulated them. We are very grateful for that and will certainly take these suggestions into account while developing the taxonomy further. We now mention the new project in the introduction and state that we need to do more work on the distinctiveness of the five levels postulated for each epistemic responsibility.

Second, we have updated the publication status of footnote 1 and added references to substantiate what we say on hypercompetition, publication pressure, marginalization of the humanities, and commercialization of universities.

Third, JB thinks that we overstate the benefits of pre-registration. We partly disagree. We wrote: “Although these phenomena are understudied, it is clear that more transparency and specifically preregistration of study protocols and data analysis plans will lead to considerable improvements.” We didn’t consider this to be a particularly strong claim, but we have now changed it into: “Although these phenomena are understudied, it is to be expected that more transparency and specifically preregistration of study protocols and data analysis plans will lead to improvements.”

Finally, we have adopted JB’s suggestion for a more inclusive formulation about whose concern the big questions are. The text now reads: “The big questions are everyone’s concern, academics included.”

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

40 Views

01 Jul 2019 | for Version 1

Britt Holbrook, Department of Humanities, New Jersey Institute of Technology, Newark, NJ, USA

Bart Penders, Maastricht University, Maastricht, The Netherlands

40 Views Cite this report Responses(1)

Approved With Reservations

Despite the argument that the rubric is to be applied to entire universities, the responsibilities focus very much on individuals and groups (which seem to be multiple individuals in the ways in which they are discussed) and less on the level of structures and collectives.
Peels et al. frame irreproducibility solely as the result of sub-par science and thus as a research integrity issue. Their previous work, as well current scholarly debates on the characters and qualities or irreproducibiluty and irreplicability, takes up a much more nuanced position. Perhaps the authors will consider adding some of that nuance here.
Excusing a few universities (technical universities or polytechnics, in this case) of taking responsibility for one of the epistemic responsibilities (#4) suggests that giving humanistic inquiry and education a proper place is optional. This seems to conflict with the idea that all responsibilities are equally important. It also opens up possibilities for policies that deprioritise humanities research and teaching (cf. the current Van Rijn report in NL).

Is the topic of the opinion article discussed accurately in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Rubrics: Useful Assessment Tools. Centre for Teaching Excellence, University of Waterloo. 2019. Reference Source

Competing Interests

The reviewers are currently engaged in an ongoing debate with the authors of the paper on whether the humanities need a replication drive like that currently ongoing for the sciences. Since the reviewers take the negative side of that debate – we do not think the humanities need a replication drive – and the authors defend the affirmative side, someone might believe we are incapable of offering a fair review. The debate is cordial, however, and one of the authors (Peels) has been included as a presenter in a session conference session on the topic organized by the reviewers. We expect that (fruitful) debate to continue, although it is possible that we could reach consensus on the matter. Although we offer a critical review here, we do so in the spirit of helping the authors strengthen their arguments and not from any ill will.

Respond to this report

Responses (1)

Author Response

06 Jul 2020

Rik Peels, Philosophy Department, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

We thank the reviewers for their helpful comments and suggestions. Below, we reply to them.

Relation to Other Rankings
Holbrook and Penders (H&P) urge that we do two things. First, that we situate our taxonomy in the wider context of other rankings. Second, that we address criticisms of existing rankings, and explain why we think ours is an improvement.
H&P are right that there are many other rankings, even many other non-standard rankings that are not merely cashed out in terms of publications. Yet, many of them are based on quantitative data, such as (i) number of students; (ii) granted research proposals; (iii) Nobel prize winners; (iv) citation indices; (v) the result of surveys on the opinions on stakeholders with respect to e.g. the reputation of universities. Our proposal focuses on epistemic responsibilities. Whereas other rankings are mainly based on quantitative data and opinions of stakeholders, the instrument we propose is based on mainly qualitative data, and will lead to an ordering in normative categories. Because of these differences, it isn’t immediately clear that extant criticisms of other rankings would apply to our proposed taxonomy. But we’ll look into this more systemically in future work as part of a new project on Epistemic Progress in the University (also funded by the Templeton World Charity Foundation). We explain this in the introduction and the section on Future steps.

Why These Five?
H&P note that we don’t argue (a) that our five are the most important areas of responsibility; (b) that they are equally important; (c) that they are sufficient to capture the epistemic responsibilities of universities; (d) that universities have no other equally worthy responsibilities.
As to (a)–(c), H&P are right that more support is needed to back up these claims – the current paper lays out a proposal based on a survey of literature about epistemic responsibilities of the university and our own analysis thereof. But providing further support is exactly what we aim to do in follow-up work that’s part of the abovementioned new project.
As to (d), universities indeed have other, equally important responsibilities. Among them are moral responsibilities (like providing safe environments for students and faculty, and taking care of the wellbeing of human and animal test subjects, treating indigenous people fairly); legal responsibilities; financial responsibilities; social responsibilities (like producing useful technologies and effective medical interventions), and more. What sets universities apart from many other organizations and institutions is that their main goals are epistemic in nature: their primary aims are to produce and disseminate knowledge and understanding. That is why we focus on these. We now make this more explicit in the introduction.

The Relations between Various Values
H&P say that the values are interrelated (if there is research integrity, there will or must be such a thing as teaching virtue; and how can universities address the Big Questions without cultivating the humanities?) and that we do not acknowledge this fact. And this, they maintain, undermines our goal of offering a taxonomy—for a taxonomy “should display not only differences but also relationships and similarities”. Instead of offering a taxonomy, they urge, we could also offer rubrics.
            We agree that the epistemic responsibilities we distinguish are related to each other. For instance, responsibility 1 (to foster research integrity) surely has to do with responsibility 2 (to teach for intellectual virtue): it would be highly surprising if a university is serious about research integrity but does not teach for curiosity, open-mindedness, thoroughness, and intellectual perseverance. However, this doesn’t mean that we cannot draw analytic distinctions between the responsibilities that allow us to focus on different parts and aspects of universities’ epistemic responsibilities. It may not always be possible (or even desirable) in practice to implement these responsibilities through separate activities, interventions, or policies, but they can be distinguished through an analytical lens. We consider that an advantage of our approach. Furthermore, the exact relations between these responsibilities is up for debate, so we wanted to leave room for different opinions about the relations between these responsibilities while working towards more consensus that these are indeed key epistemic responsibilities of universities. We have added this reasoning at the end of section 2.
            As to the suggestion of explicating relationships and dependencies and developing a rubric rather than a taxonomy, we are happy to take these ideas into consideration as we continue the development of this proposal in future work.

A More Holistic Rubric?
H&P add that just as the virtues of an individual person are expressions of that person’s character, so the university has (or should have) a character, and their friendly suggestion is that the character of the university is To pursue Wissenschaft while cultivating Bildung.
We are sympathetic to this suggestion; universities can have relatively stable dispositions to pursue particular values reliably. We might call this the university’s character or, alternatively, its organizational culture. We take it, though, that this suggestion is already covered by our proposal. It is an epistemic responsibility of universities to give the humanities a proper place next to the natural and social sciences. Our focus on intellectual virtues, moreover, is very much in line with the ideal of Bildung.

Deficit/diffusion Model of Public Understanding
H&P say that we rely on the incorrect deficit/diffusion model of science communication: the public has a deficit of knowledge, and science communication aims to diffuse the deficit by providing knowledge.
            We certainly didn’t mean to assume this model, although we can see how our focus on knowledge dissemination under the epistemic responsibility of serving society might give the impression that the role of universities merely is to better inform the public. To prevent this misunderstanding, we have added a new paragraph where we explicitly state that the responsibility of serving society can be a two-way street, where citizens can offer valuable input on which research questions to pursue, how to best pursue them, and how to communicate and use research findings. In some cases, the production of scientific knowledge can even amount to co-creation between scientists, stakeholders, and citizens.
            The suggestion to include citizens and various stakeholders in proposed Delphi studies is a valuable one, that we will take onboard.

Smaller Points
H&P raise three smaller points:

We focus, they say, more on individuals than on structures and collectives. We’re not sure what gave rise to this impression – as we think of them, all five responsibilities really are responsibilities of the university as a whole, so both its leadership and the individuals affiliated with it, as well as its organizational structure, policies, incentives, etc. We are keenly aware that meeting these responsibilities is as much an issue of individual action as it is of collective action, formal and informal organizational structures and the incentives they give rise to. And, in fact, even broader structures and systems in society. In developing our proposal further, however, we will make sure to keep both the individual and the organizational level in focus.
In this paper we suggest that irreproducibility is a sign of sub-par science, so H&P say, and they suggest that we include in a further version of our paper our more nuanced (and already published) views. We’ve done this in the revised version of the paper (note 9).
We make it seem as if the humanities are optional. This may be a misunderstanding. All we wanted to do is acknowledge the existence of technical universities, which may not need fully-fledged humanities departments. We do emphasize that they too have responsibilities in addressing the big questions.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Academia’s Big Five: a normative taxonomy for the epistemic responsibilities of universities

Abstract

Keywords

1. Introduction

Table 1. Normative taxonomy of epistemic responsibilities of universities.

2. A normative taxonomy

3. Future steps

Data availability

Grant information

Footnotes

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated