F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.130188.1 Method Article Articles Autonomous, bidding, credible, decentralized, ethical, and funded (ABCDEF) publishing

[version 1; peer review: 2 approved with reservations]

 Oka Taiki Conceptualization Project Administration Visualization Writing – Original Draft Preparation Writing – Review & Editing 1 2 3 Takashima Kaito Conceptualization Funding Acquisition Visualization Writing – Original Draft Preparation Writing – Review & Editing 4 Ueda Kohei Conceptualization Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-1818-8366 4 Mori Yuki Conceptualization Visualization Writing – Original Draft Preparation Writing – Review & Editing 4 Sasaki Kyoshiro Conceptualization Funding Acquisition Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-5496-3748 5 Hamada Hiro Taiyo Conceptualization Supervision Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-7919-9125 6 Yamagata Masahito Conceptualization Supervision Writing – Original Draft Preparation Writing – Review & Editing 7 Yamada Yuki Conceptualization Funding Acquisition Project Administration Supervision Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-1431-568X a 8 Department of Neuropsychiatry, Faculty of Life Sciences, Kumamoto University, Kumamoto, Japan The Department of Decoded Neurofeedback, Computational Neuroscience Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan Clinical Psychology, Graduate School of Human Sciences, Osaka University, Suita, Japan Graduate School of Human-Environment Studies, Kyushu University, Fukuoka, Japan Faculty of Informatics, Kansai University, Takatsuki, Japan Research & Development Department, Araya Inc., Tokyo, Japan Center for Brain Science and Department of Molecular and Cellular Biology, Harvard University, Cambridge, USA Faculty of Arts and Science, Kyushu University, Fukuoka, Japan yamadayuk@gmail.com

Competing interests: TO, KT, KU, YM, KS, and YY are members who established MinDAO, while there are no conflicts of interest to report among the other authors.

 25 7 2023 2023 12 877 10 7 2023 Copyright: © 2023 Oka T et al. 2023 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Scientists write research articles, process ethics reviews, evaluate proposals and research, and seek funding. Several strategies have been proposed to optimize these operations and to decentralize access to research resources and opportunities. For instance, we previously proposed the trinity review method, combining registered reports with financing and research ethics assessments. However, previously proposed systems have a number of shortcomings, including how to implement them, e.g., who manages them, how incentives for reviewers are paid, etc. Various solutions have been proposed to address these issues, employing methods based on blockchain technologies, called “decentralized science (DeSci)”. Decentralized approaches that exploit these developments offer potentially profound improvements to the troubled scientific ecosystem. Here, we propose a system that integrates ethics reviews, peer reviews, and funding in a decentralized manner, based on Web3 technology. This new method, named ABCDEF publishing, would enhance the speed, fairness, and transparency of scientific research and publishing.

 decentralized science registered reports reproducibility scientific ecosystem funding JST SPRING JPMJSP2136 JSPS KAKENHI JP22K13881 andJP18K12015 JP20H04581 21H03784 and22K18263. This research was supported by JST SPRING: JPMJSP2136 (KT) and JSPS KAKENHI: JP22K13881 (KS), and JP18K12015, JP20H04581, 21H03784, and 22K18263 (YY). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

“By mutual confidence and mutual aid, Great deeds are done, and great discoveries made.”—Homer, “The Iliad”

Current scientific endeavors have become complicated and protracted. Science advances by sifting through numerous findings made by many individuals and organizations. However, the current ecosystem of science suffers from various problems. First, researchers spend great amounts of time writing grant proposals to obtain research funding rather than conducting scientific investigations (Herbert et al. 2013). Funding decisions rely heavily on citation metrics, such as researcher h-indexes. 1 , 2 Second, it is now standard for human and animal research to undergo ethics reviews before being initiated. However, these reviews are usually carried out behind closed doors by ethics committees composed of members selected by each institution. 3 Additionally, such reviews are often ill-suited to evaluate research methods because the committees do not include experts in pertinent fields. 4 Moreover, approved proposals must often be revised due to unforeseen changes in experimental methodology, necessitating reassessment by post hoc peer review. 5 Accordingly, much research effort is wasted. Third, infrastructure to support reproducibility and transparency has not yet been fully developed, 6 – 8 although there are a plethora of efforts to advance open science, such as open access, open data/material/code, open review reports, open peer review, pre-registration, and registered reports systems. Finally, there is excessive centralization of authority, 9 , 10 and funding is often concentrated on limited numbers of scientists and publishers. 11 , 12

To address these issues, we previously proposed a new procedure in which three types of peer review (scientific peer review, ethics review, and research funding review) are executed simultaneously on the same document. 13 This time-saving method is promising because it could solve the transparency problems, ethics review problems, and grant acquisition problems mentioned above. Nevertheless, the proposed system still relies on volunteer work of reviewers and influential publishers. Contrary to Homer’s assertion, modern scientific endeavor is generally performed in an opaque, unidirectional, biased, and centralized system.

As an alternative to this traditional system, Decentralized Science (DeSci) activities or systems are gaining popularity. 14 , 15 Multiple scientific entities are beginning to apply Web3 technology, most notably blockchain, to obtain research funding and to publish results. 16 Web3 is a general term for distributed networks where users autonomously exchange information and communicate. These entities are often run by autonomous organizations of scientists, called decentralized autonomous organizations (DAOs). Within a DAO, there is a central organization such as a working group (WG), but decision-making is decentralized, based on Web3 technology. Several DAOs are attempting to address the aforementioned problems. 17 , 18 Those DAOs deploy a review system created on a public blockchain, e.g., Ethereum, to tackle transparency and incentive problems that have impaired conventional peer review. 17 , 19 In order to promote ethical behavior and inclusiveness, DAO systems implement a gamification mechanism that allows entire communities to evaluate peer reviews and vote for the best ones. There are also several DAOs that provide open budgets or data, e.g., VitaDAO, GenomesDAO. Such DAOs collect their budgets by distributing tokens as voting rights to determine which projects are to be funded. Open data are held in decentralized storage, e.g., InterPlanetary File System: IPFS, without censorship, and data transactions can be kept in the blockchain as an open ledger. Practically, however, DeSci systems have potential drawbacks. For example, they could damage interests of current publishers. Nonetheless, we hope this emerging movement will solve presently unresolved problems, e.g., payment of incentives to reviewers, stability of reproducibility. 16 , 19 , 20

Here, we propose a system that integrates ethics reviews, peer reviews, and funding in a decentralized manner, based on Web3 technology. This autonomous, transparent, decentralized system would help shape cutting-edge scientific research and boost scientific transparency, efficiency, ethics, and reproducibility. We have already established a decentralized community called MinDAO to realize such a system as a host.

 ABCDEF publishing

Figure 1 illustrates the scheme of a system we call “Autonomous, bidding, credible, decentralized, ethical, and funded (ABCDEF) publishing.” based on MinDAO. See the original concept described in our published article and its supplementary (Mori et al. (2022), 13 Supplementary ( https://osf.io/rq5vb) for a detailed, phase-by-phase peer review scheme. After researchers write a research plan, there are five phases from reviewing to publishing. Here are how these phases would work.

Flow of ABCDEF-publishing. Phase 1. Proposal and review

A. Scientific peer review and registered reports (RR)

Each research proposal is peer-reviewed by multiple reviewers. In this process, peer reviewers evaluate the value of a research question, the rationale for the hypothesis, and the validity of methods for testing the hypothesis. RRs focus on the quality of methodology prior to data collection. Even in cases of exploratory research, peer reviewers evaluate scientific soundness, feasibility, and methodology of submitted proposals. 21

B. Ethics reviews

Several reviewers specializing in ethical analyses are assigned to each protocol to review ethics of the research. If reviewers have ethical concerns about a submitted protocol, authors must revise it in compliance with reviewer comments and judgments of editors. In cases in which ethical considerations not covered by the protocol are required, e.g., clinical research, the protocol would be examined by author institutions.

Scientific reviewers assess the importance of research questions of a manuscript, the theoretical validity of proposed hypotheses, and whether experimental methods are appropriate for testing those hypotheses. Ethics reviewers assess the protocol for ethical issues. Scientific and ethics reviews are undertaken concurrently. Once the manuscript has successfully passed both reviews and is accepted in principle, authors can proceed to the funding review. If ethics reviewers determine that an institutional ethics review is necessary, an ethics review is conducted at author institutions. In this case, a funding review is possible only after passing institutional ethics reviews.

 Phase 2 Blockchain voting for funding

Once a protocol is accepted in Phase 1, voting for funding begins. The desired amount of funding is announced at the time of Phase 1 Peer Review. Voting on Phase 1 protocols occurs at this point, and funding is awarded from pooled funds. Community members evaluate Phase 1 protocols on their potential for scientific advancement and benefit to the public, and grants are made to protocols that meet these criteria. Voting is conducted primarily by community members possessing tokens. Although votes of individuals with more tokens have greater weight, they apply a voting system, such as quadratic funding, to minimize bias 22 (see the below details). Even if members disagree with funding for the protocol, researchers can proceed to the next phase using their own grants if they have them. Alternatively, researchers can revise their protocols and resubmit them for Phase 1 scientific review to be voted on again ( Figure 1). Also, DAO members who have the right to vote can require additional actions in both cases, i.e., agreeing or disagreeing with comments by DAO members. If members disagree, but ask authors to revise their proposals, researchers can modify their protocols to obtain a grant.

In the traditional system, funding based on popularity and performance assessed with indicators such as the h-index is not necessarily a problem. However, funds are biased toward a small subset of researchers, and funding allocation may be skewed away from early-career researchers. To solve this issue, we will apply “quadratic funding,” limiting the percentage of famous and accomplished researchers receiving funds by route (√). 22 Quadratic Funding optimizes matching funds by prioritizing projects according to the number of voters. This ensures that funds are spent on projects that truly benefit the scientific community at large, not just a few prominent researchers.

Under this mechanism, the total amount of funding for a given issue is “the square root of each donation for that issue, i.e., the square of the total amount”. Even small donations are strongly encouraged and funding democratically balances funds for the public good. After a successful funding vote, researchers can be supported by pooled funding sources. Participation in voting is restricted to credibility gained via activities 23 or Token holdings.

 Phase 3 Research and writing

This is the phase in which actual research and writing proceed, following the same flow as typical funded research. As described below, methods, experiments, and analyses conducted here are tied to the Phase 1 protocol in the blockchain. Therefore, deviations from or additions to the RR protocol and ethical review are explicitly indicated. Researchers conduct their studies following procedures accepted in Phase 1. Procedures are linked to the Phase 2 article using the blockchain. Therefore, if researchers deviate from their original protocol, this is apparent to readers.

 Phase 4 Peer review for the final report

This is a review resulting from protocols proposed and accepted in principle in Phase 1. Reviewers evaluate the quality of modifications and protocol deviations or ethical violations. In the event of rejection, after consultation, researchers would be asked to return the funds. Of course, peer reviewers should attempt to improve manuscripts by providing constructive discussions, as in traditional scientific peer review. Authors resubmit a Phase 2 article after data collection and analysis that contains the “Introduction” and “Methods” from the initial submission together with the “Results and Discussion” section. After the Phase 2 scientific evaluation is finished, the final manuscript will be published.

 Phase 5 Publishing in a friendly journal

The paper is published as a journal article. The community and the recommender make acceptance decisions as in Peer Community In (PCI) ( https://peercommunityin.org/), 24 and ABCDEF-friendly journals accept manuscripts selected, based on these recommendations.

Detailed handling of tokens and funds is described in Table 1. Through these five phases, transparency is ensured as the research process proceeds. Scientific reviewers and ethics reviewers would be invited by the community based upon license tokens, guaranteeing the quality of reviewers. Possible incentives for reviewers arising at each phase could include the following. Each reviewer/author would have a track record in the form of non-fungible tokens, which could be presented as credit for acquiring funding. 25 Incentives would be paid in tokens built on the blockchain. This incentive system is similar to that proposed in previous studies. 17 , 19 Researchers could also pay open access fees with tokens or other funds. Those grants would be provided from pooled funds. Funds would be handled by utilizing multiple funding systems, including donations or investments as external funding, and pooling funds with support from funding DAOs with quadratic funding and retrospective public goods funding. There could also be a type of bounty for other community members via programmed, self-executing contracts, so-called smart contracts.

Details of tokens and funding in each phase.
When do tokens or money move? Who obtains tokens or money? What kind of tokens or money are used?
Phase 0 Submission of proposals Assignment as reviewer Anyone invited as reviewers License token
Phase 1 Proposal and review Peer-review completed Reviewers MinDAO token
Phase2 Blockchain voting for funding Agreement to protocol Authors Money withdrawn in a pool of MinDAO for funds
Phase 3 Research and writing Phase 4 Peer Review for the final manuscript Peer-review completed Reviewers MinDAO token
Phase 5 Publication in the friendly journal Paying article processing charge (APC) ABCDEF-friendly journal MinDAO token or Money
Advantages and opportunities

This section highlights advantages that this system would bring to the academic community. First, ABCDEF Publishing combines advantages of conventional review methods and blockchain-based techniques. It reduces the burden on researchers and provides incentives for all peer-review processes from the perspective of DeSci. Our past proposal integrated RRs, ethical reviews, and grant reviews. Also, as mentioned above, some previous articles have suggested incentivizing reviewers using blockchain-based tokens. ABCDEF Publishing would integrate all of these methods to improve efficiency, fairness, and transparency of the review process. Specifically, since a record of details of this review (who reviewed it and how, etc.) is kept on the blockchain or in decentralized storage, this authentication assurance is also important. This transparency enables funders to validate the security of their funding. Also, it maintains a record of how the budget was used. Use of funds can be documented. Furthermore, the blockchain would be able to link ethics statements and hypotheses that are registered in the RR system with methodologies and results. If that linkage is not made, it is possible to know that the method and analysis did not explicitly satisfy the registration.

Second, decentralized management would remove control from the hands of a few powerful administrators, leading to promotion of diverse research that contributes to science and more neutral evaluations of science and researchers. In addition, ABCDEF publishing would also benefit researchers whose institutions do not have ethics review boards and independent researchers who do not belong to specific institutions. 26 , 27 This system would enable such researchers to review their protocols from an ethics perspective quickly and easily. Moreover, ABCDEF publishing would construct a reputational system based on activities of researchers/participants in a decentralized system of reviewers, independent of institutions and publishers (as in the Tenorio-Fornes proposal 19 ). Such a reputation system would allow independent researchers to be reviewed and to expand the review process range by themselves. The system of DeSci would also improve efficiency of all publishing procedures by decreasing office work using smart contract techniques, as previously proposed. 19

These advantages would allow ABCDEF-Publishing to achieve real distributed, citizen science. Nonetheless, ABCDEF Publishing does not seek to ignore or eliminate publishers, as has been claimed regarding the previously proposed DeSci system. 17 On the contrary, the credibility and reputation of publishers are critical and need to be supported. The previous proposal also recommended a review method using a DeSci system that is organized and transparent, while working with publishers. 19 We must respect the traditional scientific system that enables articles to be scientifically validated. Using ABCDEF Publishing, publishers could develop a more effective and efficient publishing system that would be mutually beneficial. Thus, it could help reform the complex systems of some journals, and indeed existing publishers are beginning to implement DeSci-based initiatives ( e.g., https://twitter.com/ScisetsComm).

 Potential weaknesses, threats, and solutions

There are five potential pitfalls for ABCDEF Publishing.

It is still possible that a WG and some members of the DAO will hold too much power. Therefore, establishing a more democratic method of DAO management, depending on the number of non-fungible tokens (NFTs) owned, is crucial. The ideal is to employ a managerial system that is genuinely autonomous and decentralized.

As is often the case with token economics, tokens are unstable. Linking internal tokens ($MIN in our case) to current stablecoins is one option, e.g., USDC, USDT. Stablecoins are digital assets focused on price stability. Due to price volatility brought on by the lack of underlying assets, crypto assets have not yet been used as a form of payment. Stablecoins were created to spread and boost the utility of these assets.

One concern is that fraudulent accounts, fabricated peer reviews, and fake contributions could appear. This issue is widely known as the Sybil attack, a type of attack on a computer service with many pseudonymous identities and enormous influence. Two types of defenses are proposed to thwart such abuses. One system is to evaluate risks of accounts using transaction data on the blockchain with machine learning. Another system uses non-exchangeable tokens linked to specific people, so-called soulbound tokens. 28 , 29 These systems allow us to predict risks of accounts and subtract the influence of potentially risky accounts. However, such proposals are still under development.

Another concern is about pairwise coordination of contributions in projects. For example, collaborators or unfriendly competitors work on the same project together in an attempt to control DAO decision-making. A similar idea in funding design is proposed to prevent such cases by controlling contributions based on social networks. 30 Another technique to minimize the contribution level to the cap has recently been used when users assist one another based on their shared activity history. 31

There may be instances in which these approaches fail, having various impacts, even on specific fields or where RRs are unnecessary, as in the humanities. Thus, a flexible flow, such as skipping RRs, might be required to rescue projects in some circumstances.

Increased transactions may impact “gas fees”, which are transaction costs in the public blockchain. DAOs often utilize a public blockchain like Ethereum, which requires gas fees for each transaction to maintain its services on the blockchain. The gas fee fluctuates according to the number of transactions within a certain time window, since the number of transaction times is limited, and the price incentivizes people to optimize timing of token trades. DAO activities can be hindered by such matters since they accrue high costs. Some off-chain solutions are proposed to mitigate such issues. In some solutions, including optimistic rollups and zero-knowledge rollups, security is derived directly from the public blockchain. Other solutions, such as sidechains and plasma chains, create new chains, and their security is distinct from the security of the public blockchain. In this manner, the number of transactions per second increases, and increased transactions do not hinder DAO activities due to a substantial reduction in gas fees.

Compatibility of reviewer systems is also a matter of concern. Our proposed system envisions integrating a reputation system for reviewers using the blockchain. However, the reliability of the reputation system is undeveloped and uncertain. Instead of integrating this system, it should be possible to integrate existing reviewer communities, including Clarivate, with the ABCDEF publishing system. In so doing, our publishing system would be compatible to the conventional reviewer system.

With ABCDEF publishing, which utilizes blockchain technology, there are limitations to how it can address environmental concerns, but there are also potential solutions to mitigate its impact. There are efforts to develop more energy-efficient blockchain algorithms and explore alternative energy sources, such as renewable energy, to power blockchain networks. Additionally, some decentralized publishing platforms are exploring using off-chain solutions to reduce the energy consumption required for transactions. While there are limitations to how ABCDEF publishing can address environmental concerns, there are potential solutions to mitigate its impact, such as developing more energy-efficient algorithms and promoting the reuse and recycling of electronic devices. It is important to continue exploring and implementing these solutions to ensure that the benefits of blockchain technology can be realized while minimizing its negative impact on the environment.

Some scientists might insist there are strict regulations and accredited committees to review research with human subjects or animals in many countries. Such committees usually do not process applications before studies are granted. Also, they might think there are huge differences between grant and ethics applications and an introduction plus the method section of a research publication. These criticisms are not strange to researchers accustomed to the current process. However, as we have argued in our earlier paper, 13 the process of applying for a grant and then conducting an ethical review may be one factor that has led to ethical deviations. Therefore, ABCDEF proposes the reversal of such a process. Moreover, as for grand and ethical considerations that require more than the background and methods section of the paper, those issues can be addressed by providing information as Supplementary information in the registered report.

Conclusion

We believe that ABCDEF publishing can effectively solve a panoply of problems facing the scientific community today. At the time of writing, we are just beginning to construct this system, which is community-based and developed on the blockchain. Most importantly, it will be realized as a large-scale movement involving various stakeholders, including scientists, publishers, and citizens.

 Data availability

OSF: ABCDEF publishing. https://doi.org/10.17605/OSF.IO/NSVZD. 32

This project contains the following extended data:

Supplementary Information.pdf

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

 References Mryglod O Holovatch Y Kenna R : Big fish and small ponds: why the departmental h-index should not be used to rank universities. Scientometrics. 2022 Jun;127(6):3279–3292. 10.1007/s11192-022-04373-y Barnes C : The emperor’s new clothes: theh-index as a guide to resource allocation in higher education. J. High. Educ. Policy Manag. 2014 Sep 3;36(5):456–470. 10.1080/1360080X.2014.936087 Lynch HF : Opening Closed Doors: Promoting IRB Transparency. J. Law Med. Ethics. 2018 Mar 1;46(1):145–158. 10.1177/1073110518766028 Page SA Nyeboer J : Improving the process of research ethics review. Res. Integr. Peer Rev. 2017 Aug 18;2:14. 29451537 10.1186/s41073-017-0038-7 PMC5803582 Abbott L Grady C : A systematic review of the empirical literature evaluating IRBs: what we know and what we still need to learn. J. Empir. Res. Hum. Res. Ethics. 2011 Mar;6(1):3–19. 21460582 10.1525/jer.2011.6.1.3 PMC3235475 Nosek BA Lakens D : Registered reports: A method to increase the credibility of published results. Soc. Psychol. 2014;45(3):137–141. 10.1027/1864-9335/a000192 Chambers CD Tzavella L : The past, present and future of Registered Reports. Nat. Hum. Behav. 2022 Jan;6(1):29–42. 34782730 10.1038/s41562-021-01193-7 Parikh S Malcom SM Moran B : Public access is not equal access. Science. 2022 Sep 9;377:1361. 10.1126/science.ade8028 Larivière V Haustein S Mongeon P : The oligopoly of academic publishers in the digital era. PLoS One. 2015;10:e0127502. 26061978 10.1371/journal.pone.0127502 PMC4465327 Lloyd J : How academic science gave its soul to the publishing industry. Issues Sci. Technol. 2020 [cited 2022 Sep 14]. Reference Source Berezin A : The perils of centralized research funding systems. Knowl. Technol. Policy. 1998 Sep 1;11(3):5–26. 10.1007/s12130-998-1001-1 Bol T Vaan M de Rijt A van de : The Matthew effect in science funding. Proc. Natl. Acad. Sci. U. S. A. 2018 May 8;115(19):4887–4890. 29686094 10.1073/pnas.1719557115 PMC5948972 Mori Y Takashima K Ueda K : Trinity review: integrating Registered Reports with research ethics and funding reviews. BMC. Res. Notes. 2022 May 19;15(1):184. 35590337 10.1186/s13104-022-06043-x PMC9118676 Hamburg S : Call to join the decentralized science movement. Nature. 2021 Dec;600(7888):221. 34876661 10.1038/d41586-021-03642-9 Sicard F : Can decentralized science help tackle the deterioration in working conditions in academia? Front. Blockchain. 2022;5:5. 10.3389/fbloc.2022.1066294 Wang FY Ding W Wang X : The DAO to DeSci: AI for Free, Fair, and Responsibility Sensitive Sciences. IEEE Intell. Syst. 2022 Mar;37(2):16–22. 10.1109/MIS.2022.3167070 Trovò B Massari N : Ants-Review: A Privacy-Oriented Protocol for Incentivized Open Peer Reviews on Ethereum. European Conference on Parallel Processing. 2020. 10.1007/978-3-030-71593-9_2 ResearchHub: ResearchHub. [cited 2022 Aug 28]. Reference Source Tenorio-Fornés Á Tirador EP Sanchez-Ruiz AA : Decentralizing science: Towards an interoperable open peer review ecosystem using blockchain. Inf. Process. Manag. 2021 Nov 1;58(6):102724. 10.1016/j.ipm.2021.102724 Cao L : Decentralized AI: Edge Intelligence and Smart Blockchain, Metaverse, Web3, and DeSci. IEEE Intell. Syst. 2022 May;37(3):6–19. 10.1109/MIS.2022.3181504 Dirnagl U : Preregistration of exploratory research: Learning from the golden age of discovery. PLoS Biol. 2020 Mar;18(3):e3000690. 32214315 10.1371/journal.pbio.3000690 PMC7098547 Buterin V Hitzig Z Weyl EG : Liberal Radicalism: A Flexible Design For Philanthropic Matching Funds. SSRN 3243656. 2018 [cited 2022 Sep 27]. Reference Source Atwood J : Understanding Discourse trust levels. Discourse. 2018 [cited 2022 Oct 5]. Reference Source Guillemaud T Facon B Bourguet D : Peer Community In: A free process for the recommendation of unpublished scientific papers based on peer review. In: Revue Africaine de Recherche en Informatique et Mathématiques Appliquées. hal.science. 2019. Reference Source Hamburg S : A guide to DeSci, the latest Web3 movement. Future. 2022 [cited 2022 May 25]. Reference Source Albagli S Iwama AY : Citizen science and the right to research: building local knowledge of climate change impacts. Humanit. Soc. Sci. Commun. 2022 Feb 1;9(1):1–13. 10.1057/s41599-022-01040-8 Kenens Van Oudheusden Yoshizawa : Science by, with and for citizens: rethinking “citizen science”after the 2011 Fukushima disaster. Palgrave Commun. 2020;6. 10.1057/s41599-020-0434-3 Reference Source Jain S Erichsen L Weyl G : A Plural Decentralized Identity Frontier: Abstraction v. Composability Tradeoffs in Web3. arXiv [cs.CR]. 2022. Reference Source Weyl EG Ohlhaver P Buterin V : Decentralized Society: Finding Web3’s Soul. 2022 [cited 2022 Sep 15]. Reference Source Pairwise coordination subsidies: a new quadratic funding design. Ethereum Research. 2020 [cited 2022 Oct 6]. Reference Source Chou H Lin D Nakaguchi T : TMchain: A Blockchain-based Collaboration System for Teaching Materials. J. Inf. Process. 2022;30:343–351. 10.1007/978-3-030-71593-9_2 Reference Source Yamada Y : Information related to the ABCDEF publishing article.2023. 10.17605/OSF.IO/NSVZD 10.5256/f1000research.142922.r199372 Reviewer response for version 1 Mellor David 1 Referee https://orcid.org/0000-0002-3125-5888 Center for Open Science, Charlottesville, Virginia, USA

Competing interests: I am an employee of the Center for Open Science (https://cos.io), which advocates for publishing workflows such as Registered Reports, which is a major feature of this proposed work.

 19 9 2023 Copyright: © 2023 Mellor D 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

In "Autonomous, bidding, credible, decentralized, ethical, and funded (ABCDEF) publishing" the authors provide a solution to current systems in scientific research that rely on slow, linear review processes for grants, publication, and ethical review by building on their previous work ("Trinity Reviews") to propose a system that incentivizes participation in the process through blockchain technology. While I believe there are a number of benefits to the proposed system, I also see a number of challenges to overcome that could be addressed in this manuscript. 

The first, and most substantial shortcoming is the fact that the proposed workflow (Fig 1), will, for most researchers, still require a linear review process, with all of the shortcomings that that entails. In my experience, few researchers submit studies for ethical review unless they are required to do so. The "Institutional ethics review (if necessary)" box is functionally required, and changes by that process are likely to affect the study design, which would require an additional line in the author's figure from that box back up to editorial review after triage. Overcoming that inefficiency should be a more central purpose of this publishing model, and doing that would require working with the ethics review committees in individual universities. In the U.S., they are beholden to government guidelines, and so there could be substantial challenges to integrating them into this process. 

Another limitation is that the funding review is still, in all cases, separate from the publishing and ethics review. As before, concerns that arise from the funding perspective cannot be addressed without editorial agreement if the "in-principle acceptance" is to be honored (in other words, any changes have to be approved by the editor, who may request additional reviewer insight). 

If all three sections: editorial, ethics, and funding can be combined, then I would see substantial benefit to this system. Even if the funding section remains seperate (because in this model it is more distributed and focused less on review but rather on up and down votes of being worthy of funding), I still think there is need to integrate institutional ethics review with the scientific review. 

My final set of comments are minor points that I think can be easily added to this manuscript:

What type of funder is this model designed for? Are you proposing that a private research foundation support it, a government agency? It will be important to understand that funder perspective and motivation for participating in order for this model to work. Their considerations (such as funder intent for private foundations, or legal requirements and taxpayer oversight for government agencies) must be taken into account so that there is a chance for them to participate. 

In item 7, you note that this is compatible with reviewer systems. Please describe how you would overcome compatibility issues with MULTIPLE systems. Currently, ethics reviews, funder reviews, and article reviews all use different systems, and that is a major barrier to a combined workflow such as you describe. 

In item 3, you note a concern about fraudulent accounts. Would it be possible to use simpler verification systems, such as possessing an ORCID and a human reviewer verifying your identity and credentials?

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

No

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No source data required

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Open science, meta-science, policy, ecology, citizen science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Takashima Kaito Kyushu University, Japan

Competing interests: No competing interests were disclosed.

 2 10 2023

2-1. In "Autonomous, bidding, credible, decentralized, ethical, and funded (ABCDEF) publishing" the authors provide a solution to current systems in scientific research that rely on slow, linear review processes for grants, publication, and ethical review by building on their previous work ("Trinity Reviews") to propose a system that incentivizes participation in the process through blockchain technology. While I believe there are a number of benefits to the proposed system, I also see a number of challenges to overcome that could be addressed in this manuscript.

Reply: Thank you for your compliments on our proposal. Below, we provide responses to each point that you point out.

2-2. The first, and most substantial shortcoming is the fact that the proposed workflow (Fig 1), will, for most researchers, still require a linear review process, with all of the shortcomings that that entails. In my experience, few researchers submit studies for ethical review unless they are required to do so. The "Institutional ethics review (if necessary)" box is functionally required, and changes by that process are likely to affect the study design, which would require an additional line in the author's figure from that box back up to editorial review after triage. Overcoming that inefficiency should be a more central purpose of this publishing model, and doing that would require working with the ethics review committees in individual universities. In the U.S., they are beholden to government guidelines, and so there could be substantial challenges to integrating them into this process.

Reply: As you see in the reply to Reviewer 1, we proposed the system as a community for researchers for whom institutional ethics review is not possible in the first place, as we indicate in our previous (Mori et al., 2022) and present papers. As you pointed out, there would be situations where our system needs the cooperation of existing IRBs, and we need to be recognized by the committees. On the other hand, as the previous researchers indicated in the following paper, it is a stark fact that many researchers are unable to conduct research because they do not have their own ethics review board, or because the cost of asking an independent review board is impractical. Your point is important that we have added it as an issue in "Potential weaknesses, threats, and solutions". Simultaneously, we have added the point that proceeding on the premise of the current institutional ethics reviews are themselves an obstacle to broadening the scope of science in the Introduction section:

https://psycnet.apa.org/doiLanding?doi=10.1037%2Fpst0000166

https://explore.bps.org.uk/content/report-guideline/bpsrep.2021.inf180/chapter/bpsrep.2021.inf180.12

***Introduction

"Furthermore, proceeding on the assumption of the current institutional ethical reviews impede broadening the scope of science, considering that many researchers suffer from the lack of opportunities for institutional review."

***Potential weaknesses, threats, and solutions- item9

"Of course, our proposed system may require the cooperation of existing IRBs in various situations. Additionally, this system must gain recognition from such institutions IRB and other relevant entities. The arrangement that enables cooperation with the respective institutional ethics review committees and publicity of our proposed system should be considered as issues to be addressed in the future." 

2-3. Another limitation is that the funding review is still, in all cases, separate from the publishing and ethics review. As before, concerns that arise from the funding perspective cannot be addressed without editorial agreement if the "in-principle acceptance" is to be honored (in other words, any changes have to be approved by the editor, who may request additional reviewer insight).

Reply: Basically, it is not envisioned that the funding phase will be reverted back to another previous phase after the IPA. If the (funding) voters feel that there are any problems with the research protocol/content at voting, they can simply not vote the project. If the researcher wants to modify the protocol and still receive funding from the system as a result of rejection, they can withdraw and resubmit the manuscript to our system. This part is not considered a limitation, but rather, the review of research funding is an evaluation and voting process, and asking for specific comments on the content and modifications would be very complicated; it may discourage anyone from joining the funding voting.

2-4. If all three sections: editorial, ethics, and funding can be combined, then I would see substantial benefit to this system. Even if the funding section remains separate (because in this model it is more distributed and focused less on review but rather on up and down votes of being worthy of funding), I still think there is need to integrate institutional ethics review with the scientific review.

Reply: As we replied to your comment above, we do not require institutional review in our framework only, but at the same time we understand that some studies should actually be required.

2-5. My final set of comments are minor points that I think can be easily added to this manuscript:

What type of funder is this model designed for? Are you proposing that a private research foundation support it, a government agency? It will be important to understand that funder perspective and motivation for participating in order for this model to work. Their considerations (such as funder intent for private foundations, or legal requirements and taxpayer oversight for government agencies) must be taken into account so that there is a chance for them to participate.

Reply: Thank you for your important remark As for funders, we are assuming private research foundations, government agencies, or investors, as you have indicated. The format of the fund could be in the form of other DAOs, crowdfunding, or, as reviewer 1 gave us previous examples, the system launching a grant project with an open call for proposals. However, this model is not limited to these specific funders or formats, but aims to support research projects with a wide range of funders and diverse funding sources.

2-6. In item 7, you note that this is compatible with reviewer systems. Please describe how you would overcome compatibility issues with MULTIPLE systems. Currently, ethics reviews, funder reviews, and article reviews all use different systems, and that is a major barrier to a combined workflow such as you describe.

Reply: We apologize for the misleading wording here. We don’t want to say that limitation 7 here is compatible with the traditional peer review system, but rather that we are advocating the creation of a reliable reviewer pool and making it as functional as the existing peer review system. As you pointed out, it is difficult to prepare reviewer pools for each of ethics and grant scientific review, but by working with the existing community and using a unified format for evaluation, we could bring the three types of reviews together. The sentences has been revised to be clear as follows:

***Potential weaknesses, threats, and solutions- item9

"Compatibility of reviewer reputational systems is also a matter of concern. Our proposed system envisions integrating a reputation system for reviewers using the blockchain. However, the reliability of the reputation system is undeveloped and uncertain. Instead of integrating this system, we should create a trusted pool of reviewers and make it as functional as the traditional peer review system"

2-7. In item 3, you note a concern about fraudulent accounts. Would it be possible to use simpler verification systems, such as possessing an ORCID and a human reviewer verifying your identity and credentials?

Reply: We think it is possible. More to the point, it is possible to automatically create a non-exchangeable ID in the form of a combination of ORCID and soulbound tokens, not by human hands. However, the problem of fraudulent accounts has already been pointed out even with ORCID (Teixeira da Silva, 2021. . https://doi.org/10.1016/j.jemep.2021.100692), and tying them together may not solve the problem. Rather, we suggest that it would be more realistic to establish a more reliable ID by linking researcher information and research history with a decentralized token such as the one proposed in our manuscript. However, it might lower the overall risk if it can be tied to not only ORCID but also other accounts e.g., google accounts, github accounts, and so on. We add the combined method to prevent fraudulent accounts as follows:

***Potential weaknesses, threats, and solutions- item9

"Moreover, it might lower the overall risk if it can be tied to other accounts (e.g., ORCID, google accounts, github accounts). "

References

Teixeira da Silva, J. A. (2021). Non-compliance with ethical rules caused by misuse of ORCID accounts: Implications for medical publications in the COVID-19 era. Ethics, Medicine, and Public Health, 18, 100692.

 10.5256/f1000research.142922.r193287 Reviewer response for version 1 Errington Timothy M. 1 Referee https://orcid.org/0000-0002-4959-5143 Center for Open Science, Charlottesville, Virginia, USA

Competing interests: No competing interests were disclosed.

 31 8 2023 Copyright: © 2023 Errington TM 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

This manuscript describes a vision of an alternative publishing process that would combine various emerging features being deployed to decrease bias and increase efficiency using a decentralized approach. Below is a list of recommendations for the author to consider.

In the introduction, why do the authors say “Accordingly, much research effort is wasted.”. This point is on ethical reviews. Are the authors suggesting there is waste because of having to review modified ethics protocols due to unforeseen changes? That doesn’t seem like waste per se, it’s important to revisit ethical protocols when deviations occur. And the unforeseen aspect doesn’t seem like waste because it was unknown during the initial protocol. I think the authors are trying to say there is research waste because those unforeseen changes by the authors might have been caught if methodological expertise was done alongside the ethical review in the first place. I’d suggest rewriting this to make that clearer. As a note, I also am unsure if I see that as waste – more inefficiency in a system.

In this model it assumes research progresses in single study/experimental settings, but most research papers currently are a collection of multiple studies/experiments. How will that work in this system? For example, Registered Reports can be submitted with pilot data – or the Registered Report study might be a replication of previous research. How does that work in this system?

For Phase 1, section B – this does not seem to align with how most research is done. The ethical reviews being independent of the institutional ethical review committees seems like it is adding an additional layer of review. In most journals that implement Registered Reports it is an expectation that the ethical review has already occurred, so approval is in place if the study is given IPA. Some research even requires initial ethical review at the grant review stage. I agree that having ethical and scientific review running in parallel is a good idea since these two areas are different and can influence each other, but if I’m understanding the diagram correctly, it seems like a lot of review might go into the process before it reaches the institutional review committee, which would only add additional revisions of the scientific peer review if ethical considerations need to change. Why not have the institutional ethics review running concurrently with the scientific review opposed to this institutional independent ethics review?

Related to this, in the ‘Advantages and opportunities’ section that this approach would help researchers who do not have institutional affiliations or institutional ethics committees. There already exists independent ethic review boards (e.g., BRANY) to help with this situation, which are similar to the institutional ethics reviews, how would this be integrated?

For Phase 2, funding – have the authors considered different workflows for having the funding and protocol peer review done? There are pilots experimenting with having a Registered Report reviewed by funders and journals. Here is a feasibility study of different models ( https://doi.org/10.12688/wellcomeopenres.17028.1) and here is a journal consortium model being implemented ( https://www.cos.io/consciousness).

Finally, I think the article format is not appropriate for this article. This reads more like an opinion article ( https://f1000research.com/for-authors/article-guidelines/opinion-articles) than a methods article ( https://f1000research.com/for-authors/article-guidelines/method-articles). For example, the last sentence of the paper makes quite a strong statement (i.e., ‘it will be realized’) without any evidence – but the authors are of course open to having that opinion of what they believe could happen (e.g., ‘we believe it could be realized’).

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No source data required

Are sufficient details provided to allow replication of the method development and its use by others?

No

Reviewer Expertise:

NA

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : Evaluating Registered Reports Funding Partnerships: a feasibility study. Wellcome Open Research .2021;6: 10.12688/wellcomeopenres.17028.1 10.12688/wellcomeopenres.17028.1 Takashima Kaito Kyushu University, Japan

Competing interests: No competing interests were disclosed.

 6 10 2023

1-1. This manuscript describes a vision of an alternative publishing process that would combine various emerging features being deployed to decrease bias and increase efficiency using a decentralized approach. Below is a list of recommendations for the author to consider.

Reply: Thank you for the important insights. Below, we provide responses to each recommendation or question.

1-2. In the introduction, why do the authors say “Accordingly, much research effort is wasted.”. This point is on ethical reviews. Are the authors suggesting there is waste because of having to review modified ethics protocols due to unforeseen changes? That doesn’t seem like waste per se, it’s important to revisit ethical protocols when deviations occur. And the unforeseen aspect doesn’t seem like waste because it was unknown during the initial protocol. I think the authors are trying to say there is research waste because those unforeseen changes by the authors might have been caught if methodological expertise was done alongside the ethical review in the first place. I’d suggest rewriting this to make that clearer. As a note, I also am unsure if I see that as waste – more inefficiency in a system.

Reply: As you pointed out, the explanation was misleading. We revised it as follows:

***Introduction

"Moreover, approved proposals must often be revised due to unforeseen changes in experimental methodology, necessitating reassessment by post hoc ethics or peer review.5 This rework could be prevented if methodological expertise review is done alongside the ethical review. "

1-3. In this model it assumes research progresses in single study/experimental settings, but most research papers currently are a collection of multiple studies/experiments. How will that work in this system? For example, Registered Reports can be submitted with pilot data – or the Registered Report study might be a replication of previous research. How does that work in this system?

Reply: Thank you for pointing this out. We apologize for the lack of explanation. We consider this system to be based on a Registered Report (RR) as we described in the manuscript. Therefore, the system naturally accepts such cases as you have suggested. We clearly explain in the manuscript that we accept such cases in the Advantages and opportunities as follows:

Advantages and opportunities

***"In addition, as seen in previous RRs, the ABCDEF system will accept submissions along with pilot data, replications of previous studies, and collections of various experiments and studies, allowing each content to be tied to the blockchain for reference."

1-4. For Phase 1, section B – this does not seem to align with how most research is done. The ethical reviews being independent of the institutional ethical review committees seems like it is adding an additional layer of review. In most journals that implement Registered Reports it is an expectation that the ethical review has already occurred, so approval is in place if the study is given IPA. Some research even requires initial ethical review at the grant review stage. I agree that having ethical and scientific review running in parallel is a good idea since these two areas are different and can influence each other, but if I’m understanding the diagram correctly, it seems like a lot of review might go into the process before it reaches the institutional review committee, which would only add additional revisions of the scientific peer review if ethical considerations need to change. Why not have the institutional ethics review running concurrently with the scientific review opposed to this institutional independent ethics review?

Reply: We apologize the position of "Ethical Review at the Institution" in the figure is confusing. In our proposed framework, undergoing ethical review at the institution was considered a special case. This primarily occurs when an ethics review is conducted simultaneously with the initial scientific review, and it is recommended that reviewers cannot make a clear judgment, or it is recommended to undergo an institutional ethics review as well. Therefore, in line with the original concept, it is not in its current position but should be included within the Phase 1 Combined Review and the revision loop. We have modified Figure 1 to align with the original intention.

1-5. Related to this, in the ‘Advantages and opportunities’ section that this approach would help researchers who do not have institutional affiliations or institutional ethics committees. There already exists independent ethic review boards (e.g., BRANY) to help with this situation, which are similar to the institutional ethics reviews, how would this be integrated?

Reply: Indeed, there is already an independent ethics review committee for those who do not have an ethics review committee within their affiliation. However, as pointed out in the background of our manuscript, the researchers must submit documents that are almost identical in content to those submitted for academic peer review in Stage 1 for even such institutions in their own format. When considering this structural and inefficient problem, our proposal system could exclude the barrier to undergoing an ethics review. Also, we’d appreciate it if you check our reply to Reviewer 2. We add the related sentence in the section as follows:

***Advantages and opportunities

"Although there is already an independent ethics review committee for such researchers, they must submit documents that are almost identical in content to those submitted for academic peer review in Stage 1 in their format. Our proposal system could exclude the barrier by integrating all stages."

1-6. For Phase 2, funding – have the authors considered different workflows for having the funding and protocol peer review done? There are pilots experimenting with having a Registered Report reviewed by funders and journals. Here is a feasibility study of different models ( https://doi.org/10.12688/wellcomeopenres.17028.1) and here is a journal consortium model being implemented ( https://www.cos.io/consciousness).

Reply: Thank you for sharing this very innovative and important precedent! The direction that journals and funders, such as the ones you mentioned, are taking in setting up grant projects and having manuscripts submitted to them is a very compatible. For your question, we have not considered the sperate flows between funding and RR peer review because it might waste the time of researchers. This is one of our motivation to propose the system. However, the precedential funding system which you introduced can be applied in our case with necessary modification. We added the possibility of combination in the Phase 5 explanation as follows:

“Another approach to grants involves journals of funding organizations initiating grant projects and allowing the submission of manuscript to them  (Clark et al. 2021; Center for Open Science)” 

1-7. Finally, I think the article format is not appropriate for this article. This reads more like an opinion article ( https://f1000research.com/for-authors/article-guidelines/opinion-articles) than a methods article ( https://f1000research.com/for-authors/article-guidelines/method-articles). For example, the last sentence of the paper makes quite a strong statement (i.e., ‘it will be realized’) without any evidence – but the authors are of course open to having that opinion of what they believe could happen (e.g., ‘we believe it could be realized’).

Reply: We initially submitted the article as an Opinion article, but changed it to Methods article after consulting with the Editor on the submission. While we agree with your opinion, we would like to leave the final decision to the Editor.

References

Clark, R., Drax, K., Chambers, C. D., Munafò, M., & Thompson, J. (2021). Evaluating Registered Reports Funding Partnerships: a feasibility study. Wellcome Open Research, 6, 231.  https://doi.org/10.12688/wellcomeopenres.17028.1