Open peer review at four STEM journals: an observational overview

Open peer review, peer review where authors' and reviewers' identities are disclosed to one another, is a growing trend in scholarly publishing. Through observation of four journals in STEM disciplines, PLOS One, Atmospheric Chemistry & Physics, PeerJ, and F1000Research, an observational overview is conducted. The overview relies on defined characteristics of open peer review. Results show that despite differing open peer review implementations, each journal retains editorial involvement in scholarly publishing. Further, the analysis shows that only one of these implementations is fully transparent in its peer review and decision making process. Finally, the overview contends that journals should clearly outline peer review and editorial processes in order to allow for open peer review to be better understood and adopted by authors, reviewers, editors, and readers of science communications.


Introduction
In scholarly publishing open peer review (OPR) is an emerging form of peer review that incorporates disclosure of author and referee identities to one another. Although in its infancy, OPR has been adopted and implemented in a number of disciplines and their respective scholarly publications. In this article I provide some background on OPR, addressing controversies and divergent opinions. Next I describe, examine, and discuss OPR implementations at four different science, technology, engineering, and mathematics (STEM) journals: PLOS ONE, Atmospheric Chemistry & Physics, PeerJ, and F1000Research. These observations contribute to our understanding of scholarly publication and scientific communication, as we watch the evolution of scientific vetting and validity determination processes.

Open Peer Review: a definition
Unlike double-blind peer review, which is clearly defined, has clear parameters, and an arguably universal understanding by the scholarly community in how to implement it, OPR is approached and implemented in a variety of ways. There is no one universally accepted definition of OPR, which complicates investigations of its practices. As such, I rely on my previous definition, which broadly understands OPR as any scholarly review mechanism providing disclosure of author and referee identities to one another at any point during the peer review or publication process (Ford, 2013). This definition is used as a starting point via which to observe OPR processes, and further analyze differing OPR implementations.
Other terms used to discuss OPR are peer-to-peer review and open review. Both these phrases insinuate OPR, but some have approached it as supplementary to formal peer review processes. For example, these review implementations rely on a community's members to post comments on articles at pre-print servers, such as arXiv, or using comment features via journal websites, such as British Medical Journal and BioMed Central. It should be noted that when I mention OPR, I discuss it as the formal process via which scholarly articles are vetted for publication.

Mentions of OPR in scholarly literature date back to Michael
McGiffert's 1988 article, "Is Justice Blind? An Inquiry into Peer Review", in which McGiffert argues, based on survey results, that editors should protect the identity of authors, but that editors, "...should leave referees free to decide for themselves whether or not to make themselves known [to the author]" (p. 47, McGiffert, 1988). Over time attitudes toward OPR have evolved and support of OPR has grown, although it still remains debated. Although OPR is a phenomenon occurring across the academic disciplines, those in STEM are the most prolific. Perhaps the oldest implementation of OPR occurred at Atmospheric Chemistry and Physics with its launch in 2001, which is discussed later in this article.

Why Open Peer Review?
For many OPR addresses inherent issues in what has been the gold standard of double-blind peer review. Some see blind review processes as faulty in that referee anonymity allows for referee abuse. Others view OPR as a means to hold referees and authors accountable for their communications (Cope & Kalantzis, 2009;Fitzpatrick, 2010;Mulligan, 2008). It has also been argued that OPR allows for easier identification of scientific misconduct (Boldt, 2011), and that over time the quality of submitted articles will improve (Hu et al., 2010;Prug, 2010). OPR affords referees the ability to gain credit for and cite their contributions to science communication (Boldt, 2011;Bornmann & Daniel, 2010;Fitzpatrick, 2010;Prug, 2010;Pöschl, 2009). More broadly speaking, some OPR implementations provide the scholarly community an insight into author/referee conversations during the review process. Surfacing these conversations provides readers an expanded contextual discussion of the subject at hand, and enriches science communication for all stakeholders (Fitzpatrick, 2010;Friedman et al., 2010;Lipworth & Kerridge, 2011;Maharg & Duncan, 2007). Finally, perhaps the most convincing pro argument for OPR asserts that OPR processes allow for quicker publication and dissemination of scientific findings (Cope & Kalantzis, 2009;Hu et al., 2010;Pöschl, 2004).
One of the major arguments against OPR is the perceived protection afforded both authors and reviewers in a blind process. For junior researchers serving as reviewers, blind review may allow them to feel more able to provide honest constructive criticism to senior researchers. Similarly, as authors, blind review is perceived as protecting junior researchers from public humiliation (Godlee, 2003). It has also been noted that some reviewers refuse to participate in OPR implementations, or still have concerns about them (Janowicz & Hitzler, 2012). These concerns still pervade conversations about OPR. Most recently, a survey of BMC Pharmacology and Toxicology Editorial Board Members surfaced continuing concerns regarding OPR at the journal. Despite these concerns BMC Pharmacology and Toxicology decided to "...continue with open peer review at BMC Pharmacology and Toxicology because of the ethical grounds for doing so and because the potential benefits outweigh the negatives", (p. 4, Moylan et al., 2014). This is evidence that despite continuing concerns and resistance to it, OPR will continue to be implemented and evolve in STEM publishing. As such, scholars should understand OPR implementations in order to further innovate and evolve scholarly publishing and scientific communication.

Methods
Four STEM journals claiming to use OPR processes were chosen for observation. These four journals were selected because they represent a difference in relative age, their perceived stature or authority in STEM, and for the salience of information regarding OPR on their respective websites. To review and understand the four different peer review implementations, this observation relied on the eight OPR characteristics I identified in 2013. It should be noted that I relied on these characteristics because there is no other documented common vocabulary used to discuss and analyze open peer review. I do not argue, however, that each characteristic carries the same weight and influence on review. Rather, publishers of publications exhibiting these characteristics may weigh their resulting reviews different in publishing decisions. The characteristics are cited in full below: • Signed review refers to submitted reviews signed by the referee that are either published alongside articles at the time of publication or are signed when an author receives them.
• Disclosed review refers to a process in which referees and authors know each others' identities during the peer review process, enabling them to engage in discussion or discourse.
• Editor-mediated review is a characteristic found in most open peer review processes. Editor mediation is any work done by a journal editor to facilitate open peer review. This may include editorial preselection of articles and/or final decision-making for acceptance or rejection of articles. The editor-mediated portion of any open peer review process may or may not be publicly disclosed.
• Transparent review refers to complete openness to a distinct community or the public. It allows a public community to watch peer review unfold. Authors and the public know referees' identities, and referees know authors' identities. Author responses to referee comments are public. In transparent review the public can see manuscripts, reviews, and replies from authors and public reviewers as well as the published articles.
• Crowd-sourced review is a public review process in which any community member may contribute to the article review. In crowd-sourced review there is no limit to the number of comments or reviews an article may receive. In some proposed implementations of crowdsourced review, there is little editorial mediation of article reviews. Rather, authors may simply submit papers to a preprint server or other community for crowd-sourced commentary.
• Pre-publication review occurs prior to article publication, and typically occurs in a public space such as a pre-print server.
• Synchronous review occurs at the same time as publication of the article. In the literature, synchronous review is approached only theoretically, as part of a novel and completely iterative publishing model.
• Post-publication review occurs after an article is published, much like commentary on a blog post (pp. 314-315, Ford, 2013).
Using these characteristics I examined information for authors and about each publication at their respective websites, promotional materials, blogs, and other materials discussing the OPR processes at each journal. Data for these observations was gathered in mid to late 2013. Publisher/journal policies and practices may have since changed.

PLOS ONE
PLOS ONE is an international publication of Public Library of Science, a not-for-profit publisher and open access advocacy organization. The journal was formed around the philosophy and practice that all research using scientifically sound research methods should be published regardless of its results, novelty, and/or impact. The journal publishes research articles from science and medical disciplines, including those reporting negative results. By publishing research from multiple disciplines, the journal asserts "PLOS ONE facilitates the discovery of the connections between papers whether within or between disciplines".
PLOS ONE launched in December 2006 and has since seen tremendous growth. It is indexed in numerous databases; is frequently cited as a source of research in news and popular media; and has received positive press for its review process. Even John Bohannon, a science journalist who undertook a sting operation of open access journals in an attempt to uncover poor publishing practices, acknowledged the strength of PLOS ONE's review process (Bohannon, 2013, ¶ 9).
All articles published in PLOS ONE carry Creative Commons attribution licenses. Although most authors publishing in the journal pay article processing charges (APCs), the journal makes exceptions and waives publishing fees for unfunded research. Moreover, the APC fee model at PLOS ONE takes into account an author's country of origin, and whether it is a high, lower middle, or low income nation. In this way the publication aims to make more viable open access publication for authors with disparate economic means.
Compared to the other journals I discuss in this article, PLOS ONE conservatively approaches OPR. The journal's peer review process only exhibits a few OPR characteristics, and even then these characteristics are not consistently implemented. It does, however, always use a form of editor-mediation for reviewing and publishing its content. Each submitted article is assigned an Academic Editor, who then determines whether submissions should be considered for peer review, and who facilitates the peer review process. According to the journal's review guidelines, these Academic Editors may, "...conduct the peer review themselves, based on their own knowledge and experience; they can take further advice through discussion with other members of the editorial board; they can solicit reports from further referees".
While the editorial mediation of journal articles at PLOS ONE always occurs, other characteristics of OPR do not. Signed reviews are optional, but they are strongly encouraged. According to its peer review guidelines, "If Peer Reviewers are willing, then they are also identified to the author at the time of decision." PLOS ONE does not post reviewer comments to the web alongside published articles, so readers do not benefit from reading discussions that occurred about the topic prior to publication. In addition to PLOS ONE's version of signed review, the journal enables a public commentary function for published articles. Although this technical functionality could be considered crowd-sourced reviewing, the journal does not consider post-publication public comments and discussions as part of a formal peer review process. One aspect of this crowd-sourced discussion process is that PLOS ONE surfaces any media coverage of published articles by linking to them in an article's comments section. The result is that the journal is able to create a record of the impact and conversation an article elicits outside of the PLOS ONE platform and community. Following an article's submission to ACP and brief editorial review, it is then hosted on the journal's pre-print server, Atmospheric Chemistry and Physics Discussion Papers (ACPD) for peer review and crowd-sourced discussion. On this platform reviewer comments and crowd-sourced comments are publicly available. After the discussion period for a paper ends, the ability to comment on the paper is turned off, and the journal editor makes a final publishing decision using submitted referee and public reviews. When the editor accepts an article for publication, the article is published at ACP, where it will also link to its pre-print version including referee and public comments at ACPD. ACP does not host public commentary on published articles. Although ACP views its peer review process as completely transparent (Pöschl, 2004), it is not. Reviewers may choose to disclose their identities, or they may choose to remain anonymous. True transparency of the review process can only occur when reviewers and public commenters have disclosed their identities.

Atmospheric Chemistry and Physics
In addition to editor mediation and crowd-sourced reviewing, ACP's process employs disclosed review and pre-publication review. It could be argued that synchronous review occurs, yet ACP does not consider papers posted at ACPD to be "published". Because these papers are not considered published, their review is not synchronous.

PeerJ
At the time data was collected for this article, PeerJ was an individual membership-based publisher in the biological and medical sciences. However, during the review process for this article, a reviewer surfaced that PeerJ has since changed its model and now views it as a publication plan, rather than an individual membership (Binfield, 2015 The publishing model at PeerJ is similar to the pre-print/publication relationship between ACP and ACPD in that it maintains two publishing platforms: PeerJ and PeerJ PrePrints. Arguably, PeerJ PrePrints is not a publication, but a pre-print repository service. However, unlike ACP, PeerJ considers PeerJ PrePrints a publication, so I will treat it as such alongside PeerJ. Only paid PeerJ members may publish work in PeerJ. Works may first be submitted to PeerJ PrePrints, or may be directly submitted to PeerJ. PeerJ's article review, acceptance and publication model mirrors PLOS ONE's; PeerJ accepts scientifically sound research and does not consider an article's "novelty, interest, or impact" as part of its editorial criteria. Prior to review, submissions to PeerJ undergo editorial vetting by assigned Academic Editors. These Academic Editors are responsible for facilitating peer review of assigned articles to be completed by at least two reviewers, and making final publication decisions. Additionally, Academic Editors are attributed alongside each published article. Authors and reviewers alike are "encouraged" to post the full peer review cycle online alongside final versions of articles, but the journal does not mandate it. Based on information from PeerJ's website, it is unclear whether author and reviewer identities are disclosed to one another during the review process. However, during review of this article, a reviewer clarified that PeerJ authors receive referee identity disclosure (if disclosed) at the time they receive their referee reports and publication decision from the Academic Editor (Binfield, 2015).
Unlike the crowd-sourced review occurring at ACP, PeerJ is similar to PLOS ONE in that it views discussions and commentary on articles as separate from the formal peer review process. Unique to PeerJ, too, is its utilization of a broader discussion model named Q&A. PeerJ's Q&A incorporates not only community questions and answers regarding pre-prints and articles-which can be posed at the paragraph and figure level-it also allows for free-standing questions and discussions of PeerJ community members. In this way, Q&A is intended to be a platform for anyone in the community to participate in scientific conversation. At PeerJ Q&A includes an incentive system for individual community participants. Contributing individuals are awarded points for their engagement. For example, one earns 100 points for authoring an article, 35 points for contributing an open review, etc. These points are displayed on members' profile pages.
PeerJ has future plans to expand PeerJ PrePrints. The publisher hopes to allow authors to share as much or as little of their publications as they wish. They may decide to openly publish a title, title and abstract, or the whole paper. Additionally, PeerJ PrePrints will allow for authors to share papers "privately" with only particular users, only the PeerJ community, or fully open on the web.

F1000Research
Finally, the fourth of the journals I discuss, At F1000Research articles undergo a peer review process after they are published. As such, F1000Research is the only publication I discuss that uses a post-publication review process. In this way the journal is able to speed up publication timelines to disseminate scholarly work; the journal publishes articles within one week of submission. As stated in its referee guidelines, the journal publishes submitted articles that pass initial editorial review for "content, quality, tone and format" as well as completeness, plagiarism, ethical standards, and adherence to author guidelines. In addition to reviews provided by two or three designated expert refereeswhich are attributed to the reviewer and are published online with the work-the scientific public (those affiliated with scientific or medical organizations) may comment on any published article. Any author responses to referees are also public.
In this publication model it is possible for articles to receive unanimous negative reviews. In this case, articles remain "published", but are removed from the site's default search. The site's interface clearly delineates a work's referee status and comments using icons to indicate: approved with reservations, approved, or not approved. It is important to note that F1000Research does not consider these statuses as equivalent to the accepted, accepted with revisions, and rejected statuses that one sees in closed review processes. Rather, as stated in the FAQs: • The term Approved means that the referee thinks that the article is good and has either no suggested revisions or only minor revisions. The term Approved with Reservations means that the referee agrees that the article has scientific merit and is fundamentally sound but would like the author to make further changes to the manuscript. This is approximately equivalent to a request for major revisions or several minor revisions in a traditional journal. In every case, even when all referees approve of the article, future versions are welcome.
Because F1000Research publishes and attributes all referee responses and author comments, it adheres to a fully transparent peer review process. In addition to transparent reviews provided by pre-selected referees, crowd-sourced review occurs when individuals comment on published articles. The journal exhibits most other OPR characteristics; its reviews are editor-mediated, transparent, referee and author identities are disclosed, and reviews are signed. The only OPR characteristics not exhibited by F1000Research's process are those related to review timing. Since all review at F1000Research occurs post-publication, the journal does not exhibit pre-publication and synchronous review characteristics. Once a work has been vetted by editors and is made public on the journal's site, the journal considers it published. Because review occurs post publication, authors receiving critical feedback are encouraged to revise and submit updated versions of articles that will, again, be refereed. The journal uses CrossRef's Cross-Mark identification service to assist readers in tracking these article versions and relationships. Even if an author publishes an updated article, previous versions remain published. In this way, publication at F1000Research is a good example of the iterative process of publishing and scientific knowledge and conversation.

Discussion
I examined four journals using relatively OPR processes, PLOS ONE, ACP, PeerJ, and F1000Research. None of these journals have implemented OPR in the same manner, but they do exhibit many of the same OPR characteristics. Table 1 offers a comparison of the journals' OPR characteristics, and points to a relative degree of just how open are their OPR processes.
Each journal exhibits a form of editor-mediation and each journal vets submitted articles prior to publication for basic quality, scope and adherence to author guidelines. While for some journals, such as F1000Research, this is as far as an editor's work goes before an article is published, others, such as ACP and PeerJ, allow editors to make final publishing decisions.
Each of the journals allow for some form of crowd-sourced review. At PeerJ the Q&A section includes commentary on articles; it also includes other avenues for the public to engage in scientific conversations. At PLOS ONE, however, commentary on articles occurs only after an article has been published, and also includes links to all media coverage of articles. Unlike PLOS ONE, PeerJ, and F1000Research's crowd-sourcing implementations, ACP's process only allows for crowd-sourced commentary prior to publication articles during the discussion phase of the OPR process. Crowdsourced review, although a characteristic of OPR, may be weighed differently in a publication's review process (Perakakis, 2015).
Another commonality between these publications is their varying allowance, encouragement, or mandate for reviewers to sign their commentaries. Referees may choose to remain anonymous at ACP, PeerJ, and PLOS ONE (even though disclosure is strongly encouraged), whereas F1000Resarch PLOS ONE requires referees to disclose their identities. Since author/reviewer identity disclosure is a defining factor of OPR for the purposes of this overview, it could be argued that those articles where reviewer commentary is not attributed are not truly open peer-reviewed. The motivation for publishers to encourage rather than require openness most likely stems from their desire to encourage more authors and reviewers to participate in alternative peer review processes. Publishers may also be attempting to be mindful of different discipline's accepted publishing practices. In my own view, by not mandating public attributed review, publishers are weakening the power of OPR. However, incremental steps in OPR implementation are necessary to encourage participation and to move OPR to a completely transparent standard in the future.
Two of the journals discussed above include article pre-print mechanisms. Pre-print servers and mechanisms introduce confusion into understanding publishing and OPR. Just when is something that is open to be read on the web considered "published?" Where ACP does not consider papers posted to its pre-print space, ACPD, as "published", PeerJ PrePrints is considered by its publisher a publication. Publishers' definitions of when a scholarly work is "published" will continue to evolve as OPR processes evolve. It is unlikely that any definition will be uniformly held by all scholarly publishers.
Of the four publishers discussed, I maintain that F1000Research exhibits what we should consider the gold standard of transparent and OPR processes. The publication's process is completely transparent; it publishes all commentary with attribution and makes salient referee decisions. Moreover, the mechanism it uses to track and correlate article versions and updates enhances and opens scholarly conversations. Yet, F1000Research maintains its editorial voice via editor-mediation prior to an article's publication and by suppressing from search results articles receiving unanimous negative reviews.

Conclusion
Scholarly journals are beginning to challenge traditional peer review practices by implementing OPR, yet each OPR implementation differs. By observing four different implementations of OPR I conclude that few OPR journals implement truly transparent review, yet each implementation values editorial work. Further, I maintain that distinguishing between publicly available preprints and publicly available published articles unnecessarily muddies the waters in understanding OPR. As OPR implementations proliferate, it is pertinent for journals to clearly outline any peer review process so that readers, authors, and reviewers can fully understand peer Only if reviews are signed.
Only if reviews are signed and both referees and authors opt to share all comments generated during the review process.

Yes
Crowd-sourced No. Reader comments are not considered part of the peer review process.

Yes
No. Reader comments are not considered part of the peer review process.

Yes
Pre-publication Yes Yes Yes No

Post-publication
No. Reader comments contributed to published articles are not considered a part of the peer review process.

No
No, though reader comments and the Q&A community may continue discussions regarding articles.

Synchronous No No
Possibly, since PeerJ PrePrints is considered a publication.
No. Articles are considered published once they are posted to the site.
review implementations, decision making processes, and to provide for editorial transparency.
Future research is needed in a number of areas. First, we need to understand how scholarly communities and publishers define an article's publication status. The fact that content appears readily accessible via the web does not mean publishers consider articles "published." This inherent tension, based in a print publishing paradigm, will continue to introduce confusion to those searching for and reading scholarly research and writings on the web. Additionally, OPR occurring in non-open access journals should be investigated, as should OPR implementations in non-STEM journals.

Competing interests
No competing interests were disclosed. The author has no affiliation with any of the journals discussed in this article.

Grant information
The author(s) declared that no grants were involved in supporting this work.

Thank you for this interesting observation article and for the invitation to contribute my open peer review.
In the beginning, the article tries to disambiguate different "flavours" or implementations of open peer review (OPR) by identifying a number of OPR characteristics. Regarding this effort I would like to note the following issues that I think should be somehow addressed in a future version of the article: As already noted by Peter Binfield, the definition offered is limited to only one of the identified OPR characteristics, namely whether the reviews are signed or not.
It is not clear what is the difference between reviews "signed when the author receives them" and disclosed reviews. It seems that in both cases the names of authors and reviewers are disclosed from the beginning of the review process.
Is disclosure of the reviewer's identity necessary to initiate a discussion between authors and reviewers? Perhaps what you refer to here is an infrastructure to allow multiple iterations during the review process, as provided for example by Frontiers. This is different from simply revealing the name of the reviewer.
There is a clear distinction between an unsolicited comment on an article and a formal review process by expert peers that receive a review invitation either by an experienced editor or the authors themselves.
What is the alternative to editor mediation in peer review, if we consider that unsolicited commentaries do not constitute formal peer reviews? Is there another model where formal peer review is not editor-mediated? I know we proposed such model that we called author-guided peer review in Perakakis, P., Taylor, M., Mazza, M., Trachana, V. (2010). Natural selection of academic , but I am not aware of any implementations. I repeat papers. Scientometrics, 85 (2), pp. 553-559 that I do not think we should consider crowd-sourced comments as equal to formal peer reviews. And some other comments to consider: PLOS ONE clearly does not qualify as an OPR journal by any standards. Instead, I would be more interested to see a discussion on the implementation of OPR by Frontiers.
I suspect there is a misunderstanding around the terms double-blind and single-blind review. The standard in most journals and disciplines is single-blind review where authors are blind to the name of the reviewer, but reviewers know the name of the authors. Double-blind refers to the process where neither authors nor reviewers are aware about the identity of each other.
It is not entirely correct that PLOS ONE "waives publishing fees for unfunded research". What they do is ask for proofs that authors cannot cover the APCs even from their own budget, sometimes even asking for personal financial records, which by the way I find unacceptable. Proving that there is no formal funding to publish the particular research is not enough to waive APCs.
I definitely agree that a comparison table is needed.
I detected that the references to the articles by Lawrence are missing. Please double check that all references are included. Thank you again for the opportunity to participate in this process that is very close to my ideal model of OPR and I will be looking forward to a revised version of your manuscript. 1.

2.
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed. Neither the cited reference nor the ACP journal web pages claim that the peer review process of ACP would be "completely transparent". There are good reasons to maintain an option for referees to remain anonymous, and this is also very clearly specified on the journal web pages as well as in the referenced article and in a more comprehensive review article (Pöschl, 2012 as referenced and linked below). Thus, it would indeed be wrong to claim "complete transparency" for the open peer review process for ACP, but it is also wrong to insinuate that ACP or the cited article would have raised this claim. Please discard or correct the erroneous statement accordingly. Moreover, please complement or replace the reference to Pöschl 2004 by a reference to Pöschl 2012, because the latter is more comprehensive and up-to.date: Neither the ACP journal web pages nor the articles providing an authoritative description of the ACP open peer review (Pöschl, 2004(Pöschl, & 2012 have ever indicated that the papers posted in the ACP discussion forum, ACPD, would not be "published". Well on the contrary, all relevant journal web pages emphasize explicitly that the discussion papers posted and reviewed/discussed in ACPD are publications with permanent public accessibility, archiving and citability. Also my articles explaining the concepts of interactive open access publishing and multi-stage open peer review (Pöschl, 2004 and2014) specify and emphasize that the discussion papers published in ACPDas well as in the fifteen interactive open access sister journals of the European Geosciences Union (EGU) -are indeed publications. Thus, it is inappropriate to insinuate the opposite. Please discard or correct the erroneous statement accordingly. http://www.egu.eu/about/statements/position-statement-on-the-status-of-discussion-papers-published-in-egu-int Scott Walter John T. Richardson Library, DePaul University, Chicago, IL, USA The study applies the tools of content analysis to the stated editorial policies and procedures of a small number of journals in STEM disciplines adhering (to a greater or lesser degree) to the principles of open peer review. To promote replicability of the study and/or comparison between the set of journals included in the current study and other sets of journals, the study might be revised to include a table documenting the degree to which each journal under consideration did or did not meet the stated OPR principles. While the study is limited to review of relatively new publications and publication models in a defined field, it might also be revised to suggest future research opportunities, e.g., the degree to which OPR principles are employed in more established journals, the degree to which OPR experiments are being explored in other fields.
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
No competing interests were disclosed.

Competing Interests:
Author Response 25 Jan 2015 , Portland State University, USA Emily Ford Thank you for your comments and feedback. The suggestion of a table, I think, will help to clearly communication these observations, so a revision will include one. One of the things that I have struggled with is how to represent what might be happening in other disciplinary communities. From what I have found, OPR seems to have started in STEM disciplines. I'm finding it difficult to wrap my thoughts and ideas around OPR in the humanities since humanities publishing is also changing with new ways to express scholarship, aka the digital humanities. In this way it is easier (for me) to begin observing and evaluating OPR in STEM publications.
Again, thank you for your thoughts and comments, and I look forward to more comments after I re-submit this article.
No competing interests were disclosed.

Peter Binfield
PeerJ, San Francisco, CA, USA Thank you for an interesting article which overviews an emerging form of publication / peer review

General Comments:
At an early point, the author defines OPR as simply naming the authors and reviewers ("any scholarly review mechanism providing disclosure of author and referee identities to one another at any point during ), but then goes on (e.g. in the 2 half of para 1 of the peer review or publication process" "Open Peer nd

Version 1
Reader Comment 05 Mar 2015 , Cleveland State University, USA Jason Moore I'm not sure if this should be in the scope of this paper, but at some point in history many academic journals had open peer review. I often find the correspondence between author and reviewers appended to the bottom of articles that are pre-1950 or so. My guess is that there are a fair number of journals that practiced open peer review in the past that this article fails to mention and review. Understanding why open peer review was used and disappeared could provide more insight than observing journals, like and , that have only been trying this for two years.

PeerJ F1000Research
I'd have to dig around for more examples but this article is one that has a "Communications" section appended to the end: