An international survey and modified Delphi process revealed editors’ perceptions, training needs, and ratings of competency-related statements for the development of core competencies for scientific editors of biomedical journals

Background: Scientific editors (i.e., those who make decisions on the content and policies of a journal) have a central role in the editorial process at biomedical journals. However, very little is known about the training needs of these editors or what competencies are required to perform effectively in this role. Methods: We conducted a survey of perceptions and training needs among scientific editors from major editorial organizations around the world, followed by a modified Delphi process in which we invited the same scientific editors to rate the importance of competency-related statements obtained from a previous scoping review. Results: A total of 148 participants completed the survey of perceptions and training needs. At least 80% of participants agreed on six of the 38 skill and expertise-related statements presented to them as being important or very important to their role as scientific editors. At least 80% agreed on three of the 38 statements as necessary skills they perceived themselves as possessing (well or very well). The top five items on participants’ list of top training needs were training in statistics, research methods, publication ethics, recruiting and dealing with peer reviewers, and indexing of journals. The three rounds of the Delphi were completed by 83, 83, and 73 participants, respectively, which ultimately produced a list of 23 “highly rated” competency-related statements and another 86 “included” items. Conclusion: Both the survey and the modified Delphi process will be critical for understanding knowledge and training gaps among scientific editors when designing curriculum around core competencies in the future.

(well or very well). The top five items on participants' list of top training needs were training in statistics, research methods, publication ethics, recruiting and dealing with peer reviewers, and indexing of journals. The three rounds of the Delphi were completed by 83, 83, and 73 participants, respectively, which ultimately produced a list of 23 "highly rated" competency-related statements and another 86 "included" items.
Both the survey and the modified Delphi process will be critical for Conclusion: understanding knowledge and training gaps among scientific editors when designing curriculum around core competencies in the future.

Background
The Declaration of Helsinki (2013) asks editors to ensure that the quality of what they publish is of the highest quality possible 1 . However, very little is known about the training needs of scientific editors (i.e., those who make decisions on the content and policies of a journal) or what competencies are required to meet these standards. Presently, a large portion of scientific editors' learning is informal, often learned on the job through mentoring 2 . While formal training opportunities for scientific editors do exist in the form of fellowships and intensive courses, these opportunities are limited to a very small number of editors annually. In addition, there may be variations across training opportunities, which are not evidence-based, owing to the lack of any consensus or evidence on the knowledge, skills, and abilities that editors should possess to be competent in their job.
While there is no shortage of published literature on the role of scientific editors, most of this takes the form of opinion-based editorials; very few recommendations are evidence-based. For example, a 2015 systematic review of training in writing for scholarly publication, journal editing, and peer review found no studies of formalized training programs for journal editors 3 . While a 2016 scoping review from our group 4 found 25 researchbased publications relating to scientific editors, the majority were surveys on a wide variety of topics relating to scientific editors. The same study found 136 published articles, of which 133 were non-research based editorials relating to various aspects of scientific editorship. An associated environmental scan also found an additional 35 documents that were not published in scientific journals, of which 18 were produced by journals, while nine were from associations and societies, six from organizations providing guidance to editors, and two from publishers.
In 1999, the World Association of Medical Editors conducted a global survey of journal editors regarding the characteristics of their respective journals. This survey included one item related to scientific editor training: of the 269 respondents, 75% said they wanted training for newly appointed editors. Other surveys have echoed this opinion as well 2,5,6 ; however to our knowledge, there have been no dedicated attempts to formally assess the training needs of scientific editors of academic journals. We are also unaware of any large scale collaborative effort to determine the competencies that are required for the role of scientific editor.
There are a number of examples of editorial organizations, publishers, and individuals 7 who have put forth their opinions on what makes a good scientific editor, yet there lacks clarity on how these ideas were derived, how they relate to each other, or whether they are universal. Additionally, the perceptions and training needs of scientific editors are not well represented in the literature.
The objective of this research was to better understand the training needs and perceptions of competence of scientific editors of journals. We also sought to solicit editors' opinions on the importance of particular knowledge, skills, abilities, and characteristics to carrying out their editorial duties.

Methods
The research presented here is the second step of a larger program that our team is carrying out to develop a universal, minimum set of core competencies for scientific editors of biomedical journals. In our first project, we carried out a scoping review of the published research literature and an environmental scan of non-research-based materials to identify competency-related statements found online and in previous research 4 . In the current project described here, we aimed to solicit the perspective of editors worldwide and have them narrow and refine the number of potential core competencies related to their position. In the third and final step of the process, we brought international experts together to decide on a final set of core competencies, publication of which is forthcoming.
The current research comprised a survey of scientific editors to understand their perceptions of their role as an editor and to identify any training needs, followed by a modified Delphi process whereby editors rated the importance of 200+competencyrelated statements obtained from the aforementioned previous scoping review (describing knowledge, skills, and abilities associated with the role of scientific editor) 4 . Surveying to gauge training needs and to gather consensus are common tools for creating a competency-based core curriculum in the biomedical field [8][9][10][11][12] . The study was approved by the Ottawa Health Science Network Research Ethics Board.

Participants
We approached current or former scientific editors of journals, defined as editors who make decisions on the content and policies of a journal -including editors-in-chief and associate/ academic editors. Recruitment advertisements were sent to editorial organizations and groups having a large scientific editor membership from around the world. These organizations forwarded the advertisement about the survey of editors to their members through a distribution list email or an announcement on a listserv or message board. The organizations are listed in Box 1:

Phase 1: Survey of Editor Perceptions and Training Needs
We developed an online-based assessment of editors' perceptions and training needs that was anonymous and self-administered (Supplementary File S1). The questionnaire was developed based on data collected in our previous scoping review 4, as well as with input from our research team, comprising scientific editors, representatives from publishing houses, educational experts, and specialists in publication science. Questions were designed to broadly cover major areas associated with the scientific editor role, including editors' knowledge, expertise, skills, and experience. The questionnaire was not validated; however, it was piloted among five experienced scientific editors of biomedical journals and subsequently revised based on their feedback. The revised questionnaire was uploaded to SurveyMonkey (www.surveymonkey.com) and the survey URL was sent to the editorial organizations for distribution.
The survey contained 19 demographic questions relating to participants' age, sex, education level, job title, editorial experience, the journal they edit, and their editorial training experience. The perceptions of respondents were also examined in four areas: Participants indicated in the first two instances how important they thought a series of competency-related statements was to the scientific editor role, and, in the latter two instances, how much they thought they possessed these same competencies. Response options on a 1-7 Likert scale were provided (as indicated): 1. The degree to which participants perceive that expertiserelated items are important to their job as editor (18 items) (Response options: Not Important to Very Important) 2. The degree to which participants perceive that particular skills and experience are important to their job as editor (20 items) (Response options: Not Important to Very Important) 3. The degree to which participants perceive they possess particular expertise-related items related to their job as editor (same 18 items as #1) (Response options: Not Much to Very Much) 4. The degree to which participants perceive they perform particular skills and possess particular experience related to their job as editor (same 20 items as #2) (Not Well/Not Much to Very Well/Very Much) Finally, participants were asked to create a ranked list of their top 10 training needs (from #1 being the most important to #10 being least important). Participants were also asked if they would be willing to participate in a Delphi process to rate the importance of a much larger and more detailed list of competency-related statements for scientific editors of biomedical journals. If so, they were asked to provide their email addresses and were included in the list of potential participants for the modified Delphi process.

Phase 2: Modified Delphi Process
We carried out a three-round modified Delphi process in order to rate and refine the list of potential competencies derived in a previous scoping review we conducted 4 . A Delphi process typically involves experts and takes place in the form of iterations or "rounds" (normally 2 to 4) in which data is collected anonymously, often on-line, and then fed back to the group in an aggregated and de-identified way, along with individual participants' comments 13 .
In the current project, we modified the Delphi in several ways: First, we solicited the involvement of any scientific editors of biomedical journals, not only experts, as is normally the case with a Delphi process. Next, we did not require all participants to be involved in all rounds of the modified Delphi. In addition, the number of items included in the modified Delphi was much greater than a Delphi would generally include. Finally, for the sake of efficiency (due to the large number of items), in the third round we did not ask participants to re-rate items that had reached consensus for inclusion or exclusion in Round 2.
Interested Phase 1 participants were invited to participate in the online modified Delphi, which was also administered via Survey-Monkey. For each Delphi round, an invitation was sent to the entire list of potential participants, regardless of whether they responded in the previous round.

Round 1.
Participants were asked to rate each competency-related statement on a 5-point Likert scale, from 1 (Not at all Important) to 5 (Absolutely Essential). At the end of each section, an open text box was provided for participants to include comments relating to items in that section if desired. Participants were also asked to name any potential competencies that were not included in the Delphi. Participants were reminded that all competency-related statements exclusively related to the position of Editor-in-Chief had been intentionally removed from the Delphi, as was the case for the scoping review on which the Delphi was built.
Round 2. Phase 1 participants were invited to participate in Round 2 of the Delphi, regardless of whether they completed Round 1. Along with the email invitation, they were provided the mean score for each of the items from Round 1, the participant's own score for each of the items, if relevant, and collated (de-identified) comments from the text boxes. Participants were then asked to re-rate and provide their rationale for disagreement only for those items for which they disagreed with the mean score from Round 1. Any new competency-related statements arising from the final question in Round 1 were included for rating in the Round 2 survey. Participants were asked to rate these new items in the same way that items had been rated in Round 1, that is, on a 5-point Likert scale from 1 (Not at all Important) to 5 (Absolutely Essential).

Round 3.
Phase 1 participants were invited by email to participate in Round 3 of the Delphi, regardless of whether they completed Round 1 and/or Round 2. Attached to the e-mail was a document listing the Round 1 and Round 2 average scores for all items and participants' comments from previous rounds. Items that did not reach consensus for inclusion or exclusion in Round 2 were highlighted in yellow. Participants were asked to re-rate these highlighted items in Round 3 using a 3-point Likert scale from 1 (Less Important) to 3 (Essential). The shift to a 3-point Likert scale was aimed at simplifying the process for participants by limiting the number of options to the manner in which the ratings of each competency statement would be analysed (i.e., consensus on a '1' would mean the item was excluded and consensus on a 3 would mean the item was included).

Analysis
To establish inclusion and exclusion criteria for both the survey of editors and the modified Delphi, we pre-specified the consensus level at 80% of respondents. This decision was based on the use of an 80% cut-off rate in previous Delphi studies in healthcare 14 and education 15 .
The data on the 76 perception-related survey questions were summarized by calculating mean scores for each of the questions. These scores were then classified as reaching consensus for inclusion (≥6 on a 7-point Likert scale) or exclusion (≤3 on a 7-point Likert scale), for use in a future consensus meeting. The ranked top 10 lists of training needs were collated by one author (JG) by regrouping similar statements and these groupings were then verified by another author (KDC). In

Ethical considerations
All participants provided consent through an online form preceding the survey of editors and Delphi. In order to stimulate participation and thank participants, an iPad Mini was awarded as a draw prize after the survey of editors and after each round of the Delphi. Participants who wished to be entered into the draw were asked to provide their email address. These email addresses were used for the sole purpose of the draw and removed from the data prior to analysis. All data from this research was pooled for presentation in the results section.

Results
A total of 152 participants completed either the survey of editors' perceptions and assessment of their training needs or the Delphi process ( Figure 1). As described in Figure 1, four respondents did not complete the survey of editors but completed one round each of the Delphi and their responses were included in the analysis. We presume these four respondents were invited by other participants as access was not restricted for completing the Delphi.
Editors' Perceptions and Ranking of their Training Needs Demographic data. A total of 148 editors from around the world accessed and completed the survey of their perceptions (Table 1 & Supplementary File S2). Respondents were mainly those replying to the notices for participation posted by the World Association of Medical Editors (WAME) and Cochrane. Nearly 2/3 of respondents were male. Close to 2/3 had more than seven years of experience as an editor. The majority of editors indicated that their journal had provided formal or informal training related to their job and that they had sought both formal and informal training beyond what was provided by their employer. The large majority indicated having formal or informal training in research methods and statistics.

Editor Perceptions. The degree to which participants perceive that expertise-related items are important to their job as editor.
Of the 18 questions related to participants' perceptions of the importance of particular expertise-related items to their job as editor, two items reached consensus for inclusion ( Table 2). The highest consensus was for 'expertise in research methods' (85.4%), followed by 'expertise in dealing with publication ethics, including conflicts of interest of authors, reviewers, and editors and the editorial board' (80.4%). None of the items reached consensus for exclusion.
The degree to which participants perceive that particular skills and experience are important to their job as editor.
Of the 20 questions related to perceptions of the importance of particular skills and experience related to their job as editor, four items reached consensus for inclusion ( Table 2). The highest consensus was for 'behaving with integrity/professionalism' (94.4%), followed by 'using good judgment in decision-making' (93.7%), 'language/writing skills' (90.1%), and 'author and peer reviewer correspondence; how to evaluate peer reviews, draft a revision letter and evaluate an author response letter and revision' (86.6%). No items reached consensus for exclusion.
The degree to which participants perceive they possess particular expertise-related items related to their job as editor.
Of the 18 questions related to participants' perceptions of how much they possess particular expertise related to their job as editor, no items reached consensus for inclusion ( Table 2). The highest rated item was 'expertise in research methods' (72.1%), followed by 'expertise in the publication process (decision-making aspects) for research papers, commentary, and correspondence' (70.6%), 'expertise in the subject areas in which your journal publishes' The degree to which participants perceive they perform particular skills and possess particular experience related to their job as editor. Of the 20 questions related to participants' perceptions of how much they thought they performed particular skills and possessed particular experience related to their job as editor, three items reached consensus for inclusion ( Table 2). The highest consensus was for 'behaving with integrity/professionalism' (90.2%), followed by 'using good judgment in decision-making' (87.5%), and 'language/writing skills' (81.5%). No items reached consensus for exclusion.

Ranked training needs.
Training needs from participants' ranked top ten lists were categorized into 109 unique items (Dataset 1). Of the 114 respondents to this question, the top priority listed was training in statistics; mentioned by 36.8% of respondents, with a median ranking of 2 (IQR=2). The second ranked need was for training in research methods; mentioned by 21.9% of respondents, with a median ranking of 2 (IQR=1.5). The third highest training need was in publication ethics; mentioned by 20.2% of respondents, with a median ranking of 3 (IQR=2). The fourth highest need was in recruiting and dealing with peer reviewers; mentioned by 17.5% of respondents, with a median ranking of 3 (IQR=4). The fifth highest training need was in indexing of journals; mentioned by 15.8% of respondents, with a median ranking of 2 (IQR=1). The dataset lists all of the training needs named by participants (regrouped into categories of similar items) in their respective lists of top 10 training needs from the survey of editors.

Modified Delphi
We compiled a list of 202 unique competency-related statements identified in our previous research 9 and 12 additional statements identified by participants in the survey of editors in Phase 1 (Supplementary File S3). These 214 items were categorized into seven areas: • Journal publishing (29 competency-related statements) • Publication ethics and research integrity (23 competencyrelated statements) • Journal editing (46 competency-related statements) • Journal promotion (23 competency-related statements) • Dealing with authors (27 competency-related statements) • Editor qualities and characteristics (43 competency-related statements) • Dealing with peer reviewers (23 competency-related statements) Round 1. Eighty-three people participated, all of whom had completed the survey of editors. Of the 214 items listed, 88 items reached consensus for inclusion (≥4 on a 5-point Likert scale). Only one item -'act with integrity and accountability' -was rated as 5 out of 5 by more than 80% of respondents. The items with the highest average score were 'act with integrity and accountability'  Due to the large volume of included items after 3 rounds of the Delphi, the post-hoc decision was made (JG, KDC, LS, DM) to further narrow down the list to a more manageable size. This was done by identifying items that achieved 90% consensus for inclusion in Rounds 2 or 3. This produced 23 "highly rated" items, ( The dataset is a summary of the data collected over the three rounds of the Delphi process. We considered items with 80% consensus of 4 or higher (out of 5) as "Included". and items with 90% consensus of 4.5 or higher as "Highly Ranked".

Discussion
The results from the survey of editors revealed some patterns. First, every item on the list of participant perceptions (Table 2) occupied the same positional ranking for both the degree to which participants perceived an item as important and the degree to which they perceived that they possessed the expertise or skill. Also, consensus was higher on every item for the degree to which participants perceived particular skills or expertise as important compared to the degree to which they possessed each of these skills. This would seem to indicate that the expertise and skills that editors thought were most important were also those for which they believed themselves to be most competent. However, in examining the data more closely, we can see that the largest gaps between the perceived importance and the perceived possession of particular expertise and skills occurred among items where consensus for inclusion was achieved. In particular, 'expertise in research methods', 'expertise in dealing with publication ethics…', 'language/writing skills', and 'author and peer reviewer correspondence…' all had near or above double -digit differences between perceptions of the degree of importance vs. the degree to which participants believed they possessed this expertise or skill. This finding could point to the possibility that despite having more training in the areas they deem most important, editors still may not feel adequately trained in some of these areas.
When comparing the top five items on participants' ranked list of training needs with editors' perceptions, their perceptions of competency in these areas (based on similarly-themed items in the respondent's perceptions of their own knowledge, skills, and abilities) varied from moderate competency (for research methods, publication ethics, and recruiting and dealing with peer reviewers) to low competency (for statistics and indexing).
In the Delphi, participants were quite consistent in their ratings, reflected by the fact that no new items were scored as a 4 or above by 80% of participants after the first round of ratings. Also, although it appears from our sample that editors believed that nearly all of the knowledge, skills, and abilities identified in the scoping review and environmental scan were at least somewhat important to the role of scientific editor, none of these items were rated 2 or lower by 80% of respondents in any round of the Delphi. This could indicate that editors see their role as encompassing a very large number of important interrelated skills, abilities, and knowledge.
Generally, there appears to be some agreement between the survey of editors' perceptions and the Delphi process across many items. When provided with an expanded list of potential competencies in the Delphi (from 38 items in the survey of editors to 230 items in the Delphi), many of the highest rated items from the survey of editors' perceptions remained among the highest rated in the Delphi. For example, of the six items that achieved consensus in the survey of editors' perceptions, four were similar to items on the 'highly rated' list from the Delphi, while the remaining two ('writing/language skills' and 'using good judgment in decision-making') were similar to items in the 'included' list from the Delphi. Additionally, the top 5 items listed in participants' top 10 training needs were all included in the final list of competency-related statements arising from the Delphi, with the 2 nd , 3 rd , and 4 th most cited training needs similar to items in the 'highly rated' list from the Delphi. This finding may suggest that some of the most critical skills related to the position of scientific editor may also be some of the ones for which editors feel the least trained. However, it's unclear whether this is due to the central importance of these elements (and the need for thorough, ongoing training), a true lack of training (whether in terms of availability or quality), both of these factors, or some other reason(s).

Limitations
There were a number of limitations in this research. The limited number of respondents for both the survey of editors' perceptions and the modified Delphi, the fact that the study was conducted in English, and the fact that the survey was completed primarily by medical journal editors may limit the generalizability of the findings to the wider pool of scientific editors around the world. Additionally, for the modified Delphi, we chose to only invite respondents from the survey of editors (however we did not restrict participation to only this group). This decision was made since we believed we had used all of the most pertinent communication channels to recruit participants for the survey of editors, therefore (given our biomedical focus) we were unlikely to gain many more respondents by putting out a further call for participation in the modified Delphi. Moreover, although efforts were made to regroup similar items from participants' top 10 lists of training needs, this subjective process may have failed to regroup some items that could be seen as similar while combining other items that could be judged to be different from one another. A further limitation is in regards to the interpretation of competency-related statements to be rated by participants in the Delphi. While efforts were made to preserve the original wording of competencyrelated statements in the scoping review, some of the participants' comments relayed difficulties in understanding some items and misinterpreting others. This may have led to the mis-rating of a few items by some participants. This limitation should be offset at least to some extent by our use of median scores rather than means.

Conclusion
This research provides an insight into the perceptions of scientific editors of biomedical journals from around the world regarding the importance of particular expertise and skills in their role as scientific editors and the degree to which they believe they possess these competencies. This information complements the competency-related statements identified in the scoping review, as it enabled those who might benefit from educational efforts to identify the most important competency-related statements from their standpoint and to self-identify their greatest needs. Together with a previous scoping review and environmental scan of the literature, these findings were used to inform a consensusbuilding process in which a minimum set of core competencies for scientific editors of biomedical journals was determined at a consensus meeting by a wide-ranging group of stakeholders (to be reported in a future paper). The findings from both the survey and the modified Delphi process reported on here are critical for understanding knowledge and training gaps among scientific editors when designing curriculum around these core competencies in the future.

Dataset 1. Ranked list of training needs
The dataset lists all of the training needs named by participants (regrouped into categories of similar items) in their respective lists of top 10 training needs from the survey of editors. 10.5256/f1000research.12400.d175998 16 Dataset 2. All data for Delphi The dataset is a summary of the data collected over the three rounds of the Delphi process. We considered items with 80% consensus of 4 or higher (out of 5) as "Included", and items with 90% consensus of 4.5 or higher as "Highly Ranked". 10