Introduction

F1000Research

2046-1402

F1000 Research Limited

London, UK

10.12688/f1000research.130326.1

Study Protocol

Articles

Characteristics of bibliometric analyses of the complementary, alternative, and integrative medicine literature: A scoping review protocol

[version 1; peer review: 2 approved with reservations]

Jeremy Y.

Conceptualization Methodology Project Administration Supervision Visualization Writing – Review & Editing https://orcid.org/0000-0003-0031-5873 a 1 2 Liu

Henry

Methodology Visualization Writing – Original Draft Preparation https://orcid.org/0000-0001-9880-7248 2 Shah

Aimun Qadeer

Methodology Visualization Writing – Original Draft Preparation 2 Wieland

L. Susan

Methodology Writing – Review & Editing https://orcid.org/0000-0003-2157-0603 3 Moher

David

Methodology Writing – Review & Editing https://orcid.org/0000-0003-2434-4206 1 4 1Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada 2Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada 3Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA 4School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada

a jerng@ohri.ca

No competing interests were disclosed.

13 2 2023

2023

164

24 1 2023

2023

This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: There is a growing body of literature on complementary, alternative, and integrative medicine (CAIM), which offers a holistic approach to health and the maintenance of social and cultural values. Bibliometric analyses are an increasingly commonly used method employing quantitative statistical techniques to understand trends in a particular scientific field. The objective of this scoping review is to investigate the quantity and characteristics of evidence in relation to bibliometric analyses of CAIM literature.

Methods: The following bibliographic databases will be searched: MEDLINE, EMBASE, PsycINFO, AMED, CINAHL, Scopus and Web of Science. Studies published in English, conducting any type of bibliometric analysis involving any CAIM therapies, as detailed by an operational definition of CAIM adopted by Cochrane Complementary Medicine, will be included. Conference abstracts and study protocols will be excluded. The following variables will be extracted from included studies: title, author, year, country, study objective, type of CAIM, health condition targeted, databases searched in the bibliometric analysis, the type of bibliometric variables assessed, how bibliometric information was reported, main findings, conclusions, and limitations. Findings will be summarized narratively, as well as in tabular and graphical format.

Conclusions: To the best of our knowledge, this scoping review will be the first to investigate the characteristics of evidence in relation to bibliometric analyses on CAIM literature. The findings of this review may be useful to identify variations in the objectives, methods, and results of bibliometric analyses of CAIM research literature.

bibliometric analysis complementary and alternative medicine integrative medicine scientometric analysis scoping review

National Center for Complementary and Integrative Health of the National Institutes of Health

R24AT001293

LSW was supported by the National Center for Complementary and Integrative Health of the National Institutes of Health under award number R24 AT001293.

Introduction

Complementary, alternative, and integrative medicine (CAIM) is a complex term referring to three distinct concepts related to the use of non-conventional medicine. ¹ ^, ² “Complementary medicine” describes non-conventional therapeutic approaches that are used together with conventional therapies. ¹ “Alternative medicine” describes non-conventional therapeutic approaches used in replacement of conventional therapies. ¹ “Integrative medicine” describes the combined use of both conventional and non-conventional therapies in a coordinated manner. ¹ ^, ² For the purpose of this study, each of these approaches may also incorporate elements of “traditional medicine” which is the “knowledge, skills and practices based on the theories, beliefs and experiences indigenous to different cultures, used in the maintenance of health and in the prevention, diagnosis, improvement or treatment of physical and mental illness”. ³ All these concepts will be collectively referred to as “complementary, alternative, and integrative medicine” (abbreviated as CAIM).

CAIM practitioners often emphasize a holistic approach to health, including the consideration of cultural and social values. ² ^, ⁴ Clients often perceive CAIM as better at providing individualized, person-centred care, compared to mainstream health approaches. ⁵ ^, ⁶ Prevalence of CAIM use is increasing worldwide, and, accordingly, the body of literature on CAIM research has grown immensely, with the steepest increase in CAIM publications observed between the mid-2000s and mid-2010s. ⁷ ^, ⁸ It is of interest to determine broad research trends of CAIM research literature, and identify specific CAIM topics explored (e.g., acupuncture, aromatherapy). While some CAIM therapies (e.g., yoga for depressive symptoms, ⁹ exercise therapy for reducing falls in older people ¹⁰) have been shown to be safe and effective, many other therapies have insufficient evidence to demonstrate their effectiveness or safety. ¹¹ ^, ¹² Furthermore, even when basic effectiveness and safety are established, questions often remain about key characteristics such as intervention dose and implementation or applicability to different patient populations and settings. Bibliometric analyses can be used to detect knowledge gaps and to identify research trends that help predict whether such knowledge gaps are likely to be met.

Bibliometric analysis involves the application of quantitative statistical techniques to bibliometric data (e.g., total number of citations, total number of publications) and can be used for a variety of purposes, such as identifying patterns in a given field of research. ¹³ Bibliometric analysis techniques can broadly belong to categories of performance analysis (i.e., techniques measuring contributions of research constituents) or science mapping (i.e., techniques measuring relationships between research constituents). ¹³ Examples of research constituents include authors, countries, institutions, and topics. ¹³ Performance analysis techniques can further be divided into publication-related metrics (e.g., total number of publications), citation-related metrics (e.g., average citations, total number of citations), and citation-and-publication-related metrics (e.g., h-index, g-index, proportion of cited publications). ¹³ Science mapping techniques can include methods such as citation analysis, co-citation analysis, bibliographic coupling, co-word analysis, and co-authorship analysis. ¹³ For instance, co-citation analysis examines the frequency of publications being cited together, which may reveal thematic clusters. ¹³ Enrichment techniques of network metrics (i.e., quantitative measures of research constituents’ relative importance), clustering (i.e., grouping of similar objects using clustering algorithms), and visualization (i.e., graphical visualizations of research constituents’ connections) can be employed to enhance understanding of science mapping techniques. ¹³ For instance, software like VOSviewer can be used to graphically visualize thematic clusters in co-citation analysis. ¹³

Advantages of bibliometric analyses include facilitating the examination of large datasets that are not feasible for investigation by manual review (e.g., literature reviews). ¹³ Further, the relatively low cost and rapidity of conducting bibliometric analyses allow for replicable methods. ¹³ ^, ¹⁴ Use of bibliometric techniques across different scientific fields is becoming increasingly popular. ¹⁵ ^, ¹⁶

A scoping review, which involves mapping the current literature and identifying gaps in research, ¹⁷ would be appropriate to summarize literature on CAIM bibliometric analyses. To the best of our knowledge, no systematic or scoping reviews have been conducted on bibliometric analyses of CAIM therapies. A preliminary search of the Cochrane Database of Systematic Reviews and the Scopus database revealed no existing systematic or scoping reviews on the topic. Synthesizing bibliometric analyses on CAIM will provide insight into trends, such as the types of CAIM literature typically analysed from a bibliometric lens, statistical techniques that bibliometric analyses on CAIM utilize, and more broadly, where the field of CAIM is headed. Considering the lack of standardized procedures for bibliometric analyses, this review will also improve understanding of how bibliometric analyses are currently conducted on this topic. Thus, the purpose of this review will be to understand the characteristics of bibliometric analyses of CAIM research literature, which can inform future work within the field.

Protocol Methods

Approach and eligibility criteria

The present scoping review’s research question is: “What are the characteristics of bibliometric analyses of the CAIM literature?”. The proposed scoping review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews, which recommends quantitative and descriptive analyses of main findings. ¹⁷ This protocol’s associated files (Appendix A – Search Strategies, and the PRISMA ScR Checklist) has been made openly available on Open Science Framework (see Extended data and Reporting guidelines ³³).

The search strategy will aim to locate studies published in journals only (excluding grey literature), with no date restrictions (i.e., from inception to date of search execution). The only eligible study design will be bibliometric analyses (encompassing terms for “bibliometric analysis”, “scientometric analysis”, and “citation analysis”), or articles that include both a bibliometric analysis and another study design (e.g., bibliometric analysis and systematic review). All included bibliometric analyses will be focused on one or more CAIM therapies, as defined by a recently published operational definition of CAIM. ¹⁸ This operational definition was created using a systematic search of four peer-reviewed or other quality-assessed resource types: 1) peer-reviewed articles from seven major bibliographic databases, 2) “Aims and Scope” webpages of peer-reviewed CAIM journals, 3) entries containing CAIM therapies in highly accessed online encyclopedias, and 4) highly ranked websites resulting from Health On the Net Code of Conduct (HONcode) searches. ¹⁸ To date, this operational definition includes the greatest number of evidence sources and is the only one that captures the concept of “integrative medicine”, alongside “complementary medicine” and “alternative medicine”. ¹⁸

Grey literature sources will be excluded as bibliometric analysis studies are unlikely to be found outside of traditional academic publishing channels. Conference abstracts and study protocols will also be excluded as they likely will not contain adequate information required to describe the characteristics of bibliometric analyses on CAIM literature. Finally, all non-English publications will be excluded, due to language constraints of the authors.

Search strategy

The following electronic databases will be searched: MEDLINE, EMBASE, PsycINFO, and AMED (accessed via the OVID research platform), as well as CINAHL (accessed via EBSCOhost), Scopus, and Web of Science. The search strategy will include a comprehensive search string of CAIM terms ¹⁹ encompassing 604 distinct therapies described previously in an operational definition of CAIM. ¹⁸ This search string of CAIM terms was created for OVID (e.g., MEDLINE, EMBASE, PsycINFO, AMED) and EBSCO platform (e.g., CINAHL) databases, as well as Scopus and Web of Science databases. ¹⁹ Relevant scientific names and/or synonyms were added as a term (i.e., keyword, phrase), alongside relevant boolean operators. The comprehensive search string of CAIM ¹⁹ will be combined with search terms for bibliometric analyses (e.g., bibliometric analysis, statistical bibliography, citation analysis). The search strategy, including all identified keywords and equivalent index terms, will be adapted for each included database. All search strategies that will be run are provided in Extended data, ³³ informed by PRISMA-S guidelines for reporting literature searches. ²⁰ ^, ³³

Study and source of evidence selection

Following the search, all identified citations will be collated and exported into Covidence, and duplicates will be removed. All titles/abstracts, followed by full texts, will be screened by reviewers independently and in duplicate. First, pilot title/abstract screening of a sample of twenty articles will be conducted by AQS and HL. A meeting will be held between AQS, HL, and JYN to discuss any challenges and resolve discrepancies. Following the pilot test, all titles/abstracts will be screened for inclusion by AQS and HL. Then, pilot full-text screening will be conducted, in which AQS and HL will screen ten full texts. A meeting will be held between AQS, HL, and JYN to discuss challenges and resolve discrepancies. After this pilot step, full-text screening will be completed by AQS and HL. Reasons for exclusion of full texts will be recorded. Any disagreements that arise between reviewers throughout the selection process will be resolved on a weekly basis through discussion with HL, AQS, and JYN, if disagreements still cannot be resolved. The results of the search strategy and the study screening process will be presented in a PRISMA-ScR flow diagram in the final scoping review. ²¹

Data extraction

Data extraction will be conducted using Excel software. The data extraction form that will be used in this scoping review is informed by Donthu et al., ¹³ which provides an overview of how to conduct a bibliometric analysis. The extraction form will be developed in two stages. In stage one, AQS and HL will select ten articles at random that met the inclusion criteria from a preliminary search of CAIM and bibliometric analysis search terms on Scopus. AQS and HL will independently extract information from five articles each and meet to discuss discrepancies. A meeting will be held among AQS, HL, and JYN, to discuss changes needed to improve the form. In stage two, AQS and HL will identify the ten most highly cited articles that met inclusion criteria after running a search of CAIM literature and bibliometric analyses terms on Scopus, before independently extracting information from five articles each using the latest version of the data extraction form. Another meeting will be held between AQS and HL, and then with JYN, to approve the latest version of the extraction form.

The following information will be extracted from eligible bibliometric analyses: title, author, year, country, aim of the study, secondary study design (if applicable), the type of CAIM(s), the health condition or population targeted, main findings, conclusions, and limitations. Also, the bibliometric information described will be summarized, including the databases searched, type of bibliometric methodology (i.e., performance analysis [such as citation-metrics or publication metrics] versus science mapping [such as co-word analysis, co-authorship analysis, bibliographic coupling, or enrichment techniques]), the number of studies included in the analysis, the number of metrics used, how information was reported (e.g., narrative summary, figures, tables, visualization software used), and how all variable measures align with the Donthu et al. ¹³ guideline for conducting bibliometric analyses.

To ensure consistency and quality of the extraction, an initial pilot test of data extraction will be conducted by all participating independent reviewers, assessing five articles. All reviewers will then meet with JYN, HL, and AQS to resolve discrepancies and disagreements. Based on the initial pilot testing, revisions to the data extraction form can be proposed and implemented. Upon completion of the pilot extraction step, reviewers will be divided into two teams led by HL and AQS. Teams will be further divided into pairs of two reviewers for duplicate data extraction of the same set of bibliometric full texts. These duplicate extractions will be reviewed by AQS and HL to ensure consistency. Weekly meetings will be held between reviewers and HL and AQS to standardize the data extraction process and resolve any issues identified. Any conflicts that cannot be resolved will be discussed with JYN.

We seek to conduct a scoping review. As we are going to use the JBI methodology for scoping reviews, we have elected not to conduct a risk of bias assessment.

Data analysis and presentation

The data will be summarized descriptively (e.g., frequencies of the number of studies, country, types of CAIM, outcomes reported in the bibliometric analyses) with full results presented in tabular format. Frequency of CAIM bibliometric analysis publications over time will be presented in graphical format. An additional figure will be created highlighting the types of bibliometric information each study reported.

Dissemination

The findings of this review will be disseminated in scientific journals.

Study status

The literature search is ongoing.

Discussion

We anticipate this project will advance the understanding of topics and trends in CAIM research, including what types of CAIMs are most commonly explored and what health conditions these CAIMs target. Accordingly, it can help CAIM researchers locate bibliometric sources to inform future research directions or identify gaps that warrant further investigation. It is anticipated that this review will also provide unique insights on how bibliometric analyses of CAIM literature are conducted, including the type of methodology used (such as performance analysis metrics or science mapping techniques), the number and types of outcomes reported, and how bibliometric information is presented (e.g., narratively, graphically).

As the number of scientific publications has been growing exponentially in the last fifty years, bibliometrics has become useful to quantitatively analyse publications on a particular topic. ²² Generally, there is large variability in how bibliometric analyses are conducted, as there is no authoritative guideline on bibliometric methodology. ¹³ ^, ¹⁶ Findings from the completed review can help identify whether there are any inconsistencies in the way that bibliometric analyses specifically on CAIM literature are conducted. This may be useful to inform future, standardized reporting guidelines for bibliometric analyses, generally, or with a potential focus on CAIM topics. Further, it may be expected that performance analysis metrics used to measure research constituents’ scientific impact (e.g., h-index, g-index, total citations) differ between bibliometric analyses. ²³ Given the varying capability of different performance analysis metrics in capturing scientific impact of a given research constituent, ²⁴ this review could reveal the extent to which bibliometric analyses of CAIM literature are effectively measuring scientific impact. Comparisons of different research constituents’ scientific impact are further complicated by how average values of bibliometric indicators often differ between disciplines (e.g., molecular biology, nursing). ²² This is particularly pertinent to CAIM literature, given the interdisciplinary applications of many CAIM therapies. ²⁵ ^– ²⁷

Bibliometric methodology can be influenced by database changes such as the indexing of new journals or articles. ²⁸ Due to limitations of visualization softwares like VOSviewer, often only either Scopus or Web of Science databases can be searched, which cover most but not all databases. ²⁹ If the findings of this scoping review reveal similar limitations of bibliometric analyses conducted on CAIM literature, these limitations could potentially drive investigation into techniques that will improve analysis of bibliometric results. Additionally, there are no known critical appraisal tools for bibliometric analyses. Concerns have been expressed over a lack of knowledge of good practices when conducting bibliometric analyses. ³⁰ While this is outside the scope of this present review, future research may investigate assessment tools to evaluate the quality of published bibliometric analyses. The identification of a group of bibliometric analyses in this review may potentially serve as a test set for the development and investigation of quality indicators.

Strengths and limitations

Strengths of this study will include adherence to the JBI methodology for scoping reviews ¹⁷ and use of a comprehensive systematic search strategy ¹⁹ across several databases to identify eligible articles. Another strength is that screening and data extraction will be conducted in duplicate, significantly reducing bias. There are some limitations expected in this review. By only including studies written in English, we could be missing important international work. For example, Chinese databases may contain a higher volume of CAIM articles but are unable to be searched in this study. This is especially relevant as some forms of CAIM may be practiced more frequently in non-English speaking regions of the world, such as traditional Chinese medicine in China. ³¹ ^, ³² Additionally, reported findings are expected to be descriptive in nature, such as the frequencies of the types of CAIMs examined in bibliometric studies or the frequencies of studies that engaged in science mapping versus performance analysis techniques. This makes it difficult to extrapolate themes or correlates, like which CAIMs are effective, or which bibliometric techniques are preferable.

Data availability Underlying data

No underlying data are associated with this article.

Extended data

Open Science Framework: Characteristics of Bibliometric Analyses of Complementary, Alternative, and Integrative Medicine Literature: A Scoping Review. https://doi.org/10.17605/OSF.IO/JSQWY. ³³

This project contains the following extended data: -

Appendix A - Search Strategies_Jan0923.docx (search strategies for MEDLINE, EMBASE, PsycINFO, AMED, CINAHL, Scopus and Web of Science databases).

Reporting guidelines

Open Science Framework: PRISMA-ScR checklist for ‘Characteristics of bibliometric analyses of the complementary, alternative, and integrative medicine literature: A scoping review protocol.’ https://doi.org/10.17605/OSF.IO/JSQWY. ³³

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

References 1

National Center for Complementary and Integrative Health: Complementary, alternative, or integrative health: what’s in a name? 2021. Reference Source

Boon

Thompson

: Making sense of “alternative”, “complementary”, “unconventional” and “integrative” medicine: exploring the terms and meanings through a textual analysis. BMC Complement. Altern. Med. 2016;16:134. 27206976

10.1186/s12906-016-1111-3

PMC4875612

World Health Organization: Traditional Medicine. 2021. Reference Source

Gillett

Andrews

Savelli

: Health & Society: Critical Perspectives. Oxford University Press;2016.

Bishop

Yardley

Lewith

: Treat or treatment: a qualitative study analyzing patients’ use of complementary and alternative medicine. Am. J. Public Health. 2008;98(9):1700–1705. 18172145

10.2105/AJPH.2007.110072

PMC2509600

Dodds

Bulmer

Murphy

: Consumer value in complementary and alternative medicine (CAM) health care services. Australas. Mark. J. AMJ. 2014;22(3):218–229. 10.1016/j.ausmj.2014.08.004

: Insight into the characteristics of research published in traditional, complementary, alternative, and integrative medicine journals: a bibliometric analysis. BMC Complement. Med. Ther. 2021;21(1):185. 34210316

10.1186/s12906-021-03354-7

PMC8246686

Tangkiatkumjai

Boardman

Walker

: Potential factors that influence usage of complementary and alternative medicine worldwide: a systematic review. BMC Complement. Med. Ther. 2020;20(1):363. 10.1186/s12906-020-03157-2

Cramer

Lauche

Langhorst

: Yoga for depression: a systematic review and meta-analysis. Depress. Anxiety. 2013;30(11):1068–1083. 10.1002/da.22166

Sherrington

Fairhall

Wallbank

: Exercise for preventing falls in older people living in the community. Cochrane Database Syst. Rev. 2019;2019. 10.1002/14651858.CD012424.pub2

Brinsley

Schuch

Lederman

: Effects of yoga on depressive symptoms in people with mental disorders: a systematic review and meta-analysis. Br. J. Sports Med. 2021;55(17):992–1000. 32423912

10.1136/bjsports-2019-101242

Pratt

Wieland

Ahmadzai

: A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions. Syst. Rev. 2020;9(1):97. 32354348

10.1186/s13643-020-01328-3

PMC7191816

Donthu

Kumar

Mukherjee

: How to conduct a bibliometric analysis: An overview and guidelines. J. Bus. Res. 2021;133:285–296. 10.1016/j.jbusres.2021.04.070

Rowlands

: What are we measuring? Refocusing on some fundamentals in the age of desktop bibliometrics. FEMS Microbiol. Lett. 2018;365(8). 29718194

10.1093/femsle/fny059

Ellegaard

Wallin

: The bibliometric analysis of scholarly production: How great is the impact? Scientometrics. 2015;105(3):1809–1831. 26594073

10.1007/s11192-015-1645-z

PMC4643120

González-Alcaide

: Bibliometric studies outside the information science and library science field: uncontainable or uncontrollable? Scientometrics. 2021;126(8):6837–6870. 10.1007/s11192-021-04061-3

Peters

MDJ

Godfrey

McInerney

: Chapter 11: Scoping Reviews (2020 version). Aromataris

Munn

, editors. JBI Manual for Evidence Synthesis. JBI;2020. 10.46658/JBIMES-20-12

Dhawan

Dogadova

: Operational definition of complementary, alternative, and integrative medicine derived from a systematic search. BMC Complement. Med. Ther. 2022;22(1):104. 35413882

10.1186/s12906-022-03556-7

PMC9006507

Dhawan

Dogadova

: A comprehensive search string informed by an operational definition of complementary, alternative, and integrative medicine for systematic bibliographic database search strategies. BMC Complement. Med. Ther. 2022;22(1):200. 10.1186/s12906-022-03683-1

Rethlefsen

Kirtley

Waffenschmidt

: PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst. Rev. 2021;10(1):39. 33499930

10.1186/s13643-020-01542-z

PMC7839230

Tricco

Lillie

Zarin

: PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018;169(7):467–473. 10.7326/M18-0850

Belter

: Bibliometric indicators: opportunities and limits. J. Med. Libr. Assoc. 2015;103(4):219–221. 26512227

10.3163/1536-5050.103.4.014

PMC4613388

Kumar

George

Anisha

: Bibliometric Analysis for Medical Research. Indian J. Psychol. Med. 2022;02537176221103617.

Koltun

Hafner

: The h-index is no longer an effective correlate of scientific reputation. PLoS One. 2021;16(6):e0253397. 34181681

10.1371/journal.pone.0253397

PMC8238192

Frisch

Rabinowitsch

: What's in a Definition? Holistic Nursing, Integrative Health Care, and Integrative Nursing: Report of an Integrated Literature Review. J. Holist. Nurs. 2019;37(3):260–272. 10.1177/0898010119860685

Gupta

Prakash

: Nutraceuticals for geriatrics. J. Tradit. Complement. Med. 2015;5(1):5–14. 26151003

10.1016/j.jtcme.2014.10.004

PMC4488102

Thomson-Casey

Adams

McIntyre

: Complementary medicine in psychology practice: an analysis of Australian psychology guidelines and a comparison with other psychology associations from English speaking countries. BMC Complement. Med. Ther. 2022;22(1):171. 35752820

10.1186/s12906-022-03620-2

PMC9233840

Wang

: A bibliometric model for identifying emerging research topics. J. Assoc. Inf. Sci. Technol. 2018;69(2):290–304. 10.1002/asi.23930

Echchakoui

: Why and how to merge Scopus and Web of Science during bibliometric analysis: the case of sales force literature from 1912 to 2019. J. Mark. Anal. 2020;8(3):165–184. 10.1057/s41270-020-00081-9

Mejia

Zhang

: Exploring Topics in Bibliometric Research Through Citation Networks and Semantic Analysis. Front. Res. Metr. Anal. 2021;6:742311. 34632257

10.3389/frma.2021.742311

PMC8498340

Cheung

: TCM: Made in China. Nature. 2011;480(7378):S82–S83. 10.1038/480S82a

Fjaer

Landet

McNamara

: The use of complementary and alternative medicine (CAM) in Europe. BMC Complement. Med. Ther. 2020;20(1):108. 32252735

10.1186/s12906-020-02903-w

PMC7137515

Liu

Shah

: Characteristics of Bibliometric Analyses of the Complementary, Alternative, and Integrative Medicine Literature: A Scoping Review.[dataset].2023, January 19. 10.17605/OSF.IO/JSQWY

10.5256/f1000research.143074.r163736

Reviewer response for version 1

Trager

Robert J

1 Referee https://orcid.org/0000-0002-4714-1076 1Connor Whole Health, University Hospitals Cleveland Medical Center, Cleveland, OH, USA

Competing interests: No competing interests were disclosed.

14 8 2023

2023

This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

recommendation

approve-with-reservations

Overview

I congratulate the authors on an extremely well-written, thorough, and clear protocol for a scoping review of bibliometric studies on CAIM. They adequately justify the rationale for the study and describe how their findings may impact the CAIM literature. The methodology is concise yet descriptive. I also appreciated that open-access files were included on OSF, along with an impressive search strategy. I have no major comments regarding the protocol, and only a couple of minor comments.

Minor comments

The authors note that their study will help identify “ what health conditions these CAIMs target,” and in the abstract state “ health condition targeted.” While this is certainly OK, I believe that the language could be slightly altered in these sections of text. I expect that the authors may encounter bibliometric studies of CAIM which describe not only the treatment of certain conditions, but also allude to a diagnostic, clinical reasoning, case management process, or the use of CAIM preventative purposes among individuals who are already healthy (e.g., “wellness”). For example, in our bibliometric study of chiropractic case reports, we found an increasing trend in studies describing the diagnosis of vascular disorders. While I am less familiar with other fields, I imagine a similar phenomenon may be noted for CAIM-related professions wherein providers have a broad scope of practice, requisite on diagnosis, such as osteopathy or physical therapy. I think the protocol could be therefore altered slightly to change “target(ed)” to “manage(ed)” or some other language that is broader to reflect more than just treatment, but rather an overall management of the patient.

The authors describe CAIM as “therapies” throughout the manuscript. I think they have the liberty to describe it this way, yet I would caution them that in some instances the CAIM therapy is distinct from the branch of providers that often use that therapy. For example, in my field of chiropractic, chiropractors often use spinal manipulation, yet chiropractors also perform diagnosis and referral in their management of health conditions and sometimes omit spinal manipulation altogether. One might also consider that a single CAIM therapy could be provided by several types of practitioners. To continue with the above example, osteopaths and physical therapists also use spinal manipulation. The authors could have an a priori method for handling how they categorize CAIM therapies versus practitioner types. However, given that this may be confusing and/or unnecessary to establish before seeing the results, the authors could also describe the categorization of CAIM therapies versus provider types as an iterative process, subject to change if there is overlap between the two.

Comments for clarity

Data extraction – The phrase “We seek to conduct a scoping review” is redundant and could be deleted.

This is totally optional as it’s not directly related to the manuscript but may be helpful for other readers – on OSF I could not preview the two Appendix files and had to download them to be able to see them. Consider uploading the files as a PDF in OSF so they can be viewed within an internet browser. However, I recommend also keeping the original Word documents on OSF.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

chiropractic, CAIM

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Jeremy

University of Ottawa, Canada

Competing interests: No competing interests were disclosed.

6 10 2023

We kindly thank the reviewer for their feedback.

Regarding minor comment 1, we agree that the language can be changed from “target(ed)” to “manage(d)” to account for the fact that not all bibliometric analyses may specifically discuss treatment, but rather provide a broad overview of the field in regards to diagnostics, clinical reasoning, or condition management. We have implemented these changes to the wording in the Abstract, 2nd paragraph of Data Extraction, and 1st paragraph of Discussion. Additionally, we will also be extracting main findings, conclusions and limitations of each bibliometric analyses, which is another way that we are able to capture some of the interesting findings the reviewer mentioned that goes beyond the “treatment” of conditions.

Regarding minor comment 2 and the terminology of CAIM as a “therapy”, to define the types of complementary, alternative, and integrative therapies (CAIM) eligible for inclusion in our review, we are using a Cochrane Complementary Medicine-adopted operationalized list of CAIMs ¹, which are described as “CAIM therapies”. For consistency, we are using the same terminology. However, the operationalized list is inclusive of many provider types/branches, including chiropractic and osteopathic fields. The data extraction of “CAIM therapies” will be reported in accordance with terminology used by the operationalized list.

We also find the reviewer’s point interesting and acknowledge that different CAIM practices or techniques (e.g., spinal manipulation) may be employed by different types of practitioners. However, we anticipate that not all bibliometric analyses included in our review would include the same types of metrics. Accordingly, not all bibliometric analyses may report the types of providers that are providing these CAIM therapies in the literature. In our data extraction form, we will report whether a paper has included the “types of providers” as a metric in their bibliometric analysis. However, we will unfortunately be unable to extract extensive details on which providers would be providing these therapies. This is for the sake of consistency across all the studies, and is in line with JBI scoping review methods which aims to map and provide a broad overview of the literature. To your point, it would be difficult to devise methods for categorizing CAIM therapies versus provider types as described.

Regarding the comments for clarity, we agree that the phrase “We seek to conduct a scoping review” is redundant and have removed it from the protocol. Additionally, we will upload PDF versions of the two Appendix files, alongside the Microsoft Word document files, in Open Science Framework.

References

Ng JY, Dhawan T, Dogadova E, Taghi-Zada Z, Vacca A, Wieland LS, Moher D. Operational definition of complementary, alternative, and integrative medicine derived from a systematic search. BMC. Complement. Altern. Med. 2022;22(1):104. doi: 10.1186/s12906-022-03556-7

10.5256/f1000research.143074.r181379

Reviewer response for version 1

Brodin Danell

Jenny-Ann

1 Referee 1Umeå University, Umeå, Sweden

Competing interests: No competing interests were disclosed.

14 8 2023

2023

recommendation

approve-with-reservations

Thanks for the protocol! The introduction is relevant and ok, both regarding definition of central concepts, relevance of analyzing CAIM literature, and what/how bibliometric methods can be helpful for that.

However, I would prefer a more elaborated motivation of the study. Why is it relevant to analyze the characteristics of bibliometric studies of CAIM? Are there reasons to expect them to be different from other medical/health sub-fields? Are there particular pitfalls? And isn’t it more or less necessary to compare these studies with other areas to be able to say something relevant about the patterns (at least indirectly? I have not checked the details, but there seem to be a few similar scoping reviews on other medical sub-fields.)

The search procedure/extraction seems appropriate, as well as the choice of databases. Characteristics/information that will be extracted is also relevant in relation to the scope of the study.

A minor detail; the formulations about lack of standardized procedures in bibliometric analysis just seem blunt. Bibliometrics is, as you know, a diverse field of research methods and there are never ending discussions about limitations, potential developments (etc.), but there are certainly standardized procedures.

Looking forward to the results!

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Partly

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Sociology, bibliometrics, medical sociology

Jeremy

University of Ottawa, Canada

Competing interests: No competing interests were disclosed.

6 10 2023

We kindly thank the reviewer for their feedback.

Regarding our motivations for conducting this study, we are particularly interested in mapping bibliometric analysis studies of CAIM, as, to our knowledge, no systematic or scoping reviews have been completed on this topic. The prevalence of CAIM use is increasing worldwide, and the body of CAIM literature has grown immensely in the past two decades, as we mentioned in our introduction section. Bibliometric analyses of CAIM would capture interesting trends in the field such as the publications over time of different CAIM therapies, or the countries that contribute most to particular research topics. This may be useful to CAIM researchers to identify gaps in the CAIM field and where research is headed. We have stated these points in the last paragraph in the introduction section.

Further, analyzing the characteristics of bibliometric analyses of CAIM literature can allow us to identify commonly and less commonly used bibliometric analysis metrics across studies. This could be of interest to inform the development of reporting guidelines for bibliometric analyses. As stated in the introduction section, while there are guidelines for how to conduct bibliometric analyses, there is no consensus in the literature on what should be included. We anticipate that articles may have diverse or unique bibliometric variables, or may even range in the number of variables used. As stated above, this could be useful to improve our understanding of how bibliometric analyses are conducted, and can help to inform future reporting guidelines on this methodology.

Regarding your suggestion on comparing characteristics of CAIM bibliometric analyses to other fields, we agree this would be interesting. We have not been able to locate systematic or scoping reviews that report the characteristics of bibliometric analyses (e.g., performance analysis or science mapping techniques used, main findings, health conditions managed) of CAIM literature or of other medical/health sub-fields. Accordingly, it is difficult to ascertain how our findings would compare to other medical/health sub-fields. We would be interested if the reviewer is able to share any identified review articles exploring the characteristics of bibliometric analyses in other medical fields.

Regarding the standardized procedures of bibliometrics, we acknowledge that bibliometric analyses can be conducted in diverse ways, and there are some guidelines on the types of metrics that can be used (e.g., Linnenleucke et al., 2019 ¹; Donthu et al., 2021 ²). For example, we plan to use the paper by Donthu et al. ² to inform our scoping review. However, there is a lack of consensus in the literature on what baseline information should be required as part of a bibliometric analysis, unlike other research designs (e.g., PRISMA guidelines and JBI methods have been widely adopted for reporting and conduct of systematic reviews). We have rephrased the sentence in the 5th paragraph of the Introduction section to reflect that standardized procedures exist, but that there is no universal standard or consensus in the literature on what a bibliometric analysis should at minimum entail.

References

Linnenluecke MK, Marrone M, Singh AK. Conducting systematic literature reviews and bibliometric analyses. Aust. J. Manag. 2020;45(2):175-194. doi: 10.1177/0312896219877678

Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: An overview and guidelines. J. Bus. Res. 2021;133:285-296. doi: 10.1016/j.jbusres.2021.04.070