Exploring machine learning: A bibliometric general approach using SciMAT [version 1; peer review: 2 approved with reservations]

Background: Machine learning is becoming increasingly important for companies and the scientific community. In this study, we perform a bibliometric analysis on machine learning research, in order to provide an overview of the scientific work during the period 2007-2017 in this area and to show trends that could be the basis for future developments in the field. Methods: This study is carried out using the SciMAT tool based on results extracted from Scopus. This analysis shows the strategic diagrams of evolution and a set of thematic networks. The results provide information on broad tendencies of machine learning. Results: The results show that SciMAT is a useful tool to carry out a science mapping analysis, and emphasizes the premise that machine learning has boundless applications and will continue to be an interesting research field in the future. Conclusions: Some of the conclusions exposed in this study show that classification algorithms have been widely studied and represent a relevant tool for generating different machine learning applications. Nonetheless, regression algorithms are becoming increasingly important in the scientific community, allowing the generation of solutions to predict diseases, sales, and yields, for example.


Introduction
The machine learning field researches different human learning processes, the theoretical analysis of possible learning algorithms and methods for several application domains 1 . Studies based on machine learning have allowed scientists and companies to predict mass mortality events 2 , the quality of water 3 , segment clients in private banking 4 , automatically classify text 5 or for the production of crops, such as cocoa 6 . Considering the growing interest of the scientific community in machine learning research and its challenges, it is interesting and necessary to analyze the field. A good approach for that purpose is science mapping analysis because it is a different way of visualizing information that allows a new researcher to become familiar with a field. An example of science mapping analysis providing an overview of the conceptual evolution of a field in medicine is proposed in 7. In this study, we perform a science mapping analysis to explore machine learning research. The objective of the study is to allow new data analysts to know the current knowledge base about machine learning and to have an initial point to explore applications in this field. This article has the following structure: In the Methods section, we describe the research methodology, the dataset used, the tool configuration, and how the analysis was performed. The Results section presents the results of the science mapping analysis. The conclusions are at the end of the article.

Dataset for visualization analysis
We used Scopus as the bibliographic source. We looked, in the third quarter of 2017, for references of articles and conferences about machine learning, using this concept as the search keyword ('machine AND learning'), with results ranging from 2007 to 2017(Q2). The concept was searched for in the article title, abstract and keywords of articles found. These articles were sorted by date (newest first). All the articles, between 2007 and 2017, were taken in to account for performing the analysis with the aim of obtaining a general vision of the field.
We obtained 67,475 records that were saved using RIS format in different files sorted by year. Figure 1 shows a summary of the records, and shows that research in the field of machine learning has been growing steadily. It is important to observe that the results for 2017 are not in the figure since the records are only to the second half Q2 of year and database is permanently updated. However, these results were used for the analysis because they show trends, which is the primary objective of the present study.

Parameter design
We used the records from Scopus for executing the analysis with SciMAT version 1.1.04. In this tool, the unit of analysis was Words. Primarily, we did a deduplication process, grouping similar words (by plurals) and looking for synonyms or duplicates in words with the highest number of documents and repetitions, always trying to avoid bias to include the largest number of terms. After that, we divided the time interval . We distributed the gaps this way in order to have a comparable number of articles in each one of them. Finally, we carried out the analysis with the following configuration: all the periods, author's words as the unit of analysis, a minimum frequency for data reduction of 100 (excepting 2017 (Q2) with 50) and co-occurrence as a kind of network. Other configurations were: network reduction equal to one, association strength as a normalization measure, Simple centers algorithm with a maximum net size of seven and a minimum of five, core mapper for the document mapper, h-index and Sum citations as the quality measures and, lastly, association strength for the evolution and overlapping map.

Analysis design
SciMAT shows a spatial representation of the way disciplines, fields, specialties and documents or authors related to one another [8][9][10] . For this purpose, the tool implements a longitudinal framework, which takes as its base a co-word analysis and the h-index. Co-word analysis firstly provides information on the themes of a research field and, secondly, enables to analyze and to track the evolution of a study field throughout consecutive periods of time 11 . The h-index is used to measure the impact of the various identified thematic areas 8 .
In this tool, we follow four steps as mentioned by 10: researching cluster detection, drawing strategic diagrams, plotting of thematic areas and carrying out a performance analysis. For the first one, the tool creates a network of keywords co-occurrence based on 12 and 13 and makes a clustering of keywords to topics, using the Simple center's algorithm. For the second step, according to 13, the cluster centrality and density rank values are relevant. The centrality measures the intensity of the interaction of a group with the others; if the cluster is boldly related to the field of research, then the link will be stronger. The density measures the intensity of internal links inside the group. According to 8 and 10, the themes of the clusters can be classified into four groups using these two measures: (1) Motor themes, which are both well developed and central to the research field; (2) basic and transversal themes, which are not sufficiently developed topics but significant for the area of investigation; (3) emerging or declining themes, which are weakly studied and marginal; (4) highly developed and isolated themes, which have welldeveloped internal links, but reduced external links and have only minimal importance to the field. The third step is the plotting of thematic areas, and the last one is to conduct the performance analysis. In this study, we analyzed quantitative measures, such as the number of documents and authors.

Results
To analyze the most relevant topics of investigation in different years, SciMAT uses strategic diagrams. We generated a diagram for each period of the study. The charts are divided into four quadrants: the upper-right quadrant alludes to the motor themes, the upper-left quadrant to the highly developed and isolated themes, the lower-right one to the basic and transversal themes, and the lower-left one to the emerging or declining topics 9 . of the emerging topics. SEQUENCE-ANALYSIS-PROTEIN appears as the motor theme with the highest density (0.83) and centrality (3.21) values, followed by DATABASES-PROTEIN, with centrality equal to 2.97 and density equal to 0.43. This suggests that machine learning was widely used to study the protein molecule in that period, covering, for example, studies to predict the structure or function of that molecule.
Due to the great number of themes that appear in the strategic diagram, we decided to choose three themes, giving priority to those that may have a high impact application. Figure 3A shows the net for the theme DATABASES-PROTEIN, which has a density of 0.43, a centrality of 2.97 and a document count of 322. In this net, we can observe that an important topic is PROTEIN-STRUCTURE. Figure 3B presents the network for the theme DATASETS, which has a density of 0.06, a centrality of 1.37, a document count of 273 and strongly related topics such as CLASSIFIERS, SEMI-SUPERVISED-LEARNING and RAN-DOM-FOREST. Figure 3C shows the network for the theme IMAGE-CLASSIFICATION, which has a density of 0.11, a centrality of 1.3, a document count of 273 and relevant concepts, such as NEURAL-NETWORKS, SUPPORT-VECTOR-MACHINE (SVM) and IMAGE-PROCESSING. From the second period, we selected three thematic networks. Figure 5A shows the net for the theme PROTEIN-ANALYSIS, which has a density of 0.58, a centrality of 2.18 and a document count of 229. This shows us that topics about proteins continue to be important in this period. Figure 5B presents the network for the theme CHEMISTRY, which is an emergent theme and has a density of 0.32, a centrality of 2.24 and a document count of 346. Figure 5C shows the network for the subject VIRTUAL-REALITY, which has a density of 0.07, a centrality of 0.98, a document count of 65 and is another emerging theme for the period 2010-2012.
The strategic diagram of the period 2013-2014 is shown on Figure 6, which has 32 themes. During this time frame, SENSORS, FACE-RECOGNITION and COMMERCE are some of the emerging topics. The IMAGE-INTERPRETATION-COMPUTER-ASSISTED topic appears as the motor theme, with the highest density (0.58) and centrality (2.61) values. This suggests that in this period there were studies on machine learning applied to different topics, such as sensor data, costs, and gesture recognition. We selected three thematic networks from the third period (2013-2014). Figure 7A shows the network for the theme AMINO-ACID-SEQUENCE, which has a density of 0.42, a centrality of 1.91 and a document count of 276. Figure 7B presents the network for the theme MOBILE-DEVICES, which has a density of 0.21, a centrality of 0.76, a document count of 159 and strongly related topics such as HUMAN-COMPUTER-INTERACTION, E-LEARNING, and UBIQUITOUS-COMPUTING. Figure 7C shows the network for the theme COMMERCE, which has a density of 0.07, a centrality of 0.88, a document count of 205 and relevant concepts, such as SOCIAL-NETWORKING and COSTS.   (label generated for algorithms such as Random-Forest or Decision-Trees) are some of the emerging themes. NUCLEAR-MAGNETIC-RESONANCE-IMAGING appears as the motor subject with the highest density (0.57) and centrality (2.81) values. Other important themes are COMPUTATIONAL-BIOLOGY and MEDICAL-IMAGING. We found that during this period biology and health were once again relevant topics in machine learning research.
From the fourth time frame, we selected three thematic networks. Figure 9A shows the network for the theme NUCLEAR-MAGNETIC-RESONANCE-IMAGING, which has a density of   Figure 9B presents the network for the theme COMPUTATIONAL-BIOLOGY, which is an emergent theme and has a density of 0.44, a centrality of 1.8 and a document count of 320. Figure 9C shows the network for the topic SMARTPHONES, which has a density of 0.07, a centrality of 0.13, a document count of 144 and is another emerging theme for the 2015 period.
The strategic diagram of the period 2016 is shown in Figure 10, which has 24 themes. During this time span, UBIQUITOUS-COMPUTING and COMMERCE are some of the emerging themes. The MIDDLE-AGED theme -which refers to applications developed for middle-aged people-appears as the motor topic with the highest density (0.76) and centrality (2.04) values. We selected three thematic networks from the fifth period (2016). Figure 11A shows the network for the theme MEDICAL-IMAGING, which has a density of 0.26, a centrality of 1.45 and a document count of 312. Figure 11B presents the network for the theme INTRUSION-DETECTION, which has a density of 0.3, a centrality of 0.89 and a document count of 289. Figure 11C shows the network for the theme UBIQUITOUS-COMPUTING, which has a density of 0.07, a centrality of 1.07, a document count of 132 and relevant concepts, such as AUTOMATION, SMARTPHONES and the INTERNET.   From 2017(Q2), we selected three thematic networks. Figure 13A shows the net for the theme MEDICAL-IMAGING, which has a document count of 137. Figure 13B presents the network for the topic FORESTRY (label generated for algorithms such as Random-Forest or Decision-Trees), which has a density of 0.17, a centrality of 1.84 and a document count of 220.

Conclusions
Exposing emerging trends in the field of machine learning allows researchers to increase their understanding of the changes and the evolution over time of this research field. One of the primary objectives of a science mapping analysis is to highlight trends and possible relationships between the relevant topics of a research field. SciMAT is a useful tool to carry out a study based on this approach, which offers fundamental themes, based on a cluster generation. The results of the present study show that machine learning is an important and widely studied scientific area. The tendencies indicate that machine learning applications will still be of interest to the scientific community. The use of machine learning to predict diseases such as cancer 14 or Alzheimer's disease 15 , and in fields such as biology 16 , rehabilitation system 17 , commerce 18 , smartphones 19 and ubiquitous computing 20 , will be a trend in the near future.

Competing interests
No competing interests were disclosed.

Grant information
The authors are grateful to the Telematics Engineering Group (GIT) of the University of Cauca for scientific support and Innovacción Cauca project for master's scholarship granted to J. Rincon-Patino.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The paper "Exploring machine learning: A bibliometric general approach using SciMAT" by Rincon-Patino et al. presents an overview of areas of machine learning applications. The study is conducted using science mapping analysis with the SciMAT tool. A bibliography containing "machine" and "learning" keywords over the 10-year period (2007-2017) was extracted from Scopus. This period was split into six smaller periods, and the state of machine learning in each period was illustrated by two figures, strategic diagram, and selected thematic network. The paper is well written. The following recommendations are intended to improve the readability and message of the paper. Strategic diagrams may not be familiar for users who aren't familiar with the SciMAT tool. Thus, the description of strategic diagrams given at the beginning of the Results section is better to be placed to the legend of the first figure. Also, clarify the meaning of "the basic and transversal themes." Furthermore, the description of the lower-left quadrant, "the emerging or declining topics," is confusing, it should be one or the other.  Figure 1 shows the growing number of documents containing "machine" and "learning" keywords. However, it is known that the number of publications per year grows by itself. It would be more informative to show the proportion of published documents that contain machine" and "learning" keywords. The usefulness of representing the proportion of publications can be illustrated with the following R code:

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes

Rajesh Kumar Tiwari
Amity Institute of Biotechnology, Amity University Uttar Pradesh, Lucknow, India The article highlights about the bibliometric analysis of scientific works in the area of machine learning, published in Scopus indexed Journals during the period of 2007-2017. The manuscript is well written. However, I have following suggestions to make: Correction in title of Manuscript: Exploring machine learning: A bibliometric general approach using SciMAT Tool

1.
Definition of Machine learning should be incorporated in the introduction Part. Authors can refer and cite following articles: PMID: 23740390 1 and PMID: 28302041 2 2.
Apart from mass mortality events, the quality of water, segment clients in private banking, automatically classify text, production of crops, the machine learning applications are also widely used in the Drug designing in pharmaceutical industries, academia and research. Authors should mention the same in the introduction part of the article and can refer and cite the following articles with PMID: 22346230 3 , PMID: 26526829 4 , PMID: 28382857 5 , PMID: 29256344 6 3.
Methods: As the data set is collected from Scopus, the same should be cited in the manuscript.

4.
The original source of SciMAT Tool should be cited in the manuscript. 5.
The authors must elaborate more about the basis of division of the data set. It is mentioned that the time interval divided into six smaller periods: 2007-2009, 2010-2012, 2013-2014, 2015, 2016, 2017. As the division is not uniform. Was the gap made such a way so as to have comparable number of articles in each one of them only? 6.
The machine learning algorithms are widely used in robotics and pharmaceutical properties prediction. They can also be included in conclusion part.

7.
Apart from SciMAT, are some other similar tools available? Can the comparative analysis of results using the same data set from different tools be done to know that how conclusive the reported results are? This can be included as future prospect of the current research. 8.